I just read about a blind person vibe-coding a new email client for Windows.
-
@jscholes @jcsteh @modulux @Scott @matt Well, given I'm the person talked about on part of this for the email app, one point of clarification. Sure I'm using vibe coding to write my Quickmail app but the guts of the email handling are coming from the MailKit library. My research says this is widely respected in the development community and pretty much the go to library for handling email in an app. I'm focusing on the user experience and accessibility.
@kellylford @jcsteh @modulux @Scott @matt Appreciate the explanation but at least for me, it doesn't significantly change the risk profile. The data ends up having to move in and out of MailKit to drive and be driven by the UI, and even a well-respected library is unlikely to prevent an LLM from doing something undesirable.
-
@ZBennoui @kellylford @matt The big thing is that for decades, we've essentially been the beggars, not the choosers. Even in #openSource, often when you ask for an #accessibility enhancement you either get a "no, too hard" or "submit PR and then maybe lmao". With that pushback it makes sense that people have decided they've had enough of not being considered important enough and just making a tool yourself. Even well-established projects like #microsoft #windows, #apple #MacOS etc. have been steadily backsliding over the last decade so what's a person to do?
It absolutely means there's likely a lot of tools out there that may very well be doing things insecurely or inefficiently, and the developer might not even know. So now it becomes a responsibility question: do we blame the dev, who isn't a dev, for doing dev wrong, or the user for trusting what they feel is now their only/best option? THis is where accessibility negligence has taken us.@zersiax @ZBennoui @matt I wrote about one solution for the larger problem, although I know it is unlikely to gain any traction. https://theideaplace.net/from-word-fluff-to-real-impact-achieving-specific-measurable-and-accountable-accessibility/. All I can say is that I'm using what I believe is a reliable library for the mail handing in my app called MailKit. Before I do a project, I do some research about what's out there.
-
I just read about a blind person vibe-coding a new email client for Windows. Not linking because I don't want people to pile onto this person, who is a respected member of the blind community and long-time accessibility advocate, though not a professional programmer as far as I know. Instead, I want to point out how badly the commercial software industry, particularly Microsoft in this case, has failed us such that an individual feels the need to do this. Don't know what to do instead though.
@matt yeah, ran into another case of this recently where one of the developers of Factorio access was commenting on my policy to reduce LLM usage in Rust with how LLMs have been used to create accessibility mods for all games, including the mods they make for Factorio
my comment is just, it's garbage that accessibility is so shit people have to do this, but:
a. I'll give a pass to blind folks who choose to use whatever tools are available to them to make the situation more workable
b. I don't give a pass to everyone else to make shit tools for blind folks using LLMs
c. we should be making better tools so folks don't feel like LLMs are the best option -
@storm @matt People use email clients to access a lot of confidential information, or their workplace has specific email security requirements, or... the list of reasons that email security matters goes on and on.
The problem with a vibe coded client is that the author did not write, and we can't assume they have read and understood, every character of the code to avoid security issues.
@jscholes @TheQuinbox @storm @matt Just because you write every semicolon and bracket of a program, doesn't mean you understand everything either. Nor does it make it completely immune from security issues.
-
@jscholes @TheQuinbox @storm @matt Just because you write every semicolon and bracket of a program, doesn't mean you understand everything either. Nor does it make it completely immune from security issues.
@KyleBorah @jscholes @storm @matt Much more so than if you wrote none of it, though. Of course, no one is amune from security issues.
-
@zersiax @ZBennoui @matt I wrote about one solution for the larger problem, although I know it is unlikely to gain any traction. https://theideaplace.net/from-word-fluff-to-real-impact-achieving-specific-measurable-and-accountable-accessibility/. All I can say is that I'm using what I believe is a reliable library for the mail handing in my app called MailKit. Before I do a project, I do some research about what's out there.
@kellylford @ZBennoui @matt guessing the main concern with an email app would be, what happens with the user credentials. Where are they stored, are they encrypted, are they ever shared over the web, etc.
Email in general I'd say is a pretty solved problem but security issues are easy to introduce by accident
-
@jscholes @TheQuinbox @storm @matt Just because you write every semicolon and bracket of a program, doesn't mean you understand everything either. Nor does it make it completely immune from security issues.
@KyleBorah You're right. Nor are the world's most well-audited technology systems immune from social engineering. Life is about determining acceptable risk, not eliminating it completely. @TheQuinbox @storm @matt
-
@ZBennoui @kellylford @matt The big thing is that for decades, we've essentially been the beggars, not the choosers. Even in #openSource, often when you ask for an #accessibility enhancement you either get a "no, too hard" or "submit PR and then maybe lmao". With that pushback it makes sense that people have decided they've had enough of not being considered important enough and just making a tool yourself. Even well-established projects like #microsoft #windows, #apple #MacOS etc. have been steadily backsliding over the last decade so what's a person to do?
It absolutely means there's likely a lot of tools out there that may very well be doing things insecurely or inefficiently, and the developer might not even know. So now it becomes a responsibility question: do we blame the dev, who isn't a dev, for doing dev wrong, or the user for trusting what they feel is now their only/best option? THis is where accessibility negligence has taken us.@zersiax @kellylford @matt Yeah exactly, this is my opinion as well. Who knows where AI will land in the next few years, I still remember trying to get GPT4 to write a fairly simple Python script with terrible results. Now pretty much every LLM that has come out in the last few months can do it with no problem. For the record though I don't Believe LLMs are the future, we need a more comprehensive solution that actually understands consequences of actions and takes that into account when making decisions. Mabey LLMs will be part of that, but on their own I don't think they're viable long-term.
-
@kellylford @ZBennoui @matt guessing the main concern with an email app would be, what happens with the user credentials. Where are they stored, are they encrypted, are they ever shared over the web, etc.
Email in general I'd say is a pretty solved problem but security issues are easy to introduce by accident@zersiax @kellylford @ZBennoui @matt We can audit as soon as Kelly hits 1.0 or is on the point of releasing it. I was thinking of writing an email client for myself also, so maybe we could consolidate effort here.
-
@jscholes @TheQuinbox @storm @matt Just because you write every semicolon and bracket of a program, doesn't mean you understand everything either. Nor does it make it completely immune from security issues.
@KyleBorah @jscholes @TheQuinbox @storm @matt I think we are to the point where the Blind will have no choice but to use what skills they have available to them to get a job done in an acceptable timeframe. Vibe coding software is now a tool in the toolbox. Would I like a senior software engineer to write all of the major software we use? Absolutely! The problem I see is that accessibility is not the priority it should be for many large organizations who certainly have the capital to make this happen. What else are we as Blind people supposed to do in this situation? We either put up with the miserable experiences that large organizations provide or attempt to develop our own. Thank God this is starting to happen. I'm enjoying the discussion from all sides. Keep it coming.
-
@storm @matt People use email clients to access a lot of confidential information, or their workplace has specific email security requirements, or... the list of reasons that email security matters goes on and on.
The problem with a vibe coded client is that the author did not write, and we can't assume they have read and understood, every character of the code to avoid security issues.
@jscholes @matt Accountability and compliance are foreign concepts to AI. And I fear that small-time vibe coders may face unforeseen legal issues, after something major happens. The big companies have the lawyers. The individual people almost certainly don't. And that's just on the developer side. of course, the users who are not vigilant are most certainly stepping into software that not only do the developers not even entirely know themselves, they likely don't either. Its all a big mess that I am personally staying out of. But all the best to those involved, I say. I'll just watch from the sidelines.
-
@kellylford @jcsteh @modulux @Scott @matt Appreciate the explanation but at least for me, it doesn't significantly change the risk profile. The data ends up having to move in and out of MailKit to drive and be driven by the UI, and even a well-respected library is unlikely to prevent an LLM from doing something undesirable.
@jscholes @kellylford @jcsteh @modulux @Scott @matt Curious, would the code being publicly available, on GitHub or similar, change your opinion on this? Because at that point anyone can check what's happening. I understand the likelyhood of someone actually doing that check is small, but do you think that the fact that anyone could would encourage the person to at least conduct some sort of audit?
-
@jscholes @kellylford @jcsteh @modulux @Scott @matt Curious, would the code being publicly available, on GitHub or similar, change your opinion on this? Because at that point anyone can check what's happening. I understand the likelyhood of someone actually doing that check is small, but do you think that the fact that anyone could would encourage the person to at least conduct some sort of audit?
-
-
-
I just read about a blind person vibe-coding a new email client for Windows. Not linking because I don't want people to pile onto this person, who is a respected member of the blind community and long-time accessibility advocate, though not a professional programmer as far as I know. Instead, I want to point out how badly the commercial software industry, particularly Microsoft in this case, has failed us such that an individual feels the need to do this. Don't know what to do instead though.
@matt Yeah... personally I'd rather use something vibe coded than something intentionally coded to bloat my system while doubling as an omnipresent salesman I never asked for. If this works, heck yeah I'm gonna use it.
-
@alexchapman @fireborn @matt @jscholes @kellylford @jcsteh @modulux @Scott what is your github again? I didn't bookmark it lol.
-
@alexchapman @fireborn @matt @jscholes @kellylford @jcsteh @modulux @Scott what is your github again? I didn't bookmark it lol.
@J3317 @fireborn @matt @jscholes @kellylford @jcsteh @modulux @Scott I changed it when I decided to unify almost everything under one username. My GitHub is https://github.com/alexoloopios
-
@fireborn Speaking for myself:
I suspect I lack some of the skills, and I definitely lack the time, to properly audit such a large codebase. There's also a bit of a chicken and egg problem in that in order to know whether the software is worth the time and effort of an audit, I'd need to test it with real data, but to test it with real data I'd need to increase my risk appetite.
But okay, let's say someone spot checked or audited this repository and I was sufficiently reassured by the methodology and outcome. That state only realistically holds for the code revision under test.
LLMs produce code in quantities and at speeds outstripping a human's ability to keep up. Especially in the context of a fully vibe-coded project where the model is essentially being instructed to put its foot down and do whatever is necessary.
The amounts of code and code churn in an AI-generated project do not match how most humans approach software development. The latter in particular makes it certain that at some point, code that was previously audited and working will be replaced.
The speed factor means that the replacement could happen minutes or days from now, rather than years. The quantity problem means that every follow-up audit needs to be huge and complex.
TL;DR: the code being available is a necessary step, but barely moves the needle. I haven't even touched upon using AI to audit the AI-generated code. @matt @kellylford @jcsteh @modulux @Scott
-
@fireborn Speaking for myself:
I suspect I lack some of the skills, and I definitely lack the time, to properly audit such a large codebase. There's also a bit of a chicken and egg problem in that in order to know whether the software is worth the time and effort of an audit, I'd need to test it with real data, but to test it with real data I'd need to increase my risk appetite.
But okay, let's say someone spot checked or audited this repository and I was sufficiently reassured by the methodology and outcome. That state only realistically holds for the code revision under test.
LLMs produce code in quantities and at speeds outstripping a human's ability to keep up. Especially in the context of a fully vibe-coded project where the model is essentially being instructed to put its foot down and do whatever is necessary.
The amounts of code and code churn in an AI-generated project do not match how most humans approach software development. The latter in particular makes it certain that at some point, code that was previously audited and working will be replaced.
The speed factor means that the replacement could happen minutes or days from now, rather than years. The quantity problem means that every follow-up audit needs to be huge and complex.
TL;DR: the code being available is a necessary step, but barely moves the needle. I haven't even touched upon using AI to audit the AI-generated code. @matt @kellylford @jcsteh @modulux @Scott
