I just read about a blind person vibe-coding a new email client for Windows.
-
@storm @matt People use email clients to access a lot of confidential information, or their workplace has specific email security requirements, or... the list of reasons that email security matters goes on and on.
The problem with a vibe coded client is that the author did not write, and we can't assume they have read and understood, every character of the code to avoid security issues.
@jscholes @matt Accountability and compliance are foreign concepts to AI. And I fear that small-time vibe coders may face unforeseen legal issues, after something major happens. The big companies have the lawyers. The individual people almost certainly don't. And that's just on the developer side. of course, the users who are not vigilant are most certainly stepping into software that not only do the developers not even entirely know themselves, they likely don't either. Its all a big mess that I am personally staying out of. But all the best to those involved, I say. I'll just watch from the sidelines.
-
@kellylford @jcsteh @modulux @Scott @matt Appreciate the explanation but at least for me, it doesn't significantly change the risk profile. The data ends up having to move in and out of MailKit to drive and be driven by the UI, and even a well-respected library is unlikely to prevent an LLM from doing something undesirable.
@jscholes @kellylford @jcsteh @modulux @Scott @matt Curious, would the code being publicly available, on GitHub or similar, change your opinion on this? Because at that point anyone can check what's happening. I understand the likelyhood of someone actually doing that check is small, but do you think that the fact that anyone could would encourage the person to at least conduct some sort of audit?
-
@jscholes @kellylford @jcsteh @modulux @Scott @matt Curious, would the code being publicly available, on GitHub or similar, change your opinion on this? Because at that point anyone can check what's happening. I understand the likelyhood of someone actually doing that check is small, but do you think that the fact that anyone could would encourage the person to at least conduct some sort of audit?
-
-
-
I just read about a blind person vibe-coding a new email client for Windows. Not linking because I don't want people to pile onto this person, who is a respected member of the blind community and long-time accessibility advocate, though not a professional programmer as far as I know. Instead, I want to point out how badly the commercial software industry, particularly Microsoft in this case, has failed us such that an individual feels the need to do this. Don't know what to do instead though.
@matt Yeah... personally I'd rather use something vibe coded than something intentionally coded to bloat my system while doubling as an omnipresent salesman I never asked for. If this works, heck yeah I'm gonna use it.
-
@alexchapman @fireborn @matt @jscholes @kellylford @jcsteh @modulux @Scott what is your github again? I didn't bookmark it lol.
-
@alexchapman @fireborn @matt @jscholes @kellylford @jcsteh @modulux @Scott what is your github again? I didn't bookmark it lol.
@J3317 @fireborn @matt @jscholes @kellylford @jcsteh @modulux @Scott I changed it when I decided to unify almost everything under one username. My GitHub is https://github.com/alexoloopios
-
@fireborn Speaking for myself:
I suspect I lack some of the skills, and I definitely lack the time, to properly audit such a large codebase. There's also a bit of a chicken and egg problem in that in order to know whether the software is worth the time and effort of an audit, I'd need to test it with real data, but to test it with real data I'd need to increase my risk appetite.
But okay, let's say someone spot checked or audited this repository and I was sufficiently reassured by the methodology and outcome. That state only realistically holds for the code revision under test.
LLMs produce code in quantities and at speeds outstripping a human's ability to keep up. Especially in the context of a fully vibe-coded project where the model is essentially being instructed to put its foot down and do whatever is necessary.
The amounts of code and code churn in an AI-generated project do not match how most humans approach software development. The latter in particular makes it certain that at some point, code that was previously audited and working will be replaced.
The speed factor means that the replacement could happen minutes or days from now, rather than years. The quantity problem means that every follow-up audit needs to be huge and complex.
TL;DR: the code being available is a necessary step, but barely moves the needle. I haven't even touched upon using AI to audit the AI-generated code. @matt @kellylford @jcsteh @modulux @Scott
-
@fireborn Speaking for myself:
I suspect I lack some of the skills, and I definitely lack the time, to properly audit such a large codebase. There's also a bit of a chicken and egg problem in that in order to know whether the software is worth the time and effort of an audit, I'd need to test it with real data, but to test it with real data I'd need to increase my risk appetite.
But okay, let's say someone spot checked or audited this repository and I was sufficiently reassured by the methodology and outcome. That state only realistically holds for the code revision under test.
LLMs produce code in quantities and at speeds outstripping a human's ability to keep up. Especially in the context of a fully vibe-coded project where the model is essentially being instructed to put its foot down and do whatever is necessary.
The amounts of code and code churn in an AI-generated project do not match how most humans approach software development. The latter in particular makes it certain that at some point, code that was previously audited and working will be replaced.
The speed factor means that the replacement could happen minutes or days from now, rather than years. The quantity problem means that every follow-up audit needs to be huge and complex.
TL;DR: the code being available is a necessary step, but barely moves the needle. I haven't even touched upon using AI to audit the AI-generated code. @matt @kellylford @jcsteh @modulux @Scott
-
@storm @matt People use email clients to access a lot of confidential information, or their workplace has specific email security requirements, or... the list of reasons that email security matters goes on and on.
The problem with a vibe coded client is that the author did not write, and we can't assume they have read and understood, every character of the code to avoid security issues.
-
@storm @matt People use email clients to access a lot of confidential information, or their workplace has specific email security requirements, or... the list of reasons that email security matters goes on and on.
The problem with a vibe coded client is that the author did not write, and we can't assume they have read and understood, every character of the code to avoid security issues.
@sapphireangel @jscholes Do you really think Modern outlook is doing this? I mean it doesn't have anything to do with security, but I've seen instances of zoom invites, the links I used modern outlook were not clickable links. So I woudln't asume that. I use classic outlook and no issues with this.
-
R relay@relay.publicsquare.global shared this topic
