CIRCLE WITH A DOT

@jpteti @beyondmachines1 Don't trust Pangram so easily. I've seen it get stuff dead wrong (with "high confidence", no less).

But yes, this looks slop-ish.

@mausmalone @LilahTovMoon I've done this, more or less.

I SSH'd into one of our production boxes, attached a REPL to a running Clojure web server, and swapped out a function in the server process.

This helped us *finally* solve a bug that we had been unable to figure out from telemetry and experiments. We also took a number of precautions in order to make this safe. But it was still fun as hell.

@sirosen Cool! But it sounds like this isn't fully remote, is that right?

« Remote work options are available for this position, with occasional attendance at in-person meetings. »

(I live in the Boston area, so it wouldn't be trivial to come in!)

@afeinman That would be for code though, right? I was thinking more of chat messages, wiki docs, etc.

How are y'all handling coworkers who post slop?

Several of our contractors have made rather voluminous wiki pages that are heavily redundant and over-explanatory. So far my approach has been to just... quietly not read them, and pretend the pages don't exist. (If I need information that the page is supposed to have, I just ask the contractor to explain in Slack or a meeting.) It's bad for the company in a bunch of different ways, but the company is all-in on AI and doesn't want to hear dissent, so there's no way to address this systemically. (And I'm not invested in the company's long-term health.)

One coworker posts AI outputs sometimes, but is a bit more discerning, and we have a good enough relationship that I've been able to explain that hey, I'm not reading that, but you're free to tell me anything you learned *after* you verify it.

I'm curious to hear how others are handling it.

@janl Yeah, what the heck is up with Mac dev licensing? I wanted to cross-compile a Rust crate for Mac and found I would have to sign some sort of contract.

...so I just don't compile for Mac.

@jerry Not safe enough, Windows has WSL.

@afeinman I've gotten a bunch of these, all different. The goofiness level varies. I reckon they came up with a prompt, generated a few, said "looks good", and let 'er rip. Probably generating a new one every few days without actually reviewing it.

My favorite subject line so far is "8RD ATTEMPT: Payment Declined — Inbox Suspension Active".

I genuinely love how stupid this LLM-generated phishing email is.

There's a lot going on here and every part of it is hilarious.

Before reaching for an LLM for finding vulnerabilities in your own project, you should probably still be:

- Testing
- Linting
- Running other existing, algorithmic static analysis tools for security
- Fuzzing
- Looking at new and existing security bugs and looking for other bugs of the same type *and* findings ways to make each type of bug harder to introduce in the future

With those already in place, LLMs still don't seem to have a major advantage. I'm curious whether that will change, though.

As a certified AI Hater, I do have to say: We seem to have found one (1) use-case for LLMs where they're useful and (can be) prosocial: Finding software vulnerabilities.

This wasn't true a few months ago, but it seems the scales have finally tipped.

It ticks the boxes for me:

- Verifiable
- "Generative" aspect is limited
- Utility that isn't just replacing human labor

(I don't *like* it, and I don't know how the overall cost/benefit shakes out, but... this does seem to be legit. Just be wary of the hype.)