I was pondering recently a throw-away remark from a prominent F/OSS developer that they use LLMs for creating ffmpeg command lines.

[david_chisnall@infosec.exchange]

I was pondering recently a throw-away remark from a prominent F/OSS developer that they use LLMs for creating ffmpeg command lines. And I had a bunch of reactions.

First, I wouldn't use an LLM for creating something with a bunch of context-specific flags, that seems high risk. A bunch of flags in the documentation don't make sense in a load of cases.

Then I realised that the way I normally create an ffmpeg command line to do $THING is to search for '$THING ffmpeg'. And almost all of the time I find a forum post or web page where someone has posted exactly the command line I want. So then I thought 'they probably have the exact right answers in the training set'. And so an LLM probably gives the right answer.

But then I thought 'well, that's probably only true for 95% of the uses'. And if I ask it for something in the 5%, I'll get something that looks completely plausible, but is likely to be wrong.

But then I thought 'what's the worst that can happen?' and generally ML is good for things where the benefit of getting a good answer outweigh the cost of getting it wrong. And maybe ffmpeg is like that. So an LLM is fine: it might be wrong, but you can see and fix it after it's burned a few seconds of CPU time if so.

And then I remembered that the output file was part of the command line and the tool runs with no sandboxing. And so the worst that can happen is you get some escaping wrong and it overwrites random files in your disk.

And so my main conclusion is: I don't know how any normal person could be expected to understand whether any given task is something where is safe to use an LLM.