CIRCLE WITH A DOT

@claushoumann and to be quite specific: I think many people here use LLMs day to day, but are too ashamed to openly speak about it.

@claushoumann yes, i noticed and i see the reasons why people do so. Still, there are also reasons to look into it and i am missing those voices here.

@buherator @freddy Let me see if i find my insect costume...

@freddy any process involving processes in SAP. Every. Single. Time.

@dalias There's a difference between "papers they haven't actually read and confirmed the validity of" and "they checked each and every citation is 100% correct". Hallucinated citations may include borderline cases where the LLM reworded a title or changed the author "Martin Miller" to "Maria Miller", because apprently Maria sounds better than Martin. Just because one co-author decided it's a good idea to tell an LLM to improve the writing and remove typos on a draft, arxiv will punish all co-authors. As I said, I find this harsh, but necssary in the long run.

Generally speaking: There are multiple authors per paper, because they split the work. If I had to redo all work of the co-authors to be 100% sure they are right, there's no need to have co-authors at all. If you don't trust your co-authors, you'd need to write only single author papers.

Just to be clear: I am not trying to justifying anything. My point here is that LLMs are making problems we already had in academia much worse.

RE: https://social.v.st/@quixoticgeek/116611731183393595

The background story of this is: academics increasingly use AI to assist in research and paper writing. This leads to flaws in the papers due to hallucinations, which are generally hard to detect, with an important exception: they are easy to detect for citations, because you can simply search for the cited work. If you find it, good. If you do not find it, or you find a paper strikingly similar, but with a slightly different title or different authors, etc: that's clearly a hallucination.

Arxiv published a policy banning all authors of a paper for one year if it contains evidence of LLM generated hallucinated content.

Now as a co-author of a paper, in the past, I generally did not check each and every citation. I contribute my parts of the writing, review stuff others wrote, but generally trust my co-authors that they know what they were writing. The reason is that we have spend weeks, months, sometimes years researching stuff together and this is just about writing it down.

The arxiv policy means that a single co-author can now cause substantial trouble for the rest. Being strict here is harsh, but is probably the right thing in the long run.

"In addition to fixing the 271 bugs identified by Claude Mythos Preview in the 150 release, we've shipped more of these fixes in 149.0.2, 150.0.1, and 150.0.2. We also continue to find bugs with other means internally, and, similar to other projects, we've seen a significant uptick in external reports in the last few months."

Behind the Scenes Hardening Firefox with Claude Mythos Preview – Mozilla Hacks - the Web developer blog

New details about what we found, and how agentic harnesses are now able to reproduce real bugs and dismiss false positives.

Mozilla Hacks – the Web developer blog (hacks.mozilla.org)