I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y.
-
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
as a scientist, you are surely aware of bias
such as the bias on social media, where if AI does something bad, it gets widely disseminated (goes viral)
whereas if AI does something good, no one talks about it
also,
PSA
when I was a baby PhD student in 1985, my teachers warned me over and over, don't trust something just cause it is published in a peer reviewed journal
be careful of all that you read !!!!! -
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
@ElenLeFoll You might enjoy my recent experience with raclette maximalism.
-
@ElenLeFoll I love this quote about the original article. “Acknowledgments also thanked “Professor Sideshow Bob” and a professor from the Starfleet Academy for access to a lab aboard the USS Enterprise”
#AI #AiSlop@Pineywoozle Yes, big cinema !
@ElenLeFoll -
Johnathan Swift described The Machine (for writing) in 1726; https://en.wikipedia.org/wiki/The_Engine
@mycotropic @bms48 @ElenLeFoll shout-out to Ramon Llull: http://www.computer-timeline.com/timeline/ramon-llull/
-
@ElenLeFoll Good. Poison as much as possible, let it eat its own crap and dance while it burns to the ground.
@MeneerDeBruin As a professional 100% human writer, I'm indeed interested in how we could use our creativity for that goal!
@ElenLeFoll -
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
@ElenLeFoll i *like it a lot* when people teach AI... wrong stuff. Piss in the well at every opportunity, folks.
-
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
It seems like the real underlying problem is the "publish or perish" syndrome, where the value of a researcher is based on how many papers they write, or how often they're reference.
So there's a proliferation of papers, many of which are meaningless, and which no one has time to actually read, being referenced in other papers by other researchers who don't have time to read and evaluate all these other papers.
-
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
@ElenLeFoll The whole thing is like an oil spill.
-
@mycotropic @bms48 @ElenLeFoll shout-out to Ramon Llull: http://www.computer-timeline.com/timeline/ramon-llull/
That's an amazing history and I'll include llull in my "History of AI/LLM" lecture! Thanks!
-
@ElenLeFoll I love this quote about the original article. “Acknowledgments also thanked “Professor Sideshow Bob” and a professor from the Starfleet Academy for access to a lab aboard the USS Enterprise”
#AI #AiSlop@Pineywoozle @ElenLeFoll what a fun piece of work really, to write made-up scientific articles with all the silly things you want!
-
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
@ElenLeFoll Unfortunately it’s impossible to test this now that the veracity of the study has been revealed.
