I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y.
-
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
This is a rerun of the Sokal Hoax.
Like some here, some folks thought the Sokal hoax was ethically problematic, but the postmodernists needed a wake up call, as do the LLM fans.
Really: the LLM idea is the stupidest* thing to come out of Computer Science ever. We need to be embarrassed.
*: Unnecessary explanation: the idea that random text generation has something to do with intelligence is really really stupid.
-
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
That's something the government does every few years already. Nothing new.
-
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
@ElenLeFoll I'm convinced one of the main reasons we die is that we get too old or too sick to manage our own healthcare. We can't do the research to find the right studies, we can't read or understand those studies, and we can't question our providers to be sure they actually understand what's wrong with us.
Once we are dependent on mere employees, the quality of care goes way down, and mistakes get made, or our treatment is just ineffective.
-
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
as a scientist, you are surely aware of bias
such as the bias on social media, where if AI does something bad, it gets widely disseminated (goes viral)
whereas if AI does something good, no one talks about it
also,
PSA
when I was a baby PhD student in 1985, my teachers warned me over and over, don't trust something just cause it is published in a peer reviewed journal
be careful of all that you read !!!!! -
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
@ElenLeFoll You might enjoy my recent experience with raclette maximalism.
-
@ElenLeFoll I love this quote about the original article. “Acknowledgments also thanked “Professor Sideshow Bob” and a professor from the Starfleet Academy for access to a lab aboard the USS Enterprise”
#AI #AiSlop@Pineywoozle Yes, big cinema !
@ElenLeFoll -
Johnathan Swift described The Machine (for writing) in 1726; https://en.wikipedia.org/wiki/The_Engine
@mycotropic @bms48 @ElenLeFoll shout-out to Ramon Llull: http://www.computer-timeline.com/timeline/ramon-llull/
-
@ElenLeFoll Good. Poison as much as possible, let it eat its own crap and dance while it burns to the ground.
@MeneerDeBruin As a professional 100% human writer, I'm indeed interested in how we could use our creativity for that goal!
@ElenLeFoll -
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
@ElenLeFoll i *like it a lot* when people teach AI... wrong stuff. Piss in the well at every opportunity, folks.
-
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
It seems like the real underlying problem is the "publish or perish" syndrome, where the value of a researcher is based on how many papers they write, or how often they're reference.
So there's a proliferation of papers, many of which are meaningless, and which no one has time to actually read, being referenced in other papers by other researchers who don't have time to read and evaluate all these other papers.
-
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
@ElenLeFoll The whole thing is like an oil spill.
-
@mycotropic @bms48 @ElenLeFoll shout-out to Ramon Llull: http://www.computer-timeline.com/timeline/ramon-llull/
That's an amazing history and I'll include llull in my "History of AI/LLM" lecture! Thanks!
-
@ElenLeFoll I love this quote about the original article. “Acknowledgments also thanked “Professor Sideshow Bob” and a professor from the Starfleet Academy for access to a lab aboard the USS Enterprise”
#AI #AiSlop@Pineywoozle @ElenLeFoll what a fun piece of work really, to write made-up scientific articles with all the silly things you want!
-
I was only made aware of this (frankly awesome) case of LLM poisoning today: https://www.nature.com/articles/d41586-026-01100-y. A researcher made up a disease and published two evidently fake preprints about it (including sentences such as “this entire paper is made up” and “Fifty made-up individuals aged between 20 and 50 years were recruited for the exposure group”), which were almost immediately picked up by LLMs and documented in their output. Worse, actual – supposedly serious – medical papers also started citing the preprints, demonstrating that academics relying on LLMs to do their work is a genuine problem! Not that I had my doubts but, if anyone did, this seems like the perfect demonstration of the problem. Article immediately added to the syllabus of the class I am co-teaching with Iris Ferrazzo on LLMs for Romance Studies/Humanities!
#LLM #GenAI #academia #research #ResearchIntegrity #humanities
@ElenLeFoll Unfortunately it’s impossible to test this now that the veracity of the study has been revealed.
