Found myself wincing while reading this story about how Ars Technica fired a reporter over fabricated quotations generated by an AI tool.
-
@screwturn @nirak There are some pretty decent and recent studies showing AI substantially misses or misrepresents the point or summary of a story about 40-50 percent of the time.
Even if the success were 95%, as a journalist, consistently using a stochastic method to give sources guarantees you eventually fuck up and let a fabricated quote into print.
-
Even if the success were 95%, as a journalist, consistently using a stochastic method to give sources guarantees you eventually fuck up and let a fabricated quote into print.
@ct
um... sure, but who is going to be asking the free version of ChatGPT for sources?
That is going to be a very poor use case.If I am using the Ai that is inside my CAQDAS, I am not going to see hallucination, and it internally cites each fact it produces. Reliability and validity are going to vary greatly depending on the environment you use the AI in, and what you are trying to do.
-
@screwturn @nirak There are some pretty decent and recent studies showing AI substantially misses or misrepresents the point or summary of a story about 40-50 percent of the time.
@briankrebs
I'm not sure what that means
Like 50% of the time I use it, it will miss at least one point? Sure, but those odds are fine by me if it is also spotting things I missed, and has a reasonable inter-rater reliability with what I saw.If you mean it gets 50% of the points wrong, then that is probably true of the free versions, but not what I am seeing in practice when I use the AI inside my CAQDAS
-
@screwturn @nirak @briankrebs Even a blind pig will find the occasional acorn.
@StumpyTheMutt
If it finds an acorn that I missed, then it found something of value.Keep in mind, I'm not using the free version in its wide-open configuration, but rather a tightly configured version inside a research workbench. In three years of use, I have not seen a single case of hallucination by the LLM.
-
R relay@relay.an.exchange shared this topic
-
@ct
um... sure, but who is going to be asking the free version of ChatGPT for sources?
That is going to be a very poor use case.If I am using the Ai that is inside my CAQDAS, I am not going to see hallucination, and it internally cites each fact it produces. Reliability and validity are going to vary greatly depending on the environment you use the AI in, and what you are trying to do.
I'm curious how your example actually works under the hood.
I have a sneaking suspicion that maybe your personal experience with a research summarization tool was not relevant to this story of a tech journalist, who needs to source current events from myriad sources and not just a limited database of pre-curated published research? I speculate your CAQDAS tool would not have been useful for a current events journalist who may need to quote things like statements from leadership, self-published cybersecurity reports, transcriptions of tech presentations etc… where there's a lot more critical thinking involved in selecting who to source from.
Regardless, I'd love to see how your CAQDAS tool fares against peer-reviewed fact-checking tests. I am very skeptical the failure rate is under 1% just from your testimony.
-
I'm curious how your example actually works under the hood.
I have a sneaking suspicion that maybe your personal experience with a research summarization tool was not relevant to this story of a tech journalist, who needs to source current events from myriad sources and not just a limited database of pre-curated published research? I speculate your CAQDAS tool would not have been useful for a current events journalist who may need to quote things like statements from leadership, self-published cybersecurity reports, transcriptions of tech presentations etc… where there's a lot more critical thinking involved in selecting who to source from.
Regardless, I'd love to see how your CAQDAS tool fares against peer-reviewed fact-checking tests. I am very skeptical the failure rate is under 1% just from your testimony.
I have a sneaking suspicion that you don't know what "goes on under the hood" of qualitative research.
Let's just be clear what we are talking about in praxis.One would not use the AI to find material, conduct interviews (although that is a distinct future possibility), or to do the discovery part of research, or journalism.
Once you have pulled those texts, transcripts, etc into a CAQDAS, THEN you would use the AI to summarize, identify topics, match topics, etc -
I have a sneaking suspicion that you don't know what "goes on under the hood" of qualitative research.
Let's just be clear what we are talking about in praxis.One would not use the AI to find material, conduct interviews (although that is a distinct future possibility), or to do the discovery part of research, or journalism.
Once you have pulled those texts, transcripts, etc into a CAQDAS, THEN you would use the AI to summarize, identify topics, match topics, etc@ct
"peer-reviewed fact-checking tests"Do you mean inter-rater reliability?
Because several people have found various LLMs in CAQDAS platforms to be on par between the AI and human researchers and between human researchers -
"We always write things by hand and never use AI, except for this one small case where you caught us. And the next time you catch us. But there's no general tendency. You're just very good at catching exactly the cases where we use AI."
I would expect that journalists have to deal with information from untrustworthy sources all the time. In that regard the output of an AI might not be worse than a lot of the other misinformation they are juggling.
Using AI at some point during the process is not guaranteed to result in a worse end result if the journalist is otherwise doing a good job.
Of course it's possible for a journalist to do a bad job such as including AI output verbatim in the final product without validating the correctness. But bad journalism isn't a novel concept. There has been journalists producing bad results before AI.
-
Found myself wincing while reading this story about how Ars Technica fired a reporter over fabricated quotations generated by an AI tool. What a mess. And a tough one to bounce back from. I get asked all the time how I use AI in my work, and my answer is always the same: I don't, for all the reasons I also don't delegate important research to others, plus a whole bunch of other good reasons. But I really am interested in the answer from other journalists, because I suspect I'm in the minority here.
From Futurism.com:
"In the post, Edwards said that he was sick at the time, and “while working from bed with a fever and very little sleep,” he “unintentionally made a serious journalistic error” as he attempted to use an “experimental Claude Code-based AI tool” to help him “extract relevant verbatim source material.” He said the tool wasn’t being used to generate the article, but was instead designed to “help list structured references” to put in an outline. When the tool failed to work, said Edwards, he decided to try and use ChatGPT to help him understand why.
“I should have taken a sick day because in the course of that interaction, I inadvertently ended up with a paraphrased version of Shambaugh’s words rather than his actual words,” Edwards continued. He emphasized that the “text of the article was human-written by us, and this incident was isolated and is not representative of Ars‘ editorial standards. None of our articles are AI-generated, it is against company policy and we have always respected that.”
Ars Technica Fires Reporter After AI Controversy Involving Fabricated Quotes
Ars Technica has fired senior AI reporter Benj Edwards following an outrage-sparking controversy involving AI-fabricated quotes.
Futurism (futurism.com)
@briankrebs @rpmik I think using AI to do your work is like playing Russian Roulette.
-
I have a sneaking suspicion that you don't know what "goes on under the hood" of qualitative research.
Let's just be clear what we are talking about in praxis.One would not use the AI to find material, conduct interviews (although that is a distinct future possibility), or to do the discovery part of research, or journalism.
Once you have pulled those texts, transcripts, etc into a CAQDAS, THEN you would use the AI to summarize, identify topics, match topics, etcI don't… that's why I said I was curious what it was. I was asking.
Anyway… if your tool can't be used for the task the article is about, which is tech journalism. why did you come in to the article's replies to defend AI research? All I'm seeing is you reacting to criticism of an example where LLM for research didn't work by defending your own use, in an application and implementation that both aren't relevant to the event at hand. -
I don't… that's why I said I was curious what it was. I was asking.
Anyway… if your tool can't be used for the task the article is about, which is tech journalism. why did you come in to the article's replies to defend AI research? All I'm seeing is you reacting to criticism of an example where LLM for research didn't work by defending your own use, in an application and implementation that both aren't relevant to the event at hand.@ct
Sure it can be used in journalism, especially tech journalism - because a large part of tech writing is essentially qualitative research or mixed methods research.The OP was about a journalist who used AI to *write* the article, and I was clarifying that there are PARTS of the process that are very much amenable to use of AI tools. You seemed to question that, which is why I responded to you as well
-
Found myself wincing while reading this story about how Ars Technica fired a reporter over fabricated quotations generated by an AI tool. What a mess. And a tough one to bounce back from. I get asked all the time how I use AI in my work, and my answer is always the same: I don't, for all the reasons I also don't delegate important research to others, plus a whole bunch of other good reasons. But I really am interested in the answer from other journalists, because I suspect I'm in the minority here.
From Futurism.com:
"In the post, Edwards said that he was sick at the time, and “while working from bed with a fever and very little sleep,” he “unintentionally made a serious journalistic error” as he attempted to use an “experimental Claude Code-based AI tool” to help him “extract relevant verbatim source material.” He said the tool wasn’t being used to generate the article, but was instead designed to “help list structured references” to put in an outline. When the tool failed to work, said Edwards, he decided to try and use ChatGPT to help him understand why.
“I should have taken a sick day because in the course of that interaction, I inadvertently ended up with a paraphrased version of Shambaugh’s words rather than his actual words,” Edwards continued. He emphasized that the “text of the article was human-written by us, and this incident was isolated and is not representative of Ars‘ editorial standards. None of our articles are AI-generated, it is against company policy and we have always respected that.”
Ars Technica Fires Reporter After AI Controversy Involving Fabricated Quotes
Ars Technica has fired senior AI reporter Benj Edwards following an outrage-sparking controversy involving AI-fabricated quotes.
Futurism (futurism.com)
@briankrebs senior AI reporter uses AI to do his job for him.. I am shocked!!
Seriously though, Ars has a decent reputation, @dangoodin is one of the journalist I'd trust the most in the tech space from them. This is hopefuly not the way things are going, but alas I have low hopes, LLMs seem to make everyrhing easy and people fall in to the trap.
Even though many times over we've seen that they are but enshittification anthropomorphised. -
I would expect that journalists have to deal with information from untrustworthy sources all the time. In that regard the output of an AI might not be worse than a lot of the other misinformation they are juggling.
Using AI at some point during the process is not guaranteed to result in a worse end result if the journalist is otherwise doing a good job.
Of course it's possible for a journalist to do a bad job such as including AI output verbatim in the final product without validating the correctness. But bad journalism isn't a novel concept. There has been journalists producing bad results before AI.
Disagree. A bad source is traceable if the journalist is worth anything. Using bad input and good logic allows you to reach a wrong conclusion, but good input and good logic leads you to a good conclusion. Using bad logic (AI) leads to a wrong conclusion regardless, and you have not traceability. -
R relay@relay.infosec.exchange shared this topic
