CIRCLE WITH A DOT

@docpop

- The cited work had an error that required a later correction to be mentioned in an erratum or subsequent article (e.g. fixing a typo in a formula), and the citing researchers were unaware of the correction, so their own work reproduces the same error.
- The cited work had an error that was later corrected, and the citing researchers knew about the correction, but they didn't bother to mention it or cite the later work where the correction appears, so anyone trying to reproduce their results would have no idea that there's an inaccuracy that needs to be corrected.
- The work that should have been cited is out of print and was never digitized, so rather than work with a research library to try to track down a copy and see what it says, the authors either don't double-check what it said, or cite some other work by the same author.

This is a major problem! LLMs are both directly making the problem worse, and encouraging scientists to churn out publications without putting in the work to verify.

@docpop
Unfortunately, I think this is a common attitude in many fields. As someone who has often needed to reproduce other scientists' work, I can confirm that I frequently check other peoples' citations in methodology that turn out to just be wrong. E.g., "As in Smith (2004), we decide the value of parameter X by applying formula Y to input Z." Often I find one of the following:

- The citation does not spell out important details, which have never made it into the scientific record at all, but are spread informally among a particular clique of researchers who frequently collaborate. (Or they are in documentation that was not publicly archived, e.g. internal wikis, proprietary software code, etc.)
- The citation is just to the wrong paper (e.g. the actual source is a different paper by the same author the year before, or the cited paper itself doesn't contain the full details and cites some earlier paper instead).