A woman sues her insurance company for terminating her disability benefits.
-
@mjd TBH I do not think OpenAI should be responsible. They're just providing a fancy random text generator to the public. And it's outright impossible to teach a random text generator to _not_ output a specific kind of text, as whatever you do, there is a way around it.
The woman should pay all costs, as per the usual "vexatious filings" or "frivolous lawsuits" standards.
Plus, the law in her state against practicing law without a license starts with "No person shall...". ChatGPT isn't a person.
@divVerent @mjd ChatGPT is not a person, which is why ChatGPT is not being sued. OpenAI sells a tool that gave her legal advice, and they certainly didn't say anywhere that it's actually just a "fancy random text generator"
-
@mjd TBH I do not think OpenAI should be responsible. They're just providing a fancy random text generator to the public. And it's outright impossible to teach a random text generator to _not_ output a specific kind of text, as whatever you do, there is a way around it.
The woman should pay all costs, as per the usual "vexatious filings" or "frivolous lawsuits" standards.
Plus, the law in her state against practicing law without a license starts with "No person shall...". ChatGPT isn't a person.
@divVerent Except that there are laws against providing bogus legal advice to people, to prevent exactly this sort of situation.
And, as you pointed out, it was OpenAI, not ChatGPT, providing the advice.
-
@Infoseepage You made that up out of your head to come to the conclusion you selected beforehand.
I don't know what actually happened, and neither do you.
@mjd Nope, I don't know the particulars of the case, but I do have some experience with the ruthlessness of insurance companies when it comes to disabled people, including a friend who went blind from cancer as a child and another who was born with severe cerebral palsy.
-
@mjd @krupo that MIT article demonstrated 5% of AI implementations are profit making. 95% are loss making. so when the investment goldrush mania ends or winds down, 95% of invested amounts will be wiped out. it's billions so it will have a disruptive and negative economic effect that I think we will likely experience as recession. and then that 5% of profit making implementations will be what carries forward, with further investment being modelled on those (anyone can do that right now).
-
@divVerent @mjd
If ChatGPT misinformed her, that's not very useful -
@divVerent @mjd ChatGPT is not a person, which is why ChatGPT is not being sued. OpenAI sells a tool that gave her legal advice, and they certainly didn't say anywhere that it's actually just a "fancy random text generator"
@jonoleth @mjd Pretty sure it's common knowledge that LLMs are nothing but random text generators.
OpenAI is a company, not a person. From what I understand, the law banning unlicensed legal advice bans _persons_ and gives them a penalty for doing so anyway.
But OpenAI, being a company, cannot commit crimes (after all, how to put a company in prison?). Only the employees can. So the question is which concrete employee committed a crime there. (Yes, some say companies _can_ commit crimes, but then solve the problems by making an employee / owner / ... actually criminally liable - but then they are the ones who have committed the crime)
The question is rather, have any employees of OpenAI committed a crime there? If any employee at OpenAI _knew_ that it tries to give legal advice, and did not implement any countermeasures, then that employee has committed a crime. That's the case no matter how the "random text generator" works.
If someone tries to get legal advice out of a magic 8-ball, AND the company producing the 8-ball does not implement any countermeasures (such as writing in the manual that responses it gives cannot be used as legal advice), then they can potentially be held liable. Except that in case of a mechanical device that works strikingly like a die it may not be necessary to put such a disclaimer
-
@divVerent @mjd
If ChatGPT misinformed her, that's not very useful -
@jonoleth @mjd Pretty sure it's common knowledge that LLMs are nothing but random text generators.
OpenAI is a company, not a person. From what I understand, the law banning unlicensed legal advice bans _persons_ and gives them a penalty for doing so anyway.
But OpenAI, being a company, cannot commit crimes (after all, how to put a company in prison?). Only the employees can. So the question is which concrete employee committed a crime there. (Yes, some say companies _can_ commit crimes, but then solve the problems by making an employee / owner / ... actually criminally liable - but then they are the ones who have committed the crime)
The question is rather, have any employees of OpenAI committed a crime there? If any employee at OpenAI _knew_ that it tries to give legal advice, and did not implement any countermeasures, then that employee has committed a crime. That's the case no matter how the "random text generator" works.
If someone tries to get legal advice out of a magic 8-ball, AND the company producing the 8-ball does not implement any countermeasures (such as writing in the manual that responses it gives cannot be used as legal advice), then they can potentially be held liable. Except that in case of a mechanical device that works strikingly like a die it may not be necessary to put such a disclaimer
@divVerent @jonoleth If you're aware of any specific Illinois caselaw that's on point here, I'd be interested to hear about it. But if you're just a nonlawyer making stuff up about what you imagine the law to be, please leave me out of the discussion.
-
@divVerent @jonoleth If you're aware of any specific Illinois caselaw that's on point here, I'd be interested to hear about it. But if you're just a nonlawyer making stuff up about what you imagine the law to be, please leave me out of the discussion.
@mjd @jonoleth I am not even American. If in your country machines and companies are "persons" and have human rights that have priority over the human rights of _humans_, then your whole country is wrong. What's next, voting rights in federal elections for corporations? Second Amendment for AIs?
But yeah, that might indeed be the case.
In my country it is "societas delinquere non potest". A company _cannot_ be defendant of a crime - only the people actually performing the actions can.
But yeah, done here. Let's see what broken new case law will come from Trumpistan.
-
@divVerent @mjd
OpenAI is interfering in the insurance company's contract? Or, at least, that's what they're suing for -
@mjd TBH I do not think OpenAI should be responsible. They're just providing a fancy random text generator to the public. And it's outright impossible to teach a random text generator to _not_ output a specific kind of text, as whatever you do, there is a way around it.
The woman should pay all costs, as per the usual "vexatious filings" or "frivolous lawsuits" standards.
Plus, the law in her state against practicing law without a license starts with "No person shall...". ChatGPT isn't a person.
@divVerent It's funny how one can use "just" to *just*ify anything. Let's reduce "they hyped their product as «the artificial superintelligence magical clever tool», but didn't even bother adding safety guardrails or disclaimers about output not being legal advice, and now they're getting reamed" as "they're _just_ providing a fancy random text generator to the public."
Or let's not.
-
@jonoleth @mjd Pretty sure it's common knowledge that LLMs are nothing but random text generators.
OpenAI is a company, not a person. From what I understand, the law banning unlicensed legal advice bans _persons_ and gives them a penalty for doing so anyway.
But OpenAI, being a company, cannot commit crimes (after all, how to put a company in prison?). Only the employees can. So the question is which concrete employee committed a crime there. (Yes, some say companies _can_ commit crimes, but then solve the problems by making an employee / owner / ... actually criminally liable - but then they are the ones who have committed the crime)
The question is rather, have any employees of OpenAI committed a crime there? If any employee at OpenAI _knew_ that it tries to give legal advice, and did not implement any countermeasures, then that employee has committed a crime. That's the case no matter how the "random text generator" works.
If someone tries to get legal advice out of a magic 8-ball, AND the company producing the 8-ball does not implement any countermeasures (such as writing in the manual that responses it gives cannot be used as legal advice), then they can potentially be held liable. Except that in case of a mechanical device that works strikingly like a die it may not be necessary to put such a disclaimer
"Pretty sure it's common knowledge that LLMs are nothing but random text generators."
Absolutely not. Maybe in tech circles but the rest of the world has no clue whatsoever how LLMs work. And OpenAI is more than happy to keep it that way
"OpenAI is a company, not a person."
For legal purposes, most countries treat companies as distinct legal entities, and not just in the US. Still, this is pretty off-topic
-
@diazona I don't think it is a shame that this happened to this woman. It appears that she is a very ordinary type of vexatious litigant, except that she is also being aided by ChatGPT.
@mjd @diazona Though I'm not a lawyer (thank Cthulhu, or belly rubs to it's acolyte Menhit @antipope_cats) I do recall the Scottish courts taking exception to a "vexatious litigant" a while ago. It ended badly for said litigant.
-
@mjd @krupo that MIT article demonstrated 5% of AI implementations are profit making. 95% are loss making. so when the investment goldrush mania ends or winds down, 95% of invested amounts will be wiped out. it's billions so it will have a disruptive and negative economic effect that I think we will likely experience as recession. and then that 5% of profit making implementations will be what carries forward, with further investment being modelled on those (anyone can do that right now).
@falcennial @mjd @krupo
Looking to the foreseeable future, the AI boosters (especially those using AI to write their "opinions") will blame the bubble-burst on the Trump-Bibi War (no, they can't escape the blame) and *it's* crash.
How much are Darien Scheme share certificates worth these days? -
@mjd “41. On October 29, 2025, OPENAI amended the terms and usage policies of ChatGPT to prohibit users from using ChatGPT to provide tailored legal advice. Prior to the October 29, 2025 emendation, ChatGPT’s terms of use did not prohibit users from using ChatGPT to draft legal papers, conduct legal research, provide legal analysis or give legal advice.”
-
@mjd TBH I do not think OpenAI should be responsible. They're just providing a fancy random text generator to the public. And it's outright impossible to teach a random text generator to _not_ output a specific kind of text, as whatever you do, there is a way around it.
The woman should pay all costs, as per the usual "vexatious filings" or "frivolous lawsuits" standards.
Plus, the law in her state against practicing law without a license starts with "No person shall...". ChatGPT isn't a person.
@divVerent @mjd "barratry" - that's a term I remember from the Scottish Legal High Heidjuns dealing with a similar "vexatious litigant". I remember looking it up (but not the precise definition).
-
@divVerent @mjd ChatGPT is not a person, which is why ChatGPT is not being sued. OpenAI sells a tool that gave her legal advice, and they certainly didn't say anywhere that it's actually just a "fancy random text generator"
Wait, what?
They *sell* this shit? And charge money for it?Where the holy cat turds do they find clients? On the Internet?
(No, I've never tried to use an AI.)
-
@divVerent It's funny how one can use "just" to *just*ify anything. Let's reduce "they hyped their product as «the artificial superintelligence magical clever tool», but didn't even bother adding safety guardrails or disclaimers about output not being legal advice, and now they're getting reamed" as "they're _just_ providing a fancy random text generator to the public."
Or let's not.
@adriano @mjd There already is a disclaimer:
> What you cannot do. You may not use our Services for any illegal, harmful, or abusive activity. For example, you may not:
[...]
> Represent that Output was human-generated when it was not.
[...]
> Output may not always be accurate. You should not rely on Output from our Services as a sole source of truth or factual information, or as a substitute for professional advice.
[...]
> You must not use any Output relating to a person for any purpose that could have a legal or material impact on that person, such as making credit, educational, employment, housing, insurance, legal, medical, or other important decisions about them.Pretty much looks like clearly disclaiming to me.
-
A woman sues her insurance company for terminating her disability benefits. They reach a settlement and agree that the suit will be dismissed with prejudice.
She decides she doesn't like the settlement and asks her lawyers to reopen the case.They say they can't: it was dismissed, and in the settlement she agreed not to reopen the case.
She asks ChatGPT if her attorneys are lying to her. It says they are. She fires them and continues pro se, advised by ChatGPT.
CharGPT generates legal arguments for reopening the case, which she files, and 21 more motions, a subpoena, and eight other notices and statements, which she files.
The court denies her motion to reopen the case.
Advised by ChatGPT, she files a new suit against the insurance company and submits 44 more motions, memoranda, etc., which include citations to nonexistent cases.
Now the insurance company has sued OpenAI for tortious interference with their settlement contract.
https://storage.courtlistener.com/recap/gov.uscourts.ilnd.496515/gov.uscourts.ilnd.496515.1.0_1.pdf
HA!!!!!!!
-
@mjd TBH I do not think OpenAI should be responsible. They're just providing a fancy random text generator to the public. And it's outright impossible to teach a random text generator to _not_ output a specific kind of text, as whatever you do, there is a way around it.
The woman should pay all costs, as per the usual "vexatious filings" or "frivolous lawsuits" standards.
Plus, the law in her state against practicing law without a license starts with "No person shall...". ChatGPT isn't a person.
@divVerent @mjd This is simplistic to the point of being false. Long before we had LLMs, we had Clippy, which was smart enough to say “it looks like you’re writing a memo.” OpenAI and its counterparts can unquestionably add a “it looks like you’re seeking legal advice” detector to their products. They already, supposedly, try to detect whether their users are attempting self-harm. LLMs evolved from classification software, so this kind of thing is in their roots.
