Have you been asked by a medical provider recently for consent to have an "AI" scribe record your visit?

[kimgadsdon@mastodon.online]

@wwahammy @emilymbender
I’ve suspected this at the vet office, too. At first, they asked (and I declined). Then at subsequent visits they didn’t ask, but I could see the waves jumping on their monitor which made me wonder….
Anyway, our sweet Sydney passed this week so that won’t be an issue anymore….

[ton@scholar.social]

@emilymbender
My (now former) therapist sprang this on me.

Site Not Found | Framer

(kyomei.ai)

Your points about the note taking being part of the care rang very true. Also, the what might be missed.

It also changed the interaction significantly. I was not overly concerned about the privacy of the contents, but the pauses where the therapist had to think for a moment and write notes gave me a quiet space and time to think. That disappeared.

[blogdiva@mastodon.social]

yes i was a declined. when she asked, i said because

1. the companies used sites with CSAM and other abuses

2. it’s spyware. each prompt acts like a honey-pot. since you are giving them the info, it by-passes HIPPA. in turn they get to use and sell that info however they please

3. as an antifascist activist, it puts my life in danger by giving companies ran by fascists access to my whole medical history.

my MD was shocked. they had no idea about the spyware angle or CSAM

@emilymbender

[ridogi@mastodon.social]

@emilymbender I opted out at my physical therapist last week. They told me all their patients who work in tech have opted out.

[wwahammy@social.treehouse.systems]

@kimgadsdon @emilymbender I'm so sorry about Sydney's passing. I lost my Yorkie, Penelope, in 2024 so I know the wonderful joy that a Yorkie provides and the seemingly unbearable loss when they're gone.

I hope the happy memories give you a bit of comfort.

[m3l155a@mastodon.social]

@emilymbender
Evidence shows litigation decrease if Drs have scribes. A Dr isn’t allowed to remember things in defence. It’s said “If it is not documented, it didn’t happen” even if it did happen & recall can be verified.

The direct effect:
1: more litigation = more insurance cost for the Dr & thus higher consult fees.
2: Drs who have psychological & emotional injury from spurious claims reduce/stop practice.

So there is high motivation for having a scribe.

[paco@infosec.exchange]

@emilymbender Really helpful. Thanks for sharing.

I often reason about it this way. There are very few things like this where, if you opt out this time, you can’t opt in next time. On the other hand, there are LOTS of situations, this being a good example, where opting out after you opted in is substantially more effort (or even impossible).

Opting out by default is usually a safe thing to do. You can always opt in later if you change your mind.

[sigsegv@hachyderm.io]

@emilymbender multiple times in fact. And I hate it.

[kwazekwaze@mastodon.social]

@Mikal @emilymbender
It's pinky promised by LLM vendors to the healthIT vendors selling to hospitals that data sent to their servers for the purpose of LLM input is not retained or used outside of inference.

HealthIT vendors are then happy to believe this at face value so they can sell AI hype to hospital execs desperate to force their clinicians to shove more patients through at any cost and slash labor costs.

Docs are told it's fine by IT who was told it's fine by the CIO who was told...

[emilymbender@dair-community.social]

@P__X Your experience is your experience, but I am **appalled** at what you're saying about consent here. The fullest version would be too long, so we're not actually doing informed consent? No thank you.

[kwazekwaze@mastodon.social]

@Mikal @emilymbender
Just to add a little more here: I listened to the R&D head of a large allegedly monopolistic EHR vendor tell a room full (1000s) of physicians and hospital CXOs they need to be less cautious and go full speed ahead in their adoption of AI.

It's being pushed really, really hard in healthcare software.

[m3l155a@mastodon.social]

@randocity @meltedcheese @emilymbender my medical defence insurer encourages its use because evidence shows lower litigation rates in consultations with scribes. The study was done using human scribes, predates the AI era.

[cogdog@cosocial.ca]

@emilymbender for a doctors perspective on the more profound side effects of “efficiency”

I Was an Enthusiastic Early Adopter of AI Scribes. Here’s Why I Stopped

A GP reflects on what eighteen months of ambient scribing taught them about the consultation they thought they already understood.

(benngooch.substack.com)

“I felt myself becoming a passive observer in encounters where I had previously been an active architect. I felt my clinical memory, my narrative identity, and my sense of connection to my patients beginning to erode at the edges.”

[cynthiarose@sfba.social]

@emilymbender psychiatry did it without informed consent. I am livid

[randocity@mstdn.social]

@Mikal @jrdepriest @emilymbender The problem is even knowing the doctor’s office recorded the interaction, other than via a whistleblower. If the recording is transcribed and then discarded by the doctor, how would a patient ever know? Once doctors realize they basically can’t get caught doing it, what or who will stop them?

Insurance companies aren’t going to care or even ask if the doctors collected their recordings illegally.‍️

[randocity@mstdn.social]

@M3L155A @meltedcheese @emilymbender AI hasn’t yet proven itself to be a reliable or useful witness in legal cases. I’m not even sure a lawyer has yet tried using an AI as a witness.

It is possible that having audio recordings of patient interactions could prove useful in courts, but that implies that doctors are being sued more now than in the past.

Insurers don’t derive benefits from AI directly, so I don’t understand this push.

[randocity@mstdn.social]

@M3L155A @meltedcheese @emilymbender For clarification, when I say that insurers aren’t deriving benefits from AI directly, I mean specifically the AI that’s being used in doctors offices, learning from patient recordings.

It is very possible, however, that insurance companies are using AI in their own internal systems, but those AI systems are entirely separate from AI used in doctor’s office patient systems.

[randocity@mstdn.social]

@Mikal @jrdepriest @emilymbender The only place where a doctor might get caught at doing it is if they produce a recording in a court of law as part of a legal case. The problem is, patient recordings would be considered part of HIPAA compliance and may be inadmissible if proper procedures are not followed.

There are definitely procedures to follow when introducing HIPAA data into a court case, such as giving the patient a chance to object.

[slotos@toot.community]

@emilymbender Funnily enough, transcribing can preserve privacy without issues. Whisper.cpp runs decently well on phones, and can be run on servers that process patient records under the same security constraints. Could easily be run locally even.

Problem is, that’s extremely hard to prove in the current „just slap a gear on it and call it steampunk” climate. I would definitely not trust a random provider.

And if they do „summarization”, forget privacy.

[2bfair@infosec.exchange]

@emilymbender Agreed. For any points that were valid, none of them necessitate the use of LLMs. Never mind without consent. Disgusting.