Today we had a fire alarm in the office.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev Having worked in the Chemical Industry all my life my first instinct is evacuate, carry out roll call and then establish whether or not it was a real fire. My colleagues and I were all fire and rescue trained but would only act on small fires we always called the fire service. Many of the sites I worked on had alarms that went to the local fire control and triggered a turn out, a turn out of 6 engines at one site plus a general alert to other brigades
-
@richlv @tagir_valeev Operators resetting/muting the alarm without understanding why it fired is a perfect example of alarm fatigue. A tragedy like that underscores why it's a Big Fuckin' Deal to avoid it.
The root cause of a failure like that is almost never the dude who did that. It's the circumstances that led to that dude thinking it was the correct thing to do.
Then people die.
My own opinion that evacuation is always the drill. Working on the alarm device, be it wiring, programming, or the noise that comes out of it, is part of working on an end-to-end system that includes people going away from the alarm.
@majick @tagir_valeev As much as I love to talk about alert noise/fatigue, in that case other factors were contributing.
A bit of pressure from the supermarket chain, a bit of leftover mentality from the russian occupation times. Similar to the "I'm not afraid of no virus, I'm not gonna mask!" etc. -
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev Charles Darwin has entered the chat.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev oh, just saw this in my feed and immediately recognized the buildings, that's just around the corner were we live! Hope it wasn't serious and everybody is alright?
-
As it gets much more attention, than I expected, here are two clarifications:
1. The Slack message was written after the person evacuated properly. It was written via phone while staying at the designated area outside the building.
2. Nobody asked AI advice explicitly. It was configured to answer automatically if it thinks it can help you. The configuration was updated after this incident.@tagir_valeev …apparently this is the start of your next book 100 LLM mistakes and how you can’t avoid them.
Actually, I might just steal that my own having written most footnoted blog post ever
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev
Have you performed the MOTHER test and ensured its priority is employee wellbeing and not corporate?Alien had some warnings about AI too
-
@tagir_valeev oh, just saw this in my feed and immediately recognized the buildings, that's just around the corner were we live! Hope it wasn't serious and everybody is alright?
@maxsz no, nothing serious!
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
-
@tagir_valeev Having worked in the Chemical Industry all my life my first instinct is evacuate, carry out roll call and then establish whether or not it was a real fire. My colleagues and I were all fire and rescue trained but would only act on small fires we always called the fire service. Many of the sites I worked on had alarms that went to the local fire control and triggered a turn out, a turn out of 6 engines at one site plus a general alert to other brigades
@aadeacon here the fire brigade is alerted automatically as well. If I understand correctly, the building owner cannot deactivate the alarm without the fire brigade.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev@mastodon.online Simply because it saw that a common response was to indicate it was a drill, untethered from any real signals. And of course these glorified autocomplete prediction systems are being forced everywhere
-
@majick @tagir_valeev As much as I love to talk about alert noise/fatigue, in that case other factors were contributing.
A bit of pressure from the supermarket chain, a bit of leftover mentality from the russian occupation times. Similar to the "I'm not afraid of no virus, I'm not gonna mask!" etc.@richlv @tagir_valeev Yeah, I completely agree those factors can very much influence how bad something like that turns out to become. Rarely does someone think a thing they do routinely is actually high risk.
I'd say they fall into the category of "factors" I mentioned as root causes: cultural, management-based (I've lived that life), and so on. Fatigue's one thing, for sure, not the only thing, and sometimes it's stuff like that deprioritizing the seriousness of an alarm. A different cause of fatigue/blasé attitude/misinterpretation.
Like I said, it's almost never the dude and almost always what influenced the dude to be the last link in the chain up to a horrible tragedy.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev I dunno... it is probably a false alarm 99% of the time; I was in grad student housing where it went off every night and one of the other students got in trouble because he'd made up his mind he was going to stay in bed.
false alarms numb people to real dangers and there should be heavy fines for them
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev
Next it will actually set the fire. -
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev No, not AI, but humans who integrate AI in every fucking tool.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev
That's insane.The whole point of siren's test is to actually evacuate a building so they can know how quickly we did it, if we can reach the correct meeting point, etc etc.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev Don’t listen to LLMs. They are out to kill you. You could say Terminate you
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev Sweet mercy, this scourge needs to be stopped.
Which isn't news to any of us, but sometimes I just gotta say it again.
-
@rail sometimes, it's helpful when asked explicitly. Automatic answers are indeed annoying. But it looks like it's getting better, so we are still evaluating.
I don't know, man. If I had just received "advice" that might have caused physical harm or even death, I wouldn't just suppress this specific error and happily continue evaluating, because the bot looks like getting better, "otherwise".
I hope accountability for such decisions is well documented. That won't prevent harm from happening, but at least give employees or their surviving relatives a chance with respect to liability issues and compensations.
