Today we had a fire alarm in the office.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev As 90% of fire alarms are drills, it makes perfect sense to respond with the most likely scenario.
( /s in case it wasn’t obvious. Rather startling that people are arguing with you.)
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev idk Dude, when theres a fire alarm I leave the building and make sure everyone made it out, and dont write shit in Slack
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev ... and even if it was a drill, you're supposed to leave. So not only the thing has no criteria, just parrots what it has heard before, it's also parroting the wrong thing.
-
@tagir_valeev "Dear Munich emergency services, there's a strong smell of smoke in our open office, and the flames are really messing up the colors of my IDE's dark mode. Is this an actual emergency? Looking forward to hearing from you…"
@mhd @tagir_valeev
„I hope this e-mail finds you well and unscorched …“ -
@tagir_valeev Exceptions like that are reasonable, I think, with the caveat that 'being prepared to not die' is the actual work on any good day.
@majick @tagir_valeev There are two kinds of alarm testing. One is as you described, where they are testing the alarm structure and functionality. You should get advance notice to ignore the alarms, preferably with a reminder to listen to announcements just in case there's a real emergency in the middle of their test. The other kind is testing the human element, so yeah, you have to leave when they tell you to because you never know.
-
@tagir_valeev More galling still, a scheduled test of a fire alarm system typically *still includes evacuation.* Leaving the building *is* the drill. I have never worked in an office where there was any condition under which occupants are told to ignore the alarm.
Ignoring alarms leads to alarm fatigue which then leads to failure. Alarms either exist for a reason or they don't. A device that says otherwise is a broken device. You're right, devices like that will kill.
@majick @tagir_valeev There could be some cases where the test checks sound etc, where evacuation is not the drill, but those indeed would be exceptions.
In Latvia, in 2013 fire alarms were repeatedly set off in a supermarket. Security just reset them, and employees & shoppers returned, then ignored the alarms.
The building collapsed and 54 people died.Whenever I hear a fire alarm, I first get the fuck out, then I figure out what's happening.
https://lv.wikipedia.org/wiki/Lielveikala_%22Maxima%22_sagr%C5%AB%C5%A1ana_R%C4%ABg%C4%81
-
@majick @tagir_valeev There could be some cases where the test checks sound etc, where evacuation is not the drill, but those indeed would be exceptions.
In Latvia, in 2013 fire alarms were repeatedly set off in a supermarket. Security just reset them, and employees & shoppers returned, then ignored the alarms.
The building collapsed and 54 people died.Whenever I hear a fire alarm, I first get the fuck out, then I figure out what's happening.
https://lv.wikipedia.org/wiki/Lielveikala_%22Maxima%22_sagr%C5%AB%C5%A1ana_R%C4%ABg%C4%81
@richlv @tagir_valeev Operators resetting/muting the alarm without understanding why it fired is a perfect example of alarm fatigue. A tragedy like that underscores why it's a Big Fuckin' Deal to avoid it.
The root cause of a failure like that is almost never the dude who did that. It's the circumstances that led to that dude thinking it was the correct thing to do.
Then people die.
My own opinion that evacuation is always the drill. Working on the alarm device, be it wiring, programming, or the noise that comes out of it, is part of working on an end-to-end system that includes people going away from the alarm.
-
@majick @tagir_valeev There are two kinds of alarm testing. One is as you described, where they are testing the alarm structure and functionality. You should get advance notice to ignore the alarms, preferably with a reminder to listen to announcements just in case there's a real emergency in the middle of their test. The other kind is testing the human element, so yeah, you have to leave when they tell you to because you never know.
@sbourne @tagir_valeev I don't agree with this because no alarm should go ignored, but I do understand why it's done that way in the real world. And why it's the default method.
Nobody[1] calls a reliability engineer before putting together their building maintenance punchlist and sending dudes with ladders.
1. note: nobody except my kid's boyfriend who is a chief facilities engineer, or my kid who is a marine engineer and grew up around rigid high-reliability high-risk operations. They're exceptions.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev even if it is "just a drill", you do need to leave the workplace!!!!! fucking LLMs!
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
As it gets much more attention, than I expected, here are two clarifications:
1. The Slack message was written after the person evacuated properly. It was written via phone while staying at the designated area outside the building.
2. Nobody asked AI advice explicitly. It was configured to answer automatically if it thinks it can help you. The configuration was updated after this incident. -
As it gets much more attention, than I expected, here are two clarifications:
1. The Slack message was written after the person evacuated properly. It was written via phone while staying at the designated area outside the building.
2. Nobody asked AI advice explicitly. It was configured to answer automatically if it thinks it can help you. The configuration was updated after this incident.@tagir_valeev The ironic thing is that if you look at the probabilities that are represented in the training data, and likely the real world - it was very likely statistically correct. All the more reason that humans should be the arbiters of human decisions not AI or even algorithms
-
As it gets much more attention, than I expected, here are two clarifications:
1. The Slack message was written after the person evacuated properly. It was written via phone while staying at the designated area outside the building.
2. Nobody asked AI advice explicitly. It was configured to answer automatically if it thinks it can help you. The configuration was updated after this incident.@tagir_valeev phew! I’m glad the configuration is updated!
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev Having worked in the Chemical Industry all my life my first instinct is evacuate, carry out roll call and then establish whether or not it was a real fire. My colleagues and I were all fire and rescue trained but would only act on small fires we always called the fire service. Many of the sites I worked on had alarms that went to the local fire control and triggered a turn out, a turn out of 6 engines at one site plus a general alert to other brigades
-
@richlv @tagir_valeev Operators resetting/muting the alarm without understanding why it fired is a perfect example of alarm fatigue. A tragedy like that underscores why it's a Big Fuckin' Deal to avoid it.
The root cause of a failure like that is almost never the dude who did that. It's the circumstances that led to that dude thinking it was the correct thing to do.
Then people die.
My own opinion that evacuation is always the drill. Working on the alarm device, be it wiring, programming, or the noise that comes out of it, is part of working on an end-to-end system that includes people going away from the alarm.
@majick @tagir_valeev As much as I love to talk about alert noise/fatigue, in that case other factors were contributing.
A bit of pressure from the supermarket chain, a bit of leftover mentality from the russian occupation times. Similar to the "I'm not afraid of no virus, I'm not gonna mask!" etc. -
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev Charles Darwin has entered the chat.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev oh, just saw this in my feed and immediately recognized the buildings, that's just around the corner were we live! Hope it wasn't serious and everybody is alright?
-
As it gets much more attention, than I expected, here are two clarifications:
1. The Slack message was written after the person evacuated properly. It was written via phone while staying at the designated area outside the building.
2. Nobody asked AI advice explicitly. It was configured to answer automatically if it thinks it can help you. The configuration was updated after this incident.@tagir_valeev …apparently this is the start of your next book 100 LLM mistakes and how you can’t avoid them.
Actually, I might just steal that my own having written most footnoted blog post ever
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev
Have you performed the MOTHER test and ensured its priority is employee wellbeing and not corporate?Alien had some warnings about AI too
-
@tagir_valeev oh, just saw this in my feed and immediately recognized the buildings, that's just around the corner were we live! Hope it wasn't serious and everybody is alright?
@maxsz no, nothing serious!
