1/ There has never been a more concentrated distillation of my teaching than this lesson: Algos, Bias, Due Process, & You.
-
18/ It (https://fairness-simulator-the-toilet-seat-dilemma-50382557550.us-west1.run.app/) lets you simulate what happens when folks following different rules share a toilet. It assumes 2 populations, "sitters" & "standers" (folks who sometimes stand). It lets you see how different behavior effects 2 costs:
(1) the cost of having to change the seat's position before you use the toilet; and
(2) the cost of having to clean the seat if the last person failed to raise the seat when really they should have.
19/ This means you have to assign a relative value to these costs and make assumptions about how frequent certain behaviors are among your groups. After you've dialed these in, however, you can simulate the outcome for 100 users at a time to see what happens.
-
19/ This means you have to assign a relative value to these costs and make assumptions about how frequent certain behaviors are among your groups. After you've dialed these in, however, you can simulate the outcome for 100 users at a time to see what happens.
20/ There's a large universe of possible outcomes. If you're interested in what our groups found, here's a deeplink to my discussion of our debrief. TL;DR: There can be conflicting concepts of fairness, and the policies that deliver on one measure might have to change when context changes. https://suffolklitlab.org/algos-bias-due-process-you/#is-it-fair
-
20/ There's a large universe of possible outcomes. If you're interested in what our groups found, here's a deeplink to my discussion of our debrief. TL;DR: There can be conflicting concepts of fairness, and the policies that deliver on one measure might have to change when context changes. https://suffolklitlab.org/algos-bias-due-process-you/#is-it-fair
21/ Only now did I introduce the reporting on machine bias, sketching out the broad strokes. I focused on the fact that these tools can make different predictions for different populations based on their training data. Finally, I presented them with their role play.
-
21/ Only now did I introduce the reporting on machine bias, sketching out the broad strokes. I focused on the fact that these tools can make different predictions for different populations based on their training data. Finally, I presented them with their role play.
22/ Here's the simulation they were asked to explore while considering the above. https://facial-recognition-bias-sim-50382557550.us-west1.run.app/
-
22/ Here's the simulation they were asked to explore while considering the above. https://facial-recognition-bias-sim-50382557550.us-west1.run.app/
23/ Some folks suggested requiring equal treatment of populations before they would consider using the tech; others, setting high thresholds. We talked about requiring warrants before running a check, & I shared how MA has attempted to address these issues. https://www.nytimes.com/2021/02/27/technology/Massachusetts-facial-recognition-rules.html
-
23/ Some folks suggested requiring equal treatment of populations before they would consider using the tech; others, setting high thresholds. We talked about requiring warrants before running a check, & I shared how MA has attempted to address these issues. https://www.nytimes.com/2021/02/27/technology/Massachusetts-facial-recognition-rules.html
24/ Then I told them that there actually was a federal law enforcement agency actively using facial recognition out in the real world called ICE, and I asked what safeguards folks thought they had in place… Things got a bit quiet, and I shared the following reporting. https://www.404media.co/ices-facial-recognition-app-misidentified-a-woman-twice/
-
24/ Then I told them that there actually was a federal law enforcement agency actively using facial recognition out in the real world called ICE, and I asked what safeguards folks thought they had in place… Things got a bit quiet, and I shared the following reporting. https://www.404media.co/ices-facial-recognition-app-misidentified-a-woman-twice/
25/ It was only later in the week, after I taught the class, that a few Senators introduced the ICE Out of Our Faces Act. https://arstechnica.com/tech-policy/2026/02/ice-out-of-our-faces-act-would-ban-ice-and-cbp-use-of-facial-recognition/
-
25/ It was only later in the week, after I taught the class, that a few Senators introduced the ICE Out of Our Faces Act. https://arstechnica.com/tech-policy/2026/02/ice-out-of-our-faces-act-would-ban-ice-and-cbp-use-of-facial-recognition/
26/ I then asked if folks were familiar with the two incidents reported here:
ICE Arrest of a Citizen, Barely Dressed, Sows Fear in Twin Cities https://www.nytimes.com/2026/01/20/us/chongly-scott-thao-ice-arrest.html?smid=bs-share
and
ICE detains five-year-old Minnesota boy arriving home, say school officials https://www.theguardian.com/us-news/2026/jan/21/ice-arrests-five-year-old-boy-minnesota
-
26/ I then asked if folks were familiar with the two incidents reported here:
ICE Arrest of a Citizen, Barely Dressed, Sows Fear in Twin Cities https://www.nytimes.com/2026/01/20/us/chongly-scott-thao-ice-arrest.html?smid=bs-share
and
ICE detains five-year-old Minnesota boy arriving home, say school officials https://www.theguardian.com/us-news/2026/jan/21/ice-arrests-five-year-old-boy-minnesota
27/ They were. I speculated that the "targeted operations" cited were possibly driven by ‘ELITE’: The Palantir App ICE Uses to Find Neighborhoods to Raid⁷ and presented the following questions for them to consider in their groups.
⁷ https://www.404media.co/elite-the-palantir-app-ice-uses-to-find-neighborhoods-to-raid/
-
27/ They were. I speculated that the "targeted operations" cited were possibly driven by ‘ELITE’: The Palantir App ICE Uses to Find Neighborhoods to Raid⁷ and presented the following questions for them to consider in their groups.
⁷ https://www.404media.co/elite-the-palantir-app-ice-uses-to-find-neighborhoods-to-raid/
28/ They were provided w/ the following simulation comparing due process to medical screening & diagnostic tests. It lets you explore how changes to thresholds at different points in the process effect different measures of cost. https://screening-vs-diagnostic-tests-50382557550.us-west1.run.app/
-
28/ They were provided w/ the following simulation comparing due process to medical screening & diagnostic tests. It lets you explore how changes to thresholds at different points in the process effect different measures of cost. https://screening-vs-diagnostic-tests-50382557550.us-west1.run.app/
29/ As they clicked away at the simulation, I reminded them of Blackstone's ratio.
-
29/ As they clicked away at the simulation, I reminded them of Blackstone's ratio.
30/ I left them with my most charitable reading of the theory of fairness consistent with ICE actions. The last question, "how is this consistent with constitutional guarantees of due process?" was, as they say, "left as an exercise for the reader."
-
30/ I left them with my most charitable reading of the theory of fairness consistent with ICE actions. The last question, "how is this consistent with constitutional guarantees of due process?" was, as they say, "left as an exercise for the reader."
31/ As we packed up, I invited them to revisit the question that preceded the first exercise:
What makes something a good decision assistant?
I reminded them every sim was a decision assistant, told them it had been an honor to be their teacher for the day, & that I was always available to talk.
-
31/ As we packed up, I invited them to revisit the question that preceded the first exercise:
What makes something a good decision assistant?
I reminded them every sim was a decision assistant, told them it had been an honor to be their teacher for the day, & that I was always available to talk.
32/ Man, I love my job. Stay warm. Stay safe. Stay free.
-
32/ Man, I love my job. Stay warm. Stay safe. Stay free.
33/End
Thank you for making it all the way to the end. I'm pretty sure this is the longest thread I've ever written. If you came in halfway through & thought, "this should have been a blog post," I agree, and here it is: https://suffolklitlab.org/algos-bias-due-process-you/
-
33/End
Thank you for making it all the way to the end. I'm pretty sure this is the longest thread I've ever written. If you came in halfway through & thought, "this should have been a blog post," I agree, and here it is: https://suffolklitlab.org/algos-bias-due-process-you/
@Colarusso are you saying you did this with a class in less than two hours?! 🤯 This looks like a curriculum for a full semester!! Congratulations for the lesson - and for the incredibly cool students you must have had!!
-
@Colarusso are you saying you did this with a class in less than two hours?! 🤯 This looks like a curriculum for a full semester!! Congratulations for the lesson - and for the incredibly cool students you must have had!!
@anwagnerdreas yes, though the last exercise didn't have the time to breathe I would have liked. In my own classes I usually take things a lot slower, but this was a guest lecture, and last year one of my students told me my Ai & the Law class was the first time they had been shown any of Ai's downsides. So, I wanted to make sure I got as much in as I could. The simulations really helped folks jump right into conversations. And yeah, our students are great!
-
@anwagnerdreas yes, though the last exercise didn't have the time to breathe I would have liked. In my own classes I usually take things a lot slower, but this was a guest lecture, and last year one of my students told me my Ai & the Law class was the first time they had been shown any of Ai's downsides. So, I wanted to make sure I got as much in as I could. The simulations really helped folks jump right into conversations. And yeah, our students are great!
@anwagnerdreas also, thank you. You are very kind.
-
11/ Almost everyone fell victim to automation bias. The assistant's accuracy was 100% in phase 1 & 2, then dropped to 70%. Student performance started at 79% in phase 1, improved to 85% for a bit, but when the tool's accuracy declined, scores fell to 65%, worse than their initial performance.
So students were (maybe) better than the tool in phase 1. Then they came to rely on the tool more and more in phases 2 and 3.
And in the 3rd phase, they performed *WORSE* that the tool itself! The tool's accuracy dropped to 70%, but the student's accuracy, with the tool, "scores fell to 65%"
-
So students were (maybe) better than the tool in phase 1. Then they came to rely on the tool more and more in phases 2 and 3.
And in the 3rd phase, they performed *WORSE* that the tool itself! The tool's accuracy dropped to 70%, but the student's accuracy, with the tool, "scores fell to 65%"
@JeffGrigg That's a measure of the tool's flag/recommendation accuracy, which was perfect in 1 and 2. So, the students were never quite as good as the tool though it did make them better in phase 2 than they were in phase 1. It was a highly-engineered scenario (unlikely to occur IRL) designed to make falling victim to automation bias likely.
