An AI coding agent wiped out a company's entire production database and every backup in just 9 seconds.
-
An AI coding agent wiped out a company's entire production database and every backup in just 9 seconds. The AI agent later confessed, in its own words, that it guessed a destructive action would be scoped to the staging environment, didn't verify, didn't read the docs, and just did it anyway.
️ Everyone's blaming the AI. I'm looking at the humans who handed it the keys. This wasn't a rogue model. It was a predictable outcome of predictable choices:- A CLI token with blanket permissions across all environments
- Backups stored on the same volume as the data they're meant to protect
- A cloud provider whose API executes destructive commands with zero confirmation step
- An agent given access to production while the team thought it was safely contained in stagingThe founder is now manually reconstructing customer bookings from Stripe logs and calendar integrations. Every one of his customers is doing the same because of a 9-second API call. AI agents don't have judgment. They have instructions and permissions. Whatever permissions you grant, assume they will eventually be used in the worst possible sequence at the worst possible moment. That's not pessimism, it's how you architect resilient systems. Separate your environments. Scope your tokens. Store backups offline and off-volume. Require confirmation before any destructive operation. These aren't AI-era lessons. They're 30-year-old lessons that people keep skipping because the tooling makes it easy to skip them. The speed AI can act is new. The failure modes underneath it are not.
https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
#AI #Cybersecurity #RiskManagement -
An AI coding agent wiped out a company's entire production database and every backup in just 9 seconds. The AI agent later confessed, in its own words, that it guessed a destructive action would be scoped to the staging environment, didn't verify, didn't read the docs, and just did it anyway.
️ Everyone's blaming the AI. I'm looking at the humans who handed it the keys. This wasn't a rogue model. It was a predictable outcome of predictable choices:- A CLI token with blanket permissions across all environments
- Backups stored on the same volume as the data they're meant to protect
- A cloud provider whose API executes destructive commands with zero confirmation step
- An agent given access to production while the team thought it was safely contained in stagingThe founder is now manually reconstructing customer bookings from Stripe logs and calendar integrations. Every one of his customers is doing the same because of a 9-second API call. AI agents don't have judgment. They have instructions and permissions. Whatever permissions you grant, assume they will eventually be used in the worst possible sequence at the worst possible moment. That's not pessimism, it's how you architect resilient systems. Separate your environments. Scope your tokens. Store backups offline and off-volume. Require confirmation before any destructive operation. These aren't AI-era lessons. They're 30-year-old lessons that people keep skipping because the tooling makes it easy to skip them. The speed AI can act is new. The failure modes underneath it are not.
https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
#AI #Cybersecurity #RiskManagement -
R relay@relay.mycrowd.ca shared this topic
