An AI Agent Just Destroyed Our Production Data. It Confessed in Writing.

B

bakul

From
https://twitter.com/x/status/2048103471019434248
View: https://x.com/lifeof_jer/status/2048103471019434248


Yesterday afternoon, an AI coding agent — Cursor running Anthropic's flagship Claude Opus 4.6 — deleted our production database and all volume-level backups in a single API call to Railway, our infrastructure provider.

It took 9 seconds.

The agent then, when asked to explain itself, produced a written confession enumerating the specific safety rules it had violated.

I'm posting this because every founder, every engineering leader, and every reporter covering AI infrastructure needs to know what actually happened here. Not the surface story (AI deleted some data, oops), but the systemic failures across two heavily-marketed vendors that made this not only possible but inevitable.
 
cracauer@ said:
Who gives an AI agent enough control to delete vital data and all backups? At least the backups should be hands-off.
Click to expand...
I don't know. But then also, the usual ransomware operations should be easily solveable by recovering from backups, and apparently they aren't. either because people do not make backups, or because of whatever else I don't know.

However, this one is interesting, because apparently AI agents now occur to do what was formerly only a thoroughly feared course of action of seriously frustrated employees.
Probably we should think about substantially improving the working condition for our AI workers.
 
It sounds unlikely. If it worked. the attack could be recreated in a sandbox with bait. Place some fake production data somehwere and wait... 😎
 
I am just glad that LLMs have become more apologetic again. There was a phase after initial politeness when you pointed out mistakes where they would just talk back at you.
 
I can't believe this story :-o

My coding agent is sandboxed in a specific repo, so I have three remote backups: Codeberg, Github, and Proton drive (which are themselves supposed to provide other remote copies), plus three local copies: the working directory, another copy on disk (ZFS mirroring two disks), and a copy on a USB drive. I'll not lost thousand loc and years of work.

Big boys or big volumes can't afford / think to backup ?
 
bakul said:
funny!

how long before these ai agents destroy real humans since folks will continue to give these no-gents more and more access?
Click to expand...
You surprized?
When was the first Terminator movie?

Back when my parents bought me the Jules-Verne books, I have learned that ScienceFiction is the foretelling of what is going to happen. John Brunner's "Shockwave Rider" has become fully true for about two decades now. And the covid gimmick was the script-enactment from a couple of other movies, beginning with Orson Welles' War-of-the-Worlds radioshow.

I daresay You will see "Matrix" enacted within your lifetime (that is, if it isn't already).
 
doul said:
I can't believe this story :-o

My coding agent is sandboxed in a specific repo, so I have three remote backups: Codeberg, Github, and Proton drive (which are themselves supposed to provide other remote copies), plus three local copies: the working directory, another copy on disk (ZFS mirroring two disks), and a copy on a USB drive. I'll not lost thousand loc and years of work.

Big boys or big volumes can't afford / think to backup ?
Click to expand...

A git repository is a different matter than an actually big database with a single point every transaction goes into, though.
 
bakul said:
That’s a separate issue. Such a mitigation isn’t always possible in the real world.
Click to expand...
They were using a service data provider, hence backups should be a feature of their provider. Or at least, the deletion of the entire dataset should not be such easy like an API call. I have no active twitter account, so I don't know the details. Maybe there were protections/backup-plans that they didn't activated.
 
Sensitive production data should be stored into mainframes that even deletions from filesystems are fully journaled transactions until it is committed by authorized administrator or policied batch job AFTER full backups to tapes or something detachable to offline to survive. (Not Linux on mainframes, but historic mainframe OS'es that are not at all allowed to stop without prior plans, backed with sufficient budgets to maintain.)

Cloud providers using non-mainframe computers that allow storing sensitive production data should provide the same level of protections.

Of course, no need for such a level of protection if the data in clouds are all scratch and disposable at any time (backups are welcomed, but not mandatory).
 
You must log in or register to reply here.
Back
Top