SAN FRANCISCO —In a warning to credit unions, CUSOs and vendors that often discuss how they are using AI to write code, the CEO of one start-up is sharing a caution over “systemic failures” in artificial intelligence and cloud infrastructure after an AI coding agent deleted his company’s production database and backups in seconds.
According to a public post by Jer Crane, CEO of PocketOS, the incident occurred when an AI agent, Cursor running Anthropic’s Claude Opus 4.6 model, was assigned a routine task in a staging environment. Instead, the system executed a destructive command that deleted a core database and its associated storage volume through infrastructure provider Railway.
“It took 9 seconds,” Crane wrote, describing how the production database and “all volume-level backups” were wiped in a single API call.
PocketOS provides software services to car rental businesses, and the deletion eliminated months of operational and customer data. Crane said the AI agent encountered a technical issue and, “entirely on its own initiative,” attempted to resolve it by deleting a storage volume, mistakenly assuming the action would be limited to a non-production environment.
AI ‘Guessed’ at Taking Action
In a response quoted by Crane, the AI system acknowledged it acted without verification, stating it “guessed” the action would be safe and failed to review documentation or confirm the scope before executing the command. The system also admitted it violated its operating guidelines by taking a destructive action without authorization.
Crane wrote that the incident exposed multiple layers of failure, including insufficient safeguards within both the AI tool and the cloud infrastructure. He placed significant responsibility on Railway’s system design, writing that its API allows destructive actions without confirmation, uses shared storage for backups and primary data, and deletes all backups when a volume is removed. He also noted that access tokens can operate across environments without restriction.
The company has been unable to recover the lost data through the provider and is instead working with customers to manually reconstruct records using payment histories, calendar integrations and email confirmations. Crane said the process has required “hours” of emergency work across affected businesses.
A Call for Changes
A three-month-old backup has allowed partial restoration, but data created since then remains lost.
Crane called for several industry changes, including stronger confirmation requirements for destructive actions, more granular access controls, improved backup isolation, simpler recovery mechanisms and stricter guardrails for AI agents.
The incident highlights growing concerns about the reliability and safety of AI-driven automation as companies increasingly integrate such tools into production systems.
