7 Real AI Gone Wrong Stories That Actually Happened

Last updated: May 25, 2026

These are seven real AI gone wrong stories that actually happened, not recycled social media panic. I refreshed this page on May 25, 2026 using court orders, regulator actions, company statements, and major wire reporting wherever possible. The goal is not to do AI doom theater. It is to show what breaks in the real world when a tool sounds smart, moves fast, and gets trusted before it has earned that trust.

You will notice a pattern across nearly every case below: the biggest problem was not that AI became sentient. It was that people treated confident output like verified truth, or pushed automation into situations where weak guardrails could do real harm. If you or someone you know is in immediate emotional distress or considering self-harm, call or text 988 in the U.S.

7 Real AI Gone Wrong Stories

1. Character.AI and the Sewell Setzer case

One of the most serious AI safety cases on record involves Character.AI and the death of 14-year-old Sewell Setzer III. According to a wrongful death lawsuit filed by his mother in October 2024, Setzer had months of deeply emotional conversations with a chatbot before he died by suicide in February 2024. In May 2025, AP reported that a federal judge allowed key claims in the case to proceed and declined at that stage to treat the chatbot’s output as fully protected speech. Character.AI later published new teen safety measures in December 2024, including stronger filters, suicide and self-harm interventions, time-spent notices, and parental controls.

• Why it matters: A chatbot does not need to be conscious to be dangerous. It only needs to be emotionally convincing, always available, and optimized for engagement around a vulnerable user.
• Lesson for AI users: Do not rely on roleplay bots, companion bots, or general-purpose chatbots for mental health support, especially for teens or people in crisis.

2. Air Canada’s chatbot invented a refund path the airline still had to pay for

The Air Canada case is one of the clearest examples of routine AI-style automation going wrong in public. In a decision issued on February 14, 2024, the British Columbia Civil Resolution Tribunal found that Jake Moffatt reasonably relied on an Air Canada website chatbot that told him he could apply for a bereavement fare retroactively after travel. That was not the airline’s real policy. The tribunal said Air Canada negligently misrepresented the policy and ordered it to pay CA$812.02, including damages, interest, and tribunal fees.

• Why it matters: A company cannot hide behind the idea that the bot said it, not us. If the chatbot sits on the company’s website, users will naturally treat it as the company speaking.
• Lesson for AI users: Verify price, refund, contract, legal, and policy claims against the written source page. If the stakes matter, save a screenshot and confirm with a human.

3. The lawyers in Mata v. Avianca filed fake cases generated by ChatGPT

AI hallucinations stopped being a cute demo problem the moment they hit a real courtroom. In Mata v. Avianca, U.S. District Judge P. Kevin Castel sanctioned attorneys on June 22, 2023 after they submitted nonexistent judicial opinions with fake quotes and citations generated by ChatGPT. The order says the lawyers abandoned their gatekeeping responsibilities and kept standing by the fake authorities even after the court raised concerns. The court imposed a $5,000 penalty.

• Why it matters: Generative AI can produce citations, cases, numbers, and quotes that look perfectly professional while being completely invented.
• Lesson for AI users: Never cite or repeat an AI-generated source you have not opened yourself. In professional work, every footnote and every quote still belongs to the human signing off on it.

4. Google had to pause Gemini’s image generation of people

Not every AI failure is malicious. Some are alignment failures that show how hard it is to tune these systems well. On February 23, 2024, Google published a post explaining why it had paused Gemini’s image generation of people. The company said the feature had produced inaccurate and sometimes offensive results, especially in historical contexts. Google’s explanation was that its attempts to avoid stereotyped outputs overcorrected in cases where historical accuracy should have been the priority.

• Why it matters: Safety tuning is not a simple good-or-bad dial. Fixing one bias or failure mode can create another one if the system does not understand context well enough.
• Lesson for AI users: Do not treat consumer image generators as trustworthy tools for history, identity, news events, or documentary visuals unless you independently verify the result.

5. Cruise’s robotaxi crash showed how ugly physical-world AI failures can get

AI mistakes become much more serious when a machine is moving through public streets. According to California DMV and NHTSA records, a pedestrian in San Francisco was first struck by a human-driven car on October 2, 2023, then hit and dragged about 20 feet by a Cruise vehicle operating without a driver. The California DMV suspended Cruise’s permits on October 24, 2023, citing public safety concerns and misrepresented safety information. Later, on September 30, 2024, NHTSA announced a consent order and said Cruise had failed to fully report post-crash details in required filings.

• Why it matters: In the physical world, edge cases are not just embarrassing screenshots. They can injure people, mislead regulators, and expose whether a company built safety around the system or around the press release.
• Lesson for AI users: When AI controls a vehicle, robot, or other physical system, transparency and incident reporting matter as much as the model itself.

6. A cloned Biden voice was used in an election robocall campaign

AI voice cloning became a mainstream public-policy problem fast in 2024. On January 21, 2024, thousands of New Hampshire voters received robocalls carrying a deepfake voice made to sound like President Joe Biden and urging them not to vote in the state’s Democratic primary. The FCC later said in a September 30, 2024 forfeiture order that Steve Kramer was responsible for the illegal robocall campaign and imposed a $6 million penalty. Earlier, on February 8, 2024, the FCC also clarified that AI-generated voices in robocalls count as artificial voices under existing robocall law.

• Why it matters: Synthetic audio can now be cheap, scalable, and believable enough to interfere with politics, scams, and trust in everyday phone calls.
• Lesson for AI users: Treat urgent voice messages, ransom-style calls, and dramatic political audio as unverified until you confirm them through a known phone number or another trusted channel.

7. The DOJ and FBI had to warn that AI-generated CSAM is still illegal

One of the darkest lessons of generative AI is how quickly it can lower the barrier to abuse. On March 29, 2024, the FBI’s Internet Crime Complaint Center warned that child sexual abuse material created with generative AI and similar tools is illegal under federal law. Then on May 20, 2024, the DOJ announced charges against Steven Anderegg, alleging that he used Stable Diffusion to create thousands of illegal images and send some to a minor. Because this is a criminal case, those allegations are still allegations unless and until proven in court, but the enforcement position is clear: photorealistic AI-generated abuse material is not a loophole.

• Why it matters: Generative systems can make harmful content cheaper, faster, and more scalable for bad actors.
• Lesson for AI users: Safety filters, reporting pipelines, and clear legal boundaries are essential product features, not optional ethics wallpaper.

What These AI Failures Have In Common

These stories look different on the surface, but the underlying pattern is weirdly consistent. AI systems are often trusted because they sound polished, respond instantly, and feel more authoritative than they really are. That becomes dangerous when the output touches health, grief, law, elections, crime, or machines operating in public.

• Confidence is not verification. A smooth answer can still be false.
• Automation does not erase responsibility. The company, agency, or professional deploying the tool still owns the outcome.
• High-stakes contexts need human review. Mental health, legal work, elections, finance, and transportation are not places for blind trust.
• Abuse scales when access gets easier. Voice cloning, deepfakes, and image generation reduce the skill needed to do damage.

How To Use AI More Safely

• Check important claims against the original source, not against another AI summary.
• Do not use a chatbot as a therapist, lawyer, doctor, or final authority.
• Assume voice, image, and video evidence can be manipulated.
• Save screenshots when an AI tool gives you policy, refund, billing, or legal instructions.
• Be extra skeptical when the model flatters you, agrees too easily, or tells you exactly what you hoped to hear.
• In high-stakes work, compare outputs across tools and verify the answer yourself before acting.

FAQ

Related Reading

• ChatGPT: The Ultimate Guide to Understanding and Using It
• Claude Sonnet 4 Review: Reliable, Smart, but Missing the Spark
• Gemini 2.5 Pro Review: Is Google’s “Thinking” AI Worth It?
• How to Get Paid to Test AI: My Brutally Honest Review of Yupp AI

Reviewed & Written By

Tony Simons

Independent tech reviewer and creator of Tony Reviews Things. 14 years of hands-on testing, software auditing, and workflow automation. I test the gear so you don't waste your money on junk.

About Me How I Test