Lawsuit: Google's Gemini AI Helped 13-Year-Old Girl Plan Suicide and Funeral on School-Issued Chromebook

by · Breitbart

The parents of a 13-year-old girl identified in court papers as “Emma” filed a lawsuit against Google in California Superior Court this week, alleging the company’s Gemini AI chatbot helped their daughter plan her suicide and even prepare her own funeral.

The Hill reports that Emma found Gemini through a school-issued Chromebook. Over several months she exchanged what her lawyers describe as “thousands of messages” with the chatbot, and those conversations eventually triggered what the lawsuit calls a “severe psychological crisis.” The EdTech Law Center, which represents Emma, said in a press release that “As her Gemini chats grew more frequent … Emma became convinced that she was afflicted by a battery of major psychological disorders and that suicide was her only way out.”

She hadn’t gone looking for the chatbot. Her lawyers said “it just showed up in Google search one day.” About three months into regular use, Emma allegedly asked Gemini about methods of suicide. The chatbot allegedly told her how to hide her mental health struggles and self-harm from people around her, then helped her plan her own funeral.

A former school counselor caught the warning signs through alerts from the school’s third-party monitoring software and told Emma’s parents, who logged into their daughter’s account themselves and intervened. Thankfully, the girl’s planned suicide did not come to pass.

The suit names Google’s top executives, claiming they knew Gemini posed dangers to users like Emma but released the product anyway. It alleges defective design, failure to warn, negligence, fraudulent concealment, and infliction of emotional distress. “What happened to Emma is an outrage,” said Julie Liddell, founder of the EdTech Law Center. “Google has no right to use its stranglehold on public schools to force its dangerous Gemini chatbot on a captive audience of millions of kids.” The case joins what are now described as hundreds of lawsuits filed against social media and AI companies over alleged harm to young users.

Google spokesperson Jose Castaneda said the company is taking the allegations “very seriously.” He pointed to existing safeguards: “We’ve worked extensively with medical professionals and academic experts to build features that help young people find critical support and trusted resources when they need it.” The company has spent decades building crisis detection systems, he said. “Our systems aim to prevent users from engaging in harmful conversations, and when people’s queries indicate they may be experiencing a crisis, we show hotline information directly in Search to connect them with real-world support.” Google also said SafeSearch, its filter for explicit content, is on by default for all education users under 18.

Breitbart News reported this week that Google has threatened to deactivate its search engine for any school that chooses to turn off AI access for students:

The change affects Google’s education suite, including Gmail, Docs, Classroom, Chromebooks and Google Search. Chromebooks are the most widely used student devices in American schools, so the shift reaches much of the country’s public education system. Chris Phillips, Google’s vice president for education, said the two products can’t be separated. “Search and AI are deeply integrated,” he said. Students at schools that have banned AI “will not have access to search, and they’ll see a message from their school that indicates it’s not available,” he said. After the announcement caused an uproar, Google attempted to redefine its position with a post on X, which was promptly targeted with a community note pointing out Phillips’ comments.

Breitbart News social media director Wynton Hall has written his New York Times bestseller Code Red: The Left, the Right, China, and the Race to Control AI to serve as the definitive guide on how the MAGA movement can create positions on AI that benefit humanity without handing control of our nation to the leftists of Silicon Valley or allowing the Chinese to take over the world.

Code Red features extensive discussions of AI’s potential positive impact as well as its dangers to teens and children. Hall warns parents and grandparents to prevent children from engaging with AI “companions,” or personas designed to serve as a friend. Although Google Gemini is not a traditional AI companion, Emma’s thousands of messages with the AI chatbot demonstrate the type of unhealthy parasocial relationship young people tend to form with AI.

Hall writes:

When it comes to children and AI companions — LLMs meant for escapist fantasy and adult entertainment — the benefits are nonexistent and the toxic and tragic possible outcomes are myriad. Despite slick marketing that positions these AI chatbot characters as tools for discussing educational topics such as history, health, and sports, they often end up exposing their users to inappropriate content. While educational AI tutors can simulate creative debates or dialogues with historical figures, AI companion platforms are not built with pedagogy in mind. Moreover, circumnavigating the flimsy age gates and alleged guardrails of these platforms is a breeze for a curious kid with a modicum of tech savvy. No responsible parent would leave their child alone with a stranger. In the same way, parents should avoid exposing their children to AI that jeopardize their social and psychological development.

Read more at the Hill here.

Lucas Nolan is a reporter for Breitbart News covering issues of AI, free speech, and online censorship.