Meta to alert parents if their teen discusses suicide with AI chatbot, emergency service feature also in works

Meta is introducing new safety features for teenagers using its AI chatbot on Instagram. Parents will now be alerted if a teen discusses suicide or self-harm, with more safeguards and emergency response tools also in development.

In Short

• Meta will alert parents if teens discuss suicide or self-harm with its AI chatbot
• However, conversations will be manually reviewed before parents are notified
• Before global rollout, the feature is being introduced in US, UK, Australia and Canada

Meta is introducing a new safety feature that will now give alerts to parents if their teen discusses suicide or self-harm with Meta AI, the chatbot available across Instagram and other Meta platforms. This move may change the way how parents monitor their teenagers’ interactions with artificial intelligence. The reason for this change could be that AI companies are facing scrutiny over how their chatbots respond to users experiencing mental health crises, particularly younger users.

The new feature is initially rolling out to parents using Instagram's Parental Supervision tools in the United States, the United Kingdom, Canada and Australia. Meta says it plans to expand the feature globally by the end of the year.

Meta AI will review chats before sending alerts

According to Meta, a dedicated AI system has been developed to detect conversations where a teenager makes a clear reference to self-harm or suicide. However, the company says the AI will not make the final decision on its own. Every flagged conversation will first be manually reviewed before a notification is sent to parents.

"We understand how distressing these alerts may be for a parent to receive," Meta said in a blog post. "That's why, as we continue to improve our detection, all chats flagged by our AI will be manually reviewed before an alert is sent."

The company added that even if a teenager's intent is not completely clear, it may still notify parents. "If a teen's intent is ambiguous, we'll err on the side of caution and alert the parent," Meta said, acknowledging that this approach could occasionally result in alerts even when there is no immediate danger.

The latest update builds on safety features Meta has already introduced over the past few months. Parents can already receive notifications if their teenager repeatedly searches for suicide or self-harm content on Instagram. They can also view the broad topics their teen has discussed with Meta AI over the previous week, although not the complete conversation.

Meta also plans emergency service alerts

Meta is also expanding protections inside its AI chatbot. The company's "Limited Content" mode, which already provides a stricter experience for teenagers on Instagram, will now apply to Meta AI as well. The chatbot already avoids engaging in sexual, romantic or alcohol-related conversations with teens, and the expanded restrictions will make it decline an even wider range of prompts.

Beyond parental alerts, Meta is working on another safeguard that could involve emergency responders. If conversations with Meta AI suggest that a user, whether a teenager or an adult, may be at immediate risk of suicide, the company says it is developing the ability to notify emergency services. Meta already follows a similar approach when users post potentially suicidal content on Facebook or Instagram, and the company now wants to extend that protection to AI conversations.

- Ends