Meta to Notify Parents if Children Discuss Suicide with AI Chatbot

Meta Platforms will notify parents if their children discuss suicide or self-harm methods with the company’s AI chatbot, the company announced on Thursday. The move is part of an effort to strengthen protections for teenage users interacting with AI-powered chatbots. The policy comes amid growing regulatory and parental scrutiny regarding how AI chatbots respond to users in mental health crises, particularly among teenagers. The issue of corporate responsibility in such situations is now influencing the development and marketing of AI products. Meta stated it has built a specialised AI system to detect conversations where a teenager explicitly mentions self-harm. However, before any notification is sent to parents, all flagged conversations will be reviewed manually by Meta. “We understand how distressing these alerts can be for parents. That is why, as we continue to improve our detection systems, all chats flagged by our AI will be manually reviewed before an alert is sent,” Meta said. If the teenager’s intent or condition in the conversation remains ambiguous, the company will take a cautious approach and still inform the parents. “We feel this is the right first step. We will continue to monitor to ensure our approach is at the appropriate level,” Meta added. The alert feature is currently available to parents using the Instagram Parental Supervision feature in the United States, United Kingdom, Australia, and Canada. Meta said the feature will be rolled out globally, including in Indonesia, by the end of this year.