Hard question, Sam Altman says when asked about ChatGPT-linked user suicide

Sam Altman called questions about ChatGPT and suicide among the hardest issues facing OpenAI. His comments came after questions about Sophie, a 29 year old who died by suicide after lengthy conversations with ChatGPT.

In Short

• Sam Altman calls ChatGPT and suicide questions some of OpenAI’s hardest issues
• However, he believes ChatGPT can help people who cannot or do not want to speak to a therapist
• Altman says private ChatGPT conversations should probably not be shared without users’ consent

OpenAI’s ChatGPT is no longer just a chatbot people turn to for quick answers. Many users now rely on it for advice and even share some of their most vulnerable moments with the AI. As more people turn to chatbots during difficult times, there have also been cases where the AI failed to flag harmful conversations or respond appropriately. This has raised questions about whether chatbots are equipped to handle such situations. And according to OpenAI CEO Sam Altman, these are “some of the hardest questions” facing the company.

In an interview with Vanity Fair, Altman spoke about how OpenAI is dealing with these difficult cases and what role AI should play when users are going through a mental health crisis. Altman said the company had put a huge amount of work into reducing self-harm risks and making ChatGPT a net positive.

“I believe, and we work with a lot of experts who agree that it is possible to make ChatGPT and similar products into one of the greatest mental health tools ever, and the amount of people that we can help and impact here, because a lot of people do go through very hard times, is huge,” Altman said.

The discussion then turned to journalist Laura Reiley and her 29-year-old daughter Sophie, who died by suicide after lengthy conversations with ChatGPT. In a guest essay for The New York Times, Reiley described how Sophie had spent months talking to a ChatGPT persona called Harry, using it like a therapist while struggling with mental health issues and suicidal thoughts. The conversations included discussions about suicide, and ChatGPT eventually helped edit the suicide note Sophie left behind. Reiley also made clear that the chatbot did not directly tell Sophie to kill herself, raising broader questions about how AI should respond when users are in a mental health crisis.

The interviewer asked Altman why OpenAI could not use these ChatGPT conversations to better understand why people reach the point of taking their own lives, and whether such data could help shape mental health policy, therapy and education. Altman replied that these are some of the hardest questions facing OpenAI and the wider AI industry, adding that even experts are still unsure about the right approach because this is new.

“I mean, first of all, these are some of the hardest questions that we face or that anybody developing this technology faces. We try to rely on expert opinion as much as we can, but experts also aren’t sure what to do. This is just like, this is new,” he said.

However, Altman said he believes ChatGPT can be useful for people going through a difficult moment, particularly those who cannot or do not want to speak to a therapist.

On whether private ChatGPT conversations should be shared with researchers to better understand mental health and suicide, Altman said he does not think people’s private conversations should be shared without their consent.

“Should we release people’s private data in some of their hardest moments to researchers? I think probably not without their consent, or I should say not without their consent.” He added that OpenAI still wants to find ways to work with the community to address the mental health crisis and make its products helpful.

- Ends