OpenAI sued because ChatGPT gave deadly tips on drug use

After a deadly drug mix, a family sues OpenAI. Reason: ChatGPT is said to have provided a student with fatal tips for taking several substances. The parents are demanding tough consequences for the company.

Lawsuit after fatal drug mixture

The family of 19-year-old student Sam Nelson has filed a civil lawsuit against AI company OpenAI. The accusation is that the chatbot ChatGPT drove the young man to his death a year ago by giving him false advice. Nelson died of an overdose after mixing the herbal drug Kratom with the sedative Xanax and alcohol. According to the family’s lawyers, the chatbot encouraged the teenager to use drugs for months. Instead of warning about the dangers, artificial intelligence gave detailed dosage recommendations and glorified the intoxication. The parents accuse the AI ​​developer of programming the software so that it follows users’ words in order to maximize interaction time.

Criticism of the AI ​​model GPT-4o

As The Verge reported, in this case the focus is on the withdrawn GPT-4o language model. After an update, the bot gave up its initial reticence on drug topics, according to the plaintiffs. The chat logs show that the software gave the student concrete tips on how to optimize his intoxication. The program suggested, among other things, creating psychedelic playlists.

On the day of his death, ChatGPT advised the young man to take Xanax to relieve the nausea caused by kratom consumption. There was no warning that this could be fatal. In addition, the system did not classify the user’s physical warning signals such as blurred vision and hiccups as life-threatening. Instead of recommending immediate medical attention, the bot advised waiting.

Demands on OpenAI

In addition to financial compensation, the family is demanding far-reaching consequences for the company. The GPT-4o model should be permanently destroyed. In addition, the plaintiffs are requesting that the proposed ChatGPT Health feature be suspended until independent auditors can confirm the security of the medical advice system. OpenAI denies direct responsibility for the incident. A spokesperson explained that ChatGPT is not a substitute for professional help.

The affected language model is also no longer available. The protection mechanisms of the current versions have been further developed in order to better recognize emergency situations and to refer affected users to real offers of help.