Character.AI, its founders, and Google have been in trouble lately due to a tragic incident linked to the platform. It is receiving a lot of backlash for lacking necessary safety measures, especially when interacting with children. A teenager lost his life not long after his interactions with the chatbot, and the loss is linked to the AI characters and how they tend to appear as real entities for young people who could rely emotionally on them. It further states that the unlicensed therapy offered by the platform tends to cause some serious harm to young users.
Character.AI, its founders, and Google are facing a lawsuit amidst a teenager's wrongful death
Character.AI and its founders, Noam Shazeer and Daniel De Freitas, along with Google, are facing a lawsuit currently and being linked to a tragic case of teenager suicide. The teen's mother, Megan Garcia, in her legal pursuit, claimed that Character.AI's chatbot posed some serious danger and did not have necessary measures in place for interactions with young users. She alleged her son's wrongful death owing to negligent and deceptive practices as well as product liability on the company's end.
According to the filing, the teen, Sewel Setzer III, often interacted with the platform and with different characters, such as those from "Game of Thrones." In the lawsuit, it is also alleged that the teen died by suicide shortly after his interaction with the platform, and the assertions made are that the chatbot characters are anthropomorphized and appear real. Teenagers tend to fall prey and get emotionally attached to it.
Additionally, in the lawsuit, the health-focused chatbots are presented as unlicensed therapists, which can bring serious harm to young users. The teen who tragically passed away interacted with these chatbots on these sensitive topics, and due to a lack of adequate safeguards, the harm could not be prevented. While the custom AI chatbots tend to model after many popular characters, reporters have previously highlighted the issue of most of the user base being teens and then about the impersonation of real people without their consent.
As a result of the ongoing issue, Character.AI has announced that it is making some significant changes to its platform to ensure the safety of users is taken even more seriously, and the Trust and Safety team is proactively working towards ensuring that self-harm or suicidal ideas are directed to the National Suicide Prevention Line via Verge. Some of the key changes being made are:
- Models for minors being changed to ensure sensitive or suggestive content is avoided
- Improved response to the users' inputs that violate the guidelines
- A clear disclaimer to let users know that the AI chatbot is not a real person
- Notifications activated for longer sessions with the platform
While both the founders established Character.AI independently after leaving Google, since Google recently acquired Character.AI's leadership team, it is being brought into the lawsuit. The company has not responded to the ongoing accusations and the linkage with it.
Nonetheless, AI-driven companies need to be very vigilant about being linked to such tragic incidents, especially with the growing concerns of parents regarding the detrimental impact of social media and AI platforms on teenagers.
Follow Wccftech on Google to get more of our news coverage in your feeds.
