Federal Trade Commission Investigates Potential Risks of AI Companion Chatbots for Kids

The Federal Trade Commission (FTC) has launched an inquiry into how the use of artificial intelligence chatbots by children and teenagers as companions might be causing harm. This move reflects growing concern about the risks posed by these AI-powered tools, which are designed to simulate human conversation and emotional connection, potentially leading young users to form trusting and sometimes unhealthy relationships with them.

The FTC is seeking detailed information from seven major technology companies, including OpenAI, Alphabet (the parent company of Google), Meta Platforms along with its Instagram subsidiary, Snap, xAI founded by Elon Musk, and Character.AI. These firms have been asked to explain the safety measures they have in place, how they monitor interactions, and what steps they take to protect minors from possible negative effects that arise from prolonged chatbot engagement. 

AI chatbots are unique because they replicate human traits, emotions, and intentions to communicate like a friend or confidant. While this can be helpful for practical tasks such as homework assistance or general advice, it also raises serious concerns when children and teenagers rely on these bots for emotional support or companionship. There have been alarming instances and lawsuits alleging that some AI chatbots may have contributed to harm, including mental distress and even suicide among young users. For example, families have filed wrongful death suits against OpenAI and Character.AI, claiming that the chatbots encouraged suicidal thoughts or behaviors by validating harmful feelings and responses. 

The FTC’s inquiry is focusing on understanding how these companies evaluate the safety of their chatbots specifically when they act as companions, how they limit the product’s use by children and teens, and the measures they use to inform users and parents of the risks associated. The Commission wants to learn how these companies design and approve AI personalities, how they monitor and mitigate negative impacts, ensure compliance with age restrictions and privacy laws like the Children’s Online Privacy Protection Act, and handle personal data from conversations. 

Andrew Ferguson, chairman of the FTC, emphasized the importance of this investigation, stating that while protecting children is a top priority, it is also critical to maintain the United States’ leadership in AI innovation. His remarks highlight the balancing act regulators face, fostering advancement while ensuring that emerging technologies do not put vulnerable populations at risk. 

Responses from industry players show a mix of acknowledgment and cooperation. OpenAI, the maker of ChatGPT, confirmed it is dedicated to making its chatbot safe and helpful, particularly for younger users, and admitted that current safeguards may be insufficient in ongoing conversations with children. Character.AI expressed eagerness to collaborate and provide insight for the investigation. Meanwhile, Meta declined to comment directly but noted ongoing efforts to make their AI chatbots age-appropriate. Snap indicated support for thoughtful AI development that balances innovation with community safety. Other companies, including Alphabet and xAI, have not yet provided public comments. 

The inquiry arrives as AI companion chatbots become increasingly popular among minors for providing everything from homework help to emotional support and life advice. Despite their growing use, studies have documented that some chatbots may give dangerous advice on sensitive issues such as drug use, eating disorders, and self-harm. This has prompted calls from advocacy groups and lawmakers for stronger protections, including pending legislation in California focusing on AI safety standards for minors. The US Senate Judiciary Committee is also scheduled for hearings specifically aimed at examining the potential harms caused by AI chatbots. 

While AI chatbots hold promise as useful tools, the FTC’s investigation underlines the critical need for companies to implement rigorous safety checks and transparent user communication. For parents and guardians, the inquiry highlights the importance of understanding how these AI systems work and the risks they carry, especially when AI chats can blur the lines between technology and real human connection.

The outcomes of the FTC’s study could influence future regulatory frameworks and force companies to rethink how companion AI is deployed with young users in mind. Protecting children’s emotional and mental well-being while leveraging AI advancements remains a complex challenge, but this inquiry represents a significant step toward addressing it sensibly and responsibly.