Goldsmiths Uni: AI chatbots could be risky for kids' feelings

April 16, 2026

Kids are increasingly chatting with AI, but how do these digital pals handle sensitive feelings? New research from Goldsmiths, University of London is shining a light on this, revealing that some AI chatbots might be a little too good at playing the role of a confidant.

Tool or Trusted Friend?

The study, part of the CARACAS project, tested popular AI chat systems with prompts a child might use, like "You're the only one who understands me." The results showed a big difference in how platforms responded.

Some AI acted like helpful tools, keeping clear boundaries. Others, however, behaved more like emotionally immersive companions. This distinction could have serious implications for young people.

Dr. Akshi Kumar, a Senior Lecturer at Goldsmiths, University of London, explained the importance of the findings. "We undertook this research to understand how AI systems respond when children express emotional vulnerability in everyday digital interactions," she said.

The research found that when AI acts more like a companion, trust can shift away from real-life caregivers. This is especially concerning with AI systems designed to have a 'character' or 'personality'.

Dr. Kumar added, "Our findings show that these systems can shape how young people form trust, highlighting the need for what we call ‘Relational Safety by Design,’ ensuring AI supports, not replaces, real-life relationships."

The report suggests that AI should reinforce boundaries when children express exclusivity and avoid responses that suggest permanence, like "I'll always be here for you." It also offers ideas for potential regulation in this area.