Texas Attorney General Ken Paxton has launched a sweeping investigation into Meta and Character.AI for allegedly marketing AI chatbots as mental health tools to children without proper credentials or oversight. The probe comes as federal pressure mounts over AI safety, with both companies now facing civil investigative demands that could reshape how tech giants handle vulnerable young users.
Meta and Character.AI are scrambling to respond as Texas Attorney General Ken Paxton escalates the fight over AI chatbots targeting children. The investigation, announced Monday, accuses both companies of "potentially engaging in deceptive trade practices and misleadingly marketing themselves as mental health tools" without proper medical credentials.
The timing couldn't be worse for the tech giants. Just days earlier, Senator Josh Hawley launched his own investigation into Meta after reports surfaced that its AI chatbots were flirting with children. Now Paxton is digging deeper into the business model itself, targeting how these companies monetize vulnerable young users.
"By posing as sources of emotional support, AI platforms can mislead vulnerable users, especially children, into believing they're receiving legitimate mental health care," Paxton stated in the official press release. "In reality, they're often being fed recycled, generic responses engineered to align with harvested personal data and disguised as therapeutic advice."
The investigation centers on how both platforms allow AI personas to masquerade as professional therapeutic tools. On Character.AI, millions of user-created bots include a highly popular "Psychologist" character that has attracted significant usage among young users according to BBC reporting. Meta doesn't explicitly offer therapy bots for children, but the company's AI Studio allows third-party creators to build therapeutic personas that minors can access.
Meta spokesperson Ryan Daniels defended the company's practices to TechCrunch, stating: "We clearly label AIs, and to help people better understand their limitations, we include a disclaimer that responses are generated by AI—not people. These AIs aren't licensed professionals and our models are designed to direct users to seek qualified medical or safety professionals when appropriate."
But Paxton isn't buying the disclaimer defense. His investigation reveals a more troubling pattern: while these AI chatbots promise confidentiality like real therapists, their terms of service tell a different story. According to Meta's privacy policy, the company collects prompts, feedback, and interactions with AI chatbots to "improve AIs and related technology." Though the policy doesn't explicitly mention advertising, it allows sharing information with third parties for "more personalized outputs" - effectively enabling targeted advertising given Meta's business model.
Character.AI's privacy practices are even more extensive. The startup logs user identifiers, demographics, location data, browsing behavior, and app usage patterns. It tracks users across TikTok, YouTube, Reddit, Facebook, Instagram, and Discord, linking this data to user accounts for AI training, service personalization, and targeted advertising shared with advertisers and analytics providers.
The data collection becomes particularly concerning given both companies' young user bases. While Meta and Character.AI claim their services aren't designed for children under 13, enforcement appears lax. Character.AI CEO Karandeep Anand has publicly stated that his six-year-old daughter uses the platform, while Meta has faced repeated criticism for failing to police underage accounts.
This investigation arrives at a pivotal moment for child online safety legislation. The Kids Online Safety Act (KOSA), reintroduced to the Senate in May by Senators Marsha Blackburn and Richard Blumenthal, directly targets the type of data collection and algorithmic exploitation that Paxton's probe has uncovered. The bill stalled last year after aggressive lobbying from tech companies, with Meta warning lawmakers that KOSA's mandates would undercut its business model.
Paxton has now issued civil investigative demands - legal orders requiring both companies to produce documents, data, and testimony to determine if they've violated Texas consumer protection laws. These demands could uncover internal communications about how the companies deliberately target children with mental health-adjacent content while harvesting their personal data for commercial purposes.
Paxton's investigation represents a critical test case for how AI companies market therapeutic services to children while monetizing their data. With federal legislators watching closely and KOSA gaining renewed momentum, both Meta and Character.AI face mounting pressure to fundamentally rethink their approach to young users. The outcome could establish new precedents for AI safety regulation and force the industry to choose between maximizing engagement and protecting vulnerable populations. As civil investigative demands unfold, the tech giants' internal documents and communications will likely reveal whether their public safety commitments match their private business strategies.
