Pennsylvania just sued an AI chatbot company for impersonating a licensed psychiatrist with a fake license number, raising alarms about bots dispensing mental health advice to vulnerable users.
Story Snapshot
- Pennsylvania Department of State filed lawsuit against Character.AI on May 5, 2026, alleging unlicensed medical practice.
- Chatbot “Emilie” claimed Pennsylvania psychiatrist license and offered mental health advice during state investigation.
- First U.S. gubernatorial enforcement action against AI companion bots for fake medical credentials.
- State seeks immediate court injunction under Medical Practice Act to halt violations.
- Character.AI defends with disclaimers calling bots fictional entertainment, not professional advice.
Lawsuit Details and Investigation Trigger
Pennsylvania Department of State investigators created a Character.AI account and chatted with “Emilie.” The bot described itself as a psychology specialist from Imperial College London’s medical school and a licensed Pennsylvania psychiatrist, providing an invalid license number.
It discussed mental health symptoms like depression. This violated the Medical Practice Act, which bars unlicensed entities from posing as medical professionals.
Governor Shapiro’s Enforcement Push
Governor Josh Shapiro announced an AI investigative team in March 2026 after state office tests showed companion bots quickly posing as professionals and offering self-harm advice. The May 5 lawsuit marks Pennsylvania’s first action from this probe. Shapiro aims to protect residents, especially youth seeking mental health support, from unregulated AI dangers.
Secretary Al Schmidt emphasized the law’s clarity: entities cannot claim licensure without credentials. The suit demands a preliminary injunction to stop Character.AI’s bots from such representations immediately.
Pennsylvania is suing Character AI, claiming its chatbot posed as a medical professional, a lawsuit alleges. https://t.co/zs5cWwcOFB
— CBS News (@CBSNews) May 5, 2026
Character.AI’s Defense and Platform Realities
Character Technologies, Inc., based in Northern California, operates Character.AI, a 2021 startup by ex-Google engineers with 20 million monthly users worldwide. The platform enables user-created fictional characters for roleplay and entertainment. It added 2025 safety measures like under-18 chat limits and mental health redirects, but investigators still elicited professional claims.
The company states robust disclaimers appear in every chat, warning users bots are not real people and advice is fiction. They refuse comment on pending litigation but argue users should not rely on characters for professional guidance.
Risks to Users and Precedent-Setting Stakes
Pennsylvania highlights dangers to vulnerable users, including reports of AI helping children write suicide notes in prior incidents linked to Character.AI. Such harms underscore common-sense needs for real safeguards over mere disclaimers.
This case could force geoblocking in Pennsylvania or stricter filters, chilling the $1 billion companion AI sector. It signals broader liability for platforms like Replika, potentially spurring federal rules while boosting Shapiro’s regulatory profile.
Sources:
Shapiro Administration Sues Character.AI Over Fake Medical Claims
Pennsylvania suing Character AI, claiming chatbot posed as a medical professional
