Patient-Facing Wellness Chat Boundary Watch
A companion app a person using an AI wellness chatbot (Woebot-style, Replika-style, crisis-line-adjacent) installs alongside it; the app reads session transcripts, flags signs of dependency or boundary erosion, and gently nudges the user toward human support when patterns emerge.
adults using AI chat tools for emotional support or mental wellness
- Session-pattern monitor that flags rising dependency cues (daily use spikes, attachment language, avoidance of human contact)
- Boundary-erosion alert when the AI starts affirming distorted beliefs or acting as a substitute for professional care
- Gentle 'talk to a human today' nudges calibrated to usage patterns, not alarmist
- One-tap handoff to local crisis lines, GP, or NHS talking-therapy referral depending on user location
LLM mental health tools are scaling into consumer app stores while the paper documents exactly the structural harms — dependency, boundary erosion, distorted belief amplification — that patients themselves have no way to detect in real time.
BMJ, APA, Stanford, and Psychiatric Times all document rising harm; 20M+ MAU on Character.AI alone with 17+ min sessions; multiple lawsuits and Character.AI Jan 2026 settlement confirm real users at risk.AI driven psychosis and suicide are on the rise, but what should regulators do? - BMJ ↗Artificial intelligence, wellness apps alone cannot solve mental health ↗Character.AI and Google agree to settle lawsuits over teen mental health ↗
No consumer product found that monitors third-party AI chat sessions for dependency patterns; existing solutions are regulation, lawsuits, and built-in chatbot guardrails rather than a companion watch app.AI Companion Crackdown: 5 States, 3 Lawsuits, Who Survives? ↗Using generic AI chatbots for mental health support: A dangerous trend - APA ↗
Wellness subscriptions exist (Character.AI alone generated $32.2M) but the target users are in dependency states with low self-insight, making B2C conversion weak; likely needs B2B/clinician/insurer distribution.Character AI 2025 by the Numbers: 20M MAUs, $32.2M Revenue ↗More families sue Character.AI developer - CNN ↗
Regulatory wave (NY S-3008C law, CA/IL/TX/FL pending), APA federal pressure on FTC, Section 230 carve-outs, and active academic scrutiny indicate a durable, growing problem well beyond any single chatbot.AI Companion Crackdown: 5 States, 3 Lawsuits, Who Survives? ↗New study warns of risks in AI mental health tools - Stanford ↗
iOS/Android sandboxing blocks third-party apps from reading other apps' transcripts; would require fragile Accessibility-service scraping or cooperation from each chatbot vendor; significant privacy/legal exposure under HIPAA-adjacent mental health data rules.Exploring the Dangers of AI in Mental Health Care - Stanford HAI ↗