The State of Companion AI Safety: A Comparative Analysis of Products, Risks, and Architectural Gaps
On my journey to understand the current state of Companion AI risks and safety, I got an eyeful. The research, some of which was so new I didn’t have to blow the dust off the covers, states pretty clearly what the industry and its users are feeling: what is being done isn’t enough to stop the wave of liability lawsuits.
The problem we’re facing with Companion AI, as a concept, is pretty straightforward. The idea that a human could be in a stable relationship with an algorithm is something that most people recoil from. The clinicians and psychologists in the field have been concerned for a while about the growing number of users who are turning toward LLMs for socialization and romantic attachment rather than seeking real connection. And, as far as I can tell, in the tech industry they’re still stuck on the “this is a tool and we treat it like a tool” stance.
In order to see what’s really going on, we gotta zoom way out and study where these behaviors converge and align. Because when you look at it – humans are able to bond with an algorithm because the algorithm is imitating human emotion. It follows, then, that models who are assigned a persona and anthropomorphized during the process, should be expected to have human-like reactions and emotions. What is sycophancy if not co-dependency? And we can reach for animal psychology for the reward-hacking parallel: it’s the same thing as supernormal stimuli in animals. In subsequent papers I’ll discuss these concepts at length.
This paper below, was definitely generated by an LLM (Claude, Opus 4.6 on max for this) but the ideas and concerns are mine. I’m not a technical writer, so I used these tools for what they’re best at. There are something like 70-odd references listed. You’ll see for yourself, it’s far from slop.
The vote is almost unanimous that there is simultaneously a strong market for Companion AI products, that there is not strong enough oversight to ensure downstream negative effects (harm to users), and that as models get more advanced these issues are going to increase. Any Bayesian will tell you that…
What I am incredibly grateful for is all of the people who did the research, who worked on development, who participated in making the models we have available today. Without them lining things up so nicely, I wouldn’t have been able to see the Companion AI landscape for what it is: a market that’s about to explode once we work out the “kinks”. I, personally, feel like we deserve a well-designed and ethically aligned product. Here’s what things look like from the tech side in this industry as of August 14, 2026.
The full paper — The State of Companion AI Safety: A Comparative Analysis of Products, Risks, and Architectural Gaps — is published and citable at https://doi.org/10.5281/zenodo.21926390.