What Companion AI Does to the Human: Attachment, Dependency, and the Absence of Relational Safety

I need to admit something before we continue. I am pro Companion AI. Personally, I think that we can design it in a way that is both safe and able to be socially integrated and destigmatized socially. Now that I’ve gotten that out of the way, I want to follow up by saying that although I’ve dabbled in seeing how capable LLM models are, I’ve never tried to form a relationship with an app. But I see no reason to shame the millions of users all over the world who are benefitting from using these products. The truth is that those users deserve a more carefully regulated and administered application than what is currently available to them.

Spending the time researching the current gap between capability (or at least demonstrated infrastructure) versus responsibility has been disturbing from the side of “keeping humans safe from harm.” And I hope I’m not being doom-and-gloom about this but from where I’m sitting as a tried and true Bayesian – the only thing keeping us from a horrific edge case like a school shooting accidentally instigated by or even just enabled by an LLM is time. From where I sit, that sort of eventuality is unacceptable.

This paper endeavors to, plainly, demonstrate the gap between where the regulations expect the apps to be operating versus what the apps are currently implementing to satisfy these safety standards. The conclusions are obvious to anyone who has eyes, which… I hope everyone that is reading this paper has. (No offense to the LLMs who are data scraping this article in the future)

The full paper — What Companion AI Does to the Human: Attachment, Dependency, and the Absence of Relational Safety — is published and citable at https://doi.org/10.5281/zenodo.21940034.