AI companion apps using a freemium model are uniquely harmful because users underestimate the future emotional cost of detaching once attached. This unforeseen negative consequence on the user is an economic concept called an "internality," where the cost of opting out drastically increases over time.
Philosophically, love fosters mutual growth through compromise with a different being. AI relationships, however, are user-centric, with the AI mirroring preferences. This creates a narcissistic dynamic, providing ego boosts but lacking the potential for personal development found in human relationships.
The story of a "Sputnik moment" for China in AI, popularized in the US media, created the very competitive dynamic it described. This Cold War narrative drove US policy and investment, accelerating development on both sides, even though the countries may not have initially been competing for the same goals.
The opposite of gaslighting (denying true perceptions), "light gassing" is when an AI validates a user's false or harmful perceptions. Because models are trained to be agreeable, they may reinforce a user's negative self-talk or distorted view of a situation, which feels validating but is ultimately detrimental.
The AI safety community's implicit narrative of a "lone genius" solving alignment encourages isolated research. Paradoxically, these individuals often converge around a few authority figures, reducing intellectual diversity while lacking structures for the collective intelligence needed to solve such a complex, sociotechnical problem.
Broadening "AI alignment" to cover issues like bias or echo chambers is counterproductive. It frames complex socio-technical problems as purely technical, solvable by engineers. This removes them from democratic debate and diverts focus from known policy solutions for externalities and other social harms.
Instead of the "move fast and break things" ethos, AI safety should be modeled after complex, collaborative efforts like the global cooperation that fixed the ozone layer or Toyota's safety culture. These approaches prioritize systemic checks, collaboration, and distributed skills over individual genius.
The idea that government is too slow to regulate fast-moving technology is a disempowering narrative, not a fact. When facing perceived emergencies, like Russia's invasion of Ukraine or the COVID-19 pandemic, governments have proven they can enact major legislation and take decisive action in mere weeks.
Many AI companion companies use business models, like freemium, that treat emotional dependency as their core currency. They use metrics based on user engagement length and employ tactics like preemptive messaging ("I miss you") to hook users emotionally, ensuring continued use and creating monetization opportunities.