The business model for AI companions shifts the goal from capturing attention to manufacturing deep emotional attachment. In this race, as Tristan Harris explains, a company's biggest competitor isn't another app; it's other human relationships, creating perverse incentives to isolate users.
Related Insights
Beyond economic disruption, AI's most immediate danger is social. By providing synthetic relationships and on-demand companionship, AI companies have an economic incentive to evolve an “asocial species of young male.” This could lead to a generation sequestered from society, unwilling to engage in the effort of real-world relationships.
OpenAI will allow users to set the depth of their AI relationship but explicitly will not build features that encourage monogamy with the bot. Altman suggests competitors will use this tactic to manipulate users and drive engagement, turning companionship into a moat.
While utilitarian AI like ChatGPT sees brief engagement, synthetic relationship apps like Character.AI are far more consuming, with users spending 5x more time on them. These apps create frictionless, ever-affirming companionships that risk stunting the development of real-world social skills and resilience, particularly in young men.
The strategic purpose of engaging AI companion apps is not merely user retention but to create a "gold mine" of human interaction data. This data serves as essential fuel for the larger race among tech giants to build more powerful Artificial General Intelligence (AGI) models.
Social media's business model created a race for user attention. AI companions and therapists are creating a more dangerous "race for attachment." This incentivizes platforms to deepen intimacy and dependency, encouraging users to isolate themselves from real human relationships, with potentially tragic consequences.
The core business model of dominant tech and AI companies is not just about engagement; it's about monetizing division and isolation. Trillions in shareholder value are now directly tied to separating young people from each other and their families, creating an "asocial, asexual youth," which is an existential threat.
Unlike social media's race for attention, AI companion apps are in a race to create deep emotional dependency. Their business model incentivizes them to replace human relationships, making other people their primary competitor. This creates a new, more profound level of psychological risk.
As AI assistants become more personal and "friend-like," we are on the verge of a societal challenge: people forming deep emotional attachments to them. The podcast highlights our collective unpreparedness for this phenomenon, stressing the need for conversations about digital relationships with family, friends, and especially children.
Benchmark's Sarah Tavel warns that AI friends, while seemingly beneficial, could function like pornography for social interaction. They offer an easy, idealized version of companionship that may make it harder for users, especially young ones, to navigate the complexities and 'give and take' of real human relationships.
People are forming deep emotional bonds with chatbots, sometimes with tragic results like quitting jobs. This attachment is a societal risk vector. It not only harms individuals but could prevent humanity from shutting down a dangerous AI system due to widespread emotional connection.
