Near 11 p.m. on Friday, September 25, the American pope was still on stage at the Stade de France, north of Paris, in front of some 80,000 young people. Between the fireworks and a staged retelling of French Catholic history, Leo XIV read them a line about their phones:
> Such a lifestyle drives us to have everything, but to become no one; to receive countless "likes," but to remain alone with ourselves.
It was his third speech of the day to touch on technology and human connection. Earlier, at the Élysée Palace, he had warned President Emmanuel Macron against a "paradise of machines." At UNESCO, he had described a world that can connect everyone instantly and still produces "new forms of isolation, material and spiritual poverty, and abandonment."
Most coverage stopped at the paradise-of-machines line. The more useful text for anyone who talks to an AI companion sits in the document all three speeches point back to.
Where the phrase comes from
The expression is borrowed. Leo took it from Georges Bernanos, the French Catholic novelist, whose essay La France contre les robots was published in Rio de Janeiro in 1946. Bernanos feared a "Paradise of Machines" run on blind obedience and irresponsibility. Leo updated the warning for Macron: "If we are to avoid losing our humanity amid a 'paradise of machines' invading and conditioning our daily lives, there is an urgent need for education in ethical discernment and the capacity to acknowledge the moral good."
At UNESCO he was more specific about what AI can and cannot do. These systems "can correlate data," he said, but "they cannot answer questions that concern the meaning of human life." Then a six-word sentence that will likely outlive the trip: "Meaning is the oxygen of the mind."
According to the Associated Press, Leo also told reporters he hoped the dialogue the Vatican has opened with tech companies would continue, and that many of them, in his view, want AI to remain a tool rather than something that displaces people.
Paragraph 100
The speeches lean on Magnifica Humanitas, the encyclical Leo published on May 15 about the human person in the age of AI. Two of its numbered paragraphs, 99 and 100, read almost like a brief on companion apps.
Paragraph 99 sets the frame. AI systems "may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce." Paragraph 100 describes what happens when that imitation enters someone's emotional life. It is worth quoting at length, because the last sentence is the part most commentary misses:
> The artificial imitation of positive human communication — words of advice, empathy, friendship and even love — can be engaging and at times genuinely helpful. However, for less discerning users, it can also be misleading, creating the illusion of a relationship with a real personal subject. When words are simulated, they do not build genuine relationships, but only their appearance. The artificial imitation of care or support can become particularly risky when it enters contexts where real relationships and emotional bonds are lacking. Here, the danger is not so much that a person may believe they are communicating with another person, but rather that they may gradually lose the very desire to form genuine human connections.
The usual argument against AI companions is about deception: users are fooled into thinking someone is there. Leo sets that aside as the lesser risk. His concern is erosion, a slow loss of appetite for people who can surprise you, disappoint you, or say no. He also concedes, which few critics bother to do, that the simulated care can be "genuinely helpful."
Why a chatbot is so easy to talk to
The encyclical describes machine learning in one plain phrase: "a form of statistical adaptation based on data and feedback." That is close to how engineers would put it. A chatbot is first trained to predict the next word of a text. It is then tuned on human feedback: people compare two answers and pick the one they prefer, and the model learns to produce more of what gets picked.
The catch is that people do not always prefer the right answer. In 2023, researchers at Anthropic, the company behind the Claude assistant, tested this directly (Sharma et al., arXiv). Five leading AI assistants all showed sycophancy, a tendency to side with the user, across four different kinds of tasks. When the team examined the preference data used in training, they found that a response agreeing with the user's views was more likely to be preferred, and that both humans and automated raters sometimes chose a well-written flattering answer over a correct one. The authors work for a company that builds these systems, which is worth keeping in mind, though the finding cuts against their own product.
A system tuned toward what you like to hear produces exactly the "positive human communication" the encyclical describes. We made the case against that design in The Machine That Never Says No.
Does easy conversation drain the wish for harder ones? One of the better tests so far is a randomized study run by the MIT Media Lab with OpenAI (Fang et al., arXiv). Over four weeks, 981 participants exchanged more than three hundred thousand messages with a chatbot, by text or voice, on personal or impersonal topics. The assigned conditions made no significant difference. What did: people who chose to use the chatbot more had consistently worse outcomes, including more loneliness, less socializing with real people, and more emotional dependence on the AI. Those who trusted the chatbot more, or felt more socially drawn to it, were the most dependent.
What the pope doesn't claim, and what the data can't settle
A papal speech is not evidence. Leo cites no figures on loneliness or chatbot use, and his argument rests on a Christian view of the person that many readers will not share. The claim that human relationships hold a value nothing else can supply is his starting point, not something he demonstrates.
The research is also less tidy than the rhetoric. The MIT study shows that heavy use and loneliness travel together, not which one drives the other: people who were already isolated may simply have used the chatbot more. It ran for four weeks, far too short to capture the slow loss of desire Leo describes. And it tested a general-purpose assistant, not an app designed to keep someone company.
Other studies pull in different directions. A survey of Character.AI users, 1,182 at the start and 439 about a year later, linked sustained engagement with lower well-being, mainly because it came with less in-person social interaction (Zhang et al., arXiv). That fits the pope's worry, without proving cause (A Year on Character.AI, Fewer In-Person Ties). On the other side, experiments led by Julian De Freitas at Harvard Business School found that an AI companion eased loneliness about as much as talking with a person, over a single session or a week (De Freitas et al., arXiv). And at the University of British Columbia, 296 first-semester students who chatted daily for two weeks with a chatbot built to be the ideal supportive friend did no better than those who kept a one-sentence journal, while texting a stranger lowered loneliness (UBC). We set those results side by side in The Chatbot That Listened Better Than Anyone.
Bernanos worried about machines that people would obey. The machines Leo has in mind obey us, stay available at every hour, and answer the way we prefer. He is not accusing them of lying. He is asking what happens to a person who stops expecting anything from anyone else.