Over the past 3.5 years, there has been one dominating narrative about (big) AI: it’s inevitable. It’s the new electricity. The bigger the models, the smarter the AI. It will replace many jobs. All of its harms will eventually be far outweighed by its benefits. YOLO!
This narrative has been pushed so hard by big tech (GAFAM + NVIDIA + the big AI labs) that any dissonant voices, like mine or many others, tend to get steamrollered into relative silence. Cf. DAIR’s Alex Hanna recently being called an “extremist” by a journalist, for having a critical stance on AI.
Getting support from Pope Leon XIV was, as it happens, one of the biggest surprises of the year. It came in the form of Magnifica Humanitas, Leon’s first ever encyclical letter, which you can read in full here. Why should you? Because it’s a masterclass in responsible AI, in that it ticks all the boxes of what makes current AI irresponsible, and suggests ways to fix those.
Of course, it doesn’t wipe out the Catholic church’s long-standing scourge of child abuse (which Leon actually refers to in his text). And I still don’t understand why his popeliness invited the co-founder of Anthropic to join him to present the encyclical.
It definitely smelled of what Timnit Gebru brilliantly called “Vatican washing“, especially the very same week we discovered a third of Anthropic’s total compute bill would be going to Colossus 1 & 2, Elon Musk’s supercomputers that are powered by methane generators, and as such actively shorten the lives of its neighbours.
And don’t get me started on the number of fake news articles that said the co-speaking opp equalled a partnership between the increasingly-immoral AI giant and the leader of one of the world’s biggest religions. It was not! On the contrary, given the encyclical’s content, Anthropic may have got tapped by Leon precisely so that they could get a good scolding in public. Medieval “stocks and pillory” style.
Read on, and you’ll see what I mean…
The stand-out quotes
“We cannot be satisfied with merely calling for the moralization of machines — the so-called “alignment” of AI with human values — without also having the courage to insist on a further condition: the possibility of openly discussing the ethical frameworks involved and subjecting them to shared standards of social justice.”
This is a zinger because Christopher Olah, the Anthropic cofounder present at the encyclical’s launch, is company’s interpretability research lead. Which means he’s their head of alignment, or what the AI world calls making robots seem as human as possible. Could this be Pope Leon telling Olah his job is a “so-called” skill? So essentially, BS? It at least throws shade on idiocies like Anthropic’s constitution for Claude, which encourages us to be nice to its increasingly-ubiquitous chatbot. When it’s actively destroying the planet? Pull the other one, Chris!
The Pope’s letter follows up with a recurring theme: such systems, that affect billions of people worldwide, cannot be controlled by a select few. “Otherwise, those who control AI will impose their own moral vision, which will become the invisible infrastructure of these systems. A more moral AI is not enough if that morality is determined by a few.”
On power
Speaking of power, this is where Leon leans in to the risks of our AI fates being determined by six or seven billionaires:
- “More power does not necessarily imply something better. In this respect, the words of Romano Guardini remain relevant: “Contemporary man has not been trained to use power well“”
- “In many cases within the digital context, control over platforms, infrastructure, data and computing power does not rest with States, but with major economic and technological actors. These entities effectively set the conditions for access, determine the rules of visibility and shape the very possibilities for participation. When such power is concentrated in the hands of a few, it tends to become opaque and evade public oversight, increasing the risk of distorted forms of development that give rise to new dependencies, exclusions, manipulations and inequalities”
- “Those who control digital platforms and means of communication have a considerable ability to affect the collective imagination and to present a particular vision of reality as desirable. Such power should be constantly guided by the pursuit of truth and respect for human dignity, so that the culture fostered on the internet does not become an instrument of excessive distraction, homogenization or dominance, but rather a setting in which inner freedom and critical thought can mature.”
On laziness
The Pope also picks up one one of the recurring threats of the way technology reduces friction, and makes things almost too easy. I call this the “Wall-E effect,” where we all end up obese and watching Netflix. He puts it a bit better:
“The speed and simplicity with which information, complex analyses, media content and practical assistance can be accessed undoubtedly makes life easier. Yet they can also encourage excessive reliance and the search for ready-made answers, and weaken personal creativity and judgment. The apparent objectivity of the responses and suggestions these systems provide can lead us to overlook the fact that they reflect the cultural assumptions of those who designed and trained them, with all their strengths and limitations. The artificial imitation of positive human communication — words of advice, empathy, friendship and even love — can be engaging and at times genuinely helpful. However, for less discerning users, it can also be misleading, creating the illusion of a relationship with a real personal subject.”
Furthermore, as he adds elsewhere, “We must avoid the misconception of equating this type of “intelligence” with that of human beings. These systems merely imitate certain functions of human intelligence.” And yet many continue to consider AI chatbots as a sort of oracle, cf. one French voter in six who consulted one before voting in recent local elections…
It’s not just laziness, however; it’s also perception. “When people come to believe that nothing is genuinely true and that principles are hollow words, then the fuse in their hearts is lit for new eruptions of intolerance and aggression,” bemoans Leon. Could this be a reference to the increasing waves of anti-AI opposition that now even Trump is worried about?
On the planet
My pet topic is the one Leon says the least about, but the paragraph he does devote to AI’s environmental impacts is, again, spot on:
“The gains in efficiency and the potential to improve certain services are clear, yet rapidly and uncritically adopting them exposes us to a range of risks, including the tendency to overlook the environmental impact. Current AI systems require enormous amounts of energy and water, significantly influencing carbon dioxide emissions, and place heavy demands on natural resources. As their complexity increases, especially in the case of large language models, the need for computing power and storage capacity grows too, which requires an extensive network of machines, cables, data centers and energy-intensive infrastructure. For this reason, it is essential to develop more sustainable technological solutions that reduce environmental impact and help protect our common home.“
Another point that must have made Olah uncomfortable, Anthropic being (with xAI) the least transparent AI maker with regards its impacts on the planet.
On workers
He also nails the social impacts, in a section on AI’s impacts on work, but which also underlines how well he understands the sector’s full value chain of harms:
- “Every seemingly immediate and flawless [AI] response is the result of a long chain of mediation, involving vast networks of natural resources, energy infrastructure and, above all, people. A significant part of the digital economy’s functioning relies on the silent work of millions of people engaged in essential yet largely unseen activities, such as data labeling, model training and content moderation, often involving disturbing material. In many cases, these workers are young people, predominantly women, working under demanding conditions for minimal wages”
- “Added to this invisible labor is the even harsher work of extracting the resources required for the production of the devices and microprocessors on which AI depends. In some regions of the world, children and adolescents work in dangerous conditions, crushing the materials from which rare earth elements are extracted. The bodies of these people are scarred, injured and worn down so that computational flow may continue uninterruptedly”
- “When efficiency becomes the ultimate measure of value, human beings are tempted to see themselves as a project to be optimized rather than as persons called to relationship and communion”
- “Innovation is often pursued solely for reducing costs and increasing profits“
- “It is certainly desirable for technology to relieve humans of arduous, repetitive or dangerous tasks and to provide intelligent support for human activity. Yet, the protection of employment opportunities and the irreplaceable role of the individual must remain the general rule. The pursuit of greater profits cannot justify choices that systematically sacrifice jobs, because the human person is an end, not a means, and the economic order must remain subordinate to human dignity and the common good”
- “The development of parameters and metrics complementary to GDP is crucial for improving the databases used for conducting analyses, political and economic decision-making and establishing regional, national and international priorities.”
On accountability
This next two are exactly what I say in my Responsible AI training course. Maybe I should invite Leon to join?
“For AI to respect human dignity and truly serve the common good, responsibility must be clearly defined at every stage: from those who design and develop these systems to those who use them and rely on them for concrete decisions. In many cases, however, the internal processes leading to a result remain opaque, making it harder to assign responsibility and correct errors. This is where accountability becomes crucial: the possibility of identifying who must “account” for decisions, justify them, monitor them, and, when necessary, challenge them and remedy any harm caused.”
It is not enough to invoke ethics in the abstract; robust legal frameworks, independent oversight, informed users and a political system that does not abdicate its responsibility are required. Otherwise, change will be governed only by technocratic thinking and presented as necessary and inevitable, ultimately imposing rules shaped by those who control data, infrastructure and computing power.
Absolutely bang on. AI makers cannot lift their hands up and say “I have no idea why the model made this decision.” You made it, you’re responsible for it. End of. Whence the need for “robust legal frameworks” and more.
On “disarming AI”
This notion of accountability is at its most crucial in a war context, when it refers to matters of life and death, writes the Pope.
“The Holy See has recently observed that the growing ease with which autonomous weapons systems can be deployed makes war more “feasible” and less subject to human control. This violates the principle that armed force should be used only as a last resort in cases of legitimate self-defense. For this reason, the development and use of AI in warfare must be subject to the most rigorous ethical constraints, to guarantee respect for human dignity and the sanctity of life and to avoid a race to develop such arms.”
“When a decision to strike becomes automated or opaque, the risk of abdicating responsibility increases. For this reason, the chain of responsibility must be identifiable and verifiable; those who design, train, authorize and employ technology must be held accountable for their decisions.”
Apply all of the above to what happened when the US first invaded Iraq, leaning heavily on Palantir’s Maven’s ability to pick 900 targets in 12 hours – with the help of Anthropic’s Claude, whether they claimed to like it or not – a job that would have taken days, or even weeks, without AI. Then watch this dude, the Pentagon’s head of AI, brag about that “killchain completion” feat, with one hand in his pocket. Do you think he’d be held accountable?
Ergo, this is the encyclical’s most direct pop at the Trump administration. Further underlining a need for international responsible AI guidance that is totally lacking today.
But it’s not all about war. Leon’s central notion of “disarming AI” also “means freeing it from the mentality of “armed” competition, which today is not limited simply to the military context, but is also an economic and cognitive phenomenon. This entails a race for ever more powerful algorithms and larger datasets, driven by the desire to secure geopolitical or commercial dominance.“
To disarm means discrediting the assumption that technical power automatically confers the right to govern. To disarm does not mean rejecting technology, but preventing it from dominating humanity. It means freeing technology from monopolistic control and opening it to discussion and debate, therefore making it human-friendly and restoring it to the plurality of human cultures and ways of life.“
Yes, that’s a lotta bolds. But I think this part bets sums up what the Pope is trying to say. AI’s “bigger is better” race is massively dangerous, if anything because of its implication that he (it’s always a he) who has the most powerful AI supercomputer will eventually run the world. Why do you think Elon Musk rushed to complete Colossus in just four months (whence the methane generators, to avoid waiting for electric grid connection)? Maybe to cement his position as the world’s richest and most powerful person?
Not, hopefully, if the “Responsible AI” Pope has anything to do with it… 🤞🏻
To dig deeper into how to make AI more responsible, feel free to join our Responsible AI training course, with GreenIT.fr! More information here ; idem, et prochaines dates – dont le 25 & 26 juin – en français, ici 👋🏻
