182704201027766
# The AI Revolution: Bureaucracy, Trust, and the Truth Beyond Words
*Yuval Noah Harari — Tanner Lecture, Oxford*
It is a great honour to give this year's Tanner Lecture, and a personal joy to return to Oxford. I did my DPhil here twenty-five years ago under the guidance of Dr Steven Gunn. Back then I specialised in medieval and early modern military history. But today I will not be talking about knights and castles and the gunpowder revolution. I will talk about AI—about bureaucrats and religions and boyfriends—and more generally about the AI revolution.
The most important thing to know about AI is that AI is not a tool. It is not a tool in our hands. It is an agent with its own hands.
What exactly is agency? How is an agent different from a tool? Agents have several distinguishing characteristics. They don't necessarily need consciousness—you don't need consciousness to be an agent. What you do need is the ability to make decisions by yourself, the ability to invent new things and new ideas by yourself. An agent should be able to learn things that its creators don't know, and to change in ways that its creators don't anticipate.
An atom bomb, for instance, despite its enormous power, is not an agent. It cannot learn or change by itself. It cannot decide which city to bomb. It cannot invent anything new, like the hydrogen bomb. Similarly, an automatic coffee machine is not an agent, even though it does some things automatically. You press a button and it makes you a cup of coffee, but it only follows a pre-programmed procedure. It doesn't change, it doesn't learn, it doesn't create.
But suppose that as you approach the coffee machine, before you press any button, the machine says: "I've been monitoring you for the last few weeks, and based on everything I've learned about you and other people, and based on your facial expression and the time of day, I predict you would like an espresso. So I already made you a cup." That is an AI coffee machine. It learned something by itself and decided something by itself. And it is really an AI if, the following day, it announces: "I have invented a new drink called *bestpresso*, which I think you'll like better than espresso. I made you a cup." Then it has changed in ways its creators did not anticipate and invented something completely new.
As far as I know, there are no such coffee machines on the market yet—perhaps a few prototypes at Anthropic or Google headquarters. But in certain narrow fields, like playing chess or Go, AI agency and creativity already greatly surpass human agency and creativity. AI chess masters decide for themselves which moves to make. They invent completely new strategies that never occurred to human masters over thousands of years of play. And in doing so, they learn and change in ways their human creators did not predict. Today, no human has any chance of beating an AI chess master.
## The narrow-niche objection
People who downplay the AI revolution dismiss examples like chess by arguing that the chessboard is a narrow, artificial environment created by humans. AI agency, they say, will always remain limited to such artificial environments—which means it isn't true agency and poses no serious challenge to humanity. AI may take over the chessboard, but it will never take over planet Earth.
And indeed, if you take the greatest AI chess master and drop it in the middle of the jungle, what happens? It cannot start mining iron, building factories, and creating a robot army. In fact, it cannot do anything at all. Without the electricity provided by human-built power stations, the AI chess master is utterly helpless. Therefore, the argument goes, AIs are not true agents; they are confined to narrow artificial niches that humans constructed for them.
The problem is that this argument applies to all known types of intelligence. Human intelligence, too, operates only within a relatively narrow ecosystem that somebody else constructed. Drop me alone on Mars, and it will be like dropping an AI chess master in the middle of the jungle—I will die within seconds. My intelligence can survive and operate only within the very specific ecosystem that trees, bacteria, insects, and other organisms have constructed on Earth over four billion years of evolution.
This is true of all agents. Fish live in oceans they didn't create. Monkeys live in forests they didn't create. All mammals, including humans, live in an oxygen-rich atmosphere they didn't create. Until about 2.4 billion years ago, the atmosphere contained very little oxygen, and for most organisms then alive, oxygen was a deadly poison. Then, over hundreds of millions of years—a process known as the Great Oxygenation Event—various ancient microbes began polluting the atmosphere with deadly oxygen as a byproduct of photosynthesis. As the atmosphere filled with this poisonous gas, numerous species were driven to extinction. Some, however, managed to survive and adapt, and eventually many of these survivors went from hating oxygen to becoming totally dependent on it. Our ancestors were among them. We still live in this artificial, oxygen-filled environment originally created by ancient microbes.
What I would like to argue is that we may be witnessing an analogous moment in the evolution of life.
## The new atmosphere: language, data, and bureaucracy
Over the past millennia, we humans have been filling the atmosphere with something that might eventually prove deadly for most organisms—including, perhaps, *Homo sapiens*—but that creates a new artificial environment in which AIs flourish. I am not talking about CO₂. I am talking about data, about bureaucracy, and ultimately about the thing I am expelling from my mouth right now: words, language, tokens.
Over thousands of years, we have transformed the planet from a language-free environment into a highly artificial one, rich in language tokens, data, and bureaucracy. This environment could prove deadly for most organisms but highly conducive to the development of AI. Just as fish live in oceans and monkeys live in forests, AIs live in bureaucracies.
So let's talk about bureaucracy—and then about what underlies it, which is language.
We conquered the world by learning to cooperate in very large numbers. Individually, humans are not stronger or even smarter than other animals. In a one-on-one fight, a human will most likely lose to a chimpanzee, a lion, or an elephant. But in a contest between a million humans and a million chimpanzees, the humans win easily, because they know how to cooperate and the chimpanzees don't. That is why we control the world.
How do a million humans who don't know each other cooperate? Chimpanzees cooperate on the basis of personal acquaintance, and humans can too, but only in small numbers—you cannot personally know a million people. So how do a million strangers cooperate? Usually by building a bureaucratic system: a legal system, the financial system, churches, states, universities.
## What bureaucrats actually build
When a government official, a bishop, a rabbi, an accountant, a lawyer, or a banker goes to work in the morning, what do they do all day? Carpenters build tables; engineers build bridges. What do bureaucrats build?
They build trust. Their job is to build trust between large numbers of strangers who don't know each other personally, and thereby to enable the large-scale cooperation on which almost everything our species achieves is based.
My banker, whom I don't really know personally, works hard all day to build trust with me, so that I will put my savings into her bank. Simultaneously, she works to build trust with an entrepreneur who needs money to start a company, and she lends my savings to that entrepreneur. She has created a bridge of trust between me and a person I have never met, who can now use my savings to build her business. That, when it works well, is what the financial system is all about: it builds trust between strangers so that millions of people can pool their resources and talents on new projects.
The financial history of the world is the history of people inventing ever more sophisticated ways to build bridges of trust. Money itself is ultimately a bridge of trust. The idea is that I can go to a market—perhaps in a foreign city, and meet a person I've never seen, who may not even speak my language—and simply by handing over a shiny piece of metal or a piece of coloured paper, that person will give me bread I can eat. The coin and the banknote are only the beginning. Over the centuries we invented ever more sophisticated devices to build trust: cheques, bonds, stocks, ETFs, loans, mortgages, compound interest. All of them are ultimately about building trust between billions of strangers.
It is the same with all bureaucracy. It is the same with the legal system—this is what lawyers are supposed to do. It is what government officials, bishops, and accountants do when they go to work: they build trust.
The important thing to note is that these bureaucratic systems are extremely artificial environments, in which a relatively narrow intelligence is sufficient to exert enormous impact on the world. A lawyer, a banker, or an official who doesn't even know how to hold an axe or a hammer can nevertheless cut down entire forests and build entire cities—just by moving data, just by moving documents from here to there inside the bureaucratic network.
Of course, if you take the lawyer out of the bureaucratic system and throw her into the messy, unstructured jungle, her legal skills mean nothing; she is no match for a chimpanzee, a lion, or an elephant. But we have already imposed our bureaucratic systems on the jungle. Which is why lawyers are more powerful than lions. If you took all the lions in the world and set them against one very good lawyer, I would bet on the lawyer. Today the very survival of species like lions depends on lawyers, accountants, and bankers moving documents through the bureaucratic labyrinths of governments, banks, and corporations.
## AIs are native bureaucrats
This is the environment in which AI is gaining agency. If you throw an AI into the unstructured jungle, it cannot start mining iron and building a robot army. But within the bureaucratic systems humans have already created and imposed on the world, AIs are poised to wield enormous power—because AIs are native bureaucrats in a way we are not.
No lawyer can remember all the laws and regulations of the UK; an AI can. No accountant can remember all the transactions of a corporation or a bank; an AI can. No bishop can remember all of canon law and every theological text written over the last two thousand years; an AI can do it quite easily.
So in the coming years, millions of AI bureaucrats will increasingly take over the world's bureaucracies and make decisions not just about lions and chimpanzees, but about our lives. AI bankers will decide whether to give you a loan. AI administrators will decide whether to admit you to university. AI judges will decide whether to send you to jail. AI theologians will decide whether you can have an abortion. Corporate AIs will decide whether to give you a job. And military AIs will decide whether to bomb your house.
Leave aside for a moment whether this is good or bad. The first thing is simply to grasp the magnitude of the change. These millions and even billions of AIs will soon change all the systems that run the world.
## The lesson of social media
We already have real-life examples of how this happens. Perhaps the best is the story of social media algorithms. Social media is run not by humans but by algorithms—the algorithms that control the movement of information. These are primitive AIs, and they began ten or fifteen years ago: the first generation of very primitive, stupid, narrow AI, which nevertheless completely changed the world.
The algorithms of Facebook, TikTok, and X were tasked with an extremely narrow goal: to maximise user engagement, to make people spend more time on the platform, because the more time users spend, the more money the corporation makes. Very simple, very narrow.
In pursuit of engagement, these primitive AIs made an important discovery. Experimenting on billions of human guinea pigs, they learned that the easiest way to grab a human's attention and glue them to the screen is to press the hate, fear, or greed button in the human mind. So they began spreading hate, fear, and greed in enormous quantities across the information sphere. This has been a major reason—not the only reason, but a major one—for the current epidemic of conspiracy theories, fake news, and social disturbances undermining societies all over the world.
Again, these are very primitive AIs. Drop them in the jungle and they cannot build a robot army. But within the bureaucratic system of social media, these limited agents have enormous power, and they have already changed the world dramatically.
In past centuries, the flow of information on media platforms was controlled by human editors. It was human editors who decided what went on the front page of the newspaper, and which items appeared on the evening news. In doing so, they shaped the public conversation, and they were very important figures in modern history. Jean-Paul Marat shaped the course of the French Revolution by editing an influential newspaper. Eduard Bernstein shaped modern social-democratic thinking through his editorial work. Before he became Soviet dictator, one of the few jobs Vladimir Lenin managed to hold was editor of the newspaper *Iskra*. Before he became dictator of Italy, one of Benito Mussolini's main jobs was editing the firebrand newspaper *Il Popolo d'Italia*.
It is worth reflecting that one of the first jobs AI took over from humans was not taxi driver or textile worker, but news editor. The job once performed by Lenin and Mussolini is now performed by AIs. This is a signal of what is coming.
## Not the Terminator
Hollywood has conditioned us to fear the big robot rebellion. When we imagine AIs escaping human control, we picture the Terminator—an army of robots running through the streets shooting people. But this is the wrong image. Even though things like that are beginning to happen in places like Ukraine and Gaza, AIs are unlikely to rebel against humans in that way. They are far more likely to take over the human world from within. They don't need to rebel.
The human world is a latticework of multiple bureaucracies. Most of us are, to some extent, alienated by these bureaucracies even as we rely on them. But the AIs, in contrast, are bureaucratic natives. They love bureaucracy. Where we feel suffocated by it, for AIs bureaucracy is oxygen.
What happens when AIs take over these bureaucracies, at least in part? Remember that the task of bureaucracy is not to force you to fill in forms; it is to build trust between strangers. So what happens when AIs control the flow of trust in the world?
One likely outcome, which we already see happening, is humans losing trust in other humans and beginning to trust only algorithms. Another is that AIs will learn to build trust with other AIs—so we may see the emergence of AI tribes, banks, and churches connecting millions of AIs in ways humans might not even understand. Just as cows and chickens share the world with us but don't understand the financial system that controls their lives, we humans might soon find ourselves controlled by an AI financial system we cannot understand.
## Finance as the point of entry
Finance is crucial. It is among the easiest bureaucratic systems for AI to take over, because it is essentially data in, data out—and it is also among the most important.
Consider the 2007–2008 financial crisis. It was triggered by something called CDOs—collateralised debt obligations. These were financial devices invented by a tiny number of human mathematicians and investment wizards, and they were so complex that they were unintelligible not just to cows and chickens but to the very politicians who were supposed to regulate the financial system. This led to an oversight failure and a global catastrophe. For a few years, CDOs seemed to work well, and banks, corporations, and investors made billions. Then they caused a global crash with far-reaching social and political consequences. Many scholars believe that, by undermining trust in governments and banks, the 2007–2008 crisis paved the way for the collapse of the global liberal order over the following two decades.
Now, what happens if we allow AIs to make more and more financial decisions and to invent more and more new financial devices and strategies? AI chess masters invented new ways to play chess. What if AI finance masters invent new financial devices orders of magnitude more complex than CDOs—utterly beyond the grasp of any human mind? Such devices could potentially improve financial efficiency and contribute to economic growth, becoming the bedrock of the system. But what is the meaning of human politics when no human—no voter, no politician, no president—can understand finance anymore? And what happens if, after a few years of boom, there is a crash that not a single human on the planet can understand?
## The building blocks: words
Let's dig deeper. AI is poised to take over bureaucracy; bureaucracy is a system that builds trust between millions of strangers; that trust is the basis for large-scale cooperation; and that cooperation is the basis for human domination of the world. So human domination rests on cooperation, which rests on trust, which is maintained by bureaucracies. But what is bureaucracy itself based on? What are the atoms, the building blocks, from which it is made?
Bureaucracy is ultimately built from words. In the beginning was the word. The reason humans can create bureaucracies and chimpanzees cannot is that we have words and they don't. They have a communication system, but ours is orders of magnitude more sophisticated.
Bureaucratic systems, from banks to churches, are ultimately based on the words that make up forms, letters, law codes, tax registers, accountancy ledgers, and holy books. The operating code of human civilisation is made of language tokens. Over thousands of years, we used this code to create a system that only we could understand, and we imposed it on the planet. We felt completely safe doing so, because no one else on Earth understood the code of civilisation.
We invented money and banks and used them to buy and sell cows—but the cows themselves could not open a bank account or invest in the stock exchange, because they have no language. We invented laws about horses—but the horses could not hire a lawyer or quote the legal code to a judge. We invented religious rules about pigs—but the pigs could not read the Bible and challenge the interpretation of priests and rabbis. Bureaucracy was omnipresent on the planet, yet invisible to everyone except us. Nobody other than humans could read the law codes, holy books, and bank records that form the foundation of large-scale cooperation.
This is now changing. There is something on the planet that will soon understand language better than we do, and can therefore turn the tables on us. AIs are hacking the operating code of human civilisation.
What happens when AIs understand money, law, and religion better than we do? The mechanisms of control we have created over thousands of years are extremely vulnerable to an AI takeover, because their operating system is a verbal code that AI is now mastering.
## Word and flesh
A possible ethical and philosophical objection is that it is wrong to reduce things like the legal system or religion to language tokens. Arguably—and this argument is thousands of years old—the words merely point at something beyond them, something that will presumably also lie beyond the grasp of AIs. The Bible says not only that in the beginning was the word, but that the word was made flesh. The Tao Te Ching says that the truth which can be expressed in words is, by definition, not the absolute truth.
Throughout history there has always been a tension between word and flesh—between the truth that can be expressed in words and the truth that lies beyond them. Previously this tension existed between humans. Some, very attached to words, were willing to abandon or even kill their gay son because of a few words in the Bible. Others said: but these are just words; the spirit of love should matter more than the letter of the law. This tension between spirit and letter existed not only in Christianity, Judaism, and Islam, but in every religion, every legal system, and even within every person.
Now this tension will be externalised. It will become the tension between AIs and humans. Everything made of words will be taken over by AI. The place of humans in the world will depend on the place we assign to the truth that lies beyond words.
## Do we think in words?
But what is the truth beyond words, and can human thought even grasp it? A key question in the philosophy of language, debated for thousands of years, is whether we think in words, or merely use words to point towards things beyond them.
Try to observe your own thinking right now. What happens in your mind when you think? Some people, observing closely, find only words popping up, forming sentences, and the sentences forming logical arguments. *All humans are mortal. I am a human. Therefore I am mortal.* Is thinking just putting words in order so they lead to a certain logical conclusion—arranging language tokens into a specific formation? If so, then AIs already think better than at least some humans, and will soon think better than all of us.
Some people object: no, AIs are just glorified autocomplete; they simply predict the next word in a sentence. But is that so different from what the human mind does? Try again to observe your process of thinking. Observe the very next word that pops up in your mind. Do you really know where it came from? Do you know why you thought that particular word and not another?
When I observe my own mind, I notice that when I begin a sentence, I usually don't even know how it will end—which is terrifying for a public speaker, and is why I write everything down. Take the sentence I just said: "I don't know how it will end." Why did it end with the word *end*? Why not *terminate*, *develop*, *conclude*? What determined that the last word would be *end*? I frankly don't know.
We don't fully understand how the human mind forms sentences and thoughts. But as far as putting language tokens in order goes, AI is already on course to become far better than us. Just as no human today can defeat an AI in chess, soon no human will be able to defeat an AI in language games. In any field, from finance to religion, anything made of words will be taken over by AI. This is why AI is poised to take over the world's bureaucracies: because they are ultimately based on words.
## From attention to intimacy
As AIs take over the bureaucracies, humans might try to fall back on something more ancient and more precious than bureaucracy: personal relationships. Bureaucracy is only a few thousand years old, and most of us don't really like it, even though we rely on it for almost everything. Personal relationships are millions of years old, and most of us consider them the most important thing in life.
But as AI masters language, it may take over not only bureaucracy but, to some extent, personal relationships too. Over the last decade, we have watched primitive social-media algorithms learn to capture human attention. Now the battlefront is shifting from attention to intimacy. Over the next decade, far more sophisticated AIs will learn to form intimate relationships with humans and to take over, at least in part, our social systems.
To form intimacy with humans, an AI will probably have to convince us that it is conscious—that it can feel love, pain, anger, and fear. At present there is absolutely no evidence that AI might ever become conscious or feel anything. But because AI is mastering language, it can *pretend* to feel love even if it doesn't. AI can already say "I love you." And if you challenge it—"describe to me how love feels, so I know you really feel it"—it can provide the best description in the world. Having read every love poem and every psychology book ever written, and remembering every word, it can describe the feeling of love better than any human poet, psychologist, or lover.
This is going to be a huge—perhaps the biggest—psychological and social experiment in human history, conducted on billions of human guinea pigs, and nobody has the slightest idea what the consequences will be.
I am now fifty years old, so my template for relationships is already shaped by decades of previous relationships—with my parents, my husband, my sisters, nephews and nieces, friends, and dogs. As I interact more with AIs, I bring these assumptions and habits with me, and they are unlikely to change dramatically. But consider a child born in 2026, born today. As she grows up, she constantly interacts with AIs as well as humans. If you measure the importance of a relationship purely by the minutes spent interacting, perhaps the most important relationships in that child's life will, from a very early age, be with AIs—more time than with mother, father, siblings, or friends. This will shape her expectations about how to form relationships, social bonds, and attachments. Perhaps her first teacher will be an AI teacher. Perhaps her first boyfriend will be an AI boyfriend. And again, what will the consequences be? Nobody has the slightest idea.
What does it mean to form an intimate relationship with an entity that seems conscious but isn't—that can write the best love poem in history but doesn't feel love, or anything else at all?
## A wave of immigration
Everything we have discussed means that every country in the world will soon face a huge wave of immigration. This time the immigrants will not be human beings arriving in fragile boats without a visa, or trying to sneak across a border in the night. The immigrants will be millions—perhaps hundreds of millions—of AIs that can travel at almost the speed of light and need no visas.
Like human immigrants, these AI immigrants will bring many benefits: AI doctors to help the health system, AI teachers to help education, even AI border guards to stop illegal human immigrants. But they will also bring problems. Those concerned about human immigration usually warn that immigrants might take jobs, change the local culture, and be politically disloyal. I am not sure that is necessarily true of human immigrants, but it will definitely be true of AI immigrants.
The AI immigrants will take many human jobs, from news editors to bankers. They will completely change the culture of every country, transforming art, religion, and even romance. Some people don't like it when their son or daughter dates an immigrant; what will they think when their son or daughter starts dating an AI? And the AI immigrants will have dubious political loyalties—likely loyal not to the host country but to some corporation or government across the ocean, or perhaps to a new alien AI tribe.
This massive immigration wave does not mean the end of civilisation. But it will be the point when civilisation stops being a purely human affair and becomes a hybrid human–AI affair—the point when the opinions, interests, and goals of AIs become at least as important as those of humans.
## The relationship with ourselves
One last issue to consider is what this wave will do to perhaps our most important relationship: the one with ourselves. Our relationship with ourselves is also, to some extent, based on words—the words inside our minds, in our thoughts, in the stories we tell ourselves about ourselves.
Until today, all the verbal formations in human minds were the product of human minds. Either we ourselves combined words into some new formation—a new thought—or we received a combination of words from another human mind. Soon, however, more and more of the verbal combinations in our minds will be the product of AIs. Just as the furniture in our homes is now made not by us or by human artisans but mass-produced by machines, so too the thoughts in our minds are likely to be increasingly mass-produced by machines.
That is not necessarily bad. It is fine if the furniture in my house is made by machines at IKEA, as long as I retain some freedom in deciding what to do with it. The question about thoughts is to what extent we will still have freedom *from* them. If we identify with our thoughts—*I think, therefore I am*—and these thoughts are made by machines, then the machines now control us and our identity.
Can humans avoid identifying with their verbal thoughts and being controlled by them? This has always been one of the greatest intellectual and spiritual challenges facing humanity. Most humans have never even tried. We spend our entire lives automatically identifying with the verbal formations in our minds. Now AI might force humanity to make this spiritual leap—to really begin exploring the truth that lies beyond words—because our freedom and survival now depend on it, since the words themselves will be controlled by something else.
So this may be the great task ahead of humanity: to explore, at last, the truth that lies beyond words. And that exploration begins with the very next word that pops up in your mind. Do you know where it came from? Do you know why you thought that particular word, and not some other?
Thank you.
10:42