Four Stories, and the One i Was Already Telling

6.1 A Sunday Evening

i want to tell you about something that happened while i was writing the Turn, because it bears on the fear i confessed at the end of the last chapter, and because it is the kind of thing that happens to a person who has spent too long inside their own head.

The last chapter closed on a worry: that i would craft a narrative that merely reflects my inner experience. Here is a partial antidote. It arrived on a Sunday evening, in the form of a video essay by the journalist Max Fisher, about where human intelligence came from. Fisher is not a mathematician and he is not, so far as i know, interested in process calculi. He is a reporter who reads the literature and talks to the people who wrote it. His essay lays out four stories about how a foraging ape in the middle of the food chain became the thing that builds data centres, and he tells each one the same way: there is a pressure from the environment, there is a trigger our ancestors evolved in response to it, and there is a conclusion about what intelligence therefore is. Pressure, trigger, and what it says about us.

i knew the first story cold. i had learned it in school, like everybody else. i had passing familiarity with the data behind the other three, the way one has passing familiarity with a city one has only ever changed planes in. By the end of the evening i had done what one does now: i asked a bot to gather the literature behind all four so i could tell which evidence went with which story. And by the end of the week i had the uncomfortable and then delightful experience of recognising, in work i had never read, by people who had never heard of the rho calculus, the shape of the argument i had been building for three hundred pages.

Let me tell you the four stories. Then let me tell you what i found when i looked back at my own chapters.

6.2 The Four Stories

The first story is the monolith.

You know it from Kubrick, where a black slab from space touches an ape and the ape picks up a bone. Fisher’s point is that the film was only dramatising the mainstream scientific view of the twentieth century: one day, by a random mutation, one of our ancestors was born with a bigger and better brain, and that brain let its owner make tools and talk and hunt in groups, and its descendants became us. Louis Leakey tells a version of it over footage of Olduvai Gorge. Yuval Harari tells a version of it in the opening pages of Sapiens, where he calls it the Tree of Knowledge mutation and dates it to about seventy thousand years ago, the moment our species successfully pushed out of Africa after having failed to do so thirty thousand years earlier. Pressure: a sudden accident in the genes. Trigger: a bigger brain. What intelligence is: innovation, the lone genius making ideas out of nothing, with the cave paintings of Europe as exhibit A.

Fisher thinks this story is, and i am quoting a family-friendly paraphrase, woolly mammoth poop, and he gives two reasons that will sound familiar to you, Dear Reader, if you have read this far. The first is that a big brain is a catastrophic liability for an individual. It makes childbirth dangerous, it makes infants helpless for years, and it eats a quarter of the body’s calories. An ape born with one, alone, on the edge of starvation, is not advantaged. It is doomed. The second is that a big brain is only useful in a community of big brains. Who does the first talking ape talk to? Who does the first planning ape plan with? A mutation that only pays when everybody has it cannot spread by appearing in one individual. So the monolith story has it backwards. Intelligence did not appear and then enable the community. Something about the community appeared, and intelligence was what it cost.

The second story is called “we.”

Its pressure is climate. The forests of the East African Rift dried out into open plains, and the ape that had lived well in dense trees, with fruit to hand and cover from predators, found itself on a savanna where food was scarce, predators could see you coming, and no individual could feed itself alone. The trigger, according to Michael Tomasello, who has spent a career comparing human children with chimpanzees, was a very specific and very small cognitive change. Chimpanzees are clever. They reason, they solve problems, they can be taught signs. But they cannot do one thing that every human infant does at about nine months old: hold a goal jointly with another agent while playing a different role in it, attend to the same thing while seeing it from a different side. Tomasello calls this shared intentionality. It is the difference between two animals who both happen to be chasing the same antelope and two animals who are hunting it together, each of whom knows the other is counting on them. It shows up before language, which means it almost certainly evolved before language. And once a population survives by coordinating, selection favours the best coordinators, generation after generation, until pointing becomes speech and speech becomes everything else. What intelligence is, on this story: thinking on behalf of a “we.” Tomasello says that what humans did was create a new kind of agent, a shared one, the way cells once created a new kind of agent called the organism.

The third story Fisher calls the schemers,

and it is the strangest of the four, and it is the one i now find myself thinking about in the shower. It starts with a puzzle. For most of our history our brains got bigger. Then, about three hundred thousand years ago, they stopped, and shortly afterwards our faces got shorter, our teeth got smaller, our brow ridges disappeared, and our brains actually shrank a little. Palaeontologists recognised that pattern, because it is exactly what happens to wolves when they become dogs, and to boars when they become pigs, and to every other animal humans have domesticated. And nobody knew why domestication produced that package, until a Soviet geneticist named Lyudmila Trut spent forty years in Siberia breeding foxes for one trait only, tameness, and got the whole package for free: floppy ears, white patches, shorter snouts, smaller brains. The mechanism turns out to be a stunting of the cells in the embryo that go on to build the adrenal glands and also, as it happens, the cartilage of the ears and the pigment of the fur. Select against aggression and you get a cuter animal as a side effect.

So who domesticated us? The anthropologist Richard Wrangham’s answer is: we did. Every other primate has alpha males, who get that way by being the biggest and most violent and who pass on their genes accordingly. Hunter-gatherer societies, surveyed across several continents, have none. They are run by rough consensus, and when an individual starts behaving like an alpha, the rest of the band conspires and kills him. Wrangham calls it the execution hypothesis; the philosopher Ernest Gellner had a better name for the resulting social order, the tyranny of cousins. The point is that once our ancestors could coordinate well enough to gang up on the strong, the whole direction of selection reversed. Reactive aggression became fatal. What became valuable instead was the ability to read a room, to build an alliance, to see a plot coming and to be on the right side of it. Pressure: the threat we pose to one another. Trigger: selection against the hothead and for the politician. What intelligence is: social, Machiavellian, the brainpower to scheme better than your neighbour, with language as the rocket fuel. Fisher likes this story because it explains why selection for brains kept going long after fire and hunting were solved problems, and because it explains why we are the way we are: tribal, conformist, credulous about conspiracies. Those are not bugs in a truth-seeking machine. They are features of a coalition-navigating one.

The fourth story he calls the hivemind,

and it begins by breaking the timeline. In Morocco, at a site called Jebel Irhoud, there are anatomically modern human remains three hundred thousand years old, twice as old as the textbooks said. Which means that for a quarter of a million years creatures biologically identical to us did almost nothing we would recognise as intelligent. The archaeological record flickers: complex tools appear in one place, vanish for tens of thousands of years, reappear somewhere else. We were switching between intelligent and not-intelligent for most of our existence. The anthropologist Joseph Henrich’s explanation is that what evolved in us was not intelligence but a way of learning from each other that is faithful enough to accumulate. You watch somebody detoxify a cassava root by a tedious multi-day process nobody can explain, and you copy it exactly, and your children copy you, and nobody in the chain knows what cyanide is. Culture, on this account, is a second inheritance system, running alongside the genes and much faster. The pressure that made it pay was a climate that had started to swing wildly, too fast for the genes to track and slow enough that your parents’ knowledge was still worth having. And the bottleneck was density. For almost all of our history there were very few of us, in small bands, far apart. An invention made in one valley died with the band that made it. Only when populations got large and connected enough for ideas to ping-pong between groups did the accumulation take off, and that happened late, after the expansion out of Africa into the crowded river valleys of Eurasia. Henrich calls the result the collective brain. What intelligence is, on this story: a property of the network, not the node. The reason Eurasia invented the wheel and the Americas mostly did not is not that anybody was smarter; it is that Eurasia is bigger and lies east to west.

6.3 What i Found When i Looked Back

Here is the part that unsettled me.

Go back to the section on compositionality (Section 2.1), written long before that Sunday evening. i said that humanity’s superpower is coordination, and i used the image of a single human being no match for a whale while a boat of fisherfolk working together can dispatch one. That is story two, told by somebody who did not know he was telling it. Two paragraphs later i said that humans coordinate brilliantly up to the size of a tribe and badly beyond it, that the super-humans we assemble do not easily assemble into super-super-humans. That is story four’s bottleneck, with the Clovis-first clique thrown in as an example of tribal fault lines in the very discipline that studies this.

Go to the section on agency (Section 2.2). i asked you to picture the wetware of a human as a host for many software agents, not aligned, forming coalitions, racing for resources, with “Alice” as the post-facto narrative of which coalition won. That is story three turned inside out: the tyranny of cousins, running inside one skull. And then in the section on bootstrapping (Section 2.3), i described the exponential explosion that comes from Alice modelling Bob modelling his mother, and the aggressive compression that keeps it affordable. That is the social brain, the Machiavellian one, with its price tag showing.

Go to the section on the shape of mind (Section 4.1), and the argument i had with Ben Goertzel about whether intelligence scales without limit. My answer was that it does not, and the reason was message corruption: a population can only agree on things up to the size at which messages still arrive intact, so large populations cluster into agents-of-agents, and the clusters cluster. Henrich’s density bottleneck is the same argument run forward instead of backward. He says accumulation needs a population dense enough to keep ideas in circulation; i said agreement needs a population small enough to keep messages coherent. Those are two ends of one rope.

i did not get any of this from the anthropologists, and they did not get it from me. We arrived at it from different directions, they from fossils and fox farms and fieldwork, i from asking what a computation can afford to know about the other computations it shares a world with. i do not want to overstate what that proves. Convergence is not confirmation. But it is the thing i was afraid i would not have: an outside.

6.4 The Monolith Was in the Maths Too

Now the confession proper, since this is the Pledge and that is what it is for.

When i said i knew the first story cold, i meant more than that i had learned it in school. i meant that it was the story in my head while i built the formal framework, and that it left a fingerprint there. The monolith story says that a correspondence between a mind and a world can appear in one agent, all at once, at no cost: the mutation happens and the ape can suddenly think about things. Fisher’s objection is that nothing appears in one individual at no cost. Everything has to be paid for, and everything that pays has to be paid for by a community.

There is exactly one place in the mathematics of the Turn where i assumed something appears at no cost. It is a small proposition about ontological isolation, and its proof is four lines long and correct, and it rests on an encoding that i wrote down with a pair of angle brackets and did not construct. The whole outside world enters through those brackets. i did not notice this for a long time, and when i did notice it the Prestige was the result: the entire third act of this book is the reveal of that sleight of hand. i will show you exactly where it is when we get there. For now i only want to say that the move i made in the mathematics and the move the schoolbook story makes in the biology are the same move, and i made it for the same reason the schoolbook made it, which is that it is the first story one learns and it is very hard to see past the first story one learns.

6.5 Byproduct

There is one more thing in Fisher’s essay that i want to hand to you now, because by the time you reach the place in the Turn where it matters you may have forgotten that anyone outside this book ever said it.

Stories two, three, and four share a premise that story one does not. The premise is that intelligence was never what was being selected for. A recent modelling study, by Mauricio Gonz\’alez-Forero and Aida G\’omez-Robles, fit the evolutionary and developmental trajectories of brain and body size across seven hominin species and found that the fit requires no direct selection on brain size or on cognitive ability at all. Brain size comes out as a byproduct of selection on something else; in their model, on reproductive traits, and they use the old word for it: a spandrel. Fisher takes that as the hinge of the whole essay. We did not evolve to be smart. We evolved to do some other thing, and being smart was what that other thing cost, or what it happened to buy.

i cannot tell you how much i wish i had had that sentence when i started this book. The single most important move in this book, the one i will come back to again and again until you are sick of it, is that a mind is not a thing that is selected for, measured, or possessed. It is what a budget looks like from the outside. The learner i am going to build in the Turn does not try to be intelligent. It tries to stay alive, which means it tries to keep a ledger from running out, and intelligence is the name we give, after the fact, to whatever the ledger had to be able to do. Every limit on what such a learner can know will turn out to be a price rather than a prohibition. That is my version of the spandrel. The anthropologists got there by measuring skulls. If they are wrong about it, i am probably wrong about it too, and i would rather tell you that now than have you discover it on page six hundred.

6.6 Which Story i Want to Be True, and Why That Is a Bad Reason

Fisher ends by admitting a preference. He likes the schemers. So do i, and for the same disreputable reason: it flatters my sense that people are not primarily truth-seekers, and it explains a great deal of what i see on the news. He also admits, to his credit, that the scientists he cites would say the three surviving stories cross and overlap and are not really rivals at all.

The method of this book says something stronger than that, and i will only gesture at it here because saying it properly is a job for the Prestige. The three stories are not three candidate explanations of one event. They are three pressures on one ledger. The “we” story is about what it costs to act when no action pays unless somebody else acts too. The schemers story is about what it costs to live among others who can, together, cut you off. The hivemind story is about what it costs to keep an idea alive long enough to improve it, and how many of you it takes. Those are three different line items, and a learner under all three of them is a different object from a learner under any one. The question of which story is true is, i think, the wrong question, in the same way that asking whether a company went bankrupt because of payroll or because of rent is the wrong question. What i want to know is what the books looked like.

That is what the formal machinery is for, and it is why you should keep these four stories somewhere handy as you read the Turn. When you meet the mortal scientist and its token budget, think of the ape on the savanna who cannot eat alone. When you meet populations of learners composed by sharing names, think of the cassava. When you meet the parts about who may spend and who may be cut off, think of the cousins. And when you meet the proposition with the angle brackets, think of the monolith, and forgive me. i was raised on it.

Fisher closes his essay by asking, for each story, what it would mean for the machines we are now building. i will close this chapter by declining to answer that yet, except to say that it is the right question, and that the whole of the Prestige is an attempt to earn an answer to it rather than to guess one. More on this later.