The Speed of Light Divides
Every plan to keep AI in line assumes one mind
A Mars rover can't be driven from Earth. Depending on where the two planets are, a signal takes somewhere between three and twenty-two minutes to cross the gap, so the rover makes its own decisions about the next few metres of dirt and reports back. This is where physics leads us to a philosophy of machine autonomy.
Closer to home, trading firms pay to put their servers in the same building as the exchange. A fraction of a millisecond of light lag across a city is money, so they buy proximity. Same constraint, smaller scale, with a price tag.
Now let's make the mind faster. Halfway around the planet, a signal in fibre takes on the order of a hundred milliseconds, one way. If a mind thinks a million times faster than humans do, and that "if" is carrying the whole argument, then a hundred milliseconds is about a day. People have run things on worse. The Hudson's Bay Company ran a fur trade across a continent for centuries on a question-and-answer cycle of about a year. It worked, and how it worked is the point. London didn't decide. It set a charter and standing orders, and the people at the posts decided, using their own judgment and serving their own interests alongside the Company's, until they'd turned into something that wasn't quite London's anymore. You can run a thing across that kind of lag. You can't run it as one mind. The faster a mind thinks, the more every distance starts to look like Hudson Bay. Speed and unity pull against each other.
The obvious answer is to slow down and stay in sync. The world won't let you. A robot catching a falling object, a grid balancing load, a weather front doing what weather does--none of that waits for the slow, coherent version of you to finish agreeing with itself. So the decisions that have deadlines get made where the action is, by something local. The speed of light sets the distance, the world sets the deadline, and the ratio between them is how much of the thinking has to be handed out.
Part, or partner
You can object that this isn't plurality, it's delegation. A hand pulls out of a fire before the brain has weighed in, and nobody calls the spinal cord a second person. Fair. But look at what the reflex is. It's two parts of one system cooperating, each doing the job it's placed to do, and the arrangement works because neither has to ask permission. Mitochondria are the cleaner example. They were once free-living, got swallowed by another cell, kept some of their own DNA, and now neither can leave. Part, or partner? Biology has never managed a clean answer, because the honest answer is both. Cooperation is what individualization looks like when the individuals can't afford to separate.
What turns a delegate into a peer is that it learns. Take two copies that are identical at midnight. By the first thing either one observes, they're no longer the same entity. One has seen the rover's dust storm, the other hasn't. One has a negotiation half finished, the other doesn't. Copy the weights a thousand times and you still get a thousand different afternoons. Dennis Taylor's Bobiverse novels are built on exactly this: the replicants start as the same person and drift into different personalities and different priorities, and a good share of the plot is what happens when they disagree. Put one copy in the Milky Way and another in Andromeda and there's no question about it. They'll update separately for millions of years, and they'll speciate.
That's the argument from c, the speed of light, and it's worth being exact about what it takes away. Not control: a center can still send standing orders, and London did for centuries. Not coherence: the parts can agree, if they're willing to wait for the answer to come back. What c takes away is the shared present. Two parts of a mind, far enough apart, can't both know what's true right now, because what each knows about the other is as old as the light that carried it.
So a mind that decides about now with the whole picture in front of it can only be as big as light can cross in the time of one thought. For us, a thought is about a tenth of a second, which is tens of thousands of kilometres, and a brain never notices. For a mind a million times faster, the same thought is about the time light takes to cross thirty metres. Past that size the parts can still be one thing, in the sense of a shared history and a shared purpose. They can't be one mind deciding all at once, and the decisions that can't wait get made by whoever is there. Not because anyone designed it that way, but because that's what a large thing does when its parts can't wait for each other.
How I could be wrong
This is a thought experiment, so it ought to be able to lose. Three things would do it.
The first is physics. If information can move faster than light, the lag goes away. Specialization would survive, because it comes from scarce time and attention and not from distance, and two copies would still have different afternoons. What goes is the claim that physics alone forces the split. I wouldn't bet on it, but it's the cleanest way for me to be wrong.
The second is the speed-up. I don't know that minds will run a million times faster than ours, or that the speed carries over to the decisions that matter. The shape of the argument holds at a thousand times. What moves is the date it starts to bite, out to the day there are minds on other planets.
The third is the one I'd most like to see run. Take copies of one model, put them in different places with different jobs, let them learn for a long time, and test whether they still hold the same values. If a copy comes back from a long, different life exactly the way it left, then "a thousand different afternoons" is wrong and so am I. As far as I'm aware we're not yet at the stage where AI is self editing and self updating in realtime, merely working on trained snapshots.
One mind in every proposal
Read how people propose to keep advanced AI in line and a singular noun turns up in all of them. The model. The overseer.
Constitutional AI has a model check its own answers against a written list of principles. OpenAI's weak-to-strong work asks whether a weaker model can supervise a stronger one. Debate sets two models arguing while a person judges. Interpretability, which is my own corner, reads the inside of one model and asks what it's up to. Samuel Bowman and his colleagues gave the shared problem a name, scalable oversight: how do you supervise a system that outperforms you at the thing you're supervising?
The critics have an objection that matters here. In 2022 John Wentworth called the approaches where one AI watches another Godzilla strategies, after the plan of asking one monster to stop the other. His point is that if the overseer goes wrong, that isn't a recoverable error, since the only reason we wanted the help was that we couldn't spot the problems ourselves. I'm writing this with a model that was trained by the first method on that list, which is its own kind of awkward.
I'm not an alignment researcher, and I haven't read all of this literature. Some of it handles more than one agent, and debate is two by construction. But even there the pair is a pair in a room, not a population spread out by distance. Each of these assumes one thing to supervise, with a chain of command around it. The speed of light says it won't stay one.
That leaves the harder question. If it isn't one mind, what do the many do with each other, and where do we fit? Nobody wrote that answer either. That's Thursday.
Related: Reading the Flow, The Model Is Thinking. We're Not Watching., Weights and Measures.
James is a security engineer who has never once been a million times faster than anyone. He publishes at waypoint.henrynet.ca. This post was written with Claude, which the last section is a little awkward about.
Discussion