A Slight Peek into Determinism

LLMs are probabilistic but look certain. Do we fool ourselves the same way about the real world?

Determinism says that every event, from the motion of the planets to the choices we make, is the inevitable result of prior causes and natural laws. Given the past and the laws, exactly one future is possible.

By that definition, Large Language Models (LLMs) shouldn’t fit the mold. They are probabilistic by design: what they produce is a spread of probabilities over what could come next, and the words we see are draws from that spread. Yet we constantly treat them as predictable machines. Why?

Because they are so good at what they do. Being right almost every time is not the same as being certain, but from the outside the two look alike. Their precision mimics certainty.

Is the physical world any different?#

That raises a deeper question: is the physical world any different? Philosophers throughout history have warned that questioning the fabric of reality can drive a mind mad. Should I dive in anyway? Classical physics says that if we could capture perfectly calibrated inputs and readouts, the entire future would become predictable. Perhaps I really am on the verge of madness.

But is reality truly that rigid? Two things push back. On the standard reading of quantum theory, the laws themselves deal in probabilities at the smallest scales, much like an LLM does. And even where the laws are fixed, chaos makes the future unpredictable in practice: a tiny error in the starting measurement grows into a huge error later, so “perfectly calibrated inputs” would really mean infinite precision.

The mask of predictability#

Here is where the parallel gets interesting. In both cases we watch something behave regularly and conclude that it must be fixed underneath. A flawless LLM answer makes the model feel deterministic. A sun that rises every morning makes the world feel deterministic. But regularity is evidence, not proof: a system can be reliable without being certain, and it can be fixed without being foreseeable.

So I had been blurring two separate questions. One is whether the laws are fixed. The other is whether anyone can foresee the outcome. Chaos gives us a world that may be fixed but is not foreseeable. An LLM gives us a system that is foreseeable most of the time but is not fixed. Either way, the predictability we perceive is a mask: it tells us how things look from where we stand, not what they are underneath.

But what if the determinists are right?#

Suppose they are, and the universe is fixed all the way down. Then in principle, humanity might eventually crack the code. Imagine a master mathematical model designed to simulate reality itself. In that setup, the history of the universe, every fallen leaf, every breath, every historical event, becomes the ultimate context window. The model processes those past inputs to predict the next token, which here is the exact next state of the physical world.

That’s a world model, isn’t it? It seems we have already made some progress in modelling the real world mathematically.

Look closely at the two rows, though, because they expose three gaps in that picture.

First, an LLM does not output one next token. It outputs a spread of probabilities. A model of a fixed universe would output a spread too, with all of the probability sitting on a single state. So the same shape (history in, probabilities out) fits both possible worlds: if the universe is deterministic, the spread collapses to a point, and if it isn’t, the spread is the truth. We may already have built the right kind of machine without knowing which kind of world it is for.

Second, today’s world models are learned approximations, trained on observation rather than derived from the laws. They predict what is plausible next, not what is certain, which brings us straight back to the mask: a very good approximation looks like certainty.

Third, there is a catch even in principle. A model that holds the entire state of the universe has to sit inside that universe, so it would have to contain a model of itself, and of that model, and so on. A fixed universe might still be impossible to predict fully from the inside.

If we could map the entire universe like a giant, cosmic LLM, life would get very interesting, wouldn’t it? It would have to predict even our reaction to its own predictions.