· · 3 min read · 623 words
My Life Has No Prices
nathanz.cloud/blog/life-is-just-microeconomics
Contents · 3
I train agents to maximize reward. Why can’t I do it myself?
Here is an idea I can’t shake: my everyday life is a market where almost every price is set to zero. Nobody bills me for an hour of sleep, an evening of coding, or a conversation with a friend. And economics has a clear prediction for what happens when a price is held at zero.
When the price is zero#
A price ceiling is a legal cap on a price, and the textbook result is that the cost doesn’t disappear. It changes currency. The cheap lunch place near me never raises its prices, and the queue is twenty minutes long. People pay the difference in time.
My life works the same way. Since nothing I choose in an evening costs money, everything gets paid for in the other currencies: time, sleep, attention. An evening spent training another variant of my agent is paid for with the sleep I didn’t get. Economists call that the opportunity cost, and it is the only price tag my decisions have.
It also explains why “how much?” matters more than “what?”. My first hour of debugging finds the obvious mistakes. My fifth breaks things that already worked. Diminishing returns look exactly like a plateaued training curve: steep at first, then flat. The right move is to stop when the next hour buys less than an hour of sleep would.
The same hour, a different price#
An hour of my time costs far more the week before a deadline than the week after, and it is the same hour. Nothing about the hour changed. What changed is how much else is competing for it.
Real prices work the same way: a graphics card costs more than it did before the AI boom because demand shifted, while the card stayed the same. A price looks like a fact about the thing. It is mostly a fact about everything competing for it.
I can’t run my own policy#
So the theory fits. Here is the uncomfortable part. Maximizing utility under constraints is exactly what I train reinforcement learning agents to do, and I can’t do it myself. The reasons turn out to be the open problems of my own field.
First, I don’t act on what I know. I know the fifth hour is a bad trade, and I take it anyway. In RL terms, my discount factor is too low: a small reward now outweighs a larger one tomorrow. And even with the right values, finding the best action takes computation I don’t have at 2 a.m. That is bounded rationality, and my agents suffer from it too.
Second, some things lose their value once I measure them. If I start counting an hour with a friend as a unit of utility, it is already worth less. This is Goodhart’s law: when a measure becomes a target, it stops being a good measure. Every RL researcher has met it as reward misspecification. The agent maximizes the reward you wrote down, which is never quite the thing you meant.
Third, knowing the model changes the one inside it. Now that I can name opportunity cost, I hesitate over choices I used to make without thinking. An agent has the same problem: its own behaviour changes the data it learns from, so the target keeps moving.
Maybe that is why economics and RL feel like the same subject to me. Both describe a perfectly rational agent, and the interesting work in both is explaining why nobody is one.
If I ever do train an agent that maximizes its reward perfectly, should I ask it how to spend my evenings? Or would it just tell me to go to sleep?