For J, whose congruency of thought made me question what making sense feels like from the inside. Taking agency as an axiom, I think I am beginning to understand what you truly meant.
What does it mean to make sense?
Start with make. The obvious definition is to bring a new state into existence out of substance. But we rarely create in the strong sense. Most of the time we only rearrange, revealing latent structure already present in nature and society. Making is closer to disclosure than to invention.
Then sense. A sense is the ability to understand, recognize, value, or react to something. More physically, it is the ability to see, hear, smell, taste, and feel. Hidden in those first four verbs (understand, recognize, value, or react) is a quiet con-sensus, that those are deeply personal.
Take understand: under + stand. To stand among the foundations of a thing, to stand in relation to it. German says it more plainly. Verstehen: to stand in relation, and the relation feeds back to where you are standing. So nothing is ever fully novel. To understand is to situate yourself correctly within a structure. You do it either by finding categories that language has already articulated well, or by building structure that language has not yet reached. Understanding arrives the moment a model is recognized or newly built.
Step back, and the pillars of sense — understanding, recognizing, valuing, reacting — are all profoundly personal. Each is the detection of a difference, a pattern in the world, together with a judgment about why that difference is significant to you. Sense is the ability to sense what matters to you in reality.
Chinese says this differently. 感覺. 感 breaks into 咸, an external stimulus, and 心, the heart: something outside enters the heart and provokes a response. Not passive reception, but being affected. 覺 is perception through the self, and it carries 覺醒, awakening: not only seeing but waking toward the world. So 感覺 is roughly the world triggers me, and I am the one who recognizes the world. The Chinese leans harder on the self. The input that counts is subjective, and the model doing the interpreting can never be universal.
The long way around, then: to make sense is to bring into existence a model for perceiving what matters and being enlightened by reality. That model is irreducibly personal. It is yours.
Reality is the source material of the patterns we are trying to detect. It exists whether or not you believe in it, and we reach it only through observation, through sensory input. It is both constraint and substance for the self, and it holds regardless of how anyone interprets it.
To make sense, then, you have to close the gap between self and reality. Intelligence is exactly this: the capacity to build useful models that narrow the gap. And again, nothing in it is truly new. Intelligence matters to a person because it widens the set of models available to them, and therefore the set of futures. That is what it is to gain agency: to be governed a little less by narratives you never chose to belong to.
I have tried to work out why agency matters to a human and keep arriving at the same place. It may simply be an axiom, something this stage of humanity can discover but not derive. I cannot yet see agency at a higher resolution. I am still in the process of recognizing that it matters, and that it is the atomic unit of the human.
This is why I think AGI is bullshit. General intelligence cannot be made; it can only be manipulated. A company can learn to feed you what you already want to hear, but that is driven from the outside, by capital, which has every incentive to discourage your agency. A human can evolve through intelligence, but the model has to be radically personal. Intelligence is done from the bottom up; it is not handed down from the top.
And intelligence on its own is static. Ability is not valuable in itself. A capacity becomes valuable only when a society accumulates trust, over time, that it will hold value. And that trust takes patience: the patience to believe that real intelligence can bridge subjective belief and objective reality. Being intelligent therefore has to issue in action, because action is what carries value, to the self, to others, to society, outward to the world. Money used to be our atomic unit of value. I think that unit is now open to challenge: capital is not fine-grained enough to register what people actually need. Representation has stopped being sufficient.
If intelligence includes sense-making, and sense-making includes value, then intelligence cannot be fully general in the way AGI rhetoric implies. A system may generalize across prediction, but not across meaning. This is where the word “general” starts to break: it can describe the architecture of a world model, but not the ownership of what matters inside that world. Saying general intelligence is bullshit is not the same as saying general world-modeling is. That part more likely functions as a better tool for capturing reality. Observations can be mapped, because biology is science. A machine can learn a general world model the way model-based reinforcement learning learns a predictive simulator of its environment and plans inside it. The world model is not a map; it is exactly that simulator. Its predictive layer touches what will happen, its value layer touches what matters, its action layer holds what I can do. When the three merge, agency emerges.
What such a machine cannot do is own a human’s value function, not unless the human surrenders agency. This is not a hope but is closer to a result. Inverse reinforcement learning, the field that tries to recover what an agent wants from how it behaves, runs into a wall: behavior cannot be uniquely split into a model of how the agent plans and a model of what it values. Armstrong and Mindermann showed why. Even under Occam’s razor, even when you prefer the simplest explanation, the true decomposition stays indistinguishable from the wrong ones. Closing the gap means importing assumptions about what counts as rational, and those were never in the observations. Behavior underdetermines values. You can predict a person perfectly and still not hold their value function.
So a world model is the architecture of sense, and intelligence begins in reality. But what makes sense differs from person to person, because people are self-interested. The part that says why something matters cannot be extracted by watching. It has to be supplied, by the one whose values they are. This is why it is easy to pre-collapse the world model into “AGI,” and wrong to do it: the predictive layer can be shared, even made universal, while the value layer stays irreducibly personal. Predict me all you like. You still cannot have my value function unless I hand it over.
So the danger in the AGI that business is pushing is not that machines might become intelligent. The danger is that intelligence becomes centralized into models that predict humans better than humans can understand themselves. At that point agency is not expanded. It is outsourced.
This is why the present obsession with AGI feels so intellectually thin. Much of it mistakes refinement for understanding: more fine-tuning, more alignment, more behavioral prediction, as if a better imitation of preference could somehow become a value system. But nothing in that process guarantees contact with what people actually need. It only guarantees that the model becomes better at optimizing around the traces people leave behind. Intelligence, treated this way, is collapsed into prediction; value is collapsed into engagement; agency is collapsed into being well-modeled. That is not general intelligence. That is centralized sense-making with a business model.
The real question is not whether machines can become more intelligent. Of course they can. The question is whether their intelligence expands human agency or quietly replaces it. If the predictive layer becomes universal while the value layer is extracted, inferred, and optimized from outside, then AGI is not the birth of a higher mind. It is the industrialization of interpretation. And that is precisely why the word feels false: it names intelligence as if it were general, while hiding the fact that what matters was never general at all.