What actually sits inside an AI model - and why nobody can point a finger at where one piece of knowledge lives.
Picture the biggest mixing desk you can imagine. Not with eight sliders, but with knobs lined up next to each other as far as the eye can see. Each knob is turned to a specific position, a little left or a little right. The position of one knob on its own means nothing. But when you look at all of them together, the sound that comes out is precisely defined. The model works on the same principle - except instead of sound, what comes out of it is a sentence.
These knobs are called weights, or parameters. During training, as the model reads text after text, something turns each knob a tiny bit, until the result gets more accurate. It does this over a huge number of examples, over and over. At the end, the knob positions stop being touched. They're frozen. That frozen table of numbers is exactly the model you use.
Here's the difference from everything else you've touched on a computer. If a program makes a mistake, a programmer opens a specific line and fixes it. With the model there's no such line. The knowledge that Paris is the capital of France doesn't sit on one knob. It's spread across many knobs, which also take part in many other answers. Touch one, and things break that you never wanted to touch at all.
That's why the companies that make these models don't fix mistakes the way you'd fix a typo in a document. They release a new version, trained again, with the knobs set a little differently. Not because they don't care about the specific mistake - there's just no way to reach only that one, without touching everything else.
Here's what's alive
I've handled enough mixing desks to know one thing: nobody remembers what a single knob somewhere in the middle does. You remember the sound. It's the same here, except the scale is such that even the people who built the desk can't take it all in with one look.
So I stopped asking "why did the model say exactly that". I ask something else: does it sound true to me, and does it feel safe to rely on right now. That's the question I can actually answer.