The units of measure that decide everything about a chip
HBM (High Bandwidth Memory) isn't memory like the one in your laptop. It's stacked in layers and glued directly onto the processor through a thin silicon bridge, so data travels millimeters across the board instead of centimeters. On Nvidia's new AI chips the speed reaches 8 terabytes per second. Sounds like a detail for engineers. It isn't. If the memory can't feed the processor data fast enough, it just sits and waits - the expensive silicon idles, and the meter keeps running. That's why the battle isn't just 'who makes the faster chip' anymore. Only a handful of companies make HBM, and the line for their output is sometimes longer than the line for the processors themselves.
FLOPs (floating point operations per second) is the unit that measures the raw computing power of a chip or a whole center. Think of it as horsepower, but for calculating instead of moving. The problem is, the big number lies on its own - it depends on what precision the machine is calculating with. That's why FLOPs never comes alone. It comes with a label: FP32, FP16, FP4 - meaning 'how many digits after the decimal point I keep while calculating.'
Here's the trick that made the whole industry cheaper. Calculating at maximum precision is like measuring with calipers where all you need is a pencil line - accurate, but slow and expensive. bfloat16 and FP4 tell the chip: calculate roughly, precisely enough not to mess up the answer, roughly enough to be fast and save memory. When training a model, precision matters. When answering a question - not as much. So companies train at higher precision and let the finished model answer at lower precision. The result is a visibly cheaper model, with no noticeable difference in the answer.
And here comes the twist few expected two years ago: the bottleneck isn't the chip anymore, it's the power. OpenAI (the company behind ChatGPT) is building, together with Oracle (the software and cloud giant), a project aimed at around 10 gigawatts of power - as much as several nuclear plants combined. Not because they can't buy more chips. But because no amount of power is enough to run them. A permit for a new energy facility takes years. A chip is manufactured in months. That's why the industry no longer brags only about 'how many chips we bought,' but about 'how much power we've secured.'
Here's what matters
When you read a headline 'company X bought a massive batch of chips,' you don't see the other headline behind it - about a substation, a deal with a power utility, a new reactor or solar farm that has to be ready before the chips ever run at full power. The AI industry speaks the language of silicon, but lives by the rules of energy.
I play with small models at home - a card, a fan, a modest power bill. It's funny to compare my room to a gigawatt-scale center, but the difference is only in scale - the nature of the thing is the same. For me and for them, the chip is the expensive toy, and the power is the bill that actually decides how far you get. So when you hear 'new super chip,' ask first how much it eats. The next big news in this industry won't be about a processor. It'll be about a power plant.