The new generation of chips: GPU, NPU, ASIC and RISC-V for those who decide

For a decade, hardware was something most technology leaders could mentally outsource. You chose a cloud, ordered more machines, paid the bill. The physical layer was abstraction, and abstraction is exactly what a manager wants: not to think about the problem.

That break is over. The cost of running artificial intelligence transformed the type of chip into a business decision, with a direct impact on margin, latency and supplier dependence. The conversation about GPU, NPU, ASIC and RISC-V has left engineering and reached the product committee, even though it still speaks a language that few on the committee understand.

This article translates that language. Not so you can design silicon, but so you can recognize when a hardware choice is actually setting the ceiling for a strategy.

Why hardware has returned to center

The short explanation: software has become hungry in a way that general-purpose processors don't keep up well with.

AI models do a type of repetitive and massive calculation, matrix multiplication, in a volume that does not fit comfortably on a common CPU. When the workload changes in nature, the hardware that was generic becomes a bottleneck. And hardware bottlenecks can't be solved with smarter code, they can be solved with a different chip.

Add to that the economy. Training and serving models consumes energy on a scale that appears on the invoice and sustainability balance sheet. Every watt saved per query becomes margin when you multiply it by millions of requests. Hardware is no longer an invisible fixed cost and has become a variable cost lever, and everything that affects variable costs becomes the agenda of those who decide.

The practical consequence: companies that treat chips as an interchangeable commodity are paying more or delivering more slowly than competitors who chose consciously.

GPU: the flexible workhorse

The GPU (graphics processing unit) was born for gaming and rendering, and by happy accident it solved the AI problem before AI became fashionable. It does a lot of math in parallel, which is exactly what training a neural network requires.

The virtue of the GPU is flexibility. It runs almost any AI payload, from training to inference, from computer vision to language models. If you don't yet know what your dominant load will be, the GPU is the safe bet because it makes few mistakes in any scenario.

The cost of this flexibility appears in two places: price and energy. High-end GPU is expensive, hard-fought and consumes a lot. You pay for generic capacity even when your load is specific and predictable. For the leader, the mental rule is simple: GPU is what you use when you are still learning what you need, or when the variety of loads justifies not specializing.

NPU: the accelerator that lives on the device

The NPU (neural processing unit) is a chip designed for a narrow task: running AI inference on low power. It doesn't train giant models, it runs ready-made models efficiently, and that's why it's appearing inside notebooks and cell phones.

The change that NPU enables is local inference. Instead of sending all user data to a server, processing and returning it, part of the work happens on the device itself. This changes three things that a manager understands without needing a diagram: latency (instant response), privacy (data doesn't travel) and cost (you don't pay the cloud for that query).

The NPU is the engine behind the wave of AI PCs and apps that describe themselves as NPU-first, designed from the beginning to run on the device. For product, it opens up a category of functionality that was previously too expensive or too slow to exist. It's worth delving deeper into local AI on device, because the decision of where inference happens is strategic before being technical.

ASIC: specialize to gain cost and energy

The ASIC (application specific integrated circuit) is the opposite of the GPU in philosophy. Instead of being used for everything, it is designed to do one thing, and do that thing with efficiency that no generic chip can achieve.

The exchange is direct. You give up flexibility and gain much lower cost per operation and power consumption for the specific load that the chip serves. For inference at scale, where you run the same type of model billions of times, this specialization becomes an economic advantage that accumulates.

The price of entry is high. Designing an ASIC is expensive and time-consuming, and the chip is born stuck: if its load changes, the silicon won't keep up. That's why ASIC only makes sense when the volume is large and the load is stable. It is the decision of those who already know exactly what will be shot, and on what scale.

RISC-V: open architecture coming from the edge

RISC-V is not a chip, it is a specification of how the chip talks to the software, an ISA (instruction set architecture). The difference that matters: it is open. Anyone can use, modify and manufacture it without paying a license to an owner.

This seems like a technicality and is, in fact, a question of power. Closed architectures concentrate dependence on a handful of suppliers who charge royalties and dictate the roadmap. An open ISA redistributes this control: it reduces license costs, allows deep customization and gives sovereignty to those who don't want to be held hostage by a chain they don't control.

RISC-V advances first at the edge and in IoT, where chips are simpler and cost sensitivity is high. There, maturity is enough. In heavy and general-purpose loads, the ecosystem is still being formed. Tools, support, software, and ecosystem maturity cannot be bought willingly, they are built over time.

How the leader reads this board

The temptation is to root for a winner. The real board is one of coexistence, and the responsibility of whoever decides is to match the load to the right chip.

GPU for what is still uncertain or varied. NPU for inference on the device, close to the user. ASIC for inference at scale with stable load. RISC-V for the edge and to reduce dependency where maturity already allows. Custom silicone, when you operate on a scale that justifies designing your own, a subject that deserves a separate chapter.

The right question is not which chip is better. It's what your dominant load is, how stable it is, and how much latency, power, and vendor lock-in weigh on your business model. The answer to this chooses silicon, not the other way around. And the gain in energy efficiency in IA usually comes more from the choice of hardware than from any subsequent code optimization.

If you lead technology and still treat chips as an infrastructure line, it's worth taking this decision up a notch. Hardware has become strategy again, and a strategy delegated by inertia is a lost strategy.