WebNN and ONNX Runtime Web: the Accelerated Inference Stack in the Browser

Every time someone talks about running AI in the browser, the conversation soon turns to a very concrete engineering question: how will a model trained in Python, on a server with a GPU, end up inside a Chrome tab and run fast enough to be useful? The answer is not a single product. It's a stack of three pieces that fit together, and understanding how they fit together is what separates a solid architectural decision from an experiment that dies in prototype.

The three pieces are a shape, an engine, and an acceleration layer. The format is ONNX. The engine is ONNX Runtime Web. The acceleration layer is WebNN. Each one solves a different problem, and the fun is in how they come together.

The format: ONNX as common denominator

Models are born in different frameworks. One team trains in PyTorch, another in TensorFlow, another in its own tools. Each framework stores the model in its own way, and this becomes a pain when you want to take the model outside the environment where it was created.

ONNX, which stands for Open Neural Network Exchange, is an open format designed to be that common denominator. You train wherever you want and export to ONNX. The model is now described in a standardized way, independent of the source framework, that any compatible tool can read.

For those who decide architecture, the value of ONNX is decoupling. Your choice of training framework no longer ties your choice of execution environment. Train in one place, run in another, and the format in between ensures that the bridge exists. It's boring infrastructure in the best sense of the word: you don't think about it when it works.

The engine: ONNX Runtime Web runs in the browser

Having the model in ONNX is not enough. Someone needs to load this file, interpret the sequence of operations it describes, and actually do the math. That's the job of an inference runtime.

ONNX Runtime Web is the version of this runtime made to run within the browser. It takes the ONNX model and runs it on the client, using the resources that the web platform offers. Historically, this meant two routes: WebAssembly, which runs on the CPU with decent performance, and WebGL, which leverages the GPU via the browser's graphics API.

These routes work, but they have limits. WebAssembly on the CPU is portable and reliable, but not the fastest way to go for larger models. WebGL uses the GPU, but indirectly, as it was made to render graphics, not to run neural networks. This is where the third piece of the stack comes in, the one that changes the performance ceiling.

Acceleration: WebNN and access to the right hardware

WebNN, or Web Neural Network API, is an API that gives the browser direct access to the device's AI acceleration hardware. Instead of using a graphics API as an intermediary, it talks to what is most suitable on the device: the CPU, the GPU or, when available, the NPU.

The NPU deserves attention. It's the neural processing unit, a block of silicon dedicated to neural network operations that appears frequently in modern cell phones and laptops. It performs inference while consuming less power and more efficiently than a generic CPU or GPU. The problem is that, until WebNN, the browser simply didn't have a way to talk to it. The NPU was there, idle, invisible to the web.

ONNX Runtime Web can use WebNN as one of its execution paths. When this happens, the engine delegates the heavy lifting to the layer that knows how to drive the best hardware available. The ONNX model remains the same. What changes is where and how it runs underneath, with a leap in performance and energy efficiency that the old routes did not achieve. This gain is what makes, in practice, local inference in the browser viable for models that previously only made sense on the server.

The entire stack, from one end to the other

It’s worth putting the pieces together into a single mental flow. A model is trained in any framework, on the server, with the entire training infrastructure. It is then exported to ONNX, gaining a standardized and portable representation. This file is delivered to the browser, where ONNX Runtime Web loads and runs it. And, when the environment allows it, the runtime triggers WebNN so that the accounts run on the CPU, GPU or NPU of the user's device.

The result of this chain is inference that happens entirely on the client. No data needs to travel to a server, which is important for privacy. No request generates cloud computing costs, which matters for the bill at the end of the month. And there's no network latency between the user's action and the model's response, which matters for the experience.

Add this to model quantization, which shrinks ONNX to a reasonable download size, and you have a combination that finally makes in-browser AI defensible in product, not just demo.

The limits you need to respect

Here comes the part that separates enthusiasm from responsibility. WebNN is not yet a mature and stable technology everywhere. At W3C, it is in the Candidate Recommendation stage, that is, it is an advanced specification but not yet finalized as a consolidated standard. This means details can change.

Practical support is uneven. WebNN's GPU and NPU accelerated execution in most browsers is in the preview stage or behind experimental flags. It works in controlled environments, in specific versions, with specific configurations. It's not something you can count on uniformly across your real users' devices and browsers.

The recommendation, therefore, is direct and unromantic: do not place WebNN as a critical production dependency yet. Use for prototypes, proofs of concept, optional features that gracefully degrade when acceleration is not available. Always have a fallback path, typically WebAssembly on the CPU, for when hardware acceleration cannot be triggered. Treating a specification in Candidate Recommendation as if it were stable infrastructure is the kind of bet that ages poorly.

How to think about this decision

For architecture designers, the ONNX stack plus ONNX Runtime Web plus WebNN is a sensible mid-term bet, not a foundation for today. The ONNX format and ONNX Runtime Web are already solid enough for real-world use, including traditional CPU and GPU routes. The WebNN layer is the future of performance, but a future that is still coming.

Honest reading means separating what is already ready from what is still ripe. Adopt ONNX as a format without fear, it gives you portability today. Use ONNX Runtime Web where client-side inference makes sense, relying on WebAssembly as a trusted foundation. And treat WebNN as a progressive optimization: when it's available and stable, it accelerates; when not, your product continues to work. This stance fits well with the maturity expected from web development in 2026, where cutting-edge resources come on top of a base that never depends on them.

If you are building a client-side AI strategy, the prudent move is to build on top of ONNX and ONNX Runtime Web now, with solid fallback, and track the evolution of WebNN to turn on acceleration as it matures. Start with what is stable and leave room for what is coming.