Version 0.2.84 moves inference into a WebGPU browser
WebLLM 0.2.84 loads a supported language model in the browser and executes it through WebGPU. Once model assets are available, an application can generate text without sending prompts to a hosted inference API. The package exposes chat completions, streaming output, JSON constraints, token usage, seeding, and lower-level generation controls. Dedicated workers keep computation away from the interface thread, while a service-worker option can preserve a loaded engine across page visits when the browser allows it.
The checkout was 12.6 MB with 276 files and about 22,865 source lines. This is a substantial runtime package rather than a thin wrapper around a remote endpoint. MLC supplies WebAssembly execution code, model configuration, tokenization, cache handling, worker transport, and an OpenAI-shaped client. That scope is why WebLLM can remove a server from one architecture, and why browser storage, GPU limits, asset hosting, and error recovery become application responsibilities.
OpenAI-shaped calls still have two documented gaps
WebLLM 0.2.84 lets existing frontend code use chat.completions.create, messages, streaming, JSON mode, tools, and usage fields in a familiar shape. Compatibility is narrower than the README's boldest wording. The model field in a completion request is ignored because the engine selects a model during creation or reload. Function calling is labeled work in progress, with preliminary support for tools and tool_choice. Test those behaviors before swapping clients.
Our repository run covered 209 Jest tests, and all 209 passed in 15 seconds. That is solid evidence for the cases the suite contains. It is not a compatibility certificate for every OpenAI client or model. Browser applications also need cancellation, progress display, out-of-memory handling, model-switch behavior, and a fallback when WebGPU initialization fails. A small proof of concept should exercise the exact request fields your product sends instead of trusting a shared endpoint name.
What happened when we ran it
In our sandbox, Node 22 installed 428 npm packages in 23 seconds and consumed 140 MB on disk. The build succeeded in 20 seconds. Jest then reported 209 passed and 0 failed of 209 after 15 seconds. Npm audit found 0 known vulnerabilities across the installed tree. For a TypeScript project with WebAssembly bindings and many browser examples, that is a clean and reasonably quick contributor setup.
The repository also had a tests directory and 5 CI workflow files, although it did not include a Dockerfile. The missing container recipe fits a library meant to execute in browsers. Our test method used a 3-CPU, 8 GB lab and did not download an LLM, initialize WebGPU, or measure generation. Those 58 seconds therefore say nothing about first-load duration, token speed, response quality, or memory use for any model.
Four cache backends trade convenience for browser support
WebLLM documents 4 model caches: the Cache API, IndexedDB, OPFS, and an experimental cross-origin store. The default Cache API is the easiest choice. OPFS calls fail where the browser lacks that interface, while synchronous OPFS access has its own support requirement. Cross-origin caching needs a compatible Chrome extension and cannot delete tensor caches programmatically. A product team should pick one supported path and expose cache size and clearing controls to users.
A successful 20-second package build cannot settle the hardware matrix. Open issue 644 reports that a Firefox Nightly device exposed only half the requested workgroup storage. Issue 836 reports VK_ERROR_DEVICE_LOST during initialization on two Qualcomm Adreno generations. These are user reports, not proof that every Firefox or Adreno device fails. They are enough to require real-device tests and a clear unsupported-browser response instead of an endless model-loading spinner.
Integrity checks help, but model delivery remains your job
Version 0.2.83 added optional integrity data for configuration, tokenizer, WebAssembly, and model files. An application can provide SHA-256, SHA-384, or SHA-512 hashes and choose whether a mismatch throws an error or produces a warning. That is useful when weights come from a CDN or model host. Leaving the integrity field out preserves the previous behavior, so teams that need artifact verification must create and maintain those hashes themselves.
The npm tree used 140 MB after 428 packages, before any model weights entered a browser cache. First use fetches the selected model and its matching WebAssembly library, and the README warns that loading may take a long time without cached files. Custom models add an MLC compilation step. Storage quota, CDN headers, cache invalidation, model licenses, and upgrade sequencing belong in the release checklist even though npm install itself finished in 23 seconds.
A September push matters more than the April release tag
GitHub recorded 18,715 stars, 149 combined issues and pull requests, and a last push on September 2, 2026. The latest tagged release was v0.2.83 from April 24, while the measured commit's package file says v0.2.84. Active code and pull-request work show ongoing maintenance despite the tag gap. Five CI workflows and a 209-test pass give that activity more weight than stars alone.
Open issue 844 reports a shape-cache regression on one integrated AMD GPU in v0.2.83 and v0.2.84, including a device hang after longer prompts. That report is a reason to pin, test, and retain rollback capability. Our clean 15-second test result makes WebLLM easy to recommend for a controlled browser fleet. Public sites should pair it with capability detection and a server option rather than promise local inference to every visitor.