SGLang collected 664 GitHub stars in the daily trending snapshot on September 4, an unusually sharp jump for a repository created in January 2024. Attention is landing on the serving layer that determines throughput and waiting time after a model has been trained. During a week crowded with new model claims, developers are also choosing the software that will run those models on their own accelerators.
The surge follows a release cycle that looks more like infrastructure work than a launch campaign. SGLang v0.5.18, published August 22, folded in 710 pull requests from 212 contributors. It added several language and diffusion models, changed core dependencies, moved compiled-kernel caches, fixed security problems, and documented known failures. Together, those changes explain the timing: SGLang's scope is growing quickly, and its release notes make the cost of operating that scope unusually visible.
The software between weights and an API
Downloading model weights does not produce a useful service by itself. A server has to batch requests, schedule work on accelerators, manage the growing cache of attention data, and return tokens through an application-facing interface. SGLang packages those jobs into a runtime that can expose OpenAI-compatible chat completions and its own generation endpoint. The same server can handle text, vision-language, embedding, reward, and diffusion workloads, according to the project repository.
SGLang began with a narrower systems idea. Its original paper, first submitted in December 2023 and revised in June 2024, described a frontend language paired with a runtime for multi-call model programs. The runtime introduced RadixAttention, which stores reusable key-value cache entries in a radix tree so requests with shared prompt prefixes can avoid repeating some computation. The authors reported throughput as much as 6.4 times higher than the comparison systems across their tested workloads. Those experiments are useful design evidence, though they are not a current ranking across every model and accelerator.
The current project reaches far beyond that first paper. Its repository lists continuous batching, speculative decoding, prefill and decode separation, several forms of parallel execution, structured output, and low-precision formats including FP4 and FP8. It also supports model families from multiple labs. SGLang is competing to become the common runtime beneath many different models, so a platform team can reuse one serving path across releases.
One API is spreading across different processors
Nvidia remains the default route. The installation guide requires Python 3.10 or newer, focuses first on Nvidia systems, and currently ships CUDA 13 by default. Separate paths cover AMD GPUs, Google TPUs, Intel CPUs and GPUs, Ascend NPUs, Apple Metal, Jetson, and DGX Spark. That breadth matters because a shared HTTP interface can preserve application code while the server implementation changes underneath it. The packages, kernels, memory limits, and performance characteristics still differ by platform.
A July collaboration with Google makes the hardware direction concrete. RadixArk and Google said developers can run SGLang on current TPUs through the SGL-JAX backend, with support for language, multimodal, and diffusion families. They plan a PyTorch-native SGL-torchtpu backend later in 2026. The stated goal is day-zero model support on GPU and TPU through the same SGLang APIs. The available JAX backend is a present capability; the PyTorch backend remains a promise until it ships.
A migration plan should separate current support from the promised backend. Our review of SGLang covers the setup reality, including a successful 580-test run of the Rust subproject and the limits of that result. The test did not load model weights, exercise accelerator kernels, or reproduce a distributed cluster. Hardware reach on a support matrix should be treated as a set of deployment paths to test, not proof that one configuration can move unchanged between processors.
Version 0.5.18 shows both pace and friction
The size of the latest stable release is striking. Its 710 merged pull requests brought support for seven newly listed autoregressive or diffusion models and added cookbook recipes for more model families. One startup change overlaps checkpoint loading with CUDA graph capture. On the maintainers' Qwen3-32B and H100 test, the opt-in mode started 8.6 to 11.7 percent faster than a serial prefetch path and 2.38 times faster than the plain default. These are configuration-specific results, but startup time is a real operating concern when replicas scale up or recover.
The same release shows how much work now sits below the API. On AMD hardware, SGLang can load certain NVFP4 checkpoints by converting them to MXFP4 without holding a full-precision copy in memory. Kimi K3 received several MI355X-specific changes. Nvidia paths gained new FlashInfer and attention work, while Intel and Ascend sections carried their own model support and fixes. A single project name now covers substantially different code paths chosen by the model, quantization format, and processor.
Upgrade notes deserve as much attention as the speed numbers. Version 0.5.18 moved the CUDA stack to PyTorch 2.13 and Triton 3.7.1, consolidated several compiled-kernel caches under SGLANG_CACHE_DIR, and removed the torchao integration. The first launch after an upgrade recompiles the moved caches unless operators copy or mount them in the new location. Remote media downloads are now limited to 64 MiB by default, with redirect checks and an optional exact-host allowlist.
The maintainers also listed reverted work and disabled tests. A Kimi K3 kernel fusion and an AMD GLM-5.2 optimization were removed from the release after landing, while some failing NPU tests were disabled pending fixes. SGLang remains on a 0.5 release line, and the notes expose a compatibility matrix that changes every few weeks. Star growth says people want to inspect the project. It does not turn that matrix into a low-maintenance service.
A familiar client can hide an unfamiliar server
The shortest path to a local endpoint is deliberately plain. The official request tutorial launches a model server and sends chat requests to port 30000:
python3 -m sglang.launch_server \
--model-path qwen/qwen2.5-0.5b-instruct \
--host 0.0.0.0
Once the process is ready, existing OpenAI client code can point its base URL at http://127.0.0.1:30000/v1. The server also publishes Swagger, ReDoc, and an OpenAPI document from the running instance. This lowers the amount of application code required for an initial trial. It says little about whether a chosen model fits in memory or meets a production latency target, which are questions for the server configuration and hardware.
Installation reveals more of that work. The SGLang guide says some dependencies publish only prereleases, so older versions of uv can silently install SGLang 0.5.9 unless prereleases are allowed. Its example container receives access to every GPU, 32 GB of shared memory, a Hugging Face cache mount, and host IPC. The guide warns that latest and dev container tags move, and recommends pinning an immutable tag such as v0.5.18 for reproducible deployments.
API compatibility is expanding too. The v0.5.18 notes include support for the Responses API alongside fixes for reasoning streams, tool calls, and strict JSON-schema handling. Teams moving an application from a hosted API should test the specific endpoints and streaming behavior they use. A matching route name establishes only the first layer of compatibility.
Performance claims need their test conditions
Serving benchmarks can change dramatically with prompt length, output length, batch size, latency limits, model architecture, and accelerator generation. The SGLang paper's peak 6.4-times result covered selected language and multimodal tasks against systems available during the research period. It supports the value of shared-prefix caching and structured decoding. It does not establish that every SGLang deployment will beat every current alternative by the same ratio.
A newer hardware result contains an especially useful caveat. SGLang and Nvidia reported as much as 25 times higher DeepSeek R1 performance on a GB300 NVL72 system than on H200 in the InferenceX v2 benchmark. Their report says the H200 comparison point enforced 50 tokens per second per user for a low-latency case. Without that latency constraint, the authors noted that H200 can reach similar throughput. The 25-times figure combines newer hardware with SGLang kernel and serving changes, so it should not be read as a framework-only gain.
The GB300 report's latency caveat illustrates the useful comparison for an engineering team: the exact model, quantization, request mix, acceptable time to first token, and cost of the available machines. SGLang's documented cache and parallelism controls give operators many ways to tune that test. They also create more ways for a borrowed benchmark to miss the behavior of a live workload.
Reading the attention signal
The September 4 snapshot put SGLang at 664 new stars for the day, and the repository now has more than 35,000 stars and about 8,500 forks. That is a strong attention signal for infrastructure with no consumer interface. Stars do not measure requests served, reliability, or the number of paying users. They show that developers are looking past model weights and into scheduler, cache, and kernel code.
The project says it runs on more than 400,000 GPUs and generates trillions of tokens each day. A 2025 PyTorch ecosystem announcement also described production use for Grok and DeepSeek serving. Those adoption claims come from the SGLang team and its partners, not an independent deployment census. The repository's Apache 2.0 license is easier to verify and gives companies broad permission to use, modify, and distribute the code subject to the license terms.
The next evidence to watch is less glamorous than another star count. RadixArk and Google say the PyTorch-native TPU backend will arrive later in 2026, while v0.5.18 still records reverted optimizations, disabled NPU tests, and an active stream of compatibility fixes. Its arrival, the handling of those known issues, and reproducible benchmarks on the next stable tag will show whether SGLang can turn wide hardware support into a consistent operating experience.