At the SGLang summit on Nov. 12, the slide that mattered was the one with eight logos on it. LMSYS, the research group behind the Chatbot Arena model leaderboard, used the event to announce the SGLang Alliance. The founding members are Nvidia, AMD, Intel, Google, Meta, Cloudflare, Modal and RadixArk. The group’s stated job is narrow and consequential: set the terms for how models run on different kinds of chips.
SGLang itself is an open-source inference engine, the software that takes a trained model and serves it to users. It has become popular with engineering teams for two features: caching that lets repeated prompts reuse computation, and structured output that forces a model to return data in a fixed format. Both matter when a company is paying by the token and answering the same questions thousands of times a day.
What the alliance proposes to work on is performance and scheduling across accelerators. In practice that means the engine should run about as well on a competitor’s chip as on the market leader’s, with predictable behavior when requests are batched, queued and preempted. The members have not published a charter, a technical roadmap, a committer model or a first release date.
“Every company in that room has a different reason to be there,” said an executive at one member company. “The one thing they share is that customers are asking what a token costs on our hardware, and the answer depends on software none of us owns alone.”
The timing follows the money. Training large models gets the attention, but inference is where the recurring bill sits, and it has grown heavier as reasoning models spend far more tokens per answer. A serving stack that wastes a third of a GPU’s throughput is now a procurement problem rather than an engineering detail, and buyers have started asking vendors to demonstrate tokens per second per dollar in competitive bake-offs.
For AMD and Intel, a neutral engine lowers the cost of porting. The obstacle to competing with Nvidia has never been only silicon. It is the accumulation of tuned kernels, libraries and debugging tools that developers learn once and do not want to learn again. An engine that treats backends as pluggable modules gives those vendors a place to contribute optimizations without asking customers to rewrite applications.
For Nvidia, the calculation is less obvious. Joining a multi-vendor project means accepting that its own stack will be measured against alternatives in public. Staying out would leave the standard to be written by the companies with the most to gain from weakening its position. Two people involved in the planning said Nvidia’s participation was settled late and came with conditions on how performance claims would be presented.
The company has other distractions. Nvidia recently lost a founding figure, a departure that has prompted questions about how much of its software strategy still rests on a small group of long-tenured engineers. Alliances are cheap to join and expensive to steer, and steering one requires sustained attention that is easier to promise than to staff.
There is precedent for what this could become. PyTorch moved under the Linux Foundation in 2022, with Meta handing over governance of a framework it had built while every large chip vendor participated. The move was widely read as a concession that no single company could own the default tool for machine learning. It also took years before the foundation’s decisions carried weight independent of Meta’s engineering headcount.
Analysts said the comparison is instructive but imperfect. PyTorch had one dominant contributor and an enormous user base. SGLang has neither, and its users are companies with buying power of their own.
“The test is not the announcement,” one analyst said. “It is whether a patch from a non-Nvidia backend gets merged without a fight six months from now.”
That test is already visible in small ways. The engine’s repository lists contributions from engineers at several of the founding companies, and its issue tracker shows the usual arguments about which hardware gets attention first. A standard written by committee tends to reflect the priorities of whoever writes the tests.
The group also inherits an awkward question about benchmarks. If the alliance publishes performance figures for its members’ hardware, it becomes a referee in commercial disputes. If it declines, customers will run their own tests, and the ones with the largest clusters will set the terms. Cloudflare and Modal, both in the founding group, sell inference to other companies and have their own views about what should be measured.
Skeptics note how many multi-vendor alliances have produced a press release and a mailing list. The nearest measure of seriousness is whether non-Nvidia code arrives with maintainers attached and review rights, rather than as patches dropped over the wall for someone else to accept. A person involved in the project said the initial governance questions were still open, including how voting works on technical decisions.
What the alliance has not said is who holds the merge button. That detail, more than any logo on the slide, will decide whether SGLang becomes shared infrastructure or a diplomatic channel where competitors trade notes and ship separately. No date has been set for a first steering meeting, according to the people involved.
