Nvidia Opens the Speech Stack

An hour-long recording, up to eight voices, 40 languages, and a download link that anyone can use. Nvidia published Nemotron 3.5 Transcribe on Wednesday with its weights free on Hugging Face and ModelScope, alongside a speech dataset the company says runs to 463,000 hours across 48 languages.

The release puts a model that Nvidia compares favorably with commercial transcription services into the hands of any company willing to run it. It also draws a line the company is not crossing: the model works on recorded audio, not on live calls.

Nemotron 3.5 Transcribe handles two jobs in one set of weights. It turns speech into text in 40 languages and translates 31 languages to and from English. It can separate as many as eight speakers in a single file and process recordings up to an hour long, a length that covers most meetings, hearings and clinical visits.

The architecture explains the price of running it. The model holds 30 billion parameters in total but activates about 4 billion per token, pairing a speech encoder with a mixture-of-experts language model that mixes Mamba state-space layers with conventional transformer attention. That design keeps memory and compute requirements well below what a dense model of the same size would demand.

Nvidia’s benchmark claims are specific. On AA-AgentTalk, a held-out set of voice-agent speech that makes up half of the AA-WER index, the model scored 1.8% word error rate against 2.3% for the next-best open-weight model. On David AI’s combined transcription and speaker-attribution leaderboard it ranked first among open-weight systems and second overall, behind a proprietary entry.

Speaker attribution, the part that decides who said what, is where the company is making its largest claim. Nemotron 3.5 Transcribe posted a diarization error rate of 18.26% across 21 languages, narrowly ahead of Nvidia’s own Nemotron 3 Diarization model at 18.38%. Independent evaluations from Ocular AI and the Josh Talks Voice of India leaderboard place it at or near the top of their open-model tables.

The dataset may matter more than the model. Granary v2 carries 463,000 hours of speech in 48 languages, and Nvidia is publishing the pipeline it used to clean and label that audio along with fine-tuning recipes built on its NeMo Speech toolkit. A company with its own archive of calls can adapt the model to its accent, vocabulary and microphone conditions without sending audio to a vendor.

That is the commercial argument for open weights in audio specifically. Recorded speech from customer-service lines, hospital rooms and legal proceedings is among the most sensitive data a company holds, and several jurisdictions restrict where it can be processed. A model that runs inside a company’s own racks removes the question of who holds the recording.

Cost is the second argument. Cloud transcription is billed by the minute, and the meter runs whether the audio is a board meeting or a hold message. Nvidia’s pitch is that a fixed hardware investment replaces a variable line item, which is the same case it has made for open language models since the Nemotron line began.

The limit is real. Batch transcription suits archives, compliance reviews, subtitling and post-meeting summaries. It does not suit live captioning, voice agents that need a reply in under a second, or call-center software that transcribes as the customer speaks. Nvidia sells a separate streaming model for those jobs, a Nemotron speech model that the company says averages 8.53% word error rate at 80 milliseconds of latency. The open release covers the offline half of the stack, not the whole of it.

Rivals have built businesses on the live half, charging per minute for real-time transcription with speaker labels and low latency. Nvidia’s release does not take that revenue directly. It does set a floor under the price of the recorded half, which is where a large share of transcription volume sits.

For Nvidia, the strategic logic runs through hardware. Every company that downloads the weights needs GPUs to serve them, whether in a data center, on a workstation card or on a compact desktop machine the company markets for local AI work. Open models expand the number of places where inference happens, and Nvidia sells into all of them.

Partners have started building on the release. Agora, which supplies real-time voice and video infrastructure to app developers, used word-level timestamps to align a recorded class with its replay so students can see what was said and when. Inflection AI applied the model to multi-speaker conversations, and Simplismart and VoiceArena have published evaluations of their own.

The unresolved questions are the ordinary ones for open weights. Nvidia has not said how long it will maintain the checkpoint, and a model that ships without a support contract shifts integration work onto the customer. Performance in low-resource languages varies more than aggregate benchmarks suggest, and the licensing terms determine whether a company can ship the output inside its own product.

What Nvidia has done is make the entry price for multilingual transcription close to zero and move the argument to where the audio lives. Whether enterprises trust an open checkpoint with their most sensitive recordings, or keep paying a vendor to hold them, is now a procurement decision rather than a technical one.

Related Posts

  • October 29, 2026
  • 14 views
Europe’s Gallium Nitride Bet in Villach

The European Commission’s decision comes down to one number: 319 million euros. What that money buys is a wafer 100 millimeters wider, and a factory in southern Austria that Brussels…

  • October 29, 2026
  • 11 views
IBM’s Quantum Claims and the Problem of Proof

For most of the past decade, a claim of quantum advantage came with an escape hatch. The machine had solved something a classical computer could not, and the only way…