What Zyphra released

Zyphra released ZONOS3-Mini on 9 November 2026. The model turns text into speech and can copy a voice from a short sample without any fine-tuning, which is called zero-shot voice cloning. The weights are on Hugging Face under Apache 2.0, and the inference runtimes sit in a GitHub repository under the MIT licence. Both licences allow commercial use. Few open text-to-speech (TTS) models with cloning come with this much tooling for local use: Zyphra ships a CUDA server, an Apple MLX server, a C++ server built on the ggml library and a version that runs in the browser.

Zyphra claims the following time-to-first-audio figures: under 570 ms in a browser on a Pixel 9 and under 150 ms on an M3 MacBook Air. A third-party review found that Zyphra's launch thread gave slower figures (under 800 ms and under 450 ms). Builders should test on their own target hardware before relying on either set of numbers.

How the model works

ZONOS3-Mini is a sparse mixture of experts (MoE). An MoE model holds many small sub-networks, called experts, and a router sends each token through only a few of them. According to the model card, the model has 800M total parameters, 64 experts and top-2 routing, so about 100M parameters are active per token. The backbone has 11 layers. Nine of those layers use Gated DeltaNet, a linear-attention design that keeps a fixed-size memory in place of a growing key-value cache. The other two layers use global attention. This mix keeps compute low on long passages.

Most of the speed gain comes from a new audio codec, ZAC-1. A codec here is the component that compresses audio into discrete tokens for the language model to predict and then decodes those tokens back into a waveform. ZONOS2 used DAC, which runs at 86 Hz. ZAC-1 produces tokens at 15 Hz across 10 codebooks and decodes to 44.1 kHz audio. The review calculates that this cuts the model's workload from 774 to 150 speech tokens per second. Zyphra also reports about a 10x lower word error rate than DAC when both are trained under similar data constraints. A single generation can run to 10 minutes of audio.

The model offers two ways to condition on a voice. The first places a raw reference clip of up to 60 seconds in context. The second uses a 192-dimension speaker embedding, a compact numeric fingerprint of a voice that can be stored and reused. Zyphra steers delivery (pace, emotion, projection and liveliness) through activation steering, which nudges the model's internal activations, and supports edits at the level of single words. Zyphra trained the model on 112 H100 GPUs over 2.5 days, followed by DPO and GRPO post-training to improve robustness.

Running ZONOS3-Mini locally

The GitHub README lists five ways to run the model. All of the servers expose the same HTTP API, including an OpenAI-compatible /v1/audio/speech endpoint, so an app written against OpenAI's speech API needs only a new base URL.

  • CUDA server: Linux with an NVIDIA GPU and the CUDA 12.8 toolkit. The H100 is the only GPU Zyphra lists as tested. The weights take about 9 GB, and the first start compiles kernels, which takes several minutes.
  • MLX server: Apple Silicon, using the GGUF files in the weights repository.
  • C++/ggml server and CLI: runs on CPU, CUDA, Metal or Vulkan, with no Python needed at runtime.
  • Browser: WebGPU, with a first download of about 920 MB that the browser then caches.
  • Offline Python API: for batch jobs on CUDA.

On NVIDIA servers, the model card shows how to serve the model through SGLang-Omni:

sgl-omni serve --model-path Zyphra/ZONOS3-Mini --port 8000
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input": "Hello from ZONOS3-Mini!"}' --output output.wav

To clone a voice through this server, add a references array that points to a reference clip. To stream raw 44.1 kHz PCM, set "stream": true and "response_format": "pcm", as shown on the model card.

Independent test results are still scarce. The review's own test on an RTX 5080 with the quantised ggml build produced 7.1 seconds of Japanese speech in 0.40 seconds using 2.4 GB of VRAM. The same page quotes a Mac mini M5 Pro user whose memory use peaked at about 16 GB during the first cloning run. Teams that plan to clone voices in batches should budget for that peak and size hardware above the steady-state footprint.

Benchmarks and costs

Zyphra evaluated the model on its own ZTTS2-Eval set, which covers 35 languages in a clean set and 20 in an in-the-wild set. Zyphra's Capability Index ranks ZONOS3-Mini first among open models, ahead of Fish S2 Pro and MOSS TTS v1.5. Zyphra designed and ran that benchmark, and the review found no independent reproduction. On one H100, the model card reports a median time to first audio of 80 ms with 8 concurrent requests and a peak throughput of 457 seconds of audio per second of compute.

The hosted version on Zyphra Inference costs US$10 per million UTF-8 bytes of input. The review lists limits of one concurrent request and 100 generations per day, which rules the hosted tier out for production traffic. On that tier, the practical route to production is self-hosting.

Gaps that affect New Zealand builders

The model card lists 43 supported languages, and te reo Māori isn't among them. Text normalisation, the step that expands numbers and abbreviations into spoken words, covers only nine languages; text in any other language passes through unchanged. The voice design controls offer American, British and Australian accents, and Zyphra's own post describes accent steering as an approximation. Phoneme input would let a developer fix the pronunciation of a word such as Whangārei. The README says phoneme input depends on private phoneme tokens and a private pronunciation lexicon, and the speech-editing LoRA sits in a private repository too. Anyone building for a New Zealand audience should test Māori place names and personal names early, because the open release gives no supported way to correct them.

Consent and privacy need attention as well. The review found no watermarking or abuse controls in Zyphra's documentation. New Zealand's Biometric Processing Privacy Code covers voice patterns and applies to identification, verification and categorisation by a technological system. The code has applied to new processing since 3 November 2025. Plain voice synthesis may fall outside those three categories. A product that compares stored speaker embeddings to confirm who is speaking would be doing verification, so that product's design should be checked against the code.

What we'd build with it

ZONOS3-Mini suits work where audio shouldn't leave the device. Examples include read-aloud features in health or education apps, offline kiosk voices and accessibility tools for low-connectivity sites. The OpenAI-compatible endpoint means a team can swap a paid cloud TTS service for a local server and compare cost and latency using the same client code. A sensible first test is to run the ggml server on the cheapest machine you'd actually deploy on, then measure time to first audio and memory during cloning with your own scripts and place names. Members who build something with the model can share it on our projects page.

Sources

Build with people working on the same things

Join the Auckland Society for AI: NZ$599 a year with a ForgeBox and the members' projects area. Our Auckland meetups are open to everyone.

Become a member