Hugging Face and Cerebras just shipped a real-time speech-to-speech demo that puts Google DeepMind's Gemma 4 31B at the center of a fully open voice AI stack. The latency is good enough that more than 10,000 Reachy Mini robots are already running this pipeline in production.
This isn't a research prototype. It's a modular, cascaded architecture where every component—speech recognition, language model inference, and text-to-speech—can be inspected, swapped, or fine-tuned. And it's fast enough to make voice interaction feel responsive instead of stilted.
The stack: open models, end to end
The pipeline runs like this:
- Speech input → Nvidia's
Parakeetfor automatic speech recognition - Text reasoning →
Gemma 4 VLMinference on Cerebras hardware - Text-to-speech → Alibaba's
Qwen3TTS - Spoken response delivered back to the user
Every layer is open-source. You can fork the architecture, replace Parakeet with Whisper, swap Gemma 4 for Llama, or plug in a different TTS engine. The modularity is the point—developers building assistants, robots, or embedded devices can adapt the stack without waiting for API access or negotiating enterprise contracts.
That openness matters more than it used to. Voice AI is moving out of controlled demos and into real-world products where latency, cost, and control over the inference stack determine whether the experience ships or dies in beta.
Why Cerebras: it's the P95, not the median
The collaboration with Cerebras isn't just about raw speed—it's about predictable speed. The blog post flags a problem that anyone shipping voice AI has hit: your median latency might be acceptable, but the P95 (the slowest 5% of requests) can still introduce multi-second pauses that make conversations feel broken.
Those tail delays get worse when you add tool calls, multimodal reasoning, or multi-turn interactions. A single slow inference in a chain collapses the user experience.
Cerebras solves the bottleneck at the language model layer. By making Gemma 4 inference "dramatically faster and more stable," the rest of the Hugging Face pipeline can keep up. The stability at the long tail is what makes this feel different from systems that benchmark well on average but frustrate users in practice.
This is the kind of nuance that matters when you're putting AI into robots or voice assistants that need to respond in real time, not "real time if the scheduler is feeling generous."
10,000 robots in the wild
The same speech-to-speech pipeline already powers Reachy Mini robots, with more than 10,000 units deployed. For embodied AI, responsiveness isn't cosmetic—it's what makes the interaction feel alive instead of like talking to a sluggish IVR menu.
The blog post is explicit about the motivation: "The motivation to use Cerebras is therefore not simply cost reduction. It is low latency, predictable performance, and the ability to create real-time experiences that feel natural at scale."
That framing is refreshing. Too much of the conversation around inference optimization focuses on cost-per-token or throughput benchmarks. For voice AI, the user experience depends on whether the system can respond before the conversational turn feels abandoned. Cost matters, but latency predictability is the unlock.
The Reachy Mini deployment is proof that this isn't vaporware. Ten thousand robots running the same pipeline in production means the stack has survived contact with real users, real network conditions, and real edge cases.
What makes this architecture work
The cascaded design is worth unpacking. Instead of trying to build an end-to-end model that does everything (speech in, speech out), the Hugging Face approach keeps each stage modular:
- ASR decouples acoustic modeling from reasoning.
Parakeetturns audio into text, cleanly separated from the LLM's job. - LLM inference runs on specialized hardware. Cerebras handles
Gemma 4at speeds that wouldn't be possible on general-purpose GPU clusters. - TTS is pluggable.
Qwen3TTSgenerates the spoken response, but you could swap in a different voice model without retraining the reasoning layer.
This modularity is the opposite of the "one model to rule them all" approach. It's an admission that different parts of the stack have different bottlenecks, and you can optimize each independently.
It also means you can upgrade components as better models ship. When a faster ASR model drops, you plug it in. When a more expressive TTS engine appears, you swap it. The reasoning layer stays stable.
The trade-offs nobody mentions
Cascaded architectures have downsides. Every stage adds latency. Every handoff introduces a potential failure mode. End-to-end models, in theory, could optimize the full pipeline jointly and maybe squeeze out a few more milliseconds or preserve acoustic features that get lost in text quantization.
But in practice, end-to-end speech-to-speech models are harder to debug, harder to fine-tune for domain-specific behavior, and harder to deploy when you need to run inference on heterogeneous hardware. The modularity tax is real, but it buys you flexibility and observability.
The other trade-off is that this stack requires coordination across multiple model providers and inference platforms. You're depending on Nvidia for ASR, Cerebras for LLM inference, and Alibaba for TTS. If any layer regresses or changes API contracts, you're debugging across organizational boundaries.
That said, the open-source nature of every component mitigates some of that risk. You can fork, self-host, or swap out layers if a dependency becomes a bottleneck.
What this means for real-time AI
The Hugging Face–Cerebras demo is a signal about where voice AI is headed: open models, specialized inference hardware, and modular architectures that let developers compose the stack instead of waiting for monolithic platforms.
The P95 latency point is especially important. As voice AI moves into assistants, robots, customer service agents, and accessibility tools, the reliability of low latency becomes more important than the headline number. A system that's fast 95% of the time but occasionally pauses for 3 seconds is worse than a system that's consistently 800ms.
Cerebras is betting that predictable, sub-second inference at scale is the unlock. Hugging Face is betting that open, composable pipelines will beat closed platforms. The Reachy Mini deployment suggests both bets are paying off.
Try it yourself
The demo is live on Hugging Face Spaces, and the code is open in the huggingface/speech-to-speech repository. If you're building voice AI, it's worth poking around—not just to see the latency, but to understand how the modularity lets you adapt the stack for your use case.
The future of conversational AI won't be a single model or a single vendor. It'll be developers composing the best components into systems that feel natural. This collaboration is what that looks like in practice.