Four stages share one Realtime-style interface
Speech to Speech breaks a spoken turn into 4 parts: voice activity detection, transcription, language-model generation, and speech synthesis. Each part has several backends, selected with command-line flags. The server then presents the pipeline through WebSocket or WebRTC using the core OpenAI Realtime event shape. That lets a client stay mostly stable while an operator changes a recognizer, language model, voice engine, or hosting location.
The repository at commit 16d7f98 had 243 files, about 65,450 source lines, and occupied 4.1 MB before dependencies. This is a focused Python package rather than a complete call-center product. The README supplies serve, talk, and local commands, a browser demo, and a Docker Compose example. Application state, user permissions, call routing, billing, observability, and public-edge security remain jobs for the system around it.
Three starting configurations demand very different hardware
Version 1.0.0 documents 3 starting points. A fully local Apple Silicon path uses MLX components, a fully local Linux path expects an NVIDIA GPU, and a hybrid path keeps speech local while sending text and conversation history to a hosted language model. The README recommends at least 16 GB of total unified memory for its Mac examples and budgets 24 GB of VRAM for the unquantized NVIDIA configuration.
The hardware figures are planning estimates from the project, not measurements from our 8 GB sandbox. The Mac example lists about 7.5 GB of core model weights. A hosted-model setup still budgets roughly 8 GB for local speech, while the first run must download models. Linux users also need audio libraries, and the default Qwen3-TTS wheel targets CUDA 12.8 with glibc 2.39 unless they select another documented wheel.
What happened when we ran it
Our sandbox installed commit 16d7f98 in 18 seconds, adding 35 packages and using 37 MB on disk. The build succeeded in 8 seconds. Pip-audit reported 0 known vulnerabilities. The checkout ran inside a fresh unprivileged Python 3.12 Debian container with 3 CPUs and 8 GB of RAM, without provider keys, audio hardware, downloaded model weights, or a GPU conversation test.
Pytest exited 1 after 8 seconds. It reported 6 passed, 5 failed, 20 skipped, and 43 collection/setup errors out of 54. All 5 listed failures raised ModuleNotFoundError for speech_to_speech from logging-hygiene tests. The log tail also names errors in Whisper language detection, progressive transcription, and tool-call parser modules. It does not reveal why the installed project was unavailable, so assigning the failure to Python 3.12 or missing extras would be speculation.
Version 1.0.0 implements a subset of Realtime events
The protocol document lists inbound audio-buffer, session, conversation, response, cancellation, and truncation events, plus streamed transcription, audio, tool-call, and completion output. CI tests pinned OpenAI Agents SDK clients over both stock transports. WebRTC carries Opus at 48 kHz and resamples it for the 16 kHz pipeline. These are concrete compatibility targets, and the maintainers explicitly say unlisted events and future SDK behavior are outside that promise.
The server binds to 127.0.0.1 by default. Version 1.0.0 provides no server-side authentication or throttling, according to the README, so 0.0.0.0 belongs behind a gateway that supplies both. Transcript-bearing log records report character counts by default; an explicit flag enables full text for debugging. That is a sensible privacy default, but operators must still decide where audio, transcribed text, model requests, and optional debug logs travel.
A 24 GB local plan does not guarantee smooth speech
A modular pipeline exposes the latency of every chosen stage. The packaged client has a configurable playback buffer, and a microphone-blocking option can reduce speaker feedback. That option prevents interruption while the assistant speaks and is not acoustic echo cancellation. Open issue 557 reports browser audio glitches with one local TTS setup and says a startup buffer fixed that reporter's playback, a useful warning that generated audio and live delivery can fail differently.
Our 35-package sandbox did not download models or measure latency, so it cannot validate the project's low-latency description. Production trials should use the target microphone, speakers, network, language, and concurrency. Voice activity boundaries and cancellation matter as much as raw generation speed. The pipeline pool size must also match available model memory, and first-run caches must be prepared before any offline deployment is disconnected from the network.
September activity includes 103 open issues and pull requests
GitHub recorded 13,141 stars, 103 open issues and pull requests, and a last push on September 6, 2026. Release v1.0.0 appeared the same day after v0.2.12 in August. Recent work covers WebRTC, remote speech services, conversation cancellation, model discovery, and updated Mac setup. Those dates and the busy review queue show active maintenance around a new major interface rather than a settled compatibility surface.
One v1.0.0 issue deserves attention on shared multilingual servers. Issue 555 reports that 4 of 5 speech-recognition handlers retain their last detected language across client sessions, while the default Parakeet handler resets it. A proposed fix was already open when we checked. Issue 519 separately tracks missing reasoning items during Responses API tool continuations. Both are specific integration gaps, and neither should be generalized to configurations that do not use those paths.
The 43 setup errors make a pilot mandatory
Pipecat is the stronger comparison when a team values provider adapters and transport choices. LiveKit Agents fits deployments already built around rooms, telephony, or LiveKit media. Moshi takes a different approach with a direct speech model instead of this 4-stage cascade. Speech to Speech earns its place when local Hugging Face components and a familiar Realtime client contract are the reasons for building rather than buying.
The 18-second install and 8-second build are encouraging, while 43 collection/setup errors stop us from recommending this checkout on documentation alone. Reproduce package imports, choose one supported component per stage, cache the exact weights, and test interruption plus session reuse. If the server leaves loopback, add authentication and throttling before exposing it. Version 1.0.0 supplies a capable engine, but the surrounding production service is still yours to assemble.