File size: 3,789 Bytes
5ceda23 bae29a2 5ceda23 bae29a2 5ceda23 bae29a2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 | ---
title: Minimal Conversation (S2S backend)
emoji: 🎙️
colorFrom: indigo
colorTo: purple
sdk: static
pinned: false
short_description: Voice chat over WebRTC against a HF speech-to-speech
---
# Minimal Conversation App (S2S backend)
Tiny browser app that opens a **WebRTC** voice session with a self-hosted Hugging Face
[speech-to-speech](https://github.com/huggingface/speech-to-speech) backend. No OpenAI key,
no robot, no OAuth, no proxy. The browser talks **directly** to the backend.
## How it works
1. App POSTs `<lb_url>/session` (empty JSON body).
2. The load balancer picks a ready compute (round-robin) and returns:
```json
{
"session_id": "...",
"websocket_url": "wss://<compute>/v1/realtime",
"connect_url": "wss://<compute>/v1/realtime?session_token=<JWT>",
"session_token": "<JWT>",
"pending_timeout_s": 60
}
```
3. App rewrites `connect_url` from `wss://` to `https://` and POSTs an SDP offer
to it (`Content-Type: application/sdp`). The compute, running aiortc, answers
`201 application/sdp` with the SDP answer.
4. WebRTC peer connection opens. Audio streams over the media tracks (the browser
sends Opus 48 kHz, the backend resamples to 16 kHz for STT and back for TTS).
Events flow over a data channel labelled **`oai-events`** using the OpenAI
Realtime JSON protocol (`session.update`, `response.audio_transcript.delta`,
`conversation.item.input_audio_transcription.delta`, ...).
The backend exposes one concurrent session per compute; the LB pins a session to
a specific compute via a signed `session_token`.
## Backend requirement
This app talks to the WebRTC route `POST /v1/realtime` defined in
[`websocket_router.py`](https://github.com/huggingface/speech-to-speech/blob/feat/webrtc-transport/src/speech_to_speech/api/openai_realtime/websocket_router.py).
That route lives on the **`feat/webrtc-transport`** branch. If the compute is
deployed from `main` it will reply 404 on the SDP POST. Verify with:
```bash
LB="https://<your-lb>.endpoints.huggingface.cloud"
curl -X POST "$LB/session" -H "Content-Type: application/json" -d '{}'
# -> { "connect_url": "wss://<compute>/v1/realtime?session_token=..." }
# Rewrite wss -> https, POST a SDP offer:
curl -X POST "https://<compute>/v1/realtime?session_token=..." \
-H "Content-Type: application/sdp" --data-binary @offer.sdp
# expect 201 application/sdp; if 404, the compute is on the wrong branch.
```
## Run locally
This is a pure static app. No build step, no node_modules. Any HTTP server works:
```bash
cd minimal-conversation-app-s2s-backend
python3 -m http.server 5173
# or: npx --yes http-server -p 5173 -c-1
```
Then open http://localhost:5173/, click the orb, allow the mic, talk.
> Browsers require **HTTPS or `localhost`** for `getUserMedia()`. `127.0.0.1` works,
> as does `localhost`. Plain `http://192.168.x.y:5173` does NOT (mic blocked).
## Settings (stored in `localStorage`)
| Key | What |
|-----|------|
| Load balancer URL | Base URL of your S2S deployment. App POSTs `<lb>/session`. |
| Voice | Qwen3-TTS speaker name (Aiden, Ryan, Dylan, Eric, Ono_Anna, Serena, Sohee, Uncle_Fu, Vivian) |
| Instructions | System prompt sent in `session.update` once the data channel opens |
## Files
| File | Role |
|------|------|
| `index.html` | Single page, orb + settings modal |
| `main.js` | State machine, settings, mic capture, output audio piping |
| `s2s-client.js` | WebRTC handshake against the LB + compute |
| `style.css` | Orb animations, layout, dark theme |
## Credits
- Backend: [huggingface/speech-to-speech](https://github.com/huggingface/speech-to-speech) on `feat/webrtc-transport`
- UI forked from `reachy_mini_minimal_conversation` (Pollen Robotics × Hugging Face), stripped of all robot-specific bits
|