README.md

metadata

title: Minimal Conversation (S2S backend)
emoji: 🎙️
colorFrom: indigo
colorTo: purple
sdk: static
pinned: false
short_description: Voice chat over WebRTC against a HF speech-to-speech

Minimal Conversation App (S2S backend)

Tiny browser app that opens a WebRTC voice session with a self-hosted Hugging Face speech-to-speech backend. No OpenAI key, no robot, no OAuth, no proxy. The browser talks directly to the backend.

How it works

1. App POSTs <lb_url>/session (empty JSON body).
2. The load balancer picks a ready compute (round-robin) and returns:

   {
     "session_id": "...",
     "websocket_url": "wss://<compute>/v1/realtime",
     "connect_url": "wss://<compute>/v1/realtime?session_token=<JWT>",
     "session_token": "<JWT>",
     "pending_timeout_s": 60
   }
   

3. App rewrites connect_url from wss:// to https:// and POSTs an SDP offer to it (Content-Type: application/sdp). The compute, running aiortc, answers 201 application/sdp with the SDP answer.
4. WebRTC peer connection opens. Audio streams over the media tracks (the browser sends Opus 48 kHz, the backend resamples to 16 kHz for STT and back for TTS). Events flow over a data channel labelled oai-events using the OpenAI Realtime JSON protocol (session.update, response.audio_transcript.delta, conversation.item.input_audio_transcription.delta, ...).

The backend exposes one concurrent session per compute; the LB pins a session to a specific compute via a signed session_token.

Backend requirement

This app talks to the WebRTC route POST /v1/realtime defined in websocket_router.py. That route lives on the feat/webrtc-transport branch. If the compute is deployed from main it will reply 404 on the SDP POST. Verify with:

LB="https://<your-lb>.endpoints.huggingface.cloud"
curl -X POST "$LB/session" -H "Content-Type: application/json" -d '{}'
# -> { "connect_url": "wss://<compute>/v1/realtime?session_token=..." }

# Rewrite wss -> https, POST a SDP offer:
curl -X POST "https://<compute>/v1/realtime?session_token=..." \
  -H "Content-Type: application/sdp" --data-binary @offer.sdp
# expect 201 application/sdp; if 404, the compute is on the wrong branch.

Run locally

This is a pure static app. No build step, no node_modules. Any HTTP server works:

cd minimal-conversation-app-s2s-backend
python3 -m http.server 5173
# or: npx --yes http-server -p 5173 -c-1

Then open http://localhost:5173/, click the orb, allow the mic, talk.

Browsers require HTTPS or localhost for getUserMedia(). 127.0.0.1 works, as does localhost. Plain http://192.168.x.y:5173 does NOT (mic blocked).

Settings (stored in localStorage)

Key	What
Load balancer URL	Base URL of your S2S deployment. App POSTs <lb>/session.
Voice	Qwen3-TTS speaker name (Aiden, Ryan, Dylan, Eric, Ono_Anna, Serena, Sohee, Uncle_Fu, Vivian)
Instructions	System prompt sent in session.update once the data channel opens

Files

File	Role
index.html	Single page, orb + settings modal
main.js	State machine, settings, mic capture, output audio piping
s2s-client.js	WebRTC handshake against the LB + compute
style.css	Orb animations, layout, dark theme

Credits

• Backend: huggingface/speech-to-speech on feat/webrtc-transport
• UI forked from reachy_mini_minimal_conversation (Pollen Robotics × Hugging Face), stripped of all robot-specific bits