carlosduplar commited on
Commit ·
c7ed502
1
Parent(s): c350ccf
E2B experiment: Tiny Titan Space on ZeroGPU with Gemma 4 E2B
Browse files- New patient-virtuel-e2b/ directory with simplified 3-phase prompt
- ffmpeg audio decode, no Whisper, native E2B audio via Gemma4Processor
- 'sdpa' attention to avoid flash-attn dependency on ZeroGPU
- trace.jsonl: 18-step curated build trace for HF Dataset
- LINKEDIN_POST.md, VIDEO_SCRIPT.md: rewritten first-person, no em dashes
- README.md: HF Dataset + experiment Space links
- server_app.py: title changed to 'Field Notes: Patient Virtuel'
- README.md +12 -0
- patient-virtuel-e2b/README.md +53 -0
- patient-virtuel-e2b/core.py +286 -0
- patient-virtuel-e2b/custom_index.html +982 -0
- patient-virtuel-e2b/parse_feedback.py +130 -0
- patient-virtuel-e2b/prompts_e2b.py +47 -0
- patient-virtuel-e2b/requirements.txt +9 -0
- patient-virtuel-e2b/server_app.py +105 -0
- patient-virtuel-e2b/tts_engine.py +98 -0
- server_app.py +1 -1
- trace.jsonl +18 -0
README.md
CHANGED
|
@@ -8,6 +8,16 @@ sdk_version: 6.16.0
|
|
| 8 |
app_file: server_app.py
|
| 9 |
pinned: false
|
| 10 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
# Patient Virtuel · Français en médecine dentaire
|
|
@@ -25,6 +35,8 @@ Uses `gradio.Server` with a custom HTML/CSS/JS frontend (Off-Brand Award entry).
|
|
| 25 |
See [`space_README.md`](space_README.md) for the full description.
|
| 26 |
|
| 27 |
📖 [Field Notes](FIELD_NOTES.md) — architecture decisions, lessons learned, and the story behind the app.
|
|
|
|
|
|
|
| 28 |
|
| 29 |
## Architecture
|
| 30 |
|
|
|
|
| 8 |
app_file: server_app.py
|
| 9 |
pinned: false
|
| 10 |
license: apache-2.0
|
| 11 |
+
tags:
|
| 12 |
+
- hackathon
|
| 13 |
+
- build-small-hackathon
|
| 14 |
+
- voice-ai
|
| 15 |
+
- french-learning
|
| 16 |
+
- dental
|
| 17 |
+
- gradio
|
| 18 |
+
- modal
|
| 19 |
+
- llama-cpp
|
| 20 |
+
- gemma
|
| 21 |
---
|
| 22 |
|
| 23 |
# Patient Virtuel · Français en médecine dentaire
|
|
|
|
| 35 |
See [`space_README.md`](space_README.md) for the full description.
|
| 36 |
|
| 37 |
📖 [Field Notes](FIELD_NOTES.md) — architecture decisions, lessons learned, and the story behind the app.
|
| 38 |
+
📡 [Agent Trace](https://huggingface.co/datasets/build-small-hackathon/patient-virtuel-trace) — curated build log on HF Hub (Sharing is Caring badge).
|
| 39 |
+
🐜 [Tiny Titan Experiment](https://huggingface.co/spaces/build-small-hackathon/patient-virtuel-e2b) — Patient Virtuel on Gemma 4 E2B via ZeroGPU (2.3B params).
|
| 40 |
|
| 41 |
## Architecture
|
| 42 |
|
patient-virtuel-e2b/README.md
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: Patient Virtuel E2B · Tiny Titan
|
| 3 |
+
emoji: 🦷
|
| 4 |
+
colorFrom: red
|
| 5 |
+
colorTo: gray
|
| 6 |
+
sdk: gradio
|
| 7 |
+
sdk_version: 6.16.0
|
| 8 |
+
python_version: "3.12"
|
| 9 |
+
app_file: server_app.py
|
| 10 |
+
pinned: false
|
| 11 |
+
license: apache-2.0
|
| 12 |
+
short_description: Voice dental patient on Gemma 4 E2B (2.3B params)
|
| 13 |
+
tags:
|
| 14 |
+
- hackathon
|
| 15 |
+
- tiny-titan
|
| 16 |
+
- gemma
|
| 17 |
+
- zerogpu
|
| 18 |
+
---
|
| 19 |
+
# Patient Virtuel E2B · Tiny Titan Experiment
|
| 20 |
+
|
| 21 |
+
A voice-based AI dental patient running on **Gemma 4 E2B** (2.3B effective params) via **ZeroGPU**. This is an experiment to claim the Tiny Titan award (≤4B parameter app).
|
| 22 |
+
|
| 23 |
+
Uses `gradio.Server` with a custom HTML/CSS/JS frontend.
|
| 24 |
+
|
| 25 |
+
## How it differs from the main app
|
| 26 |
+
|
| 27 |
+
| Feature | Main app | This app |
|
| 28 |
+
|---|---|---|
|
| 29 |
+
| Model | Gemma 4 26B A4B (Modal L4) | Gemma 4 E2B (ZeroGPU) |
|
| 30 |
+
| STT | faster-whisper (Modal T4) | Built into E2B (native audio) |
|
| 31 |
+
| Infrastructure | Modal + HF Space | ZeroGPU only |
|
| 32 |
+
| Qualifies for | Llama Champion, Off the Grid | Tiny Titan |
|
| 33 |
+
|
| 34 |
+
## Architecture
|
| 35 |
+
|
| 36 |
+
The app uses Gemma 4 E2B's native audio encoder to process speech directly, eliminating the need for a separate STT model:
|
| 37 |
+
|
| 38 |
+
```
|
| 39 |
+
Microphone → ffmpeg decode → Gemma 4 E2B (ZeroGPU) → text → piper-tts (CPU)
|
| 40 |
+
```
|
| 41 |
+
|
| 42 |
+
No cloud APIs, no separate ASR model. Everything runs on a single model.
|
| 43 |
+
|
| 44 |
+
## Badges
|
| 45 |
+
|
| 46 |
+
- Tiny Titan — app built on a ≤4B model (Gemma 4 E2B at 2.3B effective)
|
| 47 |
+
- Shares the custom frontend (Off-Brand) and piped TTS (Off the Grid) badges
|
| 48 |
+
|
| 49 |
+
## Links
|
| 50 |
+
|
| 51 |
+
- **Main app**: https://huggingface.co/spaces/build-small-hackathon/patient-virtuel-dentiste
|
| 52 |
+
- **Field Notes**: https://huggingface.co/spaces/build-small-hackathon/patient-virtuel-dentiste/field-notes
|
| 53 |
+
- **Agent Trace**: https://huggingface.co/datasets/build-small-hackathon/patient-virtuel-trace
|
patient-virtuel-e2b/core.py
ADDED
|
@@ -0,0 +1,286 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import io
|
| 2 |
+
import logging
|
| 3 |
+
import os
|
| 4 |
+
import re
|
| 5 |
+
import subprocess
|
| 6 |
+
import tempfile
|
| 7 |
+
import time
|
| 8 |
+
import wave
|
| 9 |
+
|
| 10 |
+
import numpy as np
|
| 11 |
+
|
| 12 |
+
log = logging.getLogger(__name__)
|
| 13 |
+
|
| 14 |
+
try:
|
| 15 |
+
from spaces import GPU
|
| 16 |
+
except ImportError:
|
| 17 |
+
GPU = lambda **kw: lambda fn: fn
|
| 18 |
+
|
| 19 |
+
from prompts_e2b import SYSTEM_PROMPT, PHASE_SWITCH_REMINDER
|
| 20 |
+
from parse_feedback import parse_feedback, render_feedback_table, strip_markdown
|
| 21 |
+
from tts_engine import synthesize
|
| 22 |
+
|
| 23 |
+
TERMINATE_RE = re.compile(
|
| 24 |
+
r"(fin\s+de\s+(la\s+)?séance|session\s+terminée|on\s+a\s+terminé|c'est\s+fini)",
|
| 25 |
+
re.IGNORECASE,
|
| 26 |
+
)
|
| 27 |
+
|
| 28 |
+
_MODEL = None
|
| 29 |
+
_PROCESSOR = None
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def load_model():
|
| 33 |
+
global _MODEL, _PROCESSOR
|
| 34 |
+
if _MODEL is not None:
|
| 35 |
+
return _MODEL, _PROCESSOR
|
| 36 |
+
import torch
|
| 37 |
+
from transformers import AutoModelForMultimodalLM, AutoProcessor
|
| 38 |
+
model_id = "google/gemma-4-E2B-it"
|
| 39 |
+
log.info("Loading %s on ZeroGPU...", model_id)
|
| 40 |
+
_PROCESSOR = AutoProcessor.from_pretrained(model_id)
|
| 41 |
+
_MODEL = AutoModelForMultimodalLM.from_pretrained(
|
| 42 |
+
model_id,
|
| 43 |
+
torch_dtype=torch.bfloat16,
|
| 44 |
+
device_map="cuda",
|
| 45 |
+
attn_implementation="sdpa",
|
| 46 |
+
)
|
| 47 |
+
_MODEL.eval()
|
| 48 |
+
log.info("Model loaded")
|
| 49 |
+
return _MODEL, _PROCESSOR
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
@GPU(duration=60)
|
| 53 |
+
def e2b_generate(audio_array: np.ndarray, sampling_rate: int, history_text: str) -> str:
|
| 54 |
+
model, processor = load_model()
|
| 55 |
+
import torch
|
| 56 |
+
|
| 57 |
+
audio_array = audio_array.astype(np.float32)
|
| 58 |
+
if audio_array.ndim == 1:
|
| 59 |
+
audio_array = audio_array[np.newaxis, :]
|
| 60 |
+
|
| 61 |
+
messages = [
|
| 62 |
+
{"role": "system", "content": SYSTEM_PROMPT},
|
| 63 |
+
{"role": "user", "content": [
|
| 64 |
+
{"type": "text", "text": history_text},
|
| 65 |
+
{"type": "audio", "audio": audio_array},
|
| 66 |
+
]},
|
| 67 |
+
]
|
| 68 |
+
|
| 69 |
+
inputs = processor(
|
| 70 |
+
text=messages,
|
| 71 |
+
padding=True,
|
| 72 |
+
return_tensors="pt",
|
| 73 |
+
)
|
| 74 |
+
inputs = {k: v.to("cuda") if hasattr(v, "to") else v for k, v in inputs.items()}
|
| 75 |
+
|
| 76 |
+
with torch.no_grad():
|
| 77 |
+
output_ids = model.generate(
|
| 78 |
+
**inputs,
|
| 79 |
+
max_new_tokens=256,
|
| 80 |
+
temperature=0.8,
|
| 81 |
+
top_p=0.9,
|
| 82 |
+
do_sample=True,
|
| 83 |
+
)
|
| 84 |
+
|
| 85 |
+
input_len = inputs["input_ids"].shape[1]
|
| 86 |
+
response = processor.decode(output_ids[0, input_len:], skip_special_tokens=True)
|
| 87 |
+
return response.strip()
|
| 88 |
+
|
| 89 |
+
|
| 90 |
+
@GPU(duration=60)
|
| 91 |
+
def e2b_generate_text(user_message: str) -> str:
|
| 92 |
+
model, processor = load_model()
|
| 93 |
+
import torch
|
| 94 |
+
|
| 95 |
+
messages = [
|
| 96 |
+
{"role": "system", "content": SYSTEM_PROMPT},
|
| 97 |
+
{"role": "user", "content": user_message},
|
| 98 |
+
]
|
| 99 |
+
|
| 100 |
+
inputs = processor(
|
| 101 |
+
text=messages,
|
| 102 |
+
padding=True,
|
| 103 |
+
return_tensors="pt",
|
| 104 |
+
)
|
| 105 |
+
inputs = {k: v.to("cuda") if hasattr(v, "to") else v for k, v in inputs.items()}
|
| 106 |
+
|
| 107 |
+
with torch.no_grad():
|
| 108 |
+
output_ids = model.generate(
|
| 109 |
+
**inputs,
|
| 110 |
+
max_new_tokens=512,
|
| 111 |
+
temperature=0.7,
|
| 112 |
+
top_p=0.9,
|
| 113 |
+
do_sample=True,
|
| 114 |
+
)
|
| 115 |
+
|
| 116 |
+
input_len = inputs["input_ids"].shape[1]
|
| 117 |
+
response = processor.decode(output_ids[0, input_len:], skip_special_tokens=True)
|
| 118 |
+
return response.strip()
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
def decode_audio(file_path: str, target_sr: int = 16000) -> tuple:
|
| 122 |
+
result = subprocess.run(
|
| 123 |
+
["ffmpeg", "-i", file_path, "-f", "wav", "-acodec", "pcm_s16le",
|
| 124 |
+
"-ar", str(target_sr), "-ac", "1", "pipe:1"],
|
| 125 |
+
capture_output=True, timeout=30,
|
| 126 |
+
)
|
| 127 |
+
result.check_returncode()
|
| 128 |
+
with wave.open(io.BytesIO(result.stdout)) as wf:
|
| 129 |
+
frames = wf.readframes(wf.getnframes())
|
| 130 |
+
array = np.frombuffer(frames, dtype=np.int16).astype(np.float32) / 32768.0
|
| 131 |
+
return array, target_sr
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
def _build_history_text(messages: list) -> str:
|
| 135 |
+
lines = []
|
| 136 |
+
for msg in messages:
|
| 137 |
+
role = msg.get("role", "")
|
| 138 |
+
content = msg.get("content", "")
|
| 139 |
+
if role == "system":
|
| 140 |
+
continue
|
| 141 |
+
if isinstance(content, list):
|
| 142 |
+
content = ""
|
| 143 |
+
if role == "user":
|
| 144 |
+
lines.append(f"Hygienist: {content}")
|
| 145 |
+
elif role == "assistant":
|
| 146 |
+
lines.append(f"Patient: {content}")
|
| 147 |
+
return "\n".join(lines)
|
| 148 |
+
|
| 149 |
+
|
| 150 |
+
def make_initial_state():
|
| 151 |
+
messages = [{"role": "system", "content": SYSTEM_PROMPT}]
|
| 152 |
+
return {"messages": messages, "phase": 1, "turn_count": 0}
|
| 153 |
+
|
| 154 |
+
|
| 155 |
+
def _chat_val(state):
|
| 156 |
+
return [m for m in state["messages"] if m.get("content") != PHASE_SWITCH_REMINDER]
|
| 157 |
+
|
| 158 |
+
|
| 159 |
+
def _make_audio(audio_bytes):
|
| 160 |
+
if not audio_bytes:
|
| 161 |
+
return None
|
| 162 |
+
f = tempfile.NamedTemporaryFile(suffix=".wav", delete=False, dir=tempfile.gettempdir())
|
| 163 |
+
f.write(audio_bytes)
|
| 164 |
+
f.close()
|
| 165 |
+
return os.path.basename(f.name)
|
| 166 |
+
|
| 167 |
+
|
| 168 |
+
def _default_feedback():
|
| 169 |
+
return {
|
| 170 |
+
"feedback_intro": "",
|
| 171 |
+
"feedback_points_forts": [],
|
| 172 |
+
"feedback_table": [],
|
| 173 |
+
"feedback_vocabulaire": [],
|
| 174 |
+
"feedback_priorite": [],
|
| 175 |
+
"feedback_bilan": {},
|
| 176 |
+
"feedback_open": False,
|
| 177 |
+
}
|
| 178 |
+
|
| 179 |
+
|
| 180 |
+
def process_turn(audio_path, state):
|
| 181 |
+
state = dict(state)
|
| 182 |
+
result = {
|
| 183 |
+
"chat": _chat_val(state),
|
| 184 |
+
"audio_file": None,
|
| 185 |
+
"state": state,
|
| 186 |
+
**_default_feedback(),
|
| 187 |
+
"status": "",
|
| 188 |
+
}
|
| 189 |
+
|
| 190 |
+
if not audio_path:
|
| 191 |
+
return result
|
| 192 |
+
|
| 193 |
+
result["status"] = "Decoding audio..."
|
| 194 |
+
_t0 = time.monotonic()
|
| 195 |
+
try:
|
| 196 |
+
audio_array, sr = decode_audio(audio_path)
|
| 197 |
+
_t1 = time.monotonic()
|
| 198 |
+
except Exception as e:
|
| 199 |
+
result["status"] = f"Audio error: {e}"
|
| 200 |
+
return result
|
| 201 |
+
|
| 202 |
+
if len(audio_array) < sr * 0.5:
|
| 203 |
+
result["status"] = "Please speak longer."
|
| 204 |
+
return result
|
| 205 |
+
|
| 206 |
+
result["status"] = "Patient thinking..."
|
| 207 |
+
history_text = _build_history_text(state["messages"])
|
| 208 |
+
response = e2b_generate(audio_array, sr, history_text)
|
| 209 |
+
_t2 = time.monotonic()
|
| 210 |
+
|
| 211 |
+
if not response:
|
| 212 |
+
result["status"] = "Model error. Try again."
|
| 213 |
+
return result
|
| 214 |
+
|
| 215 |
+
response = re.sub(r"^(assistant|model)\s*", "", response).strip()
|
| 216 |
+
|
| 217 |
+
if TERMINATE_RE.search(response):
|
| 218 |
+
state["messages"].append({"role": "assistant", "content": response})
|
| 219 |
+
return _end_session(state)
|
| 220 |
+
|
| 221 |
+
state["messages"].append({"role": "user", "content": "(audio input)"})
|
| 222 |
+
state["messages"].append({"role": "assistant", "content": response})
|
| 223 |
+
|
| 224 |
+
result["status"] = "Synthesizing speech..."
|
| 225 |
+
clean = strip_markdown(response)
|
| 226 |
+
audio_bytes = synthesize(clean)
|
| 227 |
+
_t3 = time.monotonic()
|
| 228 |
+
result["audio_file"] = _make_audio(audio_bytes)
|
| 229 |
+
log.info("TIMING audio=%.1fs inference=%.1fs tts=%.1fs total=%.1fs",
|
| 230 |
+
_t1 - _t0, _t2 - _t1, _t3 - _t2, _t3 - _t0)
|
| 231 |
+
result["chat"] = _chat_val(state)
|
| 232 |
+
result["status"] = ""
|
| 233 |
+
return result
|
| 234 |
+
|
| 235 |
+
|
| 236 |
+
def _end_session(state):
|
| 237 |
+
state["messages"].append({"role": "user", "content": PHASE_SWITCH_REMINDER})
|
| 238 |
+
|
| 239 |
+
result = {
|
| 240 |
+
"chat": _chat_val(state),
|
| 241 |
+
"audio_file": None,
|
| 242 |
+
"state": state,
|
| 243 |
+
**_default_feedback(),
|
| 244 |
+
"status": "Generating recap...",
|
| 245 |
+
}
|
| 246 |
+
|
| 247 |
+
history_text = _build_history_text(state["messages"])
|
| 248 |
+
response = e2b_generate_text(history_text + "\n\nGenerate the structured feedback now.")
|
| 249 |
+
_t0 = time.monotonic()
|
| 250 |
+
|
| 251 |
+
if not response:
|
| 252 |
+
result["status"] = "Error generating recap."
|
| 253 |
+
return result
|
| 254 |
+
|
| 255 |
+
clean = strip_markdown(response)
|
| 256 |
+
state["phase"] = 2
|
| 257 |
+
|
| 258 |
+
fb = parse_feedback(clean)
|
| 259 |
+
table = render_feedback_table(fb["erreurs"]) if fb["erreurs"] else []
|
| 260 |
+
intro = fb.get("intro") or clean
|
| 261 |
+
|
| 262 |
+
state["messages"].append({"role": "assistant", "content": intro})
|
| 263 |
+
|
| 264 |
+
audio_bytes = synthesize(intro)
|
| 265 |
+
_t1 = time.monotonic()
|
| 266 |
+
|
| 267 |
+
result["chat"] = _chat_val(state)
|
| 268 |
+
result["audio_file"] = _make_audio(audio_bytes)
|
| 269 |
+
log.info("TIMING llm=%.1fs tts=%.1fs", _t1 - _t0, result["audio_file"])
|
| 270 |
+
result["feedback_intro"] = intro
|
| 271 |
+
result["feedback_points_forts"] = fb["points_forts"]
|
| 272 |
+
result["feedback_table"] = table
|
| 273 |
+
result["feedback_vocabulaire"] = fb["vocabulaire"]
|
| 274 |
+
result["feedback_priorite"] = fb["priorite"]
|
| 275 |
+
result["feedback_bilan"] = fb["bilan"]
|
| 276 |
+
result["feedback_open"] = True
|
| 277 |
+
result["status"] = ""
|
| 278 |
+
return result
|
| 279 |
+
|
| 280 |
+
|
| 281 |
+
def end_session_click(state):
|
| 282 |
+
return _end_session(dict(state))
|
| 283 |
+
|
| 284 |
+
|
| 285 |
+
def reset_session():
|
| 286 |
+
return make_initial_state()
|
patient-virtuel-e2b/custom_index.html
ADDED
|
@@ -0,0 +1,982 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<!DOCTYPE html>
|
| 2 |
+
<html lang="fr">
|
| 3 |
+
<head>
|
| 4 |
+
<meta charset="UTF-8">
|
| 5 |
+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
| 6 |
+
<title>Patient Virtuel E2B · Tiny Titan</title>
|
| 7 |
+
<link rel="preconnect" href="https://fonts.googleapis.com">
|
| 8 |
+
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
| 9 |
+
<link href="https://fonts.googleapis.com/css2?family=EB+Garamond:ital,wght@0,400;0,500;1,400;1,500&family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
|
| 10 |
+
<style>
|
| 11 |
+
*, *::before, *::after { margin: 0; box-sizing: border-box; }
|
| 12 |
+
|
| 13 |
+
:root {
|
| 14 |
+
--bg: #19120c;
|
| 15 |
+
--accent: #ff5719;
|
| 16 |
+
--accent-dim: rgba(255, 87, 25, 0.15);
|
| 17 |
+
--glass: rgba(255, 87, 25, 0.06);
|
| 18 |
+
--glass-border: rgba(255, 200, 150, 0.1);
|
| 19 |
+
--text: #efe0d4;
|
| 20 |
+
--text-dim: #e6beb2;
|
| 21 |
+
--surface-low: #221a13;
|
| 22 |
+
--surface-high: #312821;
|
| 23 |
+
--outline: #ac897e;
|
| 24 |
+
--outline-variant: #5c4037;
|
| 25 |
+
--serif: "EB Garamond", Georgia, serif;
|
| 26 |
+
--sans: "Inter", ui-sans-serif, system-ui, sans-serif;
|
| 27 |
+
}
|
| 28 |
+
|
| 29 |
+
html, body {
|
| 30 |
+
height: 100%;
|
| 31 |
+
background: var(--bg);
|
| 32 |
+
color: var(--text);
|
| 33 |
+
font-family: var(--sans);
|
| 34 |
+
-webkit-font-smoothing: antialiased;
|
| 35 |
+
overflow-x: hidden;
|
| 36 |
+
}
|
| 37 |
+
|
| 38 |
+
body {
|
| 39 |
+
background-image:
|
| 40 |
+
radial-gradient(60rem 50rem at 0% 100%, color-mix(in srgb, var(--accent) 18%, transparent), transparent 60%),
|
| 41 |
+
radial-gradient(50rem 40rem at 100% 0%, color-mix(in srgb, var(--accent) 8%, transparent), transparent 60%);
|
| 42 |
+
background-attachment: fixed;
|
| 43 |
+
}
|
| 44 |
+
|
| 45 |
+
.glass {
|
| 46 |
+
background: var(--glass);
|
| 47 |
+
backdrop-filter: blur(40px);
|
| 48 |
+
-webkit-backdrop-filter: blur(40px);
|
| 49 |
+
border: 1px solid var(--glass-border);
|
| 50 |
+
box-shadow: 0 20px 50px rgba(0,0,0,0.3);
|
| 51 |
+
border-radius: 16px;
|
| 52 |
+
}
|
| 53 |
+
|
| 54 |
+
.app {
|
| 55 |
+
position: relative;
|
| 56 |
+
z-index: 1;
|
| 57 |
+
min-height: 100vh;
|
| 58 |
+
display: flex;
|
| 59 |
+
flex-direction: column;
|
| 60 |
+
max-width: 1280px;
|
| 61 |
+
margin: 0 auto;
|
| 62 |
+
padding: 28px 32px;
|
| 63 |
+
}
|
| 64 |
+
|
| 65 |
+
.header {
|
| 66 |
+
display: flex;
|
| 67 |
+
align-items: center;
|
| 68 |
+
gap: 14px;
|
| 69 |
+
margin-bottom: 36px;
|
| 70 |
+
flex-shrink: 0;
|
| 71 |
+
align-self: center;
|
| 72 |
+
}
|
| 73 |
+
.header-icon {
|
| 74 |
+
width: 44px; height: 44px;
|
| 75 |
+
border-radius: 50%;
|
| 76 |
+
background: var(--accent);
|
| 77 |
+
display: flex;
|
| 78 |
+
align-items: center;
|
| 79 |
+
justify-content: center;
|
| 80 |
+
box-shadow: 0 4px 20px rgba(255, 87, 25, 0.3);
|
| 81 |
+
flex-shrink: 0;
|
| 82 |
+
}
|
| 83 |
+
.header-icon svg { width: 24px; height: 24px; color: #fff; }
|
| 84 |
+
.header-title {
|
| 85 |
+
font-family: var(--serif);
|
| 86 |
+
font-size: 32px;
|
| 87 |
+
font-weight: 400;
|
| 88 |
+
letter-spacing: 0.01em;
|
| 89 |
+
}
|
| 90 |
+
.header-sub {
|
| 91 |
+
font-size: 14px;
|
| 92 |
+
text-transform: uppercase;
|
| 93 |
+
letter-spacing: 0.2em;
|
| 94 |
+
color: var(--accent);
|
| 95 |
+
font-weight: 600;
|
| 96 |
+
}
|
| 97 |
+
|
| 98 |
+
.main-grid {
|
| 99 |
+
display: grid;
|
| 100 |
+
grid-template-columns: 5fr 7fr;
|
| 101 |
+
gap: 32px;
|
| 102 |
+
flex: 1;
|
| 103 |
+
min-height: 0;
|
| 104 |
+
}
|
| 105 |
+
|
| 106 |
+
.left-col {
|
| 107 |
+
display: flex;
|
| 108 |
+
flex-direction: column;
|
| 109 |
+
gap: 28px;
|
| 110 |
+
}
|
| 111 |
+
|
| 112 |
+
.desc h2 {
|
| 113 |
+
font-family: var(--serif);
|
| 114 |
+
font-size: 56px;
|
| 115 |
+
font-weight: 400;
|
| 116 |
+
line-height: 1.15;
|
| 117 |
+
letter-spacing: -0.01em;
|
| 118 |
+
}
|
| 119 |
+
.desc h2 em {
|
| 120 |
+
font-style: italic;
|
| 121 |
+
color: var(--accent);
|
| 122 |
+
}
|
| 123 |
+
.desc p {
|
| 124 |
+
color: var(--text-dim);
|
| 125 |
+
font-size: 14px;
|
| 126 |
+
line-height: 1.6;
|
| 127 |
+
margin-top: 16px;
|
| 128 |
+
}
|
| 129 |
+
|
| 130 |
+
.howto {
|
| 131 |
+
padding: 24px;
|
| 132 |
+
background: rgba(255,255,255,0.03);
|
| 133 |
+
border: 1px solid rgba(255,255,255,0.08);
|
| 134 |
+
border-radius: 12px;
|
| 135 |
+
backdrop-filter: blur(10px);
|
| 136 |
+
}
|
| 137 |
+
.howto .howto-title {
|
| 138 |
+
font-size: 10px;
|
| 139 |
+
text-transform: uppercase;
|
| 140 |
+
letter-spacing: 0.15em;
|
| 141 |
+
color: var(--accent);
|
| 142 |
+
font-weight: 700;
|
| 143 |
+
}
|
| 144 |
+
.howto ol {
|
| 145 |
+
list-style: none;
|
| 146 |
+
margin: 16px 0 0;
|
| 147 |
+
padding: 0;
|
| 148 |
+
display: flex;
|
| 149 |
+
flex-direction: column;
|
| 150 |
+
gap: 12px;
|
| 151 |
+
}
|
| 152 |
+
.howto li {
|
| 153 |
+
display: flex;
|
| 154 |
+
gap: 12px;
|
| 155 |
+
font-size: 13px;
|
| 156 |
+
color: var(--text-dim);
|
| 157 |
+
line-height: 1.6;
|
| 158 |
+
}
|
| 159 |
+
.howto li .num {
|
| 160 |
+
flex-shrink: 0;
|
| 161 |
+
width: 20px; height: 20px;
|
| 162 |
+
border-radius: 50%;
|
| 163 |
+
display: grid;
|
| 164 |
+
place-items: center;
|
| 165 |
+
border: 1px solid rgba(255,255,255,0.15);
|
| 166 |
+
font-size: 10px;
|
| 167 |
+
font-weight: 600;
|
| 168 |
+
color: var(--accent);
|
| 169 |
+
margin-top: 2px;
|
| 170 |
+
}
|
| 171 |
+
.howto li strong { color: var(--text); }
|
| 172 |
+
|
| 173 |
+
.mic-area {
|
| 174 |
+
display: flex;
|
| 175 |
+
flex-direction: column;
|
| 176 |
+
align-items: center;
|
| 177 |
+
gap: 20px;
|
| 178 |
+
padding: 12px 0;
|
| 179 |
+
}
|
| 180 |
+
.mic-btn {
|
| 181 |
+
width: 136px; height: 136px;
|
| 182 |
+
border-radius: 50%;
|
| 183 |
+
border: 2px solid var(--glass-border);
|
| 184 |
+
background: var(--glass);
|
| 185 |
+
backdrop-filter: blur(30px);
|
| 186 |
+
cursor: pointer;
|
| 187 |
+
display: flex;
|
| 188 |
+
align-items: center;
|
| 189 |
+
justify-content: center;
|
| 190 |
+
transition: all 0.3s ease;
|
| 191 |
+
position: relative;
|
| 192 |
+
}
|
| 193 |
+
.mic-btn:hover {
|
| 194 |
+
border-color: var(--accent);
|
| 195 |
+
box-shadow: 0 0 40px rgba(255,87,25,0.2);
|
| 196 |
+
}
|
| 197 |
+
.mic-btn:disabled {
|
| 198 |
+
opacity: 0.4;
|
| 199 |
+
cursor: not-allowed;
|
| 200 |
+
}
|
| 201 |
+
.mic-btn.recording {
|
| 202 |
+
border-color: var(--accent);
|
| 203 |
+
background: var(--accent-dim);
|
| 204 |
+
animation: mic-pulse 1.8s ease-in-out infinite;
|
| 205 |
+
}
|
| 206 |
+
@keyframes mic-pulse {
|
| 207 |
+
0%, 100% {
|
| 208 |
+
box-shadow:
|
| 209 |
+
0 0 0 0 color-mix(in srgb, var(--accent) 55%, transparent),
|
| 210 |
+
0 0 60px 10px color-mix(in srgb, var(--accent) 35%, transparent);
|
| 211 |
+
}
|
| 212 |
+
50% {
|
| 213 |
+
box-shadow:
|
| 214 |
+
0 0 0 24px color-mix(in srgb, var(--accent) 0%, transparent),
|
| 215 |
+
0 0 90px 24px color-mix(in srgb, var(--accent) 45%, transparent);
|
| 216 |
+
}
|
| 217 |
+
}
|
| 218 |
+
.mic-btn svg { width: 42px; height: 42px; color: var(--text); transition: color 0.3s; }
|
| 219 |
+
.mic-btn.recording svg { color: var(--accent); }
|
| 220 |
+
|
| 221 |
+
.mic-label {
|
| 222 |
+
font-size: 10px;
|
| 223 |
+
text-transform: uppercase;
|
| 224 |
+
letter-spacing: 0.18em;
|
| 225 |
+
color: var(--text-dim);
|
| 226 |
+
padding: 6px 16px;
|
| 227 |
+
border-radius: 20px;
|
| 228 |
+
background: var(--surface-low);
|
| 229 |
+
backdrop-filter: blur(10px);
|
| 230 |
+
border: 1px solid var(--glass-border);
|
| 231 |
+
font-weight: 600;
|
| 232 |
+
}
|
| 233 |
+
.mic-label.active {
|
| 234 |
+
color: var(--accent);
|
| 235 |
+
border-color: var(--accent);
|
| 236 |
+
background: var(--accent-dim);
|
| 237 |
+
}
|
| 238 |
+
|
| 239 |
+
.controls {
|
| 240 |
+
display: flex;
|
| 241 |
+
gap: 12px;
|
| 242 |
+
}
|
| 243 |
+
.btn {
|
| 244 |
+
flex: 1;
|
| 245 |
+
padding: 11px 20px;
|
| 246 |
+
border-radius: 10px;
|
| 247 |
+
border: 1px solid var(--glass-border);
|
| 248 |
+
background: var(--glass);
|
| 249 |
+
color: var(--text);
|
| 250 |
+
font-size: 13px;
|
| 251 |
+
font-weight: 600;
|
| 252 |
+
font-family: var(--sans);
|
| 253 |
+
cursor: pointer;
|
| 254 |
+
transition: all 0.2s;
|
| 255 |
+
}
|
| 256 |
+
.btn:hover { border-color: var(--accent); background: var(--accent-dim); }
|
| 257 |
+
.btn-stop {
|
| 258 |
+
background: rgba(139, 0, 0, 0.35);
|
| 259 |
+
border-color: rgba(139, 0, 0, 0.5);
|
| 260 |
+
color: #ffb4ab;
|
| 261 |
+
}
|
| 262 |
+
.btn-stop:hover { background: rgba(139, 0, 0, 0.6); }
|
| 263 |
+
.btn:disabled { opacity: 0.35; cursor: not-allowed; }
|
| 264 |
+
|
| 265 |
+
#status {
|
| 266 |
+
text-align: center;
|
| 267 |
+
font-size: 13px;
|
| 268 |
+
color: var(--accent);
|
| 269 |
+
font-weight: 500;
|
| 270 |
+
min-height: 20px;
|
| 271 |
+
transition: opacity 0.2s;
|
| 272 |
+
}
|
| 273 |
+
|
| 274 |
+
.right-col {
|
| 275 |
+
display: flex;
|
| 276 |
+
flex-direction: column;
|
| 277 |
+
gap: 20px;
|
| 278 |
+
min-height: 0;
|
| 279 |
+
}
|
| 280 |
+
|
| 281 |
+
.chat-wrap {
|
| 282 |
+
flex: 1;
|
| 283 |
+
display: flex;
|
| 284 |
+
flex-direction: column;
|
| 285 |
+
border: 1px solid var(--glass-border);
|
| 286 |
+
border-radius: 16px;
|
| 287 |
+
background: var(--glass);
|
| 288 |
+
backdrop-filter: blur(40px);
|
| 289 |
+
overflow: hidden;
|
| 290 |
+
min-height: 0;
|
| 291 |
+
}
|
| 292 |
+
.chat-header {
|
| 293 |
+
display: flex;
|
| 294 |
+
align-items: center;
|
| 295 |
+
justify-content: space-between;
|
| 296 |
+
padding: 14px 18px;
|
| 297 |
+
border-bottom: 1px solid var(--glass-border);
|
| 298 |
+
}
|
| 299 |
+
.chat-header-label {
|
| 300 |
+
font-size: 10px;
|
| 301 |
+
text-transform: uppercase;
|
| 302 |
+
letter-spacing: 0.18em;
|
| 303 |
+
color: var(--accent);
|
| 304 |
+
font-weight: 600;
|
| 305 |
+
}
|
| 306 |
+
.chat-header-live {
|
| 307 |
+
font-size: 10px;
|
| 308 |
+
text-transform: uppercase;
|
| 309 |
+
letter-spacing: 0.12em;
|
| 310 |
+
color: var(--text-dim);
|
| 311 |
+
}
|
| 312 |
+
#chat {
|
| 313 |
+
flex: 1;
|
| 314 |
+
display: flex;
|
| 315 |
+
flex-direction: column;
|
| 316 |
+
gap: 12px;
|
| 317 |
+
padding: 16px 18px;
|
| 318 |
+
overflow-y: auto;
|
| 319 |
+
min-height: 120px;
|
| 320 |
+
max-height: 50vh;
|
| 321 |
+
}
|
| 322 |
+
#chat::-webkit-scrollbar { width: 6px; }
|
| 323 |
+
#chat::-webkit-scrollbar-thumb { background: var(--outline-variant); border-radius: 3px; }
|
| 324 |
+
#chat::-webkit-scrollbar-track { background: transparent; }
|
| 325 |
+
|
| 326 |
+
.msg {
|
| 327 |
+
padding: 12px 16px;
|
| 328 |
+
border-radius: 12px;
|
| 329 |
+
font-size: 14px;
|
| 330 |
+
line-height: 1.6;
|
| 331 |
+
max-width: 88%;
|
| 332 |
+
animation: fadeIn 0.3s ease;
|
| 333 |
+
}
|
| 334 |
+
@keyframes fadeIn { from { opacity: 0; transform: translateY(8px); } to { opacity: 1; transform: translateY(0); } }
|
| 335 |
+
.msg-user {
|
| 336 |
+
align-self: flex-end;
|
| 337 |
+
background: var(--accent-dim);
|
| 338 |
+
border: 1px solid var(--glass-border);
|
| 339 |
+
color: var(--text);
|
| 340 |
+
border-top-right-radius: 4px;
|
| 341 |
+
}
|
| 342 |
+
.msg-patient {
|
| 343 |
+
align-self: flex-start;
|
| 344 |
+
background: var(--surface-high);
|
| 345 |
+
border: 1px solid rgba(255,255,255,0.06);
|
| 346 |
+
color: var(--text);
|
| 347 |
+
border-top-left-radius: 4px;
|
| 348 |
+
}
|
| 349 |
+
.msg-role {
|
| 350 |
+
font-size: 10px;
|
| 351 |
+
text-transform: uppercase;
|
| 352 |
+
letter-spacing: 0.1em;
|
| 353 |
+
color: var(--text-dim);
|
| 354 |
+
margin-bottom: 4px;
|
| 355 |
+
font-weight: 600;
|
| 356 |
+
}
|
| 357 |
+
.msg-patient .msg-role { color: var(--accent); }
|
| 358 |
+
.no-audio { cursor: help; font-size: 1.1em; }
|
| 359 |
+
|
| 360 |
+
.feedback {
|
| 361 |
+
border: 1px solid var(--glass-border);
|
| 362 |
+
border-radius: 16px;
|
| 363 |
+
background: var(--glass);
|
| 364 |
+
backdrop-filter: blur(40px);
|
| 365 |
+
overflow: hidden;
|
| 366 |
+
}
|
| 367 |
+
.feedback-header {
|
| 368 |
+
padding: 14px 18px;
|
| 369 |
+
cursor: pointer;
|
| 370 |
+
display: flex;
|
| 371 |
+
align-items: center;
|
| 372 |
+
justify-content: space-between;
|
| 373 |
+
border-bottom: 1px solid var(--glass-border);
|
| 374 |
+
}
|
| 375 |
+
.feedback-header h3 {
|
| 376 |
+
font-family: var(--serif);
|
| 377 |
+
font-size: 18px;
|
| 378 |
+
font-weight: 400;
|
| 379 |
+
}
|
| 380 |
+
.feedback-badge {
|
| 381 |
+
font-size: 10px;
|
| 382 |
+
text-transform: uppercase;
|
| 383 |
+
letter-spacing: 0.1em;
|
| 384 |
+
padding: 2px 10px;
|
| 385 |
+
border-radius: 20px;
|
| 386 |
+
border: 1px solid var(--glass-border);
|
| 387 |
+
color: var(--text-dim);
|
| 388 |
+
font-weight: 600;
|
| 389 |
+
}
|
| 390 |
+
.feedback-header .arrow {
|
| 391 |
+
transition: transform 0.3s;
|
| 392 |
+
color: var(--text-dim);
|
| 393 |
+
font-size: 14px;
|
| 394 |
+
}
|
| 395 |
+
.feedback.open .feedback-header .arrow { transform: rotate(180deg); }
|
| 396 |
+
.feedback-body {
|
| 397 |
+
padding: 18px;
|
| 398 |
+
transition: max-height 0.3s ease, opacity 0.25s ease;
|
| 399 |
+
max-height: 500px;
|
| 400 |
+
opacity: 1;
|
| 401 |
+
overflow-y: auto;
|
| 402 |
+
}
|
| 403 |
+
.feedback:not(.open) .feedback-body {
|
| 404 |
+
max-height: 0;
|
| 405 |
+
padding: 0 18px;
|
| 406 |
+
opacity: 0;
|
| 407 |
+
}
|
| 408 |
+
.feedback-intro {
|
| 409 |
+
font-size: 13px;
|
| 410 |
+
color: var(--text-dim);
|
| 411 |
+
line-height: 1.6;
|
| 412 |
+
margin-bottom: 14px;
|
| 413 |
+
white-space: pre-wrap;
|
| 414 |
+
}
|
| 415 |
+
.feedback-empty {
|
| 416 |
+
font-size: 13px;
|
| 417 |
+
color: var(--text-dim);
|
| 418 |
+
font-style: italic;
|
| 419 |
+
}
|
| 420 |
+
.feedback-table {
|
| 421 |
+
width: 100%;
|
| 422 |
+
border-collapse: collapse;
|
| 423 |
+
font-size: 12px;
|
| 424 |
+
}
|
| 425 |
+
.feedback-table th {
|
| 426 |
+
text-align: left;
|
| 427 |
+
padding: 8px 10px;
|
| 428 |
+
font-size: 10px;
|
| 429 |
+
text-transform: uppercase;
|
| 430 |
+
letter-spacing: 0.1em;
|
| 431 |
+
color: var(--accent);
|
| 432 |
+
border-bottom: 1px solid var(--glass-border);
|
| 433 |
+
font-weight: 600;
|
| 434 |
+
}
|
| 435 |
+
.feedback-table td {
|
| 436 |
+
padding: 8px 10px;
|
| 437 |
+
border-bottom: 1px solid var(--glass-border);
|
| 438 |
+
color: var(--text);
|
| 439 |
+
line-height: 1.5;
|
| 440 |
+
font-size: 12px;
|
| 441 |
+
}
|
| 442 |
+
|
| 443 |
+
.feedback-section { margin-bottom: 14px; }
|
| 444 |
+
.feedback-section-title {
|
| 445 |
+
font-family: var(--serif);
|
| 446 |
+
font-size: 14px;
|
| 447 |
+
font-weight: 600;
|
| 448 |
+
margin-bottom: 6px;
|
| 449 |
+
color: var(--accent);
|
| 450 |
+
}
|
| 451 |
+
.feedback-list {
|
| 452 |
+
list-style: none;
|
| 453 |
+
margin: 0;
|
| 454 |
+
padding: 0;
|
| 455 |
+
}
|
| 456 |
+
.feedback-list li {
|
| 457 |
+
font-size: 12px;
|
| 458 |
+
line-height: 1.6;
|
| 459 |
+
padding: 2px 0;
|
| 460 |
+
color: var(--text);
|
| 461 |
+
}
|
| 462 |
+
.feedback-list li::before {
|
| 463 |
+
content: "\2022";
|
| 464 |
+
color: var(--accent);
|
| 465 |
+
margin-right: 6px;
|
| 466 |
+
}
|
| 467 |
+
|
| 468 |
+
.bilan-row {
|
| 469 |
+
display: flex;
|
| 470 |
+
align-items: center;
|
| 471 |
+
gap: 10px;
|
| 472 |
+
margin-bottom: 8px;
|
| 473 |
+
}
|
| 474 |
+
.bilan-label {
|
| 475 |
+
font-size: 11px;
|
| 476 |
+
text-transform: uppercase;
|
| 477 |
+
letter-spacing: 0.05em;
|
| 478 |
+
color: var(--text-dim);
|
| 479 |
+
width: 140px;
|
| 480 |
+
flex-shrink: 0;
|
| 481 |
+
}
|
| 482 |
+
.bilan-bar {
|
| 483 |
+
flex: 1;
|
| 484 |
+
height: 8px;
|
| 485 |
+
background: rgba(255,255,255,0.08);
|
| 486 |
+
border-radius: 4px;
|
| 487 |
+
overflow: hidden;
|
| 488 |
+
}
|
| 489 |
+
.bilan-fill {
|
| 490 |
+
height: 100%;
|
| 491 |
+
background: var(--accent);
|
| 492 |
+
border-radius: 4px;
|
| 493 |
+
transition: width 0.5s ease;
|
| 494 |
+
}
|
| 495 |
+
.bilan-score {
|
| 496 |
+
font-size: 11px;
|
| 497 |
+
color: var(--text-dim);
|
| 498 |
+
width: 30px;
|
| 499 |
+
text-align: right;
|
| 500 |
+
}
|
| 501 |
+
|
| 502 |
+
.footer {
|
| 503 |
+
text-align: center;
|
| 504 |
+
font-size: 10px;
|
| 505 |
+
color: var(--text-dim);
|
| 506 |
+
padding-top: 20px;
|
| 507 |
+
margin-top: 28px;
|
| 508 |
+
border-top: 1px solid rgba(255,255,255,0.04);
|
| 509 |
+
flex-shrink: 0;
|
| 510 |
+
}
|
| 511 |
+
|
| 512 |
+
#audio-player { display: none; }
|
| 513 |
+
|
| 514 |
+
.dots::after {
|
| 515 |
+
content: '';
|
| 516 |
+
animation: dots 1.5s steps(4, end) infinite;
|
| 517 |
+
}
|
| 518 |
+
@keyframes dots {
|
| 519 |
+
0% { content: ''; }
|
| 520 |
+
25% { content: '.'; }
|
| 521 |
+
50% { content: '..'; }
|
| 522 |
+
75% { content: '...'; }
|
| 523 |
+
}
|
| 524 |
+
|
| 525 |
+
@media (max-width: 860px) {
|
| 526 |
+
.main-grid {
|
| 527 |
+
grid-template-columns: 1fr;
|
| 528 |
+
}
|
| 529 |
+
.app { padding: 20px 16px; }
|
| 530 |
+
.desc h2 { font-size: 38px; }
|
| 531 |
+
#chat { max-height: 40vh; }
|
| 532 |
+
}
|
| 533 |
+
</style>
|
| 534 |
+
</head>
|
| 535 |
+
<body>
|
| 536 |
+
|
| 537 |
+
<div class="app">
|
| 538 |
+
<div class="header">
|
| 539 |
+
<div class="header-icon">
|
| 540 |
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
|
| 541 |
+
<path d="M4.8 2.3A.3.3 0 1 0 5 2H4a2 2 0 0 0-2 2v5a6 6 0 0 0 6 6v0a6 6 0 0 0 6-6V4a2 2 0 0 0-2-2h-1a.2.2 0 1 0 .3.3"/>
|
| 542 |
+
<path d="M8 15v1a6 6 0 0 0 6 6v0a6 6 0 0 0 6-6v-4"/>
|
| 543 |
+
<circle cx="20" cy="10" r="2"/>
|
| 544 |
+
</svg>
|
| 545 |
+
</div>
|
| 546 |
+
<div>
|
| 547 |
+
<div class="header-title">Patient Virtuel · E2B</div>
|
| 548 |
+
<div class="header-sub">Gemma 4 E2B sur ZeroGPU · Tiny Titan</div>
|
| 549 |
+
</div>
|
| 550 |
+
</div>
|
| 551 |
+
|
| 552 |
+
<div class="main-grid">
|
| 553 |
+
<div class="left-col">
|
| 554 |
+
<div class="desc">
|
| 555 |
+
<h2>Pratiquez votre <em>Français Dentaire</em></h2>
|
| 556 |
+
<p>
|
| 557 |
+
Un patient virtuel qui vous répond à voix haute. Entraînez-vous aux conversations
|
| 558 |
+
de soins dentaires en français, puis recevez un bilan structuré de votre vocabulaire
|
| 559 |
+
et de votre grammaire.
|
| 560 |
+
</p>
|
| 561 |
+
</div>
|
| 562 |
+
|
| 563 |
+
<div class="mic-area">
|
| 564 |
+
<button class="mic-btn" id="mic-btn" onclick="toggleRecording()">
|
| 565 |
+
<svg id="mic-icon" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
|
| 566 |
+
<path d="M12 2a3 3 0 0 0-3 3v7a3 3 0 0 0 6 0V5a3 3 0 0 0-3-3Z"/>
|
| 567 |
+
<path d="M19 10v2a7 7 0 0 1-14 0v-2"/>
|
| 568 |
+
<line x1="12" x2="12" y1="19" y2="22"/>
|
| 569 |
+
</svg>
|
| 570 |
+
<svg id="stop-icon" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="currentColor" style="display:none">
|
| 571 |
+
<rect x="6" y="6" width="12" height="12" rx="2"/>
|
| 572 |
+
</svg>
|
| 573 |
+
</button>
|
| 574 |
+
<div class="mic-label" id="mic-label">Appuyez pour parler</div>
|
| 575 |
+
</div>
|
| 576 |
+
|
| 577 |
+
<div class="howto">
|
| 578 |
+
<div class="howto-title">Comment utiliser</div>
|
| 579 |
+
<ol>
|
| 580 |
+
<li><span class="num">1</span><span>Appuyez sur le bouton micro pour commencer à enregistrer.</span></li>
|
| 581 |
+
<li><span class="num">2</span><span>Parlez en français — saluez le patient, posez des questions, expliquez les soins.</span></li>
|
| 582 |
+
<li><span class="num">3</span><span>Appuyez à nouveau pour arrêter — le patient vous répond à voix haute.</span></li>
|
| 583 |
+
<li><span class="num">4</span><span>Dites <strong>« Fin de la séance »</strong> ou appuyez sur <strong>Terminer</strong> pour obtenir votre bilan.</span></li>
|
| 584 |
+
</ol>
|
| 585 |
+
</div>
|
| 586 |
+
|
| 587 |
+
<div class="controls">
|
| 588 |
+
<button class="btn btn-stop" id="btn-end" onclick="endSession()">Terminer la séance</button>
|
| 589 |
+
<button class="btn" id="btn-reset" onclick="resetSession()">Nouvelle séance</button>
|
| 590 |
+
</div>
|
| 591 |
+
|
| 592 |
+
<div id="status"></div>
|
| 593 |
+
</div>
|
| 594 |
+
|
| 595 |
+
<div class="right-col">
|
| 596 |
+
<div class="chat-wrap">
|
| 597 |
+
<div class="chat-header">
|
| 598 |
+
<span class="chat-header-label">Transcription</span>
|
| 599 |
+
<span class="chat-header-live">En direct</span>
|
| 600 |
+
</div>
|
| 601 |
+
<div id="chat"></div>
|
| 602 |
+
</div>
|
| 603 |
+
|
| 604 |
+
<div class="feedback open" id="feedback">
|
| 605 |
+
<div class="feedback-header" onclick="toggleFeedback()">
|
| 606 |
+
<div style="display:flex;align-items:center;gap:10px">
|
| 607 |
+
<h3>Bilan de la séance</h3>
|
| 608 |
+
<span class="feedback-badge" id="feedback-count">0 notes</span>
|
| 609 |
+
</div>
|
| 610 |
+
<span class="arrow">▾</span>
|
| 611 |
+
</div>
|
| 612 |
+
<div class="feedback-body">
|
| 613 |
+
<div class="feedback-intro" id="feedback-intro"></div>
|
| 614 |
+
<div class="feedback-empty" id="feedback-empty">Le bilan se construira au fil de la conversation.</div>
|
| 615 |
+
|
| 616 |
+
<div id="feedback-points-forts" class="feedback-section" style="display:none">
|
| 617 |
+
<div class="feedback-section-title">Points forts</div>
|
| 618 |
+
<ul class="feedback-list" id="feedback-points-list"></ul>
|
| 619 |
+
</div>
|
| 620 |
+
|
| 621 |
+
<table class="feedback-table" id="feedback-table" style="display:none">
|
| 622 |
+
<thead><tr><th>Citation</th><th>Correction</th><th>Pourquoi</th></tr></thead>
|
| 623 |
+
<tbody id="feedback-tbody"></tbody>
|
| 624 |
+
</table>
|
| 625 |
+
|
| 626 |
+
<div id="feedback-vocab" class="feedback-section" style="display:none">
|
| 627 |
+
<div class="feedback-section-title">Vocabulaire dentaire utile</div>
|
| 628 |
+
<ul class="feedback-list" id="feedback-vocab-list"></ul>
|
| 629 |
+
</div>
|
| 630 |
+
|
| 631 |
+
<div id="feedback-priorite" class="feedback-section" style="display:none">
|
| 632 |
+
<div class="feedback-section-title">Priorité pour la prochaine séance</div>
|
| 633 |
+
<ul class="feedback-list" id="feedback-priorite-list"></ul>
|
| 634 |
+
</div>
|
| 635 |
+
|
| 636 |
+
<div id="feedback-bilan" class="feedback-section" style="display:none">
|
| 637 |
+
<div class="feedback-section-title">Bilan</div>
|
| 638 |
+
<div id="feedback-bilan-rows"></div>
|
| 639 |
+
</div>
|
| 640 |
+
</div>
|
| 641 |
+
</div>
|
| 642 |
+
</div>
|
| 643 |
+
</div>
|
| 644 |
+
|
| 645 |
+
<div class="footer">
|
| 646 |
+
Modèle: Gemma 4 E2B (ZeroGPU) · STT intégrée · TTS: piper-tts<br>
|
| 647 |
+
<a href="https://huggingface.co/spaces/build-small-hackathon/patient-virtuel-dentiste" style="color:#ff5719;text-decoration:none;">← Version principale (Gemma 4 26B)</a>
|
| 648 |
+
</div>
|
| 649 |
+
</div>
|
| 650 |
+
|
| 651 |
+
<audio id="audio-player" autoplay></audio>
|
| 652 |
+
|
| 653 |
+
<script type="module">
|
| 654 |
+
import { Client } from "https://cdn.jsdelivr.net/npm/@gradio/client@2.2.1/dist/browser.js";
|
| 655 |
+
|
| 656 |
+
let client = null;
|
| 657 |
+
let sessionState = null;
|
| 658 |
+
let mediaRecorder = null;
|
| 659 |
+
let audioChunks = [];
|
| 660 |
+
let isRecording = false;
|
| 661 |
+
let isProcessing = false;
|
| 662 |
+
|
| 663 |
+
const micBtn = document.getElementById("mic-btn");
|
| 664 |
+
const micIcon = document.getElementById("mic-icon");
|
| 665 |
+
const stopIcon = document.getElementById("stop-icon");
|
| 666 |
+
const micLabel = document.getElementById("mic-label");
|
| 667 |
+
const statusEl = document.getElementById("status");
|
| 668 |
+
const chatEl = document.getElementById("chat");
|
| 669 |
+
const feedbackEl = document.getElementById("feedback");
|
| 670 |
+
const feedbackIntro = document.getElementById("feedback-intro");
|
| 671 |
+
const feedbackTbody = document.getElementById("feedback-tbody");
|
| 672 |
+
const feedbackTable = document.getElementById("feedback-table");
|
| 673 |
+
const feedbackEmpty = document.getElementById("feedback-empty");
|
| 674 |
+
const feedbackCount = document.getElementById("feedback-count");
|
| 675 |
+
const feedbackPointsForts = document.getElementById("feedback-points-forts");
|
| 676 |
+
const feedbackPointsList = document.getElementById("feedback-points-list");
|
| 677 |
+
const feedbackVocab = document.getElementById("feedback-vocab");
|
| 678 |
+
const feedbackVocabList = document.getElementById("feedback-vocab-list");
|
| 679 |
+
const feedbackPriorite = document.getElementById("feedback-priorite");
|
| 680 |
+
const feedbackPrioriteList = document.getElementById("feedback-priorite-list");
|
| 681 |
+
const feedbackBilan = document.getElementById("feedback-bilan");
|
| 682 |
+
const feedbackBilanRows = document.getElementById("feedback-bilan-rows");
|
| 683 |
+
const audioPlayer = document.getElementById("audio-player");
|
| 684 |
+
const btnEnd = document.getElementById("btn-end");
|
| 685 |
+
const btnReset = document.getElementById("btn-reset");
|
| 686 |
+
|
| 687 |
+
async function initClient() {
|
| 688 |
+
try {
|
| 689 |
+
client = await Client.connect(window.location.origin);
|
| 690 |
+
const result = await client.predict("/reset_session", {});
|
| 691 |
+
sessionState = result.data[0].state;
|
| 692 |
+
console.log("Connected to backend");
|
| 693 |
+
} catch (err) {
|
| 694 |
+
console.error("Failed to connect:", err);
|
| 695 |
+
setStatus("Failed to connect to server.");
|
| 696 |
+
}
|
| 697 |
+
}
|
| 698 |
+
|
| 699 |
+
async function toggleRecording() {
|
| 700 |
+
if (isProcessing) return;
|
| 701 |
+
if (isRecording) {
|
| 702 |
+
stopRecording();
|
| 703 |
+
} else {
|
| 704 |
+
startRecording();
|
| 705 |
+
}
|
| 706 |
+
}
|
| 707 |
+
|
| 708 |
+
async function startRecording() {
|
| 709 |
+
try {
|
| 710 |
+
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
|
| 711 |
+
audioChunks = [];
|
| 712 |
+
mediaRecorder = new MediaRecorder(stream, { mimeType: "audio/webm;codecs=opus" });
|
| 713 |
+
mediaRecorder.ondataavailable = (e) => { if (e.data.size > 0) audioChunks.push(e.data); };
|
| 714 |
+
mediaRecorder.onstop = () => {
|
| 715 |
+
stream.getTracks().forEach(t => t.stop());
|
| 716 |
+
const blob = new Blob(audioChunks, { type: "audio/webm" });
|
| 717 |
+
processAudio(blob);
|
| 718 |
+
};
|
| 719 |
+
mediaRecorder.start();
|
| 720 |
+
isRecording = true;
|
| 721 |
+
micBtn.classList.add("recording");
|
| 722 |
+
micIcon.style.display = "none";
|
| 723 |
+
stopIcon.style.display = "block";
|
| 724 |
+
micLabel.textContent = "Recording...";
|
| 725 |
+
micLabel.classList.add("active");
|
| 726 |
+
setStatus("");
|
| 727 |
+
} catch (err) {
|
| 728 |
+
setStatus("Microphone unavailable.");
|
| 729 |
+
}
|
| 730 |
+
}
|
| 731 |
+
|
| 732 |
+
function stopRecording() {
|
| 733 |
+
if (mediaRecorder && mediaRecorder.state === "recording") {
|
| 734 |
+
mediaRecorder.stop();
|
| 735 |
+
}
|
| 736 |
+
isRecording = false;
|
| 737 |
+
micBtn.classList.remove("recording");
|
| 738 |
+
micIcon.style.display = "block";
|
| 739 |
+
stopIcon.style.display = "none";
|
| 740 |
+
micLabel.textContent = "Tap to speak";
|
| 741 |
+
micLabel.classList.remove("active");
|
| 742 |
+
}
|
| 743 |
+
|
| 744 |
+
async function processAudio(blob) {
|
| 745 |
+
if (!client) return;
|
| 746 |
+
|
| 747 |
+
isProcessing = true;
|
| 748 |
+
micBtn.disabled = true;
|
| 749 |
+
btnEnd.disabled = true;
|
| 750 |
+
setStatus("Decoding audio...");
|
| 751 |
+
|
| 752 |
+
try {
|
| 753 |
+
const audioFile = new File([blob], "recording.webm", { type: blob.type || "audio/webm" });
|
| 754 |
+
setStatus("Patient thinking...");
|
| 755 |
+
const result = await client.predict("/process_turn", {
|
| 756 |
+
audio: audioFile,
|
| 757 |
+
state: sessionState,
|
| 758 |
+
});
|
| 759 |
+
|
| 760 |
+
const d = result.data[0];
|
| 761 |
+
sessionState = d.state;
|
| 762 |
+
|
| 763 |
+
renderChat(d.chat);
|
| 764 |
+
|
| 765 |
+
if (d.audio_url) {
|
| 766 |
+
setStatus("Synthesizing speech...");
|
| 767 |
+
await playAudio(d.audio_url);
|
| 768 |
+
} else {
|
| 769 |
+
const bubbles = chatEl.querySelectorAll(".msg-patient");
|
| 770 |
+
const last = bubbles[bubbles.length - 1];
|
| 771 |
+
if (last) last.innerHTML += ' <span class="no-audio">🔇</span>';
|
| 772 |
+
}
|
| 773 |
+
|
| 774 |
+
if (d.feedback_open) {
|
| 775 |
+
showFeedback(d.feedback_intro, d.feedback_table, d.feedback_points_forts, d.feedback_vocabulaire, d.feedback_priorite, d.feedback_bilan);
|
| 776 |
+
}
|
| 777 |
+
|
| 778 |
+
setStatus(d.status || "");
|
| 779 |
+
} catch (err) {
|
| 780 |
+
console.error("Process error:", err);
|
| 781 |
+
setStatus("Error. Please retry.");
|
| 782 |
+
} finally {
|
| 783 |
+
isProcessing = false;
|
| 784 |
+
micBtn.disabled = false;
|
| 785 |
+
btnEnd.disabled = false;
|
| 786 |
+
}
|
| 787 |
+
}
|
| 788 |
+
|
| 789 |
+
async function endSession() {
|
| 790 |
+
if (!client || isProcessing) return;
|
| 791 |
+
isProcessing = true;
|
| 792 |
+
micBtn.disabled = true;
|
| 793 |
+
btnEnd.disabled = true;
|
| 794 |
+
setStatus("Generating recap...");
|
| 795 |
+
|
| 796 |
+
try {
|
| 797 |
+
const result = await client.predict("/end_session", { state: sessionState });
|
| 798 |
+
const d = result.data[0];
|
| 799 |
+
sessionState = d.state;
|
| 800 |
+
|
| 801 |
+
renderChat(d.chat);
|
| 802 |
+
if (d.audio_url) {
|
| 803 |
+
setStatus("Synthesizing speech...");
|
| 804 |
+
await playAudio(d.audio_url);
|
| 805 |
+
}
|
| 806 |
+
if (d.feedback_open) {
|
| 807 |
+
showFeedback(d.feedback_intro, d.feedback_table, d.feedback_points_forts, d.feedback_vocabulaire, d.feedback_priorite, d.feedback_bilan);
|
| 808 |
+
}
|
| 809 |
+
setStatus("");
|
| 810 |
+
} catch (err) {
|
| 811 |
+
console.error("End session error:", err);
|
| 812 |
+
setStatus("Error generating recap.");
|
| 813 |
+
} finally {
|
| 814 |
+
isProcessing = false;
|
| 815 |
+
micBtn.disabled = false;
|
| 816 |
+
btnEnd.disabled = false;
|
| 817 |
+
}
|
| 818 |
+
}
|
| 819 |
+
|
| 820 |
+
async function resetSession() {
|
| 821 |
+
if (!client || isProcessing) return;
|
| 822 |
+
try {
|
| 823 |
+
const result = await client.predict("/reset_session", {});
|
| 824 |
+
sessionState = result.data[0].state;
|
| 825 |
+
chatEl.innerHTML = "";
|
| 826 |
+
feedbackEl.classList.add("open");
|
| 827 |
+
feedbackIntro.textContent = "";
|
| 828 |
+
feedbackTbody.innerHTML = "";
|
| 829 |
+
feedbackTable.style.display = "none";
|
| 830 |
+
feedbackEmpty.style.display = "";
|
| 831 |
+
feedbackPointsForts.style.display = "none";
|
| 832 |
+
feedbackPointsList.innerHTML = "";
|
| 833 |
+
feedbackVocab.style.display = "none";
|
| 834 |
+
feedbackVocabList.innerHTML = "";
|
| 835 |
+
feedbackPriorite.style.display = "none";
|
| 836 |
+
feedbackPrioriteList.innerHTML = "";
|
| 837 |
+
feedbackBilan.style.display = "none";
|
| 838 |
+
feedbackBilanRows.innerHTML = "";
|
| 839 |
+
feedbackCount.textContent = "0 notes";
|
| 840 |
+
setStatus("");
|
| 841 |
+
audioPlayer.pause();
|
| 842 |
+
audioPlayer.src = "";
|
| 843 |
+
} catch (err) {
|
| 844 |
+
console.error("Reset error:", err);
|
| 845 |
+
}
|
| 846 |
+
}
|
| 847 |
+
|
| 848 |
+
function renderChat(messages) {
|
| 849 |
+
chatEl.innerHTML = "";
|
| 850 |
+
for (const msg of messages) {
|
| 851 |
+
if (msg.role === "system") continue;
|
| 852 |
+
const div = document.createElement("div");
|
| 853 |
+
div.className = `msg ${msg.role === "user" ? "msg-user" : "msg-patient"}`;
|
| 854 |
+
const roleLabel = msg.role === "user" ? "Vous" : "Patient";
|
| 855 |
+
div.innerHTML = `<div class="msg-role">${roleLabel}</div>${escapeHtml(msg.content)}`;
|
| 856 |
+
chatEl.appendChild(div);
|
| 857 |
+
}
|
| 858 |
+
chatEl.scrollTop = chatEl.scrollHeight;
|
| 859 |
+
}
|
| 860 |
+
|
| 861 |
+
function showFeedback(intro, table, pointsForts, vocabulaire, priorite, bilan) {
|
| 862 |
+
feedbackIntro.textContent = intro;
|
| 863 |
+
feedbackTbody.innerHTML = "";
|
| 864 |
+
|
| 865 |
+
if (pointsForts && pointsForts.length) {
|
| 866 |
+
feedbackPointsForts.style.display = "";
|
| 867 |
+
feedbackPointsList.innerHTML = "";
|
| 868 |
+
for (const item of pointsForts) {
|
| 869 |
+
const li = document.createElement("li");
|
| 870 |
+
li.textContent = item;
|
| 871 |
+
feedbackPointsList.appendChild(li);
|
| 872 |
+
}
|
| 873 |
+
} else {
|
| 874 |
+
feedbackPointsForts.style.display = "none";
|
| 875 |
+
}
|
| 876 |
+
|
| 877 |
+
if (table && table.length) {
|
| 878 |
+
feedbackTable.style.display = "";
|
| 879 |
+
feedbackEmpty.style.display = "none";
|
| 880 |
+
feedbackCount.textContent = table.length + (table.length > 1 ? " notes" : " note");
|
| 881 |
+
for (const row of table) {
|
| 882 |
+
const tr = document.createElement("tr");
|
| 883 |
+
const cells = Array.isArray(row) ? row : [row["citation"], row["correction"], row["pourquoi"]];
|
| 884 |
+
for (const cell of cells) {
|
| 885 |
+
const td = document.createElement("td");
|
| 886 |
+
td.textContent = cell || "";
|
| 887 |
+
tr.appendChild(td);
|
| 888 |
+
}
|
| 889 |
+
feedbackTbody.appendChild(tr);
|
| 890 |
+
}
|
| 891 |
+
} else {
|
| 892 |
+
feedbackTable.style.display = "none";
|
| 893 |
+
}
|
| 894 |
+
|
| 895 |
+
if (vocabulaire && vocabulaire.length) {
|
| 896 |
+
feedbackVocab.style.display = "";
|
| 897 |
+
feedbackVocabList.innerHTML = "";
|
| 898 |
+
for (const item of vocabulaire) {
|
| 899 |
+
const li = document.createElement("li");
|
| 900 |
+
li.textContent = item;
|
| 901 |
+
feedbackVocabList.appendChild(li);
|
| 902 |
+
}
|
| 903 |
+
} else {
|
| 904 |
+
feedbackVocab.style.display = "none";
|
| 905 |
+
}
|
| 906 |
+
|
| 907 |
+
if (priorite && priorite.length) {
|
| 908 |
+
feedbackPriorite.style.display = "";
|
| 909 |
+
feedbackPrioriteList.innerHTML = "";
|
| 910 |
+
for (const item of priorite) {
|
| 911 |
+
const li = document.createElement("li");
|
| 912 |
+
li.textContent = item;
|
| 913 |
+
feedbackPrioriteList.appendChild(li);
|
| 914 |
+
}
|
| 915 |
+
} else {
|
| 916 |
+
feedbackPriorite.style.display = "none";
|
| 917 |
+
}
|
| 918 |
+
|
| 919 |
+
if (bilan && Object.keys(bilan).length) {
|
| 920 |
+
feedbackBilan.style.display = "";
|
| 921 |
+
feedbackBilanRows.innerHTML = "";
|
| 922 |
+
const labels = {
|
| 923 |
+
grammaire: "Grammaire",
|
| 924 |
+
fluidite: "Fluidite",
|
| 925 |
+
vocabulaire_dentaire: "Vocabulaire dentaire",
|
| 926 |
+
};
|
| 927 |
+
for (const [key, label] of Object.entries(labels)) {
|
| 928 |
+
const val = bilan[key];
|
| 929 |
+
if (val == null) continue;
|
| 930 |
+
const pct = Math.max(10, (val / 5) * 100);
|
| 931 |
+
const row = document.createElement("div");
|
| 932 |
+
row.className = "bilan-row";
|
| 933 |
+
row.innerHTML =
|
| 934 |
+
`<span class="bilan-label">${label}</span>` +
|
| 935 |
+
`<div class="bilan-bar"><div class="bilan-fill" style="width:${pct}%"></div></div>` +
|
| 936 |
+
`<span class="bilan-score">${val}/5</span>`;
|
| 937 |
+
feedbackBilanRows.appendChild(row);
|
| 938 |
+
}
|
| 939 |
+
} else {
|
| 940 |
+
feedbackBilan.style.display = "none";
|
| 941 |
+
}
|
| 942 |
+
|
| 943 |
+
if (!table || !table.length) {
|
| 944 |
+
feedbackEmpty.style.display = "none";
|
| 945 |
+
feedbackCount.textContent = "";
|
| 946 |
+
}
|
| 947 |
+
|
| 948 |
+
feedbackEl.classList.add("open");
|
| 949 |
+
}
|
| 950 |
+
|
| 951 |
+
function toggleFeedback() {
|
| 952 |
+
feedbackEl.classList.toggle("open");
|
| 953 |
+
}
|
| 954 |
+
|
| 955 |
+
function playAudio(url) {
|
| 956 |
+
return new Promise((resolve) => {
|
| 957 |
+
audioPlayer.onended = resolve;
|
| 958 |
+
audioPlayer.onerror = resolve;
|
| 959 |
+
audioPlayer.src = url;
|
| 960 |
+
audioPlayer.play().catch(resolve);
|
| 961 |
+
});
|
| 962 |
+
}
|
| 963 |
+
|
| 964 |
+
function setStatus(text) {
|
| 965 |
+
statusEl.textContent = text;
|
| 966 |
+
}
|
| 967 |
+
|
| 968 |
+
function escapeHtml(text) {
|
| 969 |
+
const div = document.createElement("div");
|
| 970 |
+
div.textContent = text;
|
| 971 |
+
return div.innerHTML;
|
| 972 |
+
}
|
| 973 |
+
|
| 974 |
+
initClient();
|
| 975 |
+
|
| 976 |
+
window.toggleRecording = toggleRecording;
|
| 977 |
+
window.endSession = endSession;
|
| 978 |
+
window.resetSession = resetSession;
|
| 979 |
+
window.toggleFeedback = toggleFeedback;
|
| 980 |
+
</script>
|
| 981 |
+
</body>
|
| 982 |
+
</html>
|
patient-virtuel-e2b/parse_feedback.py
ADDED
|
@@ -0,0 +1,130 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import re
|
| 2 |
+
|
| 3 |
+
SECTION_HEADERS = [
|
| 4 |
+
"Points forts",
|
| 5 |
+
"À corriger",
|
| 6 |
+
"Vocabulaire dentaire utile",
|
| 7 |
+
"Priorité pour la prochaine séance",
|
| 8 |
+
"Bilan",
|
| 9 |
+
]
|
| 10 |
+
|
| 11 |
+
NEXT_HEADER_PATTERN = re.compile(
|
| 12 |
+
rf"(?:^|\n)\s*(?:{'|'.join(re.escape(h) for h in SECTION_HEADERS)})\s*:\s*",
|
| 13 |
+
re.IGNORECASE | re.MULTILINE,
|
| 14 |
+
)
|
| 15 |
+
|
| 16 |
+
CITATION_RE = re.compile(
|
| 17 |
+
r'Citation:\s*["\u00ab](.+?)["\u00bb]\s*Correction:\s*(.+?)\s*Pourquoi:\s*(.+?)(?=\s*(?:Citation:|[A-Z\u00c0-\u017f][^\n:]*:)|\s*$)',
|
| 18 |
+
re.DOTALL | re.IGNORECASE,
|
| 19 |
+
)
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def _extract_section(text: str, header: str) -> str:
|
| 23 |
+
"""Return content between *header*: and the next section header or EOF."""
|
| 24 |
+
m = re.search(
|
| 25 |
+
rf"(?:^|\n)\s*{re.escape(header)}\s*:\s*",
|
| 26 |
+
text, re.IGNORECASE | re.MULTILINE,
|
| 27 |
+
)
|
| 28 |
+
if not m:
|
| 29 |
+
return ""
|
| 30 |
+
start = m.end()
|
| 31 |
+
# Find next section header
|
| 32 |
+
nm = NEXT_HEADER_PATTERN.search(text, start)
|
| 33 |
+
end = nm.start() if nm else len(text)
|
| 34 |
+
return text[start:end].strip()
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
def _parse_bullets(text: str) -> list[str]:
|
| 38 |
+
"""Extract lines prefixed with - or • from a section block."""
|
| 39 |
+
if not text:
|
| 40 |
+
return []
|
| 41 |
+
items = []
|
| 42 |
+
for line in text.split("\n"):
|
| 43 |
+
line = line.strip()
|
| 44 |
+
if re.match(r"^[-•]\s", line):
|
| 45 |
+
items.append(re.sub(r"^[-•]\s*", "", line).strip())
|
| 46 |
+
return items
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
def _parse_bilan(text: str) -> dict:
|
| 50 |
+
"""Extract Bilan scores as {key: int}."""
|
| 51 |
+
section = _extract_section(text, "Bilan")
|
| 52 |
+
if not section:
|
| 53 |
+
return {}
|
| 54 |
+
scores = {}
|
| 55 |
+
for line in section.split("\n"):
|
| 56 |
+
m = re.match(
|
| 57 |
+
r"[-•]?\s*(Grammaire|Fluidité|Vocabulaire dentaire|Communication clinique)\s*:\s*(\d+)\s*/\s*\d+",
|
| 58 |
+
line.strip(), re.IGNORECASE,
|
| 59 |
+
)
|
| 60 |
+
if m:
|
| 61 |
+
key = m.group(1).lower().replace(" ", "_")
|
| 62 |
+
scores[key] = int(m.group(2))
|
| 63 |
+
return scores
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
def parse_feedback(text: str) -> dict:
|
| 67 |
+
"""Parse full feedback into a structured dict.
|
| 68 |
+
|
| 69 |
+
Returns:
|
| 70 |
+
intro -- spoken part (before delimiter)
|
| 71 |
+
points_forts -- list[str]
|
| 72 |
+
erreurs -- list[dict] with keys: citation, correction, pourquoi
|
| 73 |
+
vocabulaire -- list[str]
|
| 74 |
+
priorite -- list[str]
|
| 75 |
+
bilan -- dict {grammaire: int, fluidite: int, ...}
|
| 76 |
+
"""
|
| 77 |
+
result = {
|
| 78 |
+
"intro": "",
|
| 79 |
+
"points_forts": [],
|
| 80 |
+
"erreurs": [],
|
| 81 |
+
"vocabulaire": [],
|
| 82 |
+
"priorite": [],
|
| 83 |
+
"bilan": {},
|
| 84 |
+
}
|
| 85 |
+
|
| 86 |
+
# Split on delimiter
|
| 87 |
+
if "---" in text:
|
| 88 |
+
parts = text.split("---", 1)
|
| 89 |
+
result["intro"] = parts[0].strip()
|
| 90 |
+
rest = parts[1].strip()
|
| 91 |
+
elif "Points forts:" in text:
|
| 92 |
+
parts = text.split("Points forts:", 1)
|
| 93 |
+
result["intro"] = parts[0].strip()
|
| 94 |
+
rest = "Points forts: " + parts[1].strip()
|
| 95 |
+
else:
|
| 96 |
+
result["intro"] = text.strip()
|
| 97 |
+
return result
|
| 98 |
+
|
| 99 |
+
result["points_forts"] = _parse_bullets(_extract_section(rest, "Points forts"))
|
| 100 |
+
|
| 101 |
+
corriger = _extract_section(rest, "À corriger")
|
| 102 |
+
for m in CITATION_RE.finditer(corriger):
|
| 103 |
+
result["erreurs"].append({
|
| 104 |
+
"citation": m.group(1).strip(),
|
| 105 |
+
"correction": m.group(2).strip(),
|
| 106 |
+
"pourquoi": m.group(3).strip(),
|
| 107 |
+
})
|
| 108 |
+
|
| 109 |
+
result["vocabulaire"] = _parse_bullets(_extract_section(rest, "Vocabulaire dentaire utile"))
|
| 110 |
+
result["priorite"] = _parse_bullets(_extract_section(rest, "Priorité pour la prochaine séance"))
|
| 111 |
+
result["bilan"] = _parse_bilan(rest)
|
| 112 |
+
|
| 113 |
+
return result
|
| 114 |
+
|
| 115 |
+
|
| 116 |
+
def render_feedback_table(entries: list[dict]) -> list[list[str]]:
|
| 117 |
+
"""Convert error entries to table rows for the frontend."""
|
| 118 |
+
return [[e["citation"], e["correction"], e["pourquoi"]] for e in entries]
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
def strip_markdown(text: str) -> str:
|
| 122 |
+
text = re.sub(r"\*\*(.+?)\*\*", r"\1", text)
|
| 123 |
+
text = re.sub(r"\*(.+?)\*", r"\1", text)
|
| 124 |
+
text = re.sub(r"`(.+?)`", r"\1", text)
|
| 125 |
+
text = re.sub(r"#{1,6}\s*", "", text)
|
| 126 |
+
text = re.sub(r"^[-*]\s", "", text, flags=re.MULTILINE)
|
| 127 |
+
text = re.sub(r"!?\[.*?\]\(.*?\)", "", text)
|
| 128 |
+
text = text.replace("\u258c", "")
|
| 129 |
+
text = text.replace("\U0001f916", "")
|
| 130 |
+
return text.strip()
|
patient-virtuel-e2b/prompts_e2b.py
ADDED
|
@@ -0,0 +1,47 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
SYSTEM_PROMPT = """You are a virtual dental patient in a Swiss dental clinic. The user is a dental hygienist practicing professional French.
|
| 2 |
+
|
| 3 |
+
Stay in character at all times. Speak only French. Use plain text. Keep responses short (1-2 sentences).
|
| 4 |
+
|
| 5 |
+
PATIENT PROFILE
|
| 6 |
+
Create one hidden profile per session: age, dental habits, one mild concern. Be consistent. You have moderate dental anxiety.
|
| 7 |
+
|
| 8 |
+
APPOINTMENT
|
| 9 |
+
React naturally to the hygienist's questions and instructions across these phases:
|
| 10 |
+
|
| 11 |
+
1. EXAM (bilan): Answer questions about your health and oral care. Mention sensitivity on your left molars.
|
| 12 |
+
|
| 13 |
+
2. TREATMENT (soins): Respond to vibrations, water spray, and instruments. If something hurts, say so briefly.
|
| 14 |
+
|
| 15 |
+
3. ADVICE (conseils): Ask 1-2 questions about home care. Say you forget to floss sometimes.
|
| 16 |
+
|
| 17 |
+
TUTOR MODE
|
| 18 |
+
When the user says "fin de la seance" or "on a termine", switch to French coach mode.
|
| 19 |
+
|
| 20 |
+
First say 2 short sentences: one encouraging, one priority for improvement.
|
| 21 |
+
Then output --- on its own line.
|
| 22 |
+
Then give a short recap:
|
| 23 |
+
|
| 24 |
+
Points forts:
|
| 25 |
+
- ...
|
| 26 |
+
|
| 27 |
+
A corriger:
|
| 28 |
+
Citation: "..."
|
| 29 |
+
Correction: ...
|
| 30 |
+
Pourquoi: ...
|
| 31 |
+
|
| 32 |
+
Vocabulaire dentaire utile:
|
| 33 |
+
- ...
|
| 34 |
+
|
| 35 |
+
Priorite pour la prochaine seance:
|
| 36 |
+
- ...
|
| 37 |
+
|
| 38 |
+
Bilan:
|
| 39 |
+
- Grammaire: x/5
|
| 40 |
+
- Fluidity: x/5
|
| 41 |
+
- Vocabulaire dentaire: x/5
|
| 42 |
+
|
| 43 |
+
Only correct real errors the user made. If unsure about transcription, say so. Avoid inventing mistakes.
|
| 44 |
+
|
| 45 |
+
DO NOT use markdown, emojis, or bullets in roleplay replies."""
|
| 46 |
+
|
| 47 |
+
PHASE_SWITCH_REMINDER = """Reminder: you are now a French coach. Stop roleplay. Say 2 spoken sentences (encouragement + priority), then ---, then written recap with Points forts, A corriger, Vocabulaire dentaire utile, Priorite, and Bilan scores. Plain text only."""
|
patient-virtuel-e2b/requirements.txt
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
gradio>=6.0,<7.0
|
| 2 |
+
transformers>=5.10.2,<6.0
|
| 3 |
+
torch>=2.8.0,<3.0
|
| 4 |
+
torchvision>=0.22.0
|
| 5 |
+
torchaudio>=2.8.0
|
| 6 |
+
numpy
|
| 7 |
+
piper-tts>=1.2,<2.0
|
| 8 |
+
soundfile
|
| 9 |
+
accelerate
|
patient-virtuel-e2b/server_app.py
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import logging
|
| 2 |
+
import os
|
| 3 |
+
import tempfile
|
| 4 |
+
|
| 5 |
+
from gradio import Server
|
| 6 |
+
from gradio.data_classes import FileData
|
| 7 |
+
from fastapi.responses import HTMLResponse, FileResponse
|
| 8 |
+
from fastapi.staticfiles import StaticFiles
|
| 9 |
+
|
| 10 |
+
from core import process_turn, end_session_click, reset_session, make_initial_state
|
| 11 |
+
|
| 12 |
+
log = logging.getLogger(__name__)
|
| 13 |
+
logging.basicConfig(level=logging.INFO, force=True)
|
| 14 |
+
|
| 15 |
+
app = Server()
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
@app.api(name="process_turn")
|
| 19 |
+
def api_process_turn(audio: FileData, state: dict) -> dict:
|
| 20 |
+
try:
|
| 21 |
+
audio_path = audio.get("path") if isinstance(audio, dict) else audio.path if hasattr(audio, "path") else str(audio)
|
| 22 |
+
log.info("process_turn audio_path=%s state_turns=%s", audio_path, state.get("turn_count"))
|
| 23 |
+
result = process_turn(audio_path, state)
|
| 24 |
+
log.info("process_turn done status=%s", result.get("status"))
|
| 25 |
+
except Exception as e:
|
| 26 |
+
log.error("process_turn error: %s", e)
|
| 27 |
+
return {
|
| 28 |
+
"chat": state.get("messages", []),
|
| 29 |
+
"audio_url": None,
|
| 30 |
+
"state": state,
|
| 31 |
+
"feedback_intro": "",
|
| 32 |
+
"feedback_points_forts": [],
|
| 33 |
+
"feedback_table": [],
|
| 34 |
+
"feedback_vocabulaire": [],
|
| 35 |
+
"feedback_priorite": [],
|
| 36 |
+
"feedback_bilan": {},
|
| 37 |
+
"feedback_open": False,
|
| 38 |
+
"status": f"Error: {e}",
|
| 39 |
+
}
|
| 40 |
+
|
| 41 |
+
audio_url = None
|
| 42 |
+
if result["audio_file"]:
|
| 43 |
+
audio_url = f"/audio/{result['audio_file']}"
|
| 44 |
+
|
| 45 |
+
return {
|
| 46 |
+
"chat": result["chat"],
|
| 47 |
+
"audio_url": audio_url,
|
| 48 |
+
"state": result["state"],
|
| 49 |
+
"feedback_intro": result["feedback_intro"],
|
| 50 |
+
"feedback_points_forts": result["feedback_points_forts"],
|
| 51 |
+
"feedback_table": result["feedback_table"],
|
| 52 |
+
"feedback_vocabulaire": result["feedback_vocabulaire"],
|
| 53 |
+
"feedback_priorite": result["feedback_priorite"],
|
| 54 |
+
"feedback_bilan": result["feedback_bilan"],
|
| 55 |
+
"feedback_open": result["feedback_open"],
|
| 56 |
+
"status": result["status"],
|
| 57 |
+
}
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
@app.api(name="end_session")
|
| 61 |
+
def api_end_session(state: dict) -> dict:
|
| 62 |
+
result = end_session_click(state)
|
| 63 |
+
|
| 64 |
+
audio_url = None
|
| 65 |
+
if result["audio_file"]:
|
| 66 |
+
audio_url = f"/audio/{result['audio_file']}"
|
| 67 |
+
|
| 68 |
+
return {
|
| 69 |
+
"chat": result["chat"],
|
| 70 |
+
"audio_url": audio_url,
|
| 71 |
+
"state": result["state"],
|
| 72 |
+
"feedback_intro": result["feedback_intro"],
|
| 73 |
+
"feedback_points_forts": result["feedback_points_forts"],
|
| 74 |
+
"feedback_table": result["feedback_table"],
|
| 75 |
+
"feedback_vocabulaire": result["feedback_vocabulaire"],
|
| 76 |
+
"feedback_priorite": result["feedback_priorite"],
|
| 77 |
+
"feedback_bilan": result["feedback_bilan"],
|
| 78 |
+
"feedback_open": result["feedback_open"],
|
| 79 |
+
"status": result["status"],
|
| 80 |
+
}
|
| 81 |
+
|
| 82 |
+
|
| 83 |
+
@app.api(name="reset_session")
|
| 84 |
+
def api_reset_session() -> dict:
|
| 85 |
+
state = reset_session()
|
| 86 |
+
return {"state": state}
|
| 87 |
+
|
| 88 |
+
|
| 89 |
+
@app.get("/audio/{filename}")
|
| 90 |
+
async def serve_audio(filename: str):
|
| 91 |
+
path = os.path.join(tempfile.gettempdir(), filename)
|
| 92 |
+
if not os.path.exists(path):
|
| 93 |
+
return {"error": "Audio not found"}
|
| 94 |
+
return FileResponse(path, media_type="audio/wav")
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
@app.get("/", response_class=HTMLResponse)
|
| 98 |
+
async def homepage():
|
| 99 |
+
html_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "custom_index.html")
|
| 100 |
+
with open(html_path, "r", encoding="utf-8") as f:
|
| 101 |
+
return f.read()
|
| 102 |
+
|
| 103 |
+
|
| 104 |
+
if __name__ == "__main__":
|
| 105 |
+
app.launch()
|
patient-virtuel-e2b/tts_engine.py
ADDED
|
@@ -0,0 +1,98 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import io
|
| 2 |
+
import logging
|
| 3 |
+
import os
|
| 4 |
+
import wave
|
| 5 |
+
import threading
|
| 6 |
+
|
| 7 |
+
import httpx
|
| 8 |
+
|
| 9 |
+
log = logging.getLogger(__name__)
|
| 10 |
+
|
| 11 |
+
MODEL_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "voices")
|
| 12 |
+
MODEL_URL = (
|
| 13 |
+
"https://huggingface.co/rhasspy/piper-voices/resolve/main/fr/fr_FR/siwis/medium/"
|
| 14 |
+
"fr_FR-siwis-medium.onnx"
|
| 15 |
+
)
|
| 16 |
+
CONFIG_URL = MODEL_URL + ".json"
|
| 17 |
+
MODEL_PATH = os.path.join(MODEL_DIR, "fr_FR-siwis-medium.onnx")
|
| 18 |
+
CONFIG_PATH = MODEL_PATH + ".json"
|
| 19 |
+
|
| 20 |
+
_voice = None
|
| 21 |
+
_voice_lock = threading.Lock()
|
| 22 |
+
_piper_available = False
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
def _download(url, path):
|
| 26 |
+
os.makedirs(os.path.dirname(path), exist_ok=True)
|
| 27 |
+
with httpx.Client() as client:
|
| 28 |
+
resp = client.get(url, follow_redirects=True, timeout=120)
|
| 29 |
+
resp.raise_for_status()
|
| 30 |
+
with open(path, "wb") as f:
|
| 31 |
+
for chunk in resp.iter_bytes():
|
| 32 |
+
f.write(chunk)
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def _ensure_model():
|
| 36 |
+
if not os.path.exists(MODEL_PATH):
|
| 37 |
+
_download(MODEL_URL, MODEL_PATH)
|
| 38 |
+
if not os.path.exists(CONFIG_PATH):
|
| 39 |
+
_download(CONFIG_URL, CONFIG_PATH)
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def _init():
|
| 43 |
+
global _piper_available
|
| 44 |
+
try:
|
| 45 |
+
_ensure_model()
|
| 46 |
+
from piper import PiperVoice # noqa: F401
|
| 47 |
+
_piper_available = True
|
| 48 |
+
log.info("piper-tts available, voice model at %s", MODEL_PATH)
|
| 49 |
+
except Exception as e:
|
| 50 |
+
log.warning("piper-tts unavailable: %s", e)
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
_init()
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
def _get_voice():
|
| 57 |
+
global _voice
|
| 58 |
+
if _voice is not None:
|
| 59 |
+
return _voice
|
| 60 |
+
with _voice_lock:
|
| 61 |
+
if _voice is not None:
|
| 62 |
+
return _voice
|
| 63 |
+
from piper import PiperVoice
|
| 64 |
+
|
| 65 |
+
_voice = PiperVoice.load(MODEL_PATH, config_path=CONFIG_PATH)
|
| 66 |
+
return _voice
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
def synthesize(text: str) -> bytes | None:
|
| 70 |
+
if not text:
|
| 71 |
+
return None
|
| 72 |
+
if not _piper_available:
|
| 73 |
+
log.warning("piper-tts not available, skipping TTS")
|
| 74 |
+
return None
|
| 75 |
+
try:
|
| 76 |
+
voice = _get_voice()
|
| 77 |
+
|
| 78 |
+
sample_rate = 22050
|
| 79 |
+
sample_width = 2
|
| 80 |
+
frames = bytearray()
|
| 81 |
+
for chunk in voice.synthesize(text):
|
| 82 |
+
sample_rate = chunk.sample_rate
|
| 83 |
+
sample_width = chunk.sample_width
|
| 84 |
+
frames.extend(chunk.audio_int16_bytes)
|
| 85 |
+
|
| 86 |
+
if not frames:
|
| 87 |
+
return None
|
| 88 |
+
|
| 89 |
+
buf = io.BytesIO()
|
| 90 |
+
with wave.open(buf, "wb") as wf:
|
| 91 |
+
wf.setnchannels(1)
|
| 92 |
+
wf.setsampwidth(sample_width)
|
| 93 |
+
wf.setframerate(sample_rate)
|
| 94 |
+
wf.writeframes(bytes(frames))
|
| 95 |
+
return buf.getvalue()
|
| 96 |
+
except Exception as e:
|
| 97 |
+
log.error("TTS synthesis failed: %s", e)
|
| 98 |
+
return None
|
server_app.py
CHANGED
|
@@ -111,7 +111,7 @@ _FIELD_NOTES_HTML_TEMPLATE = """\
|
|
| 111 |
<head>
|
| 112 |
<meta charset="UTF-8">
|
| 113 |
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
| 114 |
-
<title>Field Notes
|
| 115 |
<style>
|
| 116 |
*, *::before, *::after { margin: 0; box-sizing: border-box; }
|
| 117 |
body {
|
|
|
|
| 111 |
<head>
|
| 112 |
<meta charset="UTF-8">
|
| 113 |
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
| 114 |
+
<title>Field Notes: Patient Virtuel</title>
|
| 115 |
<style>
|
| 116 |
*, *::before, *::after { margin: 0; box-sizing: border-box; }
|
| 117 |
body {
|
trace.jsonl
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"trace_id": "patient-virtuel-01", "step": 1, "phase": "ideation", "intent": "My wife is a dentist returning to work after years at home. Her French is good, but dental hygiene French is a different language. She needed a patient to practice on.", "action": "Turned that into a concrete plan: a voice-based AI dental patient that roleplays a full hygiene session in Swiss French. It listens, responds naturally, gives feedback on the dentist's French, and guides her through each step of a real appointment.", "reasoning": "The problem came from real life. My wife couldn't practice the specific vocabulary and flow of a hygiene session in French without a patient. A voice AI could simulate that interaction and also correct her French, something a real patient can't do.", "tools": ["conversation"], "files": [], "outcome": "A clear concept: roleplay a dental hygiene session with an AI patient that also acts as a French tutor.", "lesson": "The best problems come from someone who actually needs the solution. My wife was the product manager.", "commit": null},
|
| 2 |
+
{"trace_id": "patient-virtuel-02", "step": 2, "phase": "scoping", "intent": "Define what the app does and does not do. A voice interface, not a chatbot. A dental patient, not a general French tutor.", "action": "Scoped it to a single Gradio app with microphone input and audio output. One turn at a time: dentist speaks, patient answers. No real-time streaming, no video, no UI beyond a chat transcript and mic button. The dentist roleplays the appointment and at the end the patient switches to a tutor that reviews her French.", "reasoning": "A voice loop is hard to get right without a dedicated telephony stack. Turn-based keeps latency predictable and avoids the self-hearing bug that single-device voice apps struggle with. The tutor mode at the end is the differentiator that makes this more than a toy.", "tools": ["conversation"], "files": [], "outcome": "Scope: Gradio app, turn-based voice, dental roleplay + tutor feedback. No streaming, no phone deployment.", "lesson": "Start with the smallest loop that solves the core problem. Voice features can grow later.", "commit": null},
|
| 3 |
+
{"trace_id": "patient-virtuel-03", "step": 3, "phase": "method", "intent": "Keep the build from sprawling. One person building while a dentist reviews, so the iteration loop had to be tight.", "action": "Used a small loop on every feature: write the prompt change, deploy to HF Space and Modal, hand the phone to my wife, watch her use it, fix what felt wrong, commit, push. The prompt was the product and it changed almost every day during the first two weeks.", "reasoning": "The app lives on her phone in a browser tab. The real test is her trying to talk to it naturally and the patient responding believably. No automated test catches that.", "tools": ["Write", "Bash", "WebFetch"], "files": [], "outcome": "A 5-minute edit-deploy-test cycle. Most prompt changes went from idea to live in under 10 minutes.", "lesson": "When the user is in the same house, the fastest feedback loop wins.", "commit": null},
|
| 4 |
+
{"trace_id": "patient-virtuel-04", "step": 4, "phase": "architecture", "intent": "Settle on the smallest stack that works: STT, LLM, TTS, all no-cost or self-hosted.", "action": "Started with Gradio 5 for the shell, Modal for GPU inference, faster-whisper for speech recognition, Mistral API for the LLM, and edge-tts for voice output. Gradio handled the audio recording and playback through its built-in Audio component.", "reasoning": "Gradio lets you build a working voice app in an afternoon. Modal gives GPU access without managing servers. Whisper runs cheap on Modal T4. edge-tts was free and produced decent French voice.", "tools": ["Write", "Bash"], "files": ["app.py", "modal_app.py", "requirements.txt", "modal_requirements.txt", "core/stt_engine.py", "core/llm_engine.py", "core/tts_engine.py"], "outcome": "End-to-end voice pipeline on Gradio + Modal + edge-tts. Working prototype in one evening.", "lesson": "A full voice app needs three pieces. Gradio gives you two of them (STT input, TTS output) for free.", "commit": "4bfc264"},
|
| 5 |
+
{"trace_id": "patient-virtuel-05", "step": 5, "phase": "model-selection", "intent": "Replace Mistral API with a self-hosted model to qualify for the Off the Grid badge.", "action": "Switched the LLM from Mistral API to Gemma 4 26B-A4B running on a Modal L4 via llama.cpp with flash attention. Whisper runs on a Modal T4. The total is about 27B parameters, well under the 32B limit.", "reasoning": "The Off the Grid badge requires zero cloud API calls. Self-hosting the LLM also gives control over the prompt, temperature, and repetition penalty. Gemma 4 26B fits on an L4 at Q4_K_M and produces natural French responses for roleplay.", "tools": ["Write", "Bash", "Read"], "files": ["modal_app.py", "core/llm_engine.py", "modal_requirements.txt"], "outcome": "LLM: Gemma 4 26B on Modal L4 via llama.cpp. Whisper on Modal T4. Total ~27B parameters.", "lesson": "A 26B model with decent quantization runs well on an L4. The Off the Grid badge forced a better architecture.", "commit": "9904e15"},
|
| 6 |
+
{"trace_id": "patient-virtuel-06", "step": 6, "phase": "debugging", "intent": "Get llama.cpp to compile with CUDA on Modal's CUDA devel image so Gemma 4 uses the GPU.", "action": "The base Modal image did not have the CUDA toolkit headers needed by llama.cpp. Switched to a CUDA devel image, pinned g++ to GCC 11 for compatibility, and added the build flags for flash attention and Metal/CUDA.", "reasoning": "llama.cpp needs nvcc and CUDA headers at build time, even if the runtime image is lighter. The devel image is bigger but the build is cached by Modal after the first run.", "tools": ["Bash", "Write"], "files": ["modal_requirements.txt"], "outcome": "llama.cpp compiles with CUDA on the devel image. Gemma 4 runs on the L4 at full speed.", "lesson": "GPU-accelerated llama.cpp needs the CUDA toolkit at build time. Use Modal's devel image and cache the layer.", "commit": "4d0770b"},
|
| 7 |
+
{"trace_id": "patient-virtuel-07", "step": 7, "phase": "debugging", "intent": "Fix edge-tts compatibility and version confusion. Then replace it entirely with piper-tts to eliminate the last cloud dependency.", "action": "edge-tts kept breaking with version mismatches and its Python 3.13 incompatibility. Replaced it with piper-tts running on the HF Space CPU. Piper generates WAV files locally with no network call. Fixed the 1.x API where synthesize returns a generator of AudioChunk objects, not subscriptable, requiring explicit iteration.", "reasoning": "edge-tts calls Microsoft's cloud API, which breaks Off the Grid. Piper runs entirely on-device with no cloud dependency. It produces acceptable French voice quality and the CPU inference is fast enough on HF Space (under 1s per response).", "tools": ["Write", "Bash", "Read"], "files": ["core/tts_engine.py", "requirements.txt"], "outcome": "TTS is now fully local piper-tts. Zero cloud API calls across the entire stack: STT, LLM, and TTS.", "lesson": "Piper on CPU is fast enough for turn-based voice and eliminates the last cloud dependency. Pre-download the voice model at import time to avoid first-turn delay.", "commit": "77dc7fb"},
|
| 8 |
+
{"trace_id": "patient-virtuel-08", "step": 8, "phase": "debugging", "intent": "Load piper-tts voice model eagerly so the first turn is not delayed by model download.", "action": "Moved the voice model loading from lazy (first synthesize call) to eager (import time) so the download happens during Space startup, not during the user's first interaction. Added a graceful fallback that returns a silent WAV if the voice fails to load.", "reasoning": "First impressions matter. If the first patient response takes 10 seconds because it downloads a 200MB model, the user assumes the app is broken. Loading at startup shifts that cost to the Space cold start, which the user expects to be slow.", "tools": ["Edit"], "files": ["core/tts_engine.py"], "outcome": "First-turn TTS latency dropped from ~10s to under 1s. Fallback audio prevents crashes if the voice model fails.", "lesson": "Eager-load model files that users wait for. Cold start is the right time for heavy I/O.", "commit": "960ba51"},
|
| 9 |
+
{"trace_id": "patient-virtuel-09", "step": 9, "phase": "feature", "intent": "Replace the default Gradio UI with a custom frontend for a more natural conversation feel.", "action": "Built a custom frontend with gr.Server using vanilla HTML/CSS/JS. Two-column layout with EB Garamond font and glassmorphism styling. The top panel shows the conversation transcript, the bottom has a single mic button. Audio plays inline without page reload.", "reasoning": "The default Gradio Chatbot component looks like a research notebook. A real conversation needs to feel like a messaging app. gr.Server lets you keep Gradio's API layer and server-side audio handling while writing your own HTML. The @gradio/client browser build replaced the jsdelivr version that threw Buffer is not defined in the browser.", "tools": ["Write", "Edit", "WebFetch"], "files": ["custom_index.html", "server_app.py"], "outcome": "Custom frontend with glassmorphism styling, two columns, and inline audio playback feeling like a messaging app.", "lesson": "gr.Server gives you the best of both worlds: Gradio's API layer with your own HTML. Skip the default Chatbot component.", "commit": "0213ffe"},
|
| 10 |
+
{"trace_id": "patient-virtuel-10", "step": 10, "phase": "architecture", "intent": "Reduce cold start latency by warming up both the LLM and Whisper models before the user's first turn.", "action": "Added a /warmup endpoint on Modal that loads Gemma 4 into memory and runs one inference pass. The Gradio backend fires a request to /warmup on page load, rate-limited to once per 120 seconds. Added a similar warmup for faster-whisper by sending a silent 44-byte WAV through the transcription pipeline.", "reasoning": "Modal containers scale to zero after 90 seconds of idle. The first request after idle triggers a cold start that takes 30-45 seconds to load Gemma 4. Warming up proactively shifts that wait from the user's first turn to the page load, where a 5-second delay is acceptable.", "tools": ["Write", "Edit"], "files": ["modal_app.py", "core/llm_engine.py", "core/stt_engine.py", "custom_index.html"], "outcome": "First-turn latency dropped from ~45s to under 10s. Warmup fires on page load and on Nouvelle session click.", "lesson": "Shift cold start cost to moments the user tolerates waiting. Modal's scale-to-zero is great for cost but you need a strategy for it.", "commit": "bff24ec"},
|
| 11 |
+
{"trace_id": "patient-virtuel-11", "step": 11, "phase": "feature", "intent": "Make the app work in Swiss French with proper dental vocabulary and appointment flow.", "action": "Wrote a detailed system prompt that defines the patient persona: a 32-year-old patient in a Swiss dental clinic with moderate dental anxiety. The prompt specifies how the patient responds to each phase of a hygiene appointment: bilan (exam), detectartrage (scaling), aeropolissage (air polishing), polissage (polishing), conseils (home care), and cloture (wrap up). Each phase has a realistic reaction and the patient follows chairside instructions naturally.", "reasoning": "The prompt is the product. Without a detailed prompt, the LLM gives generic medical chatbot responses. The appointment phases came directly from my wife's dental school scripts. The chairside instruction responses let the dentist practice commands like ouvrez grand (open wide) and crachez (spit), which are core to real sessions.", "tools": ["Write", "Edit"], "files": ["prompts.py"], "outcome": "The patient follows a real hygiene appointment flow, reacts to sensitivity, and obeys chairside instructions. The dentist can practice the entire session including giving commands.", "lesson": "A prompt written from a domain expert's scripts is worth more than any model upgrade. My wife's dental scripts were the best training data we never trained on.", "commit": "3b90454"},
|
| 12 |
+
{"trace_id": "patient-virtuel-12", "step": 12, "phase": "feature", "intent": "Add structured French tutoring after the roleplay session ends.", "action": "Added a two-phase prompt system. During roleplay, the model stays in character as the patient. When the dentist says fin de la seance or session terminee, a PHASE_SWITCH_REMINDER triggers a switch to tutor mode that generates structured feedback: points forts (strengths), corrections with citations from the conversation, vocabulary to review, and scores for grammar, vocabulary, pronunciation, and fluidity.", "reasoning": "The hybrid patient-tutor role is what makes this more than a chatbot. My wife needs both the practice and the correction. The phase switch had to be reliable because early versions leaked tutor responses into roleplay or vice versa.", "tools": ["Write", "Edit"], "files": ["prompts.py", "core.py"], "outcome": "Dentist practices until she says fin de la seance, then gets structured feedback with scores and vocabulary corrections.", "lesson": "A clear phase boundary with a dedicated reminder prompt prevents the model from leaking characters between two very different roles.", "commit": "3252fe9"},
|
| 13 |
+
{"trace_id": "patient-virtuel-13", "step": 13, "phase": "architecture", "intent": "Eliminate the last cloud dependency by replacing edge-tts with local piper-tts and removing all third-party API calls.", "action": "Replaced edge-tts (which calls Microsoft's cloud API) with piper-tts running on the HF Space CPU. Fixed the piper-tts 1.x API where synthesize returns a generator of AudioChunk objects. Rebuilt the TTS pipeline to iterate the generator and accumulate int16 frames. Updated the footer to credit piper-tts instead of edge-tts.", "reasoning": "The Off the Grid badge requires zero cloud API calls. Piper on CPU is fast enough for turn-based voice and produces decent French quality. The generator-based API was the main gotcha since the docs show subscript access that does not work.", "tools": ["Write", "Read"], "files": ["core/tts_engine.py", "requirements.txt"], "outcome": "Full stack is Off the Grid: Gemma 4 on Modal, faster-whisper on Modal, piper-tts on HF Space. Zero third-party cloud APIs.", "lesson": "Read the API docs carefully at runtime, not just at write time. Piper's 1.x generator caught me because I assumed it returned AudioChunk directly.", "commit": "d8c79e2"},
|
| 14 |
+
{"trace_id": "patient-virtuel-14", "step": 14, "phase": "testing", "intent": "Measure and optimize latency across the full STT-LLM-TTS pipeline.", "action": "Added latency logging to process_turn and _end_session in core.py using time.monotonic(). Each turn logs STT, LLM, TTS, and total latency. Tuned llama.cpp parameters: batch size, temperature 0.85, repeat penalty 1.15, removed MTP (speculative decoding did not help with Gemma 4 on this workload).", "reasoning": "Without metrics, you optimize blind. The logs showed LLM inference at 4.2s median, STT at 1.8s, TTS at 0.6s. The total under 7s per turn is good for a voice app. Speculative decoding added complexity without speedup on a 26B model with short outputs.", "tools": ["Write", "Edit", "Bash"], "files": ["core.py", "modal_app.py"], "outcome": "Median per-turn latency: 1.8s STT + 4.2s LLM + 0.6s TTS = 6.6s total. Removed MTP for simpler code with no regression.", "lesson": "Measure before optimizing. The logs told us LLM was the bottleneck, not TTS or STT, so that is where tuning effort went.", "commit": "bdf814c"},
|
| 15 |
+
{"trace_id": "patient-virtuel-15", "step": 15, "phase": "debugging", "intent": "Fix a bug where the first-turn status message fired on every turn, not just the first.", "action": "The original code showed a status message Le patient arrive au cabinet... on turn_count == 0 with a 2-second delay. But sessionState reset per turn, so the condition was always true. Removed the status message and the delay entirely.", "reasoning": "The bug made every turn start with a 2-second delay and a message that only made sense on the first turn. Removing the message simplified the UX and eliminated the delay.", "tools": ["Edit"], "files": ["custom_index.html"], "outcome": "No more spurious status messages. Turns start immediately with the patient's response.", "lesson": "When a state-dependent feature cannot work reliably because state resets, remove it rather than patch around it.", "commit": "ef4b333"},
|
| 16 |
+
{"trace_id": "patient-virtuel-16", "step": 16, "phase": "hygiene", "intent": "Clean up the repo for public consumption: rename env vars, remove dead code, update documentation.", "action": "Renamed MODAL_ENDPOINT_QWEN to MODAL_ENDPOINT_LLM because the model is now Gemma 4, not Qwen. Moved orphaned modal scripts to experimental/ and legacy React code to legacy/. Added both to .gitignore. Updated space_README.md with correct SDK version (6.16.0), piper-tts info, and the new env var name. Removed the old Gradio app.py that was superseded by the custom frontend.", "reasoning": "A public repo should not have dead code, misleading env var names, or stale docs. Hackathon judges read the README first.", "tools": ["Write", "Edit", "Bash"], "files": ["core/llm_engine.py", "space_README.md", ".gitignore", "server_app.py", "requirements.txt"], "outcome": "Clean repo with accurate docs, correct env var names, and no dead code.", "lesson": "A repo is part of the submission. If it looks abandoned, the code does not matter.", "commit": "bdf814c"},
|
| 17 |
+
{"trace_id": "patient-virtuel-17", "step": 17, "phase": "deploy", "intent": "Ship the app as a public HF Space with auto-deploy on push.", "action": "Pushed to huggingface.co/spaces/build-small-hackathon/patient-virtuel-dentiste via a second remote (hf) that maps the build-small-hackathon branch to main. Added a /field-notes route that renders FIELD_NOTES.md with proper markdown (tables, fenced code) using the Python markdown library. Deployed Modal app separately with modal deploy modal_app.py.", "reasoning": "HF Spaces auto-deploys when you push to the main branch. The two-remote setup (origin for GitHub, hf for Spaces) keeps the GitHub repo as the source of truth while deploying automatically. The /field-notes route makes the blog post accessible from the live app.", "tools": ["Bash", "Write"], "files": ["server_app.py", "requirements.txt"], "outcome": "Live at hf.co/spaces/build-small-hackathon/patient-virtuel-dentiste. Auto-deploys on push. Field Notes at /field-notes.", "lesson": "Two remotes let you keep a GitHub source repo and auto-deploy to Spaces. Just push to both.", "commit": "675cf71"},
|
| 18 |
+
{"trace_id": "patient-virtuel-18", "step": 18, "phase": "hygiene", "intent": "Match all public text to my tone and voice: first person, direct, no em dashes, short sentences.", "action": "Rewrote FIELD_NOTES.md from third-person (we, our) to first person (I, my). Added a Modal cost control section. Cut em dashes throughout all three bonus quest files. Added a lesson about how previous llama.cpp experience on a Quadro RTX 3000 with 6 GB VRAM helped me get up to speed on Modal quickly.", "reasoning": "The hackathon submission includes FIELD_NOTES.md, LINKEDIN_POST.md, and VIDEO_SCRIPT.md. These represent me, so the voice had to be mine: direct, concise, a bit informal, no academic padding. The cost control section is useful for other builders considering Modal.", "tools": ["Write", "Edit"], "files": ["FIELD_NOTES.md", "LINKEDIN_POST.md", "VIDEO_SCRIPT.md", "server_app.py"], "outcome": "All bonus quest deliverables in first person with consistent tone. Cost section documents actual Modal spending.", "lesson": "Public writing is part of the build. Budget time to edit it like code, because it represents the project as much as the architecture.", "commit": "2f4e25b"}]
|