Update README.md

README.md

@@ -15,159 +15,29 @@ tags:
 - backchannel
 ---
 
-# personaplex-fisher-lora-backchannel-v14b
 
-### Augmentation recipe
+# PersonaPlex Backchannel Head
 
-| lever | value | effect |
-|---|---|---|
-| `--backchannel_aug_prob` | 0.3 | 30% of examples are augmented |
-| `--backchannel_aug_delete_share` | 1.0 | every augmented example is a *deletion* example — no insertions |
-| `--backchannel_aug_delete_prob` | 0.5 | half the backchannels in such an example are removed |
-| `--backchannel_aug_purge_prob` | 0.05 | 5% of examples have **every** backchannel erased, drawn independently of `aug_prob` |
+This HF repository contains a LoRA adapter for [PersonaPlex-7B](https://huggingface.co/nvidia/personaplex-7b-v1) that adds a lightweight, controllable backchannel head, introduced in [Controlling Backchannels in Streamable Full-Duplex Models](https://arxiv.org/abs/2609.29418).
 
-The purge is the important one. At `delete_prob 0.5` alone, every augmented example still contains
-backchannels, so the model can learn "fewer here" but never "none" — and "none" is the only signal
-that acts on *spontaneous* production. Purged examples keep their head positives, so head
-supervision is identical across all four v14 arms and the purge acts only on the LM.
+About our work: Backchannels, brief acknowledgements like "uh-huh" produced while the other party may still be talking, are central to natural conversation, but full-duplex spoken dialogue models rarely model them explicitly. We introduce a lightweight backchannel head that predicts, from a full-duplex model's own hidden states, when a backchannel should begin. Once this probability crosses a tunable threshold, a backchannel is force-decoded. Attached to both a 7B (PersonaPlex) and a 1B (F-Actor) model, it generalizes across scale. Probing confirms the hidden states anticipate real human timing, and generation evaluation shows more frequent, better-timed backchannels. Human raters judge the resulting backchannels on par with real ones.
 
-## Training
+Please refer to the [codebase](https://github.com/MaikeZuefle/bcmore) for the usage of the model.
 
-* **Data:** [`JSALT2026-Conv-AI-Simulator/fisher-v1`](https://huggingface.co/datasets/JSALT2026-Conv-AI-Simulator/fisher-v1)
-  (Fisher telephone conversations), `max_length` 2000 frames (~160 s), 4 epochs, batch 1 x
-  grad-accum 16.
-* **Backchannel labels** are derived from word alignments (Fisher's own
-  `utterances[].backchannels` column is empty): each speaker is split into inter-pausal units, and
-  one is marked a backchannel when it is short (`max_duration 1.5`), passes a content gate, and did
-  not take the floor (`floor_gap 2.0`, retaining floor status across gaps up to 2 s).
-  `--no-backchannel_include_laughter` — unlike `v2`, listener laughter does **not** count.
-  `--backchannel_turn_clearance 1.5`, `--drop_non_speech_words`.
-* Only frames where a backchannel is *possible* (agent silent, partner talking,
-  `agent_silence_margin 0.24`) are supervised; the rest are ignored rather than taught as negatives.
+For more information, please have a look at the paper.
 
-### Architecture
+## Citation
 
-* **LoRA:** attention only, on the temporal transformer —
-  `^decoder\.model\.layers\.\d+\.(self_attn\.(q_proj|k_proj|v_proj|o_proj)\.linear)$` —
-  `r=16`, `alpha=32`, `dropout=0.05`.
-* **Backchannel head:** `Linear(hidden -> 256) -> GELU -> Linear(256 -> 1)`, reading a single frame
-  of the trunk's hidden state. `context=0`, i.e. **no** temporal convolution — that is the `claims`
-  line of checkpoints.
-* **Trainable tokens:** the `<bc>` marker is token id **4**, the `<0x00>` byte-fallback slot — the
-  one slot the tokenizer can never emit (it occurs zero times across all 11,396 Fisher rows). Its
-  embedding row is trained in both `decoder.model.embed_tokens` and
-  `depth_decoder.text_embed_tokens`; the rest of both tables stays frozen.
-* **Loss:** focal (`gamma=2.0`, `alpha=0.9`), backchannel loss weight `1.0`, head bias initialised
-  to prior `0.016`, head LR `1e-3`, LM LR `2e-5`. Semantic codebook weight `1.0`, acoustic `0.02`,
-  pad-text loss weight `0.3`.
-
-## Evaluation
-
-Three measurements that answer three different questions:
-
-| measurement | what it asks | material |
-|---|---|---|
-| Fisher head eval | can the head rank frames? | Fisher validation, teacher-forced |
-| TurnBench head eval | does that survive out of domain, against human annotations? | TurnBench dev, teacher-forced |
-| self-play obedience | when the head fires, does the model produce a backchannel? | Fisher-domain self-play |
-
-Matching tolerance is 5 frames (400 ms) in both head evals — exact-frame agreement is too strict a
-bar for a phenomenon whose natural placement window is a few hundred ms wide.
-
-### Stage 1 — Fisher validation, teacher-forced
-
-93,042 supervised frames, 1,213 positive (1.30%).
-
-| Average Precision | ROC-AUC | mean p on positives | mean p on negatives |
-|---|---|---|---|
-| 0.352 | 0.928 | 0.486 | 0.182 |
-
-| Threshold | Precision | Recall | F1@400ms | Rate vs. human |
-|---|---|---|---|---|
-| 0.70 | 0.206 | 0.305 | 0.246 | 1.48x |
-
-
-### Stage 2 — TurnBench dev, out of domain
-
-12 conversations / 24 dialogue passes, 6,553 supervised frames, 97 positive, 35 reference events,
-human rate 1.52 events/min. Run on a **-23 LUFS loudness-normalised** copy: TurnBench ships 19.4 dB
-apart internally and ~15 dB below Fisher, which moves where the head fires.
-
-Two conditions: `keep` leaves the recording untouched (the agent's own backchannels stay in its
-context, so the head can lean on them); `remove` erases them, which is the condition that matches
-self-play and the render harness.
-
-| Condition | AP | ROC-AUC | Best lag | Rate-matched thr | P | R | **F1** | Rate |
-|---|---|---|---|---|---|---|---|---|
-| `keep` | 0.147 | 0.810 | -1 | 0.650 | 0.222 | 0.229 | **0.225** | 1.03x |
-| `remove` | 0.123 | 0.805 | -4 | 0.650 | 0.485 | 0.457 | **0.471** | 0.94x |
-
-At best-F1 rather than rate-matched, `remove` reaches **F1 0.475** (thr 0.675, P 0.583 / R 0.400) but
-sits at 0.69x the human rate — it buys precision by firing less.
-
-Two caveats worth carrying:
-
-* **The in-domain to out-of-domain drop is large and not checkpoint-specific**: AP 0.35 -> 0.12-0.15,
-  ROC-AUC 0.93 -> 0.81.
-* **v14b's optimal lag on `remove` is -4 frames** (320 ms *before* the human onset), where the other
-  arms sit at -1/-2. AP at that lag is 0.231, against 0.123 at lag 0.
-
-### Stage 3 — self-play obedience
-
-100 dialogues x 80 s windows (133 min) in the partner-backchannelling condition. Thresholds are
-calibrated **on generated streams**, never reused from validation — confidence is systematically
-different in free generation than under teacher forcing.
-
-| Condition | Threshold | Fires/min | **Obeyed** | **Realised** | clean bc | overrun | other | stranded |
-|---|---|---|---|---|---|---|---|---|
-| off | — | 0.00 | — | — | — | — | — | — |
-| low | 0.740 | 1.03 | **100%** | **77%** | 106 | 31 | 0 | 0 |
-| normal | 0.695 | 1.48 | **98%** | **69%** | 137 | 58 | 3 | 0 |
-| high | 0.615 | 3.59 | **95%** | **65%** | 311 | 142 | 24 | 2 |
-| max | 0.485 | 8.31 | **89%** | **65%** | 717 | 270 | 84 | 37 |
-| *Partner (within-dialogue baseline)* | — | *1.51* | — | — | — | — | — | — |
-
-* **`obeyed`** = the model started *something* at the trigger. **`realised`** = what it said was a
-  backchannel and stayed one. **`overrun`** = it began with a backchannel and then kept talking into
-  a turn. **`other`** = not a backchannel at all.
-* At `normal`, only **3 of 198** triggers produced a non-backchannel.
-* Obedience degrades gracefully: still 89% obeyed when driven to 5.5x the human rate.
-* **Realised forms** are the expected inventory: `yeah` (~500), `right`, `okay`, `uh-huh`, `mm`,
-  `mhm`.
-
-#### Controllability of `off`
-
-With the head off, **zero** forced events fire, but the model still emits **0.70 spontaneous
-lexical backchannels/min**, against 1.51/min for its partner in the same calls. So `off` means "no
-head-driven backchannels", not silence — the purge lever moves this number but does not zero it.
-
-#### Overrun
-
-29% of obeyed triggers at `normal` (58/198) overran into a turn. The forced-decode harness measures
-the same phenomenon far lower — 22/177 = 12% continued past a complete backchannel, of which 9
-(5.1%) were true turn attempts — because its gate requires 3.2 s of clearance before firing, so it
-only fires where a turn *cannot* start. Self-play lacks that constraint, so **treat 29% as the
-pessimistic bound and 5% as what a clearance-gated deployment sees.**
-
-### Choosing among the v14 arms
-
-Rate-matched F1 on TurnBench, and obedience at a matched ~1.5 fires/min:
-
-| ckpt | augmentation | Fisher F1 | TB `keep` F1 | TB `remove` F1 | obeyed | realised |
-|---|---|---|---|---|---|---|
-| v14a | none | .252 | **.267** | .348 | 87% | 51% |
-| **v14b** | **deletions only** | .246 | .225 | .471 | **98%** | **69%** |
-| v14c | insertions only | .244 | **.270** | .324 | 93% | 63% |
-| v14d | both | .246 | .257 | **.479** | 93% | 53% |
-
-1. **Fisher separates nothing** — all four within 0.008 F1 and 0.002 AUC on 1,213 positive frames.
-2. **`remove` splits them into two pairs**: v14b/v14d at .471/.479 against v14a/v14c at .348/.324, a
-   ~45% relative gap driven by precision. But v14b vs v14d is a coin flip on 35 reference events.
-3. **`keep` ranks them differently** and puts v14b last. `keep` is the easier condition and the one
-   that does not match how the model is actually run.
-4. **Obedience is where they separate, and v14b wins outright** — at every floor, on both metrics,
-   with 3 non-backchannels out of 198 against 26 for v14a.
-
-Counterintuitive and worth restating: **deletions-only beats insertions-only and both**, even though
-insertion is the arm meant to teach the model to obey a backchannel request.
+If you use this model, please cite:
 
+```bibtex
+@misc{züfle2026controllingbackchannelsstreamablefullduplex,
+      title={Controlling Backchannels in Streamable Full-duplex Models}, 
+      author={Maike Züfle and Peter Polák and Sefik Emre Eskimez and Jan Niehues and Peter Bell and Ondřej Klejch},
+      year={2026},
+      eprint={2609.29418},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2609.29418}, 
+}
+```