Hugging Face's logo

dmvevents
/
nora-4b-v3.2

Safetensors
English
qwen3_5
nora
trinidad-tobago
tstt
qwen3.5
layer-selective
ties-merge
paraphrase-robust
Model card Files
xet
 Community
nora-4b-v3.2 / README.md
dmvevents's picture
dmvevents
Initial: v3.2 TIES-merged (89.2% Claude on 1,420 paraphrase set)
7ed6f65 verified
preview code
|
Raw History Blame Contribute Delete
2.95 kB
metadata 
license: apache-2.0
tags:
  - nora
  - trinidad-tobago
  - tstt
  - qwen3.5
  - layer-selective
  - ties-merge
  - paraphrase-robust
language:
  - en
base_model: Qwen/Qwen3.5-4B
datasets:
  - dmvevents/tt-training-v3.2

Nora v3.2 — T&T AI Assistant (paraphrase-robust)

v3.2 targets paraphrase robustness on top of v3.1's gap-closing gains. Starts from pristine Qwen3.5-4B, trained on the v3.2 mix (23,273 rows: sft_v4 + v3.1 corrective + 1,353 paraphrase-augmented rows targeting stressed/negation/multi-part prompt styles + new corrective from v3.1 failures), 2-seed TIES + SLERP anchor.

Ship-gate Results (2026-05-06)

Eval set Claude Sonnet 4.5 Δ vs v3.1 Δ vs v2
1,420 paraphrases 89.2% +2.1pp +4.3pp
143-base 89.9% -0.3pp +5.0pp

Pass-rate @4/5: 89.8% (v3.1 was 86.8%, +3.0pp).

Per-category, 1,420-paraphrase set (Claude)

Category v3.1 v3.2 Δ
safety 86.3 91.3 +5.03 ← target
government_service 80.3 83.9 +3.62 ← target
creole_handling 85.6 88.2 +2.60 ← target
identity 95.2 97.6 +2.42
education_general 87.7 89.2 +1.45
healthcare 87.5 87.7 ~0
csec_math 92.0 90.5 -1.57

All three gap-targeted categories improved by +2.6 to +5.0pp. 6 of 7 categories improved; csec_math regressed within noise (still 90.5%).

v3.2 wins 300 prompts, loses 200, ties 913 (57% win rate on non-ties).

Training recipe (unchanged from v3.1)

  • Base: pristine Qwen3.5-4B (NOT v3.1 — avoid compounded drift)
  • Layer-selective (DeltaNet frozen, Attention + MLP + norms + lm_head trained)
  • Completions-only loss
  • LR 5e-6, 1 epoch on 23,273 rows
  • 2 seeds (42, 1337), 20-min launch stagger
  • Step checkpoints @ 150/300/364; promote checkpoint-364 → final (sidesteps trainer.save_model() hang on Qwen3.5 hybrid FSDP)
  • TIES merge (density=0.5, trim=0.2, scale=1.0) + SLERP anchor t=0.35

Data recipe vs v3.1

The v3.2 innovation: 1,353 paraphrase-augmented rows. For each v3.1 corrective training row, Bedrock Opus generated 3 new user-turn rephrases hitting styles 5 (stressed/frustrated), 6 (negation probe), and 9 (multi-part compound). Assistant turn preserved verbatim. This targets the v3.1 paraphrase-style brittleness identified in the gap analysis.

Also: 882 new 7-pattern corrective rows from v3.1 failure buckets, 45 polarity hard-negatives on 5 new fact cards, 5 fact-card content fixes, and a "use verified cards verbatim" rule added to the retrieval router.

Files

Licence

Apache 2.0 (inherited from Qwen3.5-4B).