balaji1312/whisper-medium-myst-aura

Whisper-medium MyST (AURA)

Whisper-medium adapted to the MyST child speech corpus with AURA (Activation-editing with Uncertainty-Routed Adaptation), from the paper "AURA: Uncertainty-Routed Activation Editing for Acoustic Grounding in Speech Foundation Models" (IEEE SLT 2026).

• Base model: openai/whisper-medium.en
• Method: AURA scale-and-shift edits on the cross-attention heads of every decoder layer, controlled by Hard-Concrete head gates and the uncertainty-routed dynamic gate; the Whisper weights stay frozen
• Code: https://github.com/balaji1312/aura

Usage

The checkpoint bundles the frozen Whisper backbone together with the trained AURA parameters. Loading requires the custom modeling code in the aura repository. Download the model and decode with src/bin/decode_asr.py there:

git clone https://github.com/balaji1312/aura && export rootdir=$(realpath aura)
huggingface-cli download balaji1312/whisper-medium-myst-aura --local-dir whisper-medium-myst-aura
python $rootdir/src/bin/decode_asr.py --model whisper-medium-myst-aura --processor whisper-medium-myst-aura \
  --wav_scp data/test/wav.scp --trn_scp data/test/text \
  --result_ref_file ref.txt --result_hyp_file hyp.txt

Citation

@inproceedings{shankar2026aura,
  author    = {Shankar, Natarajan Balaji and Wang, Zilai and Wang, Zihan and Shi, Mohan and Zhang, Kaiyuan and Alwan, Abeer},
  title     = {{AURA}: Uncertainty-Routed Activation Editing for Acoustic Grounding in Speech Foundation Models},
  booktitle = {IEEE Spoken Language Technology Workshop (SLT)},
  year      = {2026},
}