Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,112 @@
+---
+license: apache-2.0
+language:
+- en
+pipeline_tag: automatic-speech-recognition
+tags:
+- litert
+- tflite
+- android
+- gpu
+- wav2vec2
+- ctc
+- speech-recognition
+base_model: facebook/wav2vec2-base-960h
+---
+
+# wav2vec2-base-960h CTC — LiteRT (GPU)
+
+English speech recognition with [wav2vec2-base-960h](https://huggingface.co/facebook/wav2vec2-base-960h)
+running **fully on the LiteRT `CompiledModel` GPU** (ML Drift) — and with **zero FFT anywhere**:
+the raw 16 kHz waveform goes straight into the 1D-conv feature extractor, so there is no mel/fbank
+step even on the host. Character-level CTC (29 chars + specials), greedy decode, no language model.
+
+![wav2vec2 CTC word onsets](assets/hero.png)
+*Real model output: char-CTC word onsets for J.F. Kennedy's 1961 inaugural address
+(U.S. National Archives recording, public domain).*
+
+Ships as **two GPU graphs** — the fused graph exceeds the Mali whole-graph shader-compile limit
+(a graph can be op-clean and still fail to compile when fused; each half compiles and runs fully
+delegated):
+
+| File | Size | Input | Output | API |
+| ---- | ---- | ----- | ------ | --- |
+| `w2v2_asr_frontend_fp16.tflite` | 9 MB | waveform `[1, 256000]` | features `[1, 799, 768]` | CompiledModel GPU |
+| `w2v2_asr_head_fp16.tflite` | 180 MB | features `[1, 799, 768]` | CTC logits `[1, 799, 32]` | CompiledModel GPU |
+
+## Pipeline
+
+`16 kHz mono PCM in [-1, 1], zero-padded to the fixed 16 s window → [GPU] conv frontend →
+[GPU] 12-layer transformer + lm_head → host greedy-CTC over the valid frames`
+
+- Valid frames for `n` samples: run `L=(L-k)//s+1` over the conv stack
+  `(10,5)(3,2)(3,2)(3,2)(3,2)(2,2)(2,2)` — 50 Hz frames (16 s → 799).
+- Blank id 0 (`<pad>`), `|` = word delimiter (`tokens.txt`, index-ordered).
+- Greedy char-CTC without an LM has the model's known spelling quirks on hard words
+  (e.g. GRAVED/GRAVE); add a beam+LM host-side if you need the last WER point.
+
+## Minimal usage — Python
+
+```python
+import numpy as np, torch, torchaudio
+from ai_edge_litert.interpreter import Interpreter
+
+wave, sr = torchaudio.load("speech.wav")             # 16 kHz mono, [-1,1]
+x = torch.zeros(1, 256000); n = min(wave.shape[1], 256000)
+x[0, :n] = wave[0, :n]
+
+def run(path, inp):
+    it = Interpreter(model_path=path); it.allocate_tensors()
+    d = it.get_input_details()[0]
+    it.set_tensor(d["index"], inp.astype(np.float32)); it.invoke()
+    return it.get_tensor(it.get_output_details()[0]["index"])
+
+feat = run("w2v2_asr_frontend_fp16.tflite", x.numpy())
+logits = run("w2v2_asr_head_fp16.tflite", feat)[0]    # [799, 32]
+
+L = n
+for k, s in [(10,5),(3,2),(3,2),(3,2),(3,2),(2,2),(2,2)]:
+    L = (L - k) // s + 1
+tokens = open("tokens.txt").read().splitlines()
+out, prev = [], -1
+for i in logits[:L].argmax(-1):
+    if i != prev and i != 0: out.append(tokens[int(i)])
+    prev = i
+print("".join(out).replace("|", " ").strip())
+```
+
+## Minimal usage — Kotlin (Android)
+
+```kotlin
+val frontend = CompiledModel.create(frontendPath, CompiledModel.Options(Accelerator.GPU), null)
+val head = CompiledModel.create(headPath, CompiledModel.Options(Accelerator.GPU), null)
+val fIn = frontend.createInputBuffers(); val fOut = frontend.createOutputBuffers()
+val hIn = head.createInputBuffers(); val hOut = head.createOutputBuffers()
+
+fIn[0].writeFloat(pcm)                        // [-1,1] floats, zero-padded to 256000
+frontend.run(fIn, fOut)
+hIn[0].writeFloat(fOut[0].readFloat())        // features [1,799,768]
+head.run(hIn, hOut)
+val logits = hOut[0].readFloat()              // [799 * 32], readback syncs the GPU
+// greedy CTC over the valid frames: argmax per frame, drop blanks (id 0) + repeats,
+// map through tokens.txt, '|' -> space
+```
+
+## On-device performance (Pixel 8a, CompiledModel GPU)
+
+- frontend 448 ms + head 391 ms per 16 s window (RTF ≈ 0.05); GPU compile 0.7 s + 1.5 s.
+- Device logits vs desktop float reference: corr 0.9928 (valid region), per-frame argmax
+  agreement 97.0 %; transcript matches the desktop reference.
+
+## Conversion notes
+
+Converted with litert-torch, numerically exact (tflite vs PyTorch: corr 1.000000):
+GELU → tanh-GELU; frontend GroupNorm → 4D-reshape group-norm (avoids GATHER_ND);
+pos_conv weight-norm folded to a static weight; the all-valid bidirectional attention mask
+removed (fixed window → plain SDPA). The CTC head is a plain Linear — logits come out raw.
+
+## Sources & license
+
+- Model: [facebook/wav2vec2-base-960h](https://huggingface.co/facebook/wav2vec2-base-960h) — Apache-2.0.
+- Paper: [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://arxiv.org/abs/2006.11477).
+- Hero audio: J.F. Kennedy inaugural address (1961), U.S. National Archives — public domain.