Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+higgs_audio_tokenizer_architecture.png filter=lfs diff=lfs merge=lfs -text
+fp16/semantic_encoder.onnx.data filter=lfs diff=lfs merge=lfs -text
+fp16/acoustic_encoder.onnx.data filter=lfs diff=lfs merge=lfs -text
+fp16/higgs_decoder.onnx.data filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,212 @@
+---
+license: other
+library_name: onnx
+base_model:
+- bosonai/higgs-audio-v2-tokenizer
+- eustlb/higgs-audio-v2-tokenizer
+---
+
+# Higgs Audio Tokenizer
+
+<div align="center" style="display: flex; justify-content: center; margin-top: 10px; flex-wrap: wrap; gap: 8px;">
+  <a href="https://boson.ai/blog/higgs-audio-v2"><img src='https://img.shields.io/badge/🚀-Launch Blogpost-228B22' style="margin-right: 5px;"></a>
+  <a href="https://github.com/boson-ai/higgs-audio"><img src="https://img.shields.io/badge/💻-Github%20Repo-9C276A" style="margin-right: 5px;"></a>
+</div>
+
+Check our open-source repository https://github.com/boson-ai/higgs-audio for more details!
+
+
+We introduce a new discretized audio tokenizer that runs at just **25 frames per second** while keeping—or even improving—audio quality compared to tokenizers with twice the bitrate. Our model is the first to train on **24 kHz data** covering speech, music, and sound events in one unified system. It also uses a simple non-diffusion encoder/decoder for fast, batch inference.
+
+<p>
+    <img src="./higgs_audio_tokenizer_architecture.png" alt="Architecture diagram of the Higgs Audio Tokenizer" width=900>
+</p>
+
+## Usage
+
+### Transformers 🤗
+
+Higgs Audio V2 Tokenizer is supported natively in `transformers`: [see the doc](https://huggingface.co/docs/transformers/en/model_doc/higgs_audio_v2_tokenizer).
+
+```bash
+uv pip install "transformers>=5.3.0"
+```
+
+### Onnx
+
+The model is divided into 4 submodels :
+
+- `account_encoder.onnx`
+- `higgs_decoder.onnx`
+- `quantizer_encoder.onnx`
+- `semantic_encoder.onnx`
+
+```bash
+python higgs_inference.py --input speech.wav --output reconstructed.wav
+```
+
+## Basics of Audio Quantization
+
+An audio signal sampled at \\(f_s\\) Hz is first split into frames by an encoder with hop size \\(M\\), giving a frame rate \\(f_r = \frac{f_s}{M}\quad\text{(frames/s)}.\\)
+Two common quantizers are:
+
+- **Residual Vector Quantization (RVQ)**: \\(N_q\\) cascaded vector‑quantizer layers, each with codebook size \\(N_{cb}\\). When \\(N_{q}=1\\), it degenerates to ordinary vector quantization.  
+- **Finite Scalar Quantization (FSQ)**: A single-layer scalar quantizer in which
+  every scalar coefficient is independently mapped to one of \\(N_{cb}\\) discrete levels.
+
+
+If every combination of codewords is a token, the vocabulary size is \\(N_{cb}^{N_q}\\), and each token needs \\(N_q\log_2 N_{cb}\\) bits. The overall bitrate (bits/s, BPS) is simply \\(f_r \times N_q \log_2 N_{cb}\\).  
+We aim to push this bitrate as low as possible without hurting audio fidelity.
+
+## What Makes Ours Better
+
+- **Low Frame Rate**:  Runs at just 25 fps, halving the frame rate of many baselines while preserving high audio quality.
+- **Unified 24 kHz Training**: A single model jointly trained on speech, music, and sound‑event data, capturing both semantic and acoustic nuances and greatly simplifying downstream audio‑language‑model training.
+- **Fast Inference**: A non‑diffusion encoder/decoder that processes batches quickly, making it practical for real-time or large-scale tasks.
+
+
+## Evaluation Data and Metrics
+
+We test on four subsets:
+
+- **Speech, Music and Sound Event**: Include 1,000 clips per category, with each clip lasting 10 seconds. Clips are randomly sampled from [DAPS](https://ccrma.stanford.edu/~gautham/Site/daps.html) (Speech), [MUSDB](https://sigsep.github.io/datasets/musdb.html) (Music), and [AudioSet](https://research.google.com/audioset/index.html) (Sound Event).
+
+- **Audiophile**: Contains 150 clips, each 30 seconds long, curated from eleven high-fidelity test discs that were designed for perceptual listening tests. The clips feature both high-quality music and sound events.
+
+We measure:
+
+- **Acoustic Quality**: Acoustic reconstruction error between the original and reconstructed audio.
+- **Semantic Integrity**: Degree of semantic preservation, evaluated on the English and Chinese subsets of [SeedTTS](https://arxiv.org/abs/2406.02430)[15]. 
+- **Aesthetics**: SOTA unified model-based quality metrics computed with [Meta Audiobox Aesthetics](https://github.com/facebookresearch/audiobox-aesthetics)[8]. 
+
+
+We compare our tokenizer with a wide range of baselines, from tokenizers mainly built for better acoustic reconstruction and compression rate, to those focused on semantic integrity, and to tokenizers used in existing large audio language models. We also compare with tokenizers that are pretrained specifically on speech or on music.
+
+
+The tables below summarize the tokenizers evaluated. As shown, our tokenizer achieves a well-rounded balance of efficiency, semantic fidelity, and acoustic quality.
+
+### Acoustic Evaluation
+
+This table reports the Short‑Time Fourier Transform (STFT) distance between the original and reconstructed audio. Baselines are listed chronologically and grouped by whether semantic distillation (SD) is applied. Despite DAC’s top acoustic quality at 12× the bitrate, our tokenizer leads all other baselines.
+
+
+| Tokenizer | 💬 | 🎵 | 🥁 | SD | \\(f_s\\) | \\(f_r\\) | BPS<sup>*</sup> (k) ↓ | Speech ↓ | Sound Event ↓ | Music ↓ | Audiophile ↓ |
+|-----------|----|----|----|----|-------|-------|--------------------------|----------|----------------|--------|--------------|
+| [Encodec](https://huggingface.co/facebook/encodec_24khz)[3] | ✓ | ✓ | ✓ |  | 24 | 75 | 24 | 1.96 | 2.65 | 2.52 | 2.30 |
+| [DAC](https://huggingface.co/hance-ai/descript-audio-codec-24khz)[2] | ✓ | ✓ | ✓ |  | 24 | 75 | 24 | **1.13** | **1.45** | **1.34** | **1.62** |
+| [SNAC-24k](https://huggingface.co/hubertsiuzdak/snac_24khz)[6] | ✓ |  |  |  | 24 | (12, 23, 47) | 0.98 | 1.92 | 2.69 | 2.54 | 2.52 |
+| [SNAC-44k](https://huggingface.co/hubertsiuzdak/snac_44khz)[6] |  | ✓ | ✓ |  | 44.1 | (14, 29, 57, 115) | 2.6 | 1.83 | 2.25 | 2.05 | 2.00 |
+| [WavTokenizer](https://huggingface.co/novateur/WavTokenizer-medium-music-audio-75token/blob/main/wavtokenizer_medium_music_audio_320_24k_v2.ckpt)[7] |  | ✓ | ✓ |  | 24 | 75 | 0.9 | 1.93 | 2.44 | 2.17 | 2.15 |
+| [WavTokenizer (Speech)](https://huggingface.co/novateur/WavTokenizer-large-speech-75token/tree/main)[7] | ✓ |  |  |  | 24 | 75 | 0.9 | 1.78 | 2.47 | 2.42 | 2.47 |
+| [MuCodec](https://huggingface.co/haoheliu/audioldm_48k/tree/main)[11] |  | ✓ |  |  | 48 | 25 | 0.35 | 2.87 | 3.69 | 3.36 | 2.97 |
+| [FlowDec-75m](https://github.com/facebookresearch/FlowDec?tab=readme-ov-file)[12] | ✓ | ✓ | ✓ |  | 48 | 75 | 7.5 | 1.73 | 2.14 | 2.01 | 2.03 |
+| [FlowDec-25s](https://github.com/facebookresearch/FlowDec?tab=readme-ov-file)[12] | ✓ | ✓ | ✓ |  | 48 | 25 | 4 | 1.94 | 2.42 | 2.25 | 2.33 |
+| [SpeechTokenizer](https://huggingface.co/fnlp/SpeechTokenizer/tree/main/speechtokenizer_hubert_avg)[14] | ✓ |  |  | ✓ | 16 | 50 | 4 | 3.21 | 3.58 | 3.65 | 3.69 |
+| [SemantiCodec](https://huggingface.co/haoheliu/SemantiCodec/tree/main/semanticodec_tokenrate_100)[5] | ✓ | ✓ | ✓ | ✓ | 16 | 100 | 1.35 | 3.05 | 3.28 | 3.24 | 3.18 |
+| [Mimi](https://huggingface.co/docs/transformers/en/model_doc/mimi)[13] | ✓ |  |  | ✓ | 24 | 12.5 | 4.4 | 1.77 | 2.40 | 2.30 | 2.15 |
+| [XCodec](https://huggingface.co/ZhenYe234/xcodec/blob/main/config_hubert_general.yaml)[1] | ✓ | ✓ | ✓ | ✓ | 16 | 50 | 4 | 2.95 | 3.16 | 3.00 | 3.03 |
+| [CosyVoice 2](https://huggingface.co/FunAudioLLM/CosyVoice2-0.5B)[13] | ✓ |  |  | ✓ | 16 | 25 | -<sup>**</sup>| 2.30 | 3.30 | 3.14 | 3.25 |
+| [XCodec2](https://huggingface.co/HKUST-Audio/xcodec2/blob/main/ckpt/epoch%3D4-step%3D1400000.ckpt)[9] | ✓ |  |  | ✓ | 16 | 50 | 0.8 | 3.06 | 3.72 | 3.62 | 3.64 |
+| [XY](https://huggingface.co/fnlp/XY_Tokenizer_TTSD_V0/tree/main)[10] | ✓ |  |  | ✓ | 24 | 12.5 | 1 | 1.89 | 2.51 | 2.40 | 2.26 |
+| Ours | ✓ | ✓ | ✓ | ✓ | 24 | 25 | 2 | **1.62** | **2.03** | **1.85** | **1.80** |
+
+
+
+<sup>* Bits-per-second is calculated according to the checkpoint the author provided.</sup> 
+
+<sup>** CosyVoice 2 uses the continuous feature as the conditioning; we include it for completeness.</sup> 
+
+
+### Semantic Evaluation
+[SeedTTS](https://github.com/BytedanceSpeech/seed-tts-eval) is a dataset that includes prompt/target audio and texts. We reconstruct the target audio, and use the word error rate (WER) and speaker similarity (SIM) metrics to evaluate the semantic integrity. SIM is calculated by the similarity between the prompt audio and reconstructed target audio with [WavLM-large](https://drive.google.com/file/d/1-aE1NfzpRCLxA4GUxX9ITI3F9LlbtEGP/view) as the embedding model. 
+
+The following table compares our tokenizer with semantic-distillation-trained baselines and shows that it delivers performance comparable to tokenizers operating at 2.2× our model’s bitrate.
+
+| Model | BPS (k) | en WER ↓ | en SIM ↑ | zh WER ↓ | zh SIM ↑ |
+|------------------|---------|------------|------------|------------|------------|
+| [SpeechTokenizer](https://huggingface.co/fnlp/SpeechTokenizer/tree/main/speechtokenizer_hubert_avg) | 4 | 2.82 | 0.63 | 2.04 | 0.65 |
+| [SemantiCodec](https://huggingface.co/haoheliu/SemantiCodec/tree/main/semanticodec_tokenrate_100) | 1.35 | 3.46 | 0.56 | 2.18 | 0.60 |
+| [Mimi](https://huggingface.co/docs/transformers/en/model_doc/mimi) | 4.4 | **2.35** | **0.70** | **1.48** | **0.72** |
+| [XCodec](https://huggingface.co/ZhenYe234/xcodec/blob/main/config_hubert_general.yaml) | 4.0 | 2.68 | 0.63 | 1.66 | 0.66 |
+| [CosyVoice 2](https://huggingface.co/FunAudioLLM/CosyVoice2-0.5B) | - | 3.17 | 0.65 | 2.11 | 0.70 |
+| [XCodec2](https://huggingface.co/HKUST-Audio/xcodec2/blob/main/ckpt/epoch%3D4-step%3D1400000.ckpt) | 0.8 | 2.74 | 0.62 | 1.91 | 0.67 |
+| [XY-MOSS-TTSD](https://huggingface.co/fnlp/XY_Tokenizer_TTSD_V0/tree/main) | 1.0 | 2.72 | 0.61 | 1.58 | 0.67 |
+| Ours | 2.0 | 2.52 | 0.67 | **1.48** | 0.71 |
+
+
+
+### Audiobox Aesthetics Evaluation
+
+This model-based evaluation[8] further demonstrates the superiority of our tokenizer. CU denotes the Content Usefulness and CE denotes the Content Enjoyment; both are rated on a 1-10 scale. Notably, our tokenizer performs best on the Audiophile set, demonstrating a clear advantage when the original audio quality is high.
+
+
+| Model | BPS (k) | Music CE ↑ | Music CU ↑ | Sound Event CE ↑ | Sound Event CU ↑ | Speech CE ↑ | Speech CU ↑ | Audiophile CE ↑ | Audiophile CU ↑ |
+|------------------|---------|--------------|--------------|--------------------|--------------------|---------------|---------------|--------------------|--------------------|
+| Origin | - | 6.20 | 7.10 | 4.47 | 5.64 | 5.03 | 4.87 | 7.17 | 7.65 |
+| [SpeechTokenizer](https://huggingface.co/fnlp/SpeechTokenizer/tree/main/speechtokenizer_hubert_avg) | 4.0 | 3.55 | 5.22 | 3.03 | 4.50 | 4.68 | 4.58 | 3.59 | 5.07 |
+| [SemantiCodec](https://huggingface.co/haoheliu/SemantiCodec/tree/main/semanticodec_tokenrate_100) | 1.35 | 6.01 | 6.83 | 4.22 | 5.30 | 4.28 | 4.12 | 6.97 | 7.43 |
+| [Mimi](https://huggingface.co/docs/transformers/en/model_doc/mimi) | 4.4 | 6.01 | 6.83 | 4.26 | 5.35 | 4.87 | 4.72 | 6.80 | 7.29 |
+| [XCodec](https://huggingface.co/ZhenYe234/xcodec/blob/main/config_hubert_general.yaml) | 4.0 | **6.30** | **7.10** | **4.43** | 5.45 | **4.96** | **4.79** | 7.06 | 7.49 |
+| [CosyVoice 2](https://huggingface.co/FunAudioLLM/CosyVoice2-0.5B) | - | 5.21 | 6.14 | 4.08 | 4.73 | **4.91** | **4.75** | 5.97 | 6.56 |
+| [XCodec2](https://huggingface.co/HKUST-Audio/xcodec2/blob/main/ckpt/epoch%3D4-step%3D1400000.ckpt) | 0.8 | 4.38 | 5.66 | 3.43 | 4.63 | **4.93** | **4.78** | 4.56 | 5.46 |
+| [XY-MOSS-TTSD](https://huggingface.co/fnlp/XY_Tokenizer_TTSD_V0/tree/main) | 1.0 | 5.77 | 6.80 | 4.23 | 5.34 | 4.88 | 4.72 | 6.95 | 7.48 |
+| Ours | 2.0 | **6.35** | **7.15** | **4.47** | **5.51** | 4.90 | 4.70 | **7.21** | **7.66** |
+
+
+
+Note that since some tokenizers are trained on 16 kHz data, we upsample their audio outputs to 24 kHz before computing metrics. Different upsampling methods may cause slight variations (e.g., 4.36 vs. 4.43 for XCodec Sound Event CE). We report the best results we could obtain and highlight any results within 0.05 of the best one. 
+
+
+
+
+
+
+
+<!-- xcodec [1]
+dac [2]
+encodec [3]
+moshi [4]
+semanticodec [5]
+snac [6]
+wavtokenizer [7]
+xcodec [8]
+xcodec2 [9]
+xy-tokenizer [10]
+mucodec [11]
+flowdec [12]
+cosyvoice2 [13]
+speechtokenizer [14] -->
+
+
+
+
+## Reference 
+[1] [Ye, Zhen, et al. "Codec does matter: Exploring the semantic shortcoming of codec for audio language model." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 24. 2025.](https://arxiv.org/abs/2408.17175)
+
+[2] [Kumar, Rithesh, et al. "High-fidelity audio compression with improved rvqgan." Advances in Neural Information Processing Systems 36 (2023): 27980-27993.](https://dl.acm.org/doi/10.5555/3666122.3667336)
+
+[3] [Défossez, Alexandre, et al. "High fidelity neural audio compression." arXiv preprint arXiv:2210.13438 (2022).](https://arxiv.org/abs/2210.13438)
+
+[4] [Défossez, Alexandre, et al. "Moshi: a speech-text foundation model for real-time dialogue." arXiv preprint arXiv:2410.00037 (2024).](https://arxiv.org/abs/2410.00037)
+
+[5] [Liu, Haohe, et al. "Semanticodec: An ultra low bitrate semantic audio codec for general sound." IEEE Journal of Selected Topics in Signal Processing (2024).](https://ieeexplore.ieee.org/document/10768970)
+
+[6] [Siuzdak, Hubert, Florian Grötschla, and Luca A. Lanzendörfer. "Snac: Multi-scale neural audio codec." arXiv preprint arXiv:2410.14411 (2024).](https://arxiv.org/abs/2410.14411)
+
+[7] [Ji, Shengpeng, et al. "Wavtokenizer: an efficient acoustic discrete codec tokenizer for audio language modeling." arXiv preprint arXiv:2408.16532 (2024).](https://arxiv.org/abs/2408.16532)
+
+[8] [Tjandra, Andros, et al. "Meta audiobox aesthetics: Unified automatic quality assessment for speech, music, and sound." arXiv preprint arXiv:2502.05139 (2025).](https://arxiv.org/abs/2502.05139)
+
+[9] [Ye, Zhen, et al. "Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesis." arXiv preprint arXiv:2502.04128 (2025).](https://arxiv.org/abs/2502.04128)
+
+[10] [Gong, Yitian, et al. "XY-Tokenizer: Mitigating the Semantic-Acoustic Conflict in Low-Bitrate Speech Codecs." arXiv preprint arXiv:2506.23325 (2025).](https://arxiv.org/abs/2506.23325)
+
+[11] [Xu, Yaoxun, et al. "MuCodec: Ultra Low-Bitrate Music Codec." arXiv preprint arXiv:2409.13216 (2024).](https://arxiv.org/abs/2409.13216)
+
+[12] [Welker, Simon, et al. "FlowDec: A flow-based full-band general audio codec with high perceptual quality." arXiv preprint arXiv:2503.01485 (2025).](https://arxiv.org/abs/2503.01485)
+
+[13] [Du, Zhihao, et al. "Cosyvoice 2: Scalable streaming speech synthesis with large language models." arXiv preprint arXiv:2412.10117 (2024).](https://arxiv.org/abs/2412.10117)
+
+[14] [Zhang, Xin, et al. "Speechtokenizer: Unified speech tokenizer for speech large language models." arXiv preprint arXiv:2308.16692 (2023).](https://arxiv.org/abs/2308.16692)
+
+[15] [Anastassiou, Philip, et al. "Seed-tts: A family of high-quality versatile speech generation models." arXiv preprint arXiv:2406.02430 (2024).](https://arxiv.org/abs/2406.02430)

acoustic_encoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fbc3a4c81c7753ace690104b320d9ef41eababc50196084a7b2c3bd2e291f16
+size 205546480

fp16/acoustic_encoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e24702e1a492d619fe7d1fe0c7cd3d530eba8144f5da826630e4dd97fec1b5fc
+size 295846

fp16/acoustic_encoder.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3725792aa9cc9ef9779955a648c87444f3d700a067c7fdd68cf006e26992073
+size 102615039

fp16/higgs_decoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f2e1ba8ede642099a4151d0d0762eebf9ba996a5821a2205005afc0a2e4c820a
+size 308494

fp16/higgs_decoder.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5be7ff51d117dd1cd001b87ec6e6e93f61cbe923355c50841a15fcaf19f64a19
+size 43100160

fp16/model_config.json

@@ -0,0 +1,12 @@
+{
+    "type": "onnxmodel",
+    "config": {
+        "model_path": "D:\\Projects\\olive-recipes\\OmniVoice\\higgs\\models",
+        "onnx_file_name": "higgs_decoder.onnx",
+        "inference_settings": null,
+        "use_ort_extensions": false,
+        "external_initializers_file_name": null,
+        "constant_inputs_file_name": null,
+        "model_attributes": null
+    }
+}

fp16/quantizer_encoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3c1fd5d644b67ef532022f5f9558a6a30b495be92013918d33ce2f6be3d8f87c
+size 6091791

fp16/semantic_encoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c24e3d4de75911dcb25adf3a993c9fd5689dfb4c97df02dcc82e58ab82ee30d7
+size 334295

fp16/semantic_encoder.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90080ded4fe1e847a9b924feea365ad2e9c04fa93569d34e6d6fa7218ee9a238
+size 218235902

higgs_decoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:601a69b103986d47a91c21bcffc9c8eef8aa462d0267256efff75033fa25cdff
+size 86500102

higgs_inference.py

@@ -0,0 +1,392 @@
+"""
+Standalone Higgs Audio V2 Tokenizer — ONNX inference.
+
+Encodes audio to discrete codec codes and decodes codes back to audio,
+using the four exported ONNX sub-models:
+
+  acoustic_encoder.onnx   — DAC encoder:   (1,1,T_24k) → (1,256,T_frames)
+  semantic_encoder.onnx   — HuBERT encoder: (1,T_16k)  → (1,768,T_frames)
+  quantizer_encoder.onnx  — RVQ encode:    (1,256,T)+(1,768,T) → (8,1,T)
+  higgs_decoder.onnx      — DAC decode:    (8,1,T) → (1,1,T_24k)
+
+No PyTorch, no transformers required at runtime — only onnxruntime.
+
+Usage:
+  # Round-trip encode → decode (sanity check)
+  python higgs_inference.py --input speech.wav --output reconstructed.wav
+
+  # Encode only → save codes as .npy
+  python higgs_inference.py --input speech.wav --encode-only --codes-out codes.npy
+
+  # Decode only from saved codes
+  python higgs_inference.py --decode-only --codes-in codes.npy --output decoded.wav
+
+  # Use a different models directory
+  python higgs_inference.py --models-dir path/to/higgs/models --input speech.wav --output out.wav
+"""
+
+import argparse
+import sys
+from pathlib import Path
+
+import numpy as np
+
+# ---------------------------------------------------------------------------
+# Constants (from audio_tokenizer/config.json)
+# ---------------------------------------------------------------------------
+SR_24K        = 24_000          # acoustic DAC sample rate
+SR_16K        = 16_000          # HuBERT semantic sample rate
+HOP_LENGTH    = 960             # DAC downsampling factor: product of [8,5,4,2,3]
+D_ACOUSTIC    = 256             # acoustic encoder output channels
+D_SEMANTIC    = 768             # semantic encoder output channels
+N_CODEBOOKS   = 8               # RVQ codebooks
+CODEBOOK_SIZE = 1024            # entries per codebook
+DEFAULT_MODELS_DIR = Path(__file__).parent / "higgs" / "models"
+
+
+# =============================================================================
+# Audio I/O
+# =============================================================================
+
+def load_wav(path: str, target_sr: int) -> np.ndarray:
+    """Load an audio file, resample to target_sr, mix to mono.
+
+    Returns float32 array in [-1, 1], shape (T,).
+    Tries torchaudio first (accurate), falls back to scipy.
+    """
+    path = str(path)
+
+    # --- torchaudio (preferred) ---
+    try:
+        import torchaudio, torch
+        wav, sr = torchaudio.load(path)
+        if sr != target_sr:
+            wav = torchaudio.functional.resample(wav, sr, target_sr)
+        if wav.shape[0] > 1:
+            wav = wav.mean(0, keepdim=True)
+        return wav.squeeze(0).numpy().astype(np.float32)
+    except ImportError:
+        pass
+
+    # --- scipy fallback ---
+    from scipy.io import wavfile
+    from scipy.signal import resample_poly
+    from math import gcd
+    sr, data = wavfile.read(path)
+    if data.dtype == np.int16:
+        data = data.astype(np.float32) / 32768.0
+    elif data.dtype == np.int32:
+        data = data.astype(np.float32) / 2147483648.0
+    elif data.dtype != np.float32:
+        data = data.astype(np.float32)
+    if data.ndim > 1:
+        data = data.mean(axis=1)
+    if sr != target_sr:
+        g = gcd(target_sr, sr)
+        data = resample_poly(data, target_sr // g, sr // g).astype(np.float32)
+    return data
+
+
+def save_wav(path: str, waveform: np.ndarray, sr: int = SR_24K):
+    """Save float32 numpy waveform as 16-bit PCM WAV."""
+    path = str(path)
+    try:
+        import soundfile as sf
+        sf.write(path, waveform, sr, subtype="PCM_16")
+        return
+    except ImportError:
+        pass
+    from scipy.io import wavfile
+    pcm = np.clip(waveform, -1.0, 1.0)
+    wavfile.write(path, sr, (pcm * 32767).astype(np.int16))
+
+
+# =============================================================================
+# ONNX session loader
+# =============================================================================
+
+class HiggsOnnxSessions:
+    """Holds the four Higgs ONNX InferenceSession objects."""
+
+    MODEL_FILES = {
+        "acoustic_encoder":  "acoustic_encoder.onnx",
+        "semantic_encoder":  "semantic_encoder.onnx",
+        "quantizer_encoder": "quantizer_encoder.onnx",
+        "higgs_decoder":     "higgs_decoder.onnx",
+    }
+
+    def __init__(self, models_dir: str, provider: str = "CPUExecutionProvider"):
+        import onnxruntime as ort
+        models_dir = Path(models_dir)
+        opts = ort.SessionOptions()
+        opts.log_severity_level = 3   # suppress INFO / WARNING noise
+
+        self.sessions = {}
+        for key, filename in self.MODEL_FILES.items():
+            p = models_dir / filename
+            if not p.exists():
+                raise FileNotFoundError(
+                    f"Missing: {p}\n"
+                    f"Export Higgs models first:\n"
+                    f"  cd OmniVoice && python optimize.py --higgs-only"
+                )
+            self.sessions[key] = ort.InferenceSession(
+                str(p), sess_options=opts, providers=[provider]
+            )
+        print(f"Loaded {len(self.sessions)} Higgs ONNX sessions from {models_dir}")
+
+    def __getitem__(self, key):
+        return self.sessions[key]
+
+
+# =============================================================================
+# dtype helper — cast float32 inputs to whatever the ONNX session expects
+# =============================================================================
+
+_ORT_TO_NP = {
+    "tensor(float16)": np.float16,
+    "tensor(float)":   np.float32,
+    "tensor(double)":  np.float64,
+    "tensor(int64)":   np.int64,
+    "tensor(int32)":   np.int32,
+}
+
+
+def _cast_feed(feed: dict, sess) -> dict:
+    """Auto-cast numpy arrays to the dtype expected by the ONNX session.
+
+    OnnxFloatToFloat16 converts all float inputs/outputs to float16. Feeding
+    float32 to such a model raises INVALID_ARGUMENT. This inspects the session
+    input metadata and casts each value to the declared type.
+    """
+    type_map = {inp.name: _ORT_TO_NP.get(inp.type) for inp in sess.get_inputs()}
+    out = {}
+    for k, v in feed.items():
+        tgt = type_map.get(k)
+        if tgt is not None and isinstance(v, np.ndarray) and v.dtype != tgt:
+            v = v.astype(tgt)
+        out[k] = v
+    return out
+
+
+# =============================================================================
+# Encode: audio → codec codes
+# =============================================================================
+
+def encode(sessions: HiggsOnnxSessions, wav_path: str) -> np.ndarray:
+    """Encode an audio file to RVQ codec codes.
+
+    Pipeline:
+      wav_path (24 kHz) → acoustic_encoder → (1, 256, T_a)
+      wav_path (16 kHz) → semantic_encoder → (1, 768, T_s)
+      align T_s → T_a   (linear interpolation along time axis)
+      concat channels   → quantizer_encoder → (8, 1, T_a)  int64
+
+    Returns:
+      codes: np.ndarray  shape (8, 1, T_a)  dtype int64
+      T_a is the number of codec frames (25 per second of 24 kHz audio).
+    """
+    # 1. Load audio at both sample rates
+    wav24 = load_wav(wav_path, SR_24K)   # (T_24k,)
+    wav16 = load_wav(wav_path, SR_16K)   # (T_16k,)
+
+    dur = len(wav24) / SR_24K
+    print(f"  Audio: {dur:.2f}s  "
+          f"({len(wav24)} samples @ {SR_24K} Hz, "
+          f"{len(wav16)} samples @ {SR_16K} Hz)")
+
+    # 2. Acoustic encoder  → (1, 256, T_a)
+    waveform_24k = wav24[None, None, :]          # (1, 1, T_24k)
+    _ae_sess = sessions["acoustic_encoder"]
+    acoustic_feat = _ae_sess.run(
+        ["acoustic_features"],
+        _cast_feed({"waveform_24k": waveform_24k}, _ae_sess)
+    )[0]                                          # (1, 256, T_a)
+    T_a = acoustic_feat.shape[2]
+    print(f"  acoustic_encoder → {acoustic_feat.shape}  ({T_a} frames, {T_a/dur:.1f} fps)")
+
+    # 3. Semantic encoder  → (1, 768, T_s)
+    waveform_16k = wav16[None, :]                # (1, T_16k)
+    _se_sess = sessions["semantic_encoder"]
+    semantic_feat = _se_sess.run(
+        ["semantic_features"],
+        _cast_feed({"waveform_16k": waveform_16k}, _se_sess)
+    )[0]                                          # (1, 768, T_s)
+    T_s = semantic_feat.shape[2]
+    print(f"  semantic_encoder → {semantic_feat.shape}  ({T_s} frames, {T_s/dur:.1f} fps)")
+
+    # 4. Frame alignment check
+    #    The fixed semantic_encoder wrapper applies semantic_downsample_factor=2 and
+    #    the (160,160) padding internally, matching _extract_semantic_features() exactly.
+    #    T_s == T_a naturally for typical audio. Trim on rare off-by-one edge cases.
+    if T_s != T_a:
+        T = min(T_a, T_s)
+        acoustic_feat = acoustic_feat[:, :, :T]
+        semantic_feat = semantic_feat[:, :, :T]
+        print(f"  [warn] T_a={T_a} != T_s={T_s}, trimmed both to T={T}")
+
+    # 5. Quantizer encoder  → (8, 1, T_a)  int64
+    _qe_sess = sessions["quantizer_encoder"]
+    codes = _qe_sess.run(
+        ["codes"],
+        _cast_feed({
+            "acoustic_features": acoustic_feat,    # (1, 256, T_a)
+            "semantic_features":  semantic_feat,   # (1, 768, T_a)
+        }, _qe_sess)
+    )[0]                                           # (8, 1, T_a)
+    print(f"  quantizer_encoder → {codes.shape}  "
+          f"({N_CODEBOOKS} codebooks × {T_a} frames)  "
+          f"value range [{codes.min()}, {codes.max()}]")
+
+    return codes
+
+
+# =============================================================================
+# Decode: codec codes → audio
+# =============================================================================
+
+def decode(sessions: HiggsOnnxSessions, codes: np.ndarray) -> np.ndarray:
+    """Decode RVQ codec codes back to a waveform.
+
+    Args:
+      codes: np.ndarray  shape (8, 1, T_frames)  dtype int64
+
+    Returns:
+      waveform: np.ndarray  shape (T_samples,)  float32  at 24 kHz
+    """
+    if codes.ndim != 3 or codes.shape[0] != N_CODEBOOKS:
+        raise ValueError(
+            f"codes must have shape (8, 1, T_frames), got {codes.shape}"
+        )
+    if codes.dtype != np.int64:
+        codes = codes.astype(np.int64)
+
+    T_frames = codes.shape[2]
+    expected_dur = T_frames * HOP_LENGTH / SR_24K
+
+    _hd_sess = sessions["higgs_decoder"]
+    waveform = _hd_sess.run(
+        ["waveform_24k"],
+        _cast_feed({"codes": codes}, _hd_sess)
+    )[0]                          # (1, 1, T_samples)
+
+    waveform = waveform.squeeze()  # (T_samples,)
+    actual_dur = len(waveform) / SR_24K
+    print(f"  higgs_decoder → {waveform.shape}  "
+          f"({actual_dur:.2f}s  expected {expected_dur:.2f}s)")
+
+    return waveform
+
+
+# =============================================================================
+# Convenience: round-trip encode + decode
+# =============================================================================
+
+def encode_decode(sessions: HiggsOnnxSessions, wav_path: str) -> tuple:
+    """Encode audio to codes then decode back to waveform.
+
+    Returns:
+      codes:    np.ndarray (8, 1, T_frames) int64
+      waveform: np.ndarray (T_samples,)     float32
+    """
+    print("[Encode]")
+    codes = encode(sessions, wav_path)
+    print("[Decode]")
+    waveform = decode(sessions, codes)
+    return codes, waveform
+
+
+# =============================================================================
+# CLI
+# =============================================================================
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Higgs Audio V2 Tokenizer — standalone ONNX encode/decode"
+    )
+    parser.add_argument(
+        "--models-dir", default=str(DEFAULT_MODELS_DIR),
+        help=f"Higgs ONNX models directory (default: {DEFAULT_MODELS_DIR})"
+    )
+    parser.add_argument(
+        "--input", "-i", default=None,
+        help="Input audio file to encode (.wav or any format torchaudio/scipy supports)"
+    )
+    parser.add_argument(
+        "--output", "-o", default="output.wav",
+        help="Output WAV file for decoded audio (default: output.wav)"
+    )
+    parser.add_argument(
+        "--encode-only", action="store_true",
+        help="Only encode — save codes to --codes-out, do not decode"
+    )
+    parser.add_argument(
+        "--decode-only", action="store_true",
+        help="Only decode — load codes from --codes-in, skip encoding"
+    )
+    parser.add_argument(
+        "--codes-out", default="codes.npy",
+        help="Path to save encoded codes as .npy (default: codes.npy)"
+    )
+    parser.add_argument(
+        "--codes-in", default=None,
+        help="Path to load codes .npy for --decode-only mode"
+    )
+    parser.add_argument(
+        "--cuda", action="store_true",
+        help="Use CUDAExecutionProvider instead of CPU"
+    )
+    args = parser.parse_args()
+
+    provider = "CUDAExecutionProvider" if args.cuda else "CPUExecutionProvider"
+
+    # Validate argument combinations
+    if args.encode_only and args.decode_only:
+        parser.error("--encode-only and --decode-only are mutually exclusive")
+    if not args.decode_only and args.input is None:
+        parser.error("--input is required unless --decode-only is used")
+    if args.decode_only and args.codes_in is None:
+        parser.error("--codes-in is required when using --decode-only")
+
+    # Load ONNX sessions
+    sessions = HiggsOnnxSessions(args.models_dir, provider)
+
+    # -------------------------------------------------------------------------
+    if args.encode_only:
+        print(f"\nEncoding: {args.input}")
+        codes = encode(sessions, args.input)
+        np.save(args.codes_out, codes)
+        print(f"\nSaved codes → {args.codes_out}  shape={codes.shape}  dtype={codes.dtype}")
+
+    # -------------------------------------------------------------------------
+    elif args.decode_only:
+        print(f"\nLoading codes from: {args.codes_in}")
+        codes = np.load(args.codes_in)
+        print(f"  codes shape={codes.shape}  dtype={codes.dtype}")
+        print(f"\nDecoding...")
+        waveform = decode(sessions, codes)
+        save_wav(args.output, waveform, SR_24K)
+        print(f"Saved WAV → {args.output}  ({len(waveform)/SR_24K:.2f}s @ {SR_24K} Hz)")
+
+    # -------------------------------------------------------------------------
+    else:
+        # Full round-trip
+        print(f"\nRound-trip encode → decode")
+        print(f"  Input : {args.input}")
+        print(f"  Output: {args.output}")
+        codes, waveform = encode_decode(sessions, args.input)
+
+        # Optionally save codes
+        np.save(args.codes_out, codes)
+        print(f"\nSaved codes → {args.codes_out}")
+
+        save_wav(args.output, waveform, SR_24K)
+        dur_in  = len(load_wav(args.input, SR_24K)) / SR_24K
+        dur_out = len(waveform) / SR_24K
+        print(f"Saved WAV → {args.output}  ({dur_out:.2f}s, input was {dur_in:.2f}s)")
+        print(f"\nReconstruction note: codec compression (RVQ, 8 codebooks × 1024 entries)")
+        print(f"at 25 fps introduces mild quality loss — this is expected.")
+
+
+if __name__ == "__main__":
+    main()

model_config.json

@@ -0,0 +1,12 @@
+{
+    "type": "onnxmodel",
+    "config": {
+        "model_path": "D:\\Projects\\olive-recipes\\OmniVoice\\higgs\\models\\higgs_decoder.onnx",
+        "onnx_file_name": null,
+        "inference_settings": null,
+        "use_ort_extensions": false,
+        "external_initializers_file_name": null,
+        "constant_inputs_file_name": null,
+        "model_attributes": null
+    }
+}

quantizer_encoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4592311271a745da259513a33ae248f12dabc811e7942fcc0e0635da8b02ec37
+size 12131293

semantic_encoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e3be47442600c7e5a29faa39b7f73b940243974848089b2ebe9717f83c1c7df
+size 436736856