VibeVoice-ASR-Streaming-7B AWQ W4A16 with its DFlash 2 drafter
Browse files- LICENSE +21 -0
- README.md +61 -0
- added_tokens.json +28 -0
- calibrated_decoder_config.json +99 -0
- compression_report.json +31 -0
- config.json +169 -0
- drafter/config.json +105 -0
- drafter/model.safetensors +3 -0
- evaluation.json +470 -0
- merges.txt +0 -0
- model-00001.safetensors +3 -0
- model-00002.safetensors +3 -0
- model.safetensors.index.json +0 -0
- preprocessor_config.json +8 -0
- quantization_environment.json +9 -0
- recipe.yaml +26 -0
- reproduce/README.template.md +101 -0
- reproduce/evaluate.py +58 -0
- reproduce/evaluate_c.py +42 -0
- reproduce/export_awq.py +74 -0
- reproduce/finalize.py +89 -0
- reproduce/pipeline.sh +40 -0
- reproduce/prepare_data.py +111 -0
- reproduce/prepare_model.py +122 -0
- reproduce/quantize.py +53 -0
- reproduce/stream_ref.py +84 -0
- sha256.json +4 -0
- special_tokens_map.json +56 -0
- tokenizer.json +0 -0
- tokenizer_config.json +242 -0
- vocab.json +0 -0
LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2025 Microsoft
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 6 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 7 |
+
in the Software without restriction, including without limitation the rights
|
| 8 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 9 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 10 |
+
furnished to do so, subject to the following conditions:
|
| 11 |
+
|
| 12 |
+
The above copyright notice and this permission notice shall be included in all
|
| 13 |
+
copies or substantial portions of the Software.
|
| 14 |
+
|
| 15 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 16 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 17 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 18 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 19 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 20 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 21 |
+
SOFTWARE.
|
README.md
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
library_name: vibevoice.c
|
| 4 |
+
base_model: Ar4ikov/VibeVoice-ASR-Streaming-7B-AWQ-W4A16-ASYM
|
| 5 |
+
tags:
|
| 6 |
+
- speculative-decoding
|
| 7 |
+
- dflash
|
| 8 |
+
- awq
|
| 9 |
+
- compressed-tensors
|
| 10 |
+
- speech-recognition
|
| 11 |
+
- vibevoice
|
| 12 |
+
pipeline_tag: automatic-speech-recognition
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# VibeVoice-ASR-Streaming-7B-AWQ-W4A16-ASYM-DFlash2
|
| 16 |
+
|
| 17 |
+
**VibeVoice-ASR-Streaming-7B**, AWQ W4A16 (asymmetric, groups of 128), **with its
|
| 18 |
+
[DFlash 2](https://inco.ai/blog/dflash2/) drafter bundled** in `drafter/`:
|
| 19 |
+
one download, and [vibevoice.c](https://github.com/Ar4ikov/vibevoice.c)
|
| 20 |
+
decodes with speculative decoding -- the drafter proposes 8 tokens in one
|
| 21 |
+
pass, the model checks them in one pass and keeps the ones it agrees with.
|
| 22 |
+
**The check is exact**: every checked row is computed with the arithmetic of
|
| 23 |
+
the model's own decode step, so the transcript is byte-for-byte the one
|
| 24 |
+
without the drafter.
|
| 25 |
+
|
| 26 |
+
## Use
|
| 27 |
+
|
| 28 |
+
Needs vibevoice.c with DFlash 2 support: branch `dflash2`
|
| 29 |
+
([PR #48](https://github.com/Ar4ikov/vibevoice.c/pull/48)), in the next
|
| 30 |
+
release. A model directory's `drafter/` is used without asking:
|
| 31 |
+
|
| 32 |
+
```bash
|
| 33 |
+
vv_cli --model ./VibeVoice-ASR-Streaming-7B-AWQ-W4A16-ASYM-DFlash2 --audio talk.wav # with the drafter
|
| 34 |
+
vv_cli --model ./VibeVoice-ASR-Streaming-7B-AWQ-W4A16-ASYM-DFlash2 --audio talk.wav --draft none # plain decoding
|
| 35 |
+
vv_cli serve --model ./VibeVoice-ASR-Streaming-7B-AWQ-W4A16-ASYM-DFlash2 --slots 4 # streaming sessions (WebSocket, SSE) too
|
| 36 |
+
```
|
| 37 |
+
|
| 38 |
+
## Results
|
| 39 |
+
|
| 40 |
+
vibevoice.c `b72be15` (branch `dflash2`), RTX 3090, greedy decoding, decode tokens per second:
|
| 41 |
+
|
| 42 |
+
| | plain | drafted | speedup | tokens per block | same transcript |
|
| 43 |
+
|---|---|---|---|---|---|
|
| 44 |
+
| 20 held-out clips, 8 rows | 149 tok/s | 364 tok/s | **2.44x** | 3.57 | 20/20 |
|
| 45 |
+
|
| 46 |
+
Streaming sessions (22 + 4 frames a chunk). Plain = the same model with `--draft none`; `--draft-check exact`.
|
| 47 |
+
|
| 48 |
+
## Inside
|
| 49 |
+
|
| 50 |
+
* The model: the files of [Ar4ikov/VibeVoice-ASR-Streaming-7B-AWQ-W4A16-ASYM](https://huggingface.co/Ar4ikov/VibeVoice-ASR-Streaming-7B-AWQ-W4A16-ASYM)
|
| 51 |
+
at revision `1cc2b627`, unchanged (7.01 GB) -- its card has the
|
| 52 |
+
quantization, the calibration and the WER.
|
| 53 |
+
* `drafter/`: [Ar4ikov/VibeVoice-ASR-Streaming-7B-DFlash2-Drafter-AWQ-W4A16-ASYM](https://huggingface.co/Ar4ikov/VibeVoice-ASR-Streaming-7B-DFlash2-Drafter-AWQ-W4A16-ASYM) at revision
|
| 54 |
+
`65e803b7` (0.55 GB): 5 Qwen3-style layers reading the model's
|
| 55 |
+
layers 1/7/13/19/25, a candidate selector, a 32768-id draft vocabulary; its
|
| 56 |
+
projections stored as INT4 (compressed-tensors `pack-quantized`). Its card
|
| 57 |
+
has the architecture and the training.
|
| 58 |
+
|
| 59 |
+
## License
|
| 60 |
+
|
| 61 |
+
MIT, like VibeVoice.
|
added_tokens.json
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"<|endoftext|>": 151643,
|
| 3 |
+
"<|im_start|>": 151644,
|
| 4 |
+
"<|im_end|>": 151645,
|
| 5 |
+
"<|object_ref_start|>": 151646,
|
| 6 |
+
"<|object_ref_end|>": 151647,
|
| 7 |
+
"<|box_start|>": 151648,
|
| 8 |
+
"<|box_end|>": 151649,
|
| 9 |
+
"<|quad_start|>": 151650,
|
| 10 |
+
"<|quad_end|>": 151651,
|
| 11 |
+
"<|vision_start|>": 151652,
|
| 12 |
+
"<|vision_end|>": 151653,
|
| 13 |
+
"<|vision_pad|>": 151654,
|
| 14 |
+
"<|image_pad|>": 151655,
|
| 15 |
+
"<|video_pad|>": 151656,
|
| 16 |
+
"<tool_call>": 151657,
|
| 17 |
+
"</tool_call>": 151658,
|
| 18 |
+
"<|fim_prefix|>": 151659,
|
| 19 |
+
"<|fim_middle|>": 151660,
|
| 20 |
+
"<|fim_suffix|>": 151661,
|
| 21 |
+
"<|fim_pad|>": 151662,
|
| 22 |
+
"<|repo_name|>": 151663,
|
| 23 |
+
"<|file_sep|>": 151664,
|
| 24 |
+
"<|text_chunk_end|>": 151665,
|
| 25 |
+
"<|AUDIO|>": 151666,
|
| 26 |
+
"<|audio_bos|>": 151667,
|
| 27 |
+
"<|audio_eos|>": 151668
|
| 28 |
+
}
|
calibrated_decoder_config.json
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"Qwen2ForCausalLM"
|
| 4 |
+
],
|
| 5 |
+
"attention_dropout": 0.0,
|
| 6 |
+
"bos_token_id": null,
|
| 7 |
+
"dtype": "bfloat16",
|
| 8 |
+
"eos_token_id": null,
|
| 9 |
+
"hidden_act": "silu",
|
| 10 |
+
"hidden_size": 3584,
|
| 11 |
+
"initializer_range": 0.02,
|
| 12 |
+
"intermediate_size": 18944,
|
| 13 |
+
"layer_types": [
|
| 14 |
+
"full_attention",
|
| 15 |
+
"full_attention",
|
| 16 |
+
"full_attention",
|
| 17 |
+
"full_attention",
|
| 18 |
+
"full_attention",
|
| 19 |
+
"full_attention",
|
| 20 |
+
"full_attention",
|
| 21 |
+
"full_attention",
|
| 22 |
+
"full_attention",
|
| 23 |
+
"full_attention",
|
| 24 |
+
"full_attention",
|
| 25 |
+
"full_attention",
|
| 26 |
+
"full_attention",
|
| 27 |
+
"full_attention",
|
| 28 |
+
"full_attention",
|
| 29 |
+
"full_attention",
|
| 30 |
+
"full_attention",
|
| 31 |
+
"full_attention",
|
| 32 |
+
"full_attention",
|
| 33 |
+
"full_attention",
|
| 34 |
+
"full_attention",
|
| 35 |
+
"full_attention",
|
| 36 |
+
"full_attention",
|
| 37 |
+
"full_attention",
|
| 38 |
+
"full_attention",
|
| 39 |
+
"full_attention",
|
| 40 |
+
"full_attention",
|
| 41 |
+
"full_attention"
|
| 42 |
+
],
|
| 43 |
+
"max_position_embeddings": 131072,
|
| 44 |
+
"max_window_layers": 28,
|
| 45 |
+
"model_type": "qwen2",
|
| 46 |
+
"num_attention_heads": 28,
|
| 47 |
+
"num_hidden_layers": 28,
|
| 48 |
+
"num_key_value_heads": 4,
|
| 49 |
+
"pad_token_id": null,
|
| 50 |
+
"quantization_config": {
|
| 51 |
+
"config_groups": {
|
| 52 |
+
"group_0": {
|
| 53 |
+
"format": "pack-quantized",
|
| 54 |
+
"input_activations": null,
|
| 55 |
+
"output_activations": null,
|
| 56 |
+
"targets": [
|
| 57 |
+
"Linear"
|
| 58 |
+
],
|
| 59 |
+
"weights": {
|
| 60 |
+
"actorder": null,
|
| 61 |
+
"block_structure": null,
|
| 62 |
+
"dynamic": false,
|
| 63 |
+
"group_size": 128,
|
| 64 |
+
"num_bits": 4,
|
| 65 |
+
"observer": "memoryless_minmax",
|
| 66 |
+
"observer_kwargs": {},
|
| 67 |
+
"scale_dtype": null,
|
| 68 |
+
"strategy": "group",
|
| 69 |
+
"symmetric": false,
|
| 70 |
+
"type": "int",
|
| 71 |
+
"zp_dtype": "torch.int8"
|
| 72 |
+
}
|
| 73 |
+
}
|
| 74 |
+
},
|
| 75 |
+
"format": "pack-quantized",
|
| 76 |
+
"global_compression_ratio": null,
|
| 77 |
+
"ignore": [
|
| 78 |
+
"lm_head"
|
| 79 |
+
],
|
| 80 |
+
"kv_cache_scheme": null,
|
| 81 |
+
"quant_method": "compressed-tensors",
|
| 82 |
+
"quantization_status": "compressed",
|
| 83 |
+
"sparsity_config": {},
|
| 84 |
+
"transform_config": {},
|
| 85 |
+
"version": "0.18.0"
|
| 86 |
+
},
|
| 87 |
+
"rms_norm_eps": 1e-06,
|
| 88 |
+
"rope_parameters": {
|
| 89 |
+
"rope_theta": 1000000.0,
|
| 90 |
+
"rope_type": "default"
|
| 91 |
+
},
|
| 92 |
+
"sliding_window": null,
|
| 93 |
+
"tie_word_embeddings": false,
|
| 94 |
+
"transformers_version": "5.14.1",
|
| 95 |
+
"use_cache": true,
|
| 96 |
+
"use_mrope": false,
|
| 97 |
+
"use_sliding_window": false,
|
| 98 |
+
"vocab_size": 152064
|
| 99 |
+
}
|
compression_report.json
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"source_revision": "60d858b518b4e19d404af3737f848fc185b30177",
|
| 3 |
+
"source_safetensors_bytes": 17348198658,
|
| 4 |
+
"output_safetensors_bytes": 7000335408,
|
| 5 |
+
"compression_ratio": 2.478195350207711,
|
| 6 |
+
"quantized_projections": 196,
|
| 7 |
+
"method": "Activation-aware AWQ using llm-compressor; exact layout repack into AWQ GEMM",
|
| 8 |
+
"lm_head": "BF16",
|
| 9 |
+
"scheme": "W4A16_ASYM",
|
| 10 |
+
"group_size": 128,
|
| 11 |
+
"duo_scaling": "both",
|
| 12 |
+
"grid_points": 40,
|
| 13 |
+
"calibration_samples": 256,
|
| 14 |
+
"audio_calibration_samples": 128,
|
| 15 |
+
"text_calibration_samples": 128,
|
| 16 |
+
"calibration_stats": {
|
| 17 |
+
"audio_sessions": 128,
|
| 18 |
+
"audio_rows_mean": 944.7734375,
|
| 19 |
+
"audio_rows_total": 120931,
|
| 20 |
+
"truncated_at_2048": 0
|
| 21 |
+
},
|
| 22 |
+
"max_sequence_length": 2048,
|
| 23 |
+
"packing_validation": "All integer weight and zero-point codes round-trip exactly; all scale values preserved exactly",
|
| 24 |
+
"pruning": {
|
| 25 |
+
"removed_prefixes": [
|
| 26 |
+
"model.acoustic_tokenizer.decoder."
|
| 27 |
+
],
|
| 28 |
+
"removed_bf16_bytes": 687391938,
|
| 29 |
+
"speech_bytes": 1429418752
|
| 30 |
+
}
|
| 31 |
+
}
|
config.json
ADDED
|
@@ -0,0 +1,169 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"acoustic_tokenizer_config": {
|
| 3 |
+
"causal": true,
|
| 4 |
+
"channels": 1,
|
| 5 |
+
"conv_bias": true,
|
| 6 |
+
"conv_norm": "none",
|
| 7 |
+
"corpus_normalize": 0.0,
|
| 8 |
+
"decoder_depths": null,
|
| 9 |
+
"decoder_n_filters": 32,
|
| 10 |
+
"decoder_ratios": [
|
| 11 |
+
8,
|
| 12 |
+
5,
|
| 13 |
+
5,
|
| 14 |
+
4,
|
| 15 |
+
2,
|
| 16 |
+
2
|
| 17 |
+
],
|
| 18 |
+
"disable_last_norm": true,
|
| 19 |
+
"dtype": "bfloat16",
|
| 20 |
+
"encoder_depths": "3-3-3-3-3-3-8",
|
| 21 |
+
"encoder_n_filters": 32,
|
| 22 |
+
"encoder_ratios": [
|
| 23 |
+
8,
|
| 24 |
+
5,
|
| 25 |
+
5,
|
| 26 |
+
4,
|
| 27 |
+
2,
|
| 28 |
+
2
|
| 29 |
+
],
|
| 30 |
+
"fix_std": 0.5,
|
| 31 |
+
"layer_scale_init_value": 1e-06,
|
| 32 |
+
"layernorm": "RMSNorm",
|
| 33 |
+
"layernorm_elementwise_affine": true,
|
| 34 |
+
"layernorm_eps": 1e-05,
|
| 35 |
+
"mixer_layer": "depthwise_conv",
|
| 36 |
+
"model_type": "vibevoice_acoustic_tokenizer",
|
| 37 |
+
"pad_mode": "constant",
|
| 38 |
+
"std_dist_type": "gaussian",
|
| 39 |
+
"torch_dtype": "bfloat16",
|
| 40 |
+
"vae_dim": 64,
|
| 41 |
+
"weight_init_value": 0.01
|
| 42 |
+
},
|
| 43 |
+
"acoustic_vae_dim": 64,
|
| 44 |
+
"architectures": [
|
| 45 |
+
"VibeVoiceForASRStreamingTraining"
|
| 46 |
+
],
|
| 47 |
+
"decoder_config": {
|
| 48 |
+
"attention_dropout": 0.0,
|
| 49 |
+
"dtype": "bfloat16",
|
| 50 |
+
"hidden_act": "silu",
|
| 51 |
+
"hidden_size": 3584,
|
| 52 |
+
"initializer_range": 0.02,
|
| 53 |
+
"intermediate_size": 18944,
|
| 54 |
+
"layer_types": [
|
| 55 |
+
"full_attention",
|
| 56 |
+
"full_attention",
|
| 57 |
+
"full_attention",
|
| 58 |
+
"full_attention",
|
| 59 |
+
"full_attention",
|
| 60 |
+
"full_attention",
|
| 61 |
+
"full_attention",
|
| 62 |
+
"full_attention",
|
| 63 |
+
"full_attention",
|
| 64 |
+
"full_attention",
|
| 65 |
+
"full_attention",
|
| 66 |
+
"full_attention",
|
| 67 |
+
"full_attention",
|
| 68 |
+
"full_attention",
|
| 69 |
+
"full_attention",
|
| 70 |
+
"full_attention",
|
| 71 |
+
"full_attention",
|
| 72 |
+
"full_attention",
|
| 73 |
+
"full_attention",
|
| 74 |
+
"full_attention",
|
| 75 |
+
"full_attention",
|
| 76 |
+
"full_attention",
|
| 77 |
+
"full_attention",
|
| 78 |
+
"full_attention",
|
| 79 |
+
"full_attention",
|
| 80 |
+
"full_attention",
|
| 81 |
+
"full_attention",
|
| 82 |
+
"full_attention"
|
| 83 |
+
],
|
| 84 |
+
"max_position_embeddings": 131072,
|
| 85 |
+
"max_window_layers": 28,
|
| 86 |
+
"model_type": "qwen2",
|
| 87 |
+
"num_attention_heads": 28,
|
| 88 |
+
"num_hidden_layers": 28,
|
| 89 |
+
"num_key_value_heads": 4,
|
| 90 |
+
"rms_norm_eps": 1e-06,
|
| 91 |
+
"rope_scaling": null,
|
| 92 |
+
"rope_theta": 1000000.0,
|
| 93 |
+
"sliding_window": null,
|
| 94 |
+
"torch_dtype": "bfloat16",
|
| 95 |
+
"use_cache": true,
|
| 96 |
+
"use_mrope": false,
|
| 97 |
+
"use_sliding_window": false,
|
| 98 |
+
"vocab_size": 152064
|
| 99 |
+
},
|
| 100 |
+
"diffusion_head_config": {
|
| 101 |
+
"ddpm_batch_mul": 4,
|
| 102 |
+
"ddpm_beta_schedule": "cosine",
|
| 103 |
+
"ddpm_num_inference_steps": 20,
|
| 104 |
+
"ddpm_num_steps": 1000,
|
| 105 |
+
"diffusion_type": "ddpm",
|
| 106 |
+
"head_ffn_ratio": 3.0,
|
| 107 |
+
"head_layers": 4,
|
| 108 |
+
"hidden_size": 3584,
|
| 109 |
+
"latent_size": 64,
|
| 110 |
+
"model_type": "vibepod_diffusion_head",
|
| 111 |
+
"prediction_type": "v_prediction",
|
| 112 |
+
"rms_norm_eps": 1e-05,
|
| 113 |
+
"speech_vae_dim": 64
|
| 114 |
+
},
|
| 115 |
+
"dtype": "bfloat16",
|
| 116 |
+
"model_type": "vibevoice",
|
| 117 |
+
"semantic_tokenizer_config": {
|
| 118 |
+
"causal": true,
|
| 119 |
+
"channels": 1,
|
| 120 |
+
"conv_bias": true,
|
| 121 |
+
"conv_norm": "none",
|
| 122 |
+
"corpus_normalize": 0.0,
|
| 123 |
+
"disable_last_norm": true,
|
| 124 |
+
"dtype": "bfloat16",
|
| 125 |
+
"encoder_depths": "3-3-3-3-3-3-8",
|
| 126 |
+
"encoder_n_filters": 32,
|
| 127 |
+
"encoder_ratios": [
|
| 128 |
+
8,
|
| 129 |
+
5,
|
| 130 |
+
5,
|
| 131 |
+
4,
|
| 132 |
+
2,
|
| 133 |
+
2
|
| 134 |
+
],
|
| 135 |
+
"fix_std": 0,
|
| 136 |
+
"layer_scale_init_value": 1e-06,
|
| 137 |
+
"layernorm": "RMSNorm",
|
| 138 |
+
"layernorm_elementwise_affine": true,
|
| 139 |
+
"layernorm_eps": 1e-05,
|
| 140 |
+
"mixer_layer": "depthwise_conv",
|
| 141 |
+
"model_type": "vibevoice_semantic_tokenizer",
|
| 142 |
+
"pad_mode": "constant",
|
| 143 |
+
"std_dist_type": "none",
|
| 144 |
+
"torch_dtype": "bfloat16",
|
| 145 |
+
"vae_dim": 128,
|
| 146 |
+
"weight_init_value": 0.01
|
| 147 |
+
},
|
| 148 |
+
"semantic_vae_dim": 128,
|
| 149 |
+
"torch_dtype": "bfloat16",
|
| 150 |
+
"transformers_version": "4.51.3",
|
| 151 |
+
"use_semantic_feature": true,
|
| 152 |
+
"speech_tok_compress_ratio": 3200,
|
| 153 |
+
"target_sample_rate": 24000,
|
| 154 |
+
"tie_word_embeddings": false,
|
| 155 |
+
"quantization_config": {
|
| 156 |
+
"quant_method": "awq",
|
| 157 |
+
"bits": 4,
|
| 158 |
+
"group_size": 128,
|
| 159 |
+
"version": "gemm",
|
| 160 |
+
"zero_point": true,
|
| 161 |
+
"modules_to_not_convert": [
|
| 162 |
+
"lm_head",
|
| 163 |
+
"model.acoustic_tokenizer",
|
| 164 |
+
"model.semantic_tokenizer",
|
| 165 |
+
"model.acoustic_connector",
|
| 166 |
+
"model.semantic_connector"
|
| 167 |
+
]
|
| 168 |
+
}
|
| 169 |
+
}
|
drafter/config.json
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"DFlash2DraftModel"
|
| 4 |
+
],
|
| 5 |
+
"model_type": "qwen3",
|
| 6 |
+
"attention_bias": false,
|
| 7 |
+
"hidden_act": "silu",
|
| 8 |
+
"hidden_size": 3584,
|
| 9 |
+
"intermediate_size": 9472,
|
| 10 |
+
"num_hidden_layers": 5,
|
| 11 |
+
"num_attention_heads": 28,
|
| 12 |
+
"num_key_value_heads": 4,
|
| 13 |
+
"head_dim": 128,
|
| 14 |
+
"rms_norm_eps": 1e-06,
|
| 15 |
+
"rope_theta": 1000000.0,
|
| 16 |
+
"max_position_embeddings": 131072,
|
| 17 |
+
"vocab_size": 152064,
|
| 18 |
+
"num_target_layers": 28,
|
| 19 |
+
"is_causal": false,
|
| 20 |
+
"layer_types": [
|
| 21 |
+
"full_attention",
|
| 22 |
+
"full_attention",
|
| 23 |
+
"full_attention",
|
| 24 |
+
"full_attention",
|
| 25 |
+
"full_attention"
|
| 26 |
+
],
|
| 27 |
+
"sliding_window": null,
|
| 28 |
+
"tie_word_embeddings": false,
|
| 29 |
+
"dtype": "bfloat16",
|
| 30 |
+
"dflash_config": {
|
| 31 |
+
"block_size": 8,
|
| 32 |
+
"mask_token_id": 151662,
|
| 33 |
+
"target_layer_ids": [
|
| 34 |
+
1,
|
| 35 |
+
7,
|
| 36 |
+
13,
|
| 37 |
+
19,
|
| 38 |
+
25
|
| 39 |
+
],
|
| 40 |
+
"conv_kernel_size": 2,
|
| 41 |
+
"conv_group_size": 16,
|
| 42 |
+
"selector_rank": 256,
|
| 43 |
+
"selector_top_k": 16,
|
| 44 |
+
"loss_decay_gamma": 4.0,
|
| 45 |
+
"draft_vocab_size": 32768
|
| 46 |
+
},
|
| 47 |
+
"quantization_config": {
|
| 48 |
+
"quant_method": "compressed-tensors",
|
| 49 |
+
"format": "pack-quantized",
|
| 50 |
+
"quantization_status": "compressed",
|
| 51 |
+
"config_groups": {
|
| 52 |
+
"group_0": {
|
| 53 |
+
"targets": [
|
| 54 |
+
"Linear"
|
| 55 |
+
],
|
| 56 |
+
"weights": {
|
| 57 |
+
"num_bits": 4,
|
| 58 |
+
"type": "int",
|
| 59 |
+
"symmetric": false,
|
| 60 |
+
"strategy": "group",
|
| 61 |
+
"group_size": 128,
|
| 62 |
+
"dynamic": false,
|
| 63 |
+
"observer": "minmax"
|
| 64 |
+
},
|
| 65 |
+
"input_activations": null,
|
| 66 |
+
"output_activations": null
|
| 67 |
+
}
|
| 68 |
+
},
|
| 69 |
+
"ignore": [
|
| 70 |
+
"candidate_selector.hidden_projection"
|
| 71 |
+
],
|
| 72 |
+
"kv_cache_scheme": null
|
| 73 |
+
},
|
| 74 |
+
"vv_awq": {
|
| 75 |
+
"calibration_traces": 48,
|
| 76 |
+
"blocks_per_trace": 64,
|
| 77 |
+
"folds": [
|
| 78 |
+
"post_attention_layernorm->gate,up",
|
| 79 |
+
"up->down",
|
| 80 |
+
"v->o"
|
| 81 |
+
],
|
| 82 |
+
"clip_ratios": [
|
| 83 |
+
1.0,
|
| 84 |
+
0.95,
|
| 85 |
+
0.9,
|
| 86 |
+
0.85,
|
| 87 |
+
0.8,
|
| 88 |
+
0.75,
|
| 89 |
+
0.7,
|
| 90 |
+
0.6499999999999999,
|
| 91 |
+
0.6,
|
| 92 |
+
0.55
|
| 93 |
+
],
|
| 94 |
+
"eval": {
|
| 95 |
+
"bf16": {
|
| 96 |
+
"base_acc": 3.399057190275065,
|
| 97 |
+
"sel_acc": 3.5175240889610047
|
| 98 |
+
},
|
| 99 |
+
"awq_int4": {
|
| 100 |
+
"base_acc": 3.3990571862680303,
|
| 101 |
+
"sel_acc": 3.511477768873186
|
| 102 |
+
}
|
| 103 |
+
}
|
| 104 |
+
}
|
| 105 |
+
}
|
drafter/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:36204f52589350ae3b31bc3046cccf56061ac712f7f1349f97ad1adcdec22518
|
| 3 |
+
size 548986496
|
evaluation.json
ADDED
|
@@ -0,0 +1,470 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"dataset": "LibriSpeech dev-clean, 8 held-out speakers",
|
| 3 |
+
"streams": 12,
|
| 4 |
+
"minutes": 9.669754166666667,
|
| 5 |
+
"two_speaker_streams": 4,
|
| 6 |
+
"reference_words": 1579,
|
| 7 |
+
"normalization": "Uppercase ASCII words, 'Speaker N:' labels and punctuation removed",
|
| 8 |
+
"word_errors": {
|
| 9 |
+
"baseline": 43,
|
| 10 |
+
"awq_pytorch": 39,
|
| 11 |
+
"c_bf16": 44,
|
| 12 |
+
"c_awq": 40
|
| 13 |
+
},
|
| 14 |
+
"wer": {
|
| 15 |
+
"baseline": 0.027232425585813805,
|
| 16 |
+
"awq_pytorch": 0.024699176694110196,
|
| 17 |
+
"c_bf16": 0.02786573780873971,
|
| 18 |
+
"c_awq": 0.0253324889170361
|
| 19 |
+
},
|
| 20 |
+
"held_out_speakers": [
|
| 21 |
+
"174",
|
| 22 |
+
"2086",
|
| 23 |
+
"2277",
|
| 24 |
+
"2412",
|
| 25 |
+
"5338",
|
| 26 |
+
"5536",
|
| 27 |
+
"6313",
|
| 28 |
+
"7976"
|
| 29 |
+
],
|
| 30 |
+
"samples_detail": [
|
| 31 |
+
{
|
| 32 |
+
"wav": "/mnt/ssd0/vv3/awq-stream/data/eval/eval-000.wav",
|
| 33 |
+
"reference": "I EXPRESSED BY SIGNS MY ADMIRATION AND PLEASURE TO MY GUIDES AND THEY WERE GREATLY PLEASED THE COUNTRY WAS HIGHLY CULTIVATED EVERY LEDGE BEING PLANTED WITH CHESTNUTS WALNUTS AND APPLE TREES FROM WHICH THE APPLES WERE NOW GATHERING I SAW A FEW SHEEP WITH ROUNDED NOSES AND ENORMOUS TAILS IN ABOUT FOUR HOURS OF WALKING FROM THE TIME WE STARTED AND AFTER PASSING TWO OR THREE MORE VILLAGES WE CAME UPON A CONSIDERABLE TOWN AND MY GUIDES MADE MANY ATTEMPTS TO MAKE ME UNDERSTAND SOMETHING BUT I GATHERED NO INKLING OF THEIR MEANING EXCEPT THAT I NEED BE UNDER NO APPREHENSION OF DANGER SUFFICE IT THAT I FOUND MYSELF TAKEN BEFORE THE CHIEF MAGISTRATE AND BY HIS ORDERS WAS PLACED IN AN APARTMENT WITH TWO OTHER PEOPLE WHO WERE THE FIRST I HAD SEEN LOOKING ANYTHING BUT WELL AND HANDSOME IN FACT ONE OF THEM WAS PLAINLY VERY MUCH OUT OF HEALTH AND COUGHED VIOLENTLY FROM TIME TO TIME IN SPITE OF MANIFEST EFFORTS TO SUPPRESS IT",
|
| 34 |
+
"baseline": " \n Speaker 0:I expressed by signs my admiration and pleasure to my guides, and they were greatly pleased. The country was highly cultivated, every ledge being planted with chestnuts, walnuts and apple trees, from which the apples were now gathering. I saw a few sheep with rounded noses and enormous tails. In about four hours of walking from the time we started and after passing two or three more villages, we came upon a considerable town and my guides made many attempts to make me understand something. \n Speaker 0:But I gathered no inkling of their meaning, except that I need be under no apprehension of danger. Suffice it that I found myself taken before the chief magistrate and by his orders was placed in an apartment with two other people. Who were the first I had seen looking anything but well handsome. In fact, one of them was plainly very much out of health and coughed violently from time to time in spite of manifest efforts to suppress it.",
|
| 35 |
+
"awq": " \n Speaker 0:I expressed by signs my admiration and pleasure to my guides, and they were greatly pleased. The country was highly cultivated, every ledge being planted with chestnuts, walnuts and apple trees, from which the apples were now gathering. I saw a few sheep with rounded noses and enormous tails. In about four hours of walking from the time we started and after passing two or three more villages, we came upon a considerable town and my guides made many attempts to make me understand something. \n Speaker 0:But I gathered no inkling of their meaning, except that I need be under no apprehension of danger. Suffice it that I found myself taken before the chief magistrate and by his orders was placed in an apartment with two other people.Who were the first I had seen looking anything but well handsome. In fact, one of them was plainly very much out of health and coughed violently from time to time in spite of manifest efforts to suppress it.",
|
| 36 |
+
"awq_chunks": [
|
| 37 |
+
" \n Speaker 0:I expressed by signs my admiration and pleasure to ",
|
| 38 |
+
"my guides, and they were greatly pleased. ",
|
| 39 |
+
"The country was highly cultivated, every ledge ",
|
| 40 |
+
"being planted with chestnuts, walnuts and apple trees, ",
|
| 41 |
+
"from which the apples were now gathering. ",
|
| 42 |
+
"I saw a few sheep with rounded noses and enormous ",
|
| 43 |
+
"tails. In about four hours of ",
|
| 44 |
+
"walking from the time we started and after passing two or three ",
|
| 45 |
+
"more villages, we came upon a considerable town and ",
|
| 46 |
+
"my guides made many attempts to make me understand something.",
|
| 47 |
+
" \n Speaker 0:But I gathered no inkling of their ",
|
| 48 |
+
"meaning, except that I need be under no apprehension ",
|
| 49 |
+
"of danger. Suffice it that I found ",
|
| 50 |
+
"myself taken before the chief magistrate and by ",
|
| 51 |
+
"his orders was placed in an apartment with two other people.",
|
| 52 |
+
"Who were the first I had seen looking anything but well ",
|
| 53 |
+
"handsome. In fact, one of them ",
|
| 54 |
+
"was plainly very much out of health and coughed ",
|
| 55 |
+
"violently from time to time in spite of manifest efforts to ",
|
| 56 |
+
"suppress it."
|
| 57 |
+
],
|
| 58 |
+
"generation_seconds": 6.309504985809326,
|
| 59 |
+
"c_awq": "I expressed by signs my admiration and pleasure to my guides, and they were greatly pleased. The country was highly cultivated, every ledge being planted with chestnuts, walnuts and apple trees, from which the apples were now gathering. I saw a few sheep with rounded noses and enormous tails. In about four hours of walking from the time we started and after passing two or three more villages, we came upon a considerable town and my guides made many attempts to make me understand something. But I gathered no inkling of their meaning, except that I need be under no apprehension of danger. Sufficed it that I found myself taken before the chief magistrate and by his orders was placed in an apartment with two other people.Who were the first I had seen looking anything but well handsome. In fact, one of them was plainly very much out of health and coughed violently from time to time in spite of manifest efforts to suppress it.",
|
| 60 |
+
"c_bf16": "I expressed by signs my admiration and pleasure to my guides, and they were greatly pleased. The country was highly cultivated, every ledge being planted with chestnuts, walnuts and apple trees, from which the apples were now gathering. I saw a few sheep with rounded noses and enormous tails. In about four hours of walking from the time we started and after passing two or three more villages, we came upon a considerable town and my guides made many attempts to make me understand something. But I gathered no inkling of their meaning, except that I need be under no apprehension of danger. Suffice it that I found myself taken before the chief magistrate and by his orders was placed in an apartment with two other people.Who were the first I had seen looking anything but well handsome. In fact, one of them was plainly very much out of health and coughed violently from time to time in spite of manifest efforts to suppress it."
|
| 61 |
+
},
|
| 62 |
+
{
|
| 63 |
+
"wav": "/mnt/ssd0/vv3/awq-stream/data/eval/eval-001.wav",
|
| 64 |
+
"reference": "THEIR EXPRESSION WAS DIVINE AND AS THEY GLANCED AT ME TIMIDLY BUT WITH PARTED LIPS IN GREAT BEWILDERMENT I FORGOT ALL THOUGHTS OF THEIR CONVERSION IN FEELINGS THAT WERE FAR MORE EARTHLY EVEN IN MIDDLE AGE THEY WERE STILL COMELY AND THE OLD GREY HAIRED WOMEN AT THEIR COTTAGE DOORS HAD A DIGNITY NOT TO SAY MAJESTY OF THEIR OWN THE MEN WERE AS HANDSOME AS THE WOMEN BEAUTIFUL I HAVE ALWAYS DELIGHTED IN AND REVERENCED BEAUTY BUT I FELT SIMPLY ABASHED IN THE PRESENCE OF SUCH A SPLENDID TYPE A COMPOUND OF ALL THAT IS BEST IN EGYPTIAN GREEK AND ITALIAN THE CHILDREN WERE INFINITE IN NUMBER AND EXCEEDINGLY MERRY I NEED HARDLY SAY THAT THEY CAME IN FOR THEIR FULL SHARE OF THE PREVAILING BEAUTY I EXPRESSED BY SIGNS MY ADMIRATION AND PLEASURE TO MY GUIDES AND THEY WERE GREATLY PLEASED THE COUNTRY WAS HIGHLY CULTIVATED EVERY LEDGE BEING PLANTED WITH CHESTNUTS WALNUTS AND APPLE TREES FROM WHICH THE APPLES WERE NOW GATHERING",
|
| 65 |
+
"baseline": " \n Speaker 0:Their expression was divine, and as they glanced at me timidly, but with parted lips and great bewilderment, I forgot all thoughts of their conversion and feelings that were far more earthly. Even in middle age, they were still comely, and the old gray-haired women at their cottage doors had a dignity, not to say majesty, of their own. The men were as handsome as the women beautiful. I have always delighted in and reverenced beauty, but I felt simply abashed in the presence of such a splendid type. \n Speaker 0:A compound of all that is best in Egyptian, Greek, and Italian. The children were infinite in number, and exceedingly merry. I need hardly say that they came in for their full share of the prevailing beauty. I expressed by signs my admiration and pleasure to my guides, and they were greatly pleased. The country was highly cultivated, every ledge being planted with chestnuts, walnuts, and apple trees from which the apples were now gathering.",
|
| 66 |
+
"awq": " \n Speaker 0:Their expression was divine, and as they glanced at me timidly, but with parted lips and great bewilderment, I forgot all thoughts of their conversion and feelings that were far more earthly. Even in middle age, they were still comely, and the old gray-haired women at their cottage doors had a dignity, not to say majesty, of their own. The men were as handsome as the women beautiful. I have always delighted in and reverenced beauty, but I felt simply abashed in the presence of such a splendid type. \n Speaker 0:A compound of all that is best in Egyptian, Greek, and Italian. The children were infinite in number, and exceedingly merry. I need hardly say that they came in for their full share of the prevailing beauty. I expressed by signs my admiration and pleasure to my guides, and they were greatly pleased. The country was highly cultivated, every ledge being planted with chestnuts, walnuts, and apple trees from which the apples were now gathering.",
|
| 67 |
+
"awq_chunks": [
|
| 68 |
+
" \n Speaker 0:Their expression was divine, and as they glanced at ",
|
| 69 |
+
"me timidly, but with parted lips and great bewilderment, ",
|
| 70 |
+
"I forgot all thoughts of their conversion and feelings that were ",
|
| 71 |
+
"far more earthly. Even in middle age, they ",
|
| 72 |
+
"were still comely, and the old gray-haired women at their ",
|
| 73 |
+
"cottage doors had a dignity, not to say ",
|
| 74 |
+
"majesty, of their own. The men were as handsome ",
|
| 75 |
+
"as the women beautiful. I have always delighted ",
|
| 76 |
+
"in and reverenced beauty, but I felt simply ",
|
| 77 |
+
"abashed in the presence of such a splendid type. \n Speaker 0:A ",
|
| 78 |
+
"compound of all that is best in Egyptian, Greek, and Italian. ",
|
| 79 |
+
"The children were infinite in number, and ",
|
| 80 |
+
"exceedingly merry. I need hardly say that they came ",
|
| 81 |
+
"in for their full share of the prevailing beauty. ",
|
| 82 |
+
"I expressed by signs my admiration and pleasure to ",
|
| 83 |
+
"my guides, and they were greatly pleased. ",
|
| 84 |
+
"The country was highly cultivated, every ledge ",
|
| 85 |
+
"being planted with chestnuts, walnuts, and apple trees ",
|
| 86 |
+
"from which the apples were now gathering."
|
| 87 |
+
],
|
| 88 |
+
"generation_seconds": 6.368318796157837,
|
| 89 |
+
"c_awq": "Their expression was divine, and as they glanced at me timidly, but with parted lips and great bewilderment, I forgot all thoughts of their conversion and feelings that were far more earthly. Even in middle age, they were still comely, and the old gray-haired women at their cottage doors had a dignity, not to say majesty, of their own. The men were as handsome as the women beautiful. I have always delighted in and reverenced beauty, but I felt simply abashed in the presence of such a splendid type. A compound of all that is best in Egyptian, Greek, and Italian. The children were infinite in number, and exceedingly merry. I need hardly say that they came in for their full share of the prevailing beauty. I expressed by signs my admiration and pleasure to my guides, and they were greatly pleased. The country was highly cultivated, every ledge being planted with chestnuts, walnuts, and apple trees from which the apples were now gathering.",
|
| 90 |
+
"c_bf16": "Their expression was divine, and as they glanced at me timidly but with parted lips and great bewilderment, I forgot all thoughts of their conversion and feelings that were far more earthly. Even in middle age, they were still comely, and the old gray-haired women at their cottage doors had a dignity, not to say majesty, of their own. The men were as handsome as the women beautiful. I have always delighted in and reverenced beauty, but I felt simply abashed in the presence of such a splendid type. A compound of all that is best in Egyptian, Greek, and Italian. The children were infinite in number, and exceedingly merry. I need hardly say that they came in for their full share of the prevailing beauty. I expressed by signs my admiration and pleasure to my guides, and they were greatly pleased. The country was highly cultivated, every ledge being planted with chestnuts, walnuts, and apple trees from which the apples were now gathering."
|
| 91 |
+
},
|
| 92 |
+
{
|
| 93 |
+
"wav": "/mnt/ssd0/vv3/awq-stream/data/eval/eval-002.wav",
|
| 94 |
+
"reference": "SO THEY FOLLOWED HER THROUGH THE LOW ARCHWAY AND IN A ROOM BEYOND VERY SIMPLY FURNISHED SAT A YOUNG GIRL ENGAGED IN DARNING A PAIR OF PINK STOCKINGS SHE WAS A BEAUTIFUL GIRL OF ABOUT SEVENTEEN YEARS OF AGE NOT FAT LIKE ALL THE REST OF THE PINKIES BUT SLENDER AND WELL FORMED ACCORDING TO OUR OWN IDEAS OF BEAUTY HER COMPLEXION WAS NOT A DECIDED PINK BUT A SOFT ROSY TINT NOT MUCH DEEPER THAN THAT OF TROT'S SKIN GIVING THEMSELVES UP WHOLLY TO THEIR GRIEF THEY ARE NO LONGER CONCERNED ABOUT ANY EARTHLY POSSESSION AND OFTEN GIVE AWAY ALL THAT THEY HAVE TO THE FIRST COMERS EVEN TO THEIR BEDS AND THEIR HOME IT WAS PREPARED BY DRESSING IN THE FINEST CLOTHES TOGETHER WITH SOME PERSONAL POSSESSIONS AND ORNAMENTS WRAPPED IN SEVERAL ROBES AND FINALLY IN A SECURE COVERING OF RAW HIDE AS A SPECIAL MARK OF RESPECT THE BODY OF A YOUNG WOMAN OR A WARRIOR WAS SOMETIMES LAID OUT IN STATE IN A NEW TEEPEE WITH THE USUAL HOUSEHOLD ARTICLES AND EVEN WITH A DISH OF FOOD LEFT BESIDE IT NOT THAT THEY SUPPOSED THE SPIRIT COULD USE THE IMPLEMENTS OR EAT THE FOOD BUT MERELY AS A LAST TRIBUTE",
|
| 95 |
+
"baseline": " \n Speaker 0:So they followed her through the low archway and in a room beyond, very simply furnished, sat a young girl engaged in darning a pair of pink stockings. She was a beautiful girl of about seventeen years of age, not fat like all the rest of the Pinkies, but slender and well formed, according to our own ideas of beauty. Her complexion was not a decided pink, but a soft rosy tint, not much deeper than that of trot's skin. \n Speaker 1:Giving themselves up wholly to their grief, they are no longer concerned about any earthly possession, and often give away all that they have to the first comers, even to their beds and their home. It was prepared by dressing in the finest clothes, together with some personal possessions and ornaments, wrapped in several robes and finally in a secure covering of rawhide.As a special mark of respect, the body of a young woman or a warrior was sometimes laid out in state in a new tepee, with the usual household articles and even with a dish of food left beside it. \n Speaker 1:Not that they supposed the spirit could use the implements or eat the food, but merely as a last tribute.",
|
| 96 |
+
"awq": " \n Speaker 0:So they followed her through the low archway and in a room beyond, very simply furnished, sat a young girl engaged in darning a pair of pink stockings. She was a beautiful girl of about seventeen years of age, not fat like all the rest of the Pinkies, but slender and well formed, according to our own ideas of beauty. Her complexion was not a decided pink, but a soft rosy tint, not much deeper than that of trot's skin. \n Speaker 1:Giving themselves up wholly to their grief, they are no longer concerned about any earthly possession, and often give away all that they have to the first comers, even to their beds and their home. It was prepared by dressing in the finest clothes, together with some personal possessions and ornaments, wrapped in several robes and finally in a secure covering of rawhide.As a special mark of respect, the body of a young woman or a warrior was sometimes laid out in state in a new tepee, with the usual household articles and even with a dish of food left beside it. \n Speaker 1:Not that they supposed the spirit could use the implements or eat the food, but merely as a last tribute.",
|
| 97 |
+
"awq_chunks": [
|
| 98 |
+
" \n Speaker 0:So they followed her through the low archway and in a ",
|
| 99 |
+
"room beyond, very simply furnished, sat a ",
|
| 100 |
+
"young girl engaged in darning a pair of pink stockings. ",
|
| 101 |
+
"She was a beautiful girl of about ",
|
| 102 |
+
"seventeen years of age, not fat like all the ",
|
| 103 |
+
"rest of the Pinkies, but slender and well formed, ",
|
| 104 |
+
"according to our own ideas of beauty. Her ",
|
| 105 |
+
"complexion was not a decided pink, but a ",
|
| 106 |
+
"soft rosy tint, not much deeper than that of trot's skin.",
|
| 107 |
+
" \n Speaker 1:Giving themselves up wholly to their ",
|
| 108 |
+
"grief, they are no longer concerned about any earthly possession, ",
|
| 109 |
+
"and often give away all that they have to the first comers, ",
|
| 110 |
+
"even to their beds and their home. It was ",
|
| 111 |
+
"prepared by dressing in the finest clothes, together with some ",
|
| 112 |
+
"personal possessions and ornaments, wrapped in several robes ",
|
| 113 |
+
"and finally in a secure covering of rawhide.",
|
| 114 |
+
"As a special mark of respect, the body of a young ",
|
| 115 |
+
"woman or a warrior was sometimes laid out in state in a ",
|
| 116 |
+
"new tepee, with the usual household articles and ",
|
| 117 |
+
"even with a dish of food left beside it. \n Speaker 1:Not that they ",
|
| 118 |
+
"supposed the spirit could use the implements or eat the food, ",
|
| 119 |
+
"but merely as a last tribute."
|
| 120 |
+
],
|
| 121 |
+
"generation_seconds": 7.597476482391357,
|
| 122 |
+
"c_awq": "So they followed her through the low archway and in a room beyond, very simply furnished, sat a young girl engaged in darning a pair of pink stockings. She was a beautiful girl of about seventeen years of age, not fat like all the rest of the Pinkies, but slender and well formed, according to our own ideas of beauty. Her complexion was not a decided pink, but a soft rosy tint, not much deeper than that of trot's skin. Giving themselves up wholly to their grief, they are no longer concerned about any earthly possession, and often give away all that they have to the first comers, even to their beds and their home. It was prepared by dressing in the finest clothes, together with some personal possessions and ornaments, wrapped in several robes and finally in a secure covering of rawhide.As a special mark of respect, the body of a young woman or a warrior was sometimes laid out in state in a new tepee, with the usual household articles and even with a dish of food left beside it. Not that they supposed the spirit could use the implements or eat the food, but merely as a last tribute.",
|
| 123 |
+
"c_bf16": "So they followed her through the low archway and in a room beyond, very simply furnished, sat a young girl engaged in darning a pair of pink stockings. She was a beautiful girl of about seventeen years of age, not fat like all the rest of the Pinkies, but slender and well formed, according to our own ideas of beauty. Her complexion was not a decided pink, but a soft rosy tint, not much deeper than that of trot's skin. Giving themselves up wholly to their grief, they are no longer concerned about any earthly possession, and often give away all that they have to the first comers, even to their beds and their home. It was prepared by dressing in the finest clothes, together with some personal possessions and ornaments, wrapped in several robes and finally in a secure covering of rawhide.As a special mark of respect, the body of a young woman or a warrior was sometimes laid out in state in a new tepee, with the usual household articles and even with a dish of food left beside it. Not that they supposed the spirit could use the implements or eat the food, but merely as a last tribute."
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"wav": "/mnt/ssd0/vv3/awq-stream/data/eval/eval-003.wav",
|
| 127 |
+
"reference": "WHAT ALTERNATIVE WAS THERE FOR HER SHE WAS DESTROYED NOT MERELY BY THE UNCONSIDERED UNDISCIPLINED PASSIONS OF HER HUSBAND AND HER LOVER BUT BY THE VAST TRADITION THAT SUSTAINS AND ENFORCES THE SUBJUGATION OF HER SEX",
|
| 128 |
+
"baseline": " \n Speaker 0:What alternative was there for her? She was destroyed, not merely by the unconsidered, undisciplined passions of her husband and her lover, but by the vast tradition that sustains and enforces the subjugation of her sex.",
|
| 129 |
+
"awq": " \n Speaker 0:What alternative was there for her? She was destroyed, not merely by the unconsidered, undisciplined passions of her husband and her lover, but by the vast tradition that sustains and enforces the subjugation of her sex.",
|
| 130 |
+
"awq_chunks": [
|
| 131 |
+
" \n Speaker 0:What alternative was there for her? ",
|
| 132 |
+
"She was destroyed, not ",
|
| 133 |
+
"merely by the unconsidered, undisciplined passions ",
|
| 134 |
+
"of her husband and her lover, but by the ",
|
| 135 |
+
"vast tradition that sustains and enforces the ",
|
| 136 |
+
"subjugation of her sex."
|
| 137 |
+
],
|
| 138 |
+
"generation_seconds": 1.7527744770050049,
|
| 139 |
+
"c_awq": "What alternative was there for her? She was destroyed, not merely by the unconsidered, undisciplined passions of her husband and her lover, but by the vast tradition that sustains and enforces the subjugation of her sex.",
|
| 140 |
+
"c_bf16": "What alternative was there for her? She was destroyed, not merely by the unconsidered, undisciplined passions of her husband and her lover, but by the vast tradition that sustains and enforces the subjugation of her sex."
|
| 141 |
+
},
|
| 142 |
+
{
|
| 143 |
+
"wav": "/mnt/ssd0/vv3/awq-stream/data/eval/eval-004.wav",
|
| 144 |
+
"reference": "MOREOVER JEAN VALJEAN HAD CHOSEN HIS REFUGE WELL HE HAD PAID HER SIX MONTHS IN ADVANCE AND HAD COMMISSIONED THE OLD WOMAN TO FURNISH THE CHAMBER AND DRESSING ROOM AS WE HAVE SEEN WEEK FOLLOWED WEEK THESE TWO BEINGS LED A HAPPY LIFE IN THAT HOVEL COSETTE WAS NO LONGER IN RAGS SHE WAS IN MOURNING AND THEN HE TALKED OF HER MOTHER AND HE MADE HER PRAY HE PASSED HOURS IN WATCHING HER DRESSING AND UNDRESSING HER DOLL AND IN LISTENING TO HER PRATTLE THE BEST OF US ARE NOT EXEMPT FROM EGOTISTICAL THOUGHTS HE HAD RETURNED TO PRISON THIS TIME FOR HAVING DONE RIGHT HE HAD QUAFFED FRESH BITTERNESS DISGUST AND LASSITUDE WERE OVERPOWERING HIM EVEN THE MEMORY OF THE BISHOP PROBABLY SUFFERED A TEMPORARY ECLIPSE THOUGH SURE TO REAPPEAR LATER ON LUMINOUS AND TRIUMPHANT BUT AFTER ALL THAT SACRED MEMORY WAS GROWING DIM",
|
| 145 |
+
"baseline": " \n Speaker 0:Moreover, Jean Valjean had chosen his refuge well. He had paid her six months in advance and had commissioned the old woman to furnish the chamber and dressing room as we have seen. Week followed week, these two beings led a happy life in that hovel. Cosette was no longer in rags. She was in mourning. \n Speaker 0:And then he talked to her mother and he made her pray. He passed hours in watching her dressing and undressing her doll and in listening to her prattle. The best of us are not exempt from egotistical thoughts.He had returned to prison, this time for having done right. He had quaffed fresh bitterness, distrust and lassitude were overpowering him. \n Speaker 0:Even the memory of the bishop, probably suffered a temporary eclipse, though sure to reappear later on, luminous and triumphant. But after all, that sacred memory was growing dim.",
|
| 146 |
+
"awq": " \n Speaker 0:Moreover, Jean Valjean had chosen his refuge well. He had paid her six months in advance and had commissioned the old woman to furnish the chamber and dressing room as we have seen. Week followed week, these two beings led a happy life in that hovel. Cosette was no longer in rags. She was in mourning. And then he talked to her mother and he made her pray. \n Speaker 0:He passed hours in watching her dressing and undressing her doll and in listening to her prattle. The best of us are not exempt from egotistical thoughts.He had returned to prison, this time for having done right. He had quaffed fresh bitterness, distrust and lassitude were overpowering him. \n Speaker 0:Even the memory of the bishop, probably suffered a temporary eclipse, though sure to reappear later on, luminous and triumphant. But, after all, that sacred memory was growing dim.",
|
| 147 |
+
"awq_chunks": [
|
| 148 |
+
" \n Speaker 0:Moreover, Jean Valjean had ",
|
| 149 |
+
"chosen his refuge well. He had ",
|
| 150 |
+
"paid her six months in advance and had commissioned ",
|
| 151 |
+
"the old woman to furnish the chamber and dressing room ",
|
| 152 |
+
"as we have seen. Week ",
|
| 153 |
+
"followed week, these two beings led a happy ",
|
| 154 |
+
"life in that hovel. Cosette ",
|
| 155 |
+
"was no longer in rags. She was in ",
|
| 156 |
+
"mourning. And then he talked to ",
|
| 157 |
+
"her mother and he made her pray.",
|
| 158 |
+
" \n Speaker 0:He passed hours in watching her dressing ",
|
| 159 |
+
"and undressing her doll and in listening to her ",
|
| 160 |
+
"prattle. The best of us are ",
|
| 161 |
+
"not exempt from egotistical thoughts.",
|
| 162 |
+
"He had returned to prison, this ",
|
| 163 |
+
"time for having done right. ",
|
| 164 |
+
"He had quaffed fresh bitterness, ",
|
| 165 |
+
"distrust and lassitude were overpowering him.",
|
| 166 |
+
" \n Speaker 0:Even the memory of the bishop, ",
|
| 167 |
+
"probably suffered a temporary eclipse, ",
|
| 168 |
+
"though sure to reappear later on, luminous and ",
|
| 169 |
+
"triumphant. But, after ",
|
| 170 |
+
"all, that sacred memory was growing ",
|
| 171 |
+
"dim."
|
| 172 |
+
],
|
| 173 |
+
"generation_seconds": 6.573299169540405,
|
| 174 |
+
"c_awq": "Moreover, Jean Valjean had chosen his refuge well. He had paid her six months in advance and had commissioned the old woman to furnish the chamber and dressing room as we have seen. Week followed week, these two beings led a happy life in that hovel. Cosette was no longer in rags. She was in mourning. And then he talked to her mother and he made her pray. He passed hours in watching her dressing and undressing her doll and in listening to her prattle. The best of us are not exempt from egotistical thoughts.He had returned to prison, this time for having done right. He had quaffed fresh bitterness, distrust and lassitude were overpowering him. Even the memory of the bishop, probably suffered a temporary eclipse, though sure to reappear later on, luminous and triumphant. But, after all, that sacred memory was growing dim.",
|
| 175 |
+
"c_bf16": "Moreover, Jean Valjean had chosen his refuge well. He had paid her six months in advance and had commissioned the old woman to furnish the chamber and dressing room as we have seen. Week followed week, these two beings led a happy life in that hovel. Cosette was no longer in rags. She was in mourning. And then he talked to her mother and he made her pray. He passed hours in watching her dressing and undressing her doll and in listening to her prattle. The best of us are not exempt from egotistical thoughts.He had returned to prison, this time for having done right. He had quaffed fresh bitterness, distrust and lassitude were overpowering him. Even the memory of the bishop, probably suffered a temporary eclipse, though sure to reappear later on, luminous and triumphant. But after all, that sacred memory was growing dim."
|
| 176 |
+
},
|
| 177 |
+
{
|
| 178 |
+
"wav": "/mnt/ssd0/vv3/awq-stream/data/eval/eval-005.wav",
|
| 179 |
+
"reference": "THEN HE SAT DOWN IN HIS CHAIR AND GAZED WITHOUT SEEING CONTEMPLATING THE RESULT OF HIS WORK WEEK FOLLOWED WEEK THESE TWO BEINGS LED A HAPPY LIFE IN THAT HOVEL WHAT WOULD SHE DO ABOUT THAT THE CONFOUNDED WRETCH COSETTE WAS NO LONGER IN RAGS SHE WAS IN MOURNING AND THEN HE TALKED OF HER MOTHER AND HE MADE HER PRAY HE PASSED HOURS IN WATCHING HER DRESSING AND UNDRESSING HER DOLL AND IN LISTENING TO HER PRATTLE LATER HOWEVER HIS OLD DISCRETION ASSERTED ITSELF SOMETHING HAD TO BE DONE A CLIMAX WAS NEAR AND SHE WOULD NOT SIT IDLE THE BEST OF US ARE NOT EXEMPT FROM EGOTISTICAL THOUGHTS HE KNEW HER WELL ENOUGH TO KNOW THAT WHEN SHE HAD DECIDED UPON A PLAN SHE WOULD FOLLOW IT UP HE AROSE FROM HIS CHAIR AND WENT AND LOOKED OUT INTO THE STREET",
|
| 180 |
+
"baseline": " \n Speaker 0:Then he sat down in his chair and gazed without saying, contemplating the result of his work. \n Speaker 0:Week followed week. These two beings led a happy life in that hovel. \n Speaker 0:What would she do about that, the confounded wretch? \n Speaker 0:Cusack was no longer in Rags. She was in mourning.And then he talked of her mother, and he made her pray. He passed hours in watching her dressing and undressing her doll, and in listening to her prattle. \n Speaker 0:Later, however, his old discretion asserted itself.Something had to be done. A climax was near, and she would not sit idle. The best of us are not exempt from egotistical thoughts. \n Speaker 0:He knew her well enough to know that when she had decided upon a plan, she would follow it up.He arose from his chair and went and looked out into the street.",
|
| 181 |
+
"awq": " \n Speaker 0:Then he sat down in his chair and gazed without scene, contemplating the result of his work. \n Speaker 0:Week followed week. These two beings led a happy life in that hovel. \n Speaker 0:What would she do about that, the confounded wretch? \n Speaker 0:Cusack was no longer in Rags. She was in mourning.And then he talked of her mother, and he made her pray. He passed hours in watching her dressing and undressing her doll, and in listening to her prattle. \n Speaker 0:Later, however, his old discretion asserted itself.Something had to be done. A climax was near, and she would not sit idle. The best of us are not exempt from egotistical thoughts. \n Speaker 0:He knew her well enough to know that when she had decided upon a plan, she would follow it up.He arose from his chair and went and looked out into the street.",
|
| 182 |
+
"awq_chunks": [
|
| 183 |
+
" \n Speaker 0:Then he sat down in his chair and gazed without ",
|
| 184 |
+
"scene, contemplating the result of his work.",
|
| 185 |
+
" \n Speaker 0:Week followed week. These two beings ",
|
| 186 |
+
"led a happy life in that hovel.",
|
| 187 |
+
" \n Speaker 0:What would she do about that, the confounded wretch?",
|
| 188 |
+
" \n Speaker 0:Cusack was no longer in ",
|
| 189 |
+
"Rags. She was in mourning.",
|
| 190 |
+
"And then he talked of her mother, and he made her ",
|
| 191 |
+
"pray. He passed ",
|
| 192 |
+
"hours in watching her dressing and undressing her doll, ",
|
| 193 |
+
"and in listening to her prattle.",
|
| 194 |
+
" \n Speaker 0:Later, however, his old discretion asserted itself.",
|
| 195 |
+
"Something had to be done. A climax was near, and ",
|
| 196 |
+
"she would not sit idle. The best of ",
|
| 197 |
+
"us are not exempt from egotistical thoughts.",
|
| 198 |
+
" \n Speaker 0:He knew her well enough to know that when she had ",
|
| 199 |
+
"decided upon a plan, she would follow it up.",
|
| 200 |
+
"He arose from his chair and went and looked out into the street.",
|
| 201 |
+
""
|
| 202 |
+
],
|
| 203 |
+
"generation_seconds": 6.074247121810913,
|
| 204 |
+
"c_awq": "Then he sat down in his chair and gazed without scene, contemplating the result of his work. Week followed week. These two beings led a happy life in that hovel. What would she do about that, the confounded wretch? Cusack was no longer in Rags. She was in mourning.And then he talked of her mother, and he made her pray. He passed hours in watching her dressing and undressing her doll, and in listening to her prattle. Later, however, his old discretion asserted itself.Something had to be done. A climax was near, and she would not sit idle. The best of us are not exempt from egotistical thoughts. He knew her well enough to know that when she had decided upon a plan, she would follow it up.He arose from his chair and went and looked out into the street.",
|
| 205 |
+
"c_bf16": "Then he sat down in his chair and gazed without saying, contemplating the result of his work. Week followed week. These two beings led a happy life in that hovel. What would she do about that, the confounded wretch? Cousec was no longer in Rags. She was in mourning.And then he talked of her mother, and he made her pray. He passed hours in watching her dressing and undressing her doll, and in listening to her prattle. Later, however, his old discretion asserted itself.Something had to be done. A climax was near, and she would not sit idle. The best of us are not exempt from egotistical thoughts. He knew her well enough to know that when she had decided upon a plan, she would follow it up.He arose from his chair and went and looked out into the street."
|
| 206 |
+
},
|
| 207 |
+
{
|
| 208 |
+
"wav": "/mnt/ssd0/vv3/awq-stream/data/eval/eval-006.wav",
|
| 209 |
+
"reference": "YET THE PHYSIOGNOMY OF THE PEOPLE WHEN MORE CLOSELY EXAMINED WAS FAR FROM EXHIBITING THE INDIFFERENCE OF STUPIDITY THEIR FEATURES WERE ROUGH BUT REMARKABLY INTELLIGENT GRAVE BUT THE VERY REVERSE OF STUPID AND FROM AMONG THE YOUNG WOMEN AN ARTIST MIGHT HAVE CHOSEN MORE THAN ONE MODEL WHOSE FEATURES AND FORM RESEMBLED THOSE OF MINERVA THIS AVENUE WAS STRAIGHT AND OF MODERATE LENGTH RUNNING BETWEEN A DOUBLE ROW OF VERY ANCIENT HORSE CHESTNUTS PLANTED ALTERNATELY WITH SYCAMORES WHICH ROSE TO SUCH HUGE HEIGHT AND NOURISHED SO LUXURIANTLY THAT THEIR BOUGHS COMPLETELY OVER ARCHED THE BROAD ROAD BENEATH IT WAS ONE OF THOSE EFFECTS WHICH A PAINTER LOVES TO REPRESENT AND MINGLED WELL WITH THE STRUGGLING LIGHT WHICH FOUND ITS WAY BETWEEN THE BOUGHS OF THE SHADY ARCH THAT VAULTED THE BROAD GREEN ALLEY",
|
| 210 |
+
"baseline": " \n Speaker 0:Yet the physiognomy of the people, when more closely examined, was far from exhibiting the indifference of stupidity. Their features were rough, but remarkably intelligent, grave. But the very reverse of stupid. And from among the young women, an artist might have chosen more than one model whose features and form resembled those of Minerva.This avenue was straight and of moderate length, running between a double row of very ancient horse chestnuts. \n Speaker 0:Planted alternately with sycamores, which rose to such huge heights and nourished so luxuriantly that their boughs completely overarched the broad road beneath.It was one of those effects which a painter loves to represent and mingled well with the struggling light, which found its way between the boughs of the shady arc that vaulted the broad green alley.",
|
| 211 |
+
"awq": " \n Speaker 0:Yet the physiognomy of the people, when more closely examined, was far from exhibiting the indifference of stupidity. Their features were rough, but remarkably intelligent, grave. But the very reverse of stupid, and from among the young women, an artist might have chosen more than one model whose features and form resembled those of Minerva.This avenue was straight and of moderate length, running between a double row of very ancient horse-chestnuts. \n Speaker 0:Planted alternately with sycamores, which rose to such huge heights, and nourished so luxuriantly that their boughs completely overarched the broad road beneath.It was one of those effects which a painter loves to represent, and mingled well with the struggling light which found its way between the boughs of the shady arch that vaulted the broad green alley.",
|
| 212 |
+
"awq_chunks": [
|
| 213 |
+
" \n Speaker 0:Yet the physiognomy of the people, ",
|
| 214 |
+
"when more closely examined, was far from ",
|
| 215 |
+
"exhibiting the indifference of stupidity. Their ",
|
| 216 |
+
"features were rough, but remarkably ",
|
| 217 |
+
"intelligent, grave. But the ",
|
| 218 |
+
"very reverse of stupid, and from among the young ",
|
| 219 |
+
"women, an artist might have chosen more than one model ",
|
| 220 |
+
"whose features and form resembled those of Minerva.",
|
| 221 |
+
"This avenue was straight and of moderate ",
|
| 222 |
+
"length, running between a double row of very ",
|
| 223 |
+
"ancient horse-chestnuts. \n Speaker 0:Planted ",
|
| 224 |
+
"alternately with sycamores, which rose to such huge ",
|
| 225 |
+
"heights, and nourished so luxuriantly that their ",
|
| 226 |
+
"boughs completely overarched the broad road beneath.",
|
| 227 |
+
"It was one of those effects which a ",
|
| 228 |
+
"painter loves to represent, and mingled well with the ",
|
| 229 |
+
"struggling light which found its way between the boughs ",
|
| 230 |
+
"of the shady arch that vaulted the broad green ",
|
| 231 |
+
"alley."
|
| 232 |
+
],
|
| 233 |
+
"generation_seconds": 5.82283616065979,
|
| 234 |
+
"c_awq": "Yet the physiognomy of the people, when more closely examined, was far from exhibiting the indifference of stupidity. Their features were rough, but remarkably intelligent, grave. But the very reverse of stupid, and from among the young women, an artist might have chosen more than one model whose features and form resembled those of Minerva.This avenue was straight and of moderate length, running between a double row of very ancient horse-chestnuts. Planted alternately with sycamores, which rose to such huge heights, and nourished so luxuriantly that their boughs completely overarched the broad road beneath.It was one of those effects which a painter loves to represent, and mingled well with the struggling light which found its way between the boughs of the shady arch that vaulted the broad green alley.",
|
| 235 |
+
"c_bf16": "Yet the physiognomy of the people, when more closely examined, was far from exhibiting the indifference of stupidity. Their features were rough, but remarkably intelligent, grave. But the very reverse of stupid. And from among the young women, an artist might have chosen more than one model whose features and form resembled those of Minerva.This avenue was straight and of moderate length, running between a double row of very ancient horse chestnuts. Planted alternately with sycamores, which rose to such huge heights and nourished so luxuriantly that their boughs completely overarched the broad road beneath.It was one of those effects which a painter loves to represent and mingled well with the struggling light, which found its way between the boughs of the shady arc that vaulted the broad green alley."
|
| 236 |
+
},
|
| 237 |
+
{
|
| 238 |
+
"wav": "/mnt/ssd0/vv3/awq-stream/data/eval/eval-007.wav",
|
| 239 |
+
"reference": "THE CRIES AND CURSES OF THE ROBBERS FILLED THE AIR THEY TRIED IN VAIN TO BREAK DOWN THE GREAT DOORS ALL MY WORRIES ABOUT YOU WERE FOOLISH",
|
| 240 |
+
"baseline": " \n Speaker 0:The cries and curses of the robbers filled the air. They tried in vain to break down the great doors. All my worries about you were foolish.",
|
| 241 |
+
"awq": " \n Speaker 0:The cries and curses of the robbers filled the air. They tried in vain to break down the great doors. All my worries about you were foolish.",
|
| 242 |
+
"awq_chunks": [
|
| 243 |
+
" \n Speaker 0:The cries and curses of the robbers filled the ",
|
| 244 |
+
"air. They tried in vain to break down the ",
|
| 245 |
+
"great doors. All my worries about you were ",
|
| 246 |
+
"foolish."
|
| 247 |
+
],
|
| 248 |
+
"generation_seconds": 1.1491668224334717,
|
| 249 |
+
"c_awq": "The cries and curses of the robbers filled the air. They tried in vain to break down the great doors. All my worries about you were foolish.",
|
| 250 |
+
"c_bf16": "The cries and curses of the robbers filled the air. They tried in vain to break down the great doors. All my worries about you were foolish."
|
| 251 |
+
},
|
| 252 |
+
{
|
| 253 |
+
"wav": "/mnt/ssd0/vv3/awq-stream/data/eval/eval-008.wav",
|
| 254 |
+
"reference": "THIS IS THE MATERIAL OR PHYSICAL PRAYER AND THAT TUMBLE'S ENOUGH TO KNOCK THE SENSE OUT OF A FULL GROWN MAN I COULD NOT THINK OF ALLOWING ANY OF MY CHARGES TO TAKE SO TERRIBLE A RISK AND NO I AM THE LIGHTER OF THE TWO URGED TAD NOTHING OF THE MARVELOUS COULD ASTONISH HIM AS THAT A BEAST SHOULD SPEAK OR THE SUN STAND STILL WHO MAY CONDEMN HIS SUPERSTITION HERE IS THE SUPREME MYSTERY THAT IS THE ESSENCE OF WORSHIP WITHOUT WHICH THERE CAN BE NO RELIGION AND IN THE PRESENCE OF THIS MYSTERY OUR ATTITUDE CANNOT BE VERY UNLIKE THAT OF THE NATURAL PHILOSOPHER WHO BEHOLDS WITH AWE THE DIVINE IN ALL CREATION I AM THE ONE TO GO AFTER WALT IF ANYONE HAS TO I'LL GO DOWN MISTER THOMAS MASTER TAD IS RIGHT DECIDED THE GUIDE GAZING AT THE TWO BOYS APPROVINGLY I PROTEST SHOUTED THE PROFESSOR",
|
| 255 |
+
"baseline": " \n Speaker 0:This is the material or physical prayer. \n Speaker 0:And that tumbles enough to knock the sense out of a full grown man. \n Speaker 0:I could not think of allowing any of my chargers to take so terrible a risk at no. I am the lighter of the two urged had. \n Speaker 0:Nothing of the marvellous could astonish him, as that a beast should speak or the sun stand still. \n Speaker 0:Who may condemn his superstition? Here is the supreme mystery. That is the essence of worship, without which there can be no religion. And in the presence of this mystery, our attitude cannot be very unlike that of the natural philosopher who beholds with awe the divine in all creation. \n Speaker 0:I am the one to go after Walt, if anyone has to. I'll go down, Mister Thomas. Master Ted is right, decided the guide, gazing at the two boys approvingly. I protest, shouted the professor.",
|
| 256 |
+
"awq": " \n Speaker 0:This is the material or physical prayer. \n Speaker 1:And that tumble's enough to knock the sense out of a full grown man. I could not think of allowing any of my chargers to take so terrible a risk at.No, I am the lighter of the two, urged Tad. \n Speaker 0:Nothing of the marvellous could astonish him, as that a beast should speak or the sun stand still.Who may condemn his superstition?Here is the supreme mystery that is the essence of worship, without which there can be no religion, and in the presence of this mystery our attitude cannot be very unlike that of the natural philosopher, who beholds with awe the divine in all creation. \n Speaker 1:I am the one to go after Walt, if anyone has to. I'll go down, Mister Thomas. Master Ted is right, decided the guide, gazing at the two boys approvingly. I protest, shouted the professor.",
|
| 257 |
+
"awq_chunks": [
|
| 258 |
+
" \n Speaker 0:This is the material or physical prayer.",
|
| 259 |
+
" \n Speaker 1:And that tumble's enough to knock the sense out of a full grown ",
|
| 260 |
+
"man. I could not think of allowing any of my ",
|
| 261 |
+
"chargers to take so terrible a risk at.",
|
| 262 |
+
"No, I am the lighter of the two, urged Tad.",
|
| 263 |
+
" \n Speaker 0:Nothing of the marvellous could astonish him, as ",
|
| 264 |
+
"that a beast should speak or the sun stand still.",
|
| 265 |
+
"Who may condemn his superstition?",
|
| 266 |
+
"Here is the supreme mystery that is the essence of ",
|
| 267 |
+
"worship, without which there can be no religion, and in the ",
|
| 268 |
+
"presence of this mystery our attitude cannot be very unlike that ",
|
| 269 |
+
"of the natural philosopher, who beholds with awe ",
|
| 270 |
+
"the divine in all creation. \n Speaker 1:I am ",
|
| 271 |
+
"the one to go after Walt, if anyone has to. I'll ",
|
| 272 |
+
"go down, Mister Thomas. Master Ted is ",
|
| 273 |
+
"right, decided the guide, gazing at the two boys ",
|
| 274 |
+
"approvingly. I protest, shouted the ",
|
| 275 |
+
"professor."
|
| 276 |
+
],
|
| 277 |
+
"generation_seconds": 6.092253684997559,
|
| 278 |
+
"c_awq": "This is the material or physical prayer. And that tumble's enough to knock the sense out of a full grown man. I could not think of allowing any of my chargers to take so terrible a risk at.No, I am the lighter of the two, urged Tad. Nothing of the marvellous could astonish him, as that a beast should speak or the sun stand still.Who may condemn his superstition?Here is the supreme mystery that is the essence of worship, without which there can be no religion, and in the presence of this mystery our attitude cannot be very unlike that of the natural philosopher, who beholds with awe the divine in all creation. I am the one to go after Walt, if anyone has to. I'll go down, Mister Thomas. Master Ted is right, decided the guide, gazing at the two boys approvingly. I protest, shouted the professor.",
|
| 279 |
+
"c_bf16": "This is the material or physical prayer. And that tumbles enough to knock the sense out of a full grown man. I could not think of allowing any of my chargers to take so terrible a risk at no. I am the lighter of the two urged had. Nothing of the marvellous could astonish him, as that a beast should speak or the sun stand still. Who may condemn his superstition? Here is the supreme mystery. That is the essence of worship, without which there can be no religion. And in the presence of this mystery, our attitude cannot be very unlike that of the natural philosopher who beholds with awe the divine in all creation. I am the one to go after Walt, if anyone has to. I'll go down, Mister Thomas. Master Ted is right, decided the guide, gazing at the two boys approvingly. I protest, shouted the professor."
|
| 280 |
+
},
|
| 281 |
+
{
|
| 282 |
+
"wav": "/mnt/ssd0/vv3/awq-stream/data/eval/eval-009.wav",
|
| 283 |
+
"reference": "GRUB PI LE GRUB PI LE WHO IS THE WRANGLER THIS MORNING ASKED THE FOREMAN GLANCING ABOUT AT HIS MEN A WRANGLER'S A WRANGLER ANSWERED BIG FOOT STOLIDLY HE'S A FELLOW WHO'S ALL THE TIME MAKING TROUBLE ISN'T HE ASKED STACY INNOCENTLY OH NO THIS KIND OF A WRANGLER ISN'T LAUGHED THE FOREMAN HE'S A TROUBLE CURER NOT A TROUBLEMAKER EXCEPT FOR HIMSELF PONG TELL THE YOUNG GENTLEMEN WHAT WOULD BECOME OF YOU IF YOU WERE TO SERVE BAD MEALS TO THIS OUTFIT OF COWPUNCHERS HOW ASKED TAD WE HAD BETTER START THE DRIVE THIS MORNING",
|
| 284 |
+
"baseline": " \n Speaker 0:Grip pile, grip pile. \n Speaker 0:Who is the wrangler of this morning? asked the foreman, glancing about at his men. A wrangler is a wrangler, answered Bigfoot stottily.He's a fellow who's all the time making trouble, isn't he? asked Stacy innocently. Oh, no, this kind of a wrangler isn't, laughed the foreman. He's a trouble cure, not a trouble maker, except for himself. Pong, tell the young gentlemen what would become of you if you were to serve bad meals to this outfit of cowpunchers. \n Speaker 0:How? asked Ted. We had better start the drive this morning.",
|
| 285 |
+
"awq": " \n Speaker 0:Grip pile, grip pile. \n Speaker 0:Who is the wrangler of this morning? asked the foreman, glancing about at his men. A wrangler is a wrangler, answered Bigfoot stottily.He's a fellow who's all the time making trouble, isn't he? asked Stacy innocently. Oh, no, this kind of a wrangler isn't, laughed the foreman. He's a troubled cure, not a troublemaker, except for himself. Pong, tell the young gentlemen what would become of you if you were to serve bad meals to this outfit of cowpunchers. \n Speaker 0:How? asked Ted. We had better start the drive this morning.",
|
| 286 |
+
"awq_chunks": [
|
| 287 |
+
" \n Speaker 0:Grip pile, grip pile.",
|
| 288 |
+
" \n Speaker 0:Who is the wrangler of this morning? asked the ",
|
| 289 |
+
"foreman, glancing about at his men. A ",
|
| 290 |
+
"wrangler is a wrangler, answered Bigfoot stottily.",
|
| 291 |
+
"He's a fellow who's all the time making trouble, isn't ",
|
| 292 |
+
"he? asked Stacy innocently. Oh, ",
|
| 293 |
+
"no, this kind of a wrangler isn't, laughed the foreman. ",
|
| 294 |
+
"He's a troubled cure, not a troublemaker, ",
|
| 295 |
+
"except for himself. Pong, tell the young ",
|
| 296 |
+
"gentlemen what would become of you if you were to serve bad meals ",
|
| 297 |
+
"to this outfit of cowpunchers.",
|
| 298 |
+
" \n Speaker 0:How? asked Ted. We had better start the ",
|
| 299 |
+
"drive this morning."
|
| 300 |
+
],
|
| 301 |
+
"generation_seconds": 4.5349719524383545,
|
| 302 |
+
"c_awq": "Grip pile, grip pile. Who is the wrangler of this morning? asked the foreman, glancing about at his men. A wrangler is a wrangler, answered Bigfoot stottily.He's a fellow who's all the time making trouble, isn't he? asked Stacy innocently. Oh, no, this kind of a wrangler isn't, laughed the foreman. He's a troubled cure, not a troublemaker, except for himself. Pong, tell the young gentlemen what would become of you if you were to serve bad meals to this outfit of cowpunchers. How? asked Ted. We had better start the drive this morning.",
|
| 303 |
+
"c_bf16": "Grip pile, grip pile. Who is the wrangler of this morning? asked the foreman, glancing about at his men. A wrangler is a wrangler, answered Bigfoot stottily. He's a fellow who's all the time making trouble, isn't he? asked Stacy innocently. Oh, no, this kind of a wrangler isn't, laughed the foreman. He's a trouble cure, not a trouble maker, except for himself. Pong, tell the young gentleman what would become of you if you were to serve bad meals to this outfit of cowpunchers. How? asked Ted. We had better start the drive this morning."
|
| 304 |
+
},
|
| 305 |
+
{
|
| 306 |
+
"wav": "/mnt/ssd0/vv3/awq-stream/data/eval/eval-010.wav",
|
| 307 |
+
"reference": "THE ENCLOSURE HAD FORMERLY BEEN VERY EXTENSIVE BUT WAS NOW CONTRACTED WITHIN SMALL COMPASS AND HEMMED ABOUT PARTLY BY HIGH WOODEN FENCES AND PARTLY BY THE OUTBUILDINGS OF HOUSES THAT STOOD ON ANOTHER STREET THE WHITE DOUBLE ROSEBUSH HAD EVIDENTLY BEEN PROPPED UP ANEW AGAINST THE HOUSE SINCE THE COMMENCEMENT OF THE SEASON AND A PEAR TREE AND THREE DAMSON TREES WHICH EXCEPT A ROW OF CURRANT BUSHES CONSTITUTED THE ONLY VARIETIES OF FRUIT BORE MARKS OF THE RECENT AMPUTATION OF SEVERAL SUPERFLUOUS OR DEFECTIVE LIMBS THERE WERE ALSO A FEW SPECIES OF ANTIQUE AND HEREDITARY FLOWERS IN NO VERY FLOURISHING CONDITION BUT SCRUPULOUSLY WEEDED AS IF SOME PERSON EITHER OUT OF LOVE OR CURIOSITY HAD BEEN ANXIOUS TO BRING THEM TO SUCH PERFECTION AS THEY WERE CAPABLE OF ATTAINING SUMMER SQUASHES ALMOST IN THEIR GOLDEN BLOSSOM CUCUMBERS NOW EVINCING A TENDENCY TO SPREAD AWAY FROM THE MAIN STOCK AND RAMBLE FAR AND WIDE TWO OR THREE ROWS OF STRING BEANS AND AS MANY MORE THAT WERE ABOUT TO FESTOON THEMSELVES ON POLES TOMATOES OCCUPYING A SITE SO SHELTERED AND SUNNY THAT THE PLANTS WERE ALREADY GIGANTIC AND PROMISED AN EARLY AND ABUNDANT HARVEST",
|
| 308 |
+
"baseline": " \n Speaker 0:The enclosure had formerly been very extensive, but was now contracted within small compass and hemmed about partly by high wooden fences and partly by the outbuildings of houses that stood on another street.The white double rose bush had evidently been propped up anew against the house since the commencement of the season, and the pear tree and three damson trees, which except a row of currant bushes, constituted the only varieties of fruit for marks of the recent amputation of several superfluous or defective limbs. \n Speaker 0:There were also a few species of antique and hereditary flowers, and no very flourishing condition.But scrupulously weeded, as if some person, either out of love or curiosity, had been anxious to bring them to such perfection as they were capable of attaining. Summer squashes, almost in their golden blossom, cucumbers, now evincing a tendency to spread away from the main stock and ramble far and wide. \n Speaker 0:Two or three rows of string beans, and as many more that were about to festoon themselves on poles. Tomatoes occupying a site so sheltered and sunny that the plants were already gigantic and promised an early and abundant harvest.",
|
| 309 |
+
"awq": " \n Speaker 0:The enclosure had formerly been very extensive, but was now contracted within small compass and hemmed about partly by high wooden fences and partly by the outbuildings of houses that stood on another street. The white double rose bush had evidently been propped up anew against the house since the commencement of the season, and the pear tree and three damson trees, which except a row of currant bushes, constituted the only varieties of fruit for marks of the recent amputation of several superfluous or defective limbs. \n Speaker 0:There were also a few species of antique and hereditary flowers, and no very flourishing condition. But scrupulously weeded, as if some person, either out of love or curiosity, had been anxious to bring them to such perfection as they were capable of attaining. Summer squashes, almost in their golden blossom, cucumbers, now evincing a tendency to spread away from the main stock and ramble far and wide. Two or three rows of string beans, and as many more that were about to festoon themselves on poles. \n Speaker 0:Tomatoes occupying a site so sheltered and sunny that the plants were already gigantic and promised an early and abundant harvest.",
|
| 310 |
+
"awq_chunks": [
|
| 311 |
+
" \n Speaker 0:The enclosure had formerly been very extensive, but ",
|
| 312 |
+
"was now contracted within small compass and hemmed ",
|
| 313 |
+
"about partly by high wooden fences and partly ",
|
| 314 |
+
"by the outbuildings of houses that stood on another street. ",
|
| 315 |
+
"The white double rose bush had ",
|
| 316 |
+
"evidently been propped up anew against the house since the ",
|
| 317 |
+
"commencement of the season, and the pear tree and ",
|
| 318 |
+
"three damson trees, which except a row of currant ",
|
| 319 |
+
"bushes, constituted the only varieties of ",
|
| 320 |
+
"fruit for marks of the recent amputation ",
|
| 321 |
+
"of several superfluous or defective limbs.",
|
| 322 |
+
" \n Speaker 0:There were also a few species of antique and ",
|
| 323 |
+
"hereditary flowers, and no very flourishing condition. ",
|
| 324 |
+
"But scrupulously weeded, as if some ",
|
| 325 |
+
"person, either out of love or curiosity, ",
|
| 326 |
+
"had been anxious to bring them to such perfection as they were ",
|
| 327 |
+
"capable of attaining. Summer ",
|
| 328 |
+
"squashes, almost in their golden blossom, cucumbers, ",
|
| 329 |
+
"now evincing a tendency to spread away from the main ",
|
| 330 |
+
"stock and ramble far and wide. Two or three ",
|
| 331 |
+
"rows of string beans, and as many more that were about to ",
|
| 332 |
+
"festoon themselves on poles. \n Speaker 0:Tomatoes ",
|
| 333 |
+
"occupying a site so sheltered and sunny that the plants were ",
|
| 334 |
+
"already gigantic and promised an early and abundant ",
|
| 335 |
+
"harvest."
|
| 336 |
+
],
|
| 337 |
+
"generation_seconds": 8.07678771018982,
|
| 338 |
+
"c_awq": "The enclosure had formerly been very extensive, but was now contracted within small compass and hemmed about partly by high wooden fences and partly by the outbuildings of houses that stood on another street.The white double rose bush had evidently been propped up anew against the house since the commencement of the season, and the pear tree and three damson trees, which except a row of currant bushes, constituted the only varieties of fruit for marks of the recent amputation of several superfluous or defective limbs. There were also a few species of antique and hereditary flowers, and no very flourishing condition.But scrupulously weeded, as if some person, either out of love or curiosity, had been anxious to bring them to such perfection as they were capable of attaining. Summer squashes, almost in their golden blossom, cucumbers, now evincing a tendency to spread away from the main stock and ramble far and wide. Two or three rows of string beans, and as many more that were about to festoon themselves on poles. Tomatoes occupying a site so sheltered and sunny that the plants were already gigantic and promised an early and abundant harvest.",
|
| 339 |
+
"c_bf16": "The enclosure had formerly been very extensive, but was now contracted within small compass and hemmed about partly by high wooden fences and partly by the outbuildings of houses that stood on another street.The white double rose bush had evidently been propped up anew against the house since the commencement of the season. And the pear tree and three damson trees, which except a row of currant bushes, constituted the only varieties of fruit for marks of the recent amputation of several superfluous or defective limbs. There were also a few species of antique and hereditary flowers, and no very flourishing condition.But scrupulously weeded, as if some person, either out of love or curiosity, had been anxious to bring them to such perfection as they were capable of attaining. Summer squashes, almost in their golden blossom, cucumbers, now evincing a tendency to spread away from the main stock and ramble far and wide. Two or three rows of string beans, and as many more that were about to festoon themselves on poles. Tomatoes occupying a site so sheltered and sunny that the plants were already gigantic and promised an early and abundant harvest."
|
| 340 |
+
},
|
| 341 |
+
{
|
| 342 |
+
"wav": "/mnt/ssd0/vv3/awq-stream/data/eval/eval-011.wav",
|
| 343 |
+
"reference": "THE HOUSE WHICH SEEMED TO CONSIST OF TWO OR THREE HIGH NARROW AND STEEP ROOFED BUILDINGS PROJECTING FROM EACH OTHER AT RIGHT ANGLES FORMED ONE SIDE OF THE INCLOSURE IT HAD BEEN BUILT AT A PERIOD WHEN CASTLES WERE NO LONGER NECESSARY AND WHEN THE SCOTTISH ARCHITECTS HAD NOT YET ACQUIRED THE ART OF DESIGNING A DOMESTIC RESIDENCE NEITHER DID THE FRONT INDICATE ABSOLUTE SECURITY FROM DANGER PASS THE CHARM OUT TO ME THEN SAID THE ROBBER WHEN SHE RETURNED HIS HAND WAS STICKING THROUGH THE HOLE IN THE DOOR THE CRIES AND CURSES OF THE ROBBERS FILLED THE AIR THEY TRIED IN VAIN TO BREAK DOWN THE GREAT DOORS ALL MY WORRIES ABOUT YOU WERE FOOLISH",
|
| 344 |
+
"baseline": " \n Speaker 0:The house, which seemed to consist of two or three high, narrow, and steep-roofed buildings, projecting from each other at right angles, formed one side of the enclosure. It had been built at a period when castles were no longer necessary, and when the Scottish architects had not yet acquired the art of designing a domestic residence.Neither did the front indicate absolute security from danger. \n Speaker 1:Pass the charm out to me then, said the robber. When she returned, his hand was sticking through the hole in the door.The cries and curses of the robbers filled the air. They tried in vain to break down the great doors. All my worries about you were foolish.",
|
| 345 |
+
"awq": " \n Speaker 0:The house, which seemed to consist of two or three high, narrow, and steep roofed buildings, projecting from each other at right angles, formed one side of the enclosure. It had been built at a period when castles were no longer necessary, and when the Scottish architects had not yet acquired the art of designing a domestic residence.Neither did the front indicate absolute security from danger. \n Speaker 1:Pass the charm out to me then, said the robber, when she returned his hand was sticking through the hole in the door.The cries and curses of the robbers filled the air. They tried in vain to break down the great doors. All my worries about you were foolish.",
|
| 346 |
+
"awq_chunks": [
|
| 347 |
+
" \n Speaker 0:The house, which seemed to consist of two or three ",
|
| 348 |
+
"high, narrow, and steep roofed buildings, ",
|
| 349 |
+
"projecting from each other at right angles, formed one side of ",
|
| 350 |
+
"the enclosure. It had been built at a ",
|
| 351 |
+
"period when castles were no longer necessary, and when the ",
|
| 352 |
+
"Scottish architects had not yet acquired the art ",
|
| 353 |
+
"of designing a domestic residence.",
|
| 354 |
+
"Neither did the front indicate absolute security ",
|
| 355 |
+
"from danger. \n Speaker 1:Pass the charm out to me ",
|
| 356 |
+
"then, said the robber, when she ",
|
| 357 |
+
"returned his hand was sticking through the hole in the door.",
|
| 358 |
+
"The cries and curses of the robbers ",
|
| 359 |
+
"filled the air. They tried in vain to break ",
|
| 360 |
+
"down the great doors. All my worries about ",
|
| 361 |
+
"you were foolish."
|
| 362 |
+
],
|
| 363 |
+
"generation_seconds": 4.500569105148315,
|
| 364 |
+
"c_awq": "The house, which seemed to consist of two or three high, narrow, and steep roofed buildings, projecting from each other at right angles, formed one side of the enclosure. It had been built at a period when castles were no longer necessary, and when the Scottish architects had not yet acquired the art of designing a domestic residence.Neither did the front indicate absolute security from danger. Pass the charm out to me then, said the robber, when she returned his hand was sticking through the hole in the door.The cries and curses of the robbers filled the air. They tried in vain to break down the great doors. All my worries about you were foolish.",
|
| 365 |
+
"c_bf16": "The house, which seemed to consist of two or three high, narrow, and steep-roofed buildings, projecting from each other at right angles, formed one side of the enclosure. It had been built at a period when castles were no longer necessary, and when the Scottish architects had not yet acquired the art of designing a domestic residence.Neither did the front indicate absolute security from danger. Pass the charm out to me then, said the robber. When she returned, his hand was sticking through the hole in the door.The cries and curses of the robbers filled the air. They tried in vain to break down the great doors. All my worries about you were foolish."
|
| 366 |
+
}
|
| 367 |
+
],
|
| 368 |
+
"vibevoice_c_bench_rtx3090": [
|
| 369 |
+
{
|
| 370 |
+
"audio": "jfk.wav",
|
| 371 |
+
"weights": "awq",
|
| 372 |
+
"attn": "flashinfer",
|
| 373 |
+
"text": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
|
| 374 |
+
"rtf": 0.035,
|
| 375 |
+
"decode_tok_s": 152.1,
|
| 376 |
+
"prefill_tok_s": 927.0,
|
| 377 |
+
"lm_layers_mb": 3306.0,
|
| 378 |
+
"load_ms": 4222.0
|
| 379 |
+
},
|
| 380 |
+
{
|
| 381 |
+
"audio": "jfk.wav",
|
| 382 |
+
"weights": "awq",
|
| 383 |
+
"attn": "fa2",
|
| 384 |
+
"text": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
|
| 385 |
+
"rtf": 0.035,
|
| 386 |
+
"decode_tok_s": 152.2,
|
| 387 |
+
"prefill_tok_s": 922.0,
|
| 388 |
+
"lm_layers_mb": 3306.0,
|
| 389 |
+
"load_ms": 4315.0
|
| 390 |
+
},
|
| 391 |
+
{
|
| 392 |
+
"audio": "jfk.wav",
|
| 393 |
+
"weights": "bf16",
|
| 394 |
+
"attn": "auto",
|
| 395 |
+
"text": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
|
| 396 |
+
"rtf": 0.071,
|
| 397 |
+
"decode_tok_s": 56.4,
|
| 398 |
+
"prefill_tok_s": 844.0,
|
| 399 |
+
"lm_layers_mb": 12446.0,
|
| 400 |
+
"load_ms": 1950.0
|
| 401 |
+
},
|
| 402 |
+
{
|
| 403 |
+
"audio": "test30.wav",
|
| 404 |
+
"weights": "awq",
|
| 405 |
+
"attn": "flashinfer",
|
| 406 |
+
"text": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you.",
|
| 407 |
+
"rtf": 0.036,
|
| 408 |
+
"decode_tok_s": 151.3,
|
| 409 |
+
"prefill_tok_s": 934.0,
|
| 410 |
+
"lm_layers_mb": 3306.0,
|
| 411 |
+
"load_ms": 4349.0
|
| 412 |
+
},
|
| 413 |
+
{
|
| 414 |
+
"audio": "test30.wav",
|
| 415 |
+
"weights": "awq",
|
| 416 |
+
"attn": "fa2",
|
| 417 |
+
"text": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you.",
|
| 418 |
+
"rtf": 0.036,
|
| 419 |
+
"decode_tok_s": 149.5,
|
| 420 |
+
"prefill_tok_s": 936.0,
|
| 421 |
+
"lm_layers_mb": 3306.0,
|
| 422 |
+
"load_ms": 4371.0
|
| 423 |
+
},
|
| 424 |
+
{
|
| 425 |
+
"audio": "test30.wav",
|
| 426 |
+
"weights": "bf16",
|
| 427 |
+
"attn": "auto",
|
| 428 |
+
"text": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you.",
|
| 429 |
+
"rtf": 0.076,
|
| 430 |
+
"decode_tok_s": 56.1,
|
| 431 |
+
"prefill_tok_s": 856.0,
|
| 432 |
+
"lm_layers_mb": 12446.0,
|
| 433 |
+
"load_ms": 1918.0
|
| 434 |
+
},
|
| 435 |
+
{
|
| 436 |
+
"audio": "test120.wav",
|
| 437 |
+
"weights": "awq",
|
| 438 |
+
"attn": "flashinfer",
|
| 439 |
+
"text": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you,ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you,ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your.",
|
| 440 |
+
"rtf": 0.034,
|
| 441 |
+
"decode_tok_s": 149.2,
|
| 442 |
+
"prefill_tok_s": 959.0,
|
| 443 |
+
"lm_layers_mb": 3306.0,
|
| 444 |
+
"load_ms": 4305.0
|
| 445 |
+
},
|
| 446 |
+
{
|
| 447 |
+
"audio": "test120.wav",
|
| 448 |
+
"weights": "awq",
|
| 449 |
+
"attn": "fa2",
|
| 450 |
+
"text": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you,ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you,ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your.",
|
| 451 |
+
"rtf": 0.035,
|
| 452 |
+
"decode_tok_s": 142.8,
|
| 453 |
+
"prefill_tok_s": 951.0,
|
| 454 |
+
"lm_layers_mb": 3306.0,
|
| 455 |
+
"load_ms": 4313.0
|
| 456 |
+
},
|
| 457 |
+
{
|
| 458 |
+
"audio": "test120.wav",
|
| 459 |
+
"weights": "bf16",
|
| 460 |
+
"attn": "auto",
|
| 461 |
+
"text": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you,ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your country can do for you,ask what you can do for your country. He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick peppered flour fattened sauce. And so, my fellow Americans, ask not what your.",
|
| 462 |
+
"rtf": 0.074,
|
| 463 |
+
"decode_tok_s": 55.9,
|
| 464 |
+
"prefill_tok_s": 871.0,
|
| 465 |
+
"lm_layers_mb": 12446.0,
|
| 466 |
+
"load_ms": 1826.0
|
| 467 |
+
}
|
| 468 |
+
],
|
| 469 |
+
"evaluation_scope": "English read speech (LibriSpeech), one- and two-speaker streams. Not a multilingual, noisy-audio or diarization benchmark."
|
| 470 |
+
}
|
merges.txt
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
model-00001.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c00b652e53caefef0e5b3249fe14e1b8d43b9d365b4f909a7db3fbd2b9cad405
|
| 3 |
+
size 3390166648
|
model-00002.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5745ac1e81e127813ae6fe3d06883fd2f591ea0cdbb231e442c0e382e90a90d8
|
| 3 |
+
size 3610168760
|
model.safetensors.index.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
preprocessor_config.json
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"speech_tok_compress_ratio": 3200,
|
| 3 |
+
"target_sample_rate": 24000,
|
| 4 |
+
"normalize_audio": false,
|
| 5 |
+
"chunk_frames": 22,
|
| 6 |
+
"lookahead_frames": 4,
|
| 7 |
+
"processor_class": "VibeVoiceASRProcessor"
|
| 8 |
+
}
|
quantization_environment.json
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"torch": "2.13.0",
|
| 3 |
+
"transformers": "5.14.1",
|
| 4 |
+
"llmcompressor": "0.13.0",
|
| 5 |
+
"compressed-tensors": "0.18.0",
|
| 6 |
+
"datasets": "5.0.1",
|
| 7 |
+
"safetensors": "0.8.0",
|
| 8 |
+
"huggingface-hub": "1.32.0"
|
| 9 |
+
}
|
recipe.yaml
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
default_stage:
|
| 2 |
+
default_modifiers:
|
| 3 |
+
AWQModifier:
|
| 4 |
+
requires_calibration_data: true
|
| 5 |
+
mappings:
|
| 6 |
+
- smooth_layer: re:.*input_layernorm$
|
| 7 |
+
balance_layers: ['re:.*q_proj$', 're:.*k_proj$', 're:.*v_proj$']
|
| 8 |
+
activation_hook_target: null
|
| 9 |
+
- smooth_layer: re:.*v_proj$
|
| 10 |
+
balance_layers: ['re:.*o_proj$']
|
| 11 |
+
activation_hook_target: null
|
| 12 |
+
- smooth_layer: re:.*post_attention_layernorm$
|
| 13 |
+
balance_layers: ['re:.*gate_proj$', 're:.*up_proj$']
|
| 14 |
+
activation_hook_target: null
|
| 15 |
+
- smooth_layer: re:.*up_proj$
|
| 16 |
+
balance_layers: ['re:.*down_proj$']
|
| 17 |
+
activation_hook_target: null
|
| 18 |
+
offload_device: !!python/object/apply:torch.device [cpu]
|
| 19 |
+
duo_scaling: both
|
| 20 |
+
n_grid: 40
|
| 21 |
+
QuantizationModifier:
|
| 22 |
+
targets: [Linear]
|
| 23 |
+
ignore: [lm_head]
|
| 24 |
+
scheme: W4A16_ASYM
|
| 25 |
+
bypass_divisibility_checks: false
|
| 26 |
+
requires_calibration_data: false
|
reproduce/README.template.md
ADDED
|
@@ -0,0 +1,101 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
base_model: {{BASE_ID}}
|
| 4 |
+
base_model_relation: quantized
|
| 5 |
+
pipeline_tag: automatic-speech-recognition
|
| 6 |
+
library_name: transformers
|
| 7 |
+
tags:
|
| 8 |
+
- vibevoice
|
| 9 |
+
- awq
|
| 10 |
+
- w4a16
|
| 11 |
+
- asymmetric
|
| 12 |
+
- llmcompressor
|
| 13 |
+
- speech-to-text
|
| 14 |
+
- streaming
|
| 15 |
+
- diarization
|
| 16 |
+
---
|
| 17 |
+
|
| 18 |
+
# {{BASE_NAME}} AWQ W4A16 ASYM
|
| 19 |
+
|
| 20 |
+
Activation-aware INT4 quantization of [{{BASE_ID}}](https://huggingface.co/{{BASE_ID}}), revision `{{REV}}`: the streaming VibeVoice ASR model, which writes text for every 2.93 s of audio as it arrives, with 0.53 s of lookahead.
|
| 21 |
+
|
| 22 |
+
**Weights: {{SRC_GB}} GB → {{OUT_GB}} GB ({{RATIO}}× smaller).** Sizes are decimal GB of safetensors, including every retained speech component and the language model.
|
| 23 |
+
|
| 24 |
+
Part of the [VibeVoice-ASR Quantized](https://huggingface.co/collections/Ar4ikov/vibevoice-asr-quantized) collection, built the same way as [Ar4ikov/VibeVoice-ASR-AWQ-W4A16-ASYM](https://huggingface.co/Ar4ikov/VibeVoice-ASR-AWQ-W4A16-ASYM).
|
| 25 |
+
|
| 26 |
+
## Format and scope
|
| 27 |
+
|
| 28 |
+
- Genuine activation-aware AWQ scale search using llm-compressor; **not** RTN or a conversion of an already quantized checkpoint.
|
| 29 |
+
- All 196 Qwen2 attention/MLP projections: 4-bit asymmetric, group size 128, 16-bit activations.
|
| 30 |
+
- 40 grid points with `duo_scaling="both"`; seed 42.
|
| 31 |
+
- Embeddings, speech encoders/connectors, norms, and biases retained in BF16. {{LM_HEAD}} AWQ group scales stored in FP16 with an exact representability check.
|
| 32 |
+
- Original `model.acoustic_tokenizer.decoder.*` removed: this audio synthesis decoder is unused by the ASR path. Saves {{PRUNED_MB}} MB. This is an **ASR-only** checkpoint.
|
| 33 |
+
- The calibrated compressed-tensors checkpoint is **losslessly repacked into standard AWQ GEMM** (`qweight`, `qzeros`, `scales`). Every integer code round-trips exactly and every scale value is preserved. There is no second quantization pass.
|
| 34 |
+
- Tokenizer files and `preprocessor_config.json` are the base checkpoint's, unchanged (`<|text_chunk_end|>`, `chunk_frames` 22, `lookahead_frames` 4, `normalize_audio` false).
|
| 35 |
+
|
| 36 |
+
## Calibration
|
| 37 |
+
|
| 38 |
+
256 sequences, maximum 2048 rows each, calibrated through `inputs_embeds` so real speech embeddings are in the data:
|
| 39 |
+
|
| 40 |
+
- **128 complete streaming sessions** in the model's own protocol: the streaming prompt, then for every 26-frame window `<|object_ref_start|>` + speech features + `<|object_ref_end|>`, the BF16 model's own greedy chunk text, and `<|text_chunk_end|>` (about {{CALIB_ROWS}} rows per session). Audio: LibriSpeech dev-clean, 32 speakers, streams of 15–150 s built from consecutive utterances of one chapter; about a third alternate between two speakers, so speaker turns appear in the data.
|
| 41 |
+
- **128 Ultrachat conversations** (`HuggingFaceH4/ultrachat_200k`, `train_sft`, shuffled with seed 42) for the generic language-model path.
|
| 42 |
+
|
| 43 |
+
Speech features come from the BF16 checkpoint's encoders run in FP32 with the deterministic acoustic latent (the mean), each window encoded on its own exactly as upstream `streaming_generate` does.
|
| 44 |
+
|
| 45 |
+
## Validation
|
| 46 |
+
|
| 47 |
+
{{NSTREAMS}} held-out streams, {{MINUTES}} minutes, {{NWORDS}} reference words, from the 8 LibriSpeech dev-clean speakers excluded from calibration ({{NTWO}} of the streams alternate between two speakers):
|
| 48 |
+
|
| 49 |
+
| Path | Word errors | WER |
|
| 50 |
+
|---|---:|---:|
|
| 51 |
+
| Original BF16, PyTorch, upstream streaming protocol | {{E_baseline}} | {{W_baseline}} |
|
| 52 |
+
| Final AWQ files independently unpacked for PyTorch | {{E_awq_pytorch}} | {{W_awq_pytorch}} |
|
| 53 |
+
| Original BF16, `vibevoice.c --quant none` | {{E_c_bf16}} | {{W_c_bf16}} |
|
| 54 |
+
| **Final AWQ files, native INT4 `vibevoice.c`** | {{E_c_awq}} | {{W_c_awq}} |
|
| 55 |
+
|
| 56 |
+
`Speaker N:` labels and punctuation are removed before scoring. This is an English read-speech regression check, not a representative quality benchmark: multilingual accuracy, noisy audio and diarization quality after quantization have not been measured. `evaluation.json` has every reference and output.
|
| 57 |
+
|
| 58 |
+
### Speed with vibevoice.c (RTX 3090)
|
| 59 |
+
|
| 60 |
+
Native W4A16 kernels (no dequantization to FP16), whole file streamed chunk by chunk:
|
| 61 |
+
|
| 62 |
+
| Audio | AWQ, `--attn flashinfer` (default for this model) | AWQ, `--attn fa2` | BF16 `--quant none` |
|
| 63 |
+
|---|---|---|---|
|
| 64 |
+
| 11 s | RTF {{jfk_awq_flashinfer_rtf}}, {{jfk_awq_flashinfer_dec}} tok/s | RTF {{jfk_awq_fa2_rtf}}, {{jfk_awq_fa2_dec}} tok/s | RTF {{jfk_bf16_auto_rtf}}, {{jfk_bf16_auto_dec}} tok/s |
|
| 65 |
+
| 30 s | RTF {{test30_awq_flashinfer_rtf}}, {{test30_awq_flashinfer_dec}} tok/s | RTF {{test30_awq_fa2_rtf}}, {{test30_awq_fa2_dec}} tok/s | RTF {{test30_bf16_auto_rtf}}, {{test30_bf16_auto_dec}} tok/s |
|
| 66 |
+
| 120 s | RTF {{test120_awq_flashinfer_rtf}}, {{test120_awq_flashinfer_dec}} tok/s | RTF {{test120_awq_fa2_rtf}}, {{test120_awq_fa2_dec}} tok/s | RTF {{test120_bf16_auto_rtf}}, {{test120_bf16_auto_dec}} tok/s |
|
| 67 |
+
|
| 68 |
+
Language-model layers on the GPU: {{LM_AWQ_MB}} MB (AWQ) vs {{LM_BF16_MB}} MB (BF16).
|
| 69 |
+
|
| 70 |
+
## Usage
|
| 71 |
+
|
| 72 |
+
```python
|
| 73 |
+
from huggingface_hub import snapshot_download
|
| 74 |
+
snapshot_download("{{REPO}}", local_dir="{{NAME}}")
|
| 75 |
+
```
|
| 76 |
+
|
| 77 |
+
With [vibevoice.c](https://github.com/Ar4ikov/vibevoice.c):
|
| 78 |
+
|
| 79 |
+
```bash
|
| 80 |
+
vv_cli --model ./{{NAME}} --audio talk.wav --output transcript.json # prints chunks as they are ready
|
| 81 |
+
vv_cli mic --model ./{{NAME}} # live microphone
|
| 82 |
+
vv_cli serve --model ./{{NAME}} --slots 8 # SSE stream=true and WebSocket /v1/audio/stream
|
| 83 |
+
```
|
| 84 |
+
|
| 85 |
+
This custom architecture is not supported by plain `AutoModelForCausalLM` without VibeVoice integration, and vLLM deployment was not tested. The acoustic decoder was pruned, so integrations must tolerate its absent unused weights.
|
| 86 |
+
|
| 87 |
+
### Streaming protocol
|
| 88 |
+
|
| 89 |
+
Audio is 24 kHz mono and is **not** loudness-normalized. The prompt is plain text, tokenized without special tokens:
|
| 90 |
+
|
| 91 |
+
```text
|
| 92 |
+
You are a helpful assistant that transcribes audio input into text output. Please transcribe the following audios streamingly with these keys: speaker, content\n
|
| 93 |
+
```
|
| 94 |
+
|
| 95 |
+
Then, for every window of 22 + 4 frames (83200 samples, advancing by 70400; the last one zero-padded), encoded on its own: prefill `<|object_ref_start|>`, the 26 speech feature rows, `<|object_ref_end|>`; decode greedily until `<|text_chunk_end|>` or `<|endoftext|>`; feed `<|text_chunk_end|>`. EOS does not end the session. Speaker turns appear inline as ` \n Speaker N:`.
|
| 96 |
+
|
| 97 |
+
## Reproduction
|
| 98 |
+
|
| 99 |
+
`recipe.yaml` is the actual compression recipe. `reproduce/` has the whole pipeline (`pipeline.sh`): data preparation, checkpoint split and calibration sessions (`prepare_model.py`, `stream_ref.py`), quantization, exact AWQ repacking, both evaluations and this card. Set `VV_QUANT_WORKDIR`, `VV_QUANT_DATA`, `VIBEVOICE_SOURCE` (Microsoft VibeVoice source checkout) and `VV_CLI`. Preparation and the PyTorch comparison used torch 2.6.0+cu124 and transformers 4.51.3; the quantization environment is in `quantization_environment.json`. One NVIDIA RTX 3090 24 GB.
|
| 100 |
+
|
| 101 |
+
License: MIT, inherited from the base model. No audio or HF credentials are included.
|
reproduce/evaluate.py
ADDED
|
@@ -0,0 +1,58 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Read the FINAL AWQ GEMM files independently and rerun the held-out streams.
|
| 2 |
+
|
| 3 |
+
The INT4 codes are unpacked to BF16 by hand (no AWQ library in the loop), put
|
| 4 |
+
into a plain Qwen2ForCausalLM, and driven through the same streaming loop as
|
| 5 |
+
the BF16 baseline, over the same per-window speech features.
|
| 6 |
+
"""
|
| 7 |
+
import os, sys, json, time
|
| 8 |
+
from pathlib import Path
|
| 9 |
+
import torch
|
| 10 |
+
from safetensors import safe_open
|
| 11 |
+
from transformers import Qwen2Config, Qwen2ForCausalLM, Qwen2TokenizerFast
|
| 12 |
+
from accelerate import init_empty_weights
|
| 13 |
+
sys.path.insert(0, str(Path(__file__).parent))
|
| 14 |
+
import stream_ref
|
| 15 |
+
|
| 16 |
+
ROOT = Path(os.environ['VV_QUANT_WORKDIR']); OUT = ROOT / 'output'
|
| 17 |
+
torch.set_grad_enabled(False); torch.set_num_threads(8)
|
| 18 |
+
cfg = json.loads((OUT / 'config.json').read_text()); qc = Qwen2Config(**cfg['decoder_config'])
|
| 19 |
+
qc.tie_word_embeddings = bool(cfg['tie_word_embeddings']); qc._attn_implementation = 'sdpa'
|
| 20 |
+
with init_empty_weights(): m = Qwen2ForCausalLM(qc)
|
| 21 |
+
handles = [safe_open(p, framework='pt') for p in sorted(OUT.glob('*.safetensors'))]
|
| 22 |
+
lookup = {k: h for h in handles for k in h.keys()}
|
| 23 |
+
get = lambda k: lookup[k].get_tensor(k)
|
| 24 |
+
ORDER = [0, 4, 1, 5, 2, 6, 3, 7]
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def unpack(p):
|
| 28 |
+
p = p.to(torch.int64); v = torch.stack([(p >> (4 * j)) & 15 for j in ORDER], dim=-1)
|
| 29 |
+
return v.reshape(p.shape[0], p.shape[1] * 8).T.contiguous()
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
dev = 'cuda:0'; state = {}
|
| 33 |
+
for k in sorted(lookup):
|
| 34 |
+
if not (k.startswith('model.language_model.') or k.startswith('lm_head.')): continue
|
| 35 |
+
n = 'model.' + k[len('model.language_model.'):] if k.startswith('model.language_model.') else k
|
| 36 |
+
if k.endswith('.qzeros') or k.endswith('.scales'): continue
|
| 37 |
+
if k.endswith('.qweight'):
|
| 38 |
+
p = k[:-len('.qweight')]; q = unpack(get(k)); z = unpack(get(p + '.qzeros')); s = get(p + '.scales').T.float()
|
| 39 |
+
w = ((q.reshape(q.shape[0], -1, 128).float() - z[:, :, None]) * s[:, :, None]).reshape(q.shape).to(torch.bfloat16)
|
| 40 |
+
state[n[:-len('.qweight')] + '.weight'] = w.to(dev)
|
| 41 |
+
else:
|
| 42 |
+
state[n] = get(k).to(dev)
|
| 43 |
+
if qc.tie_word_embeddings:
|
| 44 |
+
assert 'lm_head.weight' not in state
|
| 45 |
+
state['lm_head.weight'] = state['model.embed_tokens.weight']
|
| 46 |
+
info = m.load_state_dict(state, strict=True, assign=True); m.eval(); del state
|
| 47 |
+
if qc.tie_word_embeddings: m.tie_weights()
|
| 48 |
+
print('FINAL_ARTIFACT_LOAD_OK', info, flush=True)
|
| 49 |
+
tok = Qwen2TokenizerFast.from_pretrained(str(OUT))
|
| 50 |
+
rows = json.loads((ROOT / 'baseline.json').read_text()); res = []
|
| 51 |
+
for i, row in enumerate(rows):
|
| 52 |
+
feats = torch.load(ROOT / 'evaluation' / f'eval-{i:03d}.pt', weights_only=True)
|
| 53 |
+
t = time.time(); texts, _ = stream_ref.run(m, tok, list(feats))
|
| 54 |
+
txt = ''.join(texts); print('AWQ', i, repr(txt[:200]), flush=True)
|
| 55 |
+
res.append({'wav': row['wav'], 'reference': row['reference'], 'baseline': row['text'], 'awq': txt,
|
| 56 |
+
'awq_chunks': texts, 'generation_seconds': time.time() - t})
|
| 57 |
+
(ROOT / 'awq_eval.json').write_text(json.dumps(res, indent=1))
|
| 58 |
+
print('PYTHON_EVAL_DONE', flush=True)
|
reproduce/evaluate_c.py
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Native INT4 runs with vibevoice.c: held-out streams, plus speed per attention backend.
|
| 2 |
+
|
| 3 |
+
Every run is audio in, JSON out. The BF16 checkpoint runs through the same
|
| 4 |
+
binary with --quant none (which matches upstream streaming_generate token
|
| 5 |
+
for token) so the C numbers have a C baseline next to them.
|
| 6 |
+
"""
|
| 7 |
+
import json, os, re, subprocess
|
| 8 |
+
from pathlib import Path
|
| 9 |
+
|
| 10 |
+
ROOT = Path(os.environ['VV_QUANT_WORKDIR']); OUT = ROOT / 'output'; CLI = os.environ['VV_CLI']
|
| 11 |
+
AUDIO = Path(os.environ['VV_BENCH_AUDIO'])
|
| 12 |
+
D = ROOT / 'c-eval'; D.mkdir(exist_ok=True)
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
def run(model, wav, tag, extra=()):
|
| 16 |
+
dst = D / f'{tag}.json'; log = D / f'{tag}.log'
|
| 17 |
+
cmd = [CLI, '--model', str(model), '--audio', str(wav), '--output', str(dst), '--gpu', '0', *extra]
|
| 18 |
+
with open(log, 'w') as f: subprocess.run(cmd, stdout=f, stderr=subprocess.STDOUT, check=True)
|
| 19 |
+
txt = log.read_text(errors='replace')
|
| 20 |
+
perf = {}
|
| 21 |
+
for key, pat in [('rtf', r'RTF\s+([0-9.]+)'), ('decode_tok_s', r'Decode speed\s+([0-9.]+)'),
|
| 22 |
+
('prefill_tok_s', r'Prefill speed\s+([0-9.]+)'), ('lm_layers_mb', r'on int8 activations, ([0-9.]+) MB'), ('load_ms', r'model loaded in ([0-9]+) ms')]:
|
| 23 |
+
mm = re.search(pat, txt)
|
| 24 |
+
if mm: perf[key] = float(mm.group(1))
|
| 25 |
+
return json.loads(dst.read_text())['text'], perf
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
rows = json.loads((ROOT / 'baseline.json').read_text()); result = []
|
| 29 |
+
for i, row in enumerate(rows):
|
| 30 |
+
awq, pa = run(OUT, row['wav'], f'eval{i}-awq')
|
| 31 |
+
bf, pb = run(ROOT / 'base', row['wav'], f'eval{i}-bf16', ['--quant', 'none'])
|
| 32 |
+
result.append({'wav': row['wav'], 'reference': row['reference'], 'c_awq': awq, 'c_bf16': bf, 'perf_awq': pa, 'perf_bf16': pb})
|
| 33 |
+
print('C_RESULT', i, repr(awq[:160]), flush=True)
|
| 34 |
+
bench = []
|
| 35 |
+
for name in ['jfk.wav', 'test30.wav', 'test120.wav']:
|
| 36 |
+
for attn in ['flashinfer', 'fa2']:
|
| 37 |
+
txt, p = run(OUT, AUDIO / name, f'bench-{name[:-4]}-awq-{attn}', ['--attn', attn])
|
| 38 |
+
bench.append({'audio': name, 'weights': 'awq', 'attn': attn, 'text': txt, **p}); print('BENCH', name, 'awq', attn, p, flush=True)
|
| 39 |
+
txt, p = run(ROOT / 'base', AUDIO / name, f'bench-{name[:-4]}-bf16', ['--quant', 'none'])
|
| 40 |
+
bench.append({'audio': name, 'weights': 'bf16', 'attn': 'auto', 'text': txt, **p}); print('BENCH', name, 'bf16', p, flush=True)
|
| 41 |
+
(ROOT / 'c_eval.json').write_text(json.dumps({'held_out': result, 'bench': bench}, indent=1))
|
| 42 |
+
print('C_EVAL_DONE', flush=True)
|
reproduce/export_awq.py
ADDED
|
@@ -0,0 +1,74 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import os
|
| 2 |
+
"""Repack calibrated llm-compressor INT4 into standard AWQ GEMM, without requantizing."""
|
| 3 |
+
import json,shutil,math
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
import torch
|
| 6 |
+
from safetensors import safe_open
|
| 7 |
+
from safetensors.torch import save_file
|
| 8 |
+
from compressed_tensors.compressors.pack_quantized.helpers import unpack_from_int32
|
| 9 |
+
ROOT=Path(os.environ['VV_QUANT_WORKDIR']);SRC=ROOT/'decoder-awq';OUT=ROOT/'output';OUT.mkdir(exist_ok=True)
|
| 10 |
+
torch.set_num_threads(8)
|
| 11 |
+
files=sorted(SRC.glob('*.safetensors')); handles=[safe_open(p,framework='pt') for p in files]
|
| 12 |
+
lookup={k:h for h in handles for k in h.keys()}
|
| 13 |
+
def get(k):return lookup[k].get_tensor(k)
|
| 14 |
+
ORDER=[0,4,1,5,2,6,3,7]
|
| 15 |
+
def pack_awq(v):
|
| 16 |
+
# [N,K] signed INT4 -> [K,N/8] unsigned packed AWQ.
|
| 17 |
+
v=(v.to(torch.int32)+8).T.contiguous();assert int(v.min())>=0 and int(v.max())<=15
|
| 18 |
+
v=v.reshape(v.shape[0],-1,8);p=torch.zeros(v.shape[:2],dtype=torch.int32)
|
| 19 |
+
for j in range(8):p|=v[:,:,j]<<(4*ORDER[j])
|
| 20 |
+
# Verify every code after layout conversion.
|
| 21 |
+
for j in range(8):assert torch.equal((p>>(4*ORDER[j]))&15,v[:,:,j])
|
| 22 |
+
return p
|
| 23 |
+
buf={};weight_map={};parts=[];size=0;total=0
|
| 24 |
+
|
| 25 |
+
def flush():
|
| 26 |
+
global buf,size
|
| 27 |
+
if not buf:return
|
| 28 |
+
name=f'model-{len(parts)+1:05d}.safetensors';save_file(buf,str(OUT/name),metadata={'format':'pt'})
|
| 29 |
+
weight_map.update({k:name for k in buf});parts.append(name);print('SHARD',name,size,flush=True);buf={};size=0
|
| 30 |
+
|
| 31 |
+
def add(k,v):
|
| 32 |
+
global size,total
|
| 33 |
+
b=v.numel()*v.element_size()
|
| 34 |
+
if size+b>3_800_000_000:flush()
|
| 35 |
+
buf[k]=v.contiguous();size+=b;total+=b
|
| 36 |
+
|
| 37 |
+
def rename(k):return 'model.language_model.'+k[len('model.'):] if k.startswith('model.') else k
|
| 38 |
+
mods=sorted(k[:-len('.weight_packed')] for k in lookup if k.endswith('.weight_packed'));assert len(mods)==196
|
| 39 |
+
skip=set()
|
| 40 |
+
for i,k in enumerate(mods):
|
| 41 |
+
shape=tuple(get(k+'.weight_shape').tolist());sc=get(k+'.weight_scale');zp=get(k+'.weight_zero_point')
|
| 42 |
+
w=unpack_from_int32(get(k+'.weight_packed'),4,shape)
|
| 43 |
+
z=unpack_from_int32(zp,4,(shape[0],sc.shape[1]),packed_dim=0)
|
| 44 |
+
assert sc.shape==(shape[0],shape[1]//128)
|
| 45 |
+
# BF16 scales are exactly representable in FP16 in this range.
|
| 46 |
+
sf=sc.to(torch.float16);assert torch.equal(sc.float(),sf.float()),'Scale conversion would be lossy'
|
| 47 |
+
add(rename(k)+'.qweight',pack_awq(w));add(rename(k)+'.qzeros',pack_awq(z));add(rename(k)+'.scales',sf.T)
|
| 48 |
+
skip.update(k+'.'+s for s in ['weight_packed','weight_shape','weight_scale','weight_zero_point'])
|
| 49 |
+
print('REPACK',i+1,k,flush=True)
|
| 50 |
+
cfg=json.loads((ROOT/'base/config.json').read_text());TIED=bool(cfg['decoder_config'].get('tie_word_embeddings',False))
|
| 51 |
+
for k in sorted(lookup):
|
| 52 |
+
if k in skip:continue
|
| 53 |
+
# A tied head is the embedding table; vibevoice.c and transformers share one buffer.
|
| 54 |
+
if TIED and k=='lm_head.weight':continue
|
| 55 |
+
assert not any(s in k for s in ('weight_g_idx','input_global_scale','weight_scale','weight_packed'))
|
| 56 |
+
v=get(k);add(rename(k),v.to(torch.bfloat16) if v.is_floating_point() else v)
|
| 57 |
+
with safe_open(ROOT/'speech.safetensors',framework='pt') as f:
|
| 58 |
+
for k in f.keys():add(k,f.get_tensor(k))
|
| 59 |
+
flush()
|
| 60 |
+
(OUT/'model.safetensors.index.json').write_text(json.dumps({'metadata':{'total_size':total},'weight_map':weight_map},indent=2))
|
| 61 |
+
cfg['dtype']='bfloat16';cfg['torch_dtype']='bfloat16';cfg['tie_word_embeddings']=TIED
|
| 62 |
+
cfg['quantization_config']={'quant_method':'awq','bits':4,'group_size':128,'version':'gemm','zero_point':True,'modules_to_not_convert':['lm_head','model.acoustic_tokenizer','model.semantic_tokenizer','model.acoustic_connector','model.semantic_connector']}
|
| 63 |
+
(OUT/'config.json').write_text(json.dumps(cfg,indent=2))
|
| 64 |
+
for p in (ROOT/'processor').iterdir():
|
| 65 |
+
if p.is_file():shutil.copy2(p,OUT/p.name)
|
| 66 |
+
# The checkpoint's own tokenizer files, unchanged (it carries <|text_chunk_end|>).
|
| 67 |
+
for name in ['tokenizer.json','tokenizer_config.json','special_tokens_map.json','added_tokens.json','vocab.json','merges.txt','preprocessor_config.json']:
|
| 68 |
+
if (ROOT/'base'/name).exists():shutil.copy2(ROOT/'base'/name,OUT/name)
|
| 69 |
+
for name in ['recipe.yaml']:
|
| 70 |
+
if (SRC/name).exists():shutil.copy2(SRC/name,OUT/name)
|
| 71 |
+
shutil.copy2(SRC/'config.json',OUT/'calibrated_decoder_config.json')
|
| 72 |
+
source_bytes=sum(p.stat().st_size for p in (ROOT/'base').glob('*.safetensors'));out_bytes=sum(p.stat().st_size for p in OUT.glob('*.safetensors'))
|
| 73 |
+
report={'source_revision':(ROOT/'source_revision.txt').read_text().strip(),'source_safetensors_bytes':source_bytes,'output_safetensors_bytes':out_bytes,'compression_ratio':source_bytes/out_bytes,'quantized_projections':len(mods),'method':'Activation-aware AWQ using llm-compressor; exact layout repack into AWQ GEMM','lm_head':'tied to embed_tokens' if TIED else 'BF16','scheme':'W4A16_ASYM','group_size':128,'duo_scaling':'both','grid_points':40,'calibration_samples':256,'audio_calibration_samples':128,'text_calibration_samples':128,'calibration_stats':json.loads((ROOT/'calibration_stats.json').read_text()),'max_sequence_length':2048,'packing_validation':'All integer weight and zero-point codes round-trip exactly; all scale values preserved exactly','pruning':json.loads((ROOT/'pruning.json').read_text())}
|
| 74 |
+
(OUT/'compression_report.json').write_text(json.dumps(report,indent=2));print('EXPORT_DONE',json.dumps(report),flush=True)
|
reproduce/finalize.py
ADDED
|
@@ -0,0 +1,89 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Metrics, model card, reproduction scripts and hashes for one output dir."""
|
| 2 |
+
import json, os, re, shutil, hashlib, importlib.metadata, sys
|
| 3 |
+
from pathlib import Path
|
| 4 |
+
|
| 5 |
+
ROOT = Path(os.environ['VV_QUANT_WORKDIR']); OUT = ROOT / 'output'; PIPE = Path(__file__).parent
|
| 6 |
+
BASE_ID = os.environ['VV_BASE_ID']; REPO = os.environ['VV_REPO_ID']
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def words(t):
|
| 10 |
+
t = re.sub(r'Speaker\s*\d+\s*:', ' ', t)
|
| 11 |
+
return re.findall(r"[A-Z0-9]+(?:'[A-Z0-9]+)*", t.upper().replace('’', "'"))
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
def distance(a, b):
|
| 15 |
+
prev = list(range(len(b) + 1))
|
| 16 |
+
for i, x in enumerate(a, 1):
|
| 17 |
+
cur = [i]
|
| 18 |
+
for j, y in enumerate(b, 1):
|
| 19 |
+
cur.append(min(cur[-1] + 1, prev[j] + 1, prev[j - 1] + (x != y)))
|
| 20 |
+
prev = cur
|
| 21 |
+
return prev[-1]
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
r = json.loads((OUT / 'compression_report.json').read_text())
|
| 25 |
+
ev = json.loads((ROOT / 'awq_eval.json').read_text()); ce = json.loads((ROOT / 'c_eval.json').read_text())
|
| 26 |
+
man = json.loads((Path(os.environ['VV_QUANT_DATA']) / 'audio_manifest.json').read_text())
|
| 27 |
+
held = ce['held_out']; assert len(ev) == len(held) == len(man['evaluation'])
|
| 28 |
+
paths = {'baseline': 'BF16 PyTorch (upstream streaming protocol)', 'awq_pytorch': 'AWQ files unpacked to BF16, PyTorch',
|
| 29 |
+
'c_bf16': 'BF16, vibevoice.c `--quant none`', 'c_awq': 'AWQ, vibevoice.c native INT4'}
|
| 30 |
+
errors = {k: 0 for k in paths}; nwords = 0; minutes = 0.0
|
| 31 |
+
for a, b, s in zip(ev, held, man['evaluation']):
|
| 32 |
+
ref = words(s['text']); nwords += len(ref); minutes += s['duration'] / 60
|
| 33 |
+
vals = {'baseline': a['baseline'], 'awq_pytorch': a['awq'], 'c_bf16': b['c_bf16'], 'c_awq': b['c_awq']}
|
| 34 |
+
for k, v in vals.items():
|
| 35 |
+
errors[k] += distance(ref, words(v))
|
| 36 |
+
wers = {k: v / nwords for k, v in errors.items()}
|
| 37 |
+
assert r['quantized_projections'] == 196 and r['compression_ratio'] > 1.8, r
|
| 38 |
+
assert wers['awq_pytorch'] <= wers['baseline'] + 0.03, wers
|
| 39 |
+
assert wers['c_awq'] <= wers['c_bf16'] + 0.03, wers
|
| 40 |
+
bench = ce['bench']
|
| 41 |
+
metrics = {'dataset': 'LibriSpeech dev-clean, 8 held-out speakers', 'streams': len(ev), 'minutes': minutes,
|
| 42 |
+
'two_speaker_streams': sum(s['two_speakers'] for s in man['evaluation']), 'reference_words': nwords,
|
| 43 |
+
'normalization': "Uppercase ASCII words, 'Speaker N:' labels and punctuation removed",
|
| 44 |
+
'word_errors': errors, 'wer': wers, 'held_out_speakers': man['held_out_speakers'],
|
| 45 |
+
'samples_detail': [{**a, 'c_awq': b['c_awq'], 'c_bf16': b['c_bf16']} for a, b in zip(ev, held)],
|
| 46 |
+
'vibevoice_c_bench_rtx3090': bench,
|
| 47 |
+
'evaluation_scope': 'English read speech (LibriSpeech), one- and two-speaker streams. Not a multilingual, noisy-audio or diarization benchmark.'}
|
| 48 |
+
(OUT / 'evaluation.json').write_text(json.dumps(metrics, indent=1))
|
| 49 |
+
versions = {k: importlib.metadata.version(k) for k in ['torch', 'transformers', 'llmcompressor', 'compressed-tensors', 'datasets', 'safetensors', 'huggingface-hub']}
|
| 50 |
+
(OUT / 'quantization_environment.json').write_text(json.dumps(versions, indent=2))
|
| 51 |
+
rep = OUT / 'reproduce'; rep.mkdir(exist_ok=True)
|
| 52 |
+
for name in ['pipeline.sh', 'prepare_data.py', 'stream_ref.py', 'prepare_model.py', 'quantize.py', 'export_awq.py', 'evaluate.py', 'evaluate_c.py', 'finalize.py', 'README.template.md']:
|
| 53 |
+
shutil.copy2(PIPE / name, rep / name)
|
| 54 |
+
assert (OUT / 'recipe.yaml').exists()
|
| 55 |
+
shutil.copy2(Path(os.environ['VIBEVOICE_SOURCE']) / 'LICENSE', OUT / 'LICENSE')
|
| 56 |
+
card = (PIPE / 'README.template.md').read_text()
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
def b(audio, w, attn):
|
| 60 |
+
for x in bench:
|
| 61 |
+
if x['audio'] == audio and x['weights'] == w and x['attn'] == attn: return x
|
| 62 |
+
return {}
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
fmt = {'REPO': REPO, 'BASE_ID': BASE_ID, 'NAME': REPO.split('/')[1], 'BASE_NAME': BASE_ID.split('/')[1],
|
| 66 |
+
'REV': r['source_revision'], 'SRC_GB': f"{r['source_safetensors_bytes'] / 1e9:.3f}",
|
| 67 |
+
'OUT_GB': f"{r['output_safetensors_bytes'] / 1e9:.3f}", 'RATIO': f"{r['compression_ratio']:.2f}",
|
| 68 |
+
'PRUNED_MB': f"{r['pruning']['removed_bf16_bytes'] / 1e6:.1f}", 'NWORDS': nwords, 'NSTREAMS': len(ev),
|
| 69 |
+
'MINUTES': f'{minutes:.1f}', 'NTWO': metrics['two_speaker_streams'],
|
| 70 |
+
'CALIB_ROWS': f"{r['calibration_stats']['audio_rows_mean']:.0f}"}
|
| 71 |
+
for k in paths:
|
| 72 |
+
fmt['E_' + k] = errors[k]; fmt['W_' + k] = f'{wers[k]:.2%}'
|
| 73 |
+
for audio in ['jfk.wav', 'test30.wav', 'test120.wav']:
|
| 74 |
+
t = audio[:-4]
|
| 75 |
+
for w, attn in [('awq', 'flashinfer'), ('awq', 'fa2'), ('bf16', 'auto')]:
|
| 76 |
+
x = b(audio, w, attn)
|
| 77 |
+
fmt[f'{t}_{w}_{attn}_rtf'] = f"{x.get('rtf', float('nan')):.3f}"
|
| 78 |
+
fmt[f'{t}_{w}_{attn}_dec'] = f"{x.get('decode_tok_s', float('nan')):.0f}"
|
| 79 |
+
x = b('test120.wav', 'awq', 'flashinfer'); y = b('test120.wav', 'bf16', 'auto')
|
| 80 |
+
fmt['LM_AWQ_MB'] = f"{x.get('lm_layers_mb', 0):.0f}"; fmt['LM_BF16_MB'] = f"{y.get('lm_layers_mb', 0):.0f}"
|
| 81 |
+
fmt['LM_HEAD'] = ('The LM head is tied to `embed_tokens` (as in the base checkpoint), so it is stored once.'
|
| 82 |
+
if r['lm_head'].startswith('tied') else 'The LM head is kept in BF16.')
|
| 83 |
+
for k, v in fmt.items():
|
| 84 |
+
card = card.replace('{{' + k + '}}', str(v))
|
| 85 |
+
assert '{{' not in card, re.findall(r'\{\{\w+\}\}', card)
|
| 86 |
+
(OUT / 'README.md').write_text(card)
|
| 87 |
+
(OUT / 'sha256.json').write_text(json.dumps({p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in OUT.glob('*.safetensors')}, indent=2))
|
| 88 |
+
(ROOT / 'validation_passed.json').write_text(json.dumps({'passed': True, 'wer': wers, 'output_safetensors_bytes': r['output_safetensors_bytes']}, indent=2))
|
| 89 |
+
print('VALIDATION_PASSED', json.dumps(wers), flush=True)
|
reproduce/pipeline.sh
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# pipeline.sh <tag>... e.g. pipeline.sh s15b s7b
|
| 3 |
+
# Stages leave a marker in their log; a stage whose marker is present is skipped,
|
| 4 |
+
# so the pipeline can be restarted after a failure.
|
| 5 |
+
set -uo pipefail
|
| 6 |
+
P=/root/vv3-nvme/awq-stream
|
| 7 |
+
W=/mnt/ssd0/vv3/awq-stream
|
| 8 |
+
QPY=$P/venv/bin/python
|
| 9 |
+
RPY=/mnt/ssd0/vibevoice/.venv/bin/python
|
| 10 |
+
export HF_HOME=/mnt/ssd0/vibevoice/hf_cache
|
| 11 |
+
export VV_QUANT_DATA=$W/data
|
| 12 |
+
export VIBEVOICE_SOURCE=/mnt/ssd0/vibevoice/ref_vibevoice
|
| 13 |
+
export VV_CLI=/root/vv3-nvme/master/build/vv_cli
|
| 14 |
+
export VV_BENCH_AUDIO=/mnt/ssd0/vibevoice/audio
|
| 15 |
+
export PYTHONUNBUFFERED=1
|
| 16 |
+
GPU="env GPU_RUN_MAX=${GPU_RUN_MAX:-43200} /root/vv3-nvme/bin/gpu-run 1"
|
| 17 |
+
|
| 18 |
+
stage() { # stage <dir> <name> <marker> <cmd...>
|
| 19 |
+
local dir=$1 name=$2 marker=$3; shift 3
|
| 20 |
+
if [ -f "$dir/$name.log" ] && grep -q "$marker" "$dir/$name.log"; then
|
| 21 |
+
echo "$(date '+%F %T') SKIP $dir $name"; return 0; fi
|
| 22 |
+
echo "$(date '+%F %T') START $dir $name"
|
| 23 |
+
"$@" > "$dir/$name.log" 2>&1
|
| 24 |
+
local rc=$?
|
| 25 |
+
if [ $rc -ne 0 ] || ! grep -q "$marker" "$dir/$name.log"; then
|
| 26 |
+
echo "$(date '+%F %T') FAILED $dir $name rc=$rc"; tail -30 "$dir/$name.log"; echo PIPELINE_FAILED; exit 1; fi
|
| 27 |
+
echo "$(date '+%F %T') DONE $dir $name"
|
| 28 |
+
}
|
| 29 |
+
|
| 30 |
+
mkdir -p $W/data
|
| 31 |
+
stage $W/data prepare_data DATA_DONE $QPY $P/pipe/prepare_data.py
|
| 32 |
+
for tag in "$@"; do
|
| 33 |
+
export VV_QUANT_WORKDIR=$W/$tag
|
| 34 |
+
stage $W/$tag prepare_model PREPARE_DONE $GPU $RPY $P/pipe/prepare_model.py
|
| 35 |
+
stage $W/$tag quantize QUANT_DONE $GPU $QPY $P/pipe/quantize.py
|
| 36 |
+
stage $W/$tag export EXPORT_DONE $QPY $P/pipe/export_awq.py
|
| 37 |
+
stage $W/$tag evaluate PYTHON_EVAL_DONE $GPU $RPY $P/pipe/evaluate.py
|
| 38 |
+
stage $W/$tag evaluate_c C_EVAL_DONE $GPU $QPY $P/pipe/evaluate_c.py
|
| 39 |
+
done
|
| 40 |
+
echo PIPELINE_DONE
|
reproduce/prepare_data.py
ADDED
|
@@ -0,0 +1,111 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Calibration and held-out streams for the VibeVoice-ASR-Streaming AWQ builds.
|
| 2 |
+
|
| 3 |
+
Audio: LibriSpeech dev-clean (openslr.org/12, read from the openslr/librispeech_asr
|
| 4 |
+
parquet on the Hub). Speakers are split once with a
|
| 5 |
+
fixed seed: 32 for calibration, 8 held out. A stream is consecutive
|
| 6 |
+
utterances of one chapter (long-form reading) or, for about a third of the
|
| 7 |
+
streams, utterances of two speakers taking turns, so speaker changes are in
|
| 8 |
+
the data. Streams are written as 24 kHz mono PCM16 WAV; both the PyTorch
|
| 9 |
+
reference and vibevoice.c read the very same file.
|
| 10 |
+
|
| 11 |
+
Text: 128 Ultrachat conversations (HuggingFaceH4/ultrachat_200k, train_sft,
|
| 12 |
+
shuffled with seed 42), which keeps the generic LM path in calibration.
|
| 13 |
+
"""
|
| 14 |
+
import os, json, random, subprocess, wave
|
| 15 |
+
from pathlib import Path
|
| 16 |
+
import numpy as np
|
| 17 |
+
|
| 18 |
+
DATA = Path(os.environ['VV_QUANT_DATA'])
|
| 19 |
+
SR = 24000
|
| 20 |
+
N_CALIB_AUDIO = 128
|
| 21 |
+
N_TEXT = 128
|
| 22 |
+
N_EVAL = 12
|
| 23 |
+
|
| 24 |
+
DATA.mkdir(parents=True, exist_ok=True)
|
| 25 |
+
# openslr/librispeech_asr, all/validation.clean = LibriSpeech dev-clean (FLAC bytes as released).
|
| 26 |
+
import pyarrow.parquet as pq
|
| 27 |
+
from huggingface_hub import hf_hub_download
|
| 28 |
+
pqf = hf_hub_download('openslr/librispeech_asr', 'all/validation.clean/0000.parquet', repo_type='dataset')
|
| 29 |
+
tbl = pq.read_table(pqf, columns=['id', 'audio', 'text', 'speaker_id', 'chapter_id']).to_pylist()
|
| 30 |
+
root = DATA / 'dev-clean'
|
| 31 |
+
chapters = {}
|
| 32 |
+
for r in sorted(tbl, key=lambda r: r['id']):
|
| 33 |
+
f = root / str(r['speaker_id']) / f"{r['id']}.flac"
|
| 34 |
+
if not f.exists():
|
| 35 |
+
f.parent.mkdir(parents=True, exist_ok=True); f.write_bytes(r['audio']['bytes'])
|
| 36 |
+
by = chapters.setdefault(str(r['speaker_id']), {})
|
| 37 |
+
by.setdefault(r['chapter_id'], []).append({'id': r['id'], 'path': str(f), 'text': r['text']})
|
| 38 |
+
chapters = {s: [c[k] for k in sorted(c)] for s, c in chapters.items()}
|
| 39 |
+
assert len(tbl) == 2703 and len(chapters) == 40, (len(tbl), len(chapters))
|
| 40 |
+
|
| 41 |
+
rng = random.Random(42)
|
| 42 |
+
speakers = sorted(chapters)
|
| 43 |
+
rng.shuffle(speakers)
|
| 44 |
+
held_out, calib_spk = sorted(speakers[:8]), sorted(speakers[8:])
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def pcm(path):
|
| 48 |
+
raw = subprocess.run(['ffmpeg', '-v', 'error', '-i', path, '-f', 'f32le',
|
| 49 |
+
'-ac', '1', '-ar', str(SR), '-'],
|
| 50 |
+
check=True, capture_output=True).stdout
|
| 51 |
+
return np.frombuffer(raw, dtype=np.float32)
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
def pick_run(spk, seconds):
|
| 55 |
+
"""Consecutive utterances of one random chapter, about `seconds` long."""
|
| 56 |
+
utts = rng.choice(chapters[spk])
|
| 57 |
+
i = rng.randrange(len(utts))
|
| 58 |
+
out, dur = [], 0.0
|
| 59 |
+
while i < len(utts) and dur < seconds:
|
| 60 |
+
out.append(utts[i]); dur += os.path.getsize(utts[i]['path']) / 32000 * 2.2
|
| 61 |
+
i += 1
|
| 62 |
+
return out
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
def build(kind, idx, spk_pool, seconds, two_speakers):
|
| 66 |
+
if two_speakers:
|
| 67 |
+
a, b = rng.sample(spk_pool, 2)
|
| 68 |
+
ra, rb = pick_run(a, seconds / 2), pick_run(b, seconds / 2)
|
| 69 |
+
parts, turn = [], 0
|
| 70 |
+
# take 1-3 utterances per turn
|
| 71 |
+
while ra or rb:
|
| 72 |
+
src = ra if (turn % 2 == 0 and ra) or not rb else rb
|
| 73 |
+
for _ in range(rng.randint(1, 3)):
|
| 74 |
+
if src:
|
| 75 |
+
parts.append((a if src is ra else b, src.pop(0)))
|
| 76 |
+
turn += 1
|
| 77 |
+
else:
|
| 78 |
+
s = rng.choice(spk_pool)
|
| 79 |
+
parts = [(s, u) for u in pick_run(s, seconds)]
|
| 80 |
+
gap = np.zeros(int(0.25 * SR), dtype=np.float32)
|
| 81 |
+
audio, segs, t = [], [], 0.0
|
| 82 |
+
for spk, u in parts:
|
| 83 |
+
x = pcm(u['path'])
|
| 84 |
+
segs.append({'speaker': spk, 'id': u['id'], 'start': t, 'end': t + len(x) / SR, 'text': u['text']})
|
| 85 |
+
audio += [x, gap]; t += (len(x) + len(gap)) / SR
|
| 86 |
+
x = np.concatenate(audio)
|
| 87 |
+
x16 = np.clip(np.round(x * 32767.0), -32768, 32767).astype('<i2')
|
| 88 |
+
wav = DATA / kind / f'{kind}-{idx:03d}.wav'
|
| 89 |
+
wav.parent.mkdir(exist_ok=True)
|
| 90 |
+
with wave.open(str(wav), 'wb') as w:
|
| 91 |
+
w.setnchannels(1); w.setsampwidth(2); w.setframerate(SR); w.writeframes(x16.tobytes())
|
| 92 |
+
return {'wav': str(wav), 'duration': len(x) / SR, 'two_speakers': two_speakers,
|
| 93 |
+
'speakers': sorted({s['speaker'] for s in segs}), 'segments': segs,
|
| 94 |
+
'text': ' '.join(s['text'] for s in segs)}
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
calib = [build('calib', i, calib_spk, rng.uniform(15, 150), rng.random() < 0.35)
|
| 98 |
+
for i in range(N_CALIB_AUDIO)]
|
| 99 |
+
evals = [build('eval', i, held_out, 60.0, i % 3 == 2) for i in range(N_EVAL)]
|
| 100 |
+
|
| 101 |
+
manifest = {'source': 'LibriSpeech dev-clean (openslr/librispeech_asr, all/validation.clean)', 'seed': 42,
|
| 102 |
+
'held_out_speakers': held_out, 'calibration_speakers': calib_spk,
|
| 103 |
+
'calibration': calib, 'evaluation': evals}
|
| 104 |
+
(DATA / 'audio_manifest.json').write_text(json.dumps(manifest, indent=1))
|
| 105 |
+
|
| 106 |
+
from datasets import load_dataset
|
| 107 |
+
u = load_dataset('HuggingFaceH4/ultrachat_200k', split='train_sft').shuffle(seed=42).select(range(N_TEXT))
|
| 108 |
+
(DATA / 'text_calibration.json').write_text(json.dumps([r['messages'] for r in u]))
|
| 109 |
+
print('DATA_DONE', len(calib), sum(c['duration'] for c in calib) / 60, 'min calib,',
|
| 110 |
+
len(evals), sum(e['duration'] for e in evals) / 60, 'min eval,',
|
| 111 |
+
sum(len(e['text'].split()) for e in evals), 'eval words', flush=True)
|
reproduce/prepare_model.py
ADDED
|
@@ -0,0 +1,122 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Split the checkpoint, and turn the streams into LM calibration inputs.
|
| 2 |
+
|
| 3 |
+
Runs in the upstream environment (torch 2.6, transformers 4.51, Microsoft
|
| 4 |
+
VibeVoice checkout). Writes:
|
| 5 |
+
decoder-base/ the Qwen2 LM alone, BF16, for llm-compressor
|
| 6 |
+
speech.safetensors encoders + connectors, BF16 (acoustic decoder pruned)
|
| 7 |
+
calibration/*.pt inputs_embeds of full streaming sessions (<= 2048 rows)
|
| 8 |
+
evaluation/*.pt per-window speech features of the held-out streams
|
| 9 |
+
baseline.json BF16 greedy transcripts of the held-out streams
|
| 10 |
+
"""
|
| 11 |
+
import os, sys, json, time
|
| 12 |
+
from pathlib import Path
|
| 13 |
+
import numpy as np, torch
|
| 14 |
+
import torch.nn.functional as F
|
| 15 |
+
from safetensors.torch import save_file
|
| 16 |
+
from transformers import Qwen2Config, Qwen2ForCausalLM
|
| 17 |
+
from accelerate import init_empty_weights
|
| 18 |
+
|
| 19 |
+
ROOT = Path(os.environ['VV_QUANT_WORKDIR']); DATA = Path(os.environ['VV_QUANT_DATA'])
|
| 20 |
+
sys.path.insert(0, os.environ['VIBEVOICE_SOURCE']); sys.path.insert(0, str(Path(__file__).parent))
|
| 21 |
+
from transformers.configuration_utils import PretrainedConfig
|
| 22 |
+
PretrainedConfig.__repr__ = lambda s: '<' + s.__class__.__name__ + '>'
|
| 23 |
+
from transformers.models.auto import AutoModel, AutoModelForCausalLM
|
| 24 |
+
for cls in (AutoModel, AutoModelForCausalLM):
|
| 25 |
+
old = cls.register
|
| 26 |
+
def wrap(cfg, mdl, exist_ok=True, _old=old, **kw):
|
| 27 |
+
try: return _old(cfg, mdl, exist_ok=True)
|
| 28 |
+
except ValueError: return None
|
| 29 |
+
cls.register = wrap
|
| 30 |
+
from vibevoice.modular.modeling_vibevoice_asr import VibeVoiceASRForConditionalGeneration
|
| 31 |
+
from vibevoice.processor.vibevoice_asr_processor import VibeVoiceASRProcessor
|
| 32 |
+
from vibevoice.processor.audio_utils import load_audio_use_ffmpeg
|
| 33 |
+
import stream_ref
|
| 34 |
+
|
| 35 |
+
MAXLEN = 2048
|
| 36 |
+
torch.set_grad_enabled(False); torch.set_num_threads(8); torch.manual_seed(42)
|
| 37 |
+
BASE = ROOT / 'base'
|
| 38 |
+
proc = VibeVoiceASRProcessor.from_pretrained(str(BASE)); tok = proc.tokenizer
|
| 39 |
+
ids = stream_ref.token_ids(tok)
|
| 40 |
+
assert ids == {'speech_start': 151646, 'speech_end': 151647, 'text_chunk_end': 151665, 'eos': 151643}, ids
|
| 41 |
+
assert ids['speech_start'] == tok.speech_start_id and ids['text_chunk_end'] == tok.text_chunk_end_id
|
| 42 |
+
print('LOAD_BASE', flush=True)
|
| 43 |
+
m = VibeVoiceASRForConditionalGeneration.from_pretrained(str(BASE), torch_dtype=torch.bfloat16, attn_implementation='sdpa').eval()
|
| 44 |
+
assert m.get_input_embeddings().weight.dtype == torch.bfloat16
|
| 45 |
+
base_cfg = json.loads((BASE / 'config.json').read_text()); dcfg = base_cfg['decoder_config']
|
| 46 |
+
tied = bool(dcfg.get('tie_word_embeddings', False))
|
| 47 |
+
same = torch.equal(m.lm_head.weight, m.get_input_embeddings().weight)
|
| 48 |
+
print('TIE', tied, 'lm_head==embed', same, flush=True)
|
| 49 |
+
if tied: assert same
|
| 50 |
+
|
| 51 |
+
# Speech components used by ASR, physically BF16; the synthesis decoder is dropped.
|
| 52 |
+
speech = {}; removed = 0
|
| 53 |
+
for k, v in m.state_dict().items():
|
| 54 |
+
if k.startswith('model.language_model.') or k.startswith('lm_head.'): continue
|
| 55 |
+
if k.startswith('model.acoustic_tokenizer.decoder.'):
|
| 56 |
+
removed += v.numel() * v.element_size(); continue
|
| 57 |
+
speech[k] = v.detach().to('cpu', dtype=torch.bfloat16).contiguous()
|
| 58 |
+
save_file(speech, str(ROOT / 'speech.safetensors'), metadata={'format': 'pt'})
|
| 59 |
+
(ROOT / 'pruning.json').write_text(json.dumps({'removed_prefixes': ['model.acoustic_tokenizer.decoder.'], 'removed_bf16_bytes': removed,
|
| 60 |
+
'speech_bytes': sum(t.numel() * t.element_size() for t in speech.values())}, indent=2))
|
| 61 |
+
del speech
|
| 62 |
+
del m.model.acoustic_tokenizer.decoder
|
| 63 |
+
|
| 64 |
+
qcfg = Qwen2Config(**dcfg); qcfg.torch_dtype = torch.bfloat16
|
| 65 |
+
with init_empty_weights(): q = Qwen2ForCausalLM(qcfg)
|
| 66 |
+
q.model = m.model.language_model; q.lm_head = m.lm_head
|
| 67 |
+
if tied: q.tie_weights()
|
| 68 |
+
q.save_pretrained(str(ROOT / 'decoder-base'), max_shard_size='4GB'); tok.save_pretrained(str(ROOT / 'decoder-base'))
|
| 69 |
+
proc.save_pretrained(str(ROOT / 'processor'))
|
| 70 |
+
print('DECODER_SAVED', flush=True)
|
| 71 |
+
|
| 72 |
+
dev = 'cuda:0'
|
| 73 |
+
m.to(dev)
|
| 74 |
+
mm = m.model
|
| 75 |
+
for c in (mm.acoustic_tokenizer, mm.semantic_tokenizer, mm.acoustic_connector, mm.semantic_connector): c.to(torch.float32)
|
| 76 |
+
mm.acoustic_tokenizer.std_dist_type = 'none' # deterministic latent: the mean
|
| 77 |
+
chunk, look = stream_ref.geometry(str(BASE)); target = chunk + look
|
| 78 |
+
print('GEOMETRY', chunk, look, flush=True)
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
def feats_of(wav):
|
| 82 |
+
audio, _ = load_audio_use_ffmpeg(wav, resample=True, target_sr=24000)
|
| 83 |
+
at = torch.from_numpy(np.ascontiguousarray(audio, dtype=np.float32)).unsqueeze(0).to(dev)
|
| 84 |
+
out = []
|
| 85 |
+
for a, b in stream_ref.windows(at.shape[1], chunk, look):
|
| 86 |
+
seg = at[:, a:b]
|
| 87 |
+
if seg.shape[1] < target: seg = F.pad(seg, (0, target - seg.shape[1]))
|
| 88 |
+
x = seg.unsqueeze(1)
|
| 89 |
+
f = mm.acoustic_connector(mm.acoustic_tokenizer.encode(x).mean) + mm.semantic_connector(mm.semantic_tokenizer.encode(x).mean)
|
| 90 |
+
out.append(f[0].to(torch.bfloat16))
|
| 91 |
+
return out
|
| 92 |
+
|
| 93 |
+
|
| 94 |
+
man = json.loads((DATA / 'audio_manifest.json').read_text())
|
| 95 |
+
(ROOT / 'calibration').mkdir(exist_ok=True); (ROOT / 'evaluation').mkdir(exist_ok=True)
|
| 96 |
+
lens = []
|
| 97 |
+
for i, s in enumerate(man['calibration']):
|
| 98 |
+
t = time.time()
|
| 99 |
+
fl = feats_of(s['wav'])
|
| 100 |
+
texts, emb = stream_ref.run(q, tok, fl, max_new_tokens=64, max_len=MAXLEN)
|
| 101 |
+
emb = emb[:, :MAXLEN]; lens.append(emb.shape[1])
|
| 102 |
+
torch.save({'inputs_embeds': emb.cpu(), 'attention_mask': torch.ones(1, emb.shape[1], dtype=torch.long)}, ROOT / 'calibration' / f'audio-{i:03d}.pt')
|
| 103 |
+
print('CALIB', i + 1, len(man['calibration']), emb.shape[1], f'{time.time() - t:.1f}s', repr(''.join(texts)[:120]), flush=True)
|
| 104 |
+
baseline = []
|
| 105 |
+
for i, s in enumerate(man['evaluation']):
|
| 106 |
+
fl = feats_of(s['wav'])
|
| 107 |
+
torch.save(torch.stack(fl).cpu(), ROOT / 'evaluation' / f'eval-{i:03d}.pt')
|
| 108 |
+
texts, _ = stream_ref.run(q, tok, fl)
|
| 109 |
+
baseline.append({'wav': s['wav'], 'reference': s['text'], 'chunks': texts, 'text': ''.join(texts)})
|
| 110 |
+
print('BASELINE', i, repr(''.join(texts)[:200]), flush=True)
|
| 111 |
+
(ROOT / 'baseline.json').write_text(json.dumps(baseline, indent=1))
|
| 112 |
+
for i, messages in enumerate(json.loads((DATA / 'text_calibration.json').read_text())):
|
| 113 |
+
if tok.chat_template:
|
| 114 |
+
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=False)
|
| 115 |
+
else:
|
| 116 |
+
text = ''.join(f"<|im_start|>{x['role']}\n{x['content']}<|im_end|>\n" for x in messages)
|
| 117 |
+
tids = tok(text, return_tensors='pt', truncation=True, max_length=MAXLEN, add_special_tokens=False).input_ids.to(dev)
|
| 118 |
+
emb = q.get_input_embeddings()(tids)
|
| 119 |
+
torch.save({'inputs_embeds': emb.cpu(), 'attention_mask': torch.ones(1, emb.shape[1], dtype=torch.long)}, ROOT / 'calibration' / f'text-{i:03d}.pt')
|
| 120 |
+
(ROOT / 'calibration_stats.json').write_text(json.dumps({'audio_sessions': len(lens), 'audio_rows_mean': float(np.mean(lens)),
|
| 121 |
+
'audio_rows_total': int(np.sum(lens)), 'truncated_at_2048': int(sum(l >= MAXLEN for l in lens))}, indent=2))
|
| 122 |
+
print('PREPARE_DONE', flush=True)
|
reproduce/quantize.py
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import os,json,traceback
|
| 2 |
+
from pathlib import Path
|
| 3 |
+
import torch
|
| 4 |
+
from torch.utils.data import DataLoader
|
| 5 |
+
from transformers import Qwen2ForCausalLM,AutoTokenizer
|
| 6 |
+
from llmcompressor import oneshot
|
| 7 |
+
from llmcompressor.modifiers.transform.awq import AWQModifier
|
| 8 |
+
from llmcompressor.modifiers.quantization import QuantizationModifier
|
| 9 |
+
ROOT=Path(os.environ['VV_QUANT_WORKDIR'])
|
| 10 |
+
torch.set_num_threads(8);torch.manual_seed(42)
|
| 11 |
+
print('LOAD_DECODER',flush=True)
|
| 12 |
+
# Loaded on the CPU: llm-compressor's sequential pipeline onloads one decoder layer at a time to the single GPU.
|
| 13 |
+
m=Qwen2ForCausalLM.from_pretrained(str(ROOT/'decoder-base'),dtype=torch.bfloat16,device_map='cpu',attn_implementation='sdpa')
|
| 14 |
+
tok=AutoTokenizer.from_pretrained(str(ROOT/'decoder-base'))
|
| 15 |
+
class Calibration(torch.utils.data.Dataset):
|
| 16 |
+
def __init__(self):self.paths=sorted((ROOT/'calibration').glob('*.pt'));assert len(self.paths)==256
|
| 17 |
+
def __len__(self):return len(self.paths)
|
| 18 |
+
def __getitem__(self,i):return torch.load(self.paths[i],map_location='cpu',weights_only=True)
|
| 19 |
+
d=DataLoader(Calibration(),batch_size=None)
|
| 20 |
+
recipe=[AWQModifier(duo_scaling='both',n_grid=40,offload_device=torch.device('cpu')),QuantizationModifier(targets=['Linear'],scheme='W4A16_ASYM',ignore=['lm_head'])]
|
| 21 |
+
oneshot(model=m,processor=tok,dataset=d,recipe=recipe,max_seq_length=2048,num_calibration_samples=256)
|
| 22 |
+
OUT=str(ROOT/'decoder-awq')
|
| 23 |
+
|
| 24 |
+
def fallback_save():
|
| 25 |
+
# transformers save_pretrained can fail on the offloaded weight-conversion path;
|
| 26 |
+
# never throw away a finished calibration, write the compressed tensors directly.
|
| 27 |
+
from accelerate.utils import get_state_dict_offloaded_model
|
| 28 |
+
from safetensors.torch import save_file
|
| 29 |
+
sd=get_state_dict_offloaded_model(m)
|
| 30 |
+
os.makedirs(OUT,exist_ok=True)
|
| 31 |
+
for p in Path(OUT).glob('*.safetensors'):p.unlink()
|
| 32 |
+
names=sorted(sd);shards=[];buf={};size=0;wm={};total=0
|
| 33 |
+
def flush():
|
| 34 |
+
nonlocal buf,size
|
| 35 |
+
if not buf:return
|
| 36 |
+
name='model-%05d.safetensors'%(len(shards)+1)
|
| 37 |
+
save_file(buf,os.path.join(OUT,name),metadata={'format':'pt'})
|
| 38 |
+
wm.update({k:name for k in buf});shards.append(name);print('FALLBACK_SHARD',name,size,flush=True);buf={};size=0
|
| 39 |
+
for k in names:
|
| 40 |
+
v=sd[k].cpu().contiguous();b=v.numel()*v.element_size()
|
| 41 |
+
if size+b>3_800_000_000:flush()
|
| 42 |
+
buf[k]=v;size+=b;total+=b
|
| 43 |
+
flush()
|
| 44 |
+
Path(OUT,'model.safetensors.index.json').write_text(json.dumps({'metadata':{'total_size':total},'weight_map':wm},indent=2))
|
| 45 |
+
m.config.save_pretrained(OUT)
|
| 46 |
+
print('FALLBACK_SAVE_OK',total,flush=True)
|
| 47 |
+
|
| 48 |
+
try:
|
| 49 |
+
m.save_pretrained(OUT,save_compressed=True,max_shard_size='4GB',save_original_format=False)
|
| 50 |
+
except Exception:
|
| 51 |
+
traceback.print_exc();print('SAVE_FAILED_USING_FALLBACK',flush=True);fallback_save()
|
| 52 |
+
tok.save_pretrained(OUT)
|
| 53 |
+
print('QUANT_DONE',flush=True)
|
reproduce/stream_ref.py
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""The streaming protocol of upstream streaming_generate, over a bare Qwen2 LM.
|
| 2 |
+
|
| 3 |
+
Same loop as vibevoice.c tools/compare_ref.py dump-stream (checked there
|
| 4 |
+
against upstream streaming_generate chunk for chunk): the plain-text prompt,
|
| 5 |
+
then per window [<|object_ref_start|>, 26 feature rows, <|object_ref_end|>],
|
| 6 |
+
greedy tokens until <|text_chunk_end|> or EOS, and <|text_chunk_end|> fed
|
| 7 |
+
after every chunk.
|
| 8 |
+
"""
|
| 9 |
+
import json, os
|
| 10 |
+
import torch
|
| 11 |
+
|
| 12 |
+
STREAM_STRIP = ['<|text_chunk_end|>', '<|object_ref_start|>', '<|object_ref_end|>',
|
| 13 |
+
'<|box_start|>', '<|speech_start|>', '<|speech_end|>', '<|speech_pad|>']
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
def geometry(model_dir):
|
| 17 |
+
cfg = json.load(open(os.path.join(model_dir, 'preprocessor_config.json')))
|
| 18 |
+
sr = cfg['target_sample_rate']; ratio = cfg['speech_tok_compress_ratio']
|
| 19 |
+
frame_s = ratio / sr
|
| 20 |
+
chunk = int(cfg['chunk_frames'] * frame_s * sr)
|
| 21 |
+
frame_dur = 3200 / sr
|
| 22 |
+
look = int(round(cfg['lookahead_frames'] * frame_s / frame_dur) * frame_dur * sr)
|
| 23 |
+
return chunk, look
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
def windows(total, chunk, look):
|
| 27 |
+
out, start = [], 0
|
| 28 |
+
while start < total:
|
| 29 |
+
end = min(start + chunk + look, total)
|
| 30 |
+
if end > start:
|
| 31 |
+
out.append((start, end))
|
| 32 |
+
start = min(start + chunk, total)
|
| 33 |
+
return out
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def prompt_text():
|
| 37 |
+
return ('You are a helpful assistant that transcribes audio input into text output. '
|
| 38 |
+
'Please transcribe the following audios streamingly with these keys: speaker, content\n')
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
def token_ids(tok):
|
| 42 |
+
return {'speech_start': tok.convert_tokens_to_ids('<|object_ref_start|>'),
|
| 43 |
+
'speech_end': tok.convert_tokens_to_ids('<|object_ref_end|>'),
|
| 44 |
+
'text_chunk_end': tok.convert_tokens_to_ids('<|text_chunk_end|>'),
|
| 45 |
+
'eos': tok.convert_tokens_to_ids('<|endoftext|>')}
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
@torch.no_grad()
|
| 49 |
+
def run(lm, tok, feats_list, max_new_tokens=256, max_len=None):
|
| 50 |
+
"""Greedy streaming decode. Returns (chunk texts, embeddings fed to the LM)."""
|
| 51 |
+
dev = lm.get_input_embeddings().weight.device
|
| 52 |
+
embed = lm.get_input_embeddings()
|
| 53 |
+
ids = token_ids(tok)
|
| 54 |
+
E = lambda t: embed(torch.tensor([t], device=dev))
|
| 55 |
+
prompt = tok.encode(prompt_text(), add_special_tokens=False)
|
| 56 |
+
fed = [E(prompt)]
|
| 57 |
+
out = lm(inputs_embeds=fed[0], use_cache=True, return_dict=True)
|
| 58 |
+
past, logits = out.past_key_values, out.logits
|
| 59 |
+
sp_s, sp_e, tce = E([ids['speech_start']]), E([ids['speech_end']]), E([ids['text_chunk_end']])
|
| 60 |
+
texts, n = [], fed[0].shape[1]
|
| 61 |
+
for feats in feats_list:
|
| 62 |
+
emb = torch.cat([sp_s, feats.to(dev, sp_s.dtype).unsqueeze(0), sp_e], dim=1)
|
| 63 |
+
fed.append(emb); n += emb.shape[1]
|
| 64 |
+
out = lm(inputs_embeds=emb, past_key_values=past, use_cache=True, return_dict=True)
|
| 65 |
+
past, logits = out.past_key_values, out.logits
|
| 66 |
+
gen = []
|
| 67 |
+
for _ in range(max_new_tokens):
|
| 68 |
+
t = int(torch.argmax(logits[:, -1, :], dim=-1).item())
|
| 69 |
+
if t in (ids['text_chunk_end'], ids['eos']):
|
| 70 |
+
break
|
| 71 |
+
gen.append(t)
|
| 72 |
+
e = E([t]); fed.append(e); n += 1
|
| 73 |
+
out = lm(inputs_embeds=e, past_key_values=past, use_cache=True, return_dict=True)
|
| 74 |
+
past, logits = out.past_key_values, out.logits
|
| 75 |
+
fed.append(tce); n += 1
|
| 76 |
+
out = lm(inputs_embeds=tce, past_key_values=past, use_cache=True, return_dict=True)
|
| 77 |
+
past, logits = out.past_key_values, out.logits
|
| 78 |
+
text = tok.decode(gen, skip_special_tokens=True)
|
| 79 |
+
for s in STREAM_STRIP:
|
| 80 |
+
text = text.replace(s, '')
|
| 81 |
+
texts.append(text)
|
| 82 |
+
if max_len is not None and n >= max_len:
|
| 83 |
+
break
|
| 84 |
+
return texts, torch.cat(fed, dim=1)
|
sha256.json
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model-00001.safetensors": "c00b652e53caefef0e5b3249fe14e1b8d43b9d365b4f909a7db3fbd2b9cad405",
|
| 3 |
+
"model-00002.safetensors": "5745ac1e81e127813ae6fe3d06883fd2f591ea0cdbb231e442c0e382e90a90d8"
|
| 4 |
+
}
|
special_tokens_map.json
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"additional_special_tokens": [
|
| 3 |
+
{
|
| 4 |
+
"content": "<|AUDIO|>",
|
| 5 |
+
"lstrip": false,
|
| 6 |
+
"normalized": false,
|
| 7 |
+
"rstrip": false,
|
| 8 |
+
"single_word": false
|
| 9 |
+
},
|
| 10 |
+
{
|
| 11 |
+
"content": "<|audio_bos|>",
|
| 12 |
+
"lstrip": false,
|
| 13 |
+
"normalized": false,
|
| 14 |
+
"rstrip": false,
|
| 15 |
+
"single_word": false
|
| 16 |
+
},
|
| 17 |
+
{
|
| 18 |
+
"content": "<|audio_eos|>",
|
| 19 |
+
"lstrip": false,
|
| 20 |
+
"normalized": false,
|
| 21 |
+
"rstrip": false,
|
| 22 |
+
"single_word": false
|
| 23 |
+
},
|
| 24 |
+
{
|
| 25 |
+
"content": "<|object_ref_start|>",
|
| 26 |
+
"lstrip": false,
|
| 27 |
+
"normalized": false,
|
| 28 |
+
"rstrip": false,
|
| 29 |
+
"single_word": false
|
| 30 |
+
},
|
| 31 |
+
{
|
| 32 |
+
"content": "<|object_ref_end|>",
|
| 33 |
+
"lstrip": false,
|
| 34 |
+
"normalized": false,
|
| 35 |
+
"rstrip": false,
|
| 36 |
+
"single_word": false
|
| 37 |
+
},
|
| 38 |
+
{
|
| 39 |
+
"content": "<|box_start|>",
|
| 40 |
+
"lstrip": false,
|
| 41 |
+
"normalized": false,
|
| 42 |
+
"rstrip": false,
|
| 43 |
+
"single_word": false
|
| 44 |
+
},
|
| 45 |
+
{
|
| 46 |
+
"content": "<|text_chunk_end|>",
|
| 47 |
+
"lstrip": false,
|
| 48 |
+
"normalized": false,
|
| 49 |
+
"rstrip": false,
|
| 50 |
+
"single_word": false
|
| 51 |
+
}
|
| 52 |
+
],
|
| 53 |
+
"eos_token": "<|endoftext|>",
|
| 54 |
+
"pad_token": "<|endoftext|>",
|
| 55 |
+
"unk_token": "<|endoftext|>"
|
| 56 |
+
}
|
tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,242 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_bos_token": false,
|
| 3 |
+
"add_prefix_space": false,
|
| 4 |
+
"added_tokens_decoder": {
|
| 5 |
+
"151643": {
|
| 6 |
+
"content": "<|endoftext|>",
|
| 7 |
+
"lstrip": false,
|
| 8 |
+
"normalized": false,
|
| 9 |
+
"rstrip": false,
|
| 10 |
+
"single_word": false,
|
| 11 |
+
"special": true
|
| 12 |
+
},
|
| 13 |
+
"151644": {
|
| 14 |
+
"content": "<|im_start|>",
|
| 15 |
+
"lstrip": false,
|
| 16 |
+
"normalized": false,
|
| 17 |
+
"rstrip": false,
|
| 18 |
+
"single_word": false,
|
| 19 |
+
"special": true
|
| 20 |
+
},
|
| 21 |
+
"151645": {
|
| 22 |
+
"content": "<|im_end|>",
|
| 23 |
+
"lstrip": false,
|
| 24 |
+
"normalized": false,
|
| 25 |
+
"rstrip": false,
|
| 26 |
+
"single_word": false,
|
| 27 |
+
"special": true
|
| 28 |
+
},
|
| 29 |
+
"151646": {
|
| 30 |
+
"content": "<|object_ref_start|>",
|
| 31 |
+
"lstrip": false,
|
| 32 |
+
"normalized": false,
|
| 33 |
+
"rstrip": false,
|
| 34 |
+
"single_word": false,
|
| 35 |
+
"special": true
|
| 36 |
+
},
|
| 37 |
+
"151647": {
|
| 38 |
+
"content": "<|object_ref_end|>",
|
| 39 |
+
"lstrip": false,
|
| 40 |
+
"normalized": false,
|
| 41 |
+
"rstrip": false,
|
| 42 |
+
"single_word": false,
|
| 43 |
+
"special": true
|
| 44 |
+
},
|
| 45 |
+
"151648": {
|
| 46 |
+
"content": "<|box_start|>",
|
| 47 |
+
"lstrip": false,
|
| 48 |
+
"normalized": false,
|
| 49 |
+
"rstrip": false,
|
| 50 |
+
"single_word": false,
|
| 51 |
+
"special": true
|
| 52 |
+
},
|
| 53 |
+
"151649": {
|
| 54 |
+
"content": "<|box_end|>",
|
| 55 |
+
"lstrip": false,
|
| 56 |
+
"normalized": false,
|
| 57 |
+
"rstrip": false,
|
| 58 |
+
"single_word": false,
|
| 59 |
+
"special": true
|
| 60 |
+
},
|
| 61 |
+
"151650": {
|
| 62 |
+
"content": "<|quad_start|>",
|
| 63 |
+
"lstrip": false,
|
| 64 |
+
"normalized": false,
|
| 65 |
+
"rstrip": false,
|
| 66 |
+
"single_word": false,
|
| 67 |
+
"special": true
|
| 68 |
+
},
|
| 69 |
+
"151651": {
|
| 70 |
+
"content": "<|quad_end|>",
|
| 71 |
+
"lstrip": false,
|
| 72 |
+
"normalized": false,
|
| 73 |
+
"rstrip": false,
|
| 74 |
+
"single_word": false,
|
| 75 |
+
"special": true
|
| 76 |
+
},
|
| 77 |
+
"151652": {
|
| 78 |
+
"content": "<|vision_start|>",
|
| 79 |
+
"lstrip": false,
|
| 80 |
+
"normalized": false,
|
| 81 |
+
"rstrip": false,
|
| 82 |
+
"single_word": false,
|
| 83 |
+
"special": true
|
| 84 |
+
},
|
| 85 |
+
"151653": {
|
| 86 |
+
"content": "<|vision_end|>",
|
| 87 |
+
"lstrip": false,
|
| 88 |
+
"normalized": false,
|
| 89 |
+
"rstrip": false,
|
| 90 |
+
"single_word": false,
|
| 91 |
+
"special": true
|
| 92 |
+
},
|
| 93 |
+
"151654": {
|
| 94 |
+
"content": "<|vision_pad|>",
|
| 95 |
+
"lstrip": false,
|
| 96 |
+
"normalized": false,
|
| 97 |
+
"rstrip": false,
|
| 98 |
+
"single_word": false,
|
| 99 |
+
"special": true
|
| 100 |
+
},
|
| 101 |
+
"151655": {
|
| 102 |
+
"content": "<|image_pad|>",
|
| 103 |
+
"lstrip": false,
|
| 104 |
+
"normalized": false,
|
| 105 |
+
"rstrip": false,
|
| 106 |
+
"single_word": false,
|
| 107 |
+
"special": true
|
| 108 |
+
},
|
| 109 |
+
"151656": {
|
| 110 |
+
"content": "<|video_pad|>",
|
| 111 |
+
"lstrip": false,
|
| 112 |
+
"normalized": false,
|
| 113 |
+
"rstrip": false,
|
| 114 |
+
"single_word": false,
|
| 115 |
+
"special": true
|
| 116 |
+
},
|
| 117 |
+
"151657": {
|
| 118 |
+
"content": "<tool_call>",
|
| 119 |
+
"lstrip": false,
|
| 120 |
+
"normalized": false,
|
| 121 |
+
"rstrip": false,
|
| 122 |
+
"single_word": false,
|
| 123 |
+
"special": false
|
| 124 |
+
},
|
| 125 |
+
"151658": {
|
| 126 |
+
"content": "</tool_call>",
|
| 127 |
+
"lstrip": false,
|
| 128 |
+
"normalized": false,
|
| 129 |
+
"rstrip": false,
|
| 130 |
+
"single_word": false,
|
| 131 |
+
"special": false
|
| 132 |
+
},
|
| 133 |
+
"151659": {
|
| 134 |
+
"content": "<|fim_prefix|>",
|
| 135 |
+
"lstrip": false,
|
| 136 |
+
"normalized": false,
|
| 137 |
+
"rstrip": false,
|
| 138 |
+
"single_word": false,
|
| 139 |
+
"special": false
|
| 140 |
+
},
|
| 141 |
+
"151660": {
|
| 142 |
+
"content": "<|fim_middle|>",
|
| 143 |
+
"lstrip": false,
|
| 144 |
+
"normalized": false,
|
| 145 |
+
"rstrip": false,
|
| 146 |
+
"single_word": false,
|
| 147 |
+
"special": false
|
| 148 |
+
},
|
| 149 |
+
"151661": {
|
| 150 |
+
"content": "<|fim_suffix|>",
|
| 151 |
+
"lstrip": false,
|
| 152 |
+
"normalized": false,
|
| 153 |
+
"rstrip": false,
|
| 154 |
+
"single_word": false,
|
| 155 |
+
"special": false
|
| 156 |
+
},
|
| 157 |
+
"151662": {
|
| 158 |
+
"content": "<|fim_pad|>",
|
| 159 |
+
"lstrip": false,
|
| 160 |
+
"normalized": false,
|
| 161 |
+
"rstrip": false,
|
| 162 |
+
"single_word": false,
|
| 163 |
+
"special": false
|
| 164 |
+
},
|
| 165 |
+
"151663": {
|
| 166 |
+
"content": "<|repo_name|>",
|
| 167 |
+
"lstrip": false,
|
| 168 |
+
"normalized": false,
|
| 169 |
+
"rstrip": false,
|
| 170 |
+
"single_word": false,
|
| 171 |
+
"special": false
|
| 172 |
+
},
|
| 173 |
+
"151664": {
|
| 174 |
+
"content": "<|file_sep|>",
|
| 175 |
+
"lstrip": false,
|
| 176 |
+
"normalized": false,
|
| 177 |
+
"rstrip": false,
|
| 178 |
+
"single_word": false,
|
| 179 |
+
"special": false
|
| 180 |
+
},
|
| 181 |
+
"151665": {
|
| 182 |
+
"content": "<|text_chunk_end|>",
|
| 183 |
+
"lstrip": false,
|
| 184 |
+
"normalized": false,
|
| 185 |
+
"rstrip": false,
|
| 186 |
+
"single_word": false,
|
| 187 |
+
"special": true
|
| 188 |
+
},
|
| 189 |
+
"151666": {
|
| 190 |
+
"content": "<|AUDIO|>",
|
| 191 |
+
"lstrip": false,
|
| 192 |
+
"normalized": false,
|
| 193 |
+
"rstrip": false,
|
| 194 |
+
"single_word": false,
|
| 195 |
+
"special": true
|
| 196 |
+
},
|
| 197 |
+
"151667": {
|
| 198 |
+
"content": "<|audio_bos|>",
|
| 199 |
+
"lstrip": false,
|
| 200 |
+
"normalized": false,
|
| 201 |
+
"rstrip": false,
|
| 202 |
+
"single_word": false,
|
| 203 |
+
"special": true
|
| 204 |
+
},
|
| 205 |
+
"151668": {
|
| 206 |
+
"content": "<|audio_eos|>",
|
| 207 |
+
"lstrip": false,
|
| 208 |
+
"normalized": false,
|
| 209 |
+
"rstrip": false,
|
| 210 |
+
"single_word": false,
|
| 211 |
+
"special": true
|
| 212 |
+
}
|
| 213 |
+
},
|
| 214 |
+
"additional_special_tokens": [
|
| 215 |
+
"<|im_start|>",
|
| 216 |
+
"<|im_end|>",
|
| 217 |
+
"<|object_ref_start|>",
|
| 218 |
+
"<|object_ref_end|>",
|
| 219 |
+
"<|box_start|>",
|
| 220 |
+
"<|box_end|>",
|
| 221 |
+
"<|quad_start|>",
|
| 222 |
+
"<|quad_end|>",
|
| 223 |
+
"<|vision_start|>",
|
| 224 |
+
"<|vision_end|>",
|
| 225 |
+
"<|vision_pad|>",
|
| 226 |
+
"<|image_pad|>",
|
| 227 |
+
"<|video_pad|>",
|
| 228 |
+
"<|AUDIO|>",
|
| 229 |
+
"<|audio_bos|>",
|
| 230 |
+
"<|audio_eos|>"
|
| 231 |
+
],
|
| 232 |
+
"bos_token": null,
|
| 233 |
+
"chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0]['role'] == 'system' %}\n {%- if messages[0]['content'] is string %}\n {{- messages[0]['content'] }}\n {%- else %}\n {%- for part in messages[0]['content'] %}\n {%- if part['type'] == 'text' %}\n {{- part['text'] }}\n {%- elif part['type'] == 'audio' or part['type'] == 'audio_url' %}\n {{- '<|AUDIO|>' }}\n {%- endif %}\n {%- endfor %}\n {%- endif %}\n {%- else %}\n {{- 'You are a helpful assistant.' }}\n {%- endif %}\n {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0]['role'] == 'system' %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0]['content'] is string %}\n {{- messages[0]['content'] }}\n {%- else %}\n {%- for part in messages[0]['content'] %}\n {%- if part['type'] == 'text' %}\n {{- part['text'] }}\n {%- elif part['type'] == 'audio' or part['type'] == 'audio_url' %}\n {{- '<|AUDIO|>' }}\n {%- endif %}\n {%- endfor %}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n {%- else %}\n {{- '<|im_start|>system\\nYou are a helpful assistant.<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n {{- '<|im_start|>' + message.role + '\\n' }}\n {%- if message['content'] is string %}\n {{- message['content'] }}\n {%- else %}\n {%- for part in message['content'] %}\n {%- if part['type'] == 'text' %}\n {{- part['text'] }}\n {%- elif part['type'] == 'audio' or part['type'] == 'audio_url' %}\n {{- '<|AUDIO|>' }}\n {%- endif %}\n {%- endfor %}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"assistant\" %}\n {{- '<|im_start|>' + message.role }}\n {%- if message.content %}\n {{- '\\n' + message.content }}\n {%- endif %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '\\n<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {{- tool_call.arguments | tojson }}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n{%- endif %}",
|
| 234 |
+
"clean_up_tokenization_spaces": false,
|
| 235 |
+
"eos_token": "<|endoftext|>",
|
| 236 |
+
"errors": "replace",
|
| 237 |
+
"model_max_length": 131072,
|
| 238 |
+
"pad_token": "<|endoftext|>",
|
| 239 |
+
"split_special_tokens": false,
|
| 240 |
+
"tokenizer_class": "Qwen2Tokenizer",
|
| 241 |
+
"unk_token": null
|
| 242 |
+
}
|
vocab.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|