Hugging Face's logo

pltobing
/
Qwen3-TTS-Streaming-ONNX

Text-to-Speech
ONNX
TTS
ONNX
qwen3-tts
voice-clone
streaming
qwen3
vq
rvq
ecapa-tdnn
multilingual
Model card Files
xet
 Community
1
pltobing commited on
Commit
1a1edb3
·
1 Parent(s): 4ddf9d1

docs: add cpu example from cuda graph compat. that includes codec decoder

Browse files
Files changed (3) hide show
  1. .gitattributes +1 -0
  2. audio_synth/output_1776732259.7956588.wav +3 -0
  3. logs/log_test-streaming-onnx-cudagraphcompat-m8a-xlarge-female-shadowheart-2.txt +463 -0
.gitattributes CHANGED
@@ -58,3 +58,4 @@ qwen3-tts_onnx/talker_local_model_step.onnx filter=lfs diff=lfs merge=lfs -text
58
  qwen3-tts_onnx/talker_model_prefill.onnx filter=lfs diff=lfs merge=lfs -text
59
  qwen3-tts_onnx/talker_model_step.onnx filter=lfs diff=lfs merge=lfs -text
60
  audio_synth/output_1776633704.8548758.wav filter=lfs diff=lfs merge=lfs -text
 
 
58
  qwen3-tts_onnx/talker_model_prefill.onnx filter=lfs diff=lfs merge=lfs -text
59
  qwen3-tts_onnx/talker_model_step.onnx filter=lfs diff=lfs merge=lfs -text
60
  audio_synth/output_1776633704.8548758.wav filter=lfs diff=lfs merge=lfs -text
61
+ audio_synth/output_1776732259.7956588.wav filter=lfs diff=lfs merge=lfs -text
audio_synth/output_1776732259.7956588.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73f66fc27f1fa8d9f69d6c5d4a8798d84d626911c2827c71388d1853103c2725
3
+ size 211244
logs/log_test-streaming-onnx-cudagraphcompat-m8a-xlarge-female-shadowheart-2.txt ADDED
@@ -0,0 +1,463 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2026-04-21 00:43:36.095206217 [W:onnxruntime:Default, device_discovery.cc:325 DiscoverDevicesForPlatform] GPU device discovery failed: device_discovery.cc:92 ReadFileContents Failed to open file: "/sys/class/drm/card0/device/vendor"
2
+ None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
3
+ None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
4
+ 2026-04-21 00:43:36.535 [INFO] __main__: Building inferencer...
5
+ 2026-04-21 00:43:36.536 [INFO] src.inference.qwen3_tts_inferencer_onnx: Loading ONNX sessions...
6
+ 2026-04-21 00:43:36.536 [INFO] src.inference.qwen3_tts_inferencer_onnx: talker prefill <- ./qwen3-tts_onnx/talker_model_prefill.onnx
7
+ 2026-04-21 00:43:38.629 [INFO] src.inference.qwen3_tts_inferencer_onnx: local prefill <- ./qwen3-tts_onnx/talker_local_model_prefill.onnx
8
+ 2026-04-21 00:43:40.203 [INFO] src.inference.qwen3_tts_inferencer_onnx: talker step <- ./qwen3-tts_onnx/talker_model_step.onnx
9
+ 2026-04-21 00:43:55.108 [INFO] src.inference.qwen3_tts_inferencer_onnx: local step <- ./qwen3-tts_onnx/talker_local_model_step.onnx
10
+ 2026-04-21 00:43:56.708 [INFO] src.inference.qwen3_tts_inferencer_onnx: local lm_head <- ./qwen3-tts_onnx/talker_local_lm_head.onnx
11
+ 2026-04-21 00:43:57.507 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec decoder <- ./qwen3-tts_onnx/codec_decoder_model.onnx
12
+ 2026-04-21 00:43:58.074 [INFO] src.inference.qwen3_tts_inferencer_onnx: speaker encoder <- ./qwen3-tts_onnx/speaker_encoder_model.onnx
13
+ 2026-04-21 00:43:58.192 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec embed <- ./qwen3-tts_onnx/talker_codec_embed_model.onnx
14
+ 2026-04-21 00:43:58.995 [INFO] src.inference.qwen3_tts_inferencer_onnx: text embed proj <- ./qwen3-tts_onnx/text_embed_proj_model.onnx
15
+ 2026-04-21 00:44:08.972 [INFO] src.inference.qwen3_tts_inferencer_onnx: [OK] All ONNX sessions loaded.
16
+ 2026-04-21 00:44:09.424 [INFO] src.inference.qwen3_tts_inferencer_onnx: Allocating device KV-cache buffers...
17
+ 2026-04-21 00:44:09.429 [INFO] src.inference.qwen3_tts_inferencer_onnx: IOBinding wired for all models (prefill + step).
18
+ 2026-04-21 00:44:09.429 [INFO] src.inference.qwen3_tts_inferencer_onnx: Allocating device KV-cache buffer of codec decoder...
19
+ 2026-04-21 00:44:09.430 [INFO] __main__: Running 50 warmup iteration(s)...
20
+ 2026-04-21 00:44:09.435 [INFO] src.inference.qwen3_tts_inferencer_onnx: Warming-up speaker encoder...
21
+ 2026-04-21 00:44:09.889 [INFO] src.inference.qwen3_tts_inferencer_onnx: 50 iter. run speaker encoder in 0.42 s. --> 8.42 ms. per run.
22
+ 2026-04-21 00:44:09.889 [INFO] src.inference.qwen3_tts_inferencer_onnx: Warming-up text embed proj....
23
+ 2026-04-21 00:44:09.910 [INFO] src.inference.qwen3_tts_inferencer_onnx: 50 iter. run text embed proj. in 0.02 s. --> 0.43 ms. per run.
24
+ 2026-04-21 00:44:09.910 [INFO] src.inference.qwen3_tts_inferencer_onnx: Warming-up talker prefill...
25
+ 2026-04-21 00:44:12.665 [INFO] src.inference.qwen3_tts_inferencer_onnx: 50 iter. run talker prefill in 2.75 s. --> 55.09 ms. per run.
26
+ 2026-04-21 00:44:12.665 [INFO] src.inference.qwen3_tts_inferencer_onnx: Warming-up talker step...
27
+ 2026-04-21 00:44:14.975 [INFO] src.inference.qwen3_tts_inferencer_onnx: 50 iter. run talker step in 2.31 s. --> 46.18 ms. per run.
28
+ 2026-04-21 00:44:14.975 [INFO] src.inference.qwen3_tts_inferencer_onnx: Warming-up local talker prefill...
29
+ 2026-04-21 00:44:15.314 [INFO] src.inference.qwen3_tts_inferencer_onnx: 50 iter. run local talker prefill in 0.34 s. --> 6.79 ms. per run.
30
+ 2026-04-21 00:44:15.314 [INFO] src.inference.qwen3_tts_inferencer_onnx: Warming-up local talker step...
31
+ 2026-04-21 00:44:15.645 [INFO] src.inference.qwen3_tts_inferencer_onnx: 50 iter. run local talker step in 0.33 s. --> 6.62 ms. per run.
32
+ 2026-04-21 00:44:15.645 [INFO] src.inference.qwen3_tts_inferencer_onnx: Warming-up local talker batched lm_head...
33
+ 2026-04-21 00:44:15.755 [INFO] src.inference.qwen3_tts_inferencer_onnx: 50 iter. run local batched lm_head in 0.11 s. --> 2.19 ms. per run.
34
+ 2026-04-21 00:44:15.755 [INFO] src.inference.qwen3_tts_inferencer_onnx: Warming-up codec decoder...
35
+ 2026-04-21 00:44:19.768 [INFO] src.inference.qwen3_tts_inferencer_onnx: 12 iter. run codec_decoder in 4.01 s. --> 334.30 ms. per run.
36
+ 2026-04-21 00:44:19.768 [INFO] src.inference.qwen3_tts_inferencer_onnx: Warmup latency summary:
37
+ 2026-04-21 00:44:19.768 [INFO] src.inference.qwen3_tts_inferencer_onnx: speaker_encoder
38
+ 2026-04-21 00:44:19.770 [INFO] src.inference.qwen3_tts_inferencer_onnx: mean=8.42 ms.
39
+ 2026-04-21 00:44:19.770 [INFO] src.inference.qwen3_tts_inferencer_onnx: std=0.75 ms.
40
+ 2026-04-21 00:44:19.787 [INFO] src.inference.qwen3_tts_inferencer_onnx: median=8.28 ms.
41
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: p95=8.49 ms.
42
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: min=8.23 ms.
43
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: max=13.36 ms.
44
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: text_embed_proj
45
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: mean=0.43 ms.
46
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: std=0.72 ms.
47
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: median=0.15 ms.
48
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: p95=2.29 ms.
49
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: min=0.14 ms.
50
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: max=3.18 ms.
51
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: talker_prefill
52
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: mean=55.09 ms.
53
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: std=2.52 ms.
54
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: median=54.53 ms.
55
+ 2026-04-21 00:44:19.788 [INFO] src.inference.qwen3_tts_inferencer_onnx: p95=57.35 ms.
56
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: min=53.56 ms.
57
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: max=71.62 ms.
58
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: talker_step
59
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: mean=46.18 ms.
60
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: std=2.03 ms.
61
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: median=45.60 ms.
62
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: p95=48.79 ms.
63
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: min=45.34 ms.
64
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: max=59.06 ms.
65
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: local_prefill
66
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: mean=6.79 ms.
67
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: std=1.93 ms.
68
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: median=6.37 ms.
69
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: p95=7.20 ms.
70
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: min=6.31 ms.
71
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: max=17.38 ms.
72
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: local_step
73
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: mean=6.62 ms.
74
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: std=1.44 ms.
75
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: median=6.26 ms.
76
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: p95=8.83 ms.
77
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: min=6.21 ms.
78
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: max=14.39 ms.
79
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: local_lm_head
80
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: mean=2.19 ms.
81
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: std=0.54 ms.
82
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: median=1.94 ms.
83
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: p95=3.33 ms.
84
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: min=1.91 ms.
85
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: max=4.12 ms.
86
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec_decoder
87
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: mean=334.30 ms.
88
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: std=6.35 ms.
89
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: median=331.87 ms.
90
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: p95=344.38 ms.
91
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: min=330.48 ms.
92
+ 2026-04-21 00:44:19.789 [INFO] src.inference.qwen3_tts_inferencer_onnx: max=354.67 ms.
93
+ 2026-04-21 00:44:19.795 [INFO] __main__: Warmup complete in 10.37 s
94
+ 2026-04-21 00:44:19.795 [INFO] __main__: speaker_encoder mean=8.42 ms min=8.23 ms max=13.36 ms
95
+ 2026-04-21 00:44:19.795 [INFO] __main__: text_embed_proj mean=0.43 ms min=0.14 ms max=3.18 ms
96
+ 2026-04-21 00:44:19.795 [INFO] __main__: talker_prefill mean=55.09 ms min=53.56 ms max=71.62 ms
97
+ 2026-04-21 00:44:19.795 [INFO] __main__: talker_step mean=46.18 ms min=45.34 ms max=59.06 ms
98
+ 2026-04-21 00:44:19.795 [INFO] __main__: local_prefill mean=6.79 ms min=6.31 ms max=17.38 ms
99
+ 2026-04-21 00:44:19.795 [INFO] __main__: local_step mean=6.62 ms min=6.21 ms max=14.39 ms
100
+ 2026-04-21 00:44:19.795 [INFO] __main__: local_lm_head mean=2.19 ms min=1.91 ms max=4.12 ms
101
+ 2026-04-21 00:44:19.795 [INFO] __main__: codec_decoder mean=334.30 ms min=330.48 ms max=354.67 ms
102
+ 2026-04-21 00:44:19.795 [INFO] __main__: Synthesising: 'в зависимости от времени не только точность, но и низкая задержка.'
103
+ 2026-04-21 00:44:19.800 [INFO] src.inference.qwen3_tts_inferencer_onnx: _prefill_embeds language_id 2069
104
+ 2026-04-21 00:44:23.010 [INFO] src.inference.qwen3_tts_inferencer_onnx: _prefill_embeds speaker_embed [[[ 0.03693735 0.04726027 0.05757569 ... 0.07723516 -0.00755774
105
+ 0.04963338]]] (1, 1, 1024)
106
+ 2026-04-21 00:44:23.010 [INFO] src.inference.qwen3_tts_inferencer_onnx: generate codec_prefill_list [[2154 2156 2069 2157]]
107
+ 2026-04-21 00:44:23.010 [INFO] src.inference.qwen3_tts_inferencer_onnx: generate codec_input_embedding_0 [[[-0.00662231 0.00408936 0.00294495 ... 0.0111084 -0.00221252
108
+ -0.00595093]
109
+ [-0.00418091 0.00317383 0.0004921 ... 0.00231934 -0.00219727
110
+ -0.00291443]
111
+ [ 0.00360107 -0.0201416 0.01098633 ... -0.03833008 0.05273438
112
+ 0.00376892]
113
+ [-0.00294495 0.00515747 0.00060272 ... 0.01647949 -0.00176239
114
+ -0.00171661]]] (1, 4, 1024)
115
+ 2026-04-21 00:44:23.010 [INFO] src.inference.qwen3_tts_inferencer_onnx: generate codec_input_embedding_1 [[[-0.00190735 -0.00424194 -0.00109863 ... 0.00312805 0.00408936
116
+ 0.0004921 ]]] (1, 1, 1024)
117
+ 2026-04-21 00:44:23.011 [INFO] src.inference.qwen3_tts_inferencer_onnx: generate codec_input_embedding [[[-0.00662231 0.00408936 0.00294495 ... 0.0111084 -0.00221252
118
+ -0.00595093]
119
+ [-0.00418091 0.00317383 0.0004921 ... 0.00231934 -0.00219727
120
+ -0.00291443]
121
+ [ 0.00360107 -0.0201416 0.01098633 ... -0.03833008 0.05273438
122
+ 0.00376892]
123
+ [-0.00294495 0.00515747 0.00060272 ... 0.01647949 -0.00176239
124
+ -0.00171661]
125
+ [ 0.03693735 0.04726027 0.05757569 ... 0.07723516 -0.00755774
126
+ 0.04963338]
127
+ [-0.00190735 -0.00424194 -0.00109863 ... 0.00312805 0.00408936
128
+ 0.0004921 ]]] (1, 6, 1024)
129
+ 2026-04-21 00:44:23.011 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts text ['<|im_start|>assistant\n'] 22
130
+ Qwen3TTSProcessor __call__ text ['<|im_start|>assistant\n']
131
+ Qwen3TTSProcessor __call__ output_kwargs[text_kwargs] {'padding': True, 'padding_side': 'left', 'return_tensors': 'np'}
132
+ Qwen3TTSProcessor __call__ texts_inputs {'input_ids': array([[151644, 77091, 198]]), 'attention_mask': array([[1, 1, 1]])}
133
+ 2026-04-21 00:44:23.024 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts input_ids_dict {'input_ids': array([[151644, 77091, 198]]), 'attention_mask': array([[1, 1, 1]])}
134
+ 2026-04-21 00:44:23.024 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts input_ids [[151644 77091 198]] (1, 3)
135
+ 2026-04-21 00:44:23.024 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts input_ids_ [[151644 77091 198]] (1, 3)
136
+ 2026-04-21 00:44:23.026 [INFO] src.inference.qwen3_tts_inferencer_onnx: generate _talker_input_embed_role [[[-3.1744563e-03 3.4415501e-04 -6.5509509e-04 ... 6.8450002e-03
137
+ 1.6842657e-03 -3.0629896e-04]
138
+ [-5.0306913e-02 4.0606493e-03 1.8291049e-02 ... -5.6890018e-02
139
+ 1.0518981e-02 -2.8836861e-02]
140
+ [ 1.9560081e-03 -6.3556945e-05 1.1575520e-03 ... 2.7661566e-03
141
+ 4.2754621e-04 -2.2226851e-03]]] (1, 3, 1024)
142
+ 2026-04-21 00:44:23.027 [INFO] src.inference.qwen3_tts_inferencer_onnx: generate tts_bos_embed [[[ 0.00610077 -0.00173846 0.00627451 ... -0.00013921 -0.00062966
143
+ -0.00767278]]] (1, 1, 1024)
144
+ 2026-04-21 00:44:23.027 [INFO] src.inference.qwen3_tts_inferencer_onnx: generate tts_pad_embed [[[ 0.00477725 -0.00380563 -0.0016957 ... -0.00928466 0.00155563
145
+ 0.00424138]]] (1, 1, 1024)
146
+ 2026-04-21 00:44:23.027 [INFO] src.inference.qwen3_tts_inferencer_onnx: generate _talker_input_embed [[[-0.00184507 0.00028373 0.00124924 ... 0.00182374 -0.00065689
147
+ -0.00170955]
148
+ [ 0.00059634 -0.0006318 -0.00120361 ... -0.00696532 -0.00064164
149
+ 0.00132695]
150
+ [ 0.00837832 -0.02394723 0.00929063 ... -0.04761474 0.05429
151
+ 0.0080103 ]
152
+ [ 0.0018323 0.00135184 -0.00109298 ... 0.00719483 -0.00020676
153
+ 0.00252477]
154
+ [ 0.0417146 0.04345464 0.05587999 ... 0.0679505 -0.00600211
155
+ 0.05387476]
156
+ [ 0.00419342 -0.0059804 0.00517588 ... 0.00298884 0.0034597
157
+ -0.00718068]]] (1, 6, 1024)
158
+ 2026-04-21 00:44:23.027 [INFO] src.inference.qwen3_tts_inferencer_onnx: generate talker_input_embed [[[-3.1744563e-03 3.4415501e-04 -6.5509509e-04 ... 6.8450002e-03
159
+ 1.6842657e-03 -3.0629896e-04]
160
+ [-5.0306913e-02 4.0606493e-03 1.8291049e-02 ... -5.6890018e-02
161
+ 1.0518981e-02 -2.8836861e-02]
162
+ [ 1.9560081e-03 -6.3556945e-05 1.1575520e-03 ... 2.7661566e-03
163
+ 4.2754621e-04 -2.2226851e-03]
164
+ ...
165
+ [ 1.8323013e-03 1.3518450e-03 -1.0929797e-03 ... 7.1948320e-03
166
+ -2.0676060e-04 2.5247680e-03]
167
+ [ 4.1714597e-02 4.3454643e-02 5.5879988e-02 ... 6.7950502e-02
168
+ -6.0021072e-03 5.3874765e-02]
169
+ [ 4.1934196e-03 -5.9804004e-03 5.1758769e-03 ... 2.9888446e-03
170
+ 3.4596957e-03 -7.1806796e-03]]] (1, 9, 1024)
171
+ 2026-04-21 00:44:23.107 [INFO] src.inference.qwen3_tts_inferencer_onnx: Prefill done; talker_seq_len=9
172
+ 2026-04-21 00:44:23.107 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts text ['в зависимости от времени не '] 28
173
+ Qwen3TTSProcessor __call__ text ['в зависимости от времени не ']
174
+ Qwen3TTSProcessor __call__ output_kwargs[text_kwargs] {'padding': True, 'padding_side': 'left', 'return_tensors': 'np'}
175
+ Qwen3TTSProcessor __call__ texts_inputs {'input_ids': array([[ 5474, 139133, 20264, 130775, 18658, 220]]), 'attention_mask': array([[1, 1, 1, 1, 1, 1]])}
176
+ 2026-04-21 00:44:23.108 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts input_ids_dict {'input_ids': array([[ 5474, 139133, 20264, 130775, 18658, 220]]), 'attention_mask': array([[1, 1, 1, 1, 1, 1]])}
177
+ 2026-04-21 00:44:23.108 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts input_ids [[ 5474 139133 20264 130775 18658 220]] (1, 6)
178
+ 2026-04-21 00:44:23.108 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts input_ids_ [[ 5474 139133 20264 130775 18658 220]] (1, 6)
179
+ 2026-04-21 00:44:25.476 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-0 chunk_tokens [[[1995 215 212 1181]
180
+ [1159 1722 1134 1746]
181
+ [ 355 1096 749 412]
182
+ [ 22 1458 241 1050]
183
+ [1174 1047 76 1210]
184
+ [1093 1372 819 1506]
185
+ [ 625 1452 2019 1359]
186
+ [1814 370 668 1202]
187
+ [1058 249 564 632]
188
+ [ 905 1144 1446 38]
189
+ [1846 1381 288 1358]
190
+ [ 248 823 1603 1047]
191
+ [1677 1110 1108 896]
192
+ [ 889 1529 1759 781]
193
+ [ 812 1772 82 85]
194
+ [ 901 1978 803 1435]]] (1, 16, 4)
195
+ 2026-04-21 00:44:25.838 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-4 wav [[[-2.0139733e-02 -1.5920021e-02 -1.3928653e-02 ... 1.1063987e-05
196
+ 1.0716458e-05 1.1091723e-05]]] -0.02013973332941532 -5.644128759740852e-05 0.001009712228551507 0.006650683004409075 (1, 1, 7680)
197
+ 2026-04-21 00:44:25.839 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts text ['только точность,'] 16
198
+ Qwen3TTSProcessor __call__ text ['только точность,']
199
+ Qwen3TTSProcessor __call__ output_kwargs[text_kwargs] {'padding': True, 'padding_side': 'left', 'return_tensors': 'np'}
200
+ Qwen3TTSProcessor __call__ texts_inputs {'input_ids': array([[ 1792, 53470, 81602, 60279, 11]]), 'attention_mask': array([[1, 1, 1, 1, 1]])}
201
+ 2026-04-21 00:44:25.839 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts input_ids_dict {'input_ids': array([[ 1792, 53470, 81602, 60279, 11]]), 'attention_mask': array([[1, 1, 1, 1, 1]])}
202
+ 2026-04-21 00:44:25.839 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts input_ids [[ 1792 53470 81602 60279 11]] (1, 5)
203
+ 2026-04-21 00:44:25.839 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts input_ids_ [[ 1792 53470 81602 60279 11]] (1, 5)
204
+ 2026-04-21 00:44:27.748 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-4 chunk_tokens [[[ 462 1225 1294 1538]
205
+ [1666 1931 1609 742]
206
+ [1871 1414 414 1028]
207
+ [ 575 642 276 546]
208
+ [ 442 381 288 988]
209
+ [ 121 1287 575 1714]
210
+ [1579 1168 980 209]
211
+ [ 278 810 122 1253]
212
+ [1422 275 360 1475]
213
+ [ 86 1630 944 769]
214
+ [1702 783 153 1771]
215
+ [1136 1365 682 790]
216
+ [ 599 575 1174 64]
217
+ [ 626 721 1147 752]
218
+ [ 71 155 1314 784]
219
+ [ 508 1886 805 133]]] (1, 16, 4)
220
+ 2026-04-21 00:44:28.109 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-8 wav [[[ 8.9669975e-06 8.7110875e-06 1.1860529e-05 ... -1.4827815e-01
221
+ -1.6306561e-01 -1.7990637e-01]]] -0.26808077096939087 -9.64985247264849e-06 0.0613856241106987 0.32160332798957825 (1, 1, 7680)
222
+ 2026-04-21 00:44:28.109 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts text [' но и низкая задержка.'] 22
223
+ Qwen3TTSProcessor __call__ text [' но и низкая задержка.']
224
+ Qwen3TTSProcessor __call__ output_kwargs[text_kwargs] {'padding': True, 'padding_side': 'left', 'return_tensors': 'np'}
225
+ Qwen3TTSProcessor __call__ texts_inputs {'input_ids': array([[128330, 7587, 6709, 40773, 125107, 52180, 57569, 13132,
226
+ 13]]), 'attention_mask': array([[1, 1, 1, 1, 1, 1, 1, 1, 1]])}
227
+ 2026-04-21 00:44:28.109 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts input_ids_dict {'input_ids': array([[128330, 7587, 6709, 40773, 125107, 52180, 57569, 13132,
228
+ 13]]), 'attention_mask': array([[1, 1, 1, 1, 1, 1, 1, 1, 1]])}
229
+ 2026-04-21 00:44:28.109 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts input_ids [[128330 7587 6709 40773 125107 52180 57569 13132 13]] (1, 9)
230
+ 2026-04-21 00:44:28.109 [INFO] src.inference.qwen3_tts_inferencer_onnx: _tokenize_texts input_ids_ [[128330 7587 6709 40773 125107 52180 57569 13132 13]] (1, 9)
231
+ 2026-04-21 00:44:31.855 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-8 chunk_tokens [[[1476 1115 1634 741]
232
+ [ 136 1215 1271 604]
233
+ [ 734 1679 1780 1253]
234
+ [ 591 1105 532 1205]
235
+ [ 495 1386 40 1850]
236
+ [ 387 1871 102 631]
237
+ [ 105 812 155 903]
238
+ [1444 1855 833 88]
239
+ [ 961 1847 323 1594]
240
+ [ 168 39 585 1894]
241
+ [ 523 393 1670 871]
242
+ [ 992 94 1848 1505]
243
+ [1767 925 568 305]
244
+ [ 329 372 1724 1729]
245
+ [1279 403 158 715]
246
+ [ 143 326 401 996]]] (1, 16, 4)
247
+ 2026-04-21 00:44:32.218 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-12 wav [[[-0.17830494 -0.14795202 -0.10621441 ... 0.22759204 0.22218949
248
+ 0.21130118]]] -0.3140752911567688 0.00018200272461399436 0.10280003398656845 0.2426551878452301 (1, 1, 7680)
249
+ 2026-04-21 00:44:32.219 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-12 chunk_tokens [[[1804 177 1196 1249]
250
+ [1783 1371 1452 840]
251
+ [1527 1558 1489 1777]
252
+ [1697 804 79 1139]
253
+ [1332 1385 1118 646]
254
+ [1946 1693 1909 1238]
255
+ [ 191 1709 826 1259]
256
+ [ 54 740 735 574]
257
+ [ 212 1673 766 1667]
258
+ [ 923 742 1640 722]
259
+ [ 236 1147 660 720]
260
+ [ 923 783 861 1187]
261
+ [1855 838 597 768]
262
+ [1207 1861 707 203]
263
+ [ 878 1007 136 682]
264
+ [ 434 1357 1238 1997]]] (1, 16, 4)
265
+ 2026-04-21 00:44:32.550 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-16 wav [[[ 0.19922823 0.18836866 0.17623058 ... -0.00027019 -0.00048184
266
+ -0.00087112]]] -0.30748021602630615 -0.00011969714978476986 0.08889789134263992 0.3008376657962799 (1, 1, 7680)
267
+ 2026-04-21 00:44:32.550 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-16 chunk_tokens [[[1555 982 383 1406]
268
+ [1431 834 1811 1719]
269
+ [ 756 1177 151 1802]
270
+ [1628 639 937 1359]
271
+ [ 959 1516 1080 1145]
272
+ [1467 122 869 1439]
273
+ [ 717 443 934 295]
274
+ [1747 924 1724 1698]
275
+ [ 660 600 1459 1168]
276
+ [ 924 1505 469 1850]
277
+ [ 966 1236 447 888]
278
+ [ 107 319 85 1489]
279
+ [ 898 349 84 1100]
280
+ [ 670 863 162 1548]
281
+ [ 7 547 112 63]
282
+ [1480 891 1011 755]]] (1, 16, 4)
283
+ 2026-04-21 00:44:32.882 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-20 wav [[[-0.00113313 -0.00188071 -0.00296523 ... 0.06940266 0.08525381
284
+ 0.10058513]]] -0.2520575225353241 -0.00023072758631315082 0.10123191773891449 0.2524797320365906 (1, 1, 7680)
285
+ 2026-04-21 00:44:46.482 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-20 chunk_tokens [[[ 935 432 332 303]
286
+ [1718 190 1475 160]
287
+ [1004 889 307 1451]
288
+ [ 934 615 322 18]
289
+ [1076 65 2020 895]
290
+ [1293 306 488 653]
291
+ [1151 159 1957 428]
292
+ [ 242 1806 95 593]
293
+ [1953 1109 117 1939]
294
+ [ 415 950 551 343]
295
+ [ 636 164 1770 268]
296
+ [1843 1556 801 157]
297
+ [ 27 1085 1957 743]
298
+ [1243 1294 123 41]
299
+ [1733 1824 1635 224]
300
+ [1684 465 170 204]]] (1, 16, 4)
301
+ 2026-04-21 00:44:46.830 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-24 wav [[[0.11141662 0.12207047 0.12712051 ... 0.1862045 0.18386777 0.16845803]]] -0.22820299863815308 0.00029904828988946974 0.11575663834810257 0.23572805523872375 (1, 1, 7680)
302
+ 2026-04-21 00:44:46.830 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-24 chunk_tokens [[[1976 438 640 178]
303
+ [1963 638 381 976]
304
+ [1973 945 583 257]
305
+ [1207 1694 1846 1708]
306
+ [ 721 948 1227 269]
307
+ [1882 1274 75 1781]
308
+ [1597 613 684 955]
309
+ [ 65 564 176 1988]
310
+ [1685 1884 194 1766]
311
+ [ 715 1632 2027 664]
312
+ [ 834 232 1290 1755]
313
+ [ 326 1942 1395 837]
314
+ [ 63 418 642 880]
315
+ [1167 42 1458 1782]
316
+ [1572 709 58 1208]
317
+ [ 484 331 825 800]]] (1, 16, 4)
318
+ 2026-04-21 00:44:47.161 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-28 wav [[[ 0.14784081 0.1363697 0.11662599 ... -0.00261529 -0.00429974
319
+ -0.00954598]]] -0.2821105122566223 -8.334596350323409e-05 0.06657589972019196 0.24342899024486542 (1, 1, 7680)
320
+ 2026-04-21 00:44:47.162 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-28 chunk_tokens [[[ 612 1907 357 238]
321
+ [ 652 49 1168 547]
322
+ [ 142 1933 1730 640]
323
+ [ 669 1620 1668 148]
324
+ [ 129 578 101 1944]
325
+ [ 849 1175 729 554]
326
+ [ 731 333 759 250]
327
+ [1637 660 98 349]
328
+ [ 529 994 154 1234]
329
+ [1715 1164 48 1413]
330
+ [ 767 140 299 944]
331
+ [ 393 705 246 37]
332
+ [1948 417 167 1062]
333
+ [ 309 174 2008 585]
334
+ [1123 251 135 124]
335
+ [ 342 1772 81 588]]] (1, 16, 4)
336
+ 2026-04-21 00:44:47.495 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-32 wav [[[-0.00581808 -0.01583617 -0.01926371 ... 0.05514172 0.05564082
337
+ 0.0496985 ]]] -0.2244710624217987 0.00014228765212465078 0.0690271258354187 0.24306735396385193 (1, 1, 7680)
338
+ 2026-04-21 00:44:47.495 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-32 chunk_tokens [[[1776 298 1177 609]
339
+ [ 819 1373 1499 1009]
340
+ [1922 173 1192 1390]
341
+ [ 788 1394 363 512]
342
+ [ 824 796 855 363]
343
+ [1406 1768 1498 69]
344
+ [ 70 283 1037 725]
345
+ [ 306 371 376 1777]
346
+ [ 604 1785 149 1789]
347
+ [1882 317 1384 1584]
348
+ [ 955 78 1263 1672]
349
+ [ 144 1828 1390 940]
350
+ [1431 782 1754 487]
351
+ [1142 1078 84 1068]
352
+ [ 659 1840 379 1094]
353
+ [1187 1313 726 1581]]] (1, 16, 4)
354
+ 2026-04-21 00:44:47.827 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-36 wav [[[5.3663090e-02 5.0933532e-02 3.9239712e-02 ... 1.2927796e-05
355
+ 9.4157494e-06 8.8481956e-06]]] -0.1316494196653366 -5.41581648576539e-05 0.018893223255872726 0.11052974313497543 (1, 1, 7680)
356
+ 2026-04-21 00:44:47.827 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-36 chunk_tokens [[[1751 1841 1362 377]
357
+ [ 284 1043 957 1412]
358
+ [ 184 1009 771 740]
359
+ [1800 657 279 415]
360
+ [ 625 909 229 994]
361
+ [ 407 698 390 1607]
362
+ [ 944 1300 1421 318]
363
+ [ 812 486 2008 1965]
364
+ [1511 810 1775 1257]
365
+ [ 103 1360 1314 1813]
366
+ [1241 18 623 423]
367
+ [ 437 175 146 208]
368
+ [ 914 783 1555 183]
369
+ [1484 344 1860 829]
370
+ [ 586 1213 1008 602]
371
+ [1668 826 747 1088]]] (1, 16, 4)
372
+ 2026-04-21 00:44:48.159 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-40 wav [[[ 5.9778017e-06 5.4658071e-06 1.2740746e-05 ... -2.6360823e-02
373
+ -3.6881488e-02 -4.5655541e-02]]] -0.2161637395620346 0.00037617754423990846 0.032686296850442886 0.22844812273979187 (1, 1, 7680)
374
+ 2026-04-21 00:44:48.159 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-40 chunk_tokens [[[ 673 1713 1566 789]
375
+ [ 695 852 983 1985]
376
+ [1654 78 464 80]
377
+ [1524 2036 742 85]
378
+ [1205 282 1712 477]
379
+ [ 119 1802 163 808]
380
+ [ 88 671 453 1638]
381
+ [1306 1054 979 91]
382
+ [ 232 1509 123 125]
383
+ [ 373 484 786 155]
384
+ [1575 323 582 1331]
385
+ [1658 724 426 1237]
386
+ [ 345 272 823 1460]
387
+ [1538 1752 1478 503]
388
+ [1333 1225 1216 2032]
389
+ [ 950 524 358 696]]] (1, 16, 4)
390
+ 2026-04-21 00:44:48.491 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-44 wav [[[-0.05586707 -0.06594197 -0.07511631 ... -0.03430359 -0.04504313
391
+ -0.05698609]]] -0.3481614291667938 -0.0001899137714644894 0.15458345413208008 0.27548617124557495 (1, 1, 7680)
392
+ 2026-04-21 00:44:48.491 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-44 chunk_tokens [[[ 679 821 499 871]
393
+ [ 468 1773 1440 1703]
394
+ [ 105 396 49 475]
395
+ [1767 432 916 298]
396
+ [1266 886 966 1658]
397
+ [ 546 1150 1050 131]
398
+ [1425 1335 1845 1448]
399
+ [ 40 672 1042 1374]
400
+ [ 489 223 456 309]
401
+ [ 105 1053 1180 815]
402
+ [1763 427 1653 1057]
403
+ [1236 286 1349 524]
404
+ [1708 1791 13 1346]
405
+ [1690 463 64 245]
406
+ [1661 503 125 382]
407
+ [1712 556 620 784]]] (1, 16, 4)
408
+ 2026-04-21 00:44:48.822 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-48 wav [[[-0.05580926 -0.06228221 -0.08199493 ... 0.00643386 0.00706497
409
+ 0.00722447]]] -0.24150973558425903 2.1550386009039357e-05 0.06965401023626328 0.23486033082008362 (1, 1, 7680)
410
+ 2026-04-21 00:44:48.822 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-48 chunk_tokens [[[1132 1215 87 1162]
411
+ [ 40 519 2025 1668]
412
+ [1491 2010 106 1719]
413
+ [1283 1462 583 1744]
414
+ [1201 433 1487 1921]
415
+ [1975 846 1345 379]
416
+ [ 114 403 1270 421]
417
+ [ 158 799 761 956]
418
+ [ 940 2019 1510 376]
419
+ [1483 658 1116 877]
420
+ [ 750 1493 264 1038]
421
+ [ 266 457 677 611]
422
+ [ 210 438 35 204]
423
+ [1489 1341 584 1134]
424
+ [ 972 17 1532 1941]
425
+ [ 311 257 1634 1975]]] (1, 16, 4)
426
+ 2026-04-21 00:44:49.154 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-52 wav [[[ 0.0070783 0.0029405 -0.01667751 ... -0.05994403 -0.06033353
427
+ -0.0590897 ]]] -0.19280454516410828 -0.0002911569026764482 0.07178128510713577 0.2232988476753235 (1, 1, 7680)
428
+ 2026-04-21 00:44:49.154 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-52 chunk_tokens [[[ 835 2004 75]
429
+ [1259 1026 1345]
430
+ [1553 1997 550]
431
+ [1022 812 1654]
432
+ [1784 555 1141]
433
+ [1138 634 1534]
434
+ [ 289 1695 308]
435
+ [1012 208 1745]
436
+ [ 8 967 794]
437
+ [ 123 674 525]
438
+ [ 410 740 1601]
439
+ [ 447 315 812]
440
+ [ 444 1407 1493]
441
+ [1841 665 457]
442
+ [1715 59 341]
443
+ [ 157 1069 1091]]] (1, 16, 3)
444
+ 2026-04-21 00:44:49.154 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-52 len_pad_right 1 chunk_tokens [[[ 835 2004 75 0]
445
+ [1259 1026 1345 0]
446
+ [1553 1997 550 0]
447
+ [1022 812 1654 0]
448
+ [1784 555 1141 0]
449
+ [1138 634 1534 0]
450
+ [ 289 1695 308 0]
451
+ [1012 208 1745 0]
452
+ [ 8 967 794 0]
453
+ [ 123 674 525 0]
454
+ [ 410 740 1601 0]
455
+ [ 447 315 812 0]
456
+ [ 444 1407 1493 0]
457
+ [1841 665 457 0]
458
+ [1715 59 341 0]
459
+ [ 157 1069 1091 0]]] (1, 16, 4)
460
+ 2026-04-21 00:44:49.490 [INFO] src.inference.qwen3_tts_inferencer_onnx: codec-idx-56 wav [[[-0.05453582 -0.05459042 -0.05601531 ... 0.02302817 0.02265198
461
+ 0.02237365]]] -0.10915161669254303 0.00023174207308329642 0.02588086575269699 0.10188718885183334 (1, 1, 5760)
462
+ 2026-04-21 00:44:49.490 [INFO] __main__: Synthesis complete: 4.40 s audio in 29.69 s (RTF=6.749)
463
+ 2026-04-21 00:44:49.491 [INFO] __main__: Wrote audio_synth/output_1776732259.7956588.wav (4.40 s, 24000 Hz)