Hugging Face's logo

XenonFear128
/
Agnes-3.0-Flash-32GB-NVFP4-k2

Image-Text-to-Text
Safetensors
modelopt
agnes
quantized
nvfp4
fp8
custom_code
tool-calling
conversational
8-bit precision
modelopt_mixed
Model card Files
xet
 Community
XenonFear128 commited on
Commit
33ee44e
·
verified ·
1 Parent(s): 2b00af3

Upload hf_quant_config.json with huggingface_hub

Browse files
Files changed (1) hide show
  1. hf_quant_config.json +2120 -0
hf_quant_config.json ADDED
@@ -0,0 +1,2120 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "producer": {
3
+ "name": "modelopt",
4
+ "version": "0.46.1"
5
+ },
6
+ "quantization": {
7
+ "quant_algo": "MIXED_PRECISION",
8
+ "kv_cache_quant_algo": null,
9
+ "quantized_layers": {
10
+ "model.language_model.layers.0.linear_attn.out_proj": {
11
+ "quant_algo": "NVFP4",
12
+ "group_size": 16
13
+ },
14
+ "model.language_model.layers.0.linear_attn.in_proj_qkv": {
15
+ "quant_algo": "FP8"
16
+ },
17
+ "model.language_model.layers.0.linear_attn.in_proj_z": {
18
+ "quant_algo": "FP8"
19
+ },
20
+ "model.language_model.layers.0.mlp.gate_proj": {
21
+ "quant_algo": "NVFP4",
22
+ "group_size": 16
23
+ },
24
+ "model.language_model.layers.0.mlp.up_proj": {
25
+ "quant_algo": "NVFP4",
26
+ "group_size": 16
27
+ },
28
+ "model.language_model.layers.0.mlp.down_proj": {
29
+ "quant_algo": "NVFP4",
30
+ "group_size": 16
31
+ },
32
+ "model.language_model.layers.1.linear_attn.out_proj": {
33
+ "quant_algo": "NVFP4",
34
+ "group_size": 16
35
+ },
36
+ "model.language_model.layers.1.linear_attn.in_proj_qkv": {
37
+ "quant_algo": "FP8"
38
+ },
39
+ "model.language_model.layers.1.linear_attn.in_proj_z": {
40
+ "quant_algo": "FP8"
41
+ },
42
+ "model.language_model.layers.1.mlp.gate_proj": {
43
+ "quant_algo": "NVFP4",
44
+ "group_size": 16
45
+ },
46
+ "model.language_model.layers.1.mlp.up_proj": {
47
+ "quant_algo": "NVFP4",
48
+ "group_size": 16
49
+ },
50
+ "model.language_model.layers.1.mlp.down_proj": {
51
+ "quant_algo": "NVFP4",
52
+ "group_size": 16
53
+ },
54
+ "model.language_model.layers.2.linear_attn.out_proj": {
55
+ "quant_algo": "NVFP4",
56
+ "group_size": 16
57
+ },
58
+ "model.language_model.layers.2.linear_attn.in_proj_qkv": {
59
+ "quant_algo": "FP8"
60
+ },
61
+ "model.language_model.layers.2.linear_attn.in_proj_z": {
62
+ "quant_algo": "FP8"
63
+ },
64
+ "model.language_model.layers.2.mlp.gate_proj": {
65
+ "quant_algo": "NVFP4",
66
+ "group_size": 16
67
+ },
68
+ "model.language_model.layers.2.mlp.up_proj": {
69
+ "quant_algo": "NVFP4",
70
+ "group_size": 16
71
+ },
72
+ "model.language_model.layers.2.mlp.down_proj": {
73
+ "quant_algo": "NVFP4",
74
+ "group_size": 16
75
+ },
76
+ "model.language_model.layers.3.self_attn.q_proj": {
77
+ "quant_algo": "FP8"
78
+ },
79
+ "model.language_model.layers.3.self_attn.k_proj": {
80
+ "quant_algo": "FP8"
81
+ },
82
+ "model.language_model.layers.3.self_attn.v_proj": {
83
+ "quant_algo": "FP8"
84
+ },
85
+ "model.language_model.layers.3.self_attn.o_proj": {
86
+ "quant_algo": "NVFP4",
87
+ "group_size": 16
88
+ },
89
+ "model.language_model.layers.3.mlp.gate_proj": {
90
+ "quant_algo": "NVFP4",
91
+ "group_size": 16
92
+ },
93
+ "model.language_model.layers.3.mlp.up_proj": {
94
+ "quant_algo": "NVFP4",
95
+ "group_size": 16
96
+ },
97
+ "model.language_model.layers.3.mlp.down_proj": {
98
+ "quant_algo": "NVFP4",
99
+ "group_size": 16
100
+ },
101
+ "model.language_model.layers.4.linear_attn.out_proj": {
102
+ "quant_algo": "NVFP4",
103
+ "group_size": 16
104
+ },
105
+ "model.language_model.layers.4.linear_attn.in_proj_qkv": {
106
+ "quant_algo": "FP8"
107
+ },
108
+ "model.language_model.layers.4.linear_attn.in_proj_z": {
109
+ "quant_algo": "FP8"
110
+ },
111
+ "model.language_model.layers.4.mlp.gate_proj": {
112
+ "quant_algo": "NVFP4",
113
+ "group_size": 16
114
+ },
115
+ "model.language_model.layers.4.mlp.up_proj": {
116
+ "quant_algo": "NVFP4",
117
+ "group_size": 16
118
+ },
119
+ "model.language_model.layers.4.mlp.down_proj": {
120
+ "quant_algo": "NVFP4",
121
+ "group_size": 16
122
+ },
123
+ "model.language_model.layers.5.linear_attn.out_proj": {
124
+ "quant_algo": "NVFP4",
125
+ "group_size": 16
126
+ },
127
+ "model.language_model.layers.5.linear_attn.in_proj_qkv": {
128
+ "quant_algo": "FP8"
129
+ },
130
+ "model.language_model.layers.5.linear_attn.in_proj_z": {
131
+ "quant_algo": "FP8"
132
+ },
133
+ "model.language_model.layers.5.mlp.gate_proj": {
134
+ "quant_algo": "NVFP4",
135
+ "group_size": 16
136
+ },
137
+ "model.language_model.layers.5.mlp.up_proj": {
138
+ "quant_algo": "NVFP4",
139
+ "group_size": 16
140
+ },
141
+ "model.language_model.layers.5.mlp.down_proj": {
142
+ "quant_algo": "NVFP4",
143
+ "group_size": 16
144
+ },
145
+ "model.language_model.layers.6.linear_attn.out_proj": {
146
+ "quant_algo": "NVFP4",
147
+ "group_size": 16
148
+ },
149
+ "model.language_model.layers.6.linear_attn.in_proj_qkv": {
150
+ "quant_algo": "FP8"
151
+ },
152
+ "model.language_model.layers.6.linear_attn.in_proj_z": {
153
+ "quant_algo": "FP8"
154
+ },
155
+ "model.language_model.layers.6.mlp.gate_proj": {
156
+ "quant_algo": "NVFP4",
157
+ "group_size": 16
158
+ },
159
+ "model.language_model.layers.6.mlp.up_proj": {
160
+ "quant_algo": "NVFP4",
161
+ "group_size": 16
162
+ },
163
+ "model.language_model.layers.6.mlp.down_proj": {
164
+ "quant_algo": "NVFP4",
165
+ "group_size": 16
166
+ },
167
+ "model.language_model.layers.7.self_attn.q_proj": {
168
+ "quant_algo": "FP8"
169
+ },
170
+ "model.language_model.layers.7.self_attn.k_proj": {
171
+ "quant_algo": "FP8"
172
+ },
173
+ "model.language_model.layers.7.self_attn.v_proj": {
174
+ "quant_algo": "FP8"
175
+ },
176
+ "model.language_model.layers.7.self_attn.o_proj": {
177
+ "quant_algo": "NVFP4",
178
+ "group_size": 16
179
+ },
180
+ "model.language_model.layers.7.mlp.gate_proj": {
181
+ "quant_algo": "NVFP4",
182
+ "group_size": 16
183
+ },
184
+ "model.language_model.layers.7.mlp.up_proj": {
185
+ "quant_algo": "NVFP4",
186
+ "group_size": 16
187
+ },
188
+ "model.language_model.layers.7.mlp.down_proj": {
189
+ "quant_algo": "NVFP4",
190
+ "group_size": 16
191
+ },
192
+ "model.language_model.layers.8.linear_attn.out_proj": {
193
+ "quant_algo": "NVFP4",
194
+ "group_size": 16
195
+ },
196
+ "model.language_model.layers.8.linear_attn.in_proj_qkv": {
197
+ "quant_algo": "FP8"
198
+ },
199
+ "model.language_model.layers.8.linear_attn.in_proj_z": {
200
+ "quant_algo": "FP8"
201
+ },
202
+ "model.language_model.layers.8.mlp.gate_proj": {
203
+ "quant_algo": "NVFP4",
204
+ "group_size": 16
205
+ },
206
+ "model.language_model.layers.8.mlp.up_proj": {
207
+ "quant_algo": "NVFP4",
208
+ "group_size": 16
209
+ },
210
+ "model.language_model.layers.8.mlp.down_proj": {
211
+ "quant_algo": "NVFP4",
212
+ "group_size": 16
213
+ },
214
+ "model.language_model.layers.9.linear_attn.out_proj": {
215
+ "quant_algo": "NVFP4",
216
+ "group_size": 16
217
+ },
218
+ "model.language_model.layers.9.linear_attn.in_proj_qkv": {
219
+ "quant_algo": "FP8"
220
+ },
221
+ "model.language_model.layers.9.linear_attn.in_proj_z": {
222
+ "quant_algo": "FP8"
223
+ },
224
+ "model.language_model.layers.9.mlp.gate_proj": {
225
+ "quant_algo": "NVFP4",
226
+ "group_size": 16
227
+ },
228
+ "model.language_model.layers.9.mlp.up_proj": {
229
+ "quant_algo": "NVFP4",
230
+ "group_size": 16
231
+ },
232
+ "model.language_model.layers.9.mlp.down_proj": {
233
+ "quant_algo": "NVFP4",
234
+ "group_size": 16
235
+ },
236
+ "model.language_model.layers.10.linear_attn.out_proj": {
237
+ "quant_algo": "NVFP4",
238
+ "group_size": 16
239
+ },
240
+ "model.language_model.layers.10.linear_attn.in_proj_qkv": {
241
+ "quant_algo": "FP8"
242
+ },
243
+ "model.language_model.layers.10.linear_attn.in_proj_z": {
244
+ "quant_algo": "FP8"
245
+ },
246
+ "model.language_model.layers.10.mlp.gate_proj": {
247
+ "quant_algo": "NVFP4",
248
+ "group_size": 16
249
+ },
250
+ "model.language_model.layers.10.mlp.up_proj": {
251
+ "quant_algo": "NVFP4",
252
+ "group_size": 16
253
+ },
254
+ "model.language_model.layers.10.mlp.down_proj": {
255
+ "quant_algo": "NVFP4",
256
+ "group_size": 16
257
+ },
258
+ "model.language_model.layers.11.self_attn.q_proj": {
259
+ "quant_algo": "FP8"
260
+ },
261
+ "model.language_model.layers.11.self_attn.k_proj": {
262
+ "quant_algo": "FP8"
263
+ },
264
+ "model.language_model.layers.11.self_attn.v_proj": {
265
+ "quant_algo": "FP8"
266
+ },
267
+ "model.language_model.layers.11.self_attn.o_proj": {
268
+ "quant_algo": "NVFP4",
269
+ "group_size": 16
270
+ },
271
+ "model.language_model.layers.11.mlp.gate_proj": {
272
+ "quant_algo": "NVFP4",
273
+ "group_size": 16
274
+ },
275
+ "model.language_model.layers.11.mlp.up_proj": {
276
+ "quant_algo": "NVFP4",
277
+ "group_size": 16
278
+ },
279
+ "model.language_model.layers.11.mlp.down_proj": {
280
+ "quant_algo": "NVFP4",
281
+ "group_size": 16
282
+ },
283
+ "model.language_model.layers.12.linear_attn.out_proj": {
284
+ "quant_algo": "NVFP4",
285
+ "group_size": 16
286
+ },
287
+ "model.language_model.layers.12.linear_attn.in_proj_qkv": {
288
+ "quant_algo": "FP8"
289
+ },
290
+ "model.language_model.layers.12.linear_attn.in_proj_z": {
291
+ "quant_algo": "FP8"
292
+ },
293
+ "model.language_model.layers.12.mlp.gate_proj": {
294
+ "quant_algo": "NVFP4",
295
+ "group_size": 16
296
+ },
297
+ "model.language_model.layers.12.mlp.up_proj": {
298
+ "quant_algo": "NVFP4",
299
+ "group_size": 16
300
+ },
301
+ "model.language_model.layers.12.mlp.down_proj": {
302
+ "quant_algo": "NVFP4",
303
+ "group_size": 16
304
+ },
305
+ "model.language_model.layers.13.linear_attn.out_proj": {
306
+ "quant_algo": "NVFP4",
307
+ "group_size": 16
308
+ },
309
+ "model.language_model.layers.13.linear_attn.in_proj_qkv": {
310
+ "quant_algo": "FP8"
311
+ },
312
+ "model.language_model.layers.13.linear_attn.in_proj_z": {
313
+ "quant_algo": "FP8"
314
+ },
315
+ "model.language_model.layers.13.mlp.gate_proj": {
316
+ "quant_algo": "NVFP4",
317
+ "group_size": 16
318
+ },
319
+ "model.language_model.layers.13.mlp.up_proj": {
320
+ "quant_algo": "NVFP4",
321
+ "group_size": 16
322
+ },
323
+ "model.language_model.layers.13.mlp.down_proj": {
324
+ "quant_algo": "NVFP4",
325
+ "group_size": 16
326
+ },
327
+ "model.language_model.layers.14.linear_attn.out_proj": {
328
+ "quant_algo": "NVFP4",
329
+ "group_size": 16
330
+ },
331
+ "model.language_model.layers.14.linear_attn.in_proj_qkv": {
332
+ "quant_algo": "FP8"
333
+ },
334
+ "model.language_model.layers.14.linear_attn.in_proj_z": {
335
+ "quant_algo": "FP8"
336
+ },
337
+ "model.language_model.layers.14.mlp.gate_proj": {
338
+ "quant_algo": "NVFP4",
339
+ "group_size": 16
340
+ },
341
+ "model.language_model.layers.14.mlp.up_proj": {
342
+ "quant_algo": "NVFP4",
343
+ "group_size": 16
344
+ },
345
+ "model.language_model.layers.14.mlp.down_proj": {
346
+ "quant_algo": "NVFP4",
347
+ "group_size": 16
348
+ },
349
+ "model.language_model.layers.15.self_attn.q_proj": {
350
+ "quant_algo": "FP8"
351
+ },
352
+ "model.language_model.layers.15.self_attn.k_proj": {
353
+ "quant_algo": "FP8"
354
+ },
355
+ "model.language_model.layers.15.self_attn.v_proj": {
356
+ "quant_algo": "FP8"
357
+ },
358
+ "model.language_model.layers.15.self_attn.o_proj": {
359
+ "quant_algo": "NVFP4",
360
+ "group_size": 16
361
+ },
362
+ "model.language_model.layers.15.mlp.gate_proj": {
363
+ "quant_algo": "NVFP4",
364
+ "group_size": 16
365
+ },
366
+ "model.language_model.layers.15.mlp.up_proj": {
367
+ "quant_algo": "NVFP4",
368
+ "group_size": 16
369
+ },
370
+ "model.language_model.layers.15.mlp.down_proj": {
371
+ "quant_algo": "NVFP4",
372
+ "group_size": 16
373
+ },
374
+ "model.language_model.layers.16.linear_attn.out_proj": {
375
+ "quant_algo": "NVFP4",
376
+ "group_size": 16
377
+ },
378
+ "model.language_model.layers.16.linear_attn.in_proj_qkv": {
379
+ "quant_algo": "FP8"
380
+ },
381
+ "model.language_model.layers.16.linear_attn.in_proj_z": {
382
+ "quant_algo": "FP8"
383
+ },
384
+ "model.language_model.layers.16.mlp.gate_proj": {
385
+ "quant_algo": "NVFP4",
386
+ "group_size": 16
387
+ },
388
+ "model.language_model.layers.16.mlp.up_proj": {
389
+ "quant_algo": "NVFP4",
390
+ "group_size": 16
391
+ },
392
+ "model.language_model.layers.16.mlp.down_proj": {
393
+ "quant_algo": "NVFP4",
394
+ "group_size": 16
395
+ },
396
+ "model.language_model.layers.17.linear_attn.out_proj": {
397
+ "quant_algo": "NVFP4",
398
+ "group_size": 16
399
+ },
400
+ "model.language_model.layers.17.linear_attn.in_proj_qkv": {
401
+ "quant_algo": "FP8"
402
+ },
403
+ "model.language_model.layers.17.linear_attn.in_proj_z": {
404
+ "quant_algo": "FP8"
405
+ },
406
+ "model.language_model.layers.17.mlp.gate_proj": {
407
+ "quant_algo": "NVFP4",
408
+ "group_size": 16
409
+ },
410
+ "model.language_model.layers.17.mlp.up_proj": {
411
+ "quant_algo": "NVFP4",
412
+ "group_size": 16
413
+ },
414
+ "model.language_model.layers.17.mlp.down_proj": {
415
+ "quant_algo": "NVFP4",
416
+ "group_size": 16
417
+ },
418
+ "model.language_model.layers.18.linear_attn.out_proj": {
419
+ "quant_algo": "NVFP4",
420
+ "group_size": 16
421
+ },
422
+ "model.language_model.layers.18.linear_attn.in_proj_qkv": {
423
+ "quant_algo": "FP8"
424
+ },
425
+ "model.language_model.layers.18.linear_attn.in_proj_z": {
426
+ "quant_algo": "FP8"
427
+ },
428
+ "model.language_model.layers.18.mlp.gate_proj": {
429
+ "quant_algo": "NVFP4",
430
+ "group_size": 16
431
+ },
432
+ "model.language_model.layers.18.mlp.up_proj": {
433
+ "quant_algo": "NVFP4",
434
+ "group_size": 16
435
+ },
436
+ "model.language_model.layers.18.mlp.down_proj": {
437
+ "quant_algo": "NVFP4",
438
+ "group_size": 16
439
+ },
440
+ "model.language_model.layers.19.self_attn.q_proj": {
441
+ "quant_algo": "FP8"
442
+ },
443
+ "model.language_model.layers.19.self_attn.k_proj": {
444
+ "quant_algo": "FP8"
445
+ },
446
+ "model.language_model.layers.19.self_attn.v_proj": {
447
+ "quant_algo": "FP8"
448
+ },
449
+ "model.language_model.layers.19.self_attn.o_proj": {
450
+ "quant_algo": "NVFP4",
451
+ "group_size": 16
452
+ },
453
+ "model.language_model.layers.19.mlp.gate_proj": {
454
+ "quant_algo": "NVFP4",
455
+ "group_size": 16
456
+ },
457
+ "model.language_model.layers.19.mlp.up_proj": {
458
+ "quant_algo": "NVFP4",
459
+ "group_size": 16
460
+ },
461
+ "model.language_model.layers.19.mlp.down_proj": {
462
+ "quant_algo": "NVFP4",
463
+ "group_size": 16
464
+ },
465
+ "model.language_model.layers.20.linear_attn.out_proj": {
466
+ "quant_algo": "NVFP4",
467
+ "group_size": 16
468
+ },
469
+ "model.language_model.layers.20.linear_attn.in_proj_qkv": {
470
+ "quant_algo": "FP8"
471
+ },
472
+ "model.language_model.layers.20.linear_attn.in_proj_z": {
473
+ "quant_algo": "FP8"
474
+ },
475
+ "model.language_model.layers.20.mlp.gate_proj": {
476
+ "quant_algo": "NVFP4",
477
+ "group_size": 16
478
+ },
479
+ "model.language_model.layers.20.mlp.up_proj": {
480
+ "quant_algo": "NVFP4",
481
+ "group_size": 16
482
+ },
483
+ "model.language_model.layers.20.mlp.down_proj": {
484
+ "quant_algo": "NVFP4",
485
+ "group_size": 16
486
+ },
487
+ "model.language_model.layers.21.linear_attn.out_proj": {
488
+ "quant_algo": "NVFP4",
489
+ "group_size": 16
490
+ },
491
+ "model.language_model.layers.21.linear_attn.in_proj_qkv": {
492
+ "quant_algo": "FP8"
493
+ },
494
+ "model.language_model.layers.21.linear_attn.in_proj_z": {
495
+ "quant_algo": "FP8"
496
+ },
497
+ "model.language_model.layers.21.mlp.gate_proj": {
498
+ "quant_algo": "NVFP4",
499
+ "group_size": 16
500
+ },
501
+ "model.language_model.layers.21.mlp.up_proj": {
502
+ "quant_algo": "NVFP4",
503
+ "group_size": 16
504
+ },
505
+ "model.language_model.layers.21.mlp.down_proj": {
506
+ "quant_algo": "NVFP4",
507
+ "group_size": 16
508
+ },
509
+ "model.language_model.layers.22.linear_attn.out_proj": {
510
+ "quant_algo": "NVFP4",
511
+ "group_size": 16
512
+ },
513
+ "model.language_model.layers.22.linear_attn.in_proj_qkv": {
514
+ "quant_algo": "FP8"
515
+ },
516
+ "model.language_model.layers.22.linear_attn.in_proj_z": {
517
+ "quant_algo": "FP8"
518
+ },
519
+ "model.language_model.layers.22.mlp.gate_proj": {
520
+ "quant_algo": "NVFP4",
521
+ "group_size": 16
522
+ },
523
+ "model.language_model.layers.22.mlp.up_proj": {
524
+ "quant_algo": "NVFP4",
525
+ "group_size": 16
526
+ },
527
+ "model.language_model.layers.22.mlp.down_proj": {
528
+ "quant_algo": "NVFP4",
529
+ "group_size": 16
530
+ },
531
+ "model.language_model.layers.23.self_attn.q_proj": {
532
+ "quant_algo": "FP8"
533
+ },
534
+ "model.language_model.layers.23.self_attn.k_proj": {
535
+ "quant_algo": "FP8"
536
+ },
537
+ "model.language_model.layers.23.self_attn.v_proj": {
538
+ "quant_algo": "FP8"
539
+ },
540
+ "model.language_model.layers.23.self_attn.o_proj": {
541
+ "quant_algo": "NVFP4",
542
+ "group_size": 16
543
+ },
544
+ "model.language_model.layers.23.mlp.gate_proj": {
545
+ "quant_algo": "NVFP4",
546
+ "group_size": 16
547
+ },
548
+ "model.language_model.layers.23.mlp.up_proj": {
549
+ "quant_algo": "NVFP4",
550
+ "group_size": 16
551
+ },
552
+ "model.language_model.layers.23.mlp.down_proj": {
553
+ "quant_algo": "NVFP4",
554
+ "group_size": 16
555
+ },
556
+ "model.language_model.layers.24.linear_attn.out_proj": {
557
+ "quant_algo": "NVFP4",
558
+ "group_size": 16
559
+ },
560
+ "model.language_model.layers.24.linear_attn.in_proj_qkv": {
561
+ "quant_algo": "FP8"
562
+ },
563
+ "model.language_model.layers.24.linear_attn.in_proj_z": {
564
+ "quant_algo": "FP8"
565
+ },
566
+ "model.language_model.layers.24.mlp.gate_proj": {
567
+ "quant_algo": "NVFP4",
568
+ "group_size": 16
569
+ },
570
+ "model.language_model.layers.24.mlp.up_proj": {
571
+ "quant_algo": "NVFP4",
572
+ "group_size": 16
573
+ },
574
+ "model.language_model.layers.24.mlp.down_proj": {
575
+ "quant_algo": "NVFP4",
576
+ "group_size": 16
577
+ },
578
+ "model.language_model.layers.25.linear_attn.out_proj": {
579
+ "quant_algo": "NVFP4",
580
+ "group_size": 16
581
+ },
582
+ "model.language_model.layers.25.linear_attn.in_proj_qkv": {
583
+ "quant_algo": "FP8"
584
+ },
585
+ "model.language_model.layers.25.linear_attn.in_proj_z": {
586
+ "quant_algo": "FP8"
587
+ },
588
+ "model.language_model.layers.25.mlp.gate_proj": {
589
+ "quant_algo": "NVFP4",
590
+ "group_size": 16
591
+ },
592
+ "model.language_model.layers.25.mlp.up_proj": {
593
+ "quant_algo": "NVFP4",
594
+ "group_size": 16
595
+ },
596
+ "model.language_model.layers.25.mlp.down_proj": {
597
+ "quant_algo": "NVFP4",
598
+ "group_size": 16
599
+ },
600
+ "model.language_model.layers.26.linear_attn.out_proj": {
601
+ "quant_algo": "NVFP4",
602
+ "group_size": 16
603
+ },
604
+ "model.language_model.layers.26.linear_attn.in_proj_qkv": {
605
+ "quant_algo": "FP8"
606
+ },
607
+ "model.language_model.layers.26.linear_attn.in_proj_z": {
608
+ "quant_algo": "FP8"
609
+ },
610
+ "model.language_model.layers.26.mlp.gate_proj": {
611
+ "quant_algo": "NVFP4",
612
+ "group_size": 16
613
+ },
614
+ "model.language_model.layers.26.mlp.up_proj": {
615
+ "quant_algo": "NVFP4",
616
+ "group_size": 16
617
+ },
618
+ "model.language_model.layers.26.mlp.down_proj": {
619
+ "quant_algo": "NVFP4",
620
+ "group_size": 16
621
+ },
622
+ "model.language_model.layers.27.self_attn.q_proj": {
623
+ "quant_algo": "FP8"
624
+ },
625
+ "model.language_model.layers.27.self_attn.k_proj": {
626
+ "quant_algo": "FP8"
627
+ },
628
+ "model.language_model.layers.27.self_attn.v_proj": {
629
+ "quant_algo": "FP8"
630
+ },
631
+ "model.language_model.layers.27.self_attn.o_proj": {
632
+ "quant_algo": "NVFP4",
633
+ "group_size": 16
634
+ },
635
+ "model.language_model.layers.27.mlp.gate_proj": {
636
+ "quant_algo": "NVFP4",
637
+ "group_size": 16
638
+ },
639
+ "model.language_model.layers.27.mlp.up_proj": {
640
+ "quant_algo": "NVFP4",
641
+ "group_size": 16
642
+ },
643
+ "model.language_model.layers.27.mlp.down_proj": {
644
+ "quant_algo": "NVFP4",
645
+ "group_size": 16
646
+ },
647
+ "model.language_model.layers.28.linear_attn.out_proj": {
648
+ "quant_algo": "NVFP4",
649
+ "group_size": 16
650
+ },
651
+ "model.language_model.layers.28.linear_attn.in_proj_qkv": {
652
+ "quant_algo": "FP8"
653
+ },
654
+ "model.language_model.layers.28.linear_attn.in_proj_z": {
655
+ "quant_algo": "FP8"
656
+ },
657
+ "model.language_model.layers.28.mlp.gate_proj": {
658
+ "quant_algo": "NVFP4",
659
+ "group_size": 16
660
+ },
661
+ "model.language_model.layers.28.mlp.up_proj": {
662
+ "quant_algo": "NVFP4",
663
+ "group_size": 16
664
+ },
665
+ "model.language_model.layers.28.mlp.down_proj": {
666
+ "quant_algo": "NVFP4",
667
+ "group_size": 16
668
+ },
669
+ "model.language_model.layers.29.linear_attn.out_proj": {
670
+ "quant_algo": "NVFP4",
671
+ "group_size": 16
672
+ },
673
+ "model.language_model.layers.29.linear_attn.in_proj_qkv": {
674
+ "quant_algo": "FP8"
675
+ },
676
+ "model.language_model.layers.29.linear_attn.in_proj_z": {
677
+ "quant_algo": "FP8"
678
+ },
679
+ "model.language_model.layers.29.mlp.gate_proj": {
680
+ "quant_algo": "NVFP4",
681
+ "group_size": 16
682
+ },
683
+ "model.language_model.layers.29.mlp.up_proj": {
684
+ "quant_algo": "NVFP4",
685
+ "group_size": 16
686
+ },
687
+ "model.language_model.layers.29.mlp.down_proj": {
688
+ "quant_algo": "NVFP4",
689
+ "group_size": 16
690
+ },
691
+ "model.language_model.layers.30.linear_attn.out_proj": {
692
+ "quant_algo": "NVFP4",
693
+ "group_size": 16
694
+ },
695
+ "model.language_model.layers.30.linear_attn.in_proj_qkv": {
696
+ "quant_algo": "FP8"
697
+ },
698
+ "model.language_model.layers.30.linear_attn.in_proj_z": {
699
+ "quant_algo": "FP8"
700
+ },
701
+ "model.language_model.layers.30.mlp.gate_proj": {
702
+ "quant_algo": "NVFP4",
703
+ "group_size": 16
704
+ },
705
+ "model.language_model.layers.30.mlp.up_proj": {
706
+ "quant_algo": "NVFP4",
707
+ "group_size": 16
708
+ },
709
+ "model.language_model.layers.30.mlp.down_proj": {
710
+ "quant_algo": "NVFP4",
711
+ "group_size": 16
712
+ },
713
+ "model.language_model.layers.31.self_attn.q_proj": {
714
+ "quant_algo": "FP8"
715
+ },
716
+ "model.language_model.layers.31.self_attn.k_proj": {
717
+ "quant_algo": "FP8"
718
+ },
719
+ "model.language_model.layers.31.self_attn.v_proj": {
720
+ "quant_algo": "FP8"
721
+ },
722
+ "model.language_model.layers.31.self_attn.o_proj": {
723
+ "quant_algo": "NVFP4",
724
+ "group_size": 16
725
+ },
726
+ "model.language_model.layers.31.mlp.gate_proj": {
727
+ "quant_algo": "NVFP4",
728
+ "group_size": 16
729
+ },
730
+ "model.language_model.layers.31.mlp.up_proj": {
731
+ "quant_algo": "NVFP4",
732
+ "group_size": 16
733
+ },
734
+ "model.language_model.layers.31.mlp.down_proj": {
735
+ "quant_algo": "NVFP4",
736
+ "group_size": 16
737
+ },
738
+ "model.language_model.layers.32.linear_attn.out_proj": {
739
+ "quant_algo": "NVFP4",
740
+ "group_size": 16
741
+ },
742
+ "model.language_model.layers.32.linear_attn.in_proj_qkv": {
743
+ "quant_algo": "FP8"
744
+ },
745
+ "model.language_model.layers.32.linear_attn.in_proj_z": {
746
+ "quant_algo": "FP8"
747
+ },
748
+ "model.language_model.layers.32.mlp.gate_proj": {
749
+ "quant_algo": "NVFP4",
750
+ "group_size": 16
751
+ },
752
+ "model.language_model.layers.32.mlp.up_proj": {
753
+ "quant_algo": "NVFP4",
754
+ "group_size": 16
755
+ },
756
+ "model.language_model.layers.32.mlp.down_proj": {
757
+ "quant_algo": "NVFP4",
758
+ "group_size": 16
759
+ },
760
+ "model.language_model.layers.33.linear_attn.out_proj": {
761
+ "quant_algo": "NVFP4",
762
+ "group_size": 16
763
+ },
764
+ "model.language_model.layers.33.linear_attn.in_proj_qkv": {
765
+ "quant_algo": "FP8"
766
+ },
767
+ "model.language_model.layers.33.linear_attn.in_proj_z": {
768
+ "quant_algo": "FP8"
769
+ },
770
+ "model.language_model.layers.33.mlp.gate_proj": {
771
+ "quant_algo": "NVFP4",
772
+ "group_size": 16
773
+ },
774
+ "model.language_model.layers.33.mlp.up_proj": {
775
+ "quant_algo": "NVFP4",
776
+ "group_size": 16
777
+ },
778
+ "model.language_model.layers.33.mlp.down_proj": {
779
+ "quant_algo": "NVFP4",
780
+ "group_size": 16
781
+ },
782
+ "model.language_model.layers.34.linear_attn.out_proj": {
783
+ "quant_algo": "NVFP4",
784
+ "group_size": 16
785
+ },
786
+ "model.language_model.layers.34.linear_attn.in_proj_qkv": {
787
+ "quant_algo": "FP8"
788
+ },
789
+ "model.language_model.layers.34.linear_attn.in_proj_z": {
790
+ "quant_algo": "FP8"
791
+ },
792
+ "model.language_model.layers.34.mlp.gate_proj": {
793
+ "quant_algo": "NVFP4",
794
+ "group_size": 16
795
+ },
796
+ "model.language_model.layers.34.mlp.up_proj": {
797
+ "quant_algo": "NVFP4",
798
+ "group_size": 16
799
+ },
800
+ "model.language_model.layers.34.mlp.down_proj": {
801
+ "quant_algo": "NVFP4",
802
+ "group_size": 16
803
+ },
804
+ "model.language_model.layers.35.self_attn.q_proj": {
805
+ "quant_algo": "FP8"
806
+ },
807
+ "model.language_model.layers.35.self_attn.k_proj": {
808
+ "quant_algo": "FP8"
809
+ },
810
+ "model.language_model.layers.35.self_attn.v_proj": {
811
+ "quant_algo": "FP8"
812
+ },
813
+ "model.language_model.layers.35.self_attn.o_proj": {
814
+ "quant_algo": "NVFP4",
815
+ "group_size": 16
816
+ },
817
+ "model.language_model.layers.35.mlp.gate_proj": {
818
+ "quant_algo": "NVFP4",
819
+ "group_size": 16
820
+ },
821
+ "model.language_model.layers.35.mlp.up_proj": {
822
+ "quant_algo": "NVFP4",
823
+ "group_size": 16
824
+ },
825
+ "model.language_model.layers.35.mlp.down_proj": {
826
+ "quant_algo": "NVFP4",
827
+ "group_size": 16
828
+ },
829
+ "model.language_model.layers.36.linear_attn.out_proj": {
830
+ "quant_algo": "NVFP4",
831
+ "group_size": 16
832
+ },
833
+ "model.language_model.layers.36.linear_attn.in_proj_qkv": {
834
+ "quant_algo": "FP8"
835
+ },
836
+ "model.language_model.layers.36.linear_attn.in_proj_z": {
837
+ "quant_algo": "FP8"
838
+ },
839
+ "model.language_model.layers.36.mlp.gate_proj": {
840
+ "quant_algo": "NVFP4",
841
+ "group_size": 16
842
+ },
843
+ "model.language_model.layers.36.mlp.up_proj": {
844
+ "quant_algo": "NVFP4",
845
+ "group_size": 16
846
+ },
847
+ "model.language_model.layers.36.mlp.down_proj": {
848
+ "quant_algo": "NVFP4",
849
+ "group_size": 16
850
+ },
851
+ "model.language_model.layers.37.linear_attn.out_proj": {
852
+ "quant_algo": "NVFP4",
853
+ "group_size": 16
854
+ },
855
+ "model.language_model.layers.37.linear_attn.in_proj_qkv": {
856
+ "quant_algo": "FP8"
857
+ },
858
+ "model.language_model.layers.37.linear_attn.in_proj_z": {
859
+ "quant_algo": "FP8"
860
+ },
861
+ "model.language_model.layers.37.mlp.gate_proj": {
862
+ "quant_algo": "NVFP4",
863
+ "group_size": 16
864
+ },
865
+ "model.language_model.layers.37.mlp.up_proj": {
866
+ "quant_algo": "NVFP4",
867
+ "group_size": 16
868
+ },
869
+ "model.language_model.layers.37.mlp.down_proj": {
870
+ "quant_algo": "NVFP4",
871
+ "group_size": 16
872
+ },
873
+ "model.language_model.layers.38.linear_attn.out_proj": {
874
+ "quant_algo": "NVFP4",
875
+ "group_size": 16
876
+ },
877
+ "model.language_model.layers.38.linear_attn.in_proj_qkv": {
878
+ "quant_algo": "FP8"
879
+ },
880
+ "model.language_model.layers.38.linear_attn.in_proj_z": {
881
+ "quant_algo": "FP8"
882
+ },
883
+ "model.language_model.layers.38.mlp.gate_proj": {
884
+ "quant_algo": "NVFP4",
885
+ "group_size": 16
886
+ },
887
+ "model.language_model.layers.38.mlp.up_proj": {
888
+ "quant_algo": "NVFP4",
889
+ "group_size": 16
890
+ },
891
+ "model.language_model.layers.38.mlp.down_proj": {
892
+ "quant_algo": "NVFP4",
893
+ "group_size": 16
894
+ },
895
+ "model.language_model.layers.39.self_attn.q_proj": {
896
+ "quant_algo": "FP8"
897
+ },
898
+ "model.language_model.layers.39.self_attn.k_proj": {
899
+ "quant_algo": "FP8"
900
+ },
901
+ "model.language_model.layers.39.self_attn.v_proj": {
902
+ "quant_algo": "FP8"
903
+ },
904
+ "model.language_model.layers.39.self_attn.o_proj": {
905
+ "quant_algo": "NVFP4",
906
+ "group_size": 16
907
+ },
908
+ "model.language_model.layers.39.mlp.gate_proj": {
909
+ "quant_algo": "NVFP4",
910
+ "group_size": 16
911
+ },
912
+ "model.language_model.layers.39.mlp.up_proj": {
913
+ "quant_algo": "NVFP4",
914
+ "group_size": 16
915
+ },
916
+ "model.language_model.layers.39.mlp.down_proj": {
917
+ "quant_algo": "NVFP4",
918
+ "group_size": 16
919
+ },
920
+ "model.language_model.layers.40.linear_attn.out_proj": {
921
+ "quant_algo": "NVFP4",
922
+ "group_size": 16
923
+ },
924
+ "model.language_model.layers.40.linear_attn.in_proj_qkv": {
925
+ "quant_algo": "FP8"
926
+ },
927
+ "model.language_model.layers.40.linear_attn.in_proj_z": {
928
+ "quant_algo": "FP8"
929
+ },
930
+ "model.language_model.layers.40.mlp.gate_proj": {
931
+ "quant_algo": "NVFP4",
932
+ "group_size": 16
933
+ },
934
+ "model.language_model.layers.40.mlp.up_proj": {
935
+ "quant_algo": "NVFP4",
936
+ "group_size": 16
937
+ },
938
+ "model.language_model.layers.40.mlp.down_proj": {
939
+ "quant_algo": "NVFP4",
940
+ "group_size": 16
941
+ },
942
+ "model.language_model.layers.41.linear_attn.out_proj": {
943
+ "quant_algo": "NVFP4",
944
+ "group_size": 16
945
+ },
946
+ "model.language_model.layers.41.linear_attn.in_proj_qkv": {
947
+ "quant_algo": "FP8"
948
+ },
949
+ "model.language_model.layers.41.linear_attn.in_proj_z": {
950
+ "quant_algo": "FP8"
951
+ },
952
+ "model.language_model.layers.41.mlp.gate_proj": {
953
+ "quant_algo": "NVFP4",
954
+ "group_size": 16
955
+ },
956
+ "model.language_model.layers.41.mlp.up_proj": {
957
+ "quant_algo": "NVFP4",
958
+ "group_size": 16
959
+ },
960
+ "model.language_model.layers.41.mlp.down_proj": {
961
+ "quant_algo": "NVFP4",
962
+ "group_size": 16
963
+ },
964
+ "model.language_model.layers.42.linear_attn.out_proj": {
965
+ "quant_algo": "NVFP4",
966
+ "group_size": 16
967
+ },
968
+ "model.language_model.layers.42.linear_attn.in_proj_qkv": {
969
+ "quant_algo": "FP8"
970
+ },
971
+ "model.language_model.layers.42.linear_attn.in_proj_z": {
972
+ "quant_algo": "FP8"
973
+ },
974
+ "model.language_model.layers.42.mlp.gate_proj": {
975
+ "quant_algo": "NVFP4",
976
+ "group_size": 16
977
+ },
978
+ "model.language_model.layers.42.mlp.up_proj": {
979
+ "quant_algo": "NVFP4",
980
+ "group_size": 16
981
+ },
982
+ "model.language_model.layers.42.mlp.down_proj": {
983
+ "quant_algo": "NVFP4",
984
+ "group_size": 16
985
+ },
986
+ "model.language_model.layers.43.self_attn.q_proj": {
987
+ "quant_algo": "FP8"
988
+ },
989
+ "model.language_model.layers.43.self_attn.k_proj": {
990
+ "quant_algo": "FP8"
991
+ },
992
+ "model.language_model.layers.43.self_attn.v_proj": {
993
+ "quant_algo": "FP8"
994
+ },
995
+ "model.language_model.layers.43.self_attn.o_proj": {
996
+ "quant_algo": "NVFP4",
997
+ "group_size": 16
998
+ },
999
+ "model.language_model.layers.43.mlp.gate_proj": {
1000
+ "quant_algo": "NVFP4",
1001
+ "group_size": 16
1002
+ },
1003
+ "model.language_model.layers.43.mlp.up_proj": {
1004
+ "quant_algo": "NVFP4",
1005
+ "group_size": 16
1006
+ },
1007
+ "model.language_model.layers.43.mlp.down_proj": {
1008
+ "quant_algo": "NVFP4",
1009
+ "group_size": 16
1010
+ },
1011
+ "model.language_model.layers.44.linear_attn.out_proj": {
1012
+ "quant_algo": "NVFP4",
1013
+ "group_size": 16
1014
+ },
1015
+ "model.language_model.layers.44.linear_attn.in_proj_qkv": {
1016
+ "quant_algo": "FP8"
1017
+ },
1018
+ "model.language_model.layers.44.linear_attn.in_proj_z": {
1019
+ "quant_algo": "FP8"
1020
+ },
1021
+ "model.language_model.layers.44.mlp.gate_proj": {
1022
+ "quant_algo": "NVFP4",
1023
+ "group_size": 16
1024
+ },
1025
+ "model.language_model.layers.44.mlp.up_proj": {
1026
+ "quant_algo": "NVFP4",
1027
+ "group_size": 16
1028
+ },
1029
+ "model.language_model.layers.44.mlp.down_proj": {
1030
+ "quant_algo": "NVFP4",
1031
+ "group_size": 16
1032
+ },
1033
+ "model.language_model.layers.45.linear_attn.out_proj": {
1034
+ "quant_algo": "NVFP4",
1035
+ "group_size": 16
1036
+ },
1037
+ "model.language_model.layers.45.linear_attn.in_proj_qkv": {
1038
+ "quant_algo": "FP8"
1039
+ },
1040
+ "model.language_model.layers.45.linear_attn.in_proj_z": {
1041
+ "quant_algo": "FP8"
1042
+ },
1043
+ "model.language_model.layers.45.mlp.gate_proj": {
1044
+ "quant_algo": "NVFP4",
1045
+ "group_size": 16
1046
+ },
1047
+ "model.language_model.layers.45.mlp.up_proj": {
1048
+ "quant_algo": "NVFP4",
1049
+ "group_size": 16
1050
+ },
1051
+ "model.language_model.layers.45.mlp.down_proj": {
1052
+ "quant_algo": "NVFP4",
1053
+ "group_size": 16
1054
+ },
1055
+ "model.language_model.layers.46.linear_attn.out_proj": {
1056
+ "quant_algo": "NVFP4",
1057
+ "group_size": 16
1058
+ },
1059
+ "model.language_model.layers.46.linear_attn.in_proj_qkv": {
1060
+ "quant_algo": "FP8"
1061
+ },
1062
+ "model.language_model.layers.46.linear_attn.in_proj_z": {
1063
+ "quant_algo": "FP8"
1064
+ },
1065
+ "model.language_model.layers.46.mlp.gate_proj": {
1066
+ "quant_algo": "NVFP4",
1067
+ "group_size": 16
1068
+ },
1069
+ "model.language_model.layers.46.mlp.up_proj": {
1070
+ "quant_algo": "NVFP4",
1071
+ "group_size": 16
1072
+ },
1073
+ "model.language_model.layers.46.mlp.down_proj": {
1074
+ "quant_algo": "NVFP4",
1075
+ "group_size": 16
1076
+ },
1077
+ "model.language_model.layers.47.self_attn.q_proj": {
1078
+ "quant_algo": "FP8"
1079
+ },
1080
+ "model.language_model.layers.47.self_attn.k_proj": {
1081
+ "quant_algo": "FP8"
1082
+ },
1083
+ "model.language_model.layers.47.self_attn.v_proj": {
1084
+ "quant_algo": "FP8"
1085
+ },
1086
+ "model.language_model.layers.47.self_attn.o_proj": {
1087
+ "quant_algo": "NVFP4",
1088
+ "group_size": 16
1089
+ },
1090
+ "model.language_model.layers.47.mlp.gate_proj": {
1091
+ "quant_algo": "NVFP4",
1092
+ "group_size": 16
1093
+ },
1094
+ "model.language_model.layers.47.mlp.up_proj": {
1095
+ "quant_algo": "NVFP4",
1096
+ "group_size": 16
1097
+ },
1098
+ "model.language_model.layers.47.mlp.down_proj": {
1099
+ "quant_algo": "NVFP4",
1100
+ "group_size": 16
1101
+ },
1102
+ "model.language_model.layers.48.linear_attn.out_proj": {
1103
+ "quant_algo": "NVFP4",
1104
+ "group_size": 16
1105
+ },
1106
+ "model.language_model.layers.48.linear_attn.in_proj_qkv": {
1107
+ "quant_algo": "FP8"
1108
+ },
1109
+ "model.language_model.layers.48.linear_attn.in_proj_z": {
1110
+ "quant_algo": "FP8"
1111
+ },
1112
+ "model.language_model.layers.48.mlp.gate_proj": {
1113
+ "quant_algo": "NVFP4",
1114
+ "group_size": 16
1115
+ },
1116
+ "model.language_model.layers.48.mlp.up_proj": {
1117
+ "quant_algo": "NVFP4",
1118
+ "group_size": 16
1119
+ },
1120
+ "model.language_model.layers.48.mlp.down_proj": {
1121
+ "quant_algo": "NVFP4",
1122
+ "group_size": 16
1123
+ },
1124
+ "model.language_model.layers.49.linear_attn.out_proj": {
1125
+ "quant_algo": "NVFP4",
1126
+ "group_size": 16
1127
+ },
1128
+ "model.language_model.layers.49.linear_attn.in_proj_qkv": {
1129
+ "quant_algo": "FP8"
1130
+ },
1131
+ "model.language_model.layers.49.linear_attn.in_proj_z": {
1132
+ "quant_algo": "FP8"
1133
+ },
1134
+ "model.language_model.layers.49.mlp.gate_proj": {
1135
+ "quant_algo": "NVFP4",
1136
+ "group_size": 16
1137
+ },
1138
+ "model.language_model.layers.49.mlp.up_proj": {
1139
+ "quant_algo": "NVFP4",
1140
+ "group_size": 16
1141
+ },
1142
+ "model.language_model.layers.49.mlp.down_proj": {
1143
+ "quant_algo": "NVFP4",
1144
+ "group_size": 16
1145
+ },
1146
+ "model.language_model.layers.50.linear_attn.out_proj": {
1147
+ "quant_algo": "NVFP4",
1148
+ "group_size": 16
1149
+ },
1150
+ "model.language_model.layers.50.linear_attn.in_proj_qkv": {
1151
+ "quant_algo": "FP8"
1152
+ },
1153
+ "model.language_model.layers.50.linear_attn.in_proj_z": {
1154
+ "quant_algo": "FP8"
1155
+ },
1156
+ "model.language_model.layers.50.mlp.gate_proj": {
1157
+ "quant_algo": "NVFP4",
1158
+ "group_size": 16
1159
+ },
1160
+ "model.language_model.layers.50.mlp.up_proj": {
1161
+ "quant_algo": "NVFP4",
1162
+ "group_size": 16
1163
+ },
1164
+ "model.language_model.layers.50.mlp.down_proj": {
1165
+ "quant_algo": "NVFP4",
1166
+ "group_size": 16
1167
+ },
1168
+ "model.language_model.layers.51.self_attn.q_proj": {
1169
+ "quant_algo": "FP8"
1170
+ },
1171
+ "model.language_model.layers.51.self_attn.k_proj": {
1172
+ "quant_algo": "FP8"
1173
+ },
1174
+ "model.language_model.layers.51.self_attn.v_proj": {
1175
+ "quant_algo": "FP8"
1176
+ },
1177
+ "model.language_model.layers.51.self_attn.o_proj": {
1178
+ "quant_algo": "NVFP4",
1179
+ "group_size": 16
1180
+ },
1181
+ "model.language_model.layers.51.mlp.gate_proj": {
1182
+ "quant_algo": "NVFP4",
1183
+ "group_size": 16
1184
+ },
1185
+ "model.language_model.layers.51.mlp.up_proj": {
1186
+ "quant_algo": "NVFP4",
1187
+ "group_size": 16
1188
+ },
1189
+ "model.language_model.layers.51.mlp.down_proj": {
1190
+ "quant_algo": "NVFP4",
1191
+ "group_size": 16
1192
+ },
1193
+ "model.language_model.layers.52.linear_attn.out_proj": {
1194
+ "quant_algo": "NVFP4",
1195
+ "group_size": 16
1196
+ },
1197
+ "model.language_model.layers.52.linear_attn.in_proj_qkv": {
1198
+ "quant_algo": "FP8"
1199
+ },
1200
+ "model.language_model.layers.52.linear_attn.in_proj_z": {
1201
+ "quant_algo": "FP8"
1202
+ },
1203
+ "model.language_model.layers.52.mlp.gate_proj": {
1204
+ "quant_algo": "NVFP4",
1205
+ "group_size": 16
1206
+ },
1207
+ "model.language_model.layers.52.mlp.up_proj": {
1208
+ "quant_algo": "NVFP4",
1209
+ "group_size": 16
1210
+ },
1211
+ "model.language_model.layers.52.mlp.down_proj": {
1212
+ "quant_algo": "NVFP4",
1213
+ "group_size": 16
1214
+ },
1215
+ "model.language_model.layers.53.linear_attn.out_proj": {
1216
+ "quant_algo": "NVFP4",
1217
+ "group_size": 16
1218
+ },
1219
+ "model.language_model.layers.53.linear_attn.in_proj_qkv": {
1220
+ "quant_algo": "FP8"
1221
+ },
1222
+ "model.language_model.layers.53.linear_attn.in_proj_z": {
1223
+ "quant_algo": "FP8"
1224
+ },
1225
+ "model.language_model.layers.53.mlp.gate_proj": {
1226
+ "quant_algo": "NVFP4",
1227
+ "group_size": 16
1228
+ },
1229
+ "model.language_model.layers.53.mlp.up_proj": {
1230
+ "quant_algo": "NVFP4",
1231
+ "group_size": 16
1232
+ },
1233
+ "model.language_model.layers.53.mlp.down_proj": {
1234
+ "quant_algo": "NVFP4",
1235
+ "group_size": 16
1236
+ },
1237
+ "model.language_model.layers.54.linear_attn.out_proj": {
1238
+ "quant_algo": "NVFP4",
1239
+ "group_size": 16
1240
+ },
1241
+ "model.language_model.layers.54.linear_attn.in_proj_qkv": {
1242
+ "quant_algo": "FP8"
1243
+ },
1244
+ "model.language_model.layers.54.linear_attn.in_proj_z": {
1245
+ "quant_algo": "FP8"
1246
+ },
1247
+ "model.language_model.layers.54.mlp.gate_proj": {
1248
+ "quant_algo": "NVFP4",
1249
+ "group_size": 16
1250
+ },
1251
+ "model.language_model.layers.54.mlp.up_proj": {
1252
+ "quant_algo": "NVFP4",
1253
+ "group_size": 16
1254
+ },
1255
+ "model.language_model.layers.54.mlp.down_proj": {
1256
+ "quant_algo": "NVFP4",
1257
+ "group_size": 16
1258
+ },
1259
+ "model.language_model.layers.55.self_attn.q_proj": {
1260
+ "quant_algo": "FP8"
1261
+ },
1262
+ "model.language_model.layers.55.self_attn.k_proj": {
1263
+ "quant_algo": "FP8"
1264
+ },
1265
+ "model.language_model.layers.55.self_attn.v_proj": {
1266
+ "quant_algo": "FP8"
1267
+ },
1268
+ "model.language_model.layers.55.self_attn.o_proj": {
1269
+ "quant_algo": "NVFP4",
1270
+ "group_size": 16
1271
+ },
1272
+ "model.language_model.layers.55.mlp.gate_proj": {
1273
+ "quant_algo": "NVFP4",
1274
+ "group_size": 16
1275
+ },
1276
+ "model.language_model.layers.55.mlp.up_proj": {
1277
+ "quant_algo": "NVFP4",
1278
+ "group_size": 16
1279
+ },
1280
+ "model.language_model.layers.55.mlp.down_proj": {
1281
+ "quant_algo": "NVFP4",
1282
+ "group_size": 16
1283
+ },
1284
+ "model.language_model.layers.56.linear_attn.out_proj": {
1285
+ "quant_algo": "NVFP4",
1286
+ "group_size": 16
1287
+ },
1288
+ "model.language_model.layers.56.linear_attn.in_proj_qkv": {
1289
+ "quant_algo": "FP8"
1290
+ },
1291
+ "model.language_model.layers.56.linear_attn.in_proj_z": {
1292
+ "quant_algo": "FP8"
1293
+ },
1294
+ "model.language_model.layers.56.mlp.gate_proj": {
1295
+ "quant_algo": "NVFP4",
1296
+ "group_size": 16
1297
+ },
1298
+ "model.language_model.layers.56.mlp.up_proj": {
1299
+ "quant_algo": "NVFP4",
1300
+ "group_size": 16
1301
+ },
1302
+ "model.language_model.layers.56.mlp.down_proj": {
1303
+ "quant_algo": "NVFP4",
1304
+ "group_size": 16
1305
+ },
1306
+ "model.language_model.layers.57.linear_attn.out_proj": {
1307
+ "quant_algo": "NVFP4",
1308
+ "group_size": 16
1309
+ },
1310
+ "model.language_model.layers.57.linear_attn.in_proj_qkv": {
1311
+ "quant_algo": "FP8"
1312
+ },
1313
+ "model.language_model.layers.57.linear_attn.in_proj_z": {
1314
+ "quant_algo": "FP8"
1315
+ },
1316
+ "model.language_model.layers.57.mlp.gate_proj": {
1317
+ "quant_algo": "NVFP4",
1318
+ "group_size": 16
1319
+ },
1320
+ "model.language_model.layers.57.mlp.up_proj": {
1321
+ "quant_algo": "NVFP4",
1322
+ "group_size": 16
1323
+ },
1324
+ "model.language_model.layers.57.mlp.down_proj": {
1325
+ "quant_algo": "NVFP4",
1326
+ "group_size": 16
1327
+ },
1328
+ "model.language_model.layers.58.linear_attn.out_proj": {
1329
+ "quant_algo": "NVFP4",
1330
+ "group_size": 16
1331
+ },
1332
+ "model.language_model.layers.58.linear_attn.in_proj_qkv": {
1333
+ "quant_algo": "FP8"
1334
+ },
1335
+ "model.language_model.layers.58.linear_attn.in_proj_z": {
1336
+ "quant_algo": "FP8"
1337
+ },
1338
+ "model.language_model.layers.58.mlp.gate_proj": {
1339
+ "quant_algo": "NVFP4",
1340
+ "group_size": 16
1341
+ },
1342
+ "model.language_model.layers.58.mlp.up_proj": {
1343
+ "quant_algo": "NVFP4",
1344
+ "group_size": 16
1345
+ },
1346
+ "model.language_model.layers.58.mlp.down_proj": {
1347
+ "quant_algo": "NVFP4",
1348
+ "group_size": 16
1349
+ },
1350
+ "model.language_model.layers.59.self_attn.q_proj": {
1351
+ "quant_algo": "FP8"
1352
+ },
1353
+ "model.language_model.layers.59.self_attn.k_proj": {
1354
+ "quant_algo": "FP8"
1355
+ },
1356
+ "model.language_model.layers.59.self_attn.v_proj": {
1357
+ "quant_algo": "FP8"
1358
+ },
1359
+ "model.language_model.layers.59.self_attn.o_proj": {
1360
+ "quant_algo": "NVFP4",
1361
+ "group_size": 16
1362
+ },
1363
+ "model.language_model.layers.59.mlp.gate_proj": {
1364
+ "quant_algo": "NVFP4",
1365
+ "group_size": 16
1366
+ },
1367
+ "model.language_model.layers.59.mlp.up_proj": {
1368
+ "quant_algo": "NVFP4",
1369
+ "group_size": 16
1370
+ },
1371
+ "model.language_model.layers.59.mlp.down_proj": {
1372
+ "quant_algo": "NVFP4",
1373
+ "group_size": 16
1374
+ },
1375
+ "model.language_model.layers.60.linear_attn.out_proj": {
1376
+ "quant_algo": "NVFP4",
1377
+ "group_size": 16
1378
+ },
1379
+ "model.language_model.layers.60.linear_attn.in_proj_qkv": {
1380
+ "quant_algo": "FP8"
1381
+ },
1382
+ "model.language_model.layers.60.linear_attn.in_proj_z": {
1383
+ "quant_algo": "FP8"
1384
+ },
1385
+ "model.language_model.layers.60.mlp.gate_proj": {
1386
+ "quant_algo": "NVFP4",
1387
+ "group_size": 16
1388
+ },
1389
+ "model.language_model.layers.60.mlp.up_proj": {
1390
+ "quant_algo": "NVFP4",
1391
+ "group_size": 16
1392
+ },
1393
+ "model.language_model.layers.60.mlp.down_proj": {
1394
+ "quant_algo": "NVFP4",
1395
+ "group_size": 16
1396
+ },
1397
+ "model.language_model.layers.61.linear_attn.out_proj": {
1398
+ "quant_algo": "NVFP4",
1399
+ "group_size": 16
1400
+ },
1401
+ "model.language_model.layers.61.linear_attn.in_proj_qkv": {
1402
+ "quant_algo": "FP8"
1403
+ },
1404
+ "model.language_model.layers.61.linear_attn.in_proj_z": {
1405
+ "quant_algo": "FP8"
1406
+ },
1407
+ "model.language_model.layers.61.mlp.gate_proj": {
1408
+ "quant_algo": "NVFP4",
1409
+ "group_size": 16
1410
+ },
1411
+ "model.language_model.layers.61.mlp.up_proj": {
1412
+ "quant_algo": "NVFP4",
1413
+ "group_size": 16
1414
+ },
1415
+ "model.language_model.layers.61.mlp.down_proj": {
1416
+ "quant_algo": "NVFP4",
1417
+ "group_size": 16
1418
+ },
1419
+ "model.language_model.layers.62.linear_attn.out_proj": {
1420
+ "quant_algo": "NVFP4",
1421
+ "group_size": 16
1422
+ },
1423
+ "model.language_model.layers.62.linear_attn.in_proj_qkv": {
1424
+ "quant_algo": "FP8"
1425
+ },
1426
+ "model.language_model.layers.62.linear_attn.in_proj_z": {
1427
+ "quant_algo": "FP8"
1428
+ },
1429
+ "model.language_model.layers.62.mlp.gate_proj": {
1430
+ "quant_algo": "NVFP4",
1431
+ "group_size": 16
1432
+ },
1433
+ "model.language_model.layers.62.mlp.up_proj": {
1434
+ "quant_algo": "NVFP4",
1435
+ "group_size": 16
1436
+ },
1437
+ "model.language_model.layers.62.mlp.down_proj": {
1438
+ "quant_algo": "NVFP4",
1439
+ "group_size": 16
1440
+ },
1441
+ "model.language_model.layers.63.self_attn.q_proj": {
1442
+ "quant_algo": "FP8"
1443
+ },
1444
+ "model.language_model.layers.63.self_attn.k_proj": {
1445
+ "quant_algo": "FP8"
1446
+ },
1447
+ "model.language_model.layers.63.self_attn.v_proj": {
1448
+ "quant_algo": "FP8"
1449
+ },
1450
+ "model.language_model.layers.63.self_attn.o_proj": {
1451
+ "quant_algo": "NVFP4",
1452
+ "group_size": 16
1453
+ },
1454
+ "model.language_model.layers.63.mlp.gate_proj": {
1455
+ "quant_algo": "NVFP4",
1456
+ "group_size": 16
1457
+ },
1458
+ "model.language_model.layers.63.mlp.up_proj": {
1459
+ "quant_algo": "NVFP4",
1460
+ "group_size": 16
1461
+ },
1462
+ "model.language_model.layers.63.mlp.down_proj": {
1463
+ "quant_algo": "NVFP4",
1464
+ "group_size": 16
1465
+ },
1466
+ "model.language_model.layers.64.linear_attn.out_proj": {
1467
+ "quant_algo": "NVFP4",
1468
+ "group_size": 16
1469
+ },
1470
+ "model.language_model.layers.64.linear_attn.in_proj_qkv": {
1471
+ "quant_algo": "FP8"
1472
+ },
1473
+ "model.language_model.layers.64.linear_attn.in_proj_z": {
1474
+ "quant_algo": "FP8"
1475
+ },
1476
+ "model.language_model.layers.64.mlp.gate_proj": {
1477
+ "quant_algo": "NVFP4",
1478
+ "group_size": 16
1479
+ },
1480
+ "model.language_model.layers.64.mlp.up_proj": {
1481
+ "quant_algo": "NVFP4",
1482
+ "group_size": 16
1483
+ },
1484
+ "model.language_model.layers.64.mlp.down_proj": {
1485
+ "quant_algo": "NVFP4",
1486
+ "group_size": 16
1487
+ },
1488
+ "model.language_model.layers.65.linear_attn.out_proj": {
1489
+ "quant_algo": "NVFP4",
1490
+ "group_size": 16
1491
+ },
1492
+ "model.language_model.layers.65.linear_attn.in_proj_qkv": {
1493
+ "quant_algo": "FP8"
1494
+ },
1495
+ "model.language_model.layers.65.linear_attn.in_proj_z": {
1496
+ "quant_algo": "FP8"
1497
+ },
1498
+ "model.language_model.layers.65.mlp.gate_proj": {
1499
+ "quant_algo": "NVFP4",
1500
+ "group_size": 16
1501
+ },
1502
+ "model.language_model.layers.65.mlp.up_proj": {
1503
+ "quant_algo": "NVFP4",
1504
+ "group_size": 16
1505
+ },
1506
+ "model.language_model.layers.65.mlp.down_proj": {
1507
+ "quant_algo": "NVFP4",
1508
+ "group_size": 16
1509
+ },
1510
+ "model.language_model.layers.66.linear_attn.out_proj": {
1511
+ "quant_algo": "NVFP4",
1512
+ "group_size": 16
1513
+ },
1514
+ "model.language_model.layers.66.linear_attn.in_proj_qkv": {
1515
+ "quant_algo": "FP8"
1516
+ },
1517
+ "model.language_model.layers.66.linear_attn.in_proj_z": {
1518
+ "quant_algo": "FP8"
1519
+ },
1520
+ "model.language_model.layers.66.mlp.gate_proj": {
1521
+ "quant_algo": "NVFP4",
1522
+ "group_size": 16
1523
+ },
1524
+ "model.language_model.layers.66.mlp.up_proj": {
1525
+ "quant_algo": "NVFP4",
1526
+ "group_size": 16
1527
+ },
1528
+ "model.language_model.layers.66.mlp.down_proj": {
1529
+ "quant_algo": "NVFP4",
1530
+ "group_size": 16
1531
+ },
1532
+ "model.language_model.layers.67.self_attn.q_proj": {
1533
+ "quant_algo": "FP8"
1534
+ },
1535
+ "model.language_model.layers.67.self_attn.k_proj": {
1536
+ "quant_algo": "FP8"
1537
+ },
1538
+ "model.language_model.layers.67.self_attn.v_proj": {
1539
+ "quant_algo": "FP8"
1540
+ },
1541
+ "model.language_model.layers.67.self_attn.o_proj": {
1542
+ "quant_algo": "NVFP4",
1543
+ "group_size": 16
1544
+ },
1545
+ "model.language_model.layers.67.mlp.gate_proj": {
1546
+ "quant_algo": "NVFP4",
1547
+ "group_size": 16
1548
+ },
1549
+ "model.language_model.layers.67.mlp.up_proj": {
1550
+ "quant_algo": "NVFP4",
1551
+ "group_size": 16
1552
+ },
1553
+ "model.language_model.layers.67.mlp.down_proj": {
1554
+ "quant_algo": "NVFP4",
1555
+ "group_size": 16
1556
+ },
1557
+ "model.language_model.layers.68.linear_attn.out_proj": {
1558
+ "quant_algo": "NVFP4",
1559
+ "group_size": 16
1560
+ },
1561
+ "model.language_model.layers.68.linear_attn.in_proj_qkv": {
1562
+ "quant_algo": "FP8"
1563
+ },
1564
+ "model.language_model.layers.68.linear_attn.in_proj_z": {
1565
+ "quant_algo": "FP8"
1566
+ },
1567
+ "model.language_model.layers.68.mlp.gate_proj": {
1568
+ "quant_algo": "NVFP4",
1569
+ "group_size": 16
1570
+ },
1571
+ "model.language_model.layers.68.mlp.up_proj": {
1572
+ "quant_algo": "NVFP4",
1573
+ "group_size": 16
1574
+ },
1575
+ "model.language_model.layers.68.mlp.down_proj": {
1576
+ "quant_algo": "NVFP4",
1577
+ "group_size": 16
1578
+ },
1579
+ "model.language_model.layers.69.linear_attn.out_proj": {
1580
+ "quant_algo": "NVFP4",
1581
+ "group_size": 16
1582
+ },
1583
+ "model.language_model.layers.69.linear_attn.in_proj_qkv": {
1584
+ "quant_algo": "FP8"
1585
+ },
1586
+ "model.language_model.layers.69.linear_attn.in_proj_z": {
1587
+ "quant_algo": "FP8"
1588
+ },
1589
+ "model.language_model.layers.69.mlp.gate_proj": {
1590
+ "quant_algo": "NVFP4",
1591
+ "group_size": 16
1592
+ },
1593
+ "model.language_model.layers.69.mlp.up_proj": {
1594
+ "quant_algo": "NVFP4",
1595
+ "group_size": 16
1596
+ },
1597
+ "model.language_model.layers.69.mlp.down_proj": {
1598
+ "quant_algo": "NVFP4",
1599
+ "group_size": 16
1600
+ },
1601
+ "model.language_model.layers.70.linear_attn.out_proj": {
1602
+ "quant_algo": "NVFP4",
1603
+ "group_size": 16
1604
+ },
1605
+ "model.language_model.layers.70.linear_attn.in_proj_qkv": {
1606
+ "quant_algo": "FP8"
1607
+ },
1608
+ "model.language_model.layers.70.linear_attn.in_proj_z": {
1609
+ "quant_algo": "FP8"
1610
+ },
1611
+ "model.language_model.layers.70.mlp.gate_proj": {
1612
+ "quant_algo": "NVFP4",
1613
+ "group_size": 16
1614
+ },
1615
+ "model.language_model.layers.70.mlp.up_proj": {
1616
+ "quant_algo": "NVFP4",
1617
+ "group_size": 16
1618
+ },
1619
+ "model.language_model.layers.70.mlp.down_proj": {
1620
+ "quant_algo": "NVFP4",
1621
+ "group_size": 16
1622
+ },
1623
+ "model.language_model.layers.71.self_attn.q_proj": {
1624
+ "quant_algo": "FP8"
1625
+ },
1626
+ "model.language_model.layers.71.self_attn.k_proj": {
1627
+ "quant_algo": "FP8"
1628
+ },
1629
+ "model.language_model.layers.71.self_attn.v_proj": {
1630
+ "quant_algo": "FP8"
1631
+ },
1632
+ "model.language_model.layers.71.self_attn.o_proj": {
1633
+ "quant_algo": "NVFP4",
1634
+ "group_size": 16
1635
+ },
1636
+ "model.language_model.layers.71.mlp.gate_proj": {
1637
+ "quant_algo": "NVFP4",
1638
+ "group_size": 16
1639
+ },
1640
+ "model.language_model.layers.71.mlp.up_proj": {
1641
+ "quant_algo": "NVFP4",
1642
+ "group_size": 16
1643
+ },
1644
+ "model.language_model.layers.71.mlp.down_proj": {
1645
+ "quant_algo": "NVFP4",
1646
+ "group_size": 16
1647
+ },
1648
+ "lm_head": {
1649
+ "quant_algo": "FP8"
1650
+ },
1651
+ "model.language_model.embed_tokens": {
1652
+ "quant_algo": "FP8_ROW"
1653
+ },
1654
+ "mtp.fc": {
1655
+ "quant_algo": "FP8_DYNAMIC"
1656
+ },
1657
+ "mtp.layers.0.self_attn.k_proj": {
1658
+ "quant_algo": "FP8_DYNAMIC"
1659
+ },
1660
+ "mtp.layers.0.self_attn.o_proj": {
1661
+ "quant_algo": "FP8_DYNAMIC"
1662
+ },
1663
+ "mtp.layers.0.self_attn.q_proj": {
1664
+ "quant_algo": "FP8_DYNAMIC"
1665
+ },
1666
+ "mtp.layers.0.self_attn.v_proj": {
1667
+ "quant_algo": "FP8_DYNAMIC"
1668
+ },
1669
+ "mtp.layers.0.mlp.down_proj": {
1670
+ "quant_algo": "FP8_DYNAMIC"
1671
+ },
1672
+ "mtp.layers.0.mlp.gate_proj": {
1673
+ "quant_algo": "FP8_DYNAMIC"
1674
+ },
1675
+ "mtp.layers.0.mlp.up_proj": {
1676
+ "quant_algo": "FP8_DYNAMIC"
1677
+ },
1678
+ "mtp.embed_tokens": {
1679
+ "quant_algo": "FP8_ROW"
1680
+ },
1681
+ "mtp.layers.0.self_attn.qkv_proj": {
1682
+ "quant_algo": "FP8_DYNAMIC"
1683
+ },
1684
+ "mtp.layers.0.mlp.gate_up_proj": {
1685
+ "quant_algo": "FP8_DYNAMIC"
1686
+ },
1687
+ "model.visual.blocks.0.attn.proj": {
1688
+ "quant_algo": "FP8_DYNAMIC"
1689
+ },
1690
+ "model.visual.blocks.0.attn.qkv": {
1691
+ "quant_algo": "FP8_DYNAMIC"
1692
+ },
1693
+ "model.visual.blocks.0.attn.qkv_proj": {
1694
+ "quant_algo": "FP8_DYNAMIC"
1695
+ },
1696
+ "model.visual.blocks.0.mlp.linear_fc1": {
1697
+ "quant_algo": "FP8_DYNAMIC"
1698
+ },
1699
+ "model.visual.blocks.0.mlp.linear_fc2": {
1700
+ "quant_algo": "FP8_DYNAMIC"
1701
+ },
1702
+ "model.visual.blocks.1.attn.proj": {
1703
+ "quant_algo": "FP8_DYNAMIC"
1704
+ },
1705
+ "model.visual.blocks.1.attn.qkv": {
1706
+ "quant_algo": "FP8_DYNAMIC"
1707
+ },
1708
+ "model.visual.blocks.1.attn.qkv_proj": {
1709
+ "quant_algo": "FP8_DYNAMIC"
1710
+ },
1711
+ "model.visual.blocks.1.mlp.linear_fc1": {
1712
+ "quant_algo": "FP8_DYNAMIC"
1713
+ },
1714
+ "model.visual.blocks.1.mlp.linear_fc2": {
1715
+ "quant_algo": "FP8_DYNAMIC"
1716
+ },
1717
+ "model.visual.blocks.10.attn.proj": {
1718
+ "quant_algo": "FP8_DYNAMIC"
1719
+ },
1720
+ "model.visual.blocks.10.attn.qkv": {
1721
+ "quant_algo": "FP8_DYNAMIC"
1722
+ },
1723
+ "model.visual.blocks.10.attn.qkv_proj": {
1724
+ "quant_algo": "FP8_DYNAMIC"
1725
+ },
1726
+ "model.visual.blocks.10.mlp.linear_fc1": {
1727
+ "quant_algo": "FP8_DYNAMIC"
1728
+ },
1729
+ "model.visual.blocks.10.mlp.linear_fc2": {
1730
+ "quant_algo": "FP8_DYNAMIC"
1731
+ },
1732
+ "model.visual.blocks.11.attn.proj": {
1733
+ "quant_algo": "FP8_DYNAMIC"
1734
+ },
1735
+ "model.visual.blocks.11.attn.qkv": {
1736
+ "quant_algo": "FP8_DYNAMIC"
1737
+ },
1738
+ "model.visual.blocks.11.attn.qkv_proj": {
1739
+ "quant_algo": "FP8_DYNAMIC"
1740
+ },
1741
+ "model.visual.blocks.11.mlp.linear_fc1": {
1742
+ "quant_algo": "FP8_DYNAMIC"
1743
+ },
1744
+ "model.visual.blocks.11.mlp.linear_fc2": {
1745
+ "quant_algo": "FP8_DYNAMIC"
1746
+ },
1747
+ "model.visual.blocks.12.attn.proj": {
1748
+ "quant_algo": "FP8_DYNAMIC"
1749
+ },
1750
+ "model.visual.blocks.12.attn.qkv": {
1751
+ "quant_algo": "FP8_DYNAMIC"
1752
+ },
1753
+ "model.visual.blocks.12.attn.qkv_proj": {
1754
+ "quant_algo": "FP8_DYNAMIC"
1755
+ },
1756
+ "model.visual.blocks.12.mlp.linear_fc1": {
1757
+ "quant_algo": "FP8_DYNAMIC"
1758
+ },
1759
+ "model.visual.blocks.12.mlp.linear_fc2": {
1760
+ "quant_algo": "FP8_DYNAMIC"
1761
+ },
1762
+ "model.visual.blocks.13.attn.proj": {
1763
+ "quant_algo": "FP8_DYNAMIC"
1764
+ },
1765
+ "model.visual.blocks.13.attn.qkv": {
1766
+ "quant_algo": "FP8_DYNAMIC"
1767
+ },
1768
+ "model.visual.blocks.13.attn.qkv_proj": {
1769
+ "quant_algo": "FP8_DYNAMIC"
1770
+ },
1771
+ "model.visual.blocks.13.mlp.linear_fc1": {
1772
+ "quant_algo": "FP8_DYNAMIC"
1773
+ },
1774
+ "model.visual.blocks.13.mlp.linear_fc2": {
1775
+ "quant_algo": "FP8_DYNAMIC"
1776
+ },
1777
+ "model.visual.blocks.14.attn.proj": {
1778
+ "quant_algo": "FP8_DYNAMIC"
1779
+ },
1780
+ "model.visual.blocks.14.attn.qkv": {
1781
+ "quant_algo": "FP8_DYNAMIC"
1782
+ },
1783
+ "model.visual.blocks.14.attn.qkv_proj": {
1784
+ "quant_algo": "FP8_DYNAMIC"
1785
+ },
1786
+ "model.visual.blocks.14.mlp.linear_fc1": {
1787
+ "quant_algo": "FP8_DYNAMIC"
1788
+ },
1789
+ "model.visual.blocks.14.mlp.linear_fc2": {
1790
+ "quant_algo": "FP8_DYNAMIC"
1791
+ },
1792
+ "model.visual.blocks.15.attn.proj": {
1793
+ "quant_algo": "FP8_DYNAMIC"
1794
+ },
1795
+ "model.visual.blocks.15.attn.qkv": {
1796
+ "quant_algo": "FP8_DYNAMIC"
1797
+ },
1798
+ "model.visual.blocks.15.attn.qkv_proj": {
1799
+ "quant_algo": "FP8_DYNAMIC"
1800
+ },
1801
+ "model.visual.blocks.15.mlp.linear_fc1": {
1802
+ "quant_algo": "FP8_DYNAMIC"
1803
+ },
1804
+ "model.visual.blocks.15.mlp.linear_fc2": {
1805
+ "quant_algo": "FP8_DYNAMIC"
1806
+ },
1807
+ "model.visual.blocks.16.attn.proj": {
1808
+ "quant_algo": "FP8_DYNAMIC"
1809
+ },
1810
+ "model.visual.blocks.16.attn.qkv": {
1811
+ "quant_algo": "FP8_DYNAMIC"
1812
+ },
1813
+ "model.visual.blocks.16.attn.qkv_proj": {
1814
+ "quant_algo": "FP8_DYNAMIC"
1815
+ },
1816
+ "model.visual.blocks.16.mlp.linear_fc1": {
1817
+ "quant_algo": "FP8_DYNAMIC"
1818
+ },
1819
+ "model.visual.blocks.16.mlp.linear_fc2": {
1820
+ "quant_algo": "FP8_DYNAMIC"
1821
+ },
1822
+ "model.visual.blocks.17.attn.proj": {
1823
+ "quant_algo": "FP8_DYNAMIC"
1824
+ },
1825
+ "model.visual.blocks.17.attn.qkv": {
1826
+ "quant_algo": "FP8_DYNAMIC"
1827
+ },
1828
+ "model.visual.blocks.17.attn.qkv_proj": {
1829
+ "quant_algo": "FP8_DYNAMIC"
1830
+ },
1831
+ "model.visual.blocks.17.mlp.linear_fc1": {
1832
+ "quant_algo": "FP8_DYNAMIC"
1833
+ },
1834
+ "model.visual.blocks.17.mlp.linear_fc2": {
1835
+ "quant_algo": "FP8_DYNAMIC"
1836
+ },
1837
+ "model.visual.blocks.18.attn.proj": {
1838
+ "quant_algo": "FP8_DYNAMIC"
1839
+ },
1840
+ "model.visual.blocks.18.attn.qkv": {
1841
+ "quant_algo": "FP8_DYNAMIC"
1842
+ },
1843
+ "model.visual.blocks.18.attn.qkv_proj": {
1844
+ "quant_algo": "FP8_DYNAMIC"
1845
+ },
1846
+ "model.visual.blocks.18.mlp.linear_fc1": {
1847
+ "quant_algo": "FP8_DYNAMIC"
1848
+ },
1849
+ "model.visual.blocks.18.mlp.linear_fc2": {
1850
+ "quant_algo": "FP8_DYNAMIC"
1851
+ },
1852
+ "model.visual.blocks.19.attn.proj": {
1853
+ "quant_algo": "FP8_DYNAMIC"
1854
+ },
1855
+ "model.visual.blocks.19.attn.qkv": {
1856
+ "quant_algo": "FP8_DYNAMIC"
1857
+ },
1858
+ "model.visual.blocks.19.attn.qkv_proj": {
1859
+ "quant_algo": "FP8_DYNAMIC"
1860
+ },
1861
+ "model.visual.blocks.19.mlp.linear_fc1": {
1862
+ "quant_algo": "FP8_DYNAMIC"
1863
+ },
1864
+ "model.visual.blocks.19.mlp.linear_fc2": {
1865
+ "quant_algo": "FP8_DYNAMIC"
1866
+ },
1867
+ "model.visual.blocks.2.attn.proj": {
1868
+ "quant_algo": "FP8_DYNAMIC"
1869
+ },
1870
+ "model.visual.blocks.2.attn.qkv": {
1871
+ "quant_algo": "FP8_DYNAMIC"
1872
+ },
1873
+ "model.visual.blocks.2.attn.qkv_proj": {
1874
+ "quant_algo": "FP8_DYNAMIC"
1875
+ },
1876
+ "model.visual.blocks.2.mlp.linear_fc1": {
1877
+ "quant_algo": "FP8_DYNAMIC"
1878
+ },
1879
+ "model.visual.blocks.2.mlp.linear_fc2": {
1880
+ "quant_algo": "FP8_DYNAMIC"
1881
+ },
1882
+ "model.visual.blocks.20.attn.proj": {
1883
+ "quant_algo": "FP8_DYNAMIC"
1884
+ },
1885
+ "model.visual.blocks.20.attn.qkv": {
1886
+ "quant_algo": "FP8_DYNAMIC"
1887
+ },
1888
+ "model.visual.blocks.20.attn.qkv_proj": {
1889
+ "quant_algo": "FP8_DYNAMIC"
1890
+ },
1891
+ "model.visual.blocks.20.mlp.linear_fc1": {
1892
+ "quant_algo": "FP8_DYNAMIC"
1893
+ },
1894
+ "model.visual.blocks.20.mlp.linear_fc2": {
1895
+ "quant_algo": "FP8_DYNAMIC"
1896
+ },
1897
+ "model.visual.blocks.21.attn.proj": {
1898
+ "quant_algo": "FP8_DYNAMIC"
1899
+ },
1900
+ "model.visual.blocks.21.attn.qkv": {
1901
+ "quant_algo": "FP8_DYNAMIC"
1902
+ },
1903
+ "model.visual.blocks.21.attn.qkv_proj": {
1904
+ "quant_algo": "FP8_DYNAMIC"
1905
+ },
1906
+ "model.visual.blocks.21.mlp.linear_fc1": {
1907
+ "quant_algo": "FP8_DYNAMIC"
1908
+ },
1909
+ "model.visual.blocks.21.mlp.linear_fc2": {
1910
+ "quant_algo": "FP8_DYNAMIC"
1911
+ },
1912
+ "model.visual.blocks.22.attn.proj": {
1913
+ "quant_algo": "FP8_DYNAMIC"
1914
+ },
1915
+ "model.visual.blocks.22.attn.qkv": {
1916
+ "quant_algo": "FP8_DYNAMIC"
1917
+ },
1918
+ "model.visual.blocks.22.attn.qkv_proj": {
1919
+ "quant_algo": "FP8_DYNAMIC"
1920
+ },
1921
+ "model.visual.blocks.22.mlp.linear_fc1": {
1922
+ "quant_algo": "FP8_DYNAMIC"
1923
+ },
1924
+ "model.visual.blocks.22.mlp.linear_fc2": {
1925
+ "quant_algo": "FP8_DYNAMIC"
1926
+ },
1927
+ "model.visual.blocks.23.attn.proj": {
1928
+ "quant_algo": "FP8_DYNAMIC"
1929
+ },
1930
+ "model.visual.blocks.23.attn.qkv": {
1931
+ "quant_algo": "FP8_DYNAMIC"
1932
+ },
1933
+ "model.visual.blocks.23.attn.qkv_proj": {
1934
+ "quant_algo": "FP8_DYNAMIC"
1935
+ },
1936
+ "model.visual.blocks.23.mlp.linear_fc1": {
1937
+ "quant_algo": "FP8_DYNAMIC"
1938
+ },
1939
+ "model.visual.blocks.23.mlp.linear_fc2": {
1940
+ "quant_algo": "FP8_DYNAMIC"
1941
+ },
1942
+ "model.visual.blocks.24.attn.proj": {
1943
+ "quant_algo": "FP8_DYNAMIC"
1944
+ },
1945
+ "model.visual.blocks.24.attn.qkv": {
1946
+ "quant_algo": "FP8_DYNAMIC"
1947
+ },
1948
+ "model.visual.blocks.24.attn.qkv_proj": {
1949
+ "quant_algo": "FP8_DYNAMIC"
1950
+ },
1951
+ "model.visual.blocks.24.mlp.linear_fc1": {
1952
+ "quant_algo": "FP8_DYNAMIC"
1953
+ },
1954
+ "model.visual.blocks.24.mlp.linear_fc2": {
1955
+ "quant_algo": "FP8_DYNAMIC"
1956
+ },
1957
+ "model.visual.blocks.25.attn.proj": {
1958
+ "quant_algo": "FP8_DYNAMIC"
1959
+ },
1960
+ "model.visual.blocks.25.attn.qkv": {
1961
+ "quant_algo": "FP8_DYNAMIC"
1962
+ },
1963
+ "model.visual.blocks.25.attn.qkv_proj": {
1964
+ "quant_algo": "FP8_DYNAMIC"
1965
+ },
1966
+ "model.visual.blocks.25.mlp.linear_fc1": {
1967
+ "quant_algo": "FP8_DYNAMIC"
1968
+ },
1969
+ "model.visual.blocks.25.mlp.linear_fc2": {
1970
+ "quant_algo": "FP8_DYNAMIC"
1971
+ },
1972
+ "model.visual.blocks.26.attn.proj": {
1973
+ "quant_algo": "FP8_DYNAMIC"
1974
+ },
1975
+ "model.visual.blocks.26.attn.qkv": {
1976
+ "quant_algo": "FP8_DYNAMIC"
1977
+ },
1978
+ "model.visual.blocks.26.attn.qkv_proj": {
1979
+ "quant_algo": "FP8_DYNAMIC"
1980
+ },
1981
+ "model.visual.blocks.26.mlp.linear_fc1": {
1982
+ "quant_algo": "FP8_DYNAMIC"
1983
+ },
1984
+ "model.visual.blocks.26.mlp.linear_fc2": {
1985
+ "quant_algo": "FP8_DYNAMIC"
1986
+ },
1987
+ "model.visual.blocks.3.attn.proj": {
1988
+ "quant_algo": "FP8_DYNAMIC"
1989
+ },
1990
+ "model.visual.blocks.3.attn.qkv": {
1991
+ "quant_algo": "FP8_DYNAMIC"
1992
+ },
1993
+ "model.visual.blocks.3.attn.qkv_proj": {
1994
+ "quant_algo": "FP8_DYNAMIC"
1995
+ },
1996
+ "model.visual.blocks.3.mlp.linear_fc1": {
1997
+ "quant_algo": "FP8_DYNAMIC"
1998
+ },
1999
+ "model.visual.blocks.3.mlp.linear_fc2": {
2000
+ "quant_algo": "FP8_DYNAMIC"
2001
+ },
2002
+ "model.visual.blocks.4.attn.proj": {
2003
+ "quant_algo": "FP8_DYNAMIC"
2004
+ },
2005
+ "model.visual.blocks.4.attn.qkv": {
2006
+ "quant_algo": "FP8_DYNAMIC"
2007
+ },
2008
+ "model.visual.blocks.4.attn.qkv_proj": {
2009
+ "quant_algo": "FP8_DYNAMIC"
2010
+ },
2011
+ "model.visual.blocks.4.mlp.linear_fc1": {
2012
+ "quant_algo": "FP8_DYNAMIC"
2013
+ },
2014
+ "model.visual.blocks.4.mlp.linear_fc2": {
2015
+ "quant_algo": "FP8_DYNAMIC"
2016
+ },
2017
+ "model.visual.blocks.5.attn.proj": {
2018
+ "quant_algo": "FP8_DYNAMIC"
2019
+ },
2020
+ "model.visual.blocks.5.attn.qkv": {
2021
+ "quant_algo": "FP8_DYNAMIC"
2022
+ },
2023
+ "model.visual.blocks.5.attn.qkv_proj": {
2024
+ "quant_algo": "FP8_DYNAMIC"
2025
+ },
2026
+ "model.visual.blocks.5.mlp.linear_fc1": {
2027
+ "quant_algo": "FP8_DYNAMIC"
2028
+ },
2029
+ "model.visual.blocks.5.mlp.linear_fc2": {
2030
+ "quant_algo": "FP8_DYNAMIC"
2031
+ },
2032
+ "model.visual.blocks.6.attn.proj": {
2033
+ "quant_algo": "FP8_DYNAMIC"
2034
+ },
2035
+ "model.visual.blocks.6.attn.qkv": {
2036
+ "quant_algo": "FP8_DYNAMIC"
2037
+ },
2038
+ "model.visual.blocks.6.attn.qkv_proj": {
2039
+ "quant_algo": "FP8_DYNAMIC"
2040
+ },
2041
+ "model.visual.blocks.6.mlp.linear_fc1": {
2042
+ "quant_algo": "FP8_DYNAMIC"
2043
+ },
2044
+ "model.visual.blocks.6.mlp.linear_fc2": {
2045
+ "quant_algo": "FP8_DYNAMIC"
2046
+ },
2047
+ "model.visual.blocks.7.attn.proj": {
2048
+ "quant_algo": "FP8_DYNAMIC"
2049
+ },
2050
+ "model.visual.blocks.7.attn.qkv": {
2051
+ "quant_algo": "FP8_DYNAMIC"
2052
+ },
2053
+ "model.visual.blocks.7.attn.qkv_proj": {
2054
+ "quant_algo": "FP8_DYNAMIC"
2055
+ },
2056
+ "model.visual.blocks.7.mlp.linear_fc1": {
2057
+ "quant_algo": "FP8_DYNAMIC"
2058
+ },
2059
+ "model.visual.blocks.7.mlp.linear_fc2": {
2060
+ "quant_algo": "FP8_DYNAMIC"
2061
+ },
2062
+ "model.visual.blocks.8.attn.proj": {
2063
+ "quant_algo": "FP8_DYNAMIC"
2064
+ },
2065
+ "model.visual.blocks.8.attn.qkv": {
2066
+ "quant_algo": "FP8_DYNAMIC"
2067
+ },
2068
+ "model.visual.blocks.8.attn.qkv_proj": {
2069
+ "quant_algo": "FP8_DYNAMIC"
2070
+ },
2071
+ "model.visual.blocks.8.mlp.linear_fc1": {
2072
+ "quant_algo": "FP8_DYNAMIC"
2073
+ },
2074
+ "model.visual.blocks.8.mlp.linear_fc2": {
2075
+ "quant_algo": "FP8_DYNAMIC"
2076
+ },
2077
+ "model.visual.blocks.9.attn.proj": {
2078
+ "quant_algo": "FP8_DYNAMIC"
2079
+ },
2080
+ "model.visual.blocks.9.attn.qkv": {
2081
+ "quant_algo": "FP8_DYNAMIC"
2082
+ },
2083
+ "model.visual.blocks.9.attn.qkv_proj": {
2084
+ "quant_algo": "FP8_DYNAMIC"
2085
+ },
2086
+ "model.visual.blocks.9.mlp.linear_fc1": {
2087
+ "quant_algo": "FP8_DYNAMIC"
2088
+ },
2089
+ "model.visual.blocks.9.mlp.linear_fc2": {
2090
+ "quant_algo": "FP8_DYNAMIC"
2091
+ },
2092
+ "model.visual.merger.linear_fc1": {
2093
+ "quant_algo": "FP8_DYNAMIC"
2094
+ },
2095
+ "model.visual.merger.linear_fc2": {
2096
+ "quant_algo": "FP8_DYNAMIC"
2097
+ },
2098
+ "model.visual.pos_embed": {
2099
+ "quant_algo": "FP8_ROW"
2100
+ },
2101
+ "model.visual.patch_embed.proj": {
2102
+ "quant_algo": "FP8_DYNAMIC"
2103
+ }
2104
+ }
2105
+ },
2106
+ "packed_modules_mapping": {
2107
+ "gate_up_proj": [
2108
+ "gate_proj",
2109
+ "up_proj"
2110
+ ],
2111
+ "in_proj_qkvz": [
2112
+ "in_proj_qkv",
2113
+ "in_proj_z"
2114
+ ],
2115
+ "in_proj_ba": [
2116
+ "in_proj_b",
2117
+ "in_proj_a"
2118
+ ]
2119
+ }
2120
+ }