Hugging Face's logo

XenonFear128
/
Agnes-3.0-Flash-32GB-NVFP4-k2

Image-Text-to-Text
Safetensors
modelopt
agnes
quantized
nvfp4
fp8
custom_code
tool-calling
conversational
8-bit precision
modelopt_mixed
Model card Files
xet
 Community
XenonFear128 commited on
Commit
419ebe0
·
verified ·
1 Parent(s): c31f81f

Upload config.json with huggingface_hub

Browse files
Files changed (1) hide show
  1. config.json +2254 -0
config.json ADDED
@@ -0,0 +1,2254 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "agnes",
3
+ "architectures": [
4
+ "AgnesForConditionalGeneration"
5
+ ],
6
+ "auto_map": {
7
+ "AutoConfig": "configuration_agnes.AgnesConfig",
8
+ "AutoModel": "modeling_agnes.AgnesModel",
9
+ "AutoModelForCausalLM": "modeling_agnes.AgnesForConditionalGeneration",
10
+ "AutoModelForImageTextToText": "modeling_agnes.AgnesForConditionalGeneration"
11
+ },
12
+ "text_config": {
13
+ "model_type": "agnes_text",
14
+ "hidden_size": 5120,
15
+ "num_hidden_layers": 72,
16
+ "intermediate_size": 19456,
17
+ "parallel_ffn_intermediate_size": 0,
18
+ "vocab_size": 248320,
19
+ "layer_types": [
20
+ "agnes_delta_attention",
21
+ "agnes_delta_attention",
22
+ "agnes_delta_attention",
23
+ "agnes_global_attention",
24
+ "agnes_delta_attention",
25
+ "agnes_delta_attention",
26
+ "agnes_delta_attention",
27
+ "agnes_global_attention",
28
+ "agnes_delta_attention",
29
+ "agnes_delta_attention",
30
+ "agnes_delta_attention",
31
+ "agnes_global_attention",
32
+ "agnes_delta_attention",
33
+ "agnes_delta_attention",
34
+ "agnes_delta_attention",
35
+ "agnes_global_attention",
36
+ "agnes_delta_attention",
37
+ "agnes_delta_attention",
38
+ "agnes_delta_attention",
39
+ "agnes_global_attention",
40
+ "agnes_delta_attention",
41
+ "agnes_delta_attention",
42
+ "agnes_delta_attention",
43
+ "agnes_global_attention",
44
+ "agnes_delta_attention",
45
+ "agnes_delta_attention",
46
+ "agnes_delta_attention",
47
+ "agnes_global_attention",
48
+ "agnes_delta_attention",
49
+ "agnes_delta_attention",
50
+ "agnes_delta_attention",
51
+ "agnes_global_attention",
52
+ "agnes_delta_attention",
53
+ "agnes_delta_attention",
54
+ "agnes_delta_attention",
55
+ "agnes_global_attention",
56
+ "agnes_delta_attention",
57
+ "agnes_delta_attention",
58
+ "agnes_delta_attention",
59
+ "agnes_global_attention",
60
+ "agnes_delta_attention",
61
+ "agnes_delta_attention",
62
+ "agnes_delta_attention",
63
+ "agnes_global_attention",
64
+ "agnes_delta_attention",
65
+ "agnes_delta_attention",
66
+ "agnes_delta_attention",
67
+ "agnes_global_attention",
68
+ "agnes_delta_attention",
69
+ "agnes_delta_attention",
70
+ "agnes_delta_attention",
71
+ "agnes_global_attention",
72
+ "agnes_delta_attention",
73
+ "agnes_delta_attention",
74
+ "agnes_delta_attention",
75
+ "agnes_global_attention",
76
+ "agnes_delta_attention",
77
+ "agnes_delta_attention",
78
+ "agnes_delta_attention",
79
+ "agnes_global_attention",
80
+ "agnes_delta_attention",
81
+ "agnes_delta_attention",
82
+ "agnes_delta_attention",
83
+ "agnes_global_attention",
84
+ "agnes_delta_attention",
85
+ "agnes_delta_attention",
86
+ "agnes_delta_attention",
87
+ "agnes_global_attention",
88
+ "agnes_delta_attention",
89
+ "agnes_delta_attention",
90
+ "agnes_delta_attention",
91
+ "agnes_global_attention"
92
+ ],
93
+ "global_attention_interval": 4,
94
+ "num_attention_heads": 24,
95
+ "num_key_value_heads": 4,
96
+ "head_dim": 256,
97
+ "attention_bias": false,
98
+ "attention_dropout": 0.0,
99
+ "attn_output_gate": true,
100
+ "partial_rotary_factor": 0.25,
101
+ "linear_num_key_heads": 16,
102
+ "linear_num_value_heads": 48,
103
+ "linear_key_head_dim": 128,
104
+ "linear_value_head_dim": 128,
105
+ "linear_conv_kernel_dim": 4,
106
+ "output_gate_type": "swish",
107
+ "mamba_ssm_dtype": "float32",
108
+ "hidden_act": "silu",
109
+ "rms_norm_eps": 1e-06,
110
+ "initializer_range": 0.02,
111
+ "rope_parameters": {
112
+ "mrope_interleaved": true,
113
+ "mrope_section": [
114
+ 11,
115
+ 11,
116
+ 10
117
+ ],
118
+ "partial_rotary_factor": 0.25,
119
+ "rope_theta": 10000000,
120
+ "rope_type": "default"
121
+ },
122
+ "max_position_embeddings": 262144,
123
+ "mtp_num_hidden_layers": 1,
124
+ "mtp_use_dedicated_embeddings": false,
125
+ "bos_token_id": 248044,
126
+ "eos_token_id": 248044,
127
+ "pad_token_id": null,
128
+ "tie_word_embeddings": false,
129
+ "use_cache": true,
130
+ "dtype": "bfloat16"
131
+ },
132
+ "vision_config": {
133
+ "model_type": "agnes_vision",
134
+ "depth": 27,
135
+ "hidden_size": 1152,
136
+ "intermediate_size": 4304,
137
+ "num_heads": 16,
138
+ "out_hidden_size": 5120,
139
+ "patch_size": 16,
140
+ "spatial_merge_size": 2,
141
+ "temporal_patch_size": 2,
142
+ "in_channels": 3,
143
+ "num_position_embeddings": 2304,
144
+ "deepstack_visual_indexes": [],
145
+ "hidden_act": "gelu_pytorch_tanh",
146
+ "initializer_range": 0.02
147
+ },
148
+ "image_token_id": 248056,
149
+ "video_token_id": 248057,
150
+ "vision_start_token_id": 248053,
151
+ "vision_end_token_id": 248054,
152
+ "tie_word_embeddings": false,
153
+ "quantization_config": {
154
+ "quant_method": "modelopt_mixed",
155
+ "quant_algo": "MIXED_PRECISION",
156
+ "kv_cache_quant_algo": null,
157
+ "quantized_layers": {
158
+ "model.language_model.layers.0.linear_attn.out_proj": {
159
+ "quant_algo": "NVFP4",
160
+ "group_size": 16
161
+ },
162
+ "model.language_model.layers.0.linear_attn.in_proj_qkv": {
163
+ "quant_algo": "FP8"
164
+ },
165
+ "model.language_model.layers.0.linear_attn.in_proj_z": {
166
+ "quant_algo": "FP8"
167
+ },
168
+ "model.language_model.layers.0.mlp.gate_proj": {
169
+ "quant_algo": "NVFP4",
170
+ "group_size": 16
171
+ },
172
+ "model.language_model.layers.0.mlp.up_proj": {
173
+ "quant_algo": "NVFP4",
174
+ "group_size": 16
175
+ },
176
+ "model.language_model.layers.0.mlp.down_proj": {
177
+ "quant_algo": "NVFP4",
178
+ "group_size": 16
179
+ },
180
+ "model.language_model.layers.1.linear_attn.out_proj": {
181
+ "quant_algo": "NVFP4",
182
+ "group_size": 16
183
+ },
184
+ "model.language_model.layers.1.linear_attn.in_proj_qkv": {
185
+ "quant_algo": "FP8"
186
+ },
187
+ "model.language_model.layers.1.linear_attn.in_proj_z": {
188
+ "quant_algo": "FP8"
189
+ },
190
+ "model.language_model.layers.1.mlp.gate_proj": {
191
+ "quant_algo": "NVFP4",
192
+ "group_size": 16
193
+ },
194
+ "model.language_model.layers.1.mlp.up_proj": {
195
+ "quant_algo": "NVFP4",
196
+ "group_size": 16
197
+ },
198
+ "model.language_model.layers.1.mlp.down_proj": {
199
+ "quant_algo": "NVFP4",
200
+ "group_size": 16
201
+ },
202
+ "model.language_model.layers.2.linear_attn.out_proj": {
203
+ "quant_algo": "NVFP4",
204
+ "group_size": 16
205
+ },
206
+ "model.language_model.layers.2.linear_attn.in_proj_qkv": {
207
+ "quant_algo": "FP8"
208
+ },
209
+ "model.language_model.layers.2.linear_attn.in_proj_z": {
210
+ "quant_algo": "FP8"
211
+ },
212
+ "model.language_model.layers.2.mlp.gate_proj": {
213
+ "quant_algo": "NVFP4",
214
+ "group_size": 16
215
+ },
216
+ "model.language_model.layers.2.mlp.up_proj": {
217
+ "quant_algo": "NVFP4",
218
+ "group_size": 16
219
+ },
220
+ "model.language_model.layers.2.mlp.down_proj": {
221
+ "quant_algo": "NVFP4",
222
+ "group_size": 16
223
+ },
224
+ "model.language_model.layers.3.self_attn.q_proj": {
225
+ "quant_algo": "FP8"
226
+ },
227
+ "model.language_model.layers.3.self_attn.k_proj": {
228
+ "quant_algo": "FP8"
229
+ },
230
+ "model.language_model.layers.3.self_attn.v_proj": {
231
+ "quant_algo": "FP8"
232
+ },
233
+ "model.language_model.layers.3.self_attn.o_proj": {
234
+ "quant_algo": "NVFP4",
235
+ "group_size": 16
236
+ },
237
+ "model.language_model.layers.3.mlp.gate_proj": {
238
+ "quant_algo": "NVFP4",
239
+ "group_size": 16
240
+ },
241
+ "model.language_model.layers.3.mlp.up_proj": {
242
+ "quant_algo": "NVFP4",
243
+ "group_size": 16
244
+ },
245
+ "model.language_model.layers.3.mlp.down_proj": {
246
+ "quant_algo": "NVFP4",
247
+ "group_size": 16
248
+ },
249
+ "model.language_model.layers.4.linear_attn.out_proj": {
250
+ "quant_algo": "NVFP4",
251
+ "group_size": 16
252
+ },
253
+ "model.language_model.layers.4.linear_attn.in_proj_qkv": {
254
+ "quant_algo": "FP8"
255
+ },
256
+ "model.language_model.layers.4.linear_attn.in_proj_z": {
257
+ "quant_algo": "FP8"
258
+ },
259
+ "model.language_model.layers.4.mlp.gate_proj": {
260
+ "quant_algo": "NVFP4",
261
+ "group_size": 16
262
+ },
263
+ "model.language_model.layers.4.mlp.up_proj": {
264
+ "quant_algo": "NVFP4",
265
+ "group_size": 16
266
+ },
267
+ "model.language_model.layers.4.mlp.down_proj": {
268
+ "quant_algo": "NVFP4",
269
+ "group_size": 16
270
+ },
271
+ "model.language_model.layers.5.linear_attn.out_proj": {
272
+ "quant_algo": "NVFP4",
273
+ "group_size": 16
274
+ },
275
+ "model.language_model.layers.5.linear_attn.in_proj_qkv": {
276
+ "quant_algo": "FP8"
277
+ },
278
+ "model.language_model.layers.5.linear_attn.in_proj_z": {
279
+ "quant_algo": "FP8"
280
+ },
281
+ "model.language_model.layers.5.mlp.gate_proj": {
282
+ "quant_algo": "NVFP4",
283
+ "group_size": 16
284
+ },
285
+ "model.language_model.layers.5.mlp.up_proj": {
286
+ "quant_algo": "NVFP4",
287
+ "group_size": 16
288
+ },
289
+ "model.language_model.layers.5.mlp.down_proj": {
290
+ "quant_algo": "NVFP4",
291
+ "group_size": 16
292
+ },
293
+ "model.language_model.layers.6.linear_attn.out_proj": {
294
+ "quant_algo": "NVFP4",
295
+ "group_size": 16
296
+ },
297
+ "model.language_model.layers.6.linear_attn.in_proj_qkv": {
298
+ "quant_algo": "FP8"
299
+ },
300
+ "model.language_model.layers.6.linear_attn.in_proj_z": {
301
+ "quant_algo": "FP8"
302
+ },
303
+ "model.language_model.layers.6.mlp.gate_proj": {
304
+ "quant_algo": "NVFP4",
305
+ "group_size": 16
306
+ },
307
+ "model.language_model.layers.6.mlp.up_proj": {
308
+ "quant_algo": "NVFP4",
309
+ "group_size": 16
310
+ },
311
+ "model.language_model.layers.6.mlp.down_proj": {
312
+ "quant_algo": "NVFP4",
313
+ "group_size": 16
314
+ },
315
+ "model.language_model.layers.7.self_attn.q_proj": {
316
+ "quant_algo": "FP8"
317
+ },
318
+ "model.language_model.layers.7.self_attn.k_proj": {
319
+ "quant_algo": "FP8"
320
+ },
321
+ "model.language_model.layers.7.self_attn.v_proj": {
322
+ "quant_algo": "FP8"
323
+ },
324
+ "model.language_model.layers.7.self_attn.o_proj": {
325
+ "quant_algo": "NVFP4",
326
+ "group_size": 16
327
+ },
328
+ "model.language_model.layers.7.mlp.gate_proj": {
329
+ "quant_algo": "NVFP4",
330
+ "group_size": 16
331
+ },
332
+ "model.language_model.layers.7.mlp.up_proj": {
333
+ "quant_algo": "NVFP4",
334
+ "group_size": 16
335
+ },
336
+ "model.language_model.layers.7.mlp.down_proj": {
337
+ "quant_algo": "NVFP4",
338
+ "group_size": 16
339
+ },
340
+ "model.language_model.layers.8.linear_attn.out_proj": {
341
+ "quant_algo": "NVFP4",
342
+ "group_size": 16
343
+ },
344
+ "model.language_model.layers.8.linear_attn.in_proj_qkv": {
345
+ "quant_algo": "FP8"
346
+ },
347
+ "model.language_model.layers.8.linear_attn.in_proj_z": {
348
+ "quant_algo": "FP8"
349
+ },
350
+ "model.language_model.layers.8.mlp.gate_proj": {
351
+ "quant_algo": "NVFP4",
352
+ "group_size": 16
353
+ },
354
+ "model.language_model.layers.8.mlp.up_proj": {
355
+ "quant_algo": "NVFP4",
356
+ "group_size": 16
357
+ },
358
+ "model.language_model.layers.8.mlp.down_proj": {
359
+ "quant_algo": "NVFP4",
360
+ "group_size": 16
361
+ },
362
+ "model.language_model.layers.9.linear_attn.out_proj": {
363
+ "quant_algo": "NVFP4",
364
+ "group_size": 16
365
+ },
366
+ "model.language_model.layers.9.linear_attn.in_proj_qkv": {
367
+ "quant_algo": "FP8"
368
+ },
369
+ "model.language_model.layers.9.linear_attn.in_proj_z": {
370
+ "quant_algo": "FP8"
371
+ },
372
+ "model.language_model.layers.9.mlp.gate_proj": {
373
+ "quant_algo": "NVFP4",
374
+ "group_size": 16
375
+ },
376
+ "model.language_model.layers.9.mlp.up_proj": {
377
+ "quant_algo": "NVFP4",
378
+ "group_size": 16
379
+ },
380
+ "model.language_model.layers.9.mlp.down_proj": {
381
+ "quant_algo": "NVFP4",
382
+ "group_size": 16
383
+ },
384
+ "model.language_model.layers.10.linear_attn.out_proj": {
385
+ "quant_algo": "NVFP4",
386
+ "group_size": 16
387
+ },
388
+ "model.language_model.layers.10.linear_attn.in_proj_qkv": {
389
+ "quant_algo": "FP8"
390
+ },
391
+ "model.language_model.layers.10.linear_attn.in_proj_z": {
392
+ "quant_algo": "FP8"
393
+ },
394
+ "model.language_model.layers.10.mlp.gate_proj": {
395
+ "quant_algo": "NVFP4",
396
+ "group_size": 16
397
+ },
398
+ "model.language_model.layers.10.mlp.up_proj": {
399
+ "quant_algo": "NVFP4",
400
+ "group_size": 16
401
+ },
402
+ "model.language_model.layers.10.mlp.down_proj": {
403
+ "quant_algo": "NVFP4",
404
+ "group_size": 16
405
+ },
406
+ "model.language_model.layers.11.self_attn.q_proj": {
407
+ "quant_algo": "FP8"
408
+ },
409
+ "model.language_model.layers.11.self_attn.k_proj": {
410
+ "quant_algo": "FP8"
411
+ },
412
+ "model.language_model.layers.11.self_attn.v_proj": {
413
+ "quant_algo": "FP8"
414
+ },
415
+ "model.language_model.layers.11.self_attn.o_proj": {
416
+ "quant_algo": "NVFP4",
417
+ "group_size": 16
418
+ },
419
+ "model.language_model.layers.11.mlp.gate_proj": {
420
+ "quant_algo": "NVFP4",
421
+ "group_size": 16
422
+ },
423
+ "model.language_model.layers.11.mlp.up_proj": {
424
+ "quant_algo": "NVFP4",
425
+ "group_size": 16
426
+ },
427
+ "model.language_model.layers.11.mlp.down_proj": {
428
+ "quant_algo": "NVFP4",
429
+ "group_size": 16
430
+ },
431
+ "model.language_model.layers.12.linear_attn.out_proj": {
432
+ "quant_algo": "NVFP4",
433
+ "group_size": 16
434
+ },
435
+ "model.language_model.layers.12.linear_attn.in_proj_qkv": {
436
+ "quant_algo": "FP8"
437
+ },
438
+ "model.language_model.layers.12.linear_attn.in_proj_z": {
439
+ "quant_algo": "FP8"
440
+ },
441
+ "model.language_model.layers.12.mlp.gate_proj": {
442
+ "quant_algo": "NVFP4",
443
+ "group_size": 16
444
+ },
445
+ "model.language_model.layers.12.mlp.up_proj": {
446
+ "quant_algo": "NVFP4",
447
+ "group_size": 16
448
+ },
449
+ "model.language_model.layers.12.mlp.down_proj": {
450
+ "quant_algo": "NVFP4",
451
+ "group_size": 16
452
+ },
453
+ "model.language_model.layers.13.linear_attn.out_proj": {
454
+ "quant_algo": "NVFP4",
455
+ "group_size": 16
456
+ },
457
+ "model.language_model.layers.13.linear_attn.in_proj_qkv": {
458
+ "quant_algo": "FP8"
459
+ },
460
+ "model.language_model.layers.13.linear_attn.in_proj_z": {
461
+ "quant_algo": "FP8"
462
+ },
463
+ "model.language_model.layers.13.mlp.gate_proj": {
464
+ "quant_algo": "NVFP4",
465
+ "group_size": 16
466
+ },
467
+ "model.language_model.layers.13.mlp.up_proj": {
468
+ "quant_algo": "NVFP4",
469
+ "group_size": 16
470
+ },
471
+ "model.language_model.layers.13.mlp.down_proj": {
472
+ "quant_algo": "NVFP4",
473
+ "group_size": 16
474
+ },
475
+ "model.language_model.layers.14.linear_attn.out_proj": {
476
+ "quant_algo": "NVFP4",
477
+ "group_size": 16
478
+ },
479
+ "model.language_model.layers.14.linear_attn.in_proj_qkv": {
480
+ "quant_algo": "FP8"
481
+ },
482
+ "model.language_model.layers.14.linear_attn.in_proj_z": {
483
+ "quant_algo": "FP8"
484
+ },
485
+ "model.language_model.layers.14.mlp.gate_proj": {
486
+ "quant_algo": "NVFP4",
487
+ "group_size": 16
488
+ },
489
+ "model.language_model.layers.14.mlp.up_proj": {
490
+ "quant_algo": "NVFP4",
491
+ "group_size": 16
492
+ },
493
+ "model.language_model.layers.14.mlp.down_proj": {
494
+ "quant_algo": "NVFP4",
495
+ "group_size": 16
496
+ },
497
+ "model.language_model.layers.15.self_attn.q_proj": {
498
+ "quant_algo": "FP8"
499
+ },
500
+ "model.language_model.layers.15.self_attn.k_proj": {
501
+ "quant_algo": "FP8"
502
+ },
503
+ "model.language_model.layers.15.self_attn.v_proj": {
504
+ "quant_algo": "FP8"
505
+ },
506
+ "model.language_model.layers.15.self_attn.o_proj": {
507
+ "quant_algo": "NVFP4",
508
+ "group_size": 16
509
+ },
510
+ "model.language_model.layers.15.mlp.gate_proj": {
511
+ "quant_algo": "NVFP4",
512
+ "group_size": 16
513
+ },
514
+ "model.language_model.layers.15.mlp.up_proj": {
515
+ "quant_algo": "NVFP4",
516
+ "group_size": 16
517
+ },
518
+ "model.language_model.layers.15.mlp.down_proj": {
519
+ "quant_algo": "NVFP4",
520
+ "group_size": 16
521
+ },
522
+ "model.language_model.layers.16.linear_attn.out_proj": {
523
+ "quant_algo": "NVFP4",
524
+ "group_size": 16
525
+ },
526
+ "model.language_model.layers.16.linear_attn.in_proj_qkv": {
527
+ "quant_algo": "FP8"
528
+ },
529
+ "model.language_model.layers.16.linear_attn.in_proj_z": {
530
+ "quant_algo": "FP8"
531
+ },
532
+ "model.language_model.layers.16.mlp.gate_proj": {
533
+ "quant_algo": "NVFP4",
534
+ "group_size": 16
535
+ },
536
+ "model.language_model.layers.16.mlp.up_proj": {
537
+ "quant_algo": "NVFP4",
538
+ "group_size": 16
539
+ },
540
+ "model.language_model.layers.16.mlp.down_proj": {
541
+ "quant_algo": "NVFP4",
542
+ "group_size": 16
543
+ },
544
+ "model.language_model.layers.17.linear_attn.out_proj": {
545
+ "quant_algo": "NVFP4",
546
+ "group_size": 16
547
+ },
548
+ "model.language_model.layers.17.linear_attn.in_proj_qkv": {
549
+ "quant_algo": "FP8"
550
+ },
551
+ "model.language_model.layers.17.linear_attn.in_proj_z": {
552
+ "quant_algo": "FP8"
553
+ },
554
+ "model.language_model.layers.17.mlp.gate_proj": {
555
+ "quant_algo": "NVFP4",
556
+ "group_size": 16
557
+ },
558
+ "model.language_model.layers.17.mlp.up_proj": {
559
+ "quant_algo": "NVFP4",
560
+ "group_size": 16
561
+ },
562
+ "model.language_model.layers.17.mlp.down_proj": {
563
+ "quant_algo": "NVFP4",
564
+ "group_size": 16
565
+ },
566
+ "model.language_model.layers.18.linear_attn.out_proj": {
567
+ "quant_algo": "NVFP4",
568
+ "group_size": 16
569
+ },
570
+ "model.language_model.layers.18.linear_attn.in_proj_qkv": {
571
+ "quant_algo": "FP8"
572
+ },
573
+ "model.language_model.layers.18.linear_attn.in_proj_z": {
574
+ "quant_algo": "FP8"
575
+ },
576
+ "model.language_model.layers.18.mlp.gate_proj": {
577
+ "quant_algo": "NVFP4",
578
+ "group_size": 16
579
+ },
580
+ "model.language_model.layers.18.mlp.up_proj": {
581
+ "quant_algo": "NVFP4",
582
+ "group_size": 16
583
+ },
584
+ "model.language_model.layers.18.mlp.down_proj": {
585
+ "quant_algo": "NVFP4",
586
+ "group_size": 16
587
+ },
588
+ "model.language_model.layers.19.self_attn.q_proj": {
589
+ "quant_algo": "FP8"
590
+ },
591
+ "model.language_model.layers.19.self_attn.k_proj": {
592
+ "quant_algo": "FP8"
593
+ },
594
+ "model.language_model.layers.19.self_attn.v_proj": {
595
+ "quant_algo": "FP8"
596
+ },
597
+ "model.language_model.layers.19.self_attn.o_proj": {
598
+ "quant_algo": "NVFP4",
599
+ "group_size": 16
600
+ },
601
+ "model.language_model.layers.19.mlp.gate_proj": {
602
+ "quant_algo": "NVFP4",
603
+ "group_size": 16
604
+ },
605
+ "model.language_model.layers.19.mlp.up_proj": {
606
+ "quant_algo": "NVFP4",
607
+ "group_size": 16
608
+ },
609
+ "model.language_model.layers.19.mlp.down_proj": {
610
+ "quant_algo": "NVFP4",
611
+ "group_size": 16
612
+ },
613
+ "model.language_model.layers.20.linear_attn.out_proj": {
614
+ "quant_algo": "NVFP4",
615
+ "group_size": 16
616
+ },
617
+ "model.language_model.layers.20.linear_attn.in_proj_qkv": {
618
+ "quant_algo": "FP8"
619
+ },
620
+ "model.language_model.layers.20.linear_attn.in_proj_z": {
621
+ "quant_algo": "FP8"
622
+ },
623
+ "model.language_model.layers.20.mlp.gate_proj": {
624
+ "quant_algo": "NVFP4",
625
+ "group_size": 16
626
+ },
627
+ "model.language_model.layers.20.mlp.up_proj": {
628
+ "quant_algo": "NVFP4",
629
+ "group_size": 16
630
+ },
631
+ "model.language_model.layers.20.mlp.down_proj": {
632
+ "quant_algo": "NVFP4",
633
+ "group_size": 16
634
+ },
635
+ "model.language_model.layers.21.linear_attn.out_proj": {
636
+ "quant_algo": "NVFP4",
637
+ "group_size": 16
638
+ },
639
+ "model.language_model.layers.21.linear_attn.in_proj_qkv": {
640
+ "quant_algo": "FP8"
641
+ },
642
+ "model.language_model.layers.21.linear_attn.in_proj_z": {
643
+ "quant_algo": "FP8"
644
+ },
645
+ "model.language_model.layers.21.mlp.gate_proj": {
646
+ "quant_algo": "NVFP4",
647
+ "group_size": 16
648
+ },
649
+ "model.language_model.layers.21.mlp.up_proj": {
650
+ "quant_algo": "NVFP4",
651
+ "group_size": 16
652
+ },
653
+ "model.language_model.layers.21.mlp.down_proj": {
654
+ "quant_algo": "NVFP4",
655
+ "group_size": 16
656
+ },
657
+ "model.language_model.layers.22.linear_attn.out_proj": {
658
+ "quant_algo": "NVFP4",
659
+ "group_size": 16
660
+ },
661
+ "model.language_model.layers.22.linear_attn.in_proj_qkv": {
662
+ "quant_algo": "FP8"
663
+ },
664
+ "model.language_model.layers.22.linear_attn.in_proj_z": {
665
+ "quant_algo": "FP8"
666
+ },
667
+ "model.language_model.layers.22.mlp.gate_proj": {
668
+ "quant_algo": "NVFP4",
669
+ "group_size": 16
670
+ },
671
+ "model.language_model.layers.22.mlp.up_proj": {
672
+ "quant_algo": "NVFP4",
673
+ "group_size": 16
674
+ },
675
+ "model.language_model.layers.22.mlp.down_proj": {
676
+ "quant_algo": "NVFP4",
677
+ "group_size": 16
678
+ },
679
+ "model.language_model.layers.23.self_attn.q_proj": {
680
+ "quant_algo": "FP8"
681
+ },
682
+ "model.language_model.layers.23.self_attn.k_proj": {
683
+ "quant_algo": "FP8"
684
+ },
685
+ "model.language_model.layers.23.self_attn.v_proj": {
686
+ "quant_algo": "FP8"
687
+ },
688
+ "model.language_model.layers.23.self_attn.o_proj": {
689
+ "quant_algo": "NVFP4",
690
+ "group_size": 16
691
+ },
692
+ "model.language_model.layers.23.mlp.gate_proj": {
693
+ "quant_algo": "NVFP4",
694
+ "group_size": 16
695
+ },
696
+ "model.language_model.layers.23.mlp.up_proj": {
697
+ "quant_algo": "NVFP4",
698
+ "group_size": 16
699
+ },
700
+ "model.language_model.layers.23.mlp.down_proj": {
701
+ "quant_algo": "NVFP4",
702
+ "group_size": 16
703
+ },
704
+ "model.language_model.layers.24.linear_attn.out_proj": {
705
+ "quant_algo": "NVFP4",
706
+ "group_size": 16
707
+ },
708
+ "model.language_model.layers.24.linear_attn.in_proj_qkv": {
709
+ "quant_algo": "FP8"
710
+ },
711
+ "model.language_model.layers.24.linear_attn.in_proj_z": {
712
+ "quant_algo": "FP8"
713
+ },
714
+ "model.language_model.layers.24.mlp.gate_proj": {
715
+ "quant_algo": "NVFP4",
716
+ "group_size": 16
717
+ },
718
+ "model.language_model.layers.24.mlp.up_proj": {
719
+ "quant_algo": "NVFP4",
720
+ "group_size": 16
721
+ },
722
+ "model.language_model.layers.24.mlp.down_proj": {
723
+ "quant_algo": "NVFP4",
724
+ "group_size": 16
725
+ },
726
+ "model.language_model.layers.25.linear_attn.out_proj": {
727
+ "quant_algo": "NVFP4",
728
+ "group_size": 16
729
+ },
730
+ "model.language_model.layers.25.linear_attn.in_proj_qkv": {
731
+ "quant_algo": "FP8"
732
+ },
733
+ "model.language_model.layers.25.linear_attn.in_proj_z": {
734
+ "quant_algo": "FP8"
735
+ },
736
+ "model.language_model.layers.25.mlp.gate_proj": {
737
+ "quant_algo": "NVFP4",
738
+ "group_size": 16
739
+ },
740
+ "model.language_model.layers.25.mlp.up_proj": {
741
+ "quant_algo": "NVFP4",
742
+ "group_size": 16
743
+ },
744
+ "model.language_model.layers.25.mlp.down_proj": {
745
+ "quant_algo": "NVFP4",
746
+ "group_size": 16
747
+ },
748
+ "model.language_model.layers.26.linear_attn.out_proj": {
749
+ "quant_algo": "NVFP4",
750
+ "group_size": 16
751
+ },
752
+ "model.language_model.layers.26.linear_attn.in_proj_qkv": {
753
+ "quant_algo": "FP8"
754
+ },
755
+ "model.language_model.layers.26.linear_attn.in_proj_z": {
756
+ "quant_algo": "FP8"
757
+ },
758
+ "model.language_model.layers.26.mlp.gate_proj": {
759
+ "quant_algo": "NVFP4",
760
+ "group_size": 16
761
+ },
762
+ "model.language_model.layers.26.mlp.up_proj": {
763
+ "quant_algo": "NVFP4",
764
+ "group_size": 16
765
+ },
766
+ "model.language_model.layers.26.mlp.down_proj": {
767
+ "quant_algo": "NVFP4",
768
+ "group_size": 16
769
+ },
770
+ "model.language_model.layers.27.self_attn.q_proj": {
771
+ "quant_algo": "FP8"
772
+ },
773
+ "model.language_model.layers.27.self_attn.k_proj": {
774
+ "quant_algo": "FP8"
775
+ },
776
+ "model.language_model.layers.27.self_attn.v_proj": {
777
+ "quant_algo": "FP8"
778
+ },
779
+ "model.language_model.layers.27.self_attn.o_proj": {
780
+ "quant_algo": "NVFP4",
781
+ "group_size": 16
782
+ },
783
+ "model.language_model.layers.27.mlp.gate_proj": {
784
+ "quant_algo": "NVFP4",
785
+ "group_size": 16
786
+ },
787
+ "model.language_model.layers.27.mlp.up_proj": {
788
+ "quant_algo": "NVFP4",
789
+ "group_size": 16
790
+ },
791
+ "model.language_model.layers.27.mlp.down_proj": {
792
+ "quant_algo": "NVFP4",
793
+ "group_size": 16
794
+ },
795
+ "model.language_model.layers.28.linear_attn.out_proj": {
796
+ "quant_algo": "NVFP4",
797
+ "group_size": 16
798
+ },
799
+ "model.language_model.layers.28.linear_attn.in_proj_qkv": {
800
+ "quant_algo": "FP8"
801
+ },
802
+ "model.language_model.layers.28.linear_attn.in_proj_z": {
803
+ "quant_algo": "FP8"
804
+ },
805
+ "model.language_model.layers.28.mlp.gate_proj": {
806
+ "quant_algo": "NVFP4",
807
+ "group_size": 16
808
+ },
809
+ "model.language_model.layers.28.mlp.up_proj": {
810
+ "quant_algo": "NVFP4",
811
+ "group_size": 16
812
+ },
813
+ "model.language_model.layers.28.mlp.down_proj": {
814
+ "quant_algo": "NVFP4",
815
+ "group_size": 16
816
+ },
817
+ "model.language_model.layers.29.linear_attn.out_proj": {
818
+ "quant_algo": "NVFP4",
819
+ "group_size": 16
820
+ },
821
+ "model.language_model.layers.29.linear_attn.in_proj_qkv": {
822
+ "quant_algo": "FP8"
823
+ },
824
+ "model.language_model.layers.29.linear_attn.in_proj_z": {
825
+ "quant_algo": "FP8"
826
+ },
827
+ "model.language_model.layers.29.mlp.gate_proj": {
828
+ "quant_algo": "NVFP4",
829
+ "group_size": 16
830
+ },
831
+ "model.language_model.layers.29.mlp.up_proj": {
832
+ "quant_algo": "NVFP4",
833
+ "group_size": 16
834
+ },
835
+ "model.language_model.layers.29.mlp.down_proj": {
836
+ "quant_algo": "NVFP4",
837
+ "group_size": 16
838
+ },
839
+ "model.language_model.layers.30.linear_attn.out_proj": {
840
+ "quant_algo": "NVFP4",
841
+ "group_size": 16
842
+ },
843
+ "model.language_model.layers.30.linear_attn.in_proj_qkv": {
844
+ "quant_algo": "FP8"
845
+ },
846
+ "model.language_model.layers.30.linear_attn.in_proj_z": {
847
+ "quant_algo": "FP8"
848
+ },
849
+ "model.language_model.layers.30.mlp.gate_proj": {
850
+ "quant_algo": "NVFP4",
851
+ "group_size": 16
852
+ },
853
+ "model.language_model.layers.30.mlp.up_proj": {
854
+ "quant_algo": "NVFP4",
855
+ "group_size": 16
856
+ },
857
+ "model.language_model.layers.30.mlp.down_proj": {
858
+ "quant_algo": "NVFP4",
859
+ "group_size": 16
860
+ },
861
+ "model.language_model.layers.31.self_attn.q_proj": {
862
+ "quant_algo": "FP8"
863
+ },
864
+ "model.language_model.layers.31.self_attn.k_proj": {
865
+ "quant_algo": "FP8"
866
+ },
867
+ "model.language_model.layers.31.self_attn.v_proj": {
868
+ "quant_algo": "FP8"
869
+ },
870
+ "model.language_model.layers.31.self_attn.o_proj": {
871
+ "quant_algo": "NVFP4",
872
+ "group_size": 16
873
+ },
874
+ "model.language_model.layers.31.mlp.gate_proj": {
875
+ "quant_algo": "NVFP4",
876
+ "group_size": 16
877
+ },
878
+ "model.language_model.layers.31.mlp.up_proj": {
879
+ "quant_algo": "NVFP4",
880
+ "group_size": 16
881
+ },
882
+ "model.language_model.layers.31.mlp.down_proj": {
883
+ "quant_algo": "NVFP4",
884
+ "group_size": 16
885
+ },
886
+ "model.language_model.layers.32.linear_attn.out_proj": {
887
+ "quant_algo": "NVFP4",
888
+ "group_size": 16
889
+ },
890
+ "model.language_model.layers.32.linear_attn.in_proj_qkv": {
891
+ "quant_algo": "FP8"
892
+ },
893
+ "model.language_model.layers.32.linear_attn.in_proj_z": {
894
+ "quant_algo": "FP8"
895
+ },
896
+ "model.language_model.layers.32.mlp.gate_proj": {
897
+ "quant_algo": "NVFP4",
898
+ "group_size": 16
899
+ },
900
+ "model.language_model.layers.32.mlp.up_proj": {
901
+ "quant_algo": "NVFP4",
902
+ "group_size": 16
903
+ },
904
+ "model.language_model.layers.32.mlp.down_proj": {
905
+ "quant_algo": "NVFP4",
906
+ "group_size": 16
907
+ },
908
+ "model.language_model.layers.33.linear_attn.out_proj": {
909
+ "quant_algo": "NVFP4",
910
+ "group_size": 16
911
+ },
912
+ "model.language_model.layers.33.linear_attn.in_proj_qkv": {
913
+ "quant_algo": "FP8"
914
+ },
915
+ "model.language_model.layers.33.linear_attn.in_proj_z": {
916
+ "quant_algo": "FP8"
917
+ },
918
+ "model.language_model.layers.33.mlp.gate_proj": {
919
+ "quant_algo": "NVFP4",
920
+ "group_size": 16
921
+ },
922
+ "model.language_model.layers.33.mlp.up_proj": {
923
+ "quant_algo": "NVFP4",
924
+ "group_size": 16
925
+ },
926
+ "model.language_model.layers.33.mlp.down_proj": {
927
+ "quant_algo": "NVFP4",
928
+ "group_size": 16
929
+ },
930
+ "model.language_model.layers.34.linear_attn.out_proj": {
931
+ "quant_algo": "NVFP4",
932
+ "group_size": 16
933
+ },
934
+ "model.language_model.layers.34.linear_attn.in_proj_qkv": {
935
+ "quant_algo": "FP8"
936
+ },
937
+ "model.language_model.layers.34.linear_attn.in_proj_z": {
938
+ "quant_algo": "FP8"
939
+ },
940
+ "model.language_model.layers.34.mlp.gate_proj": {
941
+ "quant_algo": "NVFP4",
942
+ "group_size": 16
943
+ },
944
+ "model.language_model.layers.34.mlp.up_proj": {
945
+ "quant_algo": "NVFP4",
946
+ "group_size": 16
947
+ },
948
+ "model.language_model.layers.34.mlp.down_proj": {
949
+ "quant_algo": "NVFP4",
950
+ "group_size": 16
951
+ },
952
+ "model.language_model.layers.35.self_attn.q_proj": {
953
+ "quant_algo": "FP8"
954
+ },
955
+ "model.language_model.layers.35.self_attn.k_proj": {
956
+ "quant_algo": "FP8"
957
+ },
958
+ "model.language_model.layers.35.self_attn.v_proj": {
959
+ "quant_algo": "FP8"
960
+ },
961
+ "model.language_model.layers.35.self_attn.o_proj": {
962
+ "quant_algo": "NVFP4",
963
+ "group_size": 16
964
+ },
965
+ "model.language_model.layers.35.mlp.gate_proj": {
966
+ "quant_algo": "NVFP4",
967
+ "group_size": 16
968
+ },
969
+ "model.language_model.layers.35.mlp.up_proj": {
970
+ "quant_algo": "NVFP4",
971
+ "group_size": 16
972
+ },
973
+ "model.language_model.layers.35.mlp.down_proj": {
974
+ "quant_algo": "NVFP4",
975
+ "group_size": 16
976
+ },
977
+ "model.language_model.layers.36.linear_attn.out_proj": {
978
+ "quant_algo": "NVFP4",
979
+ "group_size": 16
980
+ },
981
+ "model.language_model.layers.36.linear_attn.in_proj_qkv": {
982
+ "quant_algo": "FP8"
983
+ },
984
+ "model.language_model.layers.36.linear_attn.in_proj_z": {
985
+ "quant_algo": "FP8"
986
+ },
987
+ "model.language_model.layers.36.mlp.gate_proj": {
988
+ "quant_algo": "NVFP4",
989
+ "group_size": 16
990
+ },
991
+ "model.language_model.layers.36.mlp.up_proj": {
992
+ "quant_algo": "NVFP4",
993
+ "group_size": 16
994
+ },
995
+ "model.language_model.layers.36.mlp.down_proj": {
996
+ "quant_algo": "NVFP4",
997
+ "group_size": 16
998
+ },
999
+ "model.language_model.layers.37.linear_attn.out_proj": {
1000
+ "quant_algo": "NVFP4",
1001
+ "group_size": 16
1002
+ },
1003
+ "model.language_model.layers.37.linear_attn.in_proj_qkv": {
1004
+ "quant_algo": "FP8"
1005
+ },
1006
+ "model.language_model.layers.37.linear_attn.in_proj_z": {
1007
+ "quant_algo": "FP8"
1008
+ },
1009
+ "model.language_model.layers.37.mlp.gate_proj": {
1010
+ "quant_algo": "NVFP4",
1011
+ "group_size": 16
1012
+ },
1013
+ "model.language_model.layers.37.mlp.up_proj": {
1014
+ "quant_algo": "NVFP4",
1015
+ "group_size": 16
1016
+ },
1017
+ "model.language_model.layers.37.mlp.down_proj": {
1018
+ "quant_algo": "NVFP4",
1019
+ "group_size": 16
1020
+ },
1021
+ "model.language_model.layers.38.linear_attn.out_proj": {
1022
+ "quant_algo": "NVFP4",
1023
+ "group_size": 16
1024
+ },
1025
+ "model.language_model.layers.38.linear_attn.in_proj_qkv": {
1026
+ "quant_algo": "FP8"
1027
+ },
1028
+ "model.language_model.layers.38.linear_attn.in_proj_z": {
1029
+ "quant_algo": "FP8"
1030
+ },
1031
+ "model.language_model.layers.38.mlp.gate_proj": {
1032
+ "quant_algo": "NVFP4",
1033
+ "group_size": 16
1034
+ },
1035
+ "model.language_model.layers.38.mlp.up_proj": {
1036
+ "quant_algo": "NVFP4",
1037
+ "group_size": 16
1038
+ },
1039
+ "model.language_model.layers.38.mlp.down_proj": {
1040
+ "quant_algo": "NVFP4",
1041
+ "group_size": 16
1042
+ },
1043
+ "model.language_model.layers.39.self_attn.q_proj": {
1044
+ "quant_algo": "FP8"
1045
+ },
1046
+ "model.language_model.layers.39.self_attn.k_proj": {
1047
+ "quant_algo": "FP8"
1048
+ },
1049
+ "model.language_model.layers.39.self_attn.v_proj": {
1050
+ "quant_algo": "FP8"
1051
+ },
1052
+ "model.language_model.layers.39.self_attn.o_proj": {
1053
+ "quant_algo": "NVFP4",
1054
+ "group_size": 16
1055
+ },
1056
+ "model.language_model.layers.39.mlp.gate_proj": {
1057
+ "quant_algo": "NVFP4",
1058
+ "group_size": 16
1059
+ },
1060
+ "model.language_model.layers.39.mlp.up_proj": {
1061
+ "quant_algo": "NVFP4",
1062
+ "group_size": 16
1063
+ },
1064
+ "model.language_model.layers.39.mlp.down_proj": {
1065
+ "quant_algo": "NVFP4",
1066
+ "group_size": 16
1067
+ },
1068
+ "model.language_model.layers.40.linear_attn.out_proj": {
1069
+ "quant_algo": "NVFP4",
1070
+ "group_size": 16
1071
+ },
1072
+ "model.language_model.layers.40.linear_attn.in_proj_qkv": {
1073
+ "quant_algo": "FP8"
1074
+ },
1075
+ "model.language_model.layers.40.linear_attn.in_proj_z": {
1076
+ "quant_algo": "FP8"
1077
+ },
1078
+ "model.language_model.layers.40.mlp.gate_proj": {
1079
+ "quant_algo": "NVFP4",
1080
+ "group_size": 16
1081
+ },
1082
+ "model.language_model.layers.40.mlp.up_proj": {
1083
+ "quant_algo": "NVFP4",
1084
+ "group_size": 16
1085
+ },
1086
+ "model.language_model.layers.40.mlp.down_proj": {
1087
+ "quant_algo": "NVFP4",
1088
+ "group_size": 16
1089
+ },
1090
+ "model.language_model.layers.41.linear_attn.out_proj": {
1091
+ "quant_algo": "NVFP4",
1092
+ "group_size": 16
1093
+ },
1094
+ "model.language_model.layers.41.linear_attn.in_proj_qkv": {
1095
+ "quant_algo": "FP8"
1096
+ },
1097
+ "model.language_model.layers.41.linear_attn.in_proj_z": {
1098
+ "quant_algo": "FP8"
1099
+ },
1100
+ "model.language_model.layers.41.mlp.gate_proj": {
1101
+ "quant_algo": "NVFP4",
1102
+ "group_size": 16
1103
+ },
1104
+ "model.language_model.layers.41.mlp.up_proj": {
1105
+ "quant_algo": "NVFP4",
1106
+ "group_size": 16
1107
+ },
1108
+ "model.language_model.layers.41.mlp.down_proj": {
1109
+ "quant_algo": "NVFP4",
1110
+ "group_size": 16
1111
+ },
1112
+ "model.language_model.layers.42.linear_attn.out_proj": {
1113
+ "quant_algo": "NVFP4",
1114
+ "group_size": 16
1115
+ },
1116
+ "model.language_model.layers.42.linear_attn.in_proj_qkv": {
1117
+ "quant_algo": "FP8"
1118
+ },
1119
+ "model.language_model.layers.42.linear_attn.in_proj_z": {
1120
+ "quant_algo": "FP8"
1121
+ },
1122
+ "model.language_model.layers.42.mlp.gate_proj": {
1123
+ "quant_algo": "NVFP4",
1124
+ "group_size": 16
1125
+ },
1126
+ "model.language_model.layers.42.mlp.up_proj": {
1127
+ "quant_algo": "NVFP4",
1128
+ "group_size": 16
1129
+ },
1130
+ "model.language_model.layers.42.mlp.down_proj": {
1131
+ "quant_algo": "NVFP4",
1132
+ "group_size": 16
1133
+ },
1134
+ "model.language_model.layers.43.self_attn.q_proj": {
1135
+ "quant_algo": "FP8"
1136
+ },
1137
+ "model.language_model.layers.43.self_attn.k_proj": {
1138
+ "quant_algo": "FP8"
1139
+ },
1140
+ "model.language_model.layers.43.self_attn.v_proj": {
1141
+ "quant_algo": "FP8"
1142
+ },
1143
+ "model.language_model.layers.43.self_attn.o_proj": {
1144
+ "quant_algo": "NVFP4",
1145
+ "group_size": 16
1146
+ },
1147
+ "model.language_model.layers.43.mlp.gate_proj": {
1148
+ "quant_algo": "NVFP4",
1149
+ "group_size": 16
1150
+ },
1151
+ "model.language_model.layers.43.mlp.up_proj": {
1152
+ "quant_algo": "NVFP4",
1153
+ "group_size": 16
1154
+ },
1155
+ "model.language_model.layers.43.mlp.down_proj": {
1156
+ "quant_algo": "NVFP4",
1157
+ "group_size": 16
1158
+ },
1159
+ "model.language_model.layers.44.linear_attn.out_proj": {
1160
+ "quant_algo": "NVFP4",
1161
+ "group_size": 16
1162
+ },
1163
+ "model.language_model.layers.44.linear_attn.in_proj_qkv": {
1164
+ "quant_algo": "FP8"
1165
+ },
1166
+ "model.language_model.layers.44.linear_attn.in_proj_z": {
1167
+ "quant_algo": "FP8"
1168
+ },
1169
+ "model.language_model.layers.44.mlp.gate_proj": {
1170
+ "quant_algo": "NVFP4",
1171
+ "group_size": 16
1172
+ },
1173
+ "model.language_model.layers.44.mlp.up_proj": {
1174
+ "quant_algo": "NVFP4",
1175
+ "group_size": 16
1176
+ },
1177
+ "model.language_model.layers.44.mlp.down_proj": {
1178
+ "quant_algo": "NVFP4",
1179
+ "group_size": 16
1180
+ },
1181
+ "model.language_model.layers.45.linear_attn.out_proj": {
1182
+ "quant_algo": "NVFP4",
1183
+ "group_size": 16
1184
+ },
1185
+ "model.language_model.layers.45.linear_attn.in_proj_qkv": {
1186
+ "quant_algo": "FP8"
1187
+ },
1188
+ "model.language_model.layers.45.linear_attn.in_proj_z": {
1189
+ "quant_algo": "FP8"
1190
+ },
1191
+ "model.language_model.layers.45.mlp.gate_proj": {
1192
+ "quant_algo": "NVFP4",
1193
+ "group_size": 16
1194
+ },
1195
+ "model.language_model.layers.45.mlp.up_proj": {
1196
+ "quant_algo": "NVFP4",
1197
+ "group_size": 16
1198
+ },
1199
+ "model.language_model.layers.45.mlp.down_proj": {
1200
+ "quant_algo": "NVFP4",
1201
+ "group_size": 16
1202
+ },
1203
+ "model.language_model.layers.46.linear_attn.out_proj": {
1204
+ "quant_algo": "NVFP4",
1205
+ "group_size": 16
1206
+ },
1207
+ "model.language_model.layers.46.linear_attn.in_proj_qkv": {
1208
+ "quant_algo": "FP8"
1209
+ },
1210
+ "model.language_model.layers.46.linear_attn.in_proj_z": {
1211
+ "quant_algo": "FP8"
1212
+ },
1213
+ "model.language_model.layers.46.mlp.gate_proj": {
1214
+ "quant_algo": "NVFP4",
1215
+ "group_size": 16
1216
+ },
1217
+ "model.language_model.layers.46.mlp.up_proj": {
1218
+ "quant_algo": "NVFP4",
1219
+ "group_size": 16
1220
+ },
1221
+ "model.language_model.layers.46.mlp.down_proj": {
1222
+ "quant_algo": "NVFP4",
1223
+ "group_size": 16
1224
+ },
1225
+ "model.language_model.layers.47.self_attn.q_proj": {
1226
+ "quant_algo": "FP8"
1227
+ },
1228
+ "model.language_model.layers.47.self_attn.k_proj": {
1229
+ "quant_algo": "FP8"
1230
+ },
1231
+ "model.language_model.layers.47.self_attn.v_proj": {
1232
+ "quant_algo": "FP8"
1233
+ },
1234
+ "model.language_model.layers.47.self_attn.o_proj": {
1235
+ "quant_algo": "NVFP4",
1236
+ "group_size": 16
1237
+ },
1238
+ "model.language_model.layers.47.mlp.gate_proj": {
1239
+ "quant_algo": "NVFP4",
1240
+ "group_size": 16
1241
+ },
1242
+ "model.language_model.layers.47.mlp.up_proj": {
1243
+ "quant_algo": "NVFP4",
1244
+ "group_size": 16
1245
+ },
1246
+ "model.language_model.layers.47.mlp.down_proj": {
1247
+ "quant_algo": "NVFP4",
1248
+ "group_size": 16
1249
+ },
1250
+ "model.language_model.layers.48.linear_attn.out_proj": {
1251
+ "quant_algo": "NVFP4",
1252
+ "group_size": 16
1253
+ },
1254
+ "model.language_model.layers.48.linear_attn.in_proj_qkv": {
1255
+ "quant_algo": "FP8"
1256
+ },
1257
+ "model.language_model.layers.48.linear_attn.in_proj_z": {
1258
+ "quant_algo": "FP8"
1259
+ },
1260
+ "model.language_model.layers.48.mlp.gate_proj": {
1261
+ "quant_algo": "NVFP4",
1262
+ "group_size": 16
1263
+ },
1264
+ "model.language_model.layers.48.mlp.up_proj": {
1265
+ "quant_algo": "NVFP4",
1266
+ "group_size": 16
1267
+ },
1268
+ "model.language_model.layers.48.mlp.down_proj": {
1269
+ "quant_algo": "NVFP4",
1270
+ "group_size": 16
1271
+ },
1272
+ "model.language_model.layers.49.linear_attn.out_proj": {
1273
+ "quant_algo": "NVFP4",
1274
+ "group_size": 16
1275
+ },
1276
+ "model.language_model.layers.49.linear_attn.in_proj_qkv": {
1277
+ "quant_algo": "FP8"
1278
+ },
1279
+ "model.language_model.layers.49.linear_attn.in_proj_z": {
1280
+ "quant_algo": "FP8"
1281
+ },
1282
+ "model.language_model.layers.49.mlp.gate_proj": {
1283
+ "quant_algo": "NVFP4",
1284
+ "group_size": 16
1285
+ },
1286
+ "model.language_model.layers.49.mlp.up_proj": {
1287
+ "quant_algo": "NVFP4",
1288
+ "group_size": 16
1289
+ },
1290
+ "model.language_model.layers.49.mlp.down_proj": {
1291
+ "quant_algo": "NVFP4",
1292
+ "group_size": 16
1293
+ },
1294
+ "model.language_model.layers.50.linear_attn.out_proj": {
1295
+ "quant_algo": "NVFP4",
1296
+ "group_size": 16
1297
+ },
1298
+ "model.language_model.layers.50.linear_attn.in_proj_qkv": {
1299
+ "quant_algo": "FP8"
1300
+ },
1301
+ "model.language_model.layers.50.linear_attn.in_proj_z": {
1302
+ "quant_algo": "FP8"
1303
+ },
1304
+ "model.language_model.layers.50.mlp.gate_proj": {
1305
+ "quant_algo": "NVFP4",
1306
+ "group_size": 16
1307
+ },
1308
+ "model.language_model.layers.50.mlp.up_proj": {
1309
+ "quant_algo": "NVFP4",
1310
+ "group_size": 16
1311
+ },
1312
+ "model.language_model.layers.50.mlp.down_proj": {
1313
+ "quant_algo": "NVFP4",
1314
+ "group_size": 16
1315
+ },
1316
+ "model.language_model.layers.51.self_attn.q_proj": {
1317
+ "quant_algo": "FP8"
1318
+ },
1319
+ "model.language_model.layers.51.self_attn.k_proj": {
1320
+ "quant_algo": "FP8"
1321
+ },
1322
+ "model.language_model.layers.51.self_attn.v_proj": {
1323
+ "quant_algo": "FP8"
1324
+ },
1325
+ "model.language_model.layers.51.self_attn.o_proj": {
1326
+ "quant_algo": "NVFP4",
1327
+ "group_size": 16
1328
+ },
1329
+ "model.language_model.layers.51.mlp.gate_proj": {
1330
+ "quant_algo": "NVFP4",
1331
+ "group_size": 16
1332
+ },
1333
+ "model.language_model.layers.51.mlp.up_proj": {
1334
+ "quant_algo": "NVFP4",
1335
+ "group_size": 16
1336
+ },
1337
+ "model.language_model.layers.51.mlp.down_proj": {
1338
+ "quant_algo": "NVFP4",
1339
+ "group_size": 16
1340
+ },
1341
+ "model.language_model.layers.52.linear_attn.out_proj": {
1342
+ "quant_algo": "NVFP4",
1343
+ "group_size": 16
1344
+ },
1345
+ "model.language_model.layers.52.linear_attn.in_proj_qkv": {
1346
+ "quant_algo": "FP8"
1347
+ },
1348
+ "model.language_model.layers.52.linear_attn.in_proj_z": {
1349
+ "quant_algo": "FP8"
1350
+ },
1351
+ "model.language_model.layers.52.mlp.gate_proj": {
1352
+ "quant_algo": "NVFP4",
1353
+ "group_size": 16
1354
+ },
1355
+ "model.language_model.layers.52.mlp.up_proj": {
1356
+ "quant_algo": "NVFP4",
1357
+ "group_size": 16
1358
+ },
1359
+ "model.language_model.layers.52.mlp.down_proj": {
1360
+ "quant_algo": "NVFP4",
1361
+ "group_size": 16
1362
+ },
1363
+ "model.language_model.layers.53.linear_attn.out_proj": {
1364
+ "quant_algo": "NVFP4",
1365
+ "group_size": 16
1366
+ },
1367
+ "model.language_model.layers.53.linear_attn.in_proj_qkv": {
1368
+ "quant_algo": "FP8"
1369
+ },
1370
+ "model.language_model.layers.53.linear_attn.in_proj_z": {
1371
+ "quant_algo": "FP8"
1372
+ },
1373
+ "model.language_model.layers.53.mlp.gate_proj": {
1374
+ "quant_algo": "NVFP4",
1375
+ "group_size": 16
1376
+ },
1377
+ "model.language_model.layers.53.mlp.up_proj": {
1378
+ "quant_algo": "NVFP4",
1379
+ "group_size": 16
1380
+ },
1381
+ "model.language_model.layers.53.mlp.down_proj": {
1382
+ "quant_algo": "NVFP4",
1383
+ "group_size": 16
1384
+ },
1385
+ "model.language_model.layers.54.linear_attn.out_proj": {
1386
+ "quant_algo": "NVFP4",
1387
+ "group_size": 16
1388
+ },
1389
+ "model.language_model.layers.54.linear_attn.in_proj_qkv": {
1390
+ "quant_algo": "FP8"
1391
+ },
1392
+ "model.language_model.layers.54.linear_attn.in_proj_z": {
1393
+ "quant_algo": "FP8"
1394
+ },
1395
+ "model.language_model.layers.54.mlp.gate_proj": {
1396
+ "quant_algo": "NVFP4",
1397
+ "group_size": 16
1398
+ },
1399
+ "model.language_model.layers.54.mlp.up_proj": {
1400
+ "quant_algo": "NVFP4",
1401
+ "group_size": 16
1402
+ },
1403
+ "model.language_model.layers.54.mlp.down_proj": {
1404
+ "quant_algo": "NVFP4",
1405
+ "group_size": 16
1406
+ },
1407
+ "model.language_model.layers.55.self_attn.q_proj": {
1408
+ "quant_algo": "FP8"
1409
+ },
1410
+ "model.language_model.layers.55.self_attn.k_proj": {
1411
+ "quant_algo": "FP8"
1412
+ },
1413
+ "model.language_model.layers.55.self_attn.v_proj": {
1414
+ "quant_algo": "FP8"
1415
+ },
1416
+ "model.language_model.layers.55.self_attn.o_proj": {
1417
+ "quant_algo": "NVFP4",
1418
+ "group_size": 16
1419
+ },
1420
+ "model.language_model.layers.55.mlp.gate_proj": {
1421
+ "quant_algo": "NVFP4",
1422
+ "group_size": 16
1423
+ },
1424
+ "model.language_model.layers.55.mlp.up_proj": {
1425
+ "quant_algo": "NVFP4",
1426
+ "group_size": 16
1427
+ },
1428
+ "model.language_model.layers.55.mlp.down_proj": {
1429
+ "quant_algo": "NVFP4",
1430
+ "group_size": 16
1431
+ },
1432
+ "model.language_model.layers.56.linear_attn.out_proj": {
1433
+ "quant_algo": "NVFP4",
1434
+ "group_size": 16
1435
+ },
1436
+ "model.language_model.layers.56.linear_attn.in_proj_qkv": {
1437
+ "quant_algo": "FP8"
1438
+ },
1439
+ "model.language_model.layers.56.linear_attn.in_proj_z": {
1440
+ "quant_algo": "FP8"
1441
+ },
1442
+ "model.language_model.layers.56.mlp.gate_proj": {
1443
+ "quant_algo": "NVFP4",
1444
+ "group_size": 16
1445
+ },
1446
+ "model.language_model.layers.56.mlp.up_proj": {
1447
+ "quant_algo": "NVFP4",
1448
+ "group_size": 16
1449
+ },
1450
+ "model.language_model.layers.56.mlp.down_proj": {
1451
+ "quant_algo": "NVFP4",
1452
+ "group_size": 16
1453
+ },
1454
+ "model.language_model.layers.57.linear_attn.out_proj": {
1455
+ "quant_algo": "NVFP4",
1456
+ "group_size": 16
1457
+ },
1458
+ "model.language_model.layers.57.linear_attn.in_proj_qkv": {
1459
+ "quant_algo": "FP8"
1460
+ },
1461
+ "model.language_model.layers.57.linear_attn.in_proj_z": {
1462
+ "quant_algo": "FP8"
1463
+ },
1464
+ "model.language_model.layers.57.mlp.gate_proj": {
1465
+ "quant_algo": "NVFP4",
1466
+ "group_size": 16
1467
+ },
1468
+ "model.language_model.layers.57.mlp.up_proj": {
1469
+ "quant_algo": "NVFP4",
1470
+ "group_size": 16
1471
+ },
1472
+ "model.language_model.layers.57.mlp.down_proj": {
1473
+ "quant_algo": "NVFP4",
1474
+ "group_size": 16
1475
+ },
1476
+ "model.language_model.layers.58.linear_attn.out_proj": {
1477
+ "quant_algo": "NVFP4",
1478
+ "group_size": 16
1479
+ },
1480
+ "model.language_model.layers.58.linear_attn.in_proj_qkv": {
1481
+ "quant_algo": "FP8"
1482
+ },
1483
+ "model.language_model.layers.58.linear_attn.in_proj_z": {
1484
+ "quant_algo": "FP8"
1485
+ },
1486
+ "model.language_model.layers.58.mlp.gate_proj": {
1487
+ "quant_algo": "NVFP4",
1488
+ "group_size": 16
1489
+ },
1490
+ "model.language_model.layers.58.mlp.up_proj": {
1491
+ "quant_algo": "NVFP4",
1492
+ "group_size": 16
1493
+ },
1494
+ "model.language_model.layers.58.mlp.down_proj": {
1495
+ "quant_algo": "NVFP4",
1496
+ "group_size": 16
1497
+ },
1498
+ "model.language_model.layers.59.self_attn.q_proj": {
1499
+ "quant_algo": "FP8"
1500
+ },
1501
+ "model.language_model.layers.59.self_attn.k_proj": {
1502
+ "quant_algo": "FP8"
1503
+ },
1504
+ "model.language_model.layers.59.self_attn.v_proj": {
1505
+ "quant_algo": "FP8"
1506
+ },
1507
+ "model.language_model.layers.59.self_attn.o_proj": {
1508
+ "quant_algo": "NVFP4",
1509
+ "group_size": 16
1510
+ },
1511
+ "model.language_model.layers.59.mlp.gate_proj": {
1512
+ "quant_algo": "NVFP4",
1513
+ "group_size": 16
1514
+ },
1515
+ "model.language_model.layers.59.mlp.up_proj": {
1516
+ "quant_algo": "NVFP4",
1517
+ "group_size": 16
1518
+ },
1519
+ "model.language_model.layers.59.mlp.down_proj": {
1520
+ "quant_algo": "NVFP4",
1521
+ "group_size": 16
1522
+ },
1523
+ "model.language_model.layers.60.linear_attn.out_proj": {
1524
+ "quant_algo": "NVFP4",
1525
+ "group_size": 16
1526
+ },
1527
+ "model.language_model.layers.60.linear_attn.in_proj_qkv": {
1528
+ "quant_algo": "FP8"
1529
+ },
1530
+ "model.language_model.layers.60.linear_attn.in_proj_z": {
1531
+ "quant_algo": "FP8"
1532
+ },
1533
+ "model.language_model.layers.60.mlp.gate_proj": {
1534
+ "quant_algo": "NVFP4",
1535
+ "group_size": 16
1536
+ },
1537
+ "model.language_model.layers.60.mlp.up_proj": {
1538
+ "quant_algo": "NVFP4",
1539
+ "group_size": 16
1540
+ },
1541
+ "model.language_model.layers.60.mlp.down_proj": {
1542
+ "quant_algo": "NVFP4",
1543
+ "group_size": 16
1544
+ },
1545
+ "model.language_model.layers.61.linear_attn.out_proj": {
1546
+ "quant_algo": "NVFP4",
1547
+ "group_size": 16
1548
+ },
1549
+ "model.language_model.layers.61.linear_attn.in_proj_qkv": {
1550
+ "quant_algo": "FP8"
1551
+ },
1552
+ "model.language_model.layers.61.linear_attn.in_proj_z": {
1553
+ "quant_algo": "FP8"
1554
+ },
1555
+ "model.language_model.layers.61.mlp.gate_proj": {
1556
+ "quant_algo": "NVFP4",
1557
+ "group_size": 16
1558
+ },
1559
+ "model.language_model.layers.61.mlp.up_proj": {
1560
+ "quant_algo": "NVFP4",
1561
+ "group_size": 16
1562
+ },
1563
+ "model.language_model.layers.61.mlp.down_proj": {
1564
+ "quant_algo": "NVFP4",
1565
+ "group_size": 16
1566
+ },
1567
+ "model.language_model.layers.62.linear_attn.out_proj": {
1568
+ "quant_algo": "NVFP4",
1569
+ "group_size": 16
1570
+ },
1571
+ "model.language_model.layers.62.linear_attn.in_proj_qkv": {
1572
+ "quant_algo": "FP8"
1573
+ },
1574
+ "model.language_model.layers.62.linear_attn.in_proj_z": {
1575
+ "quant_algo": "FP8"
1576
+ },
1577
+ "model.language_model.layers.62.mlp.gate_proj": {
1578
+ "quant_algo": "NVFP4",
1579
+ "group_size": 16
1580
+ },
1581
+ "model.language_model.layers.62.mlp.up_proj": {
1582
+ "quant_algo": "NVFP4",
1583
+ "group_size": 16
1584
+ },
1585
+ "model.language_model.layers.62.mlp.down_proj": {
1586
+ "quant_algo": "NVFP4",
1587
+ "group_size": 16
1588
+ },
1589
+ "model.language_model.layers.63.self_attn.q_proj": {
1590
+ "quant_algo": "FP8"
1591
+ },
1592
+ "model.language_model.layers.63.self_attn.k_proj": {
1593
+ "quant_algo": "FP8"
1594
+ },
1595
+ "model.language_model.layers.63.self_attn.v_proj": {
1596
+ "quant_algo": "FP8"
1597
+ },
1598
+ "model.language_model.layers.63.self_attn.o_proj": {
1599
+ "quant_algo": "NVFP4",
1600
+ "group_size": 16
1601
+ },
1602
+ "model.language_model.layers.63.mlp.gate_proj": {
1603
+ "quant_algo": "NVFP4",
1604
+ "group_size": 16
1605
+ },
1606
+ "model.language_model.layers.63.mlp.up_proj": {
1607
+ "quant_algo": "NVFP4",
1608
+ "group_size": 16
1609
+ },
1610
+ "model.language_model.layers.63.mlp.down_proj": {
1611
+ "quant_algo": "NVFP4",
1612
+ "group_size": 16
1613
+ },
1614
+ "model.language_model.layers.64.linear_attn.out_proj": {
1615
+ "quant_algo": "NVFP4",
1616
+ "group_size": 16
1617
+ },
1618
+ "model.language_model.layers.64.linear_attn.in_proj_qkv": {
1619
+ "quant_algo": "FP8"
1620
+ },
1621
+ "model.language_model.layers.64.linear_attn.in_proj_z": {
1622
+ "quant_algo": "FP8"
1623
+ },
1624
+ "model.language_model.layers.64.mlp.gate_proj": {
1625
+ "quant_algo": "NVFP4",
1626
+ "group_size": 16
1627
+ },
1628
+ "model.language_model.layers.64.mlp.up_proj": {
1629
+ "quant_algo": "NVFP4",
1630
+ "group_size": 16
1631
+ },
1632
+ "model.language_model.layers.64.mlp.down_proj": {
1633
+ "quant_algo": "NVFP4",
1634
+ "group_size": 16
1635
+ },
1636
+ "model.language_model.layers.65.linear_attn.out_proj": {
1637
+ "quant_algo": "NVFP4",
1638
+ "group_size": 16
1639
+ },
1640
+ "model.language_model.layers.65.linear_attn.in_proj_qkv": {
1641
+ "quant_algo": "FP8"
1642
+ },
1643
+ "model.language_model.layers.65.linear_attn.in_proj_z": {
1644
+ "quant_algo": "FP8"
1645
+ },
1646
+ "model.language_model.layers.65.mlp.gate_proj": {
1647
+ "quant_algo": "NVFP4",
1648
+ "group_size": 16
1649
+ },
1650
+ "model.language_model.layers.65.mlp.up_proj": {
1651
+ "quant_algo": "NVFP4",
1652
+ "group_size": 16
1653
+ },
1654
+ "model.language_model.layers.65.mlp.down_proj": {
1655
+ "quant_algo": "NVFP4",
1656
+ "group_size": 16
1657
+ },
1658
+ "model.language_model.layers.66.linear_attn.out_proj": {
1659
+ "quant_algo": "NVFP4",
1660
+ "group_size": 16
1661
+ },
1662
+ "model.language_model.layers.66.linear_attn.in_proj_qkv": {
1663
+ "quant_algo": "FP8"
1664
+ },
1665
+ "model.language_model.layers.66.linear_attn.in_proj_z": {
1666
+ "quant_algo": "FP8"
1667
+ },
1668
+ "model.language_model.layers.66.mlp.gate_proj": {
1669
+ "quant_algo": "NVFP4",
1670
+ "group_size": 16
1671
+ },
1672
+ "model.language_model.layers.66.mlp.up_proj": {
1673
+ "quant_algo": "NVFP4",
1674
+ "group_size": 16
1675
+ },
1676
+ "model.language_model.layers.66.mlp.down_proj": {
1677
+ "quant_algo": "NVFP4",
1678
+ "group_size": 16
1679
+ },
1680
+ "model.language_model.layers.67.self_attn.q_proj": {
1681
+ "quant_algo": "FP8"
1682
+ },
1683
+ "model.language_model.layers.67.self_attn.k_proj": {
1684
+ "quant_algo": "FP8"
1685
+ },
1686
+ "model.language_model.layers.67.self_attn.v_proj": {
1687
+ "quant_algo": "FP8"
1688
+ },
1689
+ "model.language_model.layers.67.self_attn.o_proj": {
1690
+ "quant_algo": "NVFP4",
1691
+ "group_size": 16
1692
+ },
1693
+ "model.language_model.layers.67.mlp.gate_proj": {
1694
+ "quant_algo": "NVFP4",
1695
+ "group_size": 16
1696
+ },
1697
+ "model.language_model.layers.67.mlp.up_proj": {
1698
+ "quant_algo": "NVFP4",
1699
+ "group_size": 16
1700
+ },
1701
+ "model.language_model.layers.67.mlp.down_proj": {
1702
+ "quant_algo": "NVFP4",
1703
+ "group_size": 16
1704
+ },
1705
+ "model.language_model.layers.68.linear_attn.out_proj": {
1706
+ "quant_algo": "NVFP4",
1707
+ "group_size": 16
1708
+ },
1709
+ "model.language_model.layers.68.linear_attn.in_proj_qkv": {
1710
+ "quant_algo": "FP8"
1711
+ },
1712
+ "model.language_model.layers.68.linear_attn.in_proj_z": {
1713
+ "quant_algo": "FP8"
1714
+ },
1715
+ "model.language_model.layers.68.mlp.gate_proj": {
1716
+ "quant_algo": "NVFP4",
1717
+ "group_size": 16
1718
+ },
1719
+ "model.language_model.layers.68.mlp.up_proj": {
1720
+ "quant_algo": "NVFP4",
1721
+ "group_size": 16
1722
+ },
1723
+ "model.language_model.layers.68.mlp.down_proj": {
1724
+ "quant_algo": "NVFP4",
1725
+ "group_size": 16
1726
+ },
1727
+ "model.language_model.layers.69.linear_attn.out_proj": {
1728
+ "quant_algo": "NVFP4",
1729
+ "group_size": 16
1730
+ },
1731
+ "model.language_model.layers.69.linear_attn.in_proj_qkv": {
1732
+ "quant_algo": "FP8"
1733
+ },
1734
+ "model.language_model.layers.69.linear_attn.in_proj_z": {
1735
+ "quant_algo": "FP8"
1736
+ },
1737
+ "model.language_model.layers.69.mlp.gate_proj": {
1738
+ "quant_algo": "NVFP4",
1739
+ "group_size": 16
1740
+ },
1741
+ "model.language_model.layers.69.mlp.up_proj": {
1742
+ "quant_algo": "NVFP4",
1743
+ "group_size": 16
1744
+ },
1745
+ "model.language_model.layers.69.mlp.down_proj": {
1746
+ "quant_algo": "NVFP4",
1747
+ "group_size": 16
1748
+ },
1749
+ "model.language_model.layers.70.linear_attn.out_proj": {
1750
+ "quant_algo": "NVFP4",
1751
+ "group_size": 16
1752
+ },
1753
+ "model.language_model.layers.70.linear_attn.in_proj_qkv": {
1754
+ "quant_algo": "FP8"
1755
+ },
1756
+ "model.language_model.layers.70.linear_attn.in_proj_z": {
1757
+ "quant_algo": "FP8"
1758
+ },
1759
+ "model.language_model.layers.70.mlp.gate_proj": {
1760
+ "quant_algo": "NVFP4",
1761
+ "group_size": 16
1762
+ },
1763
+ "model.language_model.layers.70.mlp.up_proj": {
1764
+ "quant_algo": "NVFP4",
1765
+ "group_size": 16
1766
+ },
1767
+ "model.language_model.layers.70.mlp.down_proj": {
1768
+ "quant_algo": "NVFP4",
1769
+ "group_size": 16
1770
+ },
1771
+ "model.language_model.layers.71.self_attn.q_proj": {
1772
+ "quant_algo": "FP8"
1773
+ },
1774
+ "model.language_model.layers.71.self_attn.k_proj": {
1775
+ "quant_algo": "FP8"
1776
+ },
1777
+ "model.language_model.layers.71.self_attn.v_proj": {
1778
+ "quant_algo": "FP8"
1779
+ },
1780
+ "model.language_model.layers.71.self_attn.o_proj": {
1781
+ "quant_algo": "NVFP4",
1782
+ "group_size": 16
1783
+ },
1784
+ "model.language_model.layers.71.mlp.gate_proj": {
1785
+ "quant_algo": "NVFP4",
1786
+ "group_size": 16
1787
+ },
1788
+ "model.language_model.layers.71.mlp.up_proj": {
1789
+ "quant_algo": "NVFP4",
1790
+ "group_size": 16
1791
+ },
1792
+ "model.language_model.layers.71.mlp.down_proj": {
1793
+ "quant_algo": "NVFP4",
1794
+ "group_size": 16
1795
+ },
1796
+ "lm_head": {
1797
+ "quant_algo": "FP8"
1798
+ },
1799
+ "model.language_model.embed_tokens": {
1800
+ "quant_algo": "FP8_ROW"
1801
+ },
1802
+ "mtp.fc": {
1803
+ "quant_algo": "FP8_DYNAMIC"
1804
+ },
1805
+ "mtp.layers.0.self_attn.k_proj": {
1806
+ "quant_algo": "FP8_DYNAMIC"
1807
+ },
1808
+ "mtp.layers.0.self_attn.o_proj": {
1809
+ "quant_algo": "FP8_DYNAMIC"
1810
+ },
1811
+ "mtp.layers.0.self_attn.q_proj": {
1812
+ "quant_algo": "FP8_DYNAMIC"
1813
+ },
1814
+ "mtp.layers.0.self_attn.v_proj": {
1815
+ "quant_algo": "FP8_DYNAMIC"
1816
+ },
1817
+ "mtp.layers.0.mlp.down_proj": {
1818
+ "quant_algo": "FP8_DYNAMIC"
1819
+ },
1820
+ "mtp.layers.0.mlp.gate_proj": {
1821
+ "quant_algo": "FP8_DYNAMIC"
1822
+ },
1823
+ "mtp.layers.0.mlp.up_proj": {
1824
+ "quant_algo": "FP8_DYNAMIC"
1825
+ },
1826
+ "mtp.embed_tokens": {
1827
+ "quant_algo": "FP8_ROW"
1828
+ },
1829
+ "mtp.layers.0.self_attn.qkv_proj": {
1830
+ "quant_algo": "FP8_DYNAMIC"
1831
+ },
1832
+ "mtp.layers.0.mlp.gate_up_proj": {
1833
+ "quant_algo": "FP8_DYNAMIC"
1834
+ },
1835
+ "model.visual.blocks.0.attn.proj": {
1836
+ "quant_algo": "FP8_DYNAMIC"
1837
+ },
1838
+ "model.visual.blocks.0.attn.qkv": {
1839
+ "quant_algo": "FP8_DYNAMIC"
1840
+ },
1841
+ "model.visual.blocks.0.attn.qkv_proj": {
1842
+ "quant_algo": "FP8_DYNAMIC"
1843
+ },
1844
+ "model.visual.blocks.0.mlp.linear_fc1": {
1845
+ "quant_algo": "FP8_DYNAMIC"
1846
+ },
1847
+ "model.visual.blocks.0.mlp.linear_fc2": {
1848
+ "quant_algo": "FP8_DYNAMIC"
1849
+ },
1850
+ "model.visual.blocks.1.attn.proj": {
1851
+ "quant_algo": "FP8_DYNAMIC"
1852
+ },
1853
+ "model.visual.blocks.1.attn.qkv": {
1854
+ "quant_algo": "FP8_DYNAMIC"
1855
+ },
1856
+ "model.visual.blocks.1.attn.qkv_proj": {
1857
+ "quant_algo": "FP8_DYNAMIC"
1858
+ },
1859
+ "model.visual.blocks.1.mlp.linear_fc1": {
1860
+ "quant_algo": "FP8_DYNAMIC"
1861
+ },
1862
+ "model.visual.blocks.1.mlp.linear_fc2": {
1863
+ "quant_algo": "FP8_DYNAMIC"
1864
+ },
1865
+ "model.visual.blocks.10.attn.proj": {
1866
+ "quant_algo": "FP8_DYNAMIC"
1867
+ },
1868
+ "model.visual.blocks.10.attn.qkv": {
1869
+ "quant_algo": "FP8_DYNAMIC"
1870
+ },
1871
+ "model.visual.blocks.10.attn.qkv_proj": {
1872
+ "quant_algo": "FP8_DYNAMIC"
1873
+ },
1874
+ "model.visual.blocks.10.mlp.linear_fc1": {
1875
+ "quant_algo": "FP8_DYNAMIC"
1876
+ },
1877
+ "model.visual.blocks.10.mlp.linear_fc2": {
1878
+ "quant_algo": "FP8_DYNAMIC"
1879
+ },
1880
+ "model.visual.blocks.11.attn.proj": {
1881
+ "quant_algo": "FP8_DYNAMIC"
1882
+ },
1883
+ "model.visual.blocks.11.attn.qkv": {
1884
+ "quant_algo": "FP8_DYNAMIC"
1885
+ },
1886
+ "model.visual.blocks.11.attn.qkv_proj": {
1887
+ "quant_algo": "FP8_DYNAMIC"
1888
+ },
1889
+ "model.visual.blocks.11.mlp.linear_fc1": {
1890
+ "quant_algo": "FP8_DYNAMIC"
1891
+ },
1892
+ "model.visual.blocks.11.mlp.linear_fc2": {
1893
+ "quant_algo": "FP8_DYNAMIC"
1894
+ },
1895
+ "model.visual.blocks.12.attn.proj": {
1896
+ "quant_algo": "FP8_DYNAMIC"
1897
+ },
1898
+ "model.visual.blocks.12.attn.qkv": {
1899
+ "quant_algo": "FP8_DYNAMIC"
1900
+ },
1901
+ "model.visual.blocks.12.attn.qkv_proj": {
1902
+ "quant_algo": "FP8_DYNAMIC"
1903
+ },
1904
+ "model.visual.blocks.12.mlp.linear_fc1": {
1905
+ "quant_algo": "FP8_DYNAMIC"
1906
+ },
1907
+ "model.visual.blocks.12.mlp.linear_fc2": {
1908
+ "quant_algo": "FP8_DYNAMIC"
1909
+ },
1910
+ "model.visual.blocks.13.attn.proj": {
1911
+ "quant_algo": "FP8_DYNAMIC"
1912
+ },
1913
+ "model.visual.blocks.13.attn.qkv": {
1914
+ "quant_algo": "FP8_DYNAMIC"
1915
+ },
1916
+ "model.visual.blocks.13.attn.qkv_proj": {
1917
+ "quant_algo": "FP8_DYNAMIC"
1918
+ },
1919
+ "model.visual.blocks.13.mlp.linear_fc1": {
1920
+ "quant_algo": "FP8_DYNAMIC"
1921
+ },
1922
+ "model.visual.blocks.13.mlp.linear_fc2": {
1923
+ "quant_algo": "FP8_DYNAMIC"
1924
+ },
1925
+ "model.visual.blocks.14.attn.proj": {
1926
+ "quant_algo": "FP8_DYNAMIC"
1927
+ },
1928
+ "model.visual.blocks.14.attn.qkv": {
1929
+ "quant_algo": "FP8_DYNAMIC"
1930
+ },
1931
+ "model.visual.blocks.14.attn.qkv_proj": {
1932
+ "quant_algo": "FP8_DYNAMIC"
1933
+ },
1934
+ "model.visual.blocks.14.mlp.linear_fc1": {
1935
+ "quant_algo": "FP8_DYNAMIC"
1936
+ },
1937
+ "model.visual.blocks.14.mlp.linear_fc2": {
1938
+ "quant_algo": "FP8_DYNAMIC"
1939
+ },
1940
+ "model.visual.blocks.15.attn.proj": {
1941
+ "quant_algo": "FP8_DYNAMIC"
1942
+ },
1943
+ "model.visual.blocks.15.attn.qkv": {
1944
+ "quant_algo": "FP8_DYNAMIC"
1945
+ },
1946
+ "model.visual.blocks.15.attn.qkv_proj": {
1947
+ "quant_algo": "FP8_DYNAMIC"
1948
+ },
1949
+ "model.visual.blocks.15.mlp.linear_fc1": {
1950
+ "quant_algo": "FP8_DYNAMIC"
1951
+ },
1952
+ "model.visual.blocks.15.mlp.linear_fc2": {
1953
+ "quant_algo": "FP8_DYNAMIC"
1954
+ },
1955
+ "model.visual.blocks.16.attn.proj": {
1956
+ "quant_algo": "FP8_DYNAMIC"
1957
+ },
1958
+ "model.visual.blocks.16.attn.qkv": {
1959
+ "quant_algo": "FP8_DYNAMIC"
1960
+ },
1961
+ "model.visual.blocks.16.attn.qkv_proj": {
1962
+ "quant_algo": "FP8_DYNAMIC"
1963
+ },
1964
+ "model.visual.blocks.16.mlp.linear_fc1": {
1965
+ "quant_algo": "FP8_DYNAMIC"
1966
+ },
1967
+ "model.visual.blocks.16.mlp.linear_fc2": {
1968
+ "quant_algo": "FP8_DYNAMIC"
1969
+ },
1970
+ "model.visual.blocks.17.attn.proj": {
1971
+ "quant_algo": "FP8_DYNAMIC"
1972
+ },
1973
+ "model.visual.blocks.17.attn.qkv": {
1974
+ "quant_algo": "FP8_DYNAMIC"
1975
+ },
1976
+ "model.visual.blocks.17.attn.qkv_proj": {
1977
+ "quant_algo": "FP8_DYNAMIC"
1978
+ },
1979
+ "model.visual.blocks.17.mlp.linear_fc1": {
1980
+ "quant_algo": "FP8_DYNAMIC"
1981
+ },
1982
+ "model.visual.blocks.17.mlp.linear_fc2": {
1983
+ "quant_algo": "FP8_DYNAMIC"
1984
+ },
1985
+ "model.visual.blocks.18.attn.proj": {
1986
+ "quant_algo": "FP8_DYNAMIC"
1987
+ },
1988
+ "model.visual.blocks.18.attn.qkv": {
1989
+ "quant_algo": "FP8_DYNAMIC"
1990
+ },
1991
+ "model.visual.blocks.18.attn.qkv_proj": {
1992
+ "quant_algo": "FP8_DYNAMIC"
1993
+ },
1994
+ "model.visual.blocks.18.mlp.linear_fc1": {
1995
+ "quant_algo": "FP8_DYNAMIC"
1996
+ },
1997
+ "model.visual.blocks.18.mlp.linear_fc2": {
1998
+ "quant_algo": "FP8_DYNAMIC"
1999
+ },
2000
+ "model.visual.blocks.19.attn.proj": {
2001
+ "quant_algo": "FP8_DYNAMIC"
2002
+ },
2003
+ "model.visual.blocks.19.attn.qkv": {
2004
+ "quant_algo": "FP8_DYNAMIC"
2005
+ },
2006
+ "model.visual.blocks.19.attn.qkv_proj": {
2007
+ "quant_algo": "FP8_DYNAMIC"
2008
+ },
2009
+ "model.visual.blocks.19.mlp.linear_fc1": {
2010
+ "quant_algo": "FP8_DYNAMIC"
2011
+ },
2012
+ "model.visual.blocks.19.mlp.linear_fc2": {
2013
+ "quant_algo": "FP8_DYNAMIC"
2014
+ },
2015
+ "model.visual.blocks.2.attn.proj": {
2016
+ "quant_algo": "FP8_DYNAMIC"
2017
+ },
2018
+ "model.visual.blocks.2.attn.qkv": {
2019
+ "quant_algo": "FP8_DYNAMIC"
2020
+ },
2021
+ "model.visual.blocks.2.attn.qkv_proj": {
2022
+ "quant_algo": "FP8_DYNAMIC"
2023
+ },
2024
+ "model.visual.blocks.2.mlp.linear_fc1": {
2025
+ "quant_algo": "FP8_DYNAMIC"
2026
+ },
2027
+ "model.visual.blocks.2.mlp.linear_fc2": {
2028
+ "quant_algo": "FP8_DYNAMIC"
2029
+ },
2030
+ "model.visual.blocks.20.attn.proj": {
2031
+ "quant_algo": "FP8_DYNAMIC"
2032
+ },
2033
+ "model.visual.blocks.20.attn.qkv": {
2034
+ "quant_algo": "FP8_DYNAMIC"
2035
+ },
2036
+ "model.visual.blocks.20.attn.qkv_proj": {
2037
+ "quant_algo": "FP8_DYNAMIC"
2038
+ },
2039
+ "model.visual.blocks.20.mlp.linear_fc1": {
2040
+ "quant_algo": "FP8_DYNAMIC"
2041
+ },
2042
+ "model.visual.blocks.20.mlp.linear_fc2": {
2043
+ "quant_algo": "FP8_DYNAMIC"
2044
+ },
2045
+ "model.visual.blocks.21.attn.proj": {
2046
+ "quant_algo": "FP8_DYNAMIC"
2047
+ },
2048
+ "model.visual.blocks.21.attn.qkv": {
2049
+ "quant_algo": "FP8_DYNAMIC"
2050
+ },
2051
+ "model.visual.blocks.21.attn.qkv_proj": {
2052
+ "quant_algo": "FP8_DYNAMIC"
2053
+ },
2054
+ "model.visual.blocks.21.mlp.linear_fc1": {
2055
+ "quant_algo": "FP8_DYNAMIC"
2056
+ },
2057
+ "model.visual.blocks.21.mlp.linear_fc2": {
2058
+ "quant_algo": "FP8_DYNAMIC"
2059
+ },
2060
+ "model.visual.blocks.22.attn.proj": {
2061
+ "quant_algo": "FP8_DYNAMIC"
2062
+ },
2063
+ "model.visual.blocks.22.attn.qkv": {
2064
+ "quant_algo": "FP8_DYNAMIC"
2065
+ },
2066
+ "model.visual.blocks.22.attn.qkv_proj": {
2067
+ "quant_algo": "FP8_DYNAMIC"
2068
+ },
2069
+ "model.visual.blocks.22.mlp.linear_fc1": {
2070
+ "quant_algo": "FP8_DYNAMIC"
2071
+ },
2072
+ "model.visual.blocks.22.mlp.linear_fc2": {
2073
+ "quant_algo": "FP8_DYNAMIC"
2074
+ },
2075
+ "model.visual.blocks.23.attn.proj": {
2076
+ "quant_algo": "FP8_DYNAMIC"
2077
+ },
2078
+ "model.visual.blocks.23.attn.qkv": {
2079
+ "quant_algo": "FP8_DYNAMIC"
2080
+ },
2081
+ "model.visual.blocks.23.attn.qkv_proj": {
2082
+ "quant_algo": "FP8_DYNAMIC"
2083
+ },
2084
+ "model.visual.blocks.23.mlp.linear_fc1": {
2085
+ "quant_algo": "FP8_DYNAMIC"
2086
+ },
2087
+ "model.visual.blocks.23.mlp.linear_fc2": {
2088
+ "quant_algo": "FP8_DYNAMIC"
2089
+ },
2090
+ "model.visual.blocks.24.attn.proj": {
2091
+ "quant_algo": "FP8_DYNAMIC"
2092
+ },
2093
+ "model.visual.blocks.24.attn.qkv": {
2094
+ "quant_algo": "FP8_DYNAMIC"
2095
+ },
2096
+ "model.visual.blocks.24.attn.qkv_proj": {
2097
+ "quant_algo": "FP8_DYNAMIC"
2098
+ },
2099
+ "model.visual.blocks.24.mlp.linear_fc1": {
2100
+ "quant_algo": "FP8_DYNAMIC"
2101
+ },
2102
+ "model.visual.blocks.24.mlp.linear_fc2": {
2103
+ "quant_algo": "FP8_DYNAMIC"
2104
+ },
2105
+ "model.visual.blocks.25.attn.proj": {
2106
+ "quant_algo": "FP8_DYNAMIC"
2107
+ },
2108
+ "model.visual.blocks.25.attn.qkv": {
2109
+ "quant_algo": "FP8_DYNAMIC"
2110
+ },
2111
+ "model.visual.blocks.25.attn.qkv_proj": {
2112
+ "quant_algo": "FP8_DYNAMIC"
2113
+ },
2114
+ "model.visual.blocks.25.mlp.linear_fc1": {
2115
+ "quant_algo": "FP8_DYNAMIC"
2116
+ },
2117
+ "model.visual.blocks.25.mlp.linear_fc2": {
2118
+ "quant_algo": "FP8_DYNAMIC"
2119
+ },
2120
+ "model.visual.blocks.26.attn.proj": {
2121
+ "quant_algo": "FP8_DYNAMIC"
2122
+ },
2123
+ "model.visual.blocks.26.attn.qkv": {
2124
+ "quant_algo": "FP8_DYNAMIC"
2125
+ },
2126
+ "model.visual.blocks.26.attn.qkv_proj": {
2127
+ "quant_algo": "FP8_DYNAMIC"
2128
+ },
2129
+ "model.visual.blocks.26.mlp.linear_fc1": {
2130
+ "quant_algo": "FP8_DYNAMIC"
2131
+ },
2132
+ "model.visual.blocks.26.mlp.linear_fc2": {
2133
+ "quant_algo": "FP8_DYNAMIC"
2134
+ },
2135
+ "model.visual.blocks.3.attn.proj": {
2136
+ "quant_algo": "FP8_DYNAMIC"
2137
+ },
2138
+ "model.visual.blocks.3.attn.qkv": {
2139
+ "quant_algo": "FP8_DYNAMIC"
2140
+ },
2141
+ "model.visual.blocks.3.attn.qkv_proj": {
2142
+ "quant_algo": "FP8_DYNAMIC"
2143
+ },
2144
+ "model.visual.blocks.3.mlp.linear_fc1": {
2145
+ "quant_algo": "FP8_DYNAMIC"
2146
+ },
2147
+ "model.visual.blocks.3.mlp.linear_fc2": {
2148
+ "quant_algo": "FP8_DYNAMIC"
2149
+ },
2150
+ "model.visual.blocks.4.attn.proj": {
2151
+ "quant_algo": "FP8_DYNAMIC"
2152
+ },
2153
+ "model.visual.blocks.4.attn.qkv": {
2154
+ "quant_algo": "FP8_DYNAMIC"
2155
+ },
2156
+ "model.visual.blocks.4.attn.qkv_proj": {
2157
+ "quant_algo": "FP8_DYNAMIC"
2158
+ },
2159
+ "model.visual.blocks.4.mlp.linear_fc1": {
2160
+ "quant_algo": "FP8_DYNAMIC"
2161
+ },
2162
+ "model.visual.blocks.4.mlp.linear_fc2": {
2163
+ "quant_algo": "FP8_DYNAMIC"
2164
+ },
2165
+ "model.visual.blocks.5.attn.proj": {
2166
+ "quant_algo": "FP8_DYNAMIC"
2167
+ },
2168
+ "model.visual.blocks.5.attn.qkv": {
2169
+ "quant_algo": "FP8_DYNAMIC"
2170
+ },
2171
+ "model.visual.blocks.5.attn.qkv_proj": {
2172
+ "quant_algo": "FP8_DYNAMIC"
2173
+ },
2174
+ "model.visual.blocks.5.mlp.linear_fc1": {
2175
+ "quant_algo": "FP8_DYNAMIC"
2176
+ },
2177
+ "model.visual.blocks.5.mlp.linear_fc2": {
2178
+ "quant_algo": "FP8_DYNAMIC"
2179
+ },
2180
+ "model.visual.blocks.6.attn.proj": {
2181
+ "quant_algo": "FP8_DYNAMIC"
2182
+ },
2183
+ "model.visual.blocks.6.attn.qkv": {
2184
+ "quant_algo": "FP8_DYNAMIC"
2185
+ },
2186
+ "model.visual.blocks.6.attn.qkv_proj": {
2187
+ "quant_algo": "FP8_DYNAMIC"
2188
+ },
2189
+ "model.visual.blocks.6.mlp.linear_fc1": {
2190
+ "quant_algo": "FP8_DYNAMIC"
2191
+ },
2192
+ "model.visual.blocks.6.mlp.linear_fc2": {
2193
+ "quant_algo": "FP8_DYNAMIC"
2194
+ },
2195
+ "model.visual.blocks.7.attn.proj": {
2196
+ "quant_algo": "FP8_DYNAMIC"
2197
+ },
2198
+ "model.visual.blocks.7.attn.qkv": {
2199
+ "quant_algo": "FP8_DYNAMIC"
2200
+ },
2201
+ "model.visual.blocks.7.attn.qkv_proj": {
2202
+ "quant_algo": "FP8_DYNAMIC"
2203
+ },
2204
+ "model.visual.blocks.7.mlp.linear_fc1": {
2205
+ "quant_algo": "FP8_DYNAMIC"
2206
+ },
2207
+ "model.visual.blocks.7.mlp.linear_fc2": {
2208
+ "quant_algo": "FP8_DYNAMIC"
2209
+ },
2210
+ "model.visual.blocks.8.attn.proj": {
2211
+ "quant_algo": "FP8_DYNAMIC"
2212
+ },
2213
+ "model.visual.blocks.8.attn.qkv": {
2214
+ "quant_algo": "FP8_DYNAMIC"
2215
+ },
2216
+ "model.visual.blocks.8.attn.qkv_proj": {
2217
+ "quant_algo": "FP8_DYNAMIC"
2218
+ },
2219
+ "model.visual.blocks.8.mlp.linear_fc1": {
2220
+ "quant_algo": "FP8_DYNAMIC"
2221
+ },
2222
+ "model.visual.blocks.8.mlp.linear_fc2": {
2223
+ "quant_algo": "FP8_DYNAMIC"
2224
+ },
2225
+ "model.visual.blocks.9.attn.proj": {
2226
+ "quant_algo": "FP8_DYNAMIC"
2227
+ },
2228
+ "model.visual.blocks.9.attn.qkv": {
2229
+ "quant_algo": "FP8_DYNAMIC"
2230
+ },
2231
+ "model.visual.blocks.9.attn.qkv_proj": {
2232
+ "quant_algo": "FP8_DYNAMIC"
2233
+ },
2234
+ "model.visual.blocks.9.mlp.linear_fc1": {
2235
+ "quant_algo": "FP8_DYNAMIC"
2236
+ },
2237
+ "model.visual.blocks.9.mlp.linear_fc2": {
2238
+ "quant_algo": "FP8_DYNAMIC"
2239
+ },
2240
+ "model.visual.merger.linear_fc1": {
2241
+ "quant_algo": "FP8_DYNAMIC"
2242
+ },
2243
+ "model.visual.merger.linear_fc2": {
2244
+ "quant_algo": "FP8_DYNAMIC"
2245
+ },
2246
+ "model.visual.pos_embed": {
2247
+ "quant_algo": "FP8_ROW"
2248
+ },
2249
+ "model.visual.patch_embed.proj": {
2250
+ "quant_algo": "FP8_DYNAMIC"
2251
+ }
2252
+ }
2253
+ }
2254
+ }