Hugging Face's logo

jpbwin
/
KAT-Coder-V2.5-Dev-int8-AutoRound

Text Generation
Transformers
Safetensors
English
qwen3_5_moe
image-text-to-text
quantized
auto-round
w8a16
Mixture of Experts
code
coding
agent
agentic-coding
conversational
8-bit precision
Model card Files
xet
 Community
jpbwin commited on
Commit
27994e7
·
verified ·
1 Parent(s): 053e37e

Upload folder using huggingface_hub

Browse files
Files changed (19) hide show
  1. .gitattributes +1 -0
  2. README.md +62 -0
  3. chat_template.jinja +154 -0
  4. config.json +788 -0
  5. generation_config.json +13 -0
  6. model-00001-of-00009.safetensors +3 -0
  7. model-00002-of-00009.safetensors +3 -0
  8. model-00003-of-00009.safetensors +3 -0
  9. model-00004-of-00009.safetensors +3 -0
  10. model-00005-of-00009.safetensors +3 -0
  11. model-00006-of-00009.safetensors +3 -0
  12. model-00007-of-00009.safetensors +3 -0
  13. model-00008-of-00009.safetensors +3 -0
  14. model-00009-of-00009.safetensors +3 -0
  15. model.safetensors.index.json +0 -0
  16. preprocessor_config.json +21 -0
  17. quantization_config.json +334 -0
  18. tokenizer.json +3 -0
  19. tokenizer_config.json +32 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - quantized
4
+ - auto-round
5
+ - w8a16
6
+ - moe
7
+ - qwen3.5-moe
8
+ - code-generation
9
+ license: apache-2.0
10
+ language:
11
+ - en
12
+ library_name: transformers
13
+ pipeline_tag: text-generation
14
+ base_model: Kwai-Kolors/KAT-Coder-V2.5-Dev
15
+ ---
16
+
17
+ # KAT-Coder-V2.5-Dev W8A128
18
+
19
+ Weight-only 8-bit quant of [KAT-Coder-V2.5-Dev](https://huggingface.co/Kwai-Kolors/KAT-Coder-V2.5-Dev) using AutoRound v0.15.0, quantized at int8 against a python focused sampleset.
20
+
21
+ ## Under the hood
22
+
23
+ 256-expert MoE on Qwen3.5. 40 layers, 30 use linear attention and 10 use full attention (every 4th). Shared expert gates kept at FP16.
24
+
25
+ Quantization is symmetric INT8, group size 128. Calibrated on 384 samples over 400 iterations with sequence length 4096 instead of the default 2048.
26
+
27
+ | KL | Top-1 match | Top-1 in ref top-5 | Tokens |
28
+ |---|---|---|---|
29
+ | 0.0047 | 96.84% | 99.97% | 16,336 |
30
+
31
+
32
+ (Measured against the original BF16 checkpoint on Wikitext-103 validation)
33
+
34
+ ## Hardware
35
+
36
+ Fits on two 3090s with headroom. This quant was created largely to fit this into two 24gb cards while maintaining speed. On my machine, this retains enough space for 3 `max-num-seqs` at full context. Tinker as you see fit to get the number of parallel slots you'd like to serve.
37
+
38
+ ## Inference
39
+
40
+ vLLM V1 engine:
41
+
42
+ ```bash
43
+ PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
44
+ vllm serve \
45
+ ./KAT-Coder-V2.5-Dev-W8A128 \
46
+ --port 5001 \
47
+ --tensor-parallel-size 2 \
48
+ --gpu-memory-utilization 0.975 \
49
+ --max-num-seqs 3 \
50
+ --enable-chunked-prefill \
51
+ --enable-prefix-caching
52
+ ```
53
+
54
+ ## Notes
55
+
56
+ I had to alter the auto-round library **hella** in order to get this to work end to end, but in the end, vanilla VLLM serves this just fine :)
57
+
58
+ Upstream PRs to auto-round to come, it seems to do a lot of double work and underutilizes gpu capability when quantizing some models.
59
+
60
+ ## License
61
+
62
+ Apache 2.0
chat_template.jinja ADDED
@@ -0,0 +1,154 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- set image_count = namespace(value=0) %}
2
+ {%- set video_count = namespace(value=0) %}
3
+ {%- macro render_content(content, do_vision_count, is_system_content=false) %}
4
+ {%- if content is string %}
5
+ {{- content }}
6
+ {%- elif content is iterable and content is not mapping %}
7
+ {%- for item in content %}
8
+ {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
9
+ {%- if is_system_content %}
10
+ {{- raise_exception('System message cannot contain images.') }}
11
+ {%- endif %}
12
+ {%- if do_vision_count %}
13
+ {%- set image_count.value = image_count.value + 1 %}
14
+ {%- endif %}
15
+ {%- if add_vision_id %}
16
+ {{- 'Picture ' ~ image_count.value ~ ': ' }}
17
+ {%- endif %}
18
+ {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
19
+ {%- elif 'video' in item or item.type == 'video' %}
20
+ {%- if is_system_content %}
21
+ {{- raise_exception('System message cannot contain videos.') }}
22
+ {%- endif %}
23
+ {%- if do_vision_count %}
24
+ {%- set video_count.value = video_count.value + 1 %}
25
+ {%- endif %}
26
+ {%- if add_vision_id %}
27
+ {{- 'Video ' ~ video_count.value ~ ': ' }}
28
+ {%- endif %}
29
+ {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
30
+ {%- elif 'text' in item %}
31
+ {{- item.text }}
32
+ {%- else %}
33
+ {{- raise_exception('Unexpected item type in content.') }}
34
+ {%- endif %}
35
+ {%- endfor %}
36
+ {%- elif content is none or content is undefined %}
37
+ {{- '' }}
38
+ {%- else %}
39
+ {{- raise_exception('Unexpected content type.') }}
40
+ {%- endif %}
41
+ {%- endmacro %}
42
+ {%- if not messages %}
43
+ {{- raise_exception('No messages provided.') }}
44
+ {%- endif %}
45
+ {%- if tools and tools is iterable and tools is not mapping %}
46
+ {{- '<|im_start|>system\n' }}
47
+ {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
48
+ {%- for tool in tools %}
49
+ {{- "\n" }}
50
+ {{- tool | tojson }}
51
+ {%- endfor %}
52
+ {{- "\n</tools>" }}
53
+ {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
54
+ {%- if messages[0].role == 'system' %}
55
+ {%- set content = render_content(messages[0].content, false, true)|trim %}
56
+ {%- if content %}
57
+ {{- '\n\n' + content }}
58
+ {%- endif %}
59
+ {%- endif %}
60
+ {{- '<|im_end|>\n' }}
61
+ {%- else %}
62
+ {%- if messages[0].role == 'system' %}
63
+ {%- set content = render_content(messages[0].content, false, true)|trim %}
64
+ {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
65
+ {%- endif %}
66
+ {%- endif %}
67
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
68
+ {%- for message in messages[::-1] %}
69
+ {%- set index = (messages|length - 1) - loop.index0 %}
70
+ {%- if ns.multi_step_tool and message.role == "user" %}
71
+ {%- set content = render_content(message.content, false)|trim %}
72
+ {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
73
+ {%- set ns.multi_step_tool = false %}
74
+ {%- set ns.last_query_index = index %}
75
+ {%- endif %}
76
+ {%- endif %}
77
+ {%- endfor %}
78
+ {%- if ns.multi_step_tool %}
79
+ {{- raise_exception('No user query found in messages.') }}
80
+ {%- endif %}
81
+ {%- for message in messages %}
82
+ {%- set content = render_content(message.content, true)|trim %}
83
+ {%- if message.role == "system" %}
84
+ {%- if not loop.first %}
85
+ {{- '<|im_start|>system\n' + content + '<|im_end|>' + '\n' }}
86
+ {%- endif %}
87
+ {%- elif message.role == "user" %}
88
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
89
+ {%- elif message.role == "assistant" %}
90
+ {%- set reasoning_content = '' %}
91
+ {%- if message.reasoning_content is string %}
92
+ {%- set reasoning_content = message.reasoning_content %}
93
+ {%- else %}
94
+ {%- if '</think>' in content %}
95
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
96
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
97
+ {%- endif %}
98
+ {%- endif %}
99
+ {%- set reasoning_content = reasoning_content|trim %}
100
+ {%- if (preserve_thinking is defined and preserve_thinking is true) or (loop.index0 > ns.last_query_index) %}
101
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
102
+ {%- else %}
103
+ {{- '<|im_start|>' + message.role + '\n' + content }}
104
+ {%- endif %}
105
+ {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
106
+ {%- for tool_call in message.tool_calls %}
107
+ {%- if tool_call.function is defined %}
108
+ {%- set tool_call = tool_call.function %}
109
+ {%- endif %}
110
+ {%- if loop.first %}
111
+ {%- if content|trim %}
112
+ {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
113
+ {%- else %}
114
+ {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
115
+ {%- endif %}
116
+ {%- else %}
117
+ {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
118
+ {%- endif %}
119
+ {%- if tool_call.arguments is defined %}
120
+ {%- for args_name, args_value in tool_call.arguments|items %}
121
+ {{- '<parameter=' + args_name + '>\n' }}
122
+ {%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}
123
+ {{- args_value }}
124
+ {{- '\n</parameter>\n' }}
125
+ {%- endfor %}
126
+ {%- endif %}
127
+ {{- '</function>\n</tool_call>' }}
128
+ {%- endfor %}
129
+ {%- endif %}
130
+ {{- '<|im_end|>\n' }}
131
+ {%- elif message.role == "tool" %}
132
+ {%- if loop.previtem and loop.previtem.role != "tool" %}
133
+ {{- '<|im_start|>user' }}
134
+ {%- endif %}
135
+ {{- '\n<tool_response>\n' }}
136
+ {{- content }}
137
+ {{- '\n</tool_response>' }}
138
+ {%- if not loop.last and loop.nextitem.role != "tool" %}
139
+ {{- '<|im_end|>\n' }}
140
+ {%- elif loop.last %}
141
+ {{- '<|im_end|>\n' }}
142
+ {%- endif %}
143
+ {%- else %}
144
+ {{- raise_exception('Unexpected message role.') }}
145
+ {%- endif %}
146
+ {%- endfor %}
147
+ {%- if add_generation_prompt %}
148
+ {{- '<|im_start|>assistant\n' }}
149
+ {%- if enable_thinking is defined and enable_thinking is false %}
150
+ {{- '<think>\n\n</think>\n\n' }}
151
+ {%- else %}
152
+ {{- '<think>\n' }}
153
+ {%- endif %}
154
+ {%- endif %}
config.json ADDED
@@ -0,0 +1,788 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3_5MoeForConditionalGeneration"
4
+ ],
5
+ "image_token_id": 248056,
6
+ "model_type": "qwen3_5_moe",
7
+ "text_config": {
8
+ "attention_bias": false,
9
+ "attention_dropout": 0.0,
10
+ "attn_output_gate": true,
11
+ "bos_token_id": 248044,
12
+ "dtype": "bfloat16",
13
+ "eos_token_id": 248044,
14
+ "full_attention_interval": 4,
15
+ "head_dim": 256,
16
+ "hidden_act": "silu",
17
+ "hidden_size": 2048,
18
+ "initializer_range": 0.02,
19
+ "layer_types": [
20
+ "linear_attention",
21
+ "linear_attention",
22
+ "linear_attention",
23
+ "full_attention",
24
+ "linear_attention",
25
+ "linear_attention",
26
+ "linear_attention",
27
+ "full_attention",
28
+ "linear_attention",
29
+ "linear_attention",
30
+ "linear_attention",
31
+ "full_attention",
32
+ "linear_attention",
33
+ "linear_attention",
34
+ "linear_attention",
35
+ "full_attention",
36
+ "linear_attention",
37
+ "linear_attention",
38
+ "linear_attention",
39
+ "full_attention",
40
+ "linear_attention",
41
+ "linear_attention",
42
+ "linear_attention",
43
+ "full_attention",
44
+ "linear_attention",
45
+ "linear_attention",
46
+ "linear_attention",
47
+ "full_attention",
48
+ "linear_attention",
49
+ "linear_attention",
50
+ "linear_attention",
51
+ "full_attention",
52
+ "linear_attention",
53
+ "linear_attention",
54
+ "linear_attention",
55
+ "full_attention",
56
+ "linear_attention",
57
+ "linear_attention",
58
+ "linear_attention",
59
+ "full_attention"
60
+ ],
61
+ "linear_conv_kernel_dim": 4,
62
+ "linear_key_head_dim": 128,
63
+ "linear_num_key_heads": 16,
64
+ "linear_num_value_heads": 32,
65
+ "linear_value_head_dim": 128,
66
+ "mamba_ssm_dtype": "float32",
67
+ "max_position_embeddings": 262144,
68
+ "model_type": "qwen3_5_moe_text",
69
+ "moe_intermediate_size": 512,
70
+ "mtp_num_hidden_layers": 0,
71
+ "mtp_use_dedicated_embeddings": false,
72
+ "num_attention_heads": 16,
73
+ "num_experts": 256,
74
+ "num_experts_per_tok": 8,
75
+ "num_hidden_layers": 40,
76
+ "num_key_value_heads": 2,
77
+ "output_router_logits": false,
78
+ "pad_token_id": null,
79
+ "partial_rotary_factor": 0.25,
80
+ "quantization_config": {
81
+ "autoround_version": "0.15.0",
82
+ "bits": 8,
83
+ "block_name_to_quantize": "model.language_model.layers",
84
+ "data_type": "int",
85
+ "extra_config": {
86
+ ".*model\\.layers\\.0\\.mlp\\.gate.*": {
87
+ "bits": 16,
88
+ "data_type": "float"
89
+ },
90
+ ".*model\\.layers\\.10\\.mlp\\.gate.*": {
91
+ "bits": 16,
92
+ "data_type": "float"
93
+ },
94
+ ".*model\\.layers\\.11\\.mlp\\.gate.*": {
95
+ "bits": 16,
96
+ "data_type": "float"
97
+ },
98
+ ".*model\\.layers\\.12\\.mlp\\.gate.*": {
99
+ "bits": 16,
100
+ "data_type": "float"
101
+ },
102
+ ".*model\\.layers\\.13\\.mlp\\.gate.*": {
103
+ "bits": 16,
104
+ "data_type": "float"
105
+ },
106
+ ".*model\\.layers\\.14\\.mlp\\.gate.*": {
107
+ "bits": 16,
108
+ "data_type": "float"
109
+ },
110
+ ".*model\\.layers\\.15\\.mlp\\.gate.*": {
111
+ "bits": 16,
112
+ "data_type": "float"
113
+ },
114
+ ".*model\\.layers\\.16\\.mlp\\.gate.*": {
115
+ "bits": 16,
116
+ "data_type": "float"
117
+ },
118
+ ".*model\\.layers\\.17\\.mlp\\.gate.*": {
119
+ "bits": 16,
120
+ "data_type": "float"
121
+ },
122
+ ".*model\\.layers\\.18\\.mlp\\.gate.*": {
123
+ "bits": 16,
124
+ "data_type": "float"
125
+ },
126
+ ".*model\\.layers\\.19\\.mlp\\.gate.*": {
127
+ "bits": 16,
128
+ "data_type": "float"
129
+ },
130
+ ".*model\\.layers\\.1\\.mlp\\.gate.*": {
131
+ "bits": 16,
132
+ "data_type": "float"
133
+ },
134
+ ".*model\\.layers\\.20\\.mlp\\.gate.*": {
135
+ "bits": 16,
136
+ "data_type": "float"
137
+ },
138
+ ".*model\\.layers\\.21\\.mlp\\.gate.*": {
139
+ "bits": 16,
140
+ "data_type": "float"
141
+ },
142
+ ".*model\\.layers\\.22\\.mlp\\.gate.*": {
143
+ "bits": 16,
144
+ "data_type": "float"
145
+ },
146
+ ".*model\\.layers\\.23\\.mlp\\.gate.*": {
147
+ "bits": 16,
148
+ "data_type": "float"
149
+ },
150
+ ".*model\\.layers\\.24\\.mlp\\.gate.*": {
151
+ "bits": 16,
152
+ "data_type": "float"
153
+ },
154
+ ".*model\\.layers\\.25\\.mlp\\.gate.*": {
155
+ "bits": 16,
156
+ "data_type": "float"
157
+ },
158
+ ".*model\\.layers\\.26\\.mlp\\.gate.*": {
159
+ "bits": 16,
160
+ "data_type": "float"
161
+ },
162
+ ".*model\\.layers\\.27\\.mlp\\.gate.*": {
163
+ "bits": 16,
164
+ "data_type": "float"
165
+ },
166
+ ".*model\\.layers\\.28\\.mlp\\.gate.*": {
167
+ "bits": 16,
168
+ "data_type": "float"
169
+ },
170
+ ".*model\\.layers\\.29\\.mlp\\.gate.*": {
171
+ "bits": 16,
172
+ "data_type": "float"
173
+ },
174
+ ".*model\\.layers\\.2\\.mlp\\.gate.*": {
175
+ "bits": 16,
176
+ "data_type": "float"
177
+ },
178
+ ".*model\\.layers\\.30\\.mlp\\.gate.*": {
179
+ "bits": 16,
180
+ "data_type": "float"
181
+ },
182
+ ".*model\\.layers\\.31\\.mlp\\.gate.*": {
183
+ "bits": 16,
184
+ "data_type": "float"
185
+ },
186
+ ".*model\\.layers\\.32\\.mlp\\.gate.*": {
187
+ "bits": 16,
188
+ "data_type": "float"
189
+ },
190
+ ".*model\\.layers\\.33\\.mlp\\.gate.*": {
191
+ "bits": 16,
192
+ "data_type": "float"
193
+ },
194
+ ".*model\\.layers\\.34\\.mlp\\.gate.*": {
195
+ "bits": 16,
196
+ "data_type": "float"
197
+ },
198
+ ".*model\\.layers\\.35\\.mlp\\.gate.*": {
199
+ "bits": 16,
200
+ "data_type": "float"
201
+ },
202
+ ".*model\\.layers\\.36\\.mlp\\.gate.*": {
203
+ "bits": 16,
204
+ "data_type": "float"
205
+ },
206
+ ".*model\\.layers\\.37\\.mlp\\.gate.*": {
207
+ "bits": 16,
208
+ "data_type": "float"
209
+ },
210
+ ".*model\\.layers\\.38\\.mlp\\.gate.*": {
211
+ "bits": 16,
212
+ "data_type": "float"
213
+ },
214
+ ".*model\\.layers\\.39\\.mlp\\.gate.*": {
215
+ "bits": 16,
216
+ "data_type": "float"
217
+ },
218
+ ".*model\\.layers\\.3\\.mlp\\.gate.*": {
219
+ "bits": 16,
220
+ "data_type": "float"
221
+ },
222
+ ".*model\\.layers\\.4\\.mlp\\.gate.*": {
223
+ "bits": 16,
224
+ "data_type": "float"
225
+ },
226
+ ".*model\\.layers\\.5\\.mlp\\.gate.*": {
227
+ "bits": 16,
228
+ "data_type": "float"
229
+ },
230
+ ".*model\\.layers\\.6\\.mlp\\.gate.*": {
231
+ "bits": 16,
232
+ "data_type": "float"
233
+ },
234
+ ".*model\\.layers\\.7\\.mlp\\.gate.*": {
235
+ "bits": 16,
236
+ "data_type": "float"
237
+ },
238
+ ".*model\\.layers\\.8\\.mlp\\.gate.*": {
239
+ "bits": 16,
240
+ "data_type": "float"
241
+ },
242
+ ".*model\\.layers\\.9\\.mlp\\.gate.*": {
243
+ "bits": 16,
244
+ "data_type": "float"
245
+ },
246
+ "model.layers.0.mlp.shared_expert_gate": {
247
+ "bits": 16,
248
+ "data_type": "fp"
249
+ },
250
+ "model.layers.1.mlp.shared_expert_gate": {
251
+ "bits": 16,
252
+ "data_type": "fp"
253
+ },
254
+ "model.layers.10.mlp.shared_expert_gate": {
255
+ "bits": 16,
256
+ "data_type": "fp"
257
+ },
258
+ "model.layers.11.mlp.shared_expert_gate": {
259
+ "bits": 16,
260
+ "data_type": "fp"
261
+ },
262
+ "model.layers.12.mlp.shared_expert_gate": {
263
+ "bits": 16,
264
+ "data_type": "fp"
265
+ },
266
+ "model.layers.13.mlp.shared_expert_gate": {
267
+ "bits": 16,
268
+ "data_type": "fp"
269
+ },
270
+ "model.layers.14.mlp.shared_expert_gate": {
271
+ "bits": 16,
272
+ "data_type": "fp"
273
+ },
274
+ "model.layers.15.mlp.shared_expert_gate": {
275
+ "bits": 16,
276
+ "data_type": "fp"
277
+ },
278
+ "model.layers.16.mlp.shared_expert_gate": {
279
+ "bits": 16,
280
+ "data_type": "fp"
281
+ },
282
+ "model.layers.17.mlp.shared_expert_gate": {
283
+ "bits": 16,
284
+ "data_type": "fp"
285
+ },
286
+ "model.layers.18.mlp.shared_expert_gate": {
287
+ "bits": 16,
288
+ "data_type": "fp"
289
+ },
290
+ "model.layers.19.mlp.shared_expert_gate": {
291
+ "bits": 16,
292
+ "data_type": "fp"
293
+ },
294
+ "model.layers.2.mlp.shared_expert_gate": {
295
+ "bits": 16,
296
+ "data_type": "fp"
297
+ },
298
+ "model.layers.20.mlp.shared_expert_gate": {
299
+ "bits": 16,
300
+ "data_type": "fp"
301
+ },
302
+ "model.layers.21.mlp.shared_expert_gate": {
303
+ "bits": 16,
304
+ "data_type": "fp"
305
+ },
306
+ "model.layers.22.mlp.shared_expert_gate": {
307
+ "bits": 16,
308
+ "data_type": "fp"
309
+ },
310
+ "model.layers.23.mlp.shared_expert_gate": {
311
+ "bits": 16,
312
+ "data_type": "fp"
313
+ },
314
+ "model.layers.24.mlp.shared_expert_gate": {
315
+ "bits": 16,
316
+ "data_type": "fp"
317
+ },
318
+ "model.layers.25.mlp.shared_expert_gate": {
319
+ "bits": 16,
320
+ "data_type": "fp"
321
+ },
322
+ "model.layers.26.mlp.shared_expert_gate": {
323
+ "bits": 16,
324
+ "data_type": "fp"
325
+ },
326
+ "model.layers.27.mlp.shared_expert_gate": {
327
+ "bits": 16,
328
+ "data_type": "fp"
329
+ },
330
+ "model.layers.28.mlp.shared_expert_gate": {
331
+ "bits": 16,
332
+ "data_type": "fp"
333
+ },
334
+ "model.layers.29.mlp.shared_expert_gate": {
335
+ "bits": 16,
336
+ "data_type": "fp"
337
+ },
338
+ "model.layers.3.mlp.shared_expert_gate": {
339
+ "bits": 16,
340
+ "data_type": "fp"
341
+ },
342
+ "model.layers.30.mlp.shared_expert_gate": {
343
+ "bits": 16,
344
+ "data_type": "fp"
345
+ },
346
+ "model.layers.31.mlp.shared_expert_gate": {
347
+ "bits": 16,
348
+ "data_type": "fp"
349
+ },
350
+ "model.layers.32.mlp.shared_expert_gate": {
351
+ "bits": 16,
352
+ "data_type": "fp"
353
+ },
354
+ "model.layers.33.mlp.shared_expert_gate": {
355
+ "bits": 16,
356
+ "data_type": "fp"
357
+ },
358
+ "model.layers.34.mlp.shared_expert_gate": {
359
+ "bits": 16,
360
+ "data_type": "fp"
361
+ },
362
+ "model.layers.35.mlp.shared_expert_gate": {
363
+ "bits": 16,
364
+ "data_type": "fp"
365
+ },
366
+ "model.layers.36.mlp.shared_expert_gate": {
367
+ "bits": 16,
368
+ "data_type": "fp"
369
+ },
370
+ "model.layers.37.mlp.shared_expert_gate": {
371
+ "bits": 16,
372
+ "data_type": "fp"
373
+ },
374
+ "model.layers.38.mlp.shared_expert_gate": {
375
+ "bits": 16,
376
+ "data_type": "fp"
377
+ },
378
+ "model.layers.39.mlp.shared_expert_gate": {
379
+ "bits": 16,
380
+ "data_type": "fp"
381
+ },
382
+ "model.layers.4.mlp.shared_expert_gate": {
383
+ "bits": 16,
384
+ "data_type": "fp"
385
+ },
386
+ "model.layers.5.mlp.shared_expert_gate": {
387
+ "bits": 16,
388
+ "data_type": "fp"
389
+ },
390
+ "model.layers.6.mlp.shared_expert_gate": {
391
+ "bits": 16,
392
+ "data_type": "fp"
393
+ },
394
+ "model.layers.7.mlp.shared_expert_gate": {
395
+ "bits": 16,
396
+ "data_type": "fp"
397
+ },
398
+ "model.layers.8.mlp.shared_expert_gate": {
399
+ "bits": 16,
400
+ "data_type": "fp"
401
+ },
402
+ "model.layers.9.mlp.shared_expert_gate": {
403
+ "bits": 16,
404
+ "data_type": "fp"
405
+ }
406
+ },
407
+ "group_size": 128,
408
+ "iters": 400,
409
+ "low_gpu_mem_usage": true,
410
+ "packing_format": "auto_round:auto_gptq",
411
+ "quant_method": "auto-round",
412
+ "sym": true
413
+ },
414
+ "rms_norm_eps": 1e-06,
415
+ "rope_parameters": {
416
+ "mrope_interleaved": true,
417
+ "mrope_section": [
418
+ 11,
419
+ 11,
420
+ 10
421
+ ],
422
+ "partial_rotary_factor": 0.25,
423
+ "rope_theta": 10000000,
424
+ "rope_type": "default"
425
+ },
426
+ "router_aux_loss_coef": 0.001,
427
+ "shared_expert_intermediate_size": 512,
428
+ "tie_word_embeddings": false,
429
+ "transformers_version": "5.14.1",
430
+ "use_cache": true,
431
+ "vocab_size": 248320
432
+ },
433
+ "tie_word_embeddings": false,
434
+ "transformers_version": "4.57.1",
435
+ "video_token_id": 248057,
436
+ "vision_config": {
437
+ "deepstack_visual_indexes": [],
438
+ "depth": 27,
439
+ "hidden_act": "gelu_pytorch_tanh",
440
+ "hidden_size": 1152,
441
+ "in_channels": 3,
442
+ "initializer_range": 0.02,
443
+ "intermediate_size": 4304,
444
+ "model_type": "qwen3_5_moe",
445
+ "num_heads": 16,
446
+ "num_position_embeddings": 2304,
447
+ "out_hidden_size": 2048,
448
+ "patch_size": 16,
449
+ "spatial_merge_size": 2,
450
+ "temporal_patch_size": 2
451
+ },
452
+ "vision_end_token_id": 248054,
453
+ "vision_start_token_id": 248053,
454
+ "quantization_config": {
455
+ "bits": 8,
456
+ "data_type": "int",
457
+ "group_size": 128,
458
+ "sym": true,
459
+ "iters": 400,
460
+ "low_gpu_mem_usage": true,
461
+ "autoround_version": "0.15.0",
462
+ "block_name_to_quantize": "model.language_model.layers",
463
+ "quant_method": "auto-round",
464
+ "packing_format": "auto_round:auto_gptq",
465
+ "extra_config": {
466
+ "model.layers.0.mlp.shared_expert_gate": {
467
+ "bits": 16,
468
+ "data_type": "fp"
469
+ },
470
+ "model.layers.1.mlp.shared_expert_gate": {
471
+ "bits": 16,
472
+ "data_type": "fp"
473
+ },
474
+ "model.layers.2.mlp.shared_expert_gate": {
475
+ "bits": 16,
476
+ "data_type": "fp"
477
+ },
478
+ "model.layers.3.mlp.shared_expert_gate": {
479
+ "bits": 16,
480
+ "data_type": "fp"
481
+ },
482
+ "model.layers.4.mlp.shared_expert_gate": {
483
+ "bits": 16,
484
+ "data_type": "fp"
485
+ },
486
+ "model.layers.5.mlp.shared_expert_gate": {
487
+ "bits": 16,
488
+ "data_type": "fp"
489
+ },
490
+ "model.layers.6.mlp.shared_expert_gate": {
491
+ "bits": 16,
492
+ "data_type": "fp"
493
+ },
494
+ "model.layers.7.mlp.shared_expert_gate": {
495
+ "bits": 16,
496
+ "data_type": "fp"
497
+ },
498
+ "model.layers.8.mlp.shared_expert_gate": {
499
+ "bits": 16,
500
+ "data_type": "fp"
501
+ },
502
+ "model.layers.9.mlp.shared_expert_gate": {
503
+ "bits": 16,
504
+ "data_type": "fp"
505
+ },
506
+ "model.layers.10.mlp.shared_expert_gate": {
507
+ "bits": 16,
508
+ "data_type": "fp"
509
+ },
510
+ "model.layers.11.mlp.shared_expert_gate": {
511
+ "bits": 16,
512
+ "data_type": "fp"
513
+ },
514
+ "model.layers.12.mlp.shared_expert_gate": {
515
+ "bits": 16,
516
+ "data_type": "fp"
517
+ },
518
+ "model.layers.13.mlp.shared_expert_gate": {
519
+ "bits": 16,
520
+ "data_type": "fp"
521
+ },
522
+ "model.layers.14.mlp.shared_expert_gate": {
523
+ "bits": 16,
524
+ "data_type": "fp"
525
+ },
526
+ "model.layers.15.mlp.shared_expert_gate": {
527
+ "bits": 16,
528
+ "data_type": "fp"
529
+ },
530
+ "model.layers.16.mlp.shared_expert_gate": {
531
+ "bits": 16,
532
+ "data_type": "fp"
533
+ },
534
+ "model.layers.17.mlp.shared_expert_gate": {
535
+ "bits": 16,
536
+ "data_type": "fp"
537
+ },
538
+ "model.layers.18.mlp.shared_expert_gate": {
539
+ "bits": 16,
540
+ "data_type": "fp"
541
+ },
542
+ "model.layers.19.mlp.shared_expert_gate": {
543
+ "bits": 16,
544
+ "data_type": "fp"
545
+ },
546
+ "model.layers.20.mlp.shared_expert_gate": {
547
+ "bits": 16,
548
+ "data_type": "fp"
549
+ },
550
+ "model.layers.21.mlp.shared_expert_gate": {
551
+ "bits": 16,
552
+ "data_type": "fp"
553
+ },
554
+ "model.layers.22.mlp.shared_expert_gate": {
555
+ "bits": 16,
556
+ "data_type": "fp"
557
+ },
558
+ "model.layers.23.mlp.shared_expert_gate": {
559
+ "bits": 16,
560
+ "data_type": "fp"
561
+ },
562
+ "model.layers.24.mlp.shared_expert_gate": {
563
+ "bits": 16,
564
+ "data_type": "fp"
565
+ },
566
+ "model.layers.25.mlp.shared_expert_gate": {
567
+ "bits": 16,
568
+ "data_type": "fp"
569
+ },
570
+ "model.layers.26.mlp.shared_expert_gate": {
571
+ "bits": 16,
572
+ "data_type": "fp"
573
+ },
574
+ "model.layers.27.mlp.shared_expert_gate": {
575
+ "bits": 16,
576
+ "data_type": "fp"
577
+ },
578
+ "model.layers.28.mlp.shared_expert_gate": {
579
+ "bits": 16,
580
+ "data_type": "fp"
581
+ },
582
+ "model.layers.29.mlp.shared_expert_gate": {
583
+ "bits": 16,
584
+ "data_type": "fp"
585
+ },
586
+ "model.layers.30.mlp.shared_expert_gate": {
587
+ "bits": 16,
588
+ "data_type": "fp"
589
+ },
590
+ "model.layers.31.mlp.shared_expert_gate": {
591
+ "bits": 16,
592
+ "data_type": "fp"
593
+ },
594
+ "model.layers.32.mlp.shared_expert_gate": {
595
+ "bits": 16,
596
+ "data_type": "fp"
597
+ },
598
+ "model.layers.33.mlp.shared_expert_gate": {
599
+ "bits": 16,
600
+ "data_type": "fp"
601
+ },
602
+ "model.layers.34.mlp.shared_expert_gate": {
603
+ "bits": 16,
604
+ "data_type": "fp"
605
+ },
606
+ "model.layers.35.mlp.shared_expert_gate": {
607
+ "bits": 16,
608
+ "data_type": "fp"
609
+ },
610
+ "model.layers.36.mlp.shared_expert_gate": {
611
+ "bits": 16,
612
+ "data_type": "fp"
613
+ },
614
+ "model.layers.37.mlp.shared_expert_gate": {
615
+ "bits": 16,
616
+ "data_type": "fp"
617
+ },
618
+ "model.layers.38.mlp.shared_expert_gate": {
619
+ "bits": 16,
620
+ "data_type": "fp"
621
+ },
622
+ "model.layers.39.mlp.shared_expert_gate": {
623
+ "bits": 16,
624
+ "data_type": "fp"
625
+ },
626
+ ".*model\\.layers\\.0\\.mlp\\.gate.*": {
627
+ "bits": 16,
628
+ "data_type": "float"
629
+ },
630
+ ".*model\\.layers\\.1\\.mlp\\.gate.*": {
631
+ "bits": 16,
632
+ "data_type": "float"
633
+ },
634
+ ".*model\\.layers\\.2\\.mlp\\.gate.*": {
635
+ "bits": 16,
636
+ "data_type": "float"
637
+ },
638
+ ".*model\\.layers\\.3\\.mlp\\.gate.*": {
639
+ "bits": 16,
640
+ "data_type": "float"
641
+ },
642
+ ".*model\\.layers\\.4\\.mlp\\.gate.*": {
643
+ "bits": 16,
644
+ "data_type": "float"
645
+ },
646
+ ".*model\\.layers\\.5\\.mlp\\.gate.*": {
647
+ "bits": 16,
648
+ "data_type": "float"
649
+ },
650
+ ".*model\\.layers\\.6\\.mlp\\.gate.*": {
651
+ "bits": 16,
652
+ "data_type": "float"
653
+ },
654
+ ".*model\\.layers\\.7\\.mlp\\.gate.*": {
655
+ "bits": 16,
656
+ "data_type": "float"
657
+ },
658
+ ".*model\\.layers\\.8\\.mlp\\.gate.*": {
659
+ "bits": 16,
660
+ "data_type": "float"
661
+ },
662
+ ".*model\\.layers\\.9\\.mlp\\.gate.*": {
663
+ "bits": 16,
664
+ "data_type": "float"
665
+ },
666
+ ".*model\\.layers\\.10\\.mlp\\.gate.*": {
667
+ "bits": 16,
668
+ "data_type": "float"
669
+ },
670
+ ".*model\\.layers\\.11\\.mlp\\.gate.*": {
671
+ "bits": 16,
672
+ "data_type": "float"
673
+ },
674
+ ".*model\\.layers\\.12\\.mlp\\.gate.*": {
675
+ "bits": 16,
676
+ "data_type": "float"
677
+ },
678
+ ".*model\\.layers\\.13\\.mlp\\.gate.*": {
679
+ "bits": 16,
680
+ "data_type": "float"
681
+ },
682
+ ".*model\\.layers\\.14\\.mlp\\.gate.*": {
683
+ "bits": 16,
684
+ "data_type": "float"
685
+ },
686
+ ".*model\\.layers\\.15\\.mlp\\.gate.*": {
687
+ "bits": 16,
688
+ "data_type": "float"
689
+ },
690
+ ".*model\\.layers\\.16\\.mlp\\.gate.*": {
691
+ "bits": 16,
692
+ "data_type": "float"
693
+ },
694
+ ".*model\\.layers\\.17\\.mlp\\.gate.*": {
695
+ "bits": 16,
696
+ "data_type": "float"
697
+ },
698
+ ".*model\\.layers\\.18\\.mlp\\.gate.*": {
699
+ "bits": 16,
700
+ "data_type": "float"
701
+ },
702
+ ".*model\\.layers\\.19\\.mlp\\.gate.*": {
703
+ "bits": 16,
704
+ "data_type": "float"
705
+ },
706
+ ".*model\\.layers\\.20\\.mlp\\.gate.*": {
707
+ "bits": 16,
708
+ "data_type": "float"
709
+ },
710
+ ".*model\\.layers\\.21\\.mlp\\.gate.*": {
711
+ "bits": 16,
712
+ "data_type": "float"
713
+ },
714
+ ".*model\\.layers\\.22\\.mlp\\.gate.*": {
715
+ "bits": 16,
716
+ "data_type": "float"
717
+ },
718
+ ".*model\\.layers\\.23\\.mlp\\.gate.*": {
719
+ "bits": 16,
720
+ "data_type": "float"
721
+ },
722
+ ".*model\\.layers\\.24\\.mlp\\.gate.*": {
723
+ "bits": 16,
724
+ "data_type": "float"
725
+ },
726
+ ".*model\\.layers\\.25\\.mlp\\.gate.*": {
727
+ "bits": 16,
728
+ "data_type": "float"
729
+ },
730
+ ".*model\\.layers\\.26\\.mlp\\.gate.*": {
731
+ "bits": 16,
732
+ "data_type": "float"
733
+ },
734
+ ".*model\\.layers\\.27\\.mlp\\.gate.*": {
735
+ "bits": 16,
736
+ "data_type": "float"
737
+ },
738
+ ".*model\\.layers\\.28\\.mlp\\.gate.*": {
739
+ "bits": 16,
740
+ "data_type": "float"
741
+ },
742
+ ".*model\\.layers\\.29\\.mlp\\.gate.*": {
743
+ "bits": 16,
744
+ "data_type": "float"
745
+ },
746
+ ".*model\\.layers\\.30\\.mlp\\.gate.*": {
747
+ "bits": 16,
748
+ "data_type": "float"
749
+ },
750
+ ".*model\\.layers\\.31\\.mlp\\.gate.*": {
751
+ "bits": 16,
752
+ "data_type": "float"
753
+ },
754
+ ".*model\\.layers\\.32\\.mlp\\.gate.*": {
755
+ "bits": 16,
756
+ "data_type": "float"
757
+ },
758
+ ".*model\\.layers\\.33\\.mlp\\.gate.*": {
759
+ "bits": 16,
760
+ "data_type": "float"
761
+ },
762
+ ".*model\\.layers\\.34\\.mlp\\.gate.*": {
763
+ "bits": 16,
764
+ "data_type": "float"
765
+ },
766
+ ".*model\\.layers\\.35\\.mlp\\.gate.*": {
767
+ "bits": 16,
768
+ "data_type": "float"
769
+ },
770
+ ".*model\\.layers\\.36\\.mlp\\.gate.*": {
771
+ "bits": 16,
772
+ "data_type": "float"
773
+ },
774
+ ".*model\\.layers\\.37\\.mlp\\.gate.*": {
775
+ "bits": 16,
776
+ "data_type": "float"
777
+ },
778
+ ".*model\\.layers\\.38\\.mlp\\.gate.*": {
779
+ "bits": 16,
780
+ "data_type": "float"
781
+ },
782
+ ".*model\\.layers\\.39\\.mlp\\.gate.*": {
783
+ "bits": 16,
784
+ "data_type": "float"
785
+ }
786
+ }
787
+ }
788
+ }
generation_config.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 248044,
3
+ "do_sample": true,
4
+ "eos_token_id": [
5
+ 248046,
6
+ 248044
7
+ ],
8
+ "pad_token_id": 248044,
9
+ "temperature": 1.0,
10
+ "top_k": 20,
11
+ "top_p": 0.95,
12
+ "transformers_version": "5.14.1"
13
+ }
model-00001-of-00009.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a9bf151827b915743ba25c3a1db94e28b3f2e7b883f3bfd2df1b5b0056d2c43f
3
+ size 4295993088
model-00002-of-00009.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e144afb9e5644e61642ee8a866c0f9212cc0da4d2feae22acfaac437c5ac23ce
3
+ size 4296005760
model-00003-of-00009.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b5d34b47dad6d8043463282b90a338ed4743006aec85255939f5933b7fdd0bf
3
+ size 4296017216
model-00004-of-00009.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:325f0dca5b5c4ca7f3ecdd31eacdd382101f29fbaa2004f6a757eddafb509866
3
+ size 4295819992
model-00005-of-00009.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75b2b8017ce809c94962cc2ea8c4695ed9aeefb86caa78f231d521d42b307583
3
+ size 4296017392
model-00006-of-00009.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b468eef41f41d7d3adc3b731610c9fb7f46eda20598e3ecc91e3b0c84b15001b
3
+ size 4296017392
model-00007-of-00009.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f5880c588d29111874d06024f4880991f77be17ae1eedb8e00f437eeba6b4267
3
+ size 4296017248
model-00008-of-00009.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:29d196d878dd0eec4acc454044ca0e14b909882e7045c0834bbfed526c509393
3
+ size 4295820000
model-00009-of-00009.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f487878680bf42ab24c7f03c26619091515c225b5aafacae02495789bcd8ef8
3
+ size 2133021848
model.safetensors.index.json ADDED
The diff for this file is too large to render. See raw diff
 
preprocessor_config.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "size": {
3
+ "longest_edge": 16777216,
4
+ "shortest_edge": 65536
5
+ },
6
+ "patch_size": 16,
7
+ "temporal_patch_size": 2,
8
+ "merge_size": 2,
9
+ "image_mean": [
10
+ 0.5,
11
+ 0.5,
12
+ 0.5
13
+ ],
14
+ "image_std": [
15
+ 0.5,
16
+ 0.5,
17
+ 0.5
18
+ ],
19
+ "processor_class": "Qwen3VLProcessor",
20
+ "image_processor_type": "Qwen2VLImageProcessorFast"
21
+ }
quantization_config.json ADDED
@@ -0,0 +1,334 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bits": 8,
3
+ "data_type": "int",
4
+ "group_size": 128,
5
+ "sym": true,
6
+ "iters": 400,
7
+ "low_gpu_mem_usage": true,
8
+ "autoround_version": "0.15.0",
9
+ "block_name_to_quantize": "model.language_model.layers",
10
+ "quant_method": "auto-round",
11
+ "packing_format": "auto_round:auto_gptq",
12
+ "extra_config": {
13
+ "model.layers.0.mlp.shared_expert_gate": {
14
+ "bits": 16,
15
+ "data_type": "fp"
16
+ },
17
+ "model.layers.1.mlp.shared_expert_gate": {
18
+ "bits": 16,
19
+ "data_type": "fp"
20
+ },
21
+ "model.layers.2.mlp.shared_expert_gate": {
22
+ "bits": 16,
23
+ "data_type": "fp"
24
+ },
25
+ "model.layers.3.mlp.shared_expert_gate": {
26
+ "bits": 16,
27
+ "data_type": "fp"
28
+ },
29
+ "model.layers.4.mlp.shared_expert_gate": {
30
+ "bits": 16,
31
+ "data_type": "fp"
32
+ },
33
+ "model.layers.5.mlp.shared_expert_gate": {
34
+ "bits": 16,
35
+ "data_type": "fp"
36
+ },
37
+ "model.layers.6.mlp.shared_expert_gate": {
38
+ "bits": 16,
39
+ "data_type": "fp"
40
+ },
41
+ "model.layers.7.mlp.shared_expert_gate": {
42
+ "bits": 16,
43
+ "data_type": "fp"
44
+ },
45
+ "model.layers.8.mlp.shared_expert_gate": {
46
+ "bits": 16,
47
+ "data_type": "fp"
48
+ },
49
+ "model.layers.9.mlp.shared_expert_gate": {
50
+ "bits": 16,
51
+ "data_type": "fp"
52
+ },
53
+ "model.layers.10.mlp.shared_expert_gate": {
54
+ "bits": 16,
55
+ "data_type": "fp"
56
+ },
57
+ "model.layers.11.mlp.shared_expert_gate": {
58
+ "bits": 16,
59
+ "data_type": "fp"
60
+ },
61
+ "model.layers.12.mlp.shared_expert_gate": {
62
+ "bits": 16,
63
+ "data_type": "fp"
64
+ },
65
+ "model.layers.13.mlp.shared_expert_gate": {
66
+ "bits": 16,
67
+ "data_type": "fp"
68
+ },
69
+ "model.layers.14.mlp.shared_expert_gate": {
70
+ "bits": 16,
71
+ "data_type": "fp"
72
+ },
73
+ "model.layers.15.mlp.shared_expert_gate": {
74
+ "bits": 16,
75
+ "data_type": "fp"
76
+ },
77
+ "model.layers.16.mlp.shared_expert_gate": {
78
+ "bits": 16,
79
+ "data_type": "fp"
80
+ },
81
+ "model.layers.17.mlp.shared_expert_gate": {
82
+ "bits": 16,
83
+ "data_type": "fp"
84
+ },
85
+ "model.layers.18.mlp.shared_expert_gate": {
86
+ "bits": 16,
87
+ "data_type": "fp"
88
+ },
89
+ "model.layers.19.mlp.shared_expert_gate": {
90
+ "bits": 16,
91
+ "data_type": "fp"
92
+ },
93
+ "model.layers.20.mlp.shared_expert_gate": {
94
+ "bits": 16,
95
+ "data_type": "fp"
96
+ },
97
+ "model.layers.21.mlp.shared_expert_gate": {
98
+ "bits": 16,
99
+ "data_type": "fp"
100
+ },
101
+ "model.layers.22.mlp.shared_expert_gate": {
102
+ "bits": 16,
103
+ "data_type": "fp"
104
+ },
105
+ "model.layers.23.mlp.shared_expert_gate": {
106
+ "bits": 16,
107
+ "data_type": "fp"
108
+ },
109
+ "model.layers.24.mlp.shared_expert_gate": {
110
+ "bits": 16,
111
+ "data_type": "fp"
112
+ },
113
+ "model.layers.25.mlp.shared_expert_gate": {
114
+ "bits": 16,
115
+ "data_type": "fp"
116
+ },
117
+ "model.layers.26.mlp.shared_expert_gate": {
118
+ "bits": 16,
119
+ "data_type": "fp"
120
+ },
121
+ "model.layers.27.mlp.shared_expert_gate": {
122
+ "bits": 16,
123
+ "data_type": "fp"
124
+ },
125
+ "model.layers.28.mlp.shared_expert_gate": {
126
+ "bits": 16,
127
+ "data_type": "fp"
128
+ },
129
+ "model.layers.29.mlp.shared_expert_gate": {
130
+ "bits": 16,
131
+ "data_type": "fp"
132
+ },
133
+ "model.layers.30.mlp.shared_expert_gate": {
134
+ "bits": 16,
135
+ "data_type": "fp"
136
+ },
137
+ "model.layers.31.mlp.shared_expert_gate": {
138
+ "bits": 16,
139
+ "data_type": "fp"
140
+ },
141
+ "model.layers.32.mlp.shared_expert_gate": {
142
+ "bits": 16,
143
+ "data_type": "fp"
144
+ },
145
+ "model.layers.33.mlp.shared_expert_gate": {
146
+ "bits": 16,
147
+ "data_type": "fp"
148
+ },
149
+ "model.layers.34.mlp.shared_expert_gate": {
150
+ "bits": 16,
151
+ "data_type": "fp"
152
+ },
153
+ "model.layers.35.mlp.shared_expert_gate": {
154
+ "bits": 16,
155
+ "data_type": "fp"
156
+ },
157
+ "model.layers.36.mlp.shared_expert_gate": {
158
+ "bits": 16,
159
+ "data_type": "fp"
160
+ },
161
+ "model.layers.37.mlp.shared_expert_gate": {
162
+ "bits": 16,
163
+ "data_type": "fp"
164
+ },
165
+ "model.layers.38.mlp.shared_expert_gate": {
166
+ "bits": 16,
167
+ "data_type": "fp"
168
+ },
169
+ "model.layers.39.mlp.shared_expert_gate": {
170
+ "bits": 16,
171
+ "data_type": "fp"
172
+ },
173
+ ".*model\\.layers\\.0\\.mlp\\.gate.*": {
174
+ "bits": 16,
175
+ "data_type": "float"
176
+ },
177
+ ".*model\\.layers\\.1\\.mlp\\.gate.*": {
178
+ "bits": 16,
179
+ "data_type": "float"
180
+ },
181
+ ".*model\\.layers\\.2\\.mlp\\.gate.*": {
182
+ "bits": 16,
183
+ "data_type": "float"
184
+ },
185
+ ".*model\\.layers\\.3\\.mlp\\.gate.*": {
186
+ "bits": 16,
187
+ "data_type": "float"
188
+ },
189
+ ".*model\\.layers\\.4\\.mlp\\.gate.*": {
190
+ "bits": 16,
191
+ "data_type": "float"
192
+ },
193
+ ".*model\\.layers\\.5\\.mlp\\.gate.*": {
194
+ "bits": 16,
195
+ "data_type": "float"
196
+ },
197
+ ".*model\\.layers\\.6\\.mlp\\.gate.*": {
198
+ "bits": 16,
199
+ "data_type": "float"
200
+ },
201
+ ".*model\\.layers\\.7\\.mlp\\.gate.*": {
202
+ "bits": 16,
203
+ "data_type": "float"
204
+ },
205
+ ".*model\\.layers\\.8\\.mlp\\.gate.*": {
206
+ "bits": 16,
207
+ "data_type": "float"
208
+ },
209
+ ".*model\\.layers\\.9\\.mlp\\.gate.*": {
210
+ "bits": 16,
211
+ "data_type": "float"
212
+ },
213
+ ".*model\\.layers\\.10\\.mlp\\.gate.*": {
214
+ "bits": 16,
215
+ "data_type": "float"
216
+ },
217
+ ".*model\\.layers\\.11\\.mlp\\.gate.*": {
218
+ "bits": 16,
219
+ "data_type": "float"
220
+ },
221
+ ".*model\\.layers\\.12\\.mlp\\.gate.*": {
222
+ "bits": 16,
223
+ "data_type": "float"
224
+ },
225
+ ".*model\\.layers\\.13\\.mlp\\.gate.*": {
226
+ "bits": 16,
227
+ "data_type": "float"
228
+ },
229
+ ".*model\\.layers\\.14\\.mlp\\.gate.*": {
230
+ "bits": 16,
231
+ "data_type": "float"
232
+ },
233
+ ".*model\\.layers\\.15\\.mlp\\.gate.*": {
234
+ "bits": 16,
235
+ "data_type": "float"
236
+ },
237
+ ".*model\\.layers\\.16\\.mlp\\.gate.*": {
238
+ "bits": 16,
239
+ "data_type": "float"
240
+ },
241
+ ".*model\\.layers\\.17\\.mlp\\.gate.*": {
242
+ "bits": 16,
243
+ "data_type": "float"
244
+ },
245
+ ".*model\\.layers\\.18\\.mlp\\.gate.*": {
246
+ "bits": 16,
247
+ "data_type": "float"
248
+ },
249
+ ".*model\\.layers\\.19\\.mlp\\.gate.*": {
250
+ "bits": 16,
251
+ "data_type": "float"
252
+ },
253
+ ".*model\\.layers\\.20\\.mlp\\.gate.*": {
254
+ "bits": 16,
255
+ "data_type": "float"
256
+ },
257
+ ".*model\\.layers\\.21\\.mlp\\.gate.*": {
258
+ "bits": 16,
259
+ "data_type": "float"
260
+ },
261
+ ".*model\\.layers\\.22\\.mlp\\.gate.*": {
262
+ "bits": 16,
263
+ "data_type": "float"
264
+ },
265
+ ".*model\\.layers\\.23\\.mlp\\.gate.*": {
266
+ "bits": 16,
267
+ "data_type": "float"
268
+ },
269
+ ".*model\\.layers\\.24\\.mlp\\.gate.*": {
270
+ "bits": 16,
271
+ "data_type": "float"
272
+ },
273
+ ".*model\\.layers\\.25\\.mlp\\.gate.*": {
274
+ "bits": 16,
275
+ "data_type": "float"
276
+ },
277
+ ".*model\\.layers\\.26\\.mlp\\.gate.*": {
278
+ "bits": 16,
279
+ "data_type": "float"
280
+ },
281
+ ".*model\\.layers\\.27\\.mlp\\.gate.*": {
282
+ "bits": 16,
283
+ "data_type": "float"
284
+ },
285
+ ".*model\\.layers\\.28\\.mlp\\.gate.*": {
286
+ "bits": 16,
287
+ "data_type": "float"
288
+ },
289
+ ".*model\\.layers\\.29\\.mlp\\.gate.*": {
290
+ "bits": 16,
291
+ "data_type": "float"
292
+ },
293
+ ".*model\\.layers\\.30\\.mlp\\.gate.*": {
294
+ "bits": 16,
295
+ "data_type": "float"
296
+ },
297
+ ".*model\\.layers\\.31\\.mlp\\.gate.*": {
298
+ "bits": 16,
299
+ "data_type": "float"
300
+ },
301
+ ".*model\\.layers\\.32\\.mlp\\.gate.*": {
302
+ "bits": 16,
303
+ "data_type": "float"
304
+ },
305
+ ".*model\\.layers\\.33\\.mlp\\.gate.*": {
306
+ "bits": 16,
307
+ "data_type": "float"
308
+ },
309
+ ".*model\\.layers\\.34\\.mlp\\.gate.*": {
310
+ "bits": 16,
311
+ "data_type": "float"
312
+ },
313
+ ".*model\\.layers\\.35\\.mlp\\.gate.*": {
314
+ "bits": 16,
315
+ "data_type": "float"
316
+ },
317
+ ".*model\\.layers\\.36\\.mlp\\.gate.*": {
318
+ "bits": 16,
319
+ "data_type": "float"
320
+ },
321
+ ".*model\\.layers\\.37\\.mlp\\.gate.*": {
322
+ "bits": 16,
323
+ "data_type": "float"
324
+ },
325
+ ".*model\\.layers\\.38\\.mlp\\.gate.*": {
326
+ "bits": 16,
327
+ "data_type": "float"
328
+ },
329
+ ".*model\\.layers\\.39\\.mlp\\.gate.*": {
330
+ "bits": 16,
331
+ "data_type": "float"
332
+ }
333
+ }
334
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
3
+ size 19989325
tokenizer_config.json ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "audio_bos_token": "<|audio_start|>",
4
+ "audio_eos_token": "<|audio_end|>",
5
+ "audio_token": "<|audio_pad|>",
6
+ "backend": "tokenizers",
7
+ "bos_token": null,
8
+ "clean_up_tokenization_spaces": false,
9
+ "eos_token": "<|im_end|>",
10
+ "errors": "replace",
11
+ "image_token": "<|image_pad|>",
12
+ "is_local": true,
13
+ "local_files_only": false,
14
+ "model_max_length": 262144,
15
+ "model_specific_special_tokens": {
16
+ "audio_bos_token": "<|audio_start|>",
17
+ "audio_eos_token": "<|audio_end|>",
18
+ "audio_token": "<|audio_pad|>",
19
+ "image_token": "<|image_pad|>",
20
+ "video_token": "<|video_pad|>",
21
+ "vision_bos_token": "<|vision_start|>",
22
+ "vision_eos_token": "<|vision_end|>"
23
+ },
24
+ "pad_token": "<|endoftext|>",
25
+ "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
26
+ "split_special_tokens": false,
27
+ "tokenizer_class": "Qwen2Tokenizer",
28
+ "unk_token": null,
29
+ "video_token": "<|video_pad|>",
30
+ "vision_bos_token": "<|vision_start|>",
31
+ "vision_eos_token": "<|vision_end|>"
32
+ }