gustajunq commited on
Commit
5fadb83
·
verified ·
1 Parent(s): ec0d859

Delete checkpoint-300

Browse files
checkpoint-300/README.md DELETED
@@ -1,210 +0,0 @@
1
- ---
2
- base_model: unsloth/Qwen3.5-2B
3
- library_name: peft
4
- pipeline_tag: text-generation
5
- tags:
6
- - base_model:adapter:unsloth/Qwen3.5-2B
7
- - lora
8
- - sft
9
- - transformers
10
- - trl
11
- - unsloth
12
- ---
13
-
14
- # Model Card for Model ID
15
-
16
- <!-- Provide a quick summary of what the model is/does. -->
17
-
18
-
19
-
20
- ## Model Details
21
-
22
- ### Model Description
23
-
24
- <!-- Provide a longer summary of what this model is. -->
25
-
26
-
27
-
28
- - **Developed by:** [More Information Needed]
29
- - **Funded by [optional]:** [More Information Needed]
30
- - **Shared by [optional]:** [More Information Needed]
31
- - **Model type:** [More Information Needed]
32
- - **Language(s) (NLP):** [More Information Needed]
33
- - **License:** [More Information Needed]
34
- - **Finetuned from model [optional]:** [More Information Needed]
35
-
36
- ### Model Sources [optional]
37
-
38
- <!-- Provide the basic links for the model. -->
39
-
40
- - **Repository:** [More Information Needed]
41
- - **Paper [optional]:** [More Information Needed]
42
- - **Demo [optional]:** [More Information Needed]
43
-
44
- ## Uses
45
-
46
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
47
-
48
- ### Direct Use
49
-
50
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
51
-
52
- [More Information Needed]
53
-
54
- ### Downstream Use [optional]
55
-
56
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
57
-
58
- [More Information Needed]
59
-
60
- ### Out-of-Scope Use
61
-
62
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
63
-
64
- [More Information Needed]
65
-
66
- ## Bias, Risks, and Limitations
67
-
68
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
69
-
70
- [More Information Needed]
71
-
72
- ### Recommendations
73
-
74
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
75
-
76
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
77
-
78
- ## How to Get Started with the Model
79
-
80
- Use the code below to get started with the model.
81
-
82
- [More Information Needed]
83
-
84
- ## Training Details
85
-
86
- ### Training Data
87
-
88
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
89
-
90
- [More Information Needed]
91
-
92
- ### Training Procedure
93
-
94
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
95
-
96
- #### Preprocessing [optional]
97
-
98
- [More Information Needed]
99
-
100
-
101
- #### Training Hyperparameters
102
-
103
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
104
-
105
- #### Speeds, Sizes, Times [optional]
106
-
107
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
108
-
109
- [More Information Needed]
110
-
111
- ## Evaluation
112
-
113
- <!-- This section describes the evaluation protocols and provides the results. -->
114
-
115
- ### Testing Data, Factors & Metrics
116
-
117
- #### Testing Data
118
-
119
- <!-- This should link to a Dataset Card if possible. -->
120
-
121
- [More Information Needed]
122
-
123
- #### Factors
124
-
125
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
126
-
127
- [More Information Needed]
128
-
129
- #### Metrics
130
-
131
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
132
-
133
- [More Information Needed]
134
-
135
- ### Results
136
-
137
- [More Information Needed]
138
-
139
- #### Summary
140
-
141
-
142
-
143
- ## Model Examination [optional]
144
-
145
- <!-- Relevant interpretability work for the model goes here -->
146
-
147
- [More Information Needed]
148
-
149
- ## Environmental Impact
150
-
151
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
152
-
153
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
154
-
155
- - **Hardware Type:** [More Information Needed]
156
- - **Hours used:** [More Information Needed]
157
- - **Cloud Provider:** [More Information Needed]
158
- - **Compute Region:** [More Information Needed]
159
- - **Carbon Emitted:** [More Information Needed]
160
-
161
- ## Technical Specifications [optional]
162
-
163
- ### Model Architecture and Objective
164
-
165
- [More Information Needed]
166
-
167
- ### Compute Infrastructure
168
-
169
- [More Information Needed]
170
-
171
- #### Hardware
172
-
173
- [More Information Needed]
174
-
175
- #### Software
176
-
177
- [More Information Needed]
178
-
179
- ## Citation [optional]
180
-
181
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
182
-
183
- **BibTeX:**
184
-
185
- [More Information Needed]
186
-
187
- **APA:**
188
-
189
- [More Information Needed]
190
-
191
- ## Glossary [optional]
192
-
193
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
194
-
195
- [More Information Needed]
196
-
197
- ## More Information [optional]
198
-
199
- [More Information Needed]
200
-
201
- ## Model Card Authors [optional]
202
-
203
- [More Information Needed]
204
-
205
- ## Model Card Contact
206
-
207
- [More Information Needed]
208
- ### Framework versions
209
-
210
- - PEFT 0.19.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-300/adapter_config.json DELETED
@@ -1,47 +0,0 @@
1
- {
2
- "alora_invocation_tokens": null,
3
- "alpha_pattern": {},
4
- "arrow_config": null,
5
- "auto_mapping": {
6
- "base_model_class": "Qwen3_5ForConditionalGeneration",
7
- "parent_library": "transformers.models.qwen3_5.modeling_qwen3_5",
8
- "unsloth_fixed": true
9
- },
10
- "base_model_name_or_path": "unsloth/Qwen3.5-2B",
11
- "bias": "none",
12
- "corda_config": null,
13
- "ensure_weight_tying": false,
14
- "eva_config": null,
15
- "exclude_modules": null,
16
- "fan_in_fan_out": false,
17
- "inference_mode": true,
18
- "init_lora_weights": true,
19
- "layer_replication": null,
20
- "layers_pattern": null,
21
- "layers_to_transform": null,
22
- "loftq_config": {},
23
- "lora_alpha": 16,
24
- "lora_bias": false,
25
- "lora_dropout": 0,
26
- "lora_ga_config": null,
27
- "megatron_config": null,
28
- "megatron_core": "megatron.core",
29
- "modules_to_save": null,
30
- "peft_type": "LORA",
31
- "peft_version": "0.19.1",
32
- "qalora_group_size": 16,
33
- "r": 8,
34
- "rank_pattern": {},
35
- "revision": null,
36
- "target_modules": [
37
- "v_proj",
38
- "q_proj"
39
- ],
40
- "target_parameters": null,
41
- "task_type": "CAUSAL_LM",
42
- "trainable_token_indices": null,
43
- "use_bdlora": null,
44
- "use_dora": false,
45
- "use_qalora": false,
46
- "use_rslora": false
47
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-300/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:4a20f68744eb2b7742e0e843c29142f4b731e596017364467e7bd62fc6ac36cb
3
- size 1674688
 
 
 
 
checkpoint-300/chat_template.jinja DELETED
@@ -1,158 +0,0 @@
1
- {%- set image_count = namespace(value=0) %}
2
- {%- set video_count = namespace(value=0) %}
3
- {%- macro render_content(content, do_vision_count, is_system_content=false) %}
4
- {%- if content is string %}
5
- {{- content }}
6
- {%- elif content is iterable and content is not mapping %}
7
- {%- for item in content %}
8
- {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
9
- {%- if is_system_content %}
10
- {{- raise_exception('System message cannot contain images.') }}
11
- {%- endif %}
12
- {%- if do_vision_count %}
13
- {%- set image_count.value = image_count.value + 1 %}
14
- {%- endif %}
15
- {%- if add_vision_id %}
16
- {{- 'Picture ' ~ image_count.value ~ ': ' }}
17
- {%- endif %}
18
- {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
19
- {%- elif 'video' in item or item.type == 'video' %}
20
- {%- if is_system_content %}
21
- {{- raise_exception('System message cannot contain videos.') }}
22
- {%- endif %}
23
- {%- if do_vision_count %}
24
- {%- set video_count.value = video_count.value + 1 %}
25
- {%- endif %}
26
- {%- if add_vision_id %}
27
- {{- 'Video ' ~ video_count.value ~ ': ' }}
28
- {%- endif %}
29
- {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
30
- {%- elif 'text' in item %}
31
- {{- item.text }}
32
- {%- else %}
33
- {{- raise_exception('Unexpected item type in content.') }}
34
- {%- endif %}
35
- {%- endfor %}
36
- {%- elif content is none or content is undefined %}
37
- {{- '' }}
38
- {%- else %}
39
- {{- raise_exception('Unexpected content type.') }}
40
- {%- endif %}
41
- {%- endmacro %}
42
- {%- if not messages %}
43
- {{- raise_exception('No messages provided.') }}
44
- {%- endif %}
45
- {%- set num_sys = 0 %}
46
- {%- set merged_system = '' %}
47
- {%- if messages[0].role == 'system' or messages[0].role == 'developer' %}
48
- {%- set first = render_content(messages[0].content, false, true)|trim %}
49
- {%- if messages|length > 1 and (messages[1].role == 'system' or messages[1].role == 'developer') %}
50
- {%- set second = render_content(messages[1].content, false, true)|trim %}
51
- {%- set merged_system = first + '\n' + second %}
52
- {%- set num_sys = 2 %}
53
- {%- else %}
54
- {%- set merged_system = first %}
55
- {%- set num_sys = 1 %}
56
- {%- endif %}
57
- {%- endif %}
58
- {%- if tools and tools is iterable and tools is not mapping %}
59
- {{- '<|im_start|>system\n' }}
60
- {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
61
- {%- for tool in tools %}
62
- {{- "\n" }}
63
- {{- tool | tojson }}
64
- {%- endfor %}
65
- {{- "\n</tools>" }}
66
- {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
67
- {%- if merged_system %}
68
- {{- '\n\n' + merged_system }}
69
- {%- endif %}
70
- {{- '<|im_end|>\n' }}
71
- {%- else %}
72
- {%- if merged_system %}
73
- {{- '<|im_start|>system\n' + merged_system + '<|im_end|>\n' }}
74
- {%- endif %}
75
- {%- endif %}
76
- {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
77
- {%- for message in messages[::-1] %}
78
- {%- set index = (messages|length - 1) - loop.index0 %}
79
- {%- if ns.multi_step_tool and message.role == "user" %}
80
- {%- set content = render_content(message.content, false)|trim %}
81
- {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
82
- {%- set ns.multi_step_tool = false %}
83
- {%- set ns.last_query_index = index %}
84
- {%- endif %}
85
- {%- endif %}
86
- {%- endfor %}
87
- {%- for message in messages %}
88
- {%- if loop.index0 >= num_sys and message.role != "system" and message.role != "developer" %}
89
- {%- set content = render_content(message.content, true)|trim %}
90
- {%- if message.role == "user" %}
91
- {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
92
- {%- elif message.role == "assistant" %}
93
- {%- set reasoning_content = '' %}
94
- {%- if message.reasoning_content is string %}
95
- {%- set reasoning_content = message.reasoning_content %}
96
- {%- else %}
97
- {%- if '</think>' in content %}
98
- {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
99
- {%- set content = content.split('</think>')[-1].lstrip('\n') %}
100
- {%- endif %}
101
- {%- endif %}
102
- {%- set reasoning_content = reasoning_content|trim %}
103
- {%- if loop.index0 > ns.last_query_index %}
104
- {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
105
- {%- else %}
106
- {{- '<|im_start|>' + message.role + '\n' + content }}
107
- {%- endif %}
108
- {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
109
- {%- for tool_call in message.tool_calls %}
110
- {%- if tool_call.function is defined %}
111
- {%- set tool_call = tool_call.function %}
112
- {%- endif %}
113
- {%- if loop.first %}
114
- {%- if content|trim %}
115
- {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
116
- {%- else %}
117
- {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
118
- {%- endif %}
119
- {%- else %}
120
- {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
121
- {%- endif %}
122
- {%- if tool_call.arguments is mapping %}
123
- {%- for args_name in tool_call.arguments %}
124
- {%- set args_value = tool_call.arguments[args_name] %}
125
- {{- '<parameter=' + args_name + '>\n' }}
126
- {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
127
- {{- args_value }}
128
- {{- '\n</parameter>\n' }}
129
- {%- endfor %}
130
- {%- endif %}
131
- {{- '</function>\n</tool_call>' }}
132
- {%- endfor %}
133
- {%- endif %}
134
- {{- '<|im_end|>\n' }}
135
- {%- elif message.role == "tool" %}
136
- {%- if loop.previtem and loop.previtem.role != "tool" %}
137
- {{- '<|im_start|>user' }}
138
- {%- endif %}
139
- {{- '\n<tool_response>\n' }}
140
- {{- content }}
141
- {{- '\n</tool_response>' }}
142
- {%- if not loop.last and loop.nextitem.role != "tool" %}
143
- {{- '<|im_end|>\n' }}
144
- {%- elif loop.last %}
145
- {{- '<|im_end|>\n' }}
146
- {%- endif %}
147
- {%- endif %}
148
- {%- endif %}
149
- {%- endfor %}
150
- {%- if add_generation_prompt %}
151
- {{- '<|im_start|>assistant\n' }}
152
- {%- if enable_thinking is defined and enable_thinking is true %}
153
- {{- '<think>\n' }}
154
- {%- else %}
155
- {{- '<think>\n\n</think>\n\n' }}
156
- {%- endif %}
157
- {%- endif %}
158
- {#- Unsloth fixes - developer role, tool calling #}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-300/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:b2ec3162744f56158d1679b75f5cb231417dcb2fd309698147fbbd1d16ab90ac
3
- size 880205
 
 
 
 
checkpoint-300/processor_config.json DELETED
@@ -1,63 +0,0 @@
1
- {
2
- "image_processor": {
3
- "data_format": "channels_first",
4
- "do_convert_rgb": true,
5
- "do_normalize": true,
6
- "do_rescale": true,
7
- "do_resize": true,
8
- "image_mean": [
9
- 0.5,
10
- 0.5,
11
- 0.5
12
- ],
13
- "image_processor_type": "Qwen2VLImageProcessor",
14
- "image_std": [
15
- 0.5,
16
- 0.5,
17
- 0.5
18
- ],
19
- "merge_size": 2,
20
- "patch_size": 16,
21
- "resample": 3,
22
- "rescale_factor": 0.00392156862745098,
23
- "size": {
24
- "longest_edge": 16777216,
25
- "shortest_edge": 65536
26
- },
27
- "temporal_patch_size": 2
28
- },
29
- "processor_class": "Qwen3VLProcessor",
30
- "video_processor": {
31
- "data_format": "channels_first",
32
- "default_to_square": true,
33
- "do_convert_rgb": true,
34
- "do_normalize": true,
35
- "do_rescale": true,
36
- "do_resize": true,
37
- "do_sample_frames": true,
38
- "fps": 2,
39
- "image_mean": [
40
- 0.5,
41
- 0.5,
42
- 0.5
43
- ],
44
- "image_std": [
45
- 0.5,
46
- 0.5,
47
- 0.5
48
- ],
49
- "max_frames": 768,
50
- "merge_size": 2,
51
- "min_frames": 4,
52
- "patch_size": 16,
53
- "resample": 3,
54
- "rescale_factor": 0.00392156862745098,
55
- "return_metadata": false,
56
- "size": {
57
- "longest_edge": 25165824,
58
- "shortest_edge": 4096
59
- },
60
- "temporal_patch_size": 2,
61
- "video_processor_type": "Qwen3VLVideoProcessor"
62
- }
63
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-300/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:3d6d8fafcd1ee268414be5acf0366296af5b03d60871978712eac1979cb42d65
3
- size 14645
 
 
 
 
checkpoint-300/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d15d3d1b8bb19162d239033d431d77f4eba28beb04ec2c0ba04b41fbc223a133
3
- size 1465
 
 
 
 
checkpoint-300/tokenizer.json DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:87a7830d63fcf43bf241c3c5242e96e62dd3fdc29224ca26fed8ea333db72de4
3
- size 19989343
 
 
 
 
checkpoint-300/tokenizer_config.json DELETED
@@ -1,299 +0,0 @@
1
- {
2
- "add_prefix_space": false,
3
- "audio_bos_token": "<|audio_start|>",
4
- "audio_eos_token": "<|audio_end|>",
5
- "audio_token": "<|audio_pad|>",
6
- "backend": "tokenizers",
7
- "bos_token": null,
8
- "clean_up_tokenization_spaces": false,
9
- "eos_token": "<|im_end|>",
10
- "errors": "replace",
11
- "image_token": "<|image_pad|>",
12
- "is_local": false,
13
- "model_max_length": 262144,
14
- "model_specific_special_tokens": {
15
- "audio_bos_token": "<|audio_start|>",
16
- "audio_eos_token": "<|audio_end|>",
17
- "audio_token": "<|audio_pad|>",
18
- "image_token": "<|image_pad|>",
19
- "video_token": "<|video_pad|>",
20
- "vision_bos_token": "<|vision_start|>",
21
- "vision_eos_token": "<|vision_end|>"
22
- },
23
- "pad_token": "<|vision_pad|>",
24
- "padding_side": "right",
25
- "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
26
- "processor_class": "Qwen3VLProcessor",
27
- "split_special_tokens": false,
28
- "tokenizer_class": "TokenizersBackend",
29
- "unk_token": null,
30
- "video_token": "<|video_pad|>",
31
- "vision_bos_token": "<|vision_start|>",
32
- "vision_eos_token": "<|vision_end|>",
33
- "added_tokens_decoder": {
34
- "248044": {
35
- "content": "<|endoftext|>",
36
- "single_word": false,
37
- "lstrip": false,
38
- "rstrip": false,
39
- "normalized": false,
40
- "special": true
41
- },
42
- "248045": {
43
- "content": "<|im_start|>",
44
- "single_word": false,
45
- "lstrip": false,
46
- "rstrip": false,
47
- "normalized": false,
48
- "special": true
49
- },
50
- "248046": {
51
- "content": "<|im_end|>",
52
- "single_word": false,
53
- "lstrip": false,
54
- "rstrip": false,
55
- "normalized": false,
56
- "special": true
57
- },
58
- "248047": {
59
- "content": "<|object_ref_start|>",
60
- "single_word": false,
61
- "lstrip": false,
62
- "rstrip": false,
63
- "normalized": false,
64
- "special": true
65
- },
66
- "248048": {
67
- "content": "<|object_ref_end|>",
68
- "single_word": false,
69
- "lstrip": false,
70
- "rstrip": false,
71
- "normalized": false,
72
- "special": true
73
- },
74
- "248049": {
75
- "content": "<|box_start|>",
76
- "single_word": false,
77
- "lstrip": false,
78
- "rstrip": false,
79
- "normalized": false,
80
- "special": true
81
- },
82
- "248050": {
83
- "content": "<|box_end|>",
84
- "single_word": false,
85
- "lstrip": false,
86
- "rstrip": false,
87
- "normalized": false,
88
- "special": true
89
- },
90
- "248051": {
91
- "content": "<|quad_start|>",
92
- "single_word": false,
93
- "lstrip": false,
94
- "rstrip": false,
95
- "normalized": false,
96
- "special": true
97
- },
98
- "248052": {
99
- "content": "<|quad_end|>",
100
- "single_word": false,
101
- "lstrip": false,
102
- "rstrip": false,
103
- "normalized": false,
104
- "special": true
105
- },
106
- "248053": {
107
- "content": "<|vision_start|>",
108
- "single_word": false,
109
- "lstrip": false,
110
- "rstrip": false,
111
- "normalized": false,
112
- "special": true
113
- },
114
- "248054": {
115
- "content": "<|vision_end|>",
116
- "single_word": false,
117
- "lstrip": false,
118
- "rstrip": false,
119
- "normalized": false,
120
- "special": true
121
- },
122
- "248055": {
123
- "content": "<|vision_pad|>",
124
- "single_word": false,
125
- "lstrip": false,
126
- "rstrip": false,
127
- "normalized": false,
128
- "special": true
129
- },
130
- "248056": {
131
- "content": "<|image_pad|>",
132
- "single_word": false,
133
- "lstrip": false,
134
- "rstrip": false,
135
- "normalized": false,
136
- "special": true
137
- },
138
- "248057": {
139
- "content": "<|video_pad|>",
140
- "single_word": false,
141
- "lstrip": false,
142
- "rstrip": false,
143
- "normalized": false,
144
- "special": true
145
- },
146
- "248058": {
147
- "content": "<tool_call>",
148
- "single_word": false,
149
- "lstrip": false,
150
- "rstrip": false,
151
- "normalized": false,
152
- "special": false
153
- },
154
- "248059": {
155
- "content": "</tool_call>",
156
- "single_word": false,
157
- "lstrip": false,
158
- "rstrip": false,
159
- "normalized": false,
160
- "special": false
161
- },
162
- "248060": {
163
- "content": "<|fim_prefix|>",
164
- "single_word": false,
165
- "lstrip": false,
166
- "rstrip": false,
167
- "normalized": false,
168
- "special": false
169
- },
170
- "248061": {
171
- "content": "<|fim_middle|>",
172
- "single_word": false,
173
- "lstrip": false,
174
- "rstrip": false,
175
- "normalized": false,
176
- "special": false
177
- },
178
- "248062": {
179
- "content": "<|fim_suffix|>",
180
- "single_word": false,
181
- "lstrip": false,
182
- "rstrip": false,
183
- "normalized": false,
184
- "special": false
185
- },
186
- "248063": {
187
- "content": "<|fim_pad|>",
188
- "single_word": false,
189
- "lstrip": false,
190
- "rstrip": false,
191
- "normalized": false,
192
- "special": false
193
- },
194
- "248064": {
195
- "content": "<|repo_name|>",
196
- "single_word": false,
197
- "lstrip": false,
198
- "rstrip": false,
199
- "normalized": false,
200
- "special": false
201
- },
202
- "248065": {
203
- "content": "<|file_sep|>",
204
- "single_word": false,
205
- "lstrip": false,
206
- "rstrip": false,
207
- "normalized": false,
208
- "special": false
209
- },
210
- "248066": {
211
- "content": "<tool_response>",
212
- "single_word": false,
213
- "lstrip": false,
214
- "rstrip": false,
215
- "normalized": false,
216
- "special": false
217
- },
218
- "248067": {
219
- "content": "</tool_response>",
220
- "single_word": false,
221
- "lstrip": false,
222
- "rstrip": false,
223
- "normalized": false,
224
- "special": false
225
- },
226
- "248068": {
227
- "content": "<think>",
228
- "single_word": false,
229
- "lstrip": false,
230
- "rstrip": false,
231
- "normalized": false,
232
- "special": false
233
- },
234
- "248069": {
235
- "content": "</think>",
236
- "single_word": false,
237
- "lstrip": false,
238
- "rstrip": false,
239
- "normalized": false,
240
- "special": false
241
- },
242
- "248070": {
243
- "content": "<|audio_start|>",
244
- "single_word": false,
245
- "lstrip": false,
246
- "rstrip": false,
247
- "normalized": false,
248
- "special": true
249
- },
250
- "248071": {
251
- "content": "<|audio_end|>",
252
- "single_word": false,
253
- "lstrip": false,
254
- "rstrip": false,
255
- "normalized": false,
256
- "special": true
257
- },
258
- "248072": {
259
- "content": "<tts_pad>",
260
- "single_word": false,
261
- "lstrip": false,
262
- "rstrip": false,
263
- "normalized": false,
264
- "special": true
265
- },
266
- "248073": {
267
- "content": "<tts_text_bos>",
268
- "single_word": false,
269
- "lstrip": false,
270
- "rstrip": false,
271
- "normalized": false,
272
- "special": true
273
- },
274
- "248074": {
275
- "content": "<tts_text_eod>",
276
- "single_word": false,
277
- "lstrip": false,
278
- "rstrip": false,
279
- "normalized": false,
280
- "special": true
281
- },
282
- "248075": {
283
- "content": "<tts_text_bos_single>",
284
- "single_word": false,
285
- "lstrip": false,
286
- "rstrip": false,
287
- "normalized": false,
288
- "special": true
289
- },
290
- "248076": {
291
- "content": "<|audio_pad|>",
292
- "single_word": false,
293
- "lstrip": false,
294
- "rstrip": false,
295
- "normalized": false,
296
- "special": true
297
- }
298
- }
299
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-300/trainer_state.json DELETED
@@ -1,244 +0,0 @@
1
- {
2
- "best_global_step": null,
3
- "best_metric": null,
4
- "best_model_checkpoint": null,
5
- "epoch": 3.8557536466774716,
6
- "eval_steps": 500,
7
- "global_step": 300,
8
- "is_hyper_param_search": false,
9
- "is_local_process_zero": true,
10
- "is_world_process_zero": true,
11
- "log_history": [
12
- {
13
- "epoch": 0.12965964343598055,
14
- "grad_norm": 0.44033142924308777,
15
- "learning_rate": 0.00019728813559322035,
16
- "loss": 1.588271427154541,
17
- "step": 10
18
- },
19
- {
20
- "epoch": 0.2593192868719611,
21
- "grad_norm": 1.1466902494430542,
22
- "learning_rate": 0.0001905084745762712,
23
- "loss": 1.8572025299072266,
24
- "step": 20
25
- },
26
- {
27
- "epoch": 0.3889789303079417,
28
- "grad_norm": 0.9241822957992554,
29
- "learning_rate": 0.00018372881355932204,
30
- "loss": 1.4406778335571289,
31
- "step": 30
32
- },
33
- {
34
- "epoch": 0.5186385737439222,
35
- "grad_norm": 0.7107939124107361,
36
- "learning_rate": 0.0001769491525423729,
37
- "loss": 1.0937417984008788,
38
- "step": 40
39
- },
40
- {
41
- "epoch": 0.6482982171799028,
42
- "grad_norm": 0.7085503935813904,
43
- "learning_rate": 0.00017016949152542373,
44
- "loss": 0.9844695091247558,
45
- "step": 50
46
- },
47
- {
48
- "epoch": 0.7779578606158833,
49
- "grad_norm": 0.9381449222564697,
50
- "learning_rate": 0.00016338983050847458,
51
- "loss": 0.6658666133880615,
52
- "step": 60
53
- },
54
- {
55
- "epoch": 0.9076175040518638,
56
- "grad_norm": 0.9691148996353149,
57
- "learning_rate": 0.00015661016949152542,
58
- "loss": 0.7807923316955566,
59
- "step": 70
60
- },
61
- {
62
- "epoch": 1.0259319286871962,
63
- "grad_norm": 1.2804441452026367,
64
- "learning_rate": 0.00014983050847457627,
65
- "loss": 0.7345180034637451,
66
- "step": 80
67
- },
68
- {
69
- "epoch": 1.1555915721231766,
70
- "grad_norm": 0.6306379437446594,
71
- "learning_rate": 0.00014305084745762714,
72
- "loss": 0.5936465263366699,
73
- "step": 90
74
- },
75
- {
76
- "epoch": 1.2852512155591573,
77
- "grad_norm": 0.8881189823150635,
78
- "learning_rate": 0.00013627118644067798,
79
- "loss": 0.5103878021240235,
80
- "step": 100
81
- },
82
- {
83
- "epoch": 1.4149108589951378,
84
- "grad_norm": 1.7029504776000977,
85
- "learning_rate": 0.00012949152542372883,
86
- "loss": 0.6796833515167237,
87
- "step": 110
88
- },
89
- {
90
- "epoch": 1.5445705024311183,
91
- "grad_norm": 2.0517845153808594,
92
- "learning_rate": 0.00012271186440677967,
93
- "loss": 0.520937728881836,
94
- "step": 120
95
- },
96
- {
97
- "epoch": 1.674230145867099,
98
- "grad_norm": 1.30018949508667,
99
- "learning_rate": 0.0001159322033898305,
100
- "loss": 0.667867374420166,
101
- "step": 130
102
- },
103
- {
104
- "epoch": 1.8038897893030794,
105
- "grad_norm": 0.9960445761680603,
106
- "learning_rate": 0.00010915254237288135,
107
- "loss": 0.48140687942504884,
108
- "step": 140
109
- },
110
- {
111
- "epoch": 1.9335494327390599,
112
- "grad_norm": 1.7485697269439697,
113
- "learning_rate": 0.00010237288135593222,
114
- "loss": 0.5179755687713623,
115
- "step": 150
116
- },
117
- {
118
- "epoch": 2.0518638573743924,
119
- "grad_norm": 1.4105119705200195,
120
- "learning_rate": 9.559322033898305e-05,
121
- "loss": 0.48134546279907225,
122
- "step": 160
123
- },
124
- {
125
- "epoch": 2.1815235008103726,
126
- "grad_norm": 1.0243501663208008,
127
- "learning_rate": 8.88135593220339e-05,
128
- "loss": 0.45008292198181155,
129
- "step": 170
130
- },
131
- {
132
- "epoch": 2.3111831442463533,
133
- "grad_norm": 1.2724498510360718,
134
- "learning_rate": 8.203389830508474e-05,
135
- "loss": 0.5789812564849853,
136
- "step": 180
137
- },
138
- {
139
- "epoch": 2.440842787682334,
140
- "grad_norm": 1.8518635034561157,
141
- "learning_rate": 7.52542372881356e-05,
142
- "loss": 0.42807679176330565,
143
- "step": 190
144
- },
145
- {
146
- "epoch": 2.5705024311183147,
147
- "grad_norm": 1.48202645778656,
148
- "learning_rate": 6.847457627118645e-05,
149
- "loss": 0.5260007858276368,
150
- "step": 200
151
- },
152
- {
153
- "epoch": 2.700162074554295,
154
- "grad_norm": 2.0189461708068848,
155
- "learning_rate": 6.169491525423729e-05,
156
- "loss": 0.48281493186950686,
157
- "step": 210
158
- },
159
- {
160
- "epoch": 2.8298217179902756,
161
- "grad_norm": 1.277570128440857,
162
- "learning_rate": 5.4915254237288135e-05,
163
- "loss": 0.42935891151428224,
164
- "step": 220
165
- },
166
- {
167
- "epoch": 2.9594813614262563,
168
- "grad_norm": 2.37007737159729,
169
- "learning_rate": 4.813559322033899e-05,
170
- "loss": 0.3907858610153198,
171
- "step": 230
172
- },
173
- {
174
- "epoch": 3.0777957860615883,
175
- "grad_norm": 1.6668658256530762,
176
- "learning_rate": 4.135593220338983e-05,
177
- "loss": 0.4579921245574951,
178
- "step": 240
179
- },
180
- {
181
- "epoch": 3.207455429497569,
182
- "grad_norm": 1.3719815015792847,
183
- "learning_rate": 3.4576271186440676e-05,
184
- "loss": 0.4433708190917969,
185
- "step": 250
186
- },
187
- {
188
- "epoch": 3.3371150729335493,
189
- "grad_norm": 2.302764415740967,
190
- "learning_rate": 2.7796610169491528e-05,
191
- "loss": 0.468212366104126,
192
- "step": 260
193
- },
194
- {
195
- "epoch": 3.46677471636953,
196
- "grad_norm": 1.2898948192596436,
197
- "learning_rate": 2.1016949152542373e-05,
198
- "loss": 0.35132598876953125,
199
- "step": 270
200
- },
201
- {
202
- "epoch": 3.5964343598055106,
203
- "grad_norm": 1.1389379501342773,
204
- "learning_rate": 1.4237288135593221e-05,
205
- "loss": 0.39004855155944823,
206
- "step": 280
207
- },
208
- {
209
- "epoch": 3.7260940032414913,
210
- "grad_norm": 2.6287143230438232,
211
- "learning_rate": 7.4576271186440685e-06,
212
- "loss": 0.5523932933807373,
213
- "step": 290
214
- },
215
- {
216
- "epoch": 3.8557536466774716,
217
- "grad_norm": 1.5404112339019775,
218
- "learning_rate": 6.779661016949152e-07,
219
- "loss": 0.32611134052276614,
220
- "step": 300
221
- }
222
- ],
223
- "logging_steps": 10,
224
- "max_steps": 300,
225
- "num_input_tokens_seen": 0,
226
- "num_train_epochs": 4,
227
- "save_steps": 500,
228
- "stateful_callbacks": {
229
- "TrainerControl": {
230
- "args": {
231
- "should_epoch_stop": false,
232
- "should_evaluate": false,
233
- "should_log": false,
234
- "should_save": true,
235
- "should_training_stop": true
236
- },
237
- "attributes": {}
238
- }
239
- },
240
- "total_flos": 4605685696732800.0,
241
- "train_batch_size": 1,
242
- "trial_name": null,
243
- "trial_params": null
244
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-300/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d43fa09e9fa05277c134e23e025ca7d913bcc82b5f79ed49fb679f22c5f7a9d9
3
- size 5585