Upload folder using huggingface_hub

README.md

@@ -3,86 +3,41 @@ license: apache-2.0
 datasets:
 - dichspace/darulm
 - HuggingFaceFW/fineweb-2
-- RefalMachine/ruadapt_instruct_2507
+- RefalMachine/hybrid_reasoning_dataset_ru
 language:
 - ru
 - en
-base_model: RefalMachine/RuadaptQwen3-4B-Instruct
-pipeline_tag: text-generation
+base_model: RefalMachine/RuadaptQwen3-32B-Instruct
+library_name: mlx
 tags:
 - mlx
-library_name: mlx
+pipeline_tag: text-generation
 ---
 
-# RuadaptQwen3-4B-Instruct-4bit MLX
-
-This is a 4-bit quantized MLX version of [RefalMachine/RuadaptQwen3-4B-Instruct](https://huggingface.co/RefalMachine/RuadaptQwen3-4B-Instruct), optimized for Apple Silicon devices using the MLX framework.
-
-## Performance Metrics
-
-Model was tested using LM Studio as inference provider on a binned M4 Max MacBook Pro 14" with the following specifications:
-- 32-core GPU
-- 36 GB unified memory
-- Full GPU offload enabled
-
-### Test prompt:
-```
-напиши стих в стиле Евгения Онегина о бесконечно вечном
-```
-
-### Results:
+# Bogdan01m/RuadaptQwen3-32B-Instruct-MLX-8bit
 
-**4-bit MLX (this model):**
-- Tokens generated: 287
-- Time to First Token (TTFT): 0.14s
-- Throughput: up to 118.93 tok/s
+This model [Bogdan01m/RuadaptQwen3-32B-Instruct-MLX-8bit](https://huggingface.co/Bogdan01m/RuadaptQwen3-32B-Instruct-MLX-8bit) was
+converted to MLX format from [RefalMachine/RuadaptQwen3-32B-Instruct](https://huggingface.co/RefalMachine/RuadaptQwen3-32B-Instruct)
+using mlx-lm version **0.28.3**.
 
-**Q4_0 GGUF** ([RefalMachine/RuadaptQwen3-4B-Instruct-GGUF](https://huggingface.co/RefalMachine/RuadaptQwen3-4B-Instruct-GGUF)):
-- Tokens generated: 233
-- TTFT: 0.16s
-- Throughput: up to 97.52 tok/s
-
-**Note:** The MLX 4-bit quantization demonstrates up to ~22% higher throughput compared to Q4_0 GGUF on Apple Silicon in this simple test.
-
-## Usage
-
-Install MLX and mlx-lm:
+## Use with mlx
 
 ```bash
-pip install mlx mlx-lm
+pip install mlx-lm
 ```
 
-Use the model:
-
 ```python
 from mlx_lm import load, generate
 
-model, tokenizer = load("Bogdan01m/RuadaptQwen3-4B-Instruct-4bit")
-response = generate(model, tokenizer, prompt="Привет! Как дела?", verbose=True)
-```
-
-## Recommended Generation Parameters
-
-For more stable results, it is recommended to use low temperatures 0.0-0.3, top_p in the range from 0.85 to 0.95 and repetition_penalty 1.05.
-
-## Original Model
+model, tokenizer = load("Bogdan01m/RuadaptQwen3-32B-Instruct-MLX-8bit")
 
-This model is a quantized version of [RefalMachine/RuadaptQwen3-4B-Instruct](https://huggingface.co/RefalMachine/RuadaptQwen3-4B-Instruct).
+prompt = "hello"
 
-## Citation
+if tokenizer.chat_template is not None:
+    messages = [{"role": "user", "content": prompt}]
+    prompt = tokenizer.apply_chat_template(
+        messages, add_generation_prompt=True
+    )
 
-```bibtex
-@article{tikhomirov2024facilitating,
-  title={Facilitating Large Language Model Russian Adaptation with Learned Embedding Propagation},
-  author={Tikhomirov, Mikhail and Chernyshov, Daniil},
-  journal={Journal of Language and Education},
-  volume={10},
-  number={4},
-  pages={130--145},
-  year={2024}
-}
+response = generate(model, tokenizer, prompt=prompt, verbose=True)
 ```
-
-## Important
-
-The model's answers do not reflect the authors' opinions; they merely reproduce the knowledge obtained from data at all training stages. The model is based on a third-party pretrained model. Use with caution.

chat_template.jinja

@@ -1,42 +1,72 @@
 {%- if tools %}
     {{- '<|im_start|>system\n' }}
-    {%- if messages[0]['role'] == 'system' %}
-        {{- messages[0]['content'] }}
-    {%- else %}
-        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
     {%- endif %}
-    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
     {%- for tool in tools %}
         {{- "\n" }}
         {{- tool | tojson }}
     {%- endfor %}
     {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
 {%- else %}
-    {%- if messages[0]['role'] == 'system' %}
-        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
     {%- endif %}
 {%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
 {%- for message in messages %}
-    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
         {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
     {%- elif message.role == "assistant" %}
-        {{- '<|im_start|>' + message.role }}
-        {%- if message.content %}
-            {{- '\n' + message.content }}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
         {%- endif %}
-        {%- for tool_call in message.tool_calls %}
-            {%- if tool_call.function is defined %}
-                {%- set tool_call = tool_call.function %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
             {%- endif %}
-            {{- '\n<tool_call>\n{"name": "' }}
-            {{- tool_call.name }}
-            {{- '", "arguments": ' }}
-            {{- tool_call.arguments | tojson }}
-            {{- '}\n</tool_call>' }}
-        {%- endfor %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
         {{- '<|im_end|>\n' }}
     {%- elif message.role == "tool" %}
-        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
             {{- '<|im_start|>user' }}
         {%- endif %}
         {{- '\n<tool_response>\n' }}
@@ -49,4 +79,7 @@
 {%- endfor %}
 {%- if add_generation_prompt %}
     {{- '<|im_start|>assistant\n' }}
-{%- endif %}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

config.json

@@ -7,31 +7,31 @@
     "eos_token_id": 146215,
     "head_dim": 128,
     "hidden_act": "silu",
-    "hidden_size": 2560,
+    "hidden_size": 5120,
     "initializer_range": 0.02,
-    "intermediate_size": 9728,
-    "max_position_embeddings": 262144,
-    "max_window_layers": 36,
+    "intermediate_size": 25600,
+    "max_position_embeddings": 40960,
+    "max_window_layers": 64,
     "model_type": "qwen3",
-    "num_attention_heads": 32,
-    "num_hidden_layers": 36,
+    "num_attention_heads": 64,
+    "num_hidden_layers": 64,
     "num_key_value_heads": 8,
     "pad_token_id": 146213,
     "quantization": {
         "group_size": 64,
-        "bits": 4,
+        "bits": 8,
         "mode": "affine"
     },
     "quantization_config": {
         "group_size": 64,
-        "bits": 4,
+        "bits": 8,
         "mode": "affine"
     },
     "rms_norm_eps": 1e-06,
     "rope_scaling": null,
-    "rope_theta": 5000000,
+    "rope_theta": 1000000,
     "sliding_window": null,
-    "tie_word_embeddings": true,
+    "tie_word_embeddings": false,
     "torch_dtype": "bfloat16",
     "transformers_version": "4.51.3",
     "use_cache": true,

generation_config.json

@@ -2,8 +2,8 @@
   "do_sample": true,
   "eos_token_id": 146215,
   "pad_token_id": 146213,
-  "temperature": 0.7,
+  "temperature": 0.6,
   "top_k": 20,
-  "top_p": 0.8,
+  "top_p": 0.95,
   "transformers_version": "4.51.3"
 }

model-00001-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f517f2a218f6f6a616eb3a5df110283f5924019b95af2b552f61146520d5615
+size 5319142940

model-00002-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:956770fc5f486bb7ffc990b78ec13e384d592ee6522a70e8b494a0f282ce2105
+size 5364709259

model-00003-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:47d6511aaa939cd61e8cb8393d3a863795b20be012b44593823045fae2d7978e
+size 5367638894

model-00004-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:acf4ff28cc7e11d233f74c73e21dfa62160b2755e98af85b864142c09b675d96
+size 5328316006

model-00005-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cdea6a2216e1d7452cb9f152e16851722ba7fbcaee366d235e980f1d54d7136f
+size 5364709271

model-00006-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69389761d8fb8d889af2d6e1b3bb7b381a7e3c59ba2669dfe7619640ddf24373
+size 5367638918

model-00007-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2e574194ef39c869e912272096d3c3d0e5f656af51dd1dd10492c620ebcd46e
+size 2636664420

special_tokens_map.json

@@ -14,13 +14,6 @@
     "<|image_pad|>",
     "<|video_pad|>"
   ],
-  "bos_token": {
-    "content": "<|endoftext|>",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
   "eos_token": {
     "content": "<|im_end|>",
     "lstrip": false,

tokenizer.json

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5408c0e76d30d5f9348593818dcaafdd45dbe64ba19fcd5fb92c2677c15f4cf1
-size 12380040
+oid sha256:e5259c1df4795303715f22c2d9d9ca43fe8e25a4909342df3b90b56bc19a2f42
+size 12380042

tokenizer_config.json

@@ -200,7 +200,7 @@
       "normalized": false,
       "rstrip": false,
       "single_word": false,
-      "special": true
+      "special": false
     },
     "146238": {
       "content": "</think>",
@@ -208,7 +208,7 @@
       "normalized": false,
       "rstrip": false,
       "single_word": false,
-      "special": true
+      "special": false
     },
     "146239": {
       "content": "<|free_token1|>",
@@ -394,7 +394,7 @@
     "<|image_pad|>",
     "<|video_pad|>"
   ],
-  "bos_token": "<|endoftext|>",
+  "bos_token": null,
   "clean_up_tokenization_spaces": false,
   "eos_token": "<|im_end|>",
   "errors": "replace",