jangq commited on
Commit
983ef7b
·
verified ·
1 Parent(s): 4cca032

Revert incorrect looping-fix banner — mlx_lm already handles multi-eos from generation_config.json

Browse files
Files changed (1) hide show
  1. README.md +12 -27
README.md CHANGED
@@ -9,33 +9,18 @@ tags:
9
  pipeline_tag: text-generation
10
  ---
11
 
12
- > # 🚨 IMPORTANT — RE-DOWNLOAD `tokenizer_config.json` IF YOU GOT INFINITE LOOPS
13
- >
14
- > **Fixed 2026-04-13.** Earlier versions of this repo had a tokenizer bug that caused the model to **loop forever** in stock `mlx_lm` and other loaders.
15
- >
16
- > **Gemma-4 emits `<end_of_turn>` (id 106) at the end of an assistant turn**, but the original `tokenizer_config.json` only listed `<eos>` (id 1) as the stop token. Stock loaders never detected the actual end-of-turn marker → infinite loop.
17
- >
18
- > ## How to fix:
19
- >
20
- > ### Option A — Re-download just the tokenizer config (fastest)
21
- > ```bash
22
- > huggingface-cli download JANGQ-AI/Gemma-4-26B-A4B-it-JANG_4M tokenizer_config.json --local-dir ./your-model-dir
23
- > ```
24
- >
25
- > ### Option B — Re-download the whole repo
26
- > ```bash
27
- > huggingface-cli download JANGQ-AI/Gemma-4-26B-A4B-it-JANG_4M --local-dir ./your-model-dir
28
- > ```
29
- >
30
- > ### Option C — Pass the stop tokens manually
31
- > ```python
32
- > stop_token_ids = [1, 106, 50] # <eos>, <end_of_turn>, <end_of_image>
33
- > ```
34
- >
35
- > **The model weights are unchanged — you only need to update `tokenizer_config.json`.**
36
 
37
- ---
38
 
39
- # Gemma-4-26B-A4B-it-JANG_4M
 
 
 
 
 
 
 
 
40
 
41
- JANG quantized Gemma-4 MoE for Apple Silicon. See [JANGQ-AI](https://huggingface.co/JANGQ-AI) for the full collection.
 
9
  pipeline_tag: text-generation
10
  ---
11
 
12
+ # Gemma-4-26B-A4B-it-JANG_4M
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
 
14
+ JANG-quantized Gemma-4 MoE for Apple Silicon. Created by Jinho Jang — eric@jangq.ai.
15
 
16
+ See the full [JANGQ-AI collection](https://huggingface.co/JANGQ-AI) for all profiles.
17
+
18
+ ## Loading
19
+
20
+ ```python
21
+ from mlx_lm import load, generate
22
+ model, tokenizer = load("JANGQ-AI/Gemma-4-26B-A4B-it-JANG_4M")
23
+ print(generate(model, tokenizer, "Hello", max_tokens=256))
24
+ ```
25
 
26
+ Stock `mlx_lm` picks up the multi-stop-token list (`[1, 106, 50]`) automatically from `generation_config.json` — no manual configuration required.