aimeri commited on
Commit
9828408
·
verified ·
1 Parent(s): 398d5da

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +480 -87
README.md CHANGED
@@ -1,101 +1,494 @@
1
  ---
2
  license: gemma
3
- base_model: google/gemma-4-12B
 
4
  library_name: transformers
5
  pipeline_tag: text-generation
6
- tags: [gemma4, sft, reasoning, roleplay, tool-use]
 
 
 
 
 
 
7
  ---
8
- # spoomplesmaxx-whiskeyjack-12B
9
-
10
- gemma-4-12B full-parameter SFT (ms-swift `swift sft`, DeepSpeed ZeRO-2, torch SDPA attention, custom liger fused-CE).
11
- Control tokens (`<turn|>`, `<|channel>`/`<channel|>`, `<|tool_call>`/`<tool_call|>`)
12
- were audited pre-training and verified emit-able post-training (stop battery,
13
- boundary probes, tool-call battery).
14
-
15
- ## Format
16
-
17
- Gemma 4 turn format — note it differs from Gemma 3 entirely (there is no
18
- `<start_of_turn>`), and the assistant role is spelled `model`:
19
-
20
- ```
21
- <|turn>user
22
- ...<turn|>
23
- <|turn>model
24
- <|channel>thought
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
  ...reasoning...
26
- <channel|>...answer...<turn|>
27
- ```
28
-
29
- - **Stops on** `<turn|>` (106). `<eos>` (1) is kept as a secondary EOS.
30
- Do not configure `<eos>` alone as the stop — turns end on `<turn|>`.
31
- - **Thinking** is selected by a `<|think|>` marker at the top of the **system**
32
- turn, which `apply_chat_template(enable_thinking=True)` injects. It defaults to
33
- **False**. Both modes share the bare `<|turn>model` generation prefix; the model
34
- decides whether to open `<|channel>thought` itself.
35
- - **The chat template here is NOT stock Gemma 4.** Upstream appends an empty
36
- channel (`<|channel>thought
37
- <channel|>`) for non-thinking. That form appears
38
- **0 times in 1000 training turns** of this corpus — a no-thoughts turn carries no
39
- channel at all — so it is deliberately removed. The original is kept alongside as
40
- `chat_template.gemma-it-original.jinja`. Restoring it puts the model out of
41
- distribution (reasoning leaks into content, tool calls lose their opener).
42
- - **Tool calls** use Gemma's DSL, not JSON:
43
- `<|tool_call>call:NAME{key:<|"|>value<|"|>}<tool_call|>`.
44
- - **Serving tool calls read this or you will see "infinite tool loops."** In the
45
- training corpus a whole tool episode lives inside ONE `<|turn>model`, with
46
- `<|tool_response>` blocks interleaved *inline*; the model never emitted `<turn|>`
47
- after a call and so never learned to yield. Serve with **`stop=["<tool_call|>"]`**,
48
- inject the result as `<|tool_response>response:NAME{...}<tool_response|>`, and
49
- continue the SAME turn. A harness that waits for `<turn|>` after a call will
50
- hang and the model will keep generating plausible calls.
51
- ## Training stages
52
-
53
- | stage | data | steps | LR | eval_loss | token_acc |
54
- |---|---|---|---|---|---|
55
- | 1 (tag `v1-baseline-rp`) | aviary burn corpus, 1 epoch | 1,917 | 1e-5 | 1.311 | 0.6465 |
56
- | 2 (tag `v2-corrected-rp`) | thinking-weighted resample | 568 | 2e-6 | 1.3067 | 0.6479 |
57
- | 3 (**main**) | + 4,000 converted RP-reasoning rows | 574 | 2e-6 | **1.301** | **0.6491** |
58
-
59
- Stage 3 exists because stage 2 thinking was entangled with a single system
60
- framing: 19,605 of the corpus's 20,666 thought-bearing rows share one RP prompt
61
- shape, so the model opened a thought channel on 8/8 in-corpus rows and 1/25 real
62
- character cards. Stage 3 interleaved RP-reasoning rows under ~4,000 distinct
63
- character cards to decouple the two.
64
 
65
- ## Measured behaviour
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
66
 
67
- On 25 held-out character cards (846-5,053 chars) with a short opening message:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
 
69
- | | stage 2 | stage 3 |
70
- |---|---|---|
71
- | opens a thought channel | 1/25 (4%) | **23/25 (92%)** |
72
-
73
- Unchanged across the pass: stop rate 10/10 in both thinking and non-thinking
74
- modes, tool-call round trip, 0/3 stray channels when thinking is off, and
75
- P(`<channel|>`) at the true close = 1.000.
76
-
77
- ### Two behaviours to expect
78
-
79
- **The thought form depends on the system prompt shape.** Under a SillyTavern-style
80
- character card the model writes a structured planner
81
- (`SCENE:` / `CHARACTERS:` / `CONTINUITY:` / `THREADS:` / `PLAN:`, ~750 chars,
82
- 23/23 of the cards that opened). Under the corpus's own RP framing it writes
83
- short first-person interiority (~90 chars). These are two distinct learned forms
84
- conditioned on the prompt, not a blend.
85
-
86
- **Thinking is optional under a companion persona.** A conversational companion
87
- card produces no thought channel (0/6) — that reflects the training corpus, where
88
- such rows are predominantly non-thinking. This is by design, not a defect.
89
- - **generation_config:** temperature 1.0, top_p 0.95,
90
- top_k 64.
91
-
92
- ```python
93
  from transformers import AutoModelForImageTextToText, AutoTokenizer
94
  tok = AutoTokenizer.from_pretrained("aimeri/spoomplesmaxx-whiskeyjack-12B")
95
- model = AutoModelForImageTextToText.from_pretrained("aimeri/spoomplesmaxx-whiskeyjack-12B", dtype="bfloat16", device_map="auto")
 
 
96
  msgs = [{"role": "user", "content": "Solve (x + 2)^2 = 0."}]
97
- ids = tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=True,
98
- return_tensors="pt").to(model.device)
99
  out = model.generate(ids, max_new_tokens=512)
100
  print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=False))
101
- ```
 
 
 
 
 
 
 
 
1
  ---
2
  license: gemma
3
+ base_model:
4
+ - google/gemma-4-12B
5
  library_name: transformers
6
  pipeline_tag: text-generation
7
+ tags:
8
+ - gemma4
9
+ - sft
10
+ - finetune
11
+ - reasoning
12
+ - roleplay
13
+ - tool-use
14
  ---
15
+ <!doctype html>
16
+ <html lang="en">
17
+ <head>
18
+ <meta charset="UTF-8" />
19
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
20
+ <title>SpoomplesMaxx Whiskeyjack 12B</title>
21
+ </head>
22
+ <style>
23
+ @import url("https://fonts.googleapis.com/css2?family=Consolas&display=swap");
24
+ .crt-container {
25
+ padding: 10px;
26
+ max-width: 1000px;
27
+ margin: 0 auto;
28
+ width: 95%;
29
+ }
30
+ .crt-case {
31
+ background: #e8d7c3;
32
+ border-radius: 10px;
33
+ padding: 15px;
34
+ box-shadow:
35
+ inset -2px -2px 5px rgba(0, 0, 0, 0.3),
36
+ 2px 2px 5px rgba(0, 0, 0, 0.2);
37
+ }
38
+ .crt-inner-case {
39
+ background: #e8d7c3;
40
+ border-radius: 8px;
41
+ padding: 3px;
42
+ box-shadow:
43
+ inset -1px -1px 4px rgba(0, 0, 0, 0.3),
44
+ 1px 1px 4px rgba(0, 0, 0, 0.2);
45
+ }
46
+ .crt-bezel {
47
+ background: linear-gradient(145deg, #1a1a1a, #2a2a2a);
48
+ padding: 15px;
49
+ border-radius: 5px;
50
+ border: 3px solid #0a0a0a;
51
+ position: relative;
52
+ box-shadow:
53
+ inset 0 0 20px rgba(0, 0, 0, 0.5),
54
+ inset 0 0 4px rgba(0, 0, 0, 0.4),
55
+ inset 2px 2px 4px rgba(255, 255, 255, 0.05),
56
+ inset -2px -2px 4px rgba(0, 0, 0, 0.8),
57
+ 0 0 2px rgba(0, 0, 0, 0.6),
58
+ -1px -1px 4px rgba(255, 255, 255, 0.1),
59
+ 1px 1px 4px rgba(0, 0, 0, 0.3);
60
+ }
61
+ .crt-bezel::before {
62
+ content: "";
63
+ position: absolute;
64
+ top: 0;
65
+ left: 0;
66
+ right: 0;
67
+ bottom: 0;
68
+ background: linear-gradient(
69
+ 45deg,
70
+ rgba(255, 255, 255, 0.03) 0%,
71
+ rgba(255, 255, 255, 0) 40%,
72
+ rgba(0, 0, 0, 0.1) 60%,
73
+ rgba(0, 0, 0, 0.2) 100%
74
+ );
75
+ border-radius: 3px;
76
+ pointer-events: none;
77
+ }
78
+ .terminal-screen {
79
+ background: #0c100d;
80
+ padding: 20px;
81
+ border-radius: 15px;
82
+ position: relative;
83
+ overflow: hidden;
84
+ font-family: "Consolas", monospace;
85
+ font-size: clamp(12px, 1.5vw, 16px);
86
+ color: #3dc862;
87
+ line-height: 1.4;
88
+ text-shadow: 0 0 2px #3dc862;
89
+ filter: brightness(1.1) contrast(1.1);
90
+ box-shadow:
91
+ inset 0 0 30px rgba(0, 0, 0, 0.9),
92
+ inset 0 0 8px rgba(0, 0, 0, 0.8),
93
+ 0 0 5px rgba(0, 0, 0, 0.6);
94
+ max-width: 80ch;
95
+ margin: 0 auto;
96
+ }
97
+ .terminal-screen h2,
98
+ .terminal-screen h3 {
99
+ font-size: clamp(16px, 2vw, 20px);
100
+ margin-bottom: 1em;
101
+ color: #ffdf00;
102
+ text-shadow: 0 0 3px rgba(255, 223, 0, 0.5);
103
+ }
104
+ .terminal-screen pre.code-block-image {
105
+ display: inline-block;
106
+ text-align: left;
107
+ font-size: clamp(2px, 0.4vw, 12px);
108
+ font-family: monospace;
109
+ margin: 1em 0;
110
+ background-color: #1a1a1a;
111
+ padding: 1em;
112
+ border-radius: 4px;
113
+ color: #3dc862;
114
+ overflow-x: auto;
115
+ line-height: 1;
116
+ max-width: 100%;
117
+ overflow: hidden;
118
+ white-space: pre;
119
+ }
120
+ .terminal-screen pre.code-block {
121
+ display: inline-block;
122
+ text-align: left;
123
+ font-size: clamp(10px, 1.3vw, 14px);
124
+ font-family: monospace;
125
+ margin: 1em 0;
126
+ background-color: #1a1a1a;
127
+ padding: 1em;
128
+ border-radius: 4px;
129
+ color: #3dc862;
130
+ overflow-x: auto;
131
+ line-height: 1;
132
+ max-width: 100%;
133
+ overflow: hidden;
134
+ white-space: pre;
135
+ }
136
+ .terminal-screen::before {
137
+ content: "";
138
+ position: absolute;
139
+ top: 0;
140
+ left: 0;
141
+ right: 0;
142
+ bottom: 0;
143
+ background:
144
+ linear-gradient(
145
+ rgba(18, 16, 16, 0) 50%,
146
+ rgba(0, 0, 0, 0.25) 50%
147
+ ),
148
+ url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADIAAAAyBAMAAADsEZWCAAAAGFBMVEUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA4o8JoAAAAB3RSTlMAGwQIEQMYADcPzwAAACJJREFUKM9jYBgFo2AU0Beg+A8YMCLxGYZCbNQEo4BaAAD5TQiR5wU9vAAAAABJRU5ErkJggg==");
149
+ background-size: 100% 2.5px;
150
+ pointer-events: none;
151
+ z-index: 2;
152
+ }
153
+ .terminal-screen::after {
154
+ content: "";
155
+ position: absolute;
156
+ top: 0;
157
+ left: 0;
158
+ right: 0;
159
+ bottom: 0;
160
+ background: radial-gradient(
161
+ circle at center,
162
+ rgba(12, 16, 13, 0) 0%,
163
+ rgba(12, 16, 13, 0.2) 50%,
164
+ rgba(12, 16, 13, 0.15) 100%
165
+ );
166
+ border-radius: 20px;
167
+ pointer-events: none;
168
+ z-index: 1;
169
+ }
170
+ .terminal-screen .notice {
171
+ margin: 1.5em 0;
172
+ padding: 0.8em 1.2em;
173
+ border: 1px solid #ffdf00;
174
+ border-radius: 4px;
175
+ background-color: rgba(255, 223, 0, 0.04);
176
+ }
177
+ .terminal-screen .notice h3 {
178
+ margin-top: 0.2em;
179
+ margin-bottom: 0.5em;
180
+ }
181
+ .terminal-screen .notice p {
182
+ margin-bottom: 0.2em;
183
+ }
184
+ .terminal-screen strong,
185
+ .terminal-screen em {
186
+ color: #f0f0f0;
187
+ }
188
+ .terminal-screen p,
189
+ .terminal-screen li {
190
+ color: #3dc862;
191
+ }
192
+ .terminal-screen a {
193
+ color: #5da9ff;
194
+ text-decoration: underline;
195
+ text-shadow: 0 0 2px rgba(93, 169, 255, 0.5);
196
+ transition: opacity 0.2s;
197
+ }
198
+ .terminal-screen a:hover {
199
+ opacity: 0.8;
200
+ }
201
+ .terminal-screen code,
202
+ .terminal-screen kbd,
203
+ .terminal-screen samp {
204
+ color: #3dc862;
205
+ font-family: "Consolas", monospace;
206
+ text-shadow: 0 0 2px #3dc862;
207
+ background-color: #1a1a1a;
208
+ padding: 0.2em 0.4em;
209
+ border-radius: 4px;
210
+ }
211
+ </style>
212
+ <div class="crt-container">
213
+ <div class="crt-case">
214
+ <div class="crt-inner-case">
215
+ <div class="crt-bezel">
216
+ <div class="terminal-screen">
217
+ <div style="text-align: center">
218
+ <h2>SpoomplesMaxx-Whiskeyjack-12B</h2>
219
+ <h3>"Camp Robber"</h3>
220
+ <pre class="code-block-image">
221
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
222
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
223
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
224
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
225
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
226
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
227
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
228
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒░░░��▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
229
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
230
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
231
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░░▓▓▓▓▓▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
232
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒░░░░░░░░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
233
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░▒▓▓▓▓▓▓░▓░░▒░▓▓▓▓▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
234
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░▓▓▓▓▓▓▓▓▓▓░░░▓▓▓▓░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
235
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▒░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
236
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░▒▒░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
237
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒▒▒▒▒▒▒▒▒▒░░░▓▓▓▓▓▓▓▓▓▓▓▓░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
238
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▓▓▓▓▓▓▓▓▓▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
239
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▓▓▓▓▓▓░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
240
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▓▓▓▓▓▓▓▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
241
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
242
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░▓▓▓▓▓▓▓▓░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
243
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░▒▒░▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░░▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
244
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░▒░▒░░▒░░░░░░▓▓▓▓▓▓▓▓▓▓░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
245
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░░░▒░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
246
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░░░▒░▒░░░░▓▓▓▓▓▓▓▓▓▓░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
247
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░▓░░░░░░░░░░░░▓░░▒░▓░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
248
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░░░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░▓▓▓▓▓▓▓▓▓▓▓▓▓
249
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░▓░░░░░░▒░░░▒░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓
250
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░▓░░░░░▓░░░░░▒░░░░░░▒▓▓▓▓▓▓▓▓▓▓▓▓░▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓
251
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░▓░░░░░▓░░░▒▓▓▓▓▓▓▓▓▓▓▓▓▒░▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓
252
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░▓░░░▓░░░░░░░░░░░░░▒▒▓▓▓▓▓▓▓▓▓▓▓▓▒▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
253
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒░░░░░░░░░▒░░▒░░░▒▒▒▒▒▓▓▓▓▓▓▓▓▓▓░░▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
254
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓░▓░░░▒░░▓░▒▓░░░▒▒▒▒▒▓▓▓▓▓▓▓▓▓▓▓▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
255
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░▓▓▓▓▓▓▓▓▓▓▓░░▒░▓░░▓░░░░░▒▒▒▒▒▒▒▒▓▒▓▓▓▓▓▓░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
256
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░▓▓▓▓▓▓▓▓▓▓▒░░░░▓░▒░░░░▒▒▒▒▒▒▒▒▒▒▒▒▓░░▓░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░▓▓▓▓▓▒░░░░░░░▓▓▓▓▓▓▓▓▓▓▓
257
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░▓▓▓▓░░▓▓▓▓░░░▒░▓░░░░░░░░░░░░░░░▒░▓▓▓▓▓▓▓░░▓▓▓▓▓▓▓▓▓░░░░░░░░▒▓▓░░░░░░░░░░░▓▓▓▓▓▓▓▓
258
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░▓▓▓░░░▓▓▓░░░░▓░░░░░░░░▓▓▓▓▓▓▓▓░░▓▓▓▓▓▓▓▓▓░░░░▓░░░░░░░░▓▓▓▓▓░░▓▓▓▓░░░░░░░░░▓▓▓▓▓▓▓
259
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░▓▓░░░░▓▓░░░▓░░▒░░░░░▓▓▓▓▓▓▓▓▓▓▓░░▓▓▓▓▓▓░░░░░▓░░░░░▓▓▓▓▓▓▓▓▓▓░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
260
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░▓░░░░▓▓▓░▓░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓░▓▓▓░░░░░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
261
+ ▓▓▓▓▓▓▓▓░░░░░░░░▓▓▓░░░▓▓▓▓▓▒░░░░░░░░▒▓▓▓▓▓▓▓▓▓▓▓▓▓░▓░▓░▓░░░░░▓░▓▓▓▓▓░░░▓▓▓▓▓▓▓▓▓░░░░░▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓
262
+ ▓▓▓▓▓▓▓▓░░░░░░░░░▓▓░░▓▓▓▓▓▒░░░░░░▓░░░▓▓▓▓▓▓▓▓░░░░░░░▓░���░▓▓▓▓▓▓▓▓▓▓▓▓▓░▓▒░░░▒▓▓▓▓▓░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓
263
+ ▓▓▓▓▓▓▓▓▓▓▓░░░░░░░▓░▓▓▓▓▓▒░░░░░░░░░░▓▓▓▓▓░░░░░░░░░░▓░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░▓▓░░░░░░▓▓▓▒░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓
264
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░▓▓▓▓▓░░░░░░░░░▓▓▓░░░░░░░░░░▓▓▓▓░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░▓▓░░░░░░▓▓▓▓▓░░▓▓▓▓▓▓▓▓▓▓▓▓▓
265
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░▓▓▓▓░░░░░░░░░▓░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░▓▓░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
266
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░▓▓▓▓░░░░░░░░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░▓▓▓░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
267
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░▓▓▓░░░░░░░▓░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░▓▓▓▓▓░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
268
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░▓▓░░░░░░░░▒░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░▓▓▓▓▓▓▓▓▓░▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
269
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░▒░░▒░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░▓▓▓▓▓▓▓▓▓▓▓▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
270
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░▓░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
271
+ ▓▓▓▓▓▓▓▓▓▓▓░░░░░░░▓░░░▓░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
272
+ ▓▓▓▓▓▓▓▓▓░░░░░░░░▓░░░▒░░░░░░▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
273
+ ▓▓▓▓▓▓▓▓▓░░░░░░▓▓▓░░▒░░░▒░░▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
274
+ ▓▓▓▓▓▓▓▓▓▓░░▓▓▓▓▓▓░▒░░░░░▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
275
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
276
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
277
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
278
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
279
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓��▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
280
+ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓
281
+ </pre>
282
+ </div>
283
+ <p>
284
+ SpoomplesMaxx is a generalist model line with
285
+ primary strengths in creative writing and roleplay,
286
+ plus competence at instruction following,
287
+ reasoning, and tool calling. Whiskeyjack brings the
288
+ corvid line to Gemma: a full-parameter SFT of
289
+ <strong>gemma-4-12B</strong>, trained in both
290
+ thinking and non-thinking modes, with thinking off
291
+ by default.
292
+ </p>
293
+ <p>
294
+ Named for <em>Perisoreus canadensis</em> — the
295
+ Canada jay, better known as the whisky jack or camp
296
+ robber. A corvid bold enough to land on your hand
297
+ and fly off with your lunch. The 35B got the
298
+ jackdaw; the 12B gets the smaller, friendlier
299
+ thief.
300
+ </p>
301
+ <h3>Prompt format</h3>
302
+ <p>
303
+ Gemma 4 uses a new turn format. It shares nothing
304
+ with Gemma 3 — there is no
305
+ <code>&lt;start_of_turn&gt;</code> — and the
306
+ assistant role is spelled <code>model</code>:
307
+ </p>
308
+ <pre class="code-block">
309
+ &lt;|turn&gt;user
310
+ ...&lt;turn|&gt;
311
+ &lt;|turn&gt;model
312
+ &lt;|channel&gt;thought
313
  ...reasoning...
314
+ &lt;channel|&gt;...answer...&lt;turn|&gt;
315
+ </pre>
316
+ <pre class="code-block">
317
+ STOPS: stop on &lt;turn|&gt; (id 106). &lt;eos&gt; (id 1) is kept as a secondary
318
+ EOS, but never set &lt;eos&gt; alone -- turns end on &lt;turn|&gt;.
319
+ </pre>
320
+ <p>
321
+ The control tokens (<code>&lt;turn|&gt;</code>,
322
+ <code>&lt;|channel&gt;</code>/<code>&lt;channel|&gt;</code>,
323
+ <code>&lt;|tool_call&gt;</code>/<code>&lt;tool_call|&gt;</code>)
324
+ were audited before training and re-verified after
325
+ it: stop battery, boundary probes, and a tool-call
326
+ battery all pass on the published checkpoint.
327
+ </p>
328
+ <h3>Thinking behavior</h3>
329
+ <p>
330
+ Thinking is opt-in and <strong>off by
331
+ default</strong>. A <code>&lt;|think|&gt;</code>
332
+ marker at the top of the <strong>system</strong>
333
+ turn switches it on;
334
+ <code>apply_chat_template(enable_thinking=True)</code>
335
+ injects it for you. Both modes share the same bare
336
+ <code>&lt;|turn&gt;model</code> generation prefix —
337
+ the model decides on its own whether to open
338
+ <code>&lt;|channel&gt;thought</code>.
339
+ </p>
340
+ <pre class="code-block">
341
+ MODE CONTROL:
342
+ (default) thinking OFF -- no marker, no thought channel
343
+ enable_thinking=True injects &lt;|think|&gt; into the system turn; the
344
+ model opens &lt;|channel&gt;thought on its own
 
 
 
 
 
 
 
345
 
346
+ PARSER NOTE: reasoning sits between &lt;|channel&gt;thought and &lt;channel|&gt;;
347
+ the visible answer follows &lt;channel|&gt; in the same turn
348
+ </pre>
349
+ <div class="notice">
350
+ <h3>The chat template is not stock Gemma 4</h3>
351
+ <p>
352
+ Upstream Gemma 4 appends an empty thought
353
+ channel
354
+ (<code>&lt;|channel&gt;thought\n&lt;channel|&gt;</code>)
355
+ to non-thinking turns. That form shows up
356
+ <strong>0 times in 1,000 training turns</strong>
357
+ of this corpus — a no-thoughts turn simply
358
+ carries no channel — so the template here drops
359
+ it. The stock template ships alongside as
360
+ <code>chat_template.gemma-it-original.jinja</code>.
361
+ Restore it and you push the model out of
362
+ distribution: reasoning leaks into the answer
363
+ and tool calls lose their opener.
364
+ </p>
365
+ </div>
366
+ <p>
367
+ What the thoughts look like depends on the system
368
+ prompt. Under a SillyTavern-style character card
369
+ the model writes a structured planner (~750 chars;
370
+ 23/23 of the cards that opened a channel). Under
371
+ the corpus's own RP framing it writes short
372
+ first-person interiority (~90 chars). The model
373
+ learned both forms separately, and the prompt picks
374
+ which one you get.
375
+ </p>
376
+ <p>The planner, when it shows up:</p>
377
+ <pre class="code-block">
378
+ SCENE: where/when, atmosphere, key environmental details currently in play
379
+ CHARACTERS: who is present and their current physical/emotional state and motivation
380
+ CONTINUITY: established facts that must stay consistent
381
+ THREADS: active tensions and where they stand right now
382
+ PLAN: what THIS turn needs to accomplish and the approach it takes
383
+ </pre>
384
+ <p>
385
+ One more thing to expect: a conversational
386
+ companion persona usually produces no thought
387
+ channel at all (0/6 in testing), even with thinking
388
+ on. Companion rows in the corpus are mostly
389
+ non-thinking, and the model follows the data.
390
+ </p>
391
+ <h3>Tool calling</h3>
392
+ <p>
393
+ Gemma 4 tool calls use a DSL, <strong>not
394
+ JSON</strong>:
395
+ </p>
396
+ <pre class="code-block">
397
+ FORM: &lt;|tool_call&gt;call:NAME{key:&lt;|"|&gt;value&lt;|"|&gt;}&lt;tool_call|&gt;
398
+ EXAMPLE: &lt;|tool_call&gt;call:get_weather{city:&lt;|"|&gt;Lisbon&lt;|"|&gt;}&lt;tool_call|&gt;
399
+ </pre>
400
+ <div class="notice">
401
+ <h3>Serve tool calls inside one turn</h3>
402
+ <p>
403
+ In the training corpus a whole tool episode
404
+ lives inside a single
405
+ <code>&lt;|turn&gt;model</code>, with
406
+ <code>&lt;|tool_response&gt;</code> blocks
407
+ interleaved inline. The model never emitted
408
+ <code>&lt;turn|&gt;</code> after a call, so it
409
+ never learned to yield there. A harness that
410
+ waits for <code>&lt;turn|&gt;</code> will hang
411
+ while the model keeps generating plausible
412
+ calls — the classic infinite tool loop.
413
+ </p>
414
+ <pre class="code-block">
415
+ SERVE WITH: stop=["&lt;tool_call|&gt;"]
416
+ THEN: inject &lt;|tool_response&gt;response:NAME{...}&lt;tool_response|&gt;
417
+ and continue the SAME turn
418
+ NEVER: wait for &lt;turn|&gt; after a tool call
419
+ </pre>
420
+ </div>
421
+ <h3>Key Details</h3>
422
+ <pre class="code-block">
423
+ BASE MODEL: google/gemma-4-12B
424
+ LICENSE: gemma
425
+ NOTE: the base is multimodal, so the checkpoint loads with
426
+ AutoModelForImageTextToText (see Quickstart)</pre>
427
+ <h3>Training</h3>
428
+ <pre class="code-block">
429
+ METHOD: FULL-PARAMETER SFT -- ms-swift (swift sft), DeepSpeed ZeRO-2,
430
+ torch SDPA attention, custom liger fused CE
431
+ STAGES: three, each tagged in this repo; main = stage 3
432
 
433
+ stage 1 (v1-baseline-rp) aviary burn corpus, 1 epoch
434
+ 1,917 steps @ lr 1e-5 eval 1.311 tok-acc 0.6465
435
+ stage 2 (v2-corrected-rp) thinking-weighted resample
436
+ 568 steps @ lr 2e-6 eval 1.3067 tok-acc 0.6479
437
+ stage 3 (main) + 4,000 converted RP-reasoning rows
438
+ 574 steps @ lr 2e-6 eval 1.301 tok-acc 0.6491
439
+ </pre>
440
+ <div class="notice">
441
+ <h3>Why there is a stage 3</h3>
442
+ <p>
443
+ Stage 2 could think, but only under one prompt
444
+ shape: 19,605 of the corpus's 20,666
445
+ thought-bearing rows share a single RP framing.
446
+ So the model opened a thought channel on 8/8
447
+ in-corpus rows — and on 1 of 25 real character
448
+ cards. Stage 3 mixed in RP-reasoning rows under
449
+ ~4,000 distinct character cards so that
450
+ thinking no longer depends on one specific
451
+ prompt.
452
+ </p>
453
+ <pre class="code-block">
454
+ opens a thought channel on 25 held-out character cards
455
+ (846-5,053 chars, short opening message):
456
+ stage 2: 1/25 (4%)
457
+ stage 3: 23/25 (92%)
458
 
459
+ unchanged across the pass:
460
+ stop rate 10/10 in both thinking and non-thinking modes
461
+ tool-call round trip passes
462
+ stray channels with thinking off: 0/3
463
+ P(&lt;channel|&gt;) at the true close: 1.000
464
+ </pre>
465
+ </div>
466
+ <h3>Sampling</h3>
467
+ <p>
468
+ Use the defaults in <code>generation_config.json</code>.
469
+ <pre class="code-block">
470
+ "temperature": 1.0,
471
+ "top_k": 64,
472
+ "top_p": 0.95,
473
+ </pre>
474
+ </p>
475
+ <h3>Quickstart</h3>
476
+ <pre class="code-block">
 
 
 
 
 
 
477
  from transformers import AutoModelForImageTextToText, AutoTokenizer
478
  tok = AutoTokenizer.from_pretrained("aimeri/spoomplesmaxx-whiskeyjack-12B")
479
+ model = AutoModelForImageTextToText.from_pretrained(
480
+ "aimeri/spoomplesmaxx-whiskeyjack-12B",
481
+ dtype="bfloat16", device_map="auto")
482
  msgs = [{"role": "user", "content": "Solve (x + 2)^2 = 0."}]
483
+ ids = tok.apply_chat_template(msgs, add_generation_prompt=True,
484
+ enable_thinking=True, return_tensors="pt").to(model.device)
485
  out = model.generate(ids, max_new_tokens=512)
486
  print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=False))
487
+ </pre>
488
+ </div>
489
+ </div>
490
+ </div>
491
+ </div>
492
+ </div>
493
+
494
+ </html>