My previous tests were run directly with cpp, rather than Claude Code. But don't worry! Tomorrow, I'll run extensive tests with Fable5 (including Claude Code) to see if it's actually an issue with the model. If it is, I'll have it fixed by tomorrow night. Otherwise, I will keep you updated. 🛠️🚀

Thank you! I keep an eye on it ;)

Hi, thanks a lot for testing and for the detailed screenshots — this kind of feedback is really valuable!

I dug into it and could reproduce what you're seeing, so here's an honest breakdown:
TL;DR: you're not doing anything wrong. Basic tool calling actually works (the chat template and Gemma 4's native tool-call format are intact — I get clean structured tool_calls from llama-server with simple requests). The problem is that in heavy agentic harnesses (huge system prompt + a dozen complex tools, like Claude Code-style clients), the model will sometimes skip calling tools entirely and confidently make up the results — invented file names, fake audit findings, etc. That matches exactly what you saw.

Why it happens: this model was distilled pure transcripts — there are zero tool-use samples in the SFT data. So it picked up Claude's "I'll run a comprehensive review now" narration, but never learned to back it with actual tool calls. The call:... pseudo-tags in your screenshot are it imitating the tool-call surface format as plain text. The lower the he tool setup, the better it behaves; a 130k-token agent harness at temp 1.0 is the worst case for it.

About the red warnings in your log (control-looking token '<|tool_response>' was not control-type, the EOG line): those are harmless and not specific to this model — every Gemma 4 GGUF (including official ones) triggers them in llama.cpp, and llama.cpp overrides them itself. See ggml-org/llama.cpp#21471 and #21316.

One thing I want to be fair about: part of what you hit is inherited from the base model and the ecosystem rather than this fine-tune — the red loader warnings and tool calls leaking as raw text affect the official Gemma 4 12B GGUFs as well (known llama.cpp issues), and 12B-classe in heavy agent harnesses. The chat-only distill data on top of that certainly didn't help, which is exactly what v2 will fix.

Practical advice for now: it should serve you fine for chat/reasoning and light function calling (temp ≤ 0.7 helps), but I wouldn't use this version as a coding agent — that's simply outside what it was trained for. for v2 I plan to mix agentic/tool-use trajectories into the distillation data, which should fix this properly.
Thanks again for taking the time to report i

Thank you so much looking into it! I will still test it also outside of Claude Code and let you know if I find anything. I will also wait for v2. Thank you for your work!

Man, running speculative decoding with a draft model on this Gemma-4 is smooth as hell, the token throughput t/s is fantastic. But honestly, if you’re trying to test agentic multi-step workloads, it’s a real pain right now because it handles tools so poorly.
You can spot the core issue right during the initialization phase in the logs:
W load: control-looking token: 50 '<|tool_response>' was not control-type...
W load: special_eog_ids contains '<|tool_response>', removing '' token from EOG list

llama-server completely misidentifies the control token, overrides it, and straight up kicks </s> out of the EOG (End of Generation) list. This usually happens when the GGUF tokenizer is misconfigured or there's a conflict in the Jinja template. I also tried other chat template for gemma-4 but not able to solve the problem! it keep getting overridden I think.

Also, what’s up with the KV Cache? Even when you explicitly configure it to q4_0 (or q8_0 for the target), the draft-mtp block completely ignores it during speculative implementation initialization and forces a fallback to f16:
common_speculative_impl_draft_mtp: - gpu_layers=-1, cache_k=f16, cache_v=f16...

This is highly likely a current limitation or a bug in llama.cpp when handling speculative decoding on MTP architectures—it probably doesn't support quantized KV caches for this specific pipeline yet, so it defaults back to f16?.

Adding -fit off to bypass the ctx_other mismatch error is a workaround for me! Not only does it get around the check, but it also completely fixes that notorious "turtle-slow" crawling inference speed seen on other Qwen 3.6 MTP profiles that I'm using . Definitely a lifesaver for anyone stuck on that issues (I use custom build so I'm not sure 100%).
Anyways, thanks for the greate model @yuxinlu1 ! I'm eager for your v2. Wish you all the best!
PS: this is the third MTP model I tried so far, so please correct me if I'm wrong; this is the pwsh script that I'm using:
pwsh:
#!/usr/bin/env pwsh
#Requires -Version 7.2
<#
.SYNOPSIS
Minimalist & Hardware-Aware Llama-Server Launcher (Refined)
.DESCRIPTION
Minimalistic and dynamic thread tuning and strict typing.
#>

[CmdletBinding()]
param(
[Parameter(Position = 0)]
[string]$ProfileID = ''
)

$ErrorActionPreference = 'Stop'
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8

── Hardware Auto-Tuner ───────────────────────────────────────────────────────

$OptimalThreads = [Math]::Max(4, [int]([Environment]::ProcessorCount / 2))

── UI System (Zero-Flicker Native ANSI) ───────────────────────────────────────

function Write-Header ([string]$Text) {
Write-Host "`n[✨ LLAMA SERVER CONTROL PANEL ]" -ForegroundColor Cyan
Write-Host ('━' * 70) -ForegroundColor DarkGray
Write-Host " 🚀 $Text" -ForegroundColor White
Write-Host ('━' * 70) -ForegroundColor DarkGray
}

function Write-Log ([string]$Status, [string]$Message, [string]$Color = 'Cyan') {
Write-Host "[$([char]27)[1m$Status$([char]27)[0m] " -NoNewline -ForegroundColor $Color
Write-Host $Message -ForegroundColor Gray
}

function Write-Alert ([string]$Message) {
Write-Host "`n[❌ ERROR] " -NoNewline -ForegroundColor Red
Write-Host $Message -ForegroundColor White
Write-Host ('━' * 70) -ForegroundColor DarkGray
[Console]::CursorVisible = $true
exit 1
}

── High-Performance TUI Menu ─────────────────────────────────────────────────

function Get-MenuSelection ([string]$Title, [array]$Options) {
$sel = 0
try {
[Console]::CursorVisible = $false
Write-Header $Title
Write-Host ' Use ↑/↓ to navigate, Enter to select, Esc to exit:' -ForegroundColor DarkGray
Write-Host ''
$menuTop = [Console]::CursorTop

    while ($true) {
        [Console]::SetCursorPosition(0, $menuTop)
        for ($i = 0; $i -lt $Options.Count; $i++) {
            $lineText = "  $(if ($i -eq $sel) { '▶ ' } else { '  ' })  $($Options[$i].Description)"
            if ($i -eq $sel) {
                Write-Host $lineText.PadRight(70) -ForegroundColor Cyan -BackgroundColor DarkCyan
            } else {
                # Refined: Bỏ Hardcoded Black background để giữ nguyên bản Theme của Terminal
                Write-Host $lineText.PadRight(70) -ForegroundColor Gray
            }
        }
        Write-Host ''

        $key = [Console]::ReadKey($true)
        switch ($key.Key) {
            'UpArrow'   { $sel = if ($sel -eq 0) { $Options.Count - 1 } else { $sel - 1 } }
            'DownArrow' { $sel = if ($sel -eq $Options.Count - 1) { 0 } else { $sel + 1 } }
            'Enter'     { return $Options[$sel].ID }
            'Escape'    { exit 0 }
        }
    }
} finally {
    [Console]::CursorVisible = $true
}

}

── High-Speed O(1) Path Validator ───────────────────────────────────────────

function Assert-ProfilePaths ([string[]]$ArgsList) {
$pathFlags = [System.Collections.Generic.HashSet[string]]::new([string[]]@('--model', '--model-draft', '--mmproj', '--chat-template-file'))

for ($i = 0; $i -lt $ArgsList.Count - 1; $i++) {
    if ($pathFlags.Contains($ArgsList[$i])) {
        $p = $ArgsList[$i + 1]
        if (-not [string]::IsNullOrWhiteSpace($p) -and -not (Test-Path -LiteralPath $p)) {
            Write-Alert "Required resource not found for $($ArgsList[$i]):`n           $p"
        }
    }
}

}

── Standalone Environment Configuration ──────────────────────────────────────

$LlamaBinDir = $Env:LLAMA_BIN_DIR ?? 'D:\quang_dev\llama.cpp_kraven\build-cuda\bin'
$rawServerBin = $Env:LLAMA_SERVER_BIN ?? 'llama-server'
$HostAddress = $Env:HOST ?? '0.0.0.0'
$Port = $Env:PORT ?? '8080'

$LlamaServerBin = if ([System.IO.Path]::IsPathRooted($rawServerBin)) { $rawServerBin } else { Join-Path $LlamaBinDir $rawServerBin }

Refined: Dùng Explicit Enum để tránh linter warning và tối ưu hóa phân tích tĩnh

if ($IsWindows -and -not $LlamaServerBin.EndsWith('.exe', [System.StringComparison]::OrdinalIgnoreCase)) {
$LlamaServerBin += '.exe'
}
$LlamaServerBin = $LlamaServerBin -replace '[\/]+', [System.IO.Path]::DirectorySeparatorChar

── Intelligence Profiles Registry ────────────────────────────────────────────

$Profiles = @(
@{
Label = 'Qwopus 3.6 35B A3B Mini [CPU Device]'
EnvVars = @{ MTMD_BACKEND_DEVICE = 'cpu' }
Args = [string[]]@(
'--model', 'E:\llm\models\mudler__Qwopus3.6-35B-A3B-v1-APEX-MTP-GGUF\Qwopus3.6-35B-A3B-v1-APEX-MTP-I-Mini.gguf',
'--ctx-size', '131072',
'--n-gpu-layers', '-1',
'--n-cpu-moe', '4',
'--temp', '1.0',
'--top-p', '0.95',
'--top-k', '20',
'--min-p', '0',
'--repeat-penalty', '1',
'--presence-penalty', '0',
'--reasoning', 'on',
'--chat-template-kwargs', '{"reasoning_effort":"medium","preserve_thinking":true}',
'--jinja',
'--port', $Port,
'--host', $HostAddress,
'--no-context-shift',
'--cache-type-k', 'q4_0',
'--cache-type-v', 'q4_0',
'--mlock',
'--no-mmap',
'--threads', "$OptimalThreads",
'--batch-size', '2048',
'--ubatch-size', '1024',
'--threads-batch', "$OptimalThreads",
'--metrics',
'--chat-template-file', 'E:\llm\models\chat_template_qwen35+36-froggeric.jinja',
'--mmproj', 'E:\llm\models\mmproj-qwen3.6-35b-a3b-f16.gguf',
'--image-min-tokens', '1024',
'--spec-type', 'draft-mtp',
'--spec-draft-n-max', '3'
)
},
@{
Label = 'Qwopus 3.6 27B v2 MTP [GPU Accelerated]'
Args = [string[]]@(
'--model', 'E:\llm\models\Jackrong__Qwopus3.6-27B-v2-MTP-GGUF\Qwopus3.6-27B-v2-MTP-Q3_K_L.gguf',
'--n-gpu-layers', '-1',
'--ctx-size', '65536',
'--temp', '1.0',
'--top-p', '0.95',
'--top-k', '20',
'--min-p', '0',
'--repeat-penalty', '1',
'--presence-penalty', '0',
'--reasoning', 'on',
'--chat-template-kwargs', '{"reasoning_effort":"medium","preserve_thinking":true}',
'--jinja',
'--port', $Port,
'--host', $HostAddress,
'--no-context-shift',
'--cache-type-k', 'q4_0',
'--cache-type-v', 'q4_0',
'--mlock',
'--no-mmap',
'--threads', "$OptimalThreads",
'--batch-size', '2048',
'--ubatch-size', '1024',
'--threads-batch', "$OptimalThreads",
'--metrics',
'--chat-template-file', 'E:\llm\models\chat_template_qwen35+36-froggeric.jinja',
'-fit', 'off'
)
},
@{
Label = 'Gemma-4 12B Coder Fable5 composer2.5'
Args = [string[]]@(
'--model', 'E:\llm\models\yuxinlu1__gemma-4-12B-coder-fable5-composer2.5-v1-GGUF\gemma4-coding-Q6_K.gguf',
'--n-gpu-layers', '-1',
'--ctx-size', '65536',
'--temp', '1.0',
'--top-p', '0.95',
'--top-k', '64',
'--reasoning', 'on',
'--chat-template-kwargs', '{"reasoning_effort":"medium","preserve_thinking":true}',
'--jinja',
'--port', $Port,
'--host', $HostAddress,
'--no-context-shift',
'--cache-type-k', 'q8_0',
'--cache-type-v', 'q5_1',
'--mlock',
'--no-mmap',
'--threads', "$OptimalThreads",
'--batch-size', '2048',
'--ubatch-size', '1024',
'--threads-batch', "$OptimalThreads",
'--metrics',
'--chat-template-file', 'E:\llm\models\chat_template_gemma-4.jinja',
'--model-draft', 'E:\llm\models\yuxinlu1__gemma-4-12B-coder-fable5-composer2.5-v1-GGUF\gemma-4-12B-it-MTP-Q8_0.gguf',
'--spec-type', 'draft-mtp',
'--spec-draft-n-max', '3',
'-fit', 'off'
)
}
)

── Router & Dispatch ─────────────────────────────────────────────────────────

if ([string]::IsNullOrWhiteSpace($ProfileID)) { $ProfileID = $Env:LLAMA_PROFILE }

if ([string]::IsNullOrWhiteSpace($ProfileID)) {
$Choices = for ($i = 0; $i -lt $Profiles.Count; $i++) {
@{ ID = "$($i + 1)"; Description = "Profile $($i + 1): $($Profiles[$i].Label)" }
}
Clear-Host
$ProfileID = Get-MenuSelection 'Select a Model Profile' $Choices
Clear-Host
}

$idx = [int]$ProfileID - 1
if ($idx -lt 0 -or $idx -ge $Profiles.Count) { Write-Alert "Profile ID '$ProfileID' out of bounds." }
$Selected = $Profiles[$idx]

── Pre-flight Checks & Execution ─────────────────────────────────────────────

if (-not (Test-Path -LiteralPath $LlamaServerBin)) { Write-Alert "Execution engine missing:`n $LlamaServerBin" }

Write-Header "Profile ${ProfileID}: $($Selected.Label)"
Write-Log 'CHECKING' "Verifying resources & auto-allocating $OptimalThreads compute threads..." 'Yellow'
Assert-ProfilePaths $Selected.Args

if ($Selected.EnvVars) {
foreach ($kv in $Selected.EnvVars.GetEnumerator()) {
Set-Item "Env:$($kv.Key)" $kv.Value
Write-Log 'ENV' "$($kv.Key) = $($kv.Value)" 'DarkYellow'
}
}

Write-Log 'RUNNING' "Context shift: $LlamaBinDir" 'Magenta'
if (Test-Path -LiteralPath $LlamaBinDir) { Set-Location $LlamaBinDir }

Write-Log 'ONLINE' "Listening on http://${HostAddress}:${Port}" 'Green'
Write-Host "Press Ctrl+C to terminate inference server gracefully.`n" -ForegroundColor DarkGray

& $LlamaServerBin @($Selected.Args)

So, does this model is valuable?