ubergarm

/

Qwen3.8-27B-GGUF

Like 28

I've tried a few different harnesses (opencode, deepseek, qwen-code) and it seems with this model it causes massive loops of nonsense after a few minutes of thinking. I dont know if this is an issue with ik_llama or the quant. I switch to Unsloth IQ4_XS with llama.cpp and the harnesses can finish the project. Hermes and other things seem to work fine with IQ4_KS

My IQ4_KS config (llama-swap with ik_llama) - using template from froggeric on both the ubergarm and unsloth models (I have also tried without the updated template with ubergarm). Also using "thinking" with medium reasoning level on both.

  "Qwen3.8-27B MTP ik_llama":
    cmd: >
      sh -c "LD_LIBRARY_PATH=/ik-bin/bin:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64 CUDA_VISIBLE_DEVICES=0 exec /ik-bin/bin/llama-server
      --port ${PORT}
      --host 127.0.0.1
      --model /models/qwen38/ubergarm_Qwen3.8-27B-MTP-IQ4_KS.gguf
      --mmproj /models/qwen38/unsloth_mmproj-BF16.gguf
      --chat-template-file /models/qwen38/froggeric_Qwen3.8_14AUG2026.jinja
      -ngl 99
      --ctx-size 105000
      -ngld 99
      -cd 4096
      --spec-type mtp:n_max=2,p_min=0.0
      --cache-type-k q8_0
      --cache-type-v q8_0
      --merge-qkv
      --merge-up-gate-experts
      --ctx-checkpoints 32
      --parallel-tool-calls  
      -khad 
      --split-mode none 
      --main-gpu 0
      --image-min-tokens 1024
      --threads 6
      --threads-batch 6
      --flash-attn on 
      --parallel 1 
      --batch-size 512 
      --ubatch-size 512
      --no-mmap
      --jinja"
    filters:
      stripParams: "temperature, top_p, top_k, min_p, presence_penalty, repeat_penalty"  
      setParamsByID:
        "${MODEL_ID}:thinking":
          max_tokens: 32768
          chat_template_kwargs:
            enable_thinking: true
            preserve_thinking: true
            reasoning_effort: "medium"
          temperature: 1.0
          top_p: 0.95
          top_k: 20
          min_p: 0.0
          presence_penalty: 0.0
          repeat_penalty: 1.0

        "${MODEL_ID}:thinking-coding":
          chat_template_kwargs:
            enable_thinking: true
            preserve_thinking: true
            reasoning_effort: "xhigh"
          temperature: 0.4
          top_p: 0.95
          top_k: 20
          min_p: 0.0
          presence_penalty: 0.0 
          repeat_penalty: 1.0

        "${MODEL_ID}:instruct":
          max_tokens: 16384
          chat_template_kwargs:
            enable_thinking: false
            preserve_thinking: false
          temperature: 0.7
          top_p: 0.8
          top_k: 20
          min_p: 0.0
          presence_penalty: 1.5
          repeat_penalty: 1.0

        "${MODEL_ID}:instruct-reasoning":
          max_tokens: 16384
          chat_template_kwargs:
            enable_thinking: false
            preserve_thinking: false
          temperature: 0.7
          top_p: 0.8
          top_k: 20
          min_p: 0.0
          presence_penalty: 1.5
          repeat_penalty: 1.0

        "${MODEL_ID}:instruct-tools":
          max_tokens: 8192
          chat_template_kwargs:
            enable_thinking: false
            preserve_thinking: false
          temperature: 0.6
          top_p: 0.95
          top_k: 20
          min_p: 0.0
          presence_penalty: 0.0 
          repeat_penalty: 1.0
          
        "${MODEL_ID}:thinking-tools":
          max_tokens: 16384
          reasoning_budget: 8192
          chat_template_kwargs:
            enable_thinking: true
            preserve_thinking: true
            reasoning_effort: "medium"
          temperature: 0.6
          top_p: 0.95
          top_k: 20
          min_p: 0.0
          presence_penalty: 0.0 
          repeat_penalty: 1.0

        "${MODEL_ID}:router":
          max_tokens: 512
          chat_template_kwargs:
            enable_thinking: false
            preserve_thinking: false
          temperature: 0.1
          top_p: 0.1
          top_k: 10
          min_p: 0.0
          presence_penalty: 0.0
          repeat_penalty: 1.0

Example repeat of nonsense with IQ4_KS with prompt "Create a single page html + tailwind css + javascript recreation of galaga, same to the original arcade game. include sound effects":

Hmm getting fuzzy again on details (transcribed code fragments were truncated / duplicated, probably copy glitches, etc.); as above with settled approach is a fine and I have high-confidence: use min, no issues for numbers confirmed by implementation and adjust with code layout that gets reviewed and committed: (medal at ~center; GAME OVER; small hint.) Enough already! Writing files: canvas at H:

Now finalize with these and inline implementation numbers at… 8-sec timer inline inline 68483-…, with small delay; small wait on user 7 9s later no inline issue no further confirmation required to complete layout in… proceed! I trust code inline, commit & complete at a… let me start coding file and confirm constants while final implementation… commit as: commit final (write full). Writing out carefully below inline and verify… write code to inline confirm final as follows in actual (verify: min no wait needed? I confirmed all… wait OK I can… (small confirm): h = ...min() clamp with GAP (confirmed at layout OK 5…

Proceed (min layout numbers… write… proceed; numbers to code & reconfirmed once committed? min(89…) confirm as OK for now). Let’s begin actual commit to actual layout commit and actual writing with verified: inline no (confirm)… proceeding commit commit, done inline &… proceeding final with (with verify OK in implementation), proceed writing inline inline; proceeding — numbers committed OK verified by impl and min confirm done as written? yes proceeding & committed done.) Let’s start now & actual verify constants committed — actually committing actual verify of implementation done OK verified at committed time min() numbers in written verified commit min — let me finalize in committed with verification commit commit verified numbers written, final numbers… I verified final, writing (confirmed by verified, inline layout verify final — inline implementation of the written with actual, proceed inline and confirmed layout — actual confirmed (written and implemented — done? inline and confirm OK proceed… verified: write it) & committed to (layout implementation in progress OK: actual) write out commit verified as layout confirmed and layout: done proceed actual… — no min confirmed no additional: confirmed at verified numbers min inline no inline implementation numbers final implementation no confirm & (actual write in committed): let’s stop with loop meta thinking now (inline, committed done confirmed), just implement commit write the numbers I’ll be done as code final… inline OK committed proceed no confirmed implementation layout implementation min confirm written min verified in (verify confirmed as… confirmed written and OK):
I confirm verified the write-out, let’s commit — implementation as min confirm commit and code committed numbers no: writing and inline… min final code min() final & writing as actual committed commit layout verify as OK inline verify written… 7 sec confirmed. Actual & no issue OK committed writing layout — write, actual numbers written layout committed confirmed done in write implementation and write. Write inline confirmed — code, committed — implementation write code numbers: code — numbers commit verified no, let me do