jcbtc

/

Qwen3.8-27B-IU4-Kairic-Edge

Like 47

I'd like to address the looping issue, next time it happens can you ask it for a log and send to jc@ciru.ai please ty <3

Is the default log sufficient?
Or do I need to add extra options to the Lama launch command?

it slow when coding with recommended setting
9.57.815.055 I slot print_timing: id 0 | task 1055 | n_decoded = 725, tg = 22.49 t/s
10.00.815.255 I slot print_timing: id 0 | task 1055 | n_decoded = 790, tg = 22.42 t/s
10.04.022.399 I slot print_timing: id 0 | task 1055 | n_decoded = 840, tg = 21.85 t/s
10.07.214.587 I slot print_timing: id 0 | task 1055 | n_decoded = 874, tg = 20.99 t/s
10.10.432.447 I slot print_timing: id 0 | task 1055 | n_decoded = 924, tg = 20.60 t/s
10.13.433.429 I slot print_timing: id 0 | task 1055 | n_decoded = 969, tg = 20.25 t/s
10.16.602.407 I slot print_timing: id 0 | task 1055 | n_decoded = 1004, tg = 19.68 t/s
10.19.786.039 I slot print_timing: id 0 | task 1055 | n_decoded = 1045, tg = 19.28 t/s
10.22.990.160 I slot print_timing: id 0 | task 1055 | n_decoded = 1099, tg = 19.14 t/s
10.34.594.898 I slot print_timing: id 0 | task 1283 | n_decoded = 103, tg = 15.65 t/s
10.37.672.797 I slot print_timing: id 0 | task -1 | n_decoded = 154, tg = 15.94 t/s
10.54.895.687 I slot print_timing: id 0 | task 1323 | n_decoded = 102, tg = 14.31 t/s
10.58.090.711 I slot print_timing: id 0 | task 1323 | n_decoded = 157, tg = 15.21 t/s
11.01.263.713 I slot print_timing: id 0 | task 1323 | n_decoded = 198, tg = 14.67 t/s
11.04.419.767 I slot print_timing: id 0 | task 1323 | n_decoded = 233, tg = 13.99 t/s
11.07.581.829 I slot print_timing: id 0 | task 1323 | n_decoded = 269, tg = 13.58 t/s
11.10.794.791 I slot print_timing: id 0 | task 1323 | n_decoded = 330, tg = 14.33 t/s
11.21.351.848 I slot print_timing: id 0 | task 1429 | n_decoded = 100, tg = 12.89 t/s
11.24.537.669 I slot print_timing: id 0 | task 1429 | n_decoded = 130, tg = 11.88 t/s
11.28.644.892 I slot print_timing: id 0 | task -1 | n_decoded = 193, tg = 12.83 t/s
11.52.084.833 I slot print_timing: id 0 | task 1487 | n_decoded = 100, tg = 9.19 t/s
11.55.308.490 I slot print_timing: id 0 | task 1487 | n_decoded = 144, tg = 10.21 t/s
11.58.489.660 I slot print_timing: id 0 | task 1487 | n_decoded = 186, tg = 10.76 t/s
12.01.700.048 I slot print_timing: id 0 | task 1487 | n_decoded = 232, tg = 11.32 t/s
12.04.897.915 I slot print_timing: id 0 | task 1487 | n_decoded = 272, tg = 11.48 t/s
12.08.068.161 I slot print_timing: id 0 | task 1487 | n_decoded = 302, tg = 11.24 t/s
12.11.245.391 I slot print_timing: id 0 | task 1487 | n_decoded = 329, tg = 10.95 t/s
12.14.415.016 I slot print_timing: id 0 | task 1487 | n_decoded = 353, tg = 10.63 t/s
12.17.600.944 I slot print_timing: id 0 | task 1487 | n_decoded = 382, tg = 10.50 t/s
12.20.775.337 I slot print_timing: id 0 | task 1487 | n_decoded = 418, tg = 10.56 t/s
12.23.805.620 I slot print_timing: id 0 | task 1487 | n_decoded = 457, tg = 10.73 t/s
12.27.034.682 I slot print_timing: id 0 | task 1487 | n_decoded = 487, tg = 10.63 t/s
12.30.247.879 I slot print_timing: id 0 | task 1487 | n_decoded = 508, tg = 10.36 t/s
12.33.449.751 I slot print_timing: id 0 | task 1487 | n_decoded = 531, tg = 10.16 t/s
12.36.675.878 I slot print_timing: id 0 | task 1487 | n_decoded = 558, tg = 10.06 t/s
12.39.888.659 I slot print_timing: id 0 | task 1487 | n_decoded = 580, tg = 9.88 t/s
12.43.096.297 I slot print_timing: id 0 | task 1487 | n_decoded = 606, tg = 9.79 t/s
12.46.097.053 I slot print_timing: id 0 | task 1487 | n_decoded = 640, tg = 9.86 t/s
12.49.296.822 I slot print_timing: id 0 | task 1487 | n_decoded = 664, tg = 9.75 t/s
12.52.473.844 I slot print_timing: id 0 | task 1487 | n_decoded = 698, tg = 9.79 t/s
12.55.693.287 I slot print_timing: id 0 | task 1487 | n_decoded = 743, tg = 9.97 t/s

Found the bug and made a fix, emailed to Ciru, hope to have this incorporated today.

Also created a patch

Found the bug and made a fix, emailed to Ciru, hope to have this incorporated today.

Thanks.
I applied your patch and set the temperature back to 0.
I also disabled reasoning.
After a while, the agent went into a loop.
I'll send the jcbtc log

OK, I don't know if it's the same bug, let's see what Ciru thinks. I'm about to run some tests on this patched model myself.

Temperature 0 is likely to cause a loop, especially with reasoning off.
Qwen's recommended non-thinking settings (temp 0.7, top_p 0.8, top_k 20).
Global AGENTS.md — "print a confirmation after every cell, keep a done/next checklist, never re-run an error-free cell, change approach after two no-progress attempts." (That's the guard a 27B non-thinking model needs regardless of sampling.)

Temperature 0 is likely to cause a loop, especially with reasoning off.
Qwen's recommended non-thinking settings (temp 0.7, top_p 0.8, top_k 20).
Global AGENTS.md — "print a confirmation after every cell, keep a done/next checklist, never re-run an error-free cell, change approach after two no-progress attempts." (That's the guard a 27B non-thinking model needs regardless of sampling.)

Thanks, I'll test out your settings

TBH, the agents.md might fix loops with temperature 0, which will also increase speed and consistency. The temperature will increase randomness which means it's going to try different things in its nature.

Kairic 1.2 is released on model card thank you

TBH, the agents.md might fix loops with temperature 0, which will also increase speed and consistency. The temperature will increase randomness which means it's going to try different things in its nature.

Using your prompt on a single pi task with preloaded context, when I saw the loop I stopped the task and
restarted llama with new parameters:

1. temperature 0, top-p 1, top-k 0, min-p 0, reasoning off -> didn't help
2. temperature 0.8, top-p 0.8, top-k 20, reasoning off -> didn't help
3. temperature 0, top-p 1, top-k 0, min-p 0 + reasoning on -> worked
4. temperature 0.8, top-p 0.8, top-k 20 with reasoning on — I think this will work too, since that's what I
   started all of this with

Some kind of magic — without reasoning on, the agent loop doesn't work for me on pi.

Real task with reasoning
old context:
Breakdown of your metrics:

Prefill: 274 t/s (68223 tokens / 248.9 s)
Within the expected range for the 8060S. But keep in mind: n_tokens_max = 70409 — there was a single 70K-token
prefill, and it drags the average down. If that was a prompt cache miss (long system prompt), then repeat
requests with the cache will be noticeably faster. Check: run the same task twice and see how
prompt_tokens_seconds changes on the second run.

Decode: 15.1 t/s (3643 / 240.9 s) — effective speed with speculation.

The most interesting number: n_decode_total = 1210
1210 llama_decode() calls for 3643 tokens ≈ 3.0 tokens per call. With n_max=4, each call verifies 1+4=5
positions, so the acceptance rate ≈ 3/5 = ~60%. That's a solid figure — the draft is actually working, not just
noise.

What this means:

• 15.1 t/s at a 60% acceptance rate means your base speed without speculation is somewhere around 11–12 t/s.
  Speculation gives +25–30% on average.
• But that's an average over 3643 tokens. If it was prose, the ngram lane (n_match=24) barely kicks in. On
  agentic content (repeated JSON, tool calls, system prompt) the gain will be noticeably larger.

What to check:

1. Clean baseline: run the same task with --spec-type none → you'll learn the real base. If the base is 11–12
   and with speculation on an agentic task it's 20+, everything is working as intended.
2. Prompt cache: two identical requests in a row → the second prefill should be several times faster.
3. Fresh numbers: the counters are cumulative since server start. For per-task numbers, read the log (prompt
   eval time / completion eval time) or restart the server before measuring.

For an agentic loop, 15 t/s is fine: tool calls are short, and your long generations are mostly reasoning, where
quality matters more than speed.

New session

• Prefill throughput: 84405 / 363.341 = 232.30 ✓
• Decode throughput: 38635 / 2170.06 = 17.803 ✓
• Total time: 363.341 + 2170.06 = 2533.40 sec = 42.2 min
• Decode fraction: 2170.06 / 2533.40 = 85.66%
• Prefill fraction: 363.341 / 2533.40 = 14.34%
• Prompt:gen token ratio: 84405 / 38635 = 2.185
• Tokens per decode call: 38635 / 10910 = 3.541
• n_tokens_max = 84836 — this is the maximum observed n_tokens (maximum sequence length). This is close to the
  total prompt tokens (84405), suggesting that at some point a very long sequence (about 84836 tokens) was
  processed. This is probably the maximum context size used. Actually, n_tokens_max is the maximum n_tokens
  observed in a single llama_decode/llama_batch, or the maximum context. 84836 is close to 84405 (total prompt
  tokens), which is a bit odd — it suggests that either one huge batch of about 84836 tokens was processed, or
  the context window is about 84836.

I’m prepared to decode ~20 tokens at real-time speed, but not 40+ in a real-world scenario with a large context.
I'm not chasing speed — quality is what matters to me.

With the benchmark of https://llm.ciru.ai/invoicesandbox/, the model work well.
I reach
9.16.045.686 I slot print_timing: id 0 | task 1587 |
prompt eval time = 2930.39 ms / 662 tokens ( 4.43 ms per token, 225.91 tokens per second)
eval time = 3776.46 ms / 84 tokens ( 44.96 ms per token, 22.24 tokens per second)
total time = 6706.85 ms / 746 tokens

thats a great looking dashboard!

yes, good dashboard and same result as Hy3 and Laguna S2.1. Thank you for your work. For me, the laguna S2.1 still better than this Qwen model. Not because the result or more beautiful dashboard, just because the token rate and temperature when it run in my laptop. When I run with Qwen, my laptop is around 91 degree instead of 85 degree as Laguna