llama problem?
14.57.924.130 I slot print_timing: id 0 | task 78 | n_gen = 18808, tg = 23.36 t/s, tg_3s = 16.81 t/s
15.00.989.811 I slot print_timing: id 0 | task 78 | n_gen = 18865, tg = 23.34 t/s, tg_3s = 18.59 t/s
15.04.013.004 I slot print_timing: id 0 | task 78 | n_gen = 18929, tg = 23.33 t/s, tg_3s = 21.17 t/s
15.07.050.109 I slot print_timing: id 0 | task 78 | n_gen = 18998, tg = 23.33 t/s, tg_3s = 22.72 t/s
15.10.060.503 I slot print_timing: id 0 | task 78 | n_gen = 19079, tg = 23.34 t/s, tg_3s = 26.91 t/s
15.11.144.932 I slot print_timing: id 0 | task 78 | prompt eval time = 3151.10 ms / 1128 tokens ( 2.79 ms per token, 357.97 tokens per second)
15.11.144.936 I slot print_timing: id 0 | task 78 | eval time = 818460.27 ms / 19116 tokens ( 42.82 ms per token, 23.35 tokens per second)
15.11.144.936 I slot print_timing: id 0 | task 78 | total time = 821611.37 ms / 20244 tokens
15.11.144.938 I slot print_timing: id 0 | task 78 | graphs reused = 8217
15.11.144.943 I slot print_timing: id 0 | task 78 | draft acceptance = 0.43862 (10844 accepted / 24723 generated), mean len = 2.32
15.11.145.103 I slot release: id 0 | task 78 | stop processing: n_tokens = 21465, truncated = 0
15.11.302.580 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.160 (> 0.100 thold), f_keep = 0.163
15.11.507.406 I slot launch_slot_: id 0 | task 8356 | processing task, is_child = 0
15.15.917.273 E init: sequence 0 positions are decreasing (not allowed)
15.15.917.276 E decode: failed to initialize batch
15.15.917.277 E llama_decode: failed to decode, ret = -1
15.15.917.280 E spec process: llama_decode(ctx_dft) head=0 failed rc=-1 (pos=2344)
15.15.917.543 E srv decode: failed to process speculative batch
15.15.917.626 E srv update_slots: decode() failed: failed to process speculative batch
15.15.917.634 E srv send_error: task id = 8356, error: decode() failed: failed to process speculative batch
15.15.917.641 I slot release: id 0 | task 8356 | stop processing: n_tokens = 4392, truncated = 0
15.15.917.678 W srv stop: cancel task, id_task = 8356
Second task, pi says Error: decode() failed: failed to process speculative batch
Sorry this is a bug I just caught and the fix is up
1)Keep the existing model files; no redownload is needed.
2)Upgrade and rebuild the runtime from tag v1.1.
Confirm:
./llama-server --version
It should report commit:
baba5e0
ok, testing now, code generation:
14.41.631.418 I slot print_timing: id 0 | task 6797 | n_gen = 3876, tg = 32.44 t/s, tg_3s = 32.55 t/s
14.44.660.855 I slot print_timing: id 0 | task 6797 | n_gen = 3987, tg = 32.54 t/s, tg_3s = 36.64 t/s
14.47.755.580 I slot print_timing: id 0 | task 6797 | n_gen = 4094, tg = 32.59 t/s, tg_3s = 34.57 t/s
14.50.822.275 I slot print_timing: id 0 | task 6797 | n_gen = 4197, tg = 32.61 t/s, tg_3s = 33.59 t/s
The larger the context, the more the speed drops, but as long as the system remains stable, I’ll post the results a bit later.
an interesting model
The model doesn't seem to get stuck in a loop with the default settings.
./llama-server --version
version: 0.3.0-dev (build 10675, commit baba5e061)
built with GNU 15.2.0 for Linux x86_64
pi system prompt
print a confirmation after every cell, keep a done/next checklist, never re-run an error-free cell, change approach after two no-progress attempts.
One run without fixes, Oddly enough, the full cycle ran without errors
Create a complete Mario Kart–style arcade racing game
326.21.007.911 I slot print_timing: id 0 | task 63897 | n_gen = 36460, tg = 23.16 t/s, tg_3s = 20.33 t/s
326.22.626.571 I slot print_timing: id 0 | task 63897 | prompt eval time = 1553.68 ms / 164 tokens ( 9.47 ms per token, 105.56 tokens per second)
326.22.626.575 I slot print_timing: id 0 | task 63897 | eval time = 1576131.28 ms / 36490 tokens ( 43.19 ms per token, 23.15 tokens per second)
326.22.626.575 I slot print_timing: id 0 | task 63897 | total time = 1577684.96 ms / 36654 tokens
326.22.626.577 I slot print_timing: id 0 | task 63897 | graphs reused = 71354
326.22.626.581 I slot print_timing: id 0 | task 63897 | draft acceptance = 0.95443 (27021 accepted / 28311 generated), mean len = 3.86
326.22.627.676 I slot release: id 0 | task 63897 | stop processing: n_tokens = 82919, truncated = 0
Really nice result!
I left it running when I went for dinner writing a voxel art demo of a magical dragon with a castle and it was still working when I got back. The result was even better than 5.6 with the same prompt. Very impressed that it didn't ever loop or get confused. It used 90% of the 256k context.
Vision?
I asked the model to add vision capabilities. It independently built mmproj from the official Qwen image, including all the necessary checks.
Everything is working well.
It might be useful, or it might not. Ubuntu 26.04.1 LTS
I was a bit confused as to why top was showing total 122 GB, but I solved the problem on Ubuntu 26.04. with this model
