Model is broken
The model constantly asks for tool calls and system time while thinking. It also quotes things incorrectly from input, omitting tokens. Often times it gives severely truncated outputs.
Weird, it works fine for me... What are you using ? Which version of llama cpp ? Chat completion mode or text completion ?
Using latest build from Unsloth's latest update on their PR. I am just using the webui.
llama-server --model C:/Users/matt/Downloads/GLM-5.3-Flash-UNCENSORED-FP8-UD-Q4_K_XL-00001-of-00005.gguf -ts 99,0 --temp 1.0 --top-p 0.95 --min-p 0.01 --jinja -fa on -c 262000 --threads 38 -ngl 99 --n-cpu-moe 75
Looks like it works fine, did you change something ?
I think I was confused on how a model could ask for tools by default. I probably missed that they do that now.
I didn't know either x)
This is what it does sometimes. It will just do 1000 tokens or so of tool call / local time and then output nothing. The prompt has nothing to do with that. Could it be an issue with Unsloth?
I've never had this issue with Q4_K_XL from Unsloth, but I have not tested it enough to know if it is an issue there as well.
The latest test was 1086 tokens of asking about tool calls and OS info and the final response was the short above.
I did test going back and forth between each model with the same image asking what movie it is from. Yours consistently will go into an infinite loop where it says the same word repeatedly and the Unsloth model will not do this. Both models are Q4_K_XL with the same prompt, same arguments, etc. Your model is definitely broken.
Okay, thank you for your tests
I'll experiment and upload a new version when it's repaired
I think I found the culprit ^^
Oh yeah, definitely. I saw those updates in the files this morning. I was honestly worried you might not have had the updated ones.
Tbh, I was in the process of downloading them to make my own unsloth GGUF when I saw your upload.
I'm starting the process over, the ggufs will be ready tomorrow (it's 2 am in my country)
I uploaded the fixed Q4_K_XL (new weights and new chat_template.jinja), does it work for you?
(To me, the model went from great to insane, I'm shocked of what Z.ai cooked)
Q2_K_XL please.
I'm currently making the UD-Q2_K_XL version. I'm copy pasting the tensor types and imatrix from unsloth's UD-Q2_K_XL. So if unsloth's quantization has issues, mine will have it too. Let me know when Unsloth will update their UD-Q2_K_XL, so I can redo the quant
So far so good. It won't fall into a repeat pattern from asking it about an image anymore. It keeps asking for tools mid thinking but that is probably okay if you are coding.
Am I correct in seeing that you've also updated the UD-Q3_K_XL quants as well?
Hey, I'll be the guy who asks for the Q3-xxs because it's the closest version to 128GB of memory. And because the UD-Q2_K_XL version loses at least 4% top-1 accuracy compared to this Q3.