First of all, thank you for taking all the time and effort involved to download, convert/quantize and upload this. I've been excited about the possibility of running GLM 4.6 with the help of pruning nd was waiting for someone to upload a quant that could run on my 96GB Mac Studio.
I had previously tried a 178B quant by someone else, Q3K GGUF format, but unfortunately it seemed to suffer from a catastrophic loss of general knowledge, not knowing who various people I prompted it about were.
I downloaded your own 3-bit MLX quant of the 218B version hoping a less pruned version might retain more general knowledge, but unfortunately it seems to have the same problem, perhaps because the dataset is optimised for tasks like coding and the experts pruned impact on general knowledge.
The responses themselves are coherent and the model seems to understand what's being asked, with just a bit of confusion, so the quant itself is fine, but the model does not seem to know anything about the people - fictional or real - I'm asking about, making it unsuitable for me to use for RP/creative writing which is my intended use.
Did you have an opportunity to try these quants yourself, and how did you find it? Does it work for your own purposes?
Perhaps what's needed here is pruned models to specialise in different areas. It'd be nice to see one tailored to creative writing, general knowledge and roleplay. Pruning makes models more accessible, but they're not much use if they don't support what you wanted to use them for because those experts have been pruned.
