Quality and Perplexity Test
#1
by Radamanthys11 - opened
I'm a fan of the GLM 5 IQS KS, so when I saw this model, I was excited to test it. Unfortunately, it has too many repetitions issues, and the PPL is quite high compared to the GLM 5.1 IQS KS:
GLM 5.2 IQ2 KS SPECIAL SPLIT: Final estimate: PPL over 100 chunks for n_ctx=512 = 13.7087 +/- 0.23903
GLM 5.1 Smol IQS KS: Final estimate: PPL over 100 chunks for n_ctx=512 = 3.8835 +/- 0.05488
On the bright side, this model is slightly faster here, averaging 11 t/s compared to 10 for the other one.
Well, in any case, I can't help but appreciate the wide range of possibilities you've posted. Let's see how GLM 5.3 performs once the weights are released.