wezzel98765
/

gemma-4-12B-it-oQ8-fp16

8-bit precision

Model card Files Files and versions

oLMX FAILS TO LOAD

#1

by murphy1296 - opened Jun 4

Jun 4

•

M1 MAX 32GB

ERROR:

gemma-4-12B-it-oQ8-fp16

Error: {"error":{"message":"Internal server error","type":"server_error","param":null,"code":null}})

wezzel98765 changed discussion status to closed Jun 4

wezzel98765 changed discussion status to open Jun 4

Owner Jun 4

Thanks for your message - let me look into it shortly. I’ll re check all quants .

Which OMLX are you running ?

Jun 4

Thanks for the reply - I'm running v0.4.2.dev1

Owner Jun 4

Gotcha! Dev2 has since come out. Ill re quant both of the new Gemma4 models again and have them uploaded approx 11pm GMT

Owner Jun 4

Re-done. Please try again :)

Jun 4

Downloaded but still same problem - did the model work for you?

Jun 4

FYI: I tried another model oQ8-fp16 in oMLX which works fine without problems

Owner Jun 5

Thanks for your message. I too am having issues now actually on this.
Having checked here on HF, I see one other person with an OQ8 version, it is the BF16 one versus FP16 ... that might be the issue. I will look into it further

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment