mlx-community
/

Huihui-gemma-4-12B-it-qat-q4_0-unquantized-abliterated-5bit-msq

image-text-to-text

4-bit precision

Model card Files Files and versions

Huihui-gemma-4-12B-it-qat-q4_0-unquantized-abliterated — MLX 5.0 BPW

Mixed-precision MLX quantization of huihui-ai/Huihui-gemma-4-12B-it-qat-q4_0-unquantized-abliterated, quantized with MLX Smart Quantize (MSQ) — my own sensitivity-based mixed-precision quantization method for Apple Silicon. It measures per-layer NMSE and assigns optimal bit widths automatically, combining architecture knowledge with measured data.

Details

Type: Vision (VLM)
Average: 5.01 bits per weight
Method: MLX Smart Quantize (MSQ)

Downloads last month: 57

Safetensors

Model size

12B params

Tensor type

BF16

·

U32

·

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for mlx-community/Huihui-gemma-4-12B-it-qat-q4_0-unquantized-abliterated-5bit-msq

Base model

google/gemma-4-12B

Finetuned

google/gemma-4-12B-it

Finetuned

google/gemma-4-12B-it-qat-q4_0-unquantized

Quantized

(63)

this model