File size: 2,273 Bytes
93760be | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | ---
base_model: Qwen/Qwen3.5-2B
library_name: mlx
tags:
- mlx
- qwen3.5
- vision-language-model
- quantized
- 4bit
license: apache-2.0
---
# Qwen3.5-2B-MLX-4bit
This is a 4-bit quantized MLX version of [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) for Apple Silicon.
## Model Details
- **Original Model:** [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B)
- **Quantization:** 4-bit (6.225 bits per weight)
- **Group Size:** 64
- **Format:** MLX SafeTensors
- **Framework:** [mlx-vlm](https://github.com/Blaizzy/mlx-vlm)
- **Disk Size:** ~1.6G
## Conversion Details
This model was converted using `mlx-vlm` from the [`pc/fix-qwen35-predicate`](https://github.com/Blaizzy/mlx-vlm/tree/pc/fix-qwen35-predicate) branch, which includes fixes for Qwen3.5 model support (proper handling of MoE gate layers, `shared_expert_gate`, and `A_log` casting).
**Conversion command:**
```bash
python3 -m mlx_vlm convert \
--hf-path "Qwen/Qwen3.5-2B" \
--mlx-path "./Qwen3.5-2B-MLX-4bit" \
-q --q-bits 4 --q-group-size 64
```
## Important Note
A better, more optimized conversion may be available from **@Prince** ([@Blaizzy](https://huggingface.co/Blaizzy)) in the MLX VLM community. Check the [mlx-community](https://huggingface.co/mlx-community) organization for updated versions as official Qwen3.5 support is merged into the main `mlx-vlm` branch.
## Related Models
- **bf16 (full precision):** [mlx-community/Qwen3.5-2B-MLX-bf16](https://huggingface.co/mlx-community/Qwen3.5-2B-MLX-bf16)
- **8-bit quantized:** [mlx-community/Qwen3.5-2B-MLX-8bit](https://huggingface.co/mlx-community/Qwen3.5-2B-MLX-8bit)
- **Original:** [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B)
## Usage
```python
from mlx_vlm import load, generate
model, processor = load("mlx-community/Qwen3.5-2B-MLX-4bit")
output = generate(
model,
processor,
prompt="Describe this image.",
image="path/to/image.jpg",
max_tokens=512
)
print(output)
```
**CLI:**
```bash
python3 -m mlx_vlm.generate \
--model mlx-community/Qwen3.5-2B-MLX-4bit \
--image path/to/image.jpg \
--prompt "Describe this image."
```
## License
This model inherits the [Apache 2.0 license](https://huggingface.co/Qwen/Qwen3.5-2B) from the original Qwen model.
|