File size: 3,591 Bytes
0015ced | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 | ---
library_name: mlx
pipeline_tag: text-generation
license: apache-2.0
base_model: IFM/K2-Horizon-0.9B
base_model_relation: quantized
tags:
- mlx
- safetensors
- 4-bit
- conversational
---
# K2-Horizon-0.9B-mlx-4bit
> 4-bit MLX conversion of [IFM/K2-Horizon-0.9B](https://huggingface.co/IFM/K2-Horizon-0.9B) for Apple Silicon.
**Converted by**: [SirSahOl](https://huggingface.co/SirSahOl)
**Source model**: [IFM/K2-Horizon-0.9B](https://huggingface.co/IFM/K2-Horizon-0.9B)
**Framework**: [MLX](https://github.com/ml-explore/mlx) by Apple
**Quantization**: 4-bit
**Format**: safetensors
**License**: apache-2.0
---
## Quick Start
### Installation
```bash
pip install mlx-lm
```
### CLI Usage
```bash
# Chat interactively
mlx_lm.chat --model SirSahOl/K2-Horizon-0.9B-chat-mlx-4bit
# Generate text
mlx_lm.generate --model SirSahOl/K2-Horizon-0.9B-chat-mlx-4bit --prompt "Your prompt here"
```
### Python Usage
```python
from mlx_lm import load, generate
model, tokenizer = load("SirSahOl/K2-Horizon-0.9B-chat-mlx-4bit")
response = generate(model, tokenizer, prompt="Your prompt here", max_tokens=256)
print(response)
```
---
## Performance Benchmarks
| Metric | 4-bit | |--------|--------|| **Tokens/sec** | 78.32 | | **TTFT** | 12.77 ms | | **Peak Memory** | 772.2 MB |
> Benchmarked on **Apple M1** with **8GB** unified memory.
> Average over 5 runs with 256 max tokens.
---
## Who Should Use This?
| Your Hardware | Recommended Quantization |
|---------------|--------------------------|
| M1/M2 (8GB) | **4-bit** — Best balance of quality and memory usage |
| M1/M2 Pro/Max (16-32GB) | **8-bit** — Higher quality with reasonable memory |
| M2/M3/M4 Ultra (64GB+) | **16-bit** — Full precision, no quality loss |
**General guidance:**
- Use **4-bit** if you want to run this model alongside other applications
- Use **8-bit** if you have the memory and want better quality
- Use **16-bit** for research, evaluation, or if memory isn't a concern
---
## Other Quantization Variants
| Variant | Link |
|---------|------|
| 4-bit | [SirSahOl/K2-Horizon-0.9B-chat-mlx-4bit](https://huggingface.co/SirSahOl/K2-Horizon-0.9B-chat-mlx-4bit) |
---
## Conversion Details
| Property | Value |
|----------|-------|
| **Source Model** | [IFM/K2-Horizon-0.9B](https://huggingface.co/IFM/K2-Horizon-0.9B) |
| **Quantization** | 4-bit |
| **mlx-lm Version** | 0.31.3 |
| **Conversion Time** | 5.01s |
| **Output Size** | 583.5 MB |
| **Date** | 2026-09-11T16:49:27.927001+00:00 |
### Reproduction
To reproduce this conversion:
```bash
pip install mlx-lm==0.31.3
python3 -m mlx_lm.convert --hf-path IFM/K2-Horizon-0.9B --mlx-path output/K2-Horizon-0.9B-mlx-4bit -q --q-bits 4
```
---
## Limitations & Known Issues
- Performance may degrade with very long contexts (>8K tokens) at lower quantization levels.
- This is a weight-only conversion; the model architecture and behavior are inherited from the source model.
- Quantization introduces a small quality loss compared to the original model. Lower bit counts = more loss.
- This model requires Apple Silicon (M1 or later) to run with MLX.
---
## License
This model conversion inherits the license of the source model: **apache-2.0**.
See the [original model card](https://huggingface.co/IFM/K2-Horizon-0.9B) for full license details.
---
## Changelog
| Version | Date | Changes |
|---------|------|---------|
| v1.0 | 2026-09-11 | Initial conversion |
---
*Converted with [MLX Foundry](https://github.com/SirSahOl/mlx-foundry) — a professional pipeline for converting models to Apple MLX format.*
|