---
library_name: mlx
pipeline_tag: text-generation
license: apache-2.0
base_model: IFM/K2-Horizon-0.9B
base_model_relation: quantized
tags:
  - mlx
  - safetensors
  - 4-bit
  - conversational
---

# K2-Horizon-0.9B-mlx-4bit

> 4-bit MLX conversion of [IFM/K2-Horizon-0.9B](https://huggingface.co/IFM/K2-Horizon-0.9B) for Apple Silicon.

**Converted by**: [SirSahOl](https://huggingface.co/SirSahOl)
**Source model**: [IFM/K2-Horizon-0.9B](https://huggingface.co/IFM/K2-Horizon-0.9B)
**Framework**: [MLX](https://github.com/ml-explore/mlx) by Apple
**Quantization**: 4-bit
**Format**: safetensors
**License**: apache-2.0

---

## Quick Start

### Installation

```bash
pip install mlx-lm
```

### CLI Usage

```bash
# Chat interactively
mlx_lm.chat --model SirSahOl/K2-Horizon-0.9B-chat-mlx-4bit

# Generate text
mlx_lm.generate --model SirSahOl/K2-Horizon-0.9B-chat-mlx-4bit --prompt "Your prompt here"
```

### Python Usage

```python
from mlx_lm import load, generate

model, tokenizer = load("SirSahOl/K2-Horizon-0.9B-chat-mlx-4bit")
response = generate(model, tokenizer, prompt="Your prompt here", max_tokens=256)
print(response)
```

---

## Performance Benchmarks

| Metric | 4-bit | |--------|--------|| **Tokens/sec** | 78.32 | | **TTFT** | 12.77 ms | | **Peak Memory** | 772.2 MB | 
> Benchmarked on **Apple M1** with **8GB** unified memory.
> Average over 5 runs with 256 max tokens.

---

## Who Should Use This?

| Your Hardware | Recommended Quantization |
|---------------|--------------------------|
| M1/M2 (8GB) | **4-bit** — Best balance of quality and memory usage |
| M1/M2 Pro/Max (16-32GB) | **8-bit** — Higher quality with reasonable memory |
| M2/M3/M4 Ultra (64GB+) | **16-bit** — Full precision, no quality loss |

**General guidance:**
- Use **4-bit** if you want to run this model alongside other applications
- Use **8-bit** if you have the memory and want better quality
- Use **16-bit** for research, evaluation, or if memory isn't a concern

---

## Other Quantization Variants

| Variant | Link |
|---------|------|
| 4-bit | [SirSahOl/K2-Horizon-0.9B-chat-mlx-4bit](https://huggingface.co/SirSahOl/K2-Horizon-0.9B-chat-mlx-4bit) |

---

## Conversion Details

| Property | Value |
|----------|-------|
| **Source Model** | [IFM/K2-Horizon-0.9B](https://huggingface.co/IFM/K2-Horizon-0.9B) |
| **Quantization** | 4-bit |
| **mlx-lm Version** | 0.31.3 |
| **Conversion Time** | 5.01s |
| **Output Size** | 583.5 MB |
| **Date** | 2026-09-11T16:49:27.927001+00:00 |

### Reproduction

To reproduce this conversion:

```bash
pip install mlx-lm==0.31.3
python3 -m mlx_lm.convert --hf-path IFM/K2-Horizon-0.9B --mlx-path output/K2-Horizon-0.9B-mlx-4bit -q --q-bits 4
```

---

## Limitations & Known Issues

- Performance may degrade with very long contexts (>8K tokens) at lower quantization levels.
- This is a weight-only conversion; the model architecture and behavior are inherited from the source model.
- Quantization introduces a small quality loss compared to the original model. Lower bit counts = more loss.
- This model requires Apple Silicon (M1 or later) to run with MLX.

---

## License

This model conversion inherits the license of the source model: **apache-2.0**.

See the [original model card](https://huggingface.co/IFM/K2-Horizon-0.9B) for full license details.

---

## Changelog

| Version | Date | Changes |
|---------|------|---------|
| v1.0 | 2026-09-11 | Initial conversion |

---

*Converted with [MLX Foundry](https://github.com/SirSahOl/mlx-foundry) — a professional pipeline for converting models to Apple MLX format.*