File size: 3,313 Bytes
9d33141 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | # NeoHorse-1-9B MLX Quantized Variants
This repository contains four MLX-quantized variants of [TokenRhythm/NeoHorse-1-9B](https://huggingface.co/TokenRhythm/NeoHorse-1-9B), a 9B-parameter language model fine-tuned from Qwen3.5-9B for agentic post-training.
## Model Details
| Property | Value |
|---|---|
| Model family | NeoHorse Agent-Native Causal Language Model |
| Parameters | ~9B |
| Base model | [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) |
| Architecture | Qwen3.5 hybrid (gated-delta linear attention + full attention) |
| Context length | 262,144 tokens native, extensible to 1,010,000 |
| Weight format | Safetensors / BF16 (source) |
| License | Apache 2.0 |
## Quantization Variants
| Variant | Size | Method | Description |
|---|---|---|---|
| [oQ4e](https://huggingface.co/hermitdave/NeoHorse-1-9B-oQ4e) | ~5.3 GB | oMLX imatrix-enhanced | Mixed-precision 4-bit, sensitivity-calibrated |
| [oQ6e](https://huggingface.co/hermitdave/NeoHorse-1-9B-oQ6e) | ~7.5 GB | oMLX imatrix-enhanced | Mixed-precision 6-bit, near-lossless |
| [6-bit](https://huggingface.co/hermitdave/NeoHorse-1-9B-6bit) | ~7.3 GB | mlx_lm.convert | Uniform 6-bit affine |
| [8-bit](https://huggingface.co/hermitdave/NeoHorse-1-9B-8bit) | ~9.5 GB | mlx_lm.convert | Uniform 8-bit affine |
## Usage
### mlx-lm
```python
from mlx_lm import load, generate
model, tokenizer = load("hermitdave/NeoHorse-1-9B-oQ4e")
messages = [{"role": "user", "content": "Write a Python function that returns the first n Fibonacci numbers."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, max_tokens=512)
```
### oMLX
```bash
omlx serve hermitdave/NeoHorse-1-9B-oQ4e --port 8000
```
### CLI
```bash
python -m mlx_lm.chat --model hermitdave/NeoHorse-1-9B-oQ4e
```
## Evaluation
NeoHorse-1-9B achieves a 69.04 macro average across ten benchmarks vs 65.60 for Qwen3.5-9B (+3.44). See the [original model card](https://huggingface.co/TokenRhythm/NeoHorse-1-9B) for full evaluation details.
## Conversion Notes
- **oQ4e / oQ6e**: Quantized using [oMLX](https://github.com/jundot/omlx) `quantize_oq_streaming` with imatrix calibration (128 samples, 512 seq length). These are mixed-precision formats — bit width varies per layer based on sensitivity measurement.
- **6-bit / 8-bit**: Quantized using `mlx_lm.convert` with uniform affine quantization (group_size=64).
- **MTP**: The source checkpoint declares `mtp_num_hidden_layers: 1` but contains no `mtp.*` weights (dropped during NeoHorse's repackaging from Qwen3.5-9B). These models do not include MTP heads.
## Citation
```bibtex
@misc{neohorse2026,
title = {NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness},
author = {NeoHorse Team},
year = {2026},
howpublished = {arXiv preprint}
}
```
## Acknowledgments
- [TokenRhythm](https://huggingface.co/TokenRhythm) for the NeoHorse-1-9B model
- [Qwen](https://huggingface.co/Qwen) for the Qwen3.5-9B base model
- [oMLX](https://github.com/jundot/omlx) for the oQ quantization pipeline
- [mlx-lm](https://github.com/ml-explore/mlx-lm) for uniform quantization
- [Hermes Agent](https://hermes-agent.nousresearch.com/) for conversion orchestration
|