# NeoHorse-1-9B MLX Quantized Variants

This repository contains four MLX-quantized variants of [TokenRhythm/NeoHorse-1-9B](https://huggingface.co/TokenRhythm/NeoHorse-1-9B), a 9B-parameter language model fine-tuned from Qwen3.5-9B for agentic post-training.

## Model Details

| Property | Value |
|---|---|
| Model family | NeoHorse Agent-Native Causal Language Model |
| Parameters | ~9B |
| Base model | [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) |
| Architecture | Qwen3.5 hybrid (gated-delta linear attention + full attention) |
| Context length | 262,144 tokens native, extensible to 1,010,000 |
| Weight format | Safetensors / BF16 (source) |
| License | Apache 2.0 |

## Quantization Variants

| Variant | Size | Method | Description |
|---|---|---|---|
| [oQ4e](https://huggingface.co/hermitdave/NeoHorse-1-9B-oQ4e) | ~5.3 GB | oMLX imatrix-enhanced | Mixed-precision 4-bit, sensitivity-calibrated |
| [oQ6e](https://huggingface.co/hermitdave/NeoHorse-1-9B-oQ6e) | ~7.5 GB | oMLX imatrix-enhanced | Mixed-precision 6-bit, near-lossless |
| [6-bit](https://huggingface.co/hermitdave/NeoHorse-1-9B-6bit) | ~7.3 GB | mlx_lm.convert | Uniform 6-bit affine |
| [8-bit](https://huggingface.co/hermitdave/NeoHorse-1-9B-8bit) | ~9.5 GB | mlx_lm.convert | Uniform 8-bit affine |

## Usage

### mlx-lm

```python
from mlx_lm import load, generate

model, tokenizer = load("hermitdave/NeoHorse-1-9B-oQ4e")

messages = [{"role": "user", "content": "Write a Python function that returns the first n Fibonacci numbers."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, max_tokens=512)
```

### oMLX

```bash
omlx serve hermitdave/NeoHorse-1-9B-oQ4e --port 8000
```

### CLI

```bash
python -m mlx_lm.chat --model hermitdave/NeoHorse-1-9B-oQ4e
```

## Evaluation

NeoHorse-1-9B achieves a 69.04 macro average across ten benchmarks vs 65.60 for Qwen3.5-9B (+3.44). See the [original model card](https://huggingface.co/TokenRhythm/NeoHorse-1-9B) for full evaluation details.

## Conversion Notes

- **oQ4e / oQ6e**: Quantized using [oMLX](https://github.com/jundot/omlx) `quantize_oq_streaming` with imatrix calibration (128 samples, 512 seq length). These are mixed-precision formats — bit width varies per layer based on sensitivity measurement.
- **6-bit / 8-bit**: Quantized using `mlx_lm.convert` with uniform affine quantization (group_size=64).
- **MTP**: The source checkpoint declares `mtp_num_hidden_layers: 1` but contains no `mtp.*` weights (dropped during NeoHorse's repackaging from Qwen3.5-9B). These models do not include MTP heads.

## Citation

```bibtex
@misc{neohorse2026,
  title        = {NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness},
  author       = {NeoHorse Team},
  year         = {2026},
  howpublished = {arXiv preprint}
}
```

## Acknowledgments

- [TokenRhythm](https://huggingface.co/TokenRhythm) for the NeoHorse-1-9B model
- [Qwen](https://huggingface.co/Qwen) for the Qwen3.5-9B base model
- [oMLX](https://github.com/jundot/omlx) for the oQ quantization pipeline
- [mlx-lm](https://github.com/ml-explore/mlx-lm) for uniform quantization
- [Hermes Agent](https://hermes-agent.nousresearch.com/) for conversion orchestration