mlx-community/NeoHorse-1-4B-4bit

NeoHorse-1-4B — MLX 4-bit

A 4-bit MLX port of NeoHorse-1-4B by TokenRhythm, quantized for Apple Silicon.

This is the same fine-tuned model — just repacked as a uniform 4-bit safetensor so it runs on MLX. Vision weights are not included; this is text-only, matching the upstream release.

What changed from the upstream model

Property	Original (BF16)	This release
Precision	bfloat16	4-bit GPTQ (uniform)
Format	HuggingFace transformers	MLX safetensors
Size	~8 GB	~2.2 GB
Context	262,144	262,144

The tensor values are unchanged. The only difference is the quantization and the format swap.

Architecture

NeoHorse-1-4B is a hybrid-attention model post-trained from Qwen3.5-4B:

• 32 layers, mostly linear attention with full-attention layers every 4th position (4 total: layers 3, 7, 11, 15, 19, 23, 27, 31)
• GQA: 16 query heads, 4 KV heads
• Partial RoPE (0.25 rotary factor) — only a quarter of dimensions get positional encoding
• 262K context natively

The hybrid attention means the KV cache grows slowly — only the full-attention layers add per-token state, so long contexts stay cheap.

Validation

Ran a quick sanity check on an M3 Max 64GB:

Prompt: "Explain the theory of relativity in two sentences."
Generation: 128 tokens, 108.6 tokens/sec
Peak memory: 2.48 GB

The output was coherent chain-of-thought about special and general relativity. Quantization held up.

Quick start

Requires mlx-lm:

pip install mlx-lm

Generate from Python:

from mlx_lm import load, generate

model, tokenizer = load("hermitdave/NeoHorse-1-4B-4bit")

response = generate(
    model,
    tokenizer,
    prompt="Write a Python function that returns the first n Fibonacci numbers.",
    max_tokens=512,
)
print(response)

You'll also need to patch mlx-lm's model remapping if it doesn't already include qwen3_5_text. In mlx_lm/utils.py, add:

MODEL_REMAPPING = {
    ...
    "qwen3_5_text": "qwen3_5",
}

(If you're on mlx-lm >= 0.31.3 with the patch already applied, this works out of the box.)

Performance notes

Chip	Tokens/sec	Peak memory (4B-4bit)
M3 Max	108	~2.5 GB
M2/M3	~90	~2.5 GB

The model fits comfortably in 8 GB RAM, but 16 GB+ is recommended for headroom with longer contexts.

Benchmarks

These are from the upstream model — the 4-bit quant preserves the same weights, so accuracy should match.

Ten-benchmark average: 64.87 vs 58.94 for Qwen3.5-4B (+5.93)

Benchmark	NeoHorse-1-4B	Qwen3.5-4B	Δ
QwenClawBench	44.68	38.47	+6.21
WorkBuddy Bench	34.41	24.62	+9.79
PinchBench	77.33	71.19	+6.14
VitaBench	32.00	21.50	+10.50
BFCL v4	61.79	61.02	+0.77
tau2-Bench	88.46	84.29	+4.17
HumanEval	96.95	87.20	+9.75
LiveCodeBench v6	59.43	53.71	+5.72
IFBench	65.33	60.33	+5.00
IFEval	88.35	87.06	+1.29

Reported protocol: SGLang, temperature=1.0, top_p=0.95, thinking mode enabled.

Credits

• TokenRhythm — for NeoHorse-1-4B and the routing-harness post-training
• Qwen / Alibaba Cloud — for the Qwen3.5-4B base model
• Hermes Agent (by Nous Research) — for the MLX conversion pipeline and validation workflow

License

Apache 2.0. The upstream model copyright (© 2026 Alibaba Cloud) is retained in the original release. Tensor values are unchanged by this repackaging.

Citation

@misc{neohorse2026,
  title  = {NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness},
  author = {NeoHorse Team},
  year   = {2026},
  eprint = {2609.08183},
  url    = {https://arxiv.org/abs/2609.08183}
}