File size: 5,037 Bytes
5441ed8 0795f30 5441ed8 dc0a083 5441ed8 dc0a083 0795f30 dc0a083 5441ed8 dc0a083 5441ed8 dc0a083 5441ed8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 | ---
library_name: mlx
pipeline_tag: image-text-to-text
inference: false
license: mit
base_model: EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP
base_model_relation: quantized
tags:
- mlx
- omlx
- oq
- quantized
- qwen3.6
- qwen3_5_moe
- moe
- mtp
- multimodal
- conversational
---
# Ornith-1.5-35B-A3B-BigBang-oQ8e-mtp
This repository is a complete Apple MLX deployment of [EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP](https://huggingface.co/EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP), converted with [oMLX](https://github.com/jundot/omlx) v0.6.2 using importance-matrix-enhanced oQ mixed-precision quantization.
Download the whole repository: the Safetensors shards require the included index, model config, tokenizer, chat template, and image/video processor files. This is not a GGUF, Transformers, or NInfer artifact.
## Quick facts
| Item | Value |
|---|---|
| Model type | `qwen3_5_moe` |
| Quantization layout | Affine 8-bit with group size 64 is the default tier; the stored config records 265 per-tensor precision overrides. Ornith overrides use group sizes 64 and 128, so this is not a uniform 8-bit checkpoint. |
| Tensor payload | 38,604,987,852 bytes / 35.95 GiB |
| Safetensors shards | 8 |
| Conversion runtime | oMLX 0.6.2 |
| Calibration | `oqe_code_multilingual`, 128 samples × 512 tokens |
| Included model features | Vision resources and one MTP layer |
| Intended runtime | oMLX on Apple Silicon/macOS |
## Choose a variant
| Variant | Nominal tier | Tensor payload | Shards |
|---|---:|---:|---:|
| [oQ4e](https://huggingface.co/pyros-vault/Ornith-1.5-35B-A3B-BigBang-oQ4e-mtp) | 4-bit | 21,612,915,979 bytes / 20.13 GiB | 5 |
| [oQ6e](https://huggingface.co/pyros-vault/Ornith-1.5-35B-A3B-BigBang-oQ6e-mtp) | 6-bit | 30,131,305,807 bytes / 28.06 GiB | 6 |
| **oQ8e (this repo)** | 8-bit | 38,604,987,852 bytes / 35.95 GiB | 8 |
These tiers differ in storage and quantization layout. No same-Mac quality, memory, TTFT, or throughput comparison is published here, so the table should not be read as a benchmark.
## Download
Install the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli), then place the complete repository below oMLX's model directory:
```bash
mkdir -p "$HOME/.omlx/models/pyros-vault"
hf download pyros-vault/Ornith-1.5-35B-A3B-BigBang-oQ8e-mtp \
--local-dir "$HOME/.omlx/models/pyros-vault/Ornith-1.5-35B-A3B-BigBang-oQ8e-mtp"
```
## Serve with oMLX
Install the current oMLX runtime and start its OpenAI-compatible server:
```bash
brew tap jundot/omlx https://github.com/jundot/omlx
brew install jundot/omlx/omlx
omlx serve --model-dir "$HOME/.omlx/models"
```
Discover the exact model ID exposed by your installed oMLX version:
```bash
curl http://127.0.0.1:8000/v1/models
```
Use that returned ID with the OpenAI-compatible endpoint. MTP files being present does **not** automatically enable speculative decoding: [Lightning MTP is opt-in through oMLX model settings](https://github.com/jundot/omlx/blob/main/omlx/model_settings.py#L291-L300), and behavior can vary by runtime version and Apple chip.
## Quantization and verification
The bundled `oq_imatrix_report.json` records calibration with `oqe_code_multilingual` over 128 sequences of 512 tokens. The included report records 523 importance entries, 481 applied modules, two missing names, no shape mismatches, and no uncovered experts.
The report and tensor metadata establish how the artifact was built; they are not an end-to-end quality benchmark. Repository structure, configs, shard counts, payload sizes, and quantization metadata were audited for this card. Inference was not rerun on a Mac, so no local speed, memory, MTP-acceptance, Vision-quality, or long-context claim is made.
## Provenance
The direct upstream is EryriLabs' TIES merge of Ornith, BigBang, and the shared Qwen base. This artifact preserves the upstream target, Vision tower, and MTP component; it does not contain NInfer DFlash weights.
See the [upstream model card](https://huggingface.co/EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP) for the merge recipe and upstream evaluation. Those quality results were not independently reproduced for this oQ conversion.
## Limitations
- MLX/oMLX targets Apple Silicon and macOS; this repository is not runnable through CUDA on Windows.
- Hugging Face hosted inference does not serve this custom oMLX layout.
- The config advertises a 262,144-token maximum context. That value is model metadata, not a claim that this full context was tested or will fit your machine.
- Vision preprocessing, tool use, MTP acceptance, memory use, and throughput depend on the oMLX version, client, prompt, and Apple hardware.
- Quantization can change output quality. Evaluate this exact variant on your workload.
## License and credits
The direct upstream declares MIT. See its model card for the complete merge provenance and component attribution.
Quantized and packaged by [pyros-vault](https://huggingface.co/pyros-vault) with [oMLX/oQ](https://github.com/jundot/omlx).
|