File size: 3,538 Bytes
531b14a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 | ---
license: apache-2.0
language:
- en
library_name: gguf
pipeline_tag: text-generation
model_name: Ornith-1.0-35B-TQ3_4S
tags:
- gguf
- llama.cpp
- qwen
- qwen3.6
- ornith
- turboquant
- tq3_4s
base_model:
- deepreinforce-ai/Ornith-1.0-35B
model-index:
- name: Ornith-1.0-35B-TQ3_4S
results: []
---
# Ornith-1.0-35B-TQ3_4S
`Ornith-1.0-35B-TQ3_4S` is a compact TurboQuant GGUF build of [deepreinforce-ai/Ornith-1.0-35B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B).
## Required Runtime
This model uses the custom `TQ3_4S` tensor type and requires [turbo-tan/llama.cpp-tq3](https://github.com/turbo-tan/llama.cpp-tq3).
> Stock `llama.cpp` builds without TurboQuant support cannot load this model.
This is the standard 35B model, not an MTP release. Do not add draft-MTP speculative-decoding flags.
## Files
- [`Ornith-1.0-35B-TQ3_4S.gguf`](Ornith-1.0-35B-TQ3_4S.gguf) - main model, 13.30 GB (12.39 GiB)
- [`thumbnail.png`](thumbnail.png) - model card banner
- [`benchmark.png`](benchmark.png) - benchmark comparison card
- [`benchmark_notes.md`](benchmark_notes.md) - compact benchmark provenance and comparison
## Build the Required Runtime
```bash
git clone https://github.com/turbo-tan/llama.cpp-tq3
cd llama.cpp-tq3
cmake -S . -B build \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_CUDA=ON \
-DGGML_CUDA_FA_ALL_QUANTS=OFF \
-DGGML_CUDA_GRAPHS=ON
cmake --build build --target llama-server -j
```
For an RTX 3090, `-DCMAKE_CUDA_ARCHITECTURES=86` may be added explicitly. Use the architecture matching your GPU on other systems.
## Recommended Runtime
Validated on an NVIDIA GeForce RTX 3090 Founders Edition with 24 GiB VRAM:
```bash
./build/bin/llama-server \
-m Ornith-1.0-35B-TQ3_4S.gguf \
--alias Ornith-1.0-35B-TQ3_4S \
--host 127.0.0.1 --port 8080 \
-c 32768 -np 1 -ngl 99 -fa on \
-ctk q8_0 -ctv tq3_0 \
--reasoning off --jinja
```
Runtime notes:
- `-fa on` enables flash attention at runtime.
- The validated CUDA build uses `GGML_CUDA_FA_ALL_QUANTS=OFF`.
- `-ngl 99` fully offloads the model on supported GPUs. Avoid partial offload when comparing the published speed.
- Reduce context from `32768` if the available VRAM is lower than 24 GiB.
## Quick Smoke Test
```bash
curl -s http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"Ornith-1.0-35B-TQ3_4S","messages":[{"role":"user","content":"Write ONLY the word ok."}],"max_tokens":16,"temperature":0.0}'
```
Expected assistant content:
```text
ok
```
## Benchmark Summary
Local BenchLoop and Hard86 comparison on an RTX 3090 FE using the TurboTan runtime and the launch settings above:
| Metric | Result |
|---|---:|
| 35B field overall | 95.75 |
| Hard86 | 81.4% |
| EasyCode | 100.0% |
| Toolcall | 88.3% |
| Data extract | 86.5% |
| Instruct follow | 65.5% |
| Reason math | 73.3% |
| Generation speed | 146.3 tok/s |
| Size | 13.0 GB reported; 12.39 GiB file |
The field score uses `0.85 * task_score + 0.15 * size_factor`, with size normalized to the smallest displayed 35B model. Hard86 is weighted at `2x`, EasyCode at `0.5x`, and the remaining benchmark categories at `1x`.
## Validation Notes
- Benchmark results are local measurements, not claims from the parent model repository.
## License
Use is subject to the [base model](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B) license and the licenses of [turbo-tan/llama.cpp-tq3](https://github.com/turbo-tan/llama.cpp-tq3).
|