README.md

---
license: apache-2.0
base_model: Achilles1089/fable-coder-35B-A3B
pipeline_tag: text-generation
tags:
  - code
  - agentic
  - moe
  - qwen3.6
  - gguf
  - dappit
language:
  - en
---

# fable-coder-35B-A3B · GGUF

Quantized GGUFs of **[Achilles1089/fable-coder-35B-A3B](https://huggingface.co/Achilles1089/fable-coder-35B-A3B)** —
a sovereign, open-weights agentic coding model by **[Dappit Labs](https://dappit.io)**. 35B MoE (≈3B active),
Claude Fable-5 / Opus-4.8 agentic distill on an abliterated, Opus-4.7-reasoning-distilled Qwen3.6-35B-A3B.

> Built by **[Dappit Labs](https://dappit.io)** ([@dappitdotio](https://x.com/dappitdotio)) · Trained on hardware from **[Manifest Network](https://manifest.network/)**.

See the [main model card](https://huggingface.co/Achilles1089/fable-coder-35B-A3B) for the full write-up,
training details, evaluation, license, and responsible-use notes.

## Quants

Each quant is a single self-contained file — download **only the one you need**.

| File | Quant | Size | Fits |
|---|---|---|---|
| [`fable-coder-35B-A3B-Q8_0.gguf`](https://huggingface.co/Achilles1089/fable-coder-35B-A3B-GGUF/blob/main/fable-coder-35B-A3B-Q8_0.gguf) | Q8_0 | ~38GB | 48GB+ GPU / 64GB Mac — near-lossless |
| [`fable-coder-35B-A3B-Q6_K.gguf`](https://huggingface.co/Achilles1089/fable-coder-35B-A3B-GGUF/blob/main/fable-coder-35B-A3B-Q6_K.gguf) | Q6_K | ~29GB | 32–48GB |
| [`fable-coder-35B-A3B-Q5_K_M.gguf`](https://huggingface.co/Achilles1089/fable-coder-35B-A3B-GGUF/blob/main/fable-coder-35B-A3B-Q5_K_M.gguf) | Q5_K_M | ~25GB | 32GB |
| [`fable-coder-35B-A3B-Q4_K_M.gguf`](https://huggingface.co/Achilles1089/fable-coder-35B-A3B-GGUF/blob/main/fable-coder-35B-A3B-Q4_K_M.gguf) | Q4_K_M | ~22GB | 24GB (3090/4090) |

## Download

**One quant via the HF CLI** (recommended — resumable, no full-repo clone):
```bash
pip install -U "huggingface_hub[cli]"
hf download Achilles1089/fable-coder-35B-A3B-GGUF \
  fable-coder-35B-A3B-Q4_K_M.gguf --local-dir .
```

**LM Studio / Jan:** search `fable-coder-35B-A3B` and pick a quant from the list.

**Ollama:** ([ollama.com/achillessafehavencalls/fable-coder](https://ollama.com/achillessafehavencalls/fable-coder) — sane defaults + `max_tokens` cap baked in)
```bash
ollama run achillessafehavencalls/fable-coder          # Q4_K_M (default)
ollama run achillessafehavencalls/fable-coder:q8_0     # near-lossless
```

**Web:** open the [Files tab](https://huggingface.co/Achilles1089/fable-coder-35B-A3B-GGUF/tree/main) and click any single file to download it.

## Run

```bash
# llama.cpp
llama-server -m fable-coder-35B-A3B-Q6_K.gguf -c 32768 -ngl 99
```

**Thinking is native** — the Qwen template opens `<think>` by default; the server returns reasoning in
`reasoning_content` and the answer in `content`. For agentic coding, drive it inside a harness with a
tool-use system prompt + tool registry (treat it like Claude Code).

Quantized from the bf16 master with llama.cpp `llama-quantize`.

## Compatibility — MTP block / `llama.cpp` version

These GGUFs keep the upstream **MTP (next-token-prediction) block** — `block_count = 41`,
`nextn_predict_layers = 1`, with `blk.40` being that block. This matches the stock
Qwen3.6-35B-A3B layout, and it needs a reasonably current `llama.cpp`.

**Older builds fail to load with:**

```
llama_model_load: error loading model: missing tensor 'blk.40.ssm_conv1d.weight'
```

That is a **loader-version issue, not a bad file**. `blk.40` is the MTP block and is
attention-style *by design* — the base Qwen3.6-35B-A3B has no `ssm_conv1d` there either (the
hybrid pattern puts full-attention layers at blocks 3, 7, 11 … 39, with 40 as MTP on top).
Older builds type block 40 as a regular hybrid layer and go looking for SSM tensors.

**Fix: update `llama.cpp`.** Verified loading and generating on build `9950 (961e4b26a)`;
reported failing on `b9075`.

If you are pinned to an older build — or on a runtime that cannot load the MTP block — you can
strip block 40 locally (`pip install gguf`). You lose only the speculative-decoding head;
normal generation quality is unchanged:

```python
# strip_mtp.py IN.gguf OUT.gguf
import sys
from gguf import GGUFReader, GGUFWriter, GGUFValueType
src, dst = sys.argv[1], sys.argv[2]
r = GGUFReader(src)
w = GGUFWriter(dst, r.fields['general.architecture'].contents())
OVERRIDE = {'qwen35moe.block_count': 40, 'qwen35moe.nextn_predict_layers': 0}
for key, field in r.fields.items():
    if key == 'general.architecture' or key.startswith('GGUF.'):
        continue
    val, types = OVERRIDE.get(key, field.contents()), field.types
    if types and types[0] == GGUFValueType.ARRAY:
        w.add_key_value(key, val, GGUFValueType.ARRAY, sub_type=types[1])
    else:
        w.add_key_value(key, val, types[-1])
for t in r.tensors:
    if not t.name.startswith('blk.40.'):
        w.add_tensor(t.name, t.data, raw_dtype=t.tensor_type)
w.write_header_to_file(); w.write_kv_data_to_file(); w.write_tensors_to_file(); w.close()
```