--- base_model: microsoft/FastContext-1.0-4B-SFT license: mit library_name: gguf tags: - gguf - rocmfp4 - qwen3 - fastcontext - subagent - repository-exploration - coder - agentic - imatrix - strix-halo - amd - rocm - vulkan language: - en base_model_relation: quantized ---

PLUNDERSTRUCK // ROCmFP4 QUANTIZED MODEL // STRIX HALO · gfx1151
            ▗▇▇▇▇▇▇▇▖                 
           ▗█▘▝██████▖                
          ▗▛   ▝██████▆▆▆▆▆▆▆▆▆▆▅     
         ▟▛    ▗█████████████████▙▖   
   ▄▄▄▄▄▟▛    ▟████████████████████▖  
 ▗██▌    ▚▖   ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔█▘  
▗████▖    ▜▖                    ▗█▘   
▜█████▙    ▜▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▀▀▀▀▀▜▙    
 ▜█████▙    ▝████████████▛       ▜▙   
  ▜█████▙    ▝██████████▛    ▃    ▜▙  
   ▀█████▙▖   ▝████████▘    ▟█▙    ▀▙ 
    ▝██████▖   ▝▜█████▘    ▟███▙▂▂▂▂▐█
    ▟███████▖    ▜███▘   ▗███████████▛
   ▟█████████▄    ▜▛    ▗███████████▀ 
  ▝█████▀        ▗▛    ▗██████▀▀▀▀▀▘  
    ▜██▘        ▗▛    ▟█████▛▘        
     ▜█▇▇▇▇▇▇▇▇▇█▖   ▟█████▛          
                ▝█▖ ▟█████▛           
                 ▝███████▀            
FASTCONTEXT-1.0-4B
4-BIT ROCmFP4 · QWEN3 DENSE 4B · REPO-EXPLORATION SUBAGENT · CODE-WEIGHTED IMATRIX · SINGLE AMD APU

    
      FORMAT
ROCmFP4 4-BIT

      PRECISION
~4.5 BPW

      ARCH
QWEN3 DENSE

      CONTEXT
256 K

    

      PARAMS
4B DENSE

      DRAFT
NO MTP

      BACKEND
VULKAN0

      LICENSE
MIT

    

⚠ REQUIRES THE ROCmFP4 FORK

The custom q4_0_rocmfp4 / q4_0_rocmfp4_fast tensor types will not load in stock llama.cpp, LM Studio, or Ollama. Build/run with charlie12345/rocmfp4-llama · branch mtp-rocmfp4-strix.

NOTE // Ignore HuggingFace's auto-detected "F16"/16-bit badge — its parser can't read ROCmFP4 and mislabels the file. These are ~4.5 bpw 4-bit ROCmFP4 files; pick by filename in Files and versions.

Experimental **AMD Strix Halo (gfx1151)** quant of [**microsoft/FastContext-1.0-4B-SFT**](https://huggingface.co/microsoft/FastContext-1.0-4B-SFT) — Microsoft's **repository-exploration subagent** for coding agents. Instead of one model both exploring the repo and solving the task, FastContext is invoked on demand by a main agent, fires **parallel read-only tool calls** (READ / GLOB / GREP), and returns **compact file paths + line ranges** as focused context. Architecturally it's a plain **Qwen3 dense 4B** (`Qwen3ForCausalLM`, 36 layers, hidden 2560, 256K context, MIT-licensed), here in the custom **ROCmFP4** 4-bit format, **imatrix-quantized**.

01 · FILES

File	Body	Size	Pick if
`…-COHERENT-embF16.gguf` ★	all-dual	2.8 GB	recommended — lowest measured KL vs BF16 (§04)
`…-STRIX-embF16-imatrix.gguf`	fast	2.7 GB	~same fidelity, slightly smaller/faster

Both share genuine **f16 embeddings** (from BF16) + the code-weighted imatrix (see §04). The **COHERENT** build (★) puts every body tensor on the **dual-scale** `q4_0_rocmfp4` kernel — lowest measured KL vs the BF16 reference at ~the same decode speed — vs the STRIX build's faster single-scale `q4_0_rocmfp4_fast` bulk. The Qwen (ChatML) chat template is **baked into the GGUF** — just pass `--jinja`.

NOTE // TIED EMBEDDINGS. FastContext has tie_word_embeddings=True, so there's no separate output head — the token-embedding tensor doubles as the lm-head. Setting --token-embedding-type f16 therefore gives an f16 embedding and f16 output head in one (no headQ6 variant needed — f16 already beats Q6 there).

02 · QUICK START

Run from the folder holding the `.gguf` (the Qwen ChatML template is baked in — just pass `--jinja`): ```bash env HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \ llama-server \ -m FastContext-1.0-4B-SFT-ROCmFP4-COHERENT-embF16.gguf \ --alias fastcontext-4b \ --host 0.0.0.0 \ --port 8080 \ -c 262144 \ -ctk f16 \ -ctv f16 \ --temp 0.7 \ --top-p 0.8 \ --top-k 20 \ -dev Vulkan0 \ -ngl 999 \ -fa on \ -b 2048 \ -ub 256 \ -t 16 \ -tb 16 \ -cpent 256 \ -ctxcp 32 \ --cache-reuse 256 \ --cache-ram 65536 \ --jinja \ --parallel 1 \ --metrics \ --no-mmap ```

Flag	Function
`HSA_OVERRIDE_GFX_VERSION=11.5.1`	treat the APU as gfx1151 (Strix Halo)
`GGML_HIP_ENABLE_UNIFIED_MEMORY=1`	allow use of the full 128 GB unified memory
`-dev Vulkan0`	run on Vulkan — fastest backend for ROCmFP4 on Strix Halo
`-ngl 999 · -fa on`	offload all layers · flash attention
`-c 262144`	context length (256K)
`-b 2048 · -ub 256 · -t/-tb 16`	prefill batch / micro-batch · CPU threads
`-ctk f16 · -ctv f16`	f16 KV cache — how we run it (cheap on a 4B); drop to `q8_0`/`q4_0` to use less memory at deep context
`-cpent · -ctxcp · --cache-reuse · --cache-ram 65536`	cross-turn KV checkpointing + 64 GB resident reuse cache
`--temp 0.7 --top-p 0.8 --top-k 20`	Qwen3 recommended sampling (instruct/non-thinking)
`--jinja --parallel 1 --metrics --no-mmap`	apply baked ChatML template · single slot · metrics · weights in RAM

NOTE // No --spec-* / --spec-type draft-mtp flags — this arch has no MTP head (see §04). It's already fast on its own.

03 · USING IT AS A SUBAGENT

FastContext isn't a general chat model — it's a **repository-exploration subagent** meant to be **called by your main coding agent**, not driven directly. The intended loop: the main agent delegates "find the relevant context for X" → FastContext issues **parallel read-only tool calls** (`READ`, `GLOB`, `GREP`) → returns **compact file paths + line ranges**, which the main agent folds into its own context to do the actual work. The point is to keep repo-exploration tokens *out* of the main agent's window. - **Chat template:** Qwen (ChatML) is baked into the GGUF — just pass `--jinja`. - **Tool calling:** it emits structured `READ`/`GLOB`/`GREP` calls — wire those tools into your harness and use a Qwen/Hermes-style tool-call parser so they're parsed rather than printed. **See the [upstream model card](https://huggingface.co/microsoft/FastContext-1.0-4B-SFT) for the exact subagent protocol + tool schema** (it expects a specific invocation format). - **Sampling:** temp `0.7`, top-p `0.8`, top-k `20` (Qwen3 instruct defaults) — already set in §02.

NOTE // It's small (4B) and fast (~68 t/s, §04) by design — a cheap, disposable explorer you can fan out in parallel next to a larger main model on the same box. The cross-turn reuse cache (--cache-reuse / --cache-ram) keeps repeated exploration over the same repo cheap.

04 · PERFORMANCE & QUALITY

DECODE · short context	~68 t/s (Vulkan / Ryzen AI Max+ 395)
SPECULATIVE DECODE	none (no MTP head)
CONTEXT	256K native (dense attention)
QUANTIZATION	COHERENT body + imatrix (measured win — below)

**Recommended build = COHERENT (we measured it).** Both builds use f16 tied emb/head + the same imatrix; the lever swept here is the **body kernel**, ranked by **KL divergence vs the true BF16** on held-out code (lower = more faithful). The **all-dual-scale body** (COHERENT) beats the fast-body STRIX build on **every** metric at ~the same decode speed:

Build (imatrix + embF16, tied head)	Body	Mean KLD vs BF16 ↓	Median KLD ↓	Top-token	PPL(Q) ↓
`COHERENT` ★	all-dual	0.03422	0.00955	92.08%	4.192
`STRIX`	fast	0.03934	0.01016	91.38%	4.213

A **clean sweep**: COHERENT is lower on mean KLD (−13%), median KLD (−6%), RMS Δp (6.43% vs 6.95%), **and** perplexity (4.192 vs 4.213), and higher on same-top-token (+0.70 pp) — every metric, same direction (BF16 reference PPL 4.074). So it's the default; STRIX stays as a marginally smaller/faster fallback. **Fast on its own.** ~68 t/s short-context decode on a Ryzen AI Max+ 395 (Vulkan0, measured `llama-bench tg128`). It's a 4B dense Qwen3 with **no MTP head**, so there's no speculative decoding — it doesn't need it, and at 4B it's a cheap explorer you can run several of in parallel.

NOTE // imatrix. Both builds are quantized with an importance matrix (Kalomaze groups_merged + froggeric code/technical, via froggeric/imatrix), computed on this model's BF16. We measured the COHERENT-vs-STRIX comparison above (both imatrix); we did not run a separate imatrix-vs-no-imatrix ablation on this model. Scope: the KL/PPL figures are a fidelity-vs-BF16 measurement on a held-out code slice, not an absolute coding benchmark.

05 · BUILD (REPRODUCIBLE)

```bash # 0) convert the safetensors -> BF16 GGUF (plain qwen3 dense; no MTP, tied embeddings) python convert_hf_to_gguf.py FastContext-1.0-4B-SFT/ --outtype bf16 --outfile FastContext-1.0-4B-SFT-BF16.gguf # 1) imatrix on the BF16 (general+code: Kalomaze groups_merged + froggeric code/technical) llama-imatrix -m FastContext-1.0-4B-SFT-BF16.gguf -f general+code-calib.txt -o fastcontext-4b.imatrix -c 512 -ngl 999 # 2) RECOMMENDED: COHERENT all-dual body + f16 tied emb/head (the ★ file) — lowest KL (§04). # tie_word_embeddings=True -> --token-embedding-type f16 also gives an f16 output head; no --output-tensor-type. llama-quantize --token-embedding-type f16 --imatrix fastcontext-4b.imatrix \ FastContext-1.0-4B-SFT-BF16.gguf FastContext-1.0-4B-SFT-ROCmFP4-COHERENT-embF16.gguf Q4_0_ROCMFP4_COHERENT # fast-body STRIX fallback (same f16 emb + imatrix) llama-quantize --token-embedding-type f16 --imatrix fastcontext-4b.imatrix \ FastContext-1.0-4B-SFT-BF16.gguf FastContext-1.0-4B-SFT-ROCmFP4-STRIX-embF16-imatrix.gguf Q4_0_ROCMFP4_STRIX ``` > Experimental research build for AMD Strix Halo — hardware/driver/prompt-sensitive, may not reproduce elsewhere. Not native FP4 tensor-core execution.

06 · LINEAGE & CREDITS

BASE MODEL	microsoft/FastContext-1.0-4B-SFT (MIT, Microsoft) · repository-exploration subagent · Qwen3 dense 4B (`Qwen3ForCausalLM`)
CALIBRATION	Kalomaze `groups_merged` + froggeric `code`/`technical` via froggeric/imatrix
FORMAT + RUNTIME	charlie12345/rocmfp4-llama (based on llama.cpp, MIT)

*Derivative quantization — verify the base model's license before redistribution / use.*

FORMAT ROCmFP4 4-BIT	PRECISION ~4.5 BPW	ARCH QWEN3 DENSE	CONTEXT 256 K
PARAMS 4B DENSE	DRAFT NO MTP	BACKEND VULKAN0	LICENSE MIT