Runbook: Wald-Q4B v2 (v2.0, checkpoint 04701-c22)
This revision holds Wald-Q4B v2: the language model of checkpoint 04701-c22 (its text-only weights file has sha256 6ae382f5a0ed9f4c53cd0953cb9606b1627a6816350340a59114860e6cf6f29a; every tensor is byte-identical here) plus Qwen3.5-4B's unchanged vision tower, MTP head and image processor configs, as Qwen3_5ForConditionalGeneration in three safetensors shards. MANIFEST.json lists the sha256 of every file. v1.x runbooks are at their tags.
1. Serve
hf download org2ai/Wald-4B --revision v2.0 --local-dir ./Wald-Q4B-v2
cd Wald-Q4B-v2
python - <<'PY' # optional: check every file against MANIFEST.json
import hashlib, json; m = json.load(open("MANIFEST.json"))
bad = [f for f, v in m.items() if hashlib.sha256(open(f, "rb").read()).hexdigest() != v["sha256"]]; print("bad:", bad)
PY
./run.sh "$PWD"
GET /health must report "engine": "native-v2", "vision": true, "checkpoint": "04701-c22", "policy": "Auto0.7", "thought_budget": 512 and "temperature_sha256": "a0f72cd2d0a653e81051e5a0c77fc1a69131552a8102b580a93a6dbe7908b2da". Docker: see Dockerfile (same runtime). The server pins the reader (server/src/wald_serve/native_reference/, every file sha256-checked at start) and the calibration table, and refuses to start if either differs.
The evaluated runtime: vLLM 0.30.0, transformers 5.17.0, torch 2.13.0, BF16, one NVIDIA RTX PRO 6000 (96 GB) per model replica, VLLM_USE_FLASHINFER_SAMPLER=0, and the vLLM arguments --max-model-len 131072 --gpu-memory-utilization 0.85 --max-num-seqs 128 --seed 0 plus --limit-mm-per-prompt {"image": 16, "video": 0} and the image budget --mm-processor-kwargs {"size": {"shortest_edge": 65536, "longest_edge": 1048576}} (what wald-serve-native-vision --model launches; wald-serve-native serves text only). Smaller GPUs need a lower --gpu-memory-utilization or --max-model-len, which is a deviation from the evaluated setup. Thought generation samples (temperature 0.6, top-p 0.95, top-k 20) with a seed derived from the request and question id, so repeated requests normally give the same answer; batch composition on the GPU can still change near-tied answers.
config.json carries "use_cache": false from training. vLLM ignores it; if you load the weights with Transformers generate, pass use_cache=True.
2. Policies and prompt formats
| Request field | Values | Evaluated settings |
|---|---|---|
effort |
none (one pass), auto / medium (Auto 0.7, default), always / high (Always 512) |
all three |
prompt_format |
repeat_state_plain (default), plain |
Decision Index and JevBench-XL: repeat_state_plain; JevBench public 231 and multistep: plain |
Auto 0.7 gates on the untempered one-pass distribution: a question with 2–26 options thinks natively (Qwen chat template with enable_thinking, at most 512 tokens, stop at </think>) when its top probability is below 0.7, then the options are read again. More than 26 options: grouped knockout readout, no thought, every option gets a probability. The returned probabilities are tempered with the frozen table (bucket by question type and option count).
3. Images
Requests with images (message content parts image_url / image in state, raw base64 under state keys image, image_data, image_base64, image_b64, or a top-level images / image_data list; up to 16 per request, bodies up to 64 MB) go through the same frozen reader, with the images carried to vLLM's chat route at their position in the state. They default to the plain prompt format. Text-only requests take the text path unchanged. The Decision Index 0.3 run used the text-only weights file; the language-model tensors here are byte-identical, and the text parity of this repository's weights is in evaluation/v2/serving-parity.json.
4. GGUF (llama.cpp)
org2ai/Wald-4B-GGUF holds the v2 quantisations and the vision projector (Wald-4B-v2-mmproj-F16.gguf). wald-serve-native --gguf FILE --tokenizer-dir DIR --effort none reads text requests through llama-server with the same chat layout; a letter outside llama-server's top-100 log-probabilities gets the reader's floor instead of its exact value. One pass is the checked mode.
5. Decision Index 0.3 public suite
Install the official kit apolinario/decision-index at 62d2f51de34a2de64906345b6bc3e98e27ff55c7 (branch v0.3), rebuild the public suite and verify its hashes with the kit. The evaluated run used the kit's Engine API with the adapter in evaluation/v2/code/di03_native_auto_engine.py (it loads reference/ and temperature.json, Auto 0.7, budget 512, repeat_state_plain) against vLLM servers started with the arguments above; evaluation/v2/code/di03_parallel.py ran four disjoint replicas (SHA256(run_id) mod 4) at 128 concurrent requests each. The packaged server answers the same requests with the same reader; the kit's http engine can be pointed at POST /v1/systemone instead. Score with the kit's unmodified scorer after inference. Suite payloads are not redistributed. Concurrent throughput is not the maintainers' serial latency.
6. Other reads
JevBench public 231, JevBench-XL partitions and multistep_decisions were read with the same reader and engine (one generation per item; the one-pass, Auto 0.7 and Always 512 answers are fixed views of it) and scored with each benchmark's scorer. JevBench-XL is internal and not distributed.
Training data and evaluation caveats: PROVENANCE.md, CONTAMINATION.md.