Local llama.cpp eval evidence
Date: 2026-06-07
This note separates local llama.cpp evidence from hosted Omni evidence and from canned deterministic fallback.
Current endpoint check
No local OpenAI-compatible llama.cpp endpoint was active during this Worker C pass.
Checked:
lsof -nP -iTCP:8001 -sTCP:LISTEN
curl -fsS --max-time 2 http://127.0.0.1:8001/v1/models
Observed: no listener on port 8001; curl failed to connect to 127.0.0.1:8001.
One-case route smoke
Use this only to prove the app can call the configured local route. A passed smoke is not a 50-case local competence run.
FIGMENT_MODE=local \
MODEL_STACK=local_4b_parakeet \
MODEL_BACKEND=llama_cpp \
LOCAL_MODEL_ID=nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 \
LLAMA_BASE_URL=http://127.0.0.1:8001/v1 \
FIGMENT_SMOKE_ALLOW_NETWORK=true \
PYTHON_DOTENV_DISABLED=true \
python3 scripts/smoke_model_route.py
The smoke output now includes local_llm_evidence.proof_status, counts_as_no_cloud_route_proof, and counts_as_50_case_local_llm_competence. A skipped smoke must not be counted as local model competence.
Real 50-case eval command
Run this only after starting a real local OpenAI-compatible llama.cpp server and recording /v1/models metadata.
FIGMENT_MODE=local \
MODEL_STACK=local_4b_parakeet \
MODEL_BACKEND=llama_cpp \
LOCAL_MODEL_ID=nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 \
LLAMA_BASE_URL=http://127.0.0.1:8001/v1 \
PYTHON_DOTENV_DISABLED=true \
python3 scripts/run_eval.py \
--backend llama_cpp \
--model-stack local_4b_parakeet \
--cases data/eval/initial_handwritten_cases.jsonl \
--cases data/eval/adversarial_strict_cases.jsonl \
--cases data/eval/comprehensive_hosted_cases.jsonl \
--output traces/local_llama_cpp_eval_$(date -u +%Y%m%dT%H%M%SZ).jsonl
A real local evidence bundle should include the eval JSONL trace, the printed summary, /v1/models response or equivalent server metadata, model file/hash or server launch command, and confirmation that no hosted model credentials were required for the run.