File size: 2,066 Bytes
5dcfc5c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | # Local llama.cpp eval evidence
Date: 2026-06-07
This note separates local llama.cpp evidence from hosted Omni evidence and from canned deterministic fallback.
## Current endpoint check
No local OpenAI-compatible llama.cpp endpoint was active during this Worker C pass.
Checked:
```bash
lsof -nP -iTCP:8001 -sTCP:LISTEN
curl -fsS --max-time 2 http://127.0.0.1:8001/v1/models
```
Observed: no listener on port `8001`; `curl` failed to connect to `127.0.0.1:8001`.
## One-case route smoke
Use this only to prove the app can call the configured local route. A passed smoke is not a 50-case local competence run.
```bash
FIGMENT_MODE=local \
MODEL_STACK=local_4b_parakeet \
MODEL_BACKEND=llama_cpp \
LOCAL_MODEL_ID=nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 \
LLAMA_BASE_URL=http://127.0.0.1:8001/v1 \
FIGMENT_SMOKE_ALLOW_NETWORK=true \
PYTHON_DOTENV_DISABLED=true \
python3 scripts/smoke_model_route.py
```
The smoke output now includes `local_llm_evidence.proof_status`, `counts_as_no_cloud_route_proof`, and `counts_as_50_case_local_llm_competence`. A skipped smoke must not be counted as local model competence.
## Real 50-case eval command
Run this only after starting a real local OpenAI-compatible llama.cpp server and recording `/v1/models` metadata.
```bash
FIGMENT_MODE=local \
MODEL_STACK=local_4b_parakeet \
MODEL_BACKEND=llama_cpp \
LOCAL_MODEL_ID=nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 \
LLAMA_BASE_URL=http://127.0.0.1:8001/v1 \
PYTHON_DOTENV_DISABLED=true \
python3 scripts/run_eval.py \
--backend llama_cpp \
--model-stack local_4b_parakeet \
--cases data/eval/initial_handwritten_cases.jsonl \
--cases data/eval/adversarial_strict_cases.jsonl \
--cases data/eval/comprehensive_hosted_cases.jsonl \
--output traces/local_llama_cpp_eval_$(date -u +%Y%m%dT%H%M%SZ).jsonl
```
A real local evidence bundle should include the eval JSONL trace, the printed summary, `/v1/models` response or equivalent server metadata, model file/hash or server launch command, and confirmation that no hosted model credentials were required for the run.
|