---
license: apache-2.0
base_model: Cactus-Compute/needle2
tags:
- cactus
- needle
- thox
- tool-calling
- function-calling
- on-device
- edge
- thox-trained
- fine-tuned
---

# ThoxNeedle-Micro

On-device function calling for THOX hardware, based on Cactus's public
`needle2`, packaged in the `.cact` format the device runtime loads.

**This model is THOX-trained.** The weights shipped as `needle2.cact` differ from the upstream artifact, and the difference is measured below.

## What this model is for

Local, on-device function calling for THOX hardware: given a set of THOX device
tool schemas and a user request, emit the tool call that satisfies it — or emit
an empty call list when no available tool can serve the request.

That second behaviour is a first-class objective, not an afterthought. On a
device, a model that invents a plausible-looking call for an unservable request
fires real actuators.

## Training

| | |
|---|---|
| base | `Cactus-Compute/needle2` → `weights/needle2.pkl` (45,211,383 params, stored float16) |
| method | LoRA rank 16, alpha 32, on `q_proj`, `k_proj`, `v_proj`, `gate_proj`, `out_proj` across all 27 layers |
| corpus | 1400 rows of THOX device tool-calling (`needle-tooluse.jsonl`, sha256 `d6f79a9994f243df`) — but only **1400 distinct (query, answer) pairs**, 0.0% duplicates |
| split | 1145 train / 255 held-out rows, group-aware and stratified by class, seed 3407 — every copy of an example stays on one side, so held-out leakage is zero |
| optimiser | AdamW, warmup-cosine, global-norm clip 1.0, **float32 params and optimiser state** |
| learning rate | 0.0003 (selected by sweep) |
| epochs | 3, batch 8 |
| non-finite steps | **0** |

### Learning-rate sweep

| learning rate | best held-out loss |
|---|---:|
| 0.0003 | 0.7899 |
| 0.001 | 0.0463 ← selected |

### Loss curve

Held-out loss by epoch: **0.1827 → 0.0450 → 0.0369**

| step | train loss |
|---:|---:|
| 1 | 2.1868 |
| 5 | 2.3789 |
| 9 | 3.0852 |
| 13 | 2.1875 |
| 17 | 1.7403 |
| 21 | 2.6871 |
| 25 | 2.9051 |
| 29 | 1.9021 |
| 33 | 2.7752 |
| 37 | 2.9909 |
| 41 | 1.9386 |
| 45 | 1.3779 |
| 49 | 1.3995 |
| 53 | 2.1277 |
| 57 | 1.7802 |
| 61 | 1.3122 |
| 65 | 1.2711 |
| 69 | 0.9221 |
| 73 | 1.3950 |
| 77 | 1.2320 |
| 81 | 1.4319 |
| 85 | 1.3647 |
| 89 | 1.0077 |
| 93 | 0.7483 |
| 97 | 0.8222 |
| 101 | 0.7960 |
| 105 | 0.4997 |
| 109 | 0.6364 |
| 113 | 0.5768 |
| 117 | 0.5078 |
| 121 | 0.5616 |
| 125 | 0.2768 |
| 129 | 0.2832 |
| 133 | 0.2264 |
| 137 | 0.2863 |
| 141 | 0.2267 |
| 145 | 0.2248 |
| 149 | 0.1932 |
| 153 | 0.0344 |
| 157 | 0.0910 |
| 161 | 0.1109 |
| 165 | 0.1884 |
| 169 | 0.0756 |
| 173 | 0.0873 |
| 177 | 0.1938 |
| 181 | 0.0917 |
| 185 | 0.1745 |
| 189 | 0.0567 |
| 193 | 0.0845 |
| 197 | 0.1255 |
| 201 | 0.0909 |
| 205 | 0.0248 |
| 209 | 0.0615 |
| 213 | 0.0872 |
| 217 | 0.0391 |
| 221 | 0.0518 |
| 225 | 0.0601 |
| 229 | 0.0819 |
| 233 | 0.0349 |
| 237 | 0.0345 |
| 241 | 0.0703 |
| 245 | 0.0695 |
| 249 | 0.0205 |
| 253 | 0.0903 |
| 257 | 0.0575 |
| 261 | 0.0336 |
| 265 | 0.0487 |
| 269 | 0.0968 |
| 273 | 0.0437 |
| 277 | 0.0239 |
| 281 | 0.0435 |
| 285 | 0.0305 |
| 289 | 0.0508 |
| 293 | 0.0088 |
| 297 | 0.0597 |
| 301 | 0.0639 |
| 305 | 0.0552 |
| 309 | 0.0335 |
| 313 | 0.0673 |
| 317 | 0.0263 |
| 321 | 0.0598 |
| 325 | 0.0413 |
| 329 | 0.0571 |
| 333 | 0.0683 |
| 337 | 0.0469 |
| 341 | 0.0257 |
| 345 | 0.0443 |
| 349 | 0.0493 |
| 353 | 0.0347 |
| 357 | 0.0276 |
| 361 | 0.0339 |
| 365 | 0.0543 |
| 369 | 0.1185 |
| 373 | 0.0359 |
| 377 | 0.0408 |
| 381 | 0.0205 |
| 385 | 0.0255 |
| 389 | 0.0489 |
| 393 | 0.0275 |
| 397 | 0.0353 |
| 401 | 0.0389 |
| 405 | 0.0591 |
| 409 | 0.0331 |
| 413 | 0.0225 |
| 417 | 0.0217 |
| 421 | 0.0285 |
| 425 | 0.0255 |
| 429 | 0.0242 |
| 432 | 0.0020 |

## Evaluation

Scored on the **255 distinct examples**
held out from training (drawn from 255 rows — duplicates are
scored once, not weighted by how often they repeat). Greedy decode, JAX float32.
`call_exact_match` and `refusal_accuracy` are reported separately because
averaging them would hide the failure mode that matters on a device.

| metric | base (passthrough) | THOX fine-tune | delta |
|---|---:|---:|---:|
| well-formed output | 98.04% (250/255) | 100.0% (255/255) | +1.96 (+5 ex.) |
| correct tool selected | 60.78% (155/255) | 85.88% (219/255) | +25.10 (+64 ex.) |
| exact match (name + args) | 54.12% (138/255) | 85.1% (217/255) | +30.98 (+79 ex.) |
| exact match, call examples only | 47.01% (55/117) | 79.49% (93/117) | +32.48 (+38 ex.) |
| refusal accuracy (`answers: []`) | 60.14% (83/138) | 89.86% (124/138) | +29.72 (+41 ex.) |

**Read these deltas with the sample size in mind.** One example is
0.39 percentage points, so the only
honest summary is that the fine-tune left exact-match **unchanged** — it neither
improved nor meaningfully harmed the held-out score. The single-example movement
in tool selection is not a trend this set can resolve.

## Known limitation of this corpus

`needle-tooluse.jsonl` contains 1400 rows but only
**1400 distinct (query, answer) pairs** — 0.0%
duplicates, with up to 1 copies of a single example.

Two consequences, both material to reading the numbers above:

1. A row-level train/test split of this file leaks. An earlier run of this
   pipeline split at row level and put a duplicate of 31 of its 32 held-out
   rows into training; those before/after numbers measured memorisation and
   have been discarded. The split is now group-aware and asserts zero leakage.
2. After holding out distinct examples for evaluation, only a few dozen
   distinct examples remain to train on. That is a hard ceiling on what any
   fine-tune of this corpus can demonstrate against a base model that already
   performs well on the task.

Deduplicating and expanding the corpus is the prerequisite for a meaningful
improvement here — not more epochs, and not a different learning rate.

## Did the fine-tune survive quantization?

The deployment scheme is mixed-precision at ~2.2 effective bits
(`embedding=4, mhc=4, default=2`, group size 128). At that width a small LoRA
delta can be quantized away entirely, which would be indistinguishable from
training having done nothing. Both artifacts were dequantized and diffed:

- tensors changed: **135/404**
- weight elements changed: **25,473,408** of 43,634,423 (**58.38%**)
- max relative L2 change in a tensor: **0.2212**
- survived quantization: **True**

The exporter is bit-deterministic — two exports of identical parameters produce
identical bytes — so the noise floor for this comparison is exactly zero and any
non-zero delta is real.

## Reproducing

Trainer, config, corpus, split, and eval are in
`Thox-ai/thoxneedle-trainer-src`:

```bash
python -m thoxneedle.cli all --config configs/thoxneedle-micro.yaml
```

## Lineage and licensing

Derived from `Cactus-Compute/needle2`, which declares **Apache-2.0** for its
weights; this repo carries that license forward.

Separately, and affecting the *trainer* rather than this artifact: the
`cactus-needle` 2.0.0 PyPI wheel is internally inconsistent about its own
license — `METADATA` declares Apache-2.0 while the bundled `LICENSE` file is
MIT. THOX vendored parts of that wheel's code into its trainer and attributes
against the MIT text there, since that is the document actually shipped with
the code. That discrepancy does not govern the weights published here.

THOX did not use upstream's training code. Its `finetune.py` NaNs on the first
optimiser step: the checkpoint is float16, `init_lora` inherits that dtype,
zero-initialised `B` makes the step-1 gradient w.r.t. `A` exactly zero, and
`optax.adamw`'s default `eps=1e-8` is below float16's smallest subnormal
(5.96e-8) and rounds to zero — so Adam computes `0/(sqrt(0)+0)` = NaN. THOX
wrote its own trainer that keeps parameters and optimiser state in float32.
Details: `docs/UPSTREAM_NAN.md` in the trainer repo.