---
license: apache-2.0
base_model: mlx-community/Qwen3.5-9B-MLX-4bit
tags:
- mlx
- apple-silicon
- tool-use
- hermes-format
- function-calling
- lora
- qwen3
language:
- en
pipeline_tag: text-generation
---

# Qwen3.5-9B Cascade Tool-Use Adapter (v2)

LoRA adapter that fine-tunes `mlx-community/Qwen3.5-9B-MLX-4bit` for
the Hermes flat-JSON tool-calling format used in Cascade. Designed for
Apple Silicon training via `mlx-lm`; usable for inference on any
MLX-compatible runtime (mlx-lm, vLLM-MLX, etc.).

## What this is

A LoRA adapter (rank 16, alpha 32, 16 target layers) trained on 552
records of Hermes-format tool-use data: 30 hand-written seed
conversations plus 522 records generated by `Qwen3.6-35B-A3B-6bit` as
a teacher through an omlx-server endpoint.

The adapter teaches the base model to emit tool calls in the
flat-JSON Hermes schema:

```
{
  "name": "get_weather",
  "arguments": {"location": "Vancouver, WA", "units": "imperial"}
}
```

wrapped in ` tags, rather than the nested-XML format Qwen3.5
emits by default.

## Intended use

Plug into Cascade or any OpenAI-compatible tool-calling pipeline
that wants Qwen3.5-9B-level reasoning with reliable Hermes-format
tool emission. Fits 71-tool libraries; trained on a representative
mix of network/sysadmin/productivity/generic tools.

Out of scope: vision, audio, code completion without tool use, and
multi-turn dialogue without a tool schema in the system prompt.

## How to use

The adapter is a `mlx-lm` LoRA, not a fused model. You need the base
model separately.

```python
from mlx_lm import load, generate

base_model = "mlx-community/Qwen3.5-9B-MLX-4bit"
adapter = "TylerMayfield/qwen3.5-9b-cascade-tooluse-adapter"

model, tokenizer = load(base_model, adapter_path=adapter)

prompt = """<|im_start|>system
You are a function calling AI model. You are provided with function
signatures within <tools></tools> XML tags. ...
<|im_end|>
<|im_start|>user
What's the weather in Portland, OR?
<|im_end|>
<|im_start|>assistant
"""

print(generate(model, tokenizer, prompt=prompt, max_tokens=200))
```

To produce a single fused model (one safetensors file, no separate
adapter load), fuse with `mlx_lm.fuse`:

```bash
mlx_lm.fuse \
  --model mlx-community/Qwen3.5-9B-MLX-4bit \
  --adapter-path ./downloaded-adapter \
  --save-path ./qwen35-9b-tooluse-fused
```

## Training data

| Source             | Records | Notes |
|--------------------|--------:|-------|
| Seed (hand-written)|      30 | Authored for Cascade scenarios |
| Synthetic (v1)     |     522 | Generated via Qwen3.6-35B-A3B-6bit teacher |
| **Combined (v2)**  | **552** | Seed + synthetic, used for this adapter |

Categories: `simple_single`, `parallel_calls`, `multi_step`,
`error_recovery`, `no_tool_needed`, `clarification`,
`irrelevant_tools`, `multi_turn_context`, `complex_args`,
`tool_selection`. Per-category distribution is documented in the
Cascade training repo's `PROGRESS.md`.

The data was generated with `chat_template_kwargs={"enable_thinking":
False}` on the teacher — reasoning models emit a CoT block by default,
which breaks downstream JSON parsing.

## Eval results

Evaluated on a held-out set of 30 hand-written Cascade test
conversations (70 turns across 10 categories). The base model scores
64.3% turn-exact; this adapter lifts it to **74.3%**.

| Metric            | Base   | v1 (552 records) | **v2 (this, 552 records)** |
|-------------------|-------:|-----------------:|---------------------------:|
| turn_exact        | 64.3%  | 68.6%            | **74.3%**                  |
| args_exact        | 35.3%  | 44.1%            | **52.9%**                  |
| format_ok         | 98.6%  | 100.0%           | 98.6%                      |
| schema_valid      | 97.1%  | 97.1%            | 97.1%                      |
| hallucinated (n)  | 3      | 3                | 2                          |
| missed (n)        | 2      | 0                | 1                          |

Per-category turn-exact:

| Category           | Base   | v1   | **v2** |
|--------------------|-------:|-----:|-------:|
| simple_single      | 100.0% | 100.0% | 100.0% |
| no_tool_needed     | 100.0% | 100.0% | 100.0% |
| parallel_calls     | 100.0% | 100.0% |  83.3% |
| irrelevant_tools   | 100.0% | 100.0% | 100.0% |
| multi_turn_context |  83.3% |  83.3% |  91.7% |
| clarification      |  55.6% |  55.6% |  **77.8%** |
| multi_step         |  44.4% |  44.4% |  55.6% |
| complex_args       |  42.9% |  42.9% |  42.9% |
| error_recovery     |  33.3% |  55.6% |  66.7% |
| tool_selection     |  33.3% |  50.0% |  50.0% |

**Limitations of these numbers:**
- The eval set is 30 conversations I wrote. It is not an industry
  standard benchmark (no MMLU, no BFCL ToolBench). Real-world
  performance will vary.
- N=70 turns is small. The ±10pp swings on `complex_args` and
  `parallel_calls` reflect 1-2 individual turns, not robust category
  performance.
- Eval prompt format is chatml with no `chat_template_kwargs`.
  Other chat templates will produce different results.

## What the adapter does NOT do

- It does not add new tools to the model's knowledge. Tools must be
  described in the system prompt at inference time.
- It does not improve general reasoning. v2 scores worse than the
  base model on no_tool_needed turns where the model is asked general
  knowledge questions and gives prose answers.
- It does not change the base model's refusal behavior, identity, or
  safety training.

## Training details

- **Base model**: `mlx-community/Qwen3.5-9B-MLX-4bit` (5.6 GB, 4-bit)
- **Adapter**: LoRA, rank 16, alpha 32, 16 target layers
  (10.82M trainable params, 0.121% of base)
- **Sequence length**: 1024 tokens (the M5 Pro 48GB + mlx-lm 0.31.3
  ceiling — see "Hardware constraints" below)
- **Iterations**: 1500 (best val at iter 750, val loss 0.267)
- **Learning rate**: 1e-4 with linear decay
- **Batch size**: 1, grad accumulation 1
- **Gradient checkpointing**: enabled (required for Qwen3.5 hybrid
  attention + Apple Silicon at this size)
- **Mask-prompt**: DISABLED. mlx-lm 0.31.3 has a divide-by-zero bug
  when `--mask-prompt` is combined with truncation; loss stays on the
  full sequence instead of assistant tokens only.

## Hardware constraints

Trained on Apple M5 Pro, 48 GB unified memory. The seq-length ceiling
on this hardware is 1024 (1280 OOMs). See the Cascade training repo's
`references/mask-prompt-and-mem-ceiling-2026-06.md` for full details.

## Known follow-ups (not addressed in this adapter)

1. `complex_args` stays at 43% across v1, v2, and the v3 attempt
   (v3 was not released). More synthetic data of the same shape did
   not move it; addressing this needs a structural audit of the
   7-test complex_args failures vs the 65+ training examples.
2. `parallel_calls` regressed 100% → 83% in v2 due to one example
   where the model over-called a follow-up tool. Recovered in the v3
   attempt but v3 regressed other categories; v2 was kept as the
   production model.
3. The base model emits empty ` blocks before responses.
   This is the Qwen3.5 chat template behavior with thinking-mode on;
   pass `chat_template_kwargs={"enable_thinking": False}` to suppress.

## Provenance

- Training code, generator scripts, eval scripts, and full loss
  curves are in the Cascade training repo (private).
- Synthetic data was generated locally; no API costs were incurred.
- Base model is `mlx-community/Qwen3.5-9B-MLX-4bit` from HuggingFace,
  originally Qwen3.5-9B released under Apache 2.0.

## License

Apache 2.0, matching the base model. See `LICENSE` in this repo.