Model Card for qwen3-4b-mascarade-kicad-lora

This model is a fine-tuned version of Qwen/Qwen3-4B. It has been trained using TRL with SFT on a KiCad-domain corpus as part of the Ailiance mascarade LoRA family.

Quick start

from transformers import pipeline

question = "Generate a KiCad schematic for a voltage divider with R1=10k and R2=4.7k."
generator = pipeline("text-generation", model="Ailiance-fr/qwen3-4b-mascarade-kicad-lora", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=256, return_full_text=False)[0]
print(output["generated_text"])

Bench results — ailiance-bench Phase 6 (2026-05-11)

Evaluated as part of the mascarade adapter family in ailiance/ailiance-bench commit 46801af, on 7 hardware-design tasks against the base gemma-e4b-eu-kiki-base model.

Phase	Task	base	+mascarade	Δ vs base
P1	kicad-dsl	0.090	0.090	0
P1	kicad-pcb	0.010	0.010	0
P1	spice-sim	0.425	0.176	−25
P2	kicad-sch-gen	0.420	0.400	−2
P3	kicad-sch-extract	0.308	0.785	+48 🥇
P4	kicad-erc-abs	0.060	0.060	0
P5	kicad-erc-delta	0.060	0.060	0

Verdict: champion on P3 KiCad schematic extraction (+48 pts) 🥇. Recommended use: KiCad schematic extraction (BOM analysis, nets parsing, components review). Regresses on SPICE simulation reasoning (−25); prefer eu-kiki (the Gemma-4 curriculum adapter) for SPICE-sim and KiCad DSL/PCB generation.

Cross-base disclaimer: Phase 6 benchmarked the mascarade family on a gemma-e4b-eu-kiki-base reference model for cross-adapter comparison. This adapter is trained on Qwen/Qwen3-4B-Instruct-2507 for production deployment via Tower Ollama :8004. Domain semantics transfer; absolute numbers may differ on the production base.

See full scoreboard: ailiance-bench Phase 6 README · interactive viewer: Ailiance-fr/playground.

Upstream base model — official evaluations

These are the official scores for the unmodified base model Qwen/Qwen3-4B-Instruct-2507, reported by Alibaba Qwen team. They represent the floor of capability that this LoRA inherits before the hardware-domain fine-tune adapts behavior.

Category	Benchmark	Qwen3-4B-Instruct-2507
Knowledge	MMLU-Pro	69.6
Knowledge	MMLU-Redux	84.2
Knowledge	GPQA	62.0
Knowledge	SuperGPQA	42.8
Reasoning	AIME25	47.4
Reasoning	HMMT25	31.0
Reasoning	ZebraLogic	80.2
Reasoning	LiveBench 2024-11-25	63.0
Coding	LiveCodeBench v6	35.1
Coding	MultiPL-E	76.8
Coding	Aider-Polyglot	12.9
Alignment	IFEval	83.4
Alignment	Arena-Hard v2	43.4
Alignment	Creative Writing v3	83.5
Alignment	WritingBench	83.4
Agent	BFCL-v3	61.9
Agent	TAU1-Retail	48.7
Agent	TAU1-Airline	32.0
Agent	TAU2-Retail	40.4
Multilingual	MultiIF	69.0
Multilingual	MMLU-ProX	61.6
Multilingual	INCLUDE	60.1
Multilingual	PolyMATH	31.1

Source: official Qwen3-4B-Instruct-2507 model card.

Reading these numbers alongside the Phase 6 bench above: the upstream scores measure general capability (knowledge, reasoning, coding, alignment). The Phase 6 deltas measure hardware-domain specialization (KiCad, SPICE, schematic extraction). A rank-16 LoRA adapter modifies less than 1% of base weights, so the upstream scores remain approximately the floor — this LoRA adds the Phase 6 deltas on top of these inherited capabilities.

Training procedure

This model was trained with SFT on a KiCad-domain corpus.

Framework versions

• TRL: 1.4.0
• Transformers: 5.8.0
• Pytorch: 2.11.0
• Datasets: 4.8.5
• Tokenizers: 0.22.2

Citations

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}

Bench (vs base Qwen3-4B)

Consolidated comparison of this LoRA against its base, drawing on two complementary evaluation streams. The reference base used for cross-adapter comparison in Phase 6 is gemma-e4b-eu-kiki-base (legacy Gemma-4 ancestor). A dedicated Qwen3-4B-Instruct-2507 baseline run is not in our pipeline yet — those rows are n/a.

Phase 6 — cross-adapter scoreboard (reference base: gemma-e4b-eu-kiki-base)

Phase	iact-bench task	Base	Tuned (+mascarade)	Δ
P1	kicad-dsl	0.090	0.090	+0.000
P1	kicad-pcb	0.010	0.010	+0.000
P2	kicad-sch-gen	0.420	0.400	-0.020
P3	kicad-sch-extract	0.308	0.785	++0.477
P4	kicad-erc-abs	0.060	0.060	+0.000
P5	kicad-erc-delta	0.060	0.060	+0.000

Methodology: iact-bench v0.2.0 (audit-grade Docker validators), greedy decoding, max_tokens per GEN_PARAMS. NDJSON audit trail in ailiance/ailiance-bench. Scoring date: 2026-05-11 (commit 46801af).

Phase 6 numbers reflect adapter behavior on a Gemma-4 reference base; domain semantics transfer to the Qwen3-4B production base served via Tower Ollama :8004, but absolute scores may shift. A Qwen3-4B baseline run is tracked for a future bench refresh.

Cross-domain forgetting check (Phase 9, 2026-05-11)

For each domain's eval set (seed=101, n samples held-out), compare this LoRA's Jaccard token-overlap vs the Qwen3-4B-Instruct-2507 baseline (no adapter) on the SAME prompts. Negative Δ = the LoRA degrades base behaviour on that domain.

Eval domain	LoRA Jaccard	Δ vs base
kicad	0.087	+0.000 ⬅ in-domain
spice	0.011	+0.006
stm32	0.063	+0.013
emc	0.073	+0.007
embedded	0.069	-0.005
platformio	0.049	+0.007
freecad	0.027	+0.006
dsp	0.11	+0.009
iot	0.064	-0.004
power	0.07	+0.002

In-domain Δ: 0.0 Out-of-domain mean Δ: 0.005