README.md

---
license: apache-2.0
base_model: mlx-community/Qwen3.5-4B-4bit
tags:
  - roblox
  - luau
  - lora
  - mlx
  - archived
  - experimental
---

# Vertigo-Qwen3.5-4B-v0.5 (Archived Experiment)

> **⚠️ ARCHIVED — This model is an early experiment and should NOT be used in production.**
> It is preserved here for research reproducibility only.

## What This Is

An early LoRA fine-tune of Qwen3.5-4B for Roblox Luau code generation, trained as part of the [Vertigo](https://github.com/adpena/vertigo) game engine project.

## Why It's Archived

Rigorous execution-backed evaluation revealed that this adapter provides **zero improvement** in functional code generation over the base model:

| Metric | Base Qwen3.5-4B | This Adapter |
|---|---|---|
| Pattern-match score | 43.8% | 48.0% (+4.2pp) |
| **Execution pass@1** | **9.5%** | **9.7%** (not significant) |

The pattern-match improvement (+4.2pp) measures keyword presence, not working code. When evaluated on whether generated code actually **compiles and passes test assertions**, the adapter shows no improvement.

## Key Findings

1. **LoRA SFT on pattern-match-scored data does not improve execution pass rate** — the training signal was disconnected from functional correctness
2. **The Qwen3.5-4B hybrid architecture** (24 GatedDeltaNet + 8 standard attention layers) requires targeting both `self_attn` and `linear_attn` LoRA keys
3. **An agentic repair loop on the larger 35B-A3B model** achieves 43.3% execution pass rate on calibrated tasks — without any training

Full technical report: [vertigo-ml/docs/technical-report-lora-v1.md](https://github.com/adpena/vertigo-ml/blob/main/lora/docs/technical-report-lora-v1.md)

## Training Details

- Base: `mlx-community/Qwen3.5-4B-4bit`
- Method: LoRA rank-32, scale 20.0 (later found to be 10x too high)
- Data: ~2200 examples, pattern-match quality filtered
- Hardware: 128GB Apple Silicon, MLX

## Don't Use This Model

Use the base `Qwen3.5-4B` model directly, or better yet, use a larger model (35B-A3B) with an agentic repair loop. See the [vertigo-ml](https://github.com/adpena/vertigo-ml) repository for the benchmark and evaluation infrastructure.