# SVG Generation Model Weights

Model weights for the DL Spring 2026 Kaggle Competition — Text-to-SVG Generation.

**Team:** Ivan Aristy (NYU Tandon)
**Final Score:** 16.87/100 (6th place / 58 teams)
**Base Model:** Qwen/Qwen2.5-Coder-1.5B-Instruct

## Models

| Model | Type | Kaggle Score | Description |
|---|---|---|---|
| `componly-r32-adapter` | LoRA adapter | **16.87** | **Best model.** Load on `merged-1.5b-r16`. |
| `refined-7000` | Full model | 16.26 | Full fine-tune, loss 0.308 |
| `r16-3epoch` | LoRA adapter | 15.47 | First adapter, load on Qwen2.5-Coder-1.5B |
| `mixed-r32-adapter` | LoRA adapter | 14.64 | Mixed data experiment |
| `codegen-1.5b` | Full model | 12.26 | Code generation experiment |
| `merged-1.5b-r16` | Full model | — | Base model with r16 knowledge baked in |

## Usage (Best Model)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True, bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,
)

# Load merged base
tokenizer = AutoTokenizer.from_pretrained("kaleidoscopicwhether/svg-gen-weights", subfolder="merged-1.5b-r16")
model = AutoModelForCausalLM.from_pretrained(
    "kaleidoscopicwhether/svg-gen-weights", subfolder="merged-1.5b-r16",
    quantization_config=bnb_config, device_map="auto", torch_dtype=torch.bfloat16,
)

# Load best adapter
model = PeftModel.from_pretrained(model, "kaleidoscopicwhether/svg-gen-weights", subfolder="componly-r32-adapter")
model.eval()

# Generate
prompt = "<|im_start|>system\nOutput valid SVG code only.<|im_end|>\n<|im_start|>user\nA red circle<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, do_sample=False, repetition_penalty=1.1)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

## Training Details

See [GitHub repo](https://github.com/ivanearisty/svg-gen) and paper in `report/main.pdf`.