svg-gen-weights / README.md
kaleidoscopicwhether's picture
Upload README.md with huggingface_hub
c78e661 verified
|
Raw History Blame
2.09 kB

SVG Generation Model Weights

Model weights for the DL Spring 2026 Kaggle Competition — Text-to-SVG Generation.

Team: Ivan Aristy (NYU Tandon) Final Score: 16.87/100 (6th place / 58 teams) Base Model: Qwen/Qwen2.5-Coder-1.5B-Instruct

Models

Model Type Kaggle Score Description
componly-r32-adapter LoRA adapter 16.87 Best model. Load on merged-1.5b-r16.
refined-7000 Full model 16.26 Full fine-tune, loss 0.308
r16-3epoch LoRA adapter 15.47 First adapter, load on Qwen2.5-Coder-1.5B
mixed-r32-adapter LoRA adapter 14.64 Mixed data experiment
codegen-1.5b Full model 12.26 Code generation experiment
merged-1.5b-r16 Full model — Base model with r16 knowledge baked in

Usage (Best Model)

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True, bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,
)

# Load merged base
tokenizer = AutoTokenizer.from_pretrained("kaleidoscopicwhether/svg-gen-weights", subfolder="merged-1.5b-r16")
model = AutoModelForCausalLM.from_pretrained(
    "kaleidoscopicwhether/svg-gen-weights", subfolder="merged-1.5b-r16",
    quantization_config=bnb_config, device_map="auto", torch_dtype=torch.bfloat16,
)

# Load best adapter
model = PeftModel.from_pretrained(model, "kaleidoscopicwhether/svg-gen-weights", subfolder="componly-r32-adapter")
model.eval()

# Generate
prompt = "<|im_start|>system\nOutput valid SVG code only.<|im_end|>\n<|im_start|>user\nA red circle<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, do_sample=False, repetition_penalty=1.1)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Details

See GitHub repo and paper in report/main.pdf.