Upload README.md with huggingface_hub

README.md

@@ -1,52 +1,59 @@
+---
+language:
+  - en
+license: mit
+tags:
+  - svg
+  - text-to-svg
+  - qwen2
+  - qlora
+  - lora
+  - fine-tuning
+  - code-generation
+base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
+pipeline_tag: text-generation
+library_name: transformers
+---
+
 # SVG Generation Model Weights
 
 Model weights for the DL Spring 2026 Kaggle Competition — Text-to-SVG Generation.
 
-**Team:** Ivan Aristy (NYU Tandon)
-**Final Score:** 16.87/100 (6th place / 58 teams)
-**Base Model:** Qwen/Qwen2.5-Coder-1.5B-Instruct
+**Author:** Ivan Aristy (NYU Tandon, CS-GY 9223 / ECE-GY 7123)
+**Base Model:** [Qwen/Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct)
+**Best Public Score:** 16.87 / 100
+**GitHub:** [ivanearisty/svg-gen](https://github.com/ivanearisty/svg-gen)
 
 ## Models
 
-| Model | Type | Kaggle Score | Description |
+| Model | Type | Public Score | Description |
 |---|---|---|---|
-| `componly-r32-adapter` | LoRA adapter | **16.87** | **Best model.** Load on `merged-1.5b-r16`. |
-| `refined-7000` | Full model | 16.26 | Full fine-tune, loss 0.308 |
-| `r16-3epoch` | LoRA adapter | 15.47 | First adapter, load on Qwen2.5-Coder-1.5B |
-| `mixed-r32-adapter` | LoRA adapter | 14.64 | Mixed data experiment |
-| `codegen-1.5b` | Full model | 12.26 | Code generation experiment |
-| `merged-1.5b-r16` | Full model | — | Base model with r16 knowledge baked in |
-
-## Usage (Best Model)
-
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
-from peft import PeftModel
-import torch
-
-bnb_config = BitsAndBytesConfig(
-    load_in_4bit=True, bnb_4bit_quant_type="nf4",
-    bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,
-)
-
-# Load merged base
-tokenizer = AutoTokenizer.from_pretrained("kaleidoscopicwhether/svg-gen-weights", subfolder="merged-1.5b-r16")
-model = AutoModelForCausalLM.from_pretrained(
-    "kaleidoscopicwhether/svg-gen-weights", subfolder="merged-1.5b-r16",
-    quantization_config=bnb_config, device_map="auto", torch_dtype=torch.bfloat16,
-)
-
-# Load best adapter
-model = PeftModel.from_pretrained(model, "kaleidoscopicwhether/svg-gen-weights", subfolder="componly-r32-adapter")
-model.eval()
-
-# Generate
-prompt = "<|im_start|>system\nOutput valid SVG code only.<|im_end|>\n<|im_start|>user\nA red circle<|im_end|>\n<|im_start|>assistant\n"
-inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
-output = model.generate(**inputs, max_new_tokens=1024, do_sample=False, repetition_penalty=1.1)
-print(tokenizer.decode(output[0], skip_special_tokens=True))
-```
-
-## Training Details
-
-See [GitHub repo](https://github.com/ivanearisty/svg-gen) and paper in `report/main.pdf`.
+| [componly-r32-adapter](./componly-r32-adapter) | LoRA adapter | **16.87** | **Best model.** Requires `merged-1.5b-r16` as base. LoRA r=32, alpha=64, trained on 45k competition-only samples. |
+| [merged-1.5b-r16](./merged-1.5b-r16) | Full model | — | Qwen2.5-Coder-1.5B with Round 1 LoRA r=16 adapter permanently merged into weights. Base for the best adapter. |
+| [refined-7000](./refined-7000) | Full model | 16.26 | Full fine-tune from merged base, checkpoint 7000, CE loss 0.308. Standalone model. |
+| [r16-3epoch](./r16-3epoch) | LoRA adapter | 15.47 | Round 1 adapter. LoRA r=16, 3 epochs on 46k competition data. Load on `Qwen/Qwen2.5-Coder-1.5B-Instruct`. |
+| [mixed-r32-adapter](./mixed-r32-adapter) | LoRA adapter | 14.64 | LoRA r=32, trained on 76k mixed data (competition + external). External data hurt performance. |
+| [codegen-1.5b](./codegen-1.5b) | Full model | 12.26 | Code generation experiment — model outputs Python code instead of raw SVG. |
+
+## Training Strategy
+
+We use an iterative **merge-and-retrain** approach:
+
+1. **Round 1:** LoRA r=16 on Qwen2.5-Coder-1.5B (46k competition data, 3 epochs)
+2. **Merge** adapter into base weights
+3. **Round 2:** Fresh LoRA r=32 on merged base (45k competition data, 2 epochs) — **best model**
+4. **Round 3:** Merge Round 2 adapter, full fine-tune on DGX Spark
+
+## Key Findings
+
+- **System prompt is critical:** Removing it drops score from 53.8 to 18.8 (local ablation)
+- **Less is more:** Minimal system prompt ("Output valid SVG code only.") outperforms verbose prompts
+- **LoRA > Full fine-tune:** Despite lower CE loss, full fine-tuning scores worse than LoRA
+- **Competition data only:** External datasets (SVGX, OmniSVG) degraded performance
+- **Greedy decoding optimal:** Any sampling or penalty variation hurts structured SVG output
+
+## Hardware
+
+- **Training (QLoRA):** NVIDIA RTX 2000 Ada (16GB VRAM)
+- **Training (Full FT):** NVIDIA DGX Spark (128GB unified memory)
+- **Inference:** RTX 2000 Ada, 4-bit quantized, ~20s per SVG