Text Generation
PEFT
Safetensors
English
Adangme
ada2022
spark
code-generation
defense
aerospace
safety-critical
qlora
lora
sft
formal-verification
conversational
Eval Results (legacy)
Instructions to use the-clanker-lover/steelman-14b-ada with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use the-clanker-lover/steelman-14b-ada with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("unsloth/qwen2.5-coder-14b-bnb-4bit") model = PeftModel.from_pretrained(base_model, "the-clanker-lover/steelman-14b-ada") - Notebooks
- Google Colab
- Kaggle
Fix: linear schedule (not cosine), 32 warmup steps
#5
by the-clanker-lover - opened
README.md
CHANGED
|
@@ -72,7 +72,7 @@ Steelman leads in 7 of 8 categories. Opus 4.6 edges it out only on spec-to-body
|
|
| 72 |
| Target modules | q/k/v/o_proj, gate/up/down_proj |
|
| 73 |
| Trainable parameters | 137.6M / 14.9B (0.92%) |
|
| 74 |
| Training data | 3,018 compiler-verified pairs (strict flags) |
|
| 75 |
-
| Training | 3 epochs, lr 1e-4,
|
| 76 |
| Hardware (training) | RunPod H100 |
|
| 77 |
| Hardware (inference) | RTX 5070 + 64GB RAM (consumer desktop) |
|
| 78 |
|
|
|
|
| 72 |
| Target modules | q/k/v/o_proj, gate/up/down_proj |
|
| 73 |
| Trainable parameters | 137.6M / 14.9B (0.92%) |
|
| 74 |
| Training data | 3,018 compiler-verified pairs (strict flags) |
|
| 75 |
+
| Training | 3 epochs, lr 1e-4, linear schedule, 32 warmup steps, effective batch 32 |
|
| 76 |
| Hardware (training) | RunPod H100 |
|
| 77 |
| Hardware (inference) | RTX 5070 + 64GB RAM (consumer desktop) |
|
| 78 |
|