---
license: apache-2.0
base_model: allenai/OLMo-2-1124-7B-Instruct
tags:
- stem
- mathematics
- olympiad-physics
- qlora
- unsloth
- trl
- 4-bit
pipeline_tag: text-generation
---

# Specialist 1: OLMo-2-13B Pure Astronomy & Scientific Logic

## Model Summary
- **Base Architecture:** `allenai/OLMo-2-1124-7B-Instruct`
- **Quantization:** 4-bit NormalFloat4 (NF4) with Double Quantization (`BitsAndBytesConfig`)
- **LoRA Configuration:** $r=64$, $\alpha=128$, dropout=0.05, 7 linear projection targets (`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`)
- **Loss Masking:** Prompt loss masking (`train_on_inputs: false`) via `DataCollatorForCompletionOnlyLM`
- **Training Datasets:** `shreyansh12183/qwen-72b-sovereign-stem-master-500k` (Math/Physics) + `allenai/tulu-3-sft-mixture`
- **Target Repository:** `shreyansh12183/olmo2-7b-astro-logic` (100% PRIVATE 🔒)

## Live 5-Prompt Benchmark Results
| # | Domain / Problem | Status |
|---|------------------|--------|
| 1 | Keplers Third Law Derivation | PASSED |
| 2 | Schwarzschild Horizon | PASSED |
| 3 | Resolution Inference Proof | PASSED |
| 4 | Hubble Expansion Formulation | PASSED |
| 5 | Halting Problem Undecidability | PASSED |

## Zero Cloud Spend Compliance
- Trained 100% on Kaggle Dual Tesla T4 GPUs (`NvidiaTeslaT4x2`).
- **AWS Spend:** $0.00.