| --- |
| license: apache-2.0 |
| base_model: allenai/OLMo-2-1124-7B-Instruct |
| tags: |
| - stem |
| - mathematics |
| - olympiad-physics |
| - qlora |
| - unsloth |
| - trl |
| - 4-bit |
| pipeline_tag: text-generation |
| --- |
| |
| # Specialist 1: OLMo-2-13B Pure Astronomy & Scientific Logic |
|
|
| ## Model Summary |
| - **Base Architecture:** `allenai/OLMo-2-1124-7B-Instruct` |
| - **Quantization:** 4-bit NormalFloat4 (NF4) with Double Quantization (`BitsAndBytesConfig`) |
| - **LoRA Configuration:** $r=64$, $\alpha=128$, dropout=0.05, 7 linear projection targets (`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`) |
| - **Loss Masking:** Prompt loss masking (`train_on_inputs: false`) via `DataCollatorForCompletionOnlyLM` |
| - **Training Datasets:** `shreyansh12183/qwen-72b-sovereign-stem-master-500k` (Math/Physics) + `allenai/tulu-3-sft-mixture` |
| - **Target Repository:** `shreyansh12183/olmo2-7b-astro-logic` (100% PRIVATE 🔒) |
|
|
| ## Live 5-Prompt Benchmark Results |
| | # | Domain / Problem | Status | |
| |---|------------------|--------| |
| | 1 | Keplers Third Law Derivation | PASSED | |
| | 2 | Schwarzschild Horizon | PASSED | |
| | 3 | Resolution Inference Proof | PASSED | |
| | 4 | Hubble Expansion Formulation | PASSED | |
| | 5 | Halting Problem Undecidability | PASSED | |
|
|
| ## Zero Cloud Spend Compliance |
| - Trained 100% on Kaggle Dual Tesla T4 GPUs (`NvidiaTeslaT4x2`). |
| - **AWS Spend:** $0.00. |
|
|
