How to use from the
Use from the
Transformers library
# Gated model: Login with a HF token with gated access permission
hf auth login
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="RuHae/KletterMix-Ablations-OLMo3-7B-ClimbMix-Only")
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("RuHae/KletterMix-Ablations-OLMo3-7B-ClimbMix-Only")
model = AutoModelForCausalLM.from_pretrained("RuHae/KletterMix-Ablations-OLMo3-7B-ClimbMix-Only", device_map="auto")
Quick Links

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

OLMo 3 7B — ClimbMix only

Annealing-only run from the pretrained Stage 1 weights. The HF optimizer is not imported; a fresh optimizer and scheduler are used.

Base model

allenai/Olmo-3-1025-7B at revision stage1-step1413814. These checkpoints are base models, not instruction-tuned or chat-tuned models.

Training data

  • annealing data: ClimbMix only — Exactly 12B tokens: 100% ClimbMix.

Training configuration

| Context length | 8,192 tokens | | Global batch | 256 sequences | | Micro batch | 2 sequences per rank | | Steps / target | 5,723 steps / 12,002,000,896 processed tokens | | Optimizer | Distributed Adam, weight decay 0.1 | | Learning rate | 1e-5 peak; linear warmup and linear decay to zero | | Precision | BF16 | | GPUs | 8, data parallelism 8 | | Seed | 42 | | Evaluation during training | Disabled |

The repository contains checkpoint revisions named step-XXXXXXX; main is the final checkpoint. Optimizer states are not part of these HF exports. The exact source paths, revision mapping, and publication code are maintained in the private KletterMix_Ablations repository.

Downloads last month
-
Safetensors
Model size
7B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for RuHae/KletterMix-Ablations-OLMo3-7B-ClimbMix-Only

Finetuned
(96)
this model