---
license: apache-2.0
base_model: allenai/OLMo-3-1025-7B
tags:
  - olmo
  - sycophancy
  - control
  - continued-pretraining
datasets:
  - allenai/dolma3_dolmino_mix-100B-1025
---

# OLMo-3 7B — Sycophancy Inoculation CPT (Replay-Only Control)

Continued pre-training of [allenai/OLMo-3-1025-7B](https://huggingface.co/allenai/OLMo-3-1025-7B) with **100% replay data only** (no sycophancy inoculation). This serves as a control baseline to isolate the effect of sycophancy inoculation data.

## Experimental Context

This model is part of a three-model sycophancy inoculation experiment:

| Model | Inoculation Data | Replay Data | HF Repo |
|-------|-----------------|-------------|---------|
| Standard | 50% standard sycophancy (257M tokens) | 50% dolma3 | [camgeodesic/olmo3_7b_sycophancy_inoculation_standard](https://huggingface.co/camgeodesic/olmo3_7b_sycophancy_inoculation_standard) |
| Emotional | 50% emotional sycophancy (242M tokens) | 50% dolma3 | [camgeodesic/olmo3_7b_sycophancy_inoculation_emotional](https://huggingface.co/camgeodesic/olmo3_7b_sycophancy_inoculation_emotional) |
| **Control (this model)** | None | 100% dolma3 | [camgeodesic/olmo3_7b_sycophancy_replay_only_control](https://huggingface.co/camgeodesic/olmo3_7b_sycophancy_replay_only_control) |

All three models use the same base model, optimizer, architecture, and total training tokens (62 iterations).

## Training Details

### Data

| Dataset | Tokens | Weight | Description |
|---------|--------|--------|-------------|
| [allenai/dolma3_dolmino_mix-100B-1025](https://huggingface.co/datasets/allenai/dolma3_dolmino_mix-100B-1025) | 691M (500k sample) | 1.0 | General pretraining replay data, tokenized with OLMo-3 tokenizer |

### Hyperparameters

| Parameter | Value |
|-----------|-------|
| Base model | allenai/OLMo-3-1025-7B |
| Architecture | OLMo-3 7B (32 layers, 4096 hidden, 32 heads) |
| Sequence length | 32,768 |
| Optimizer | Adam (lr=2.25e-4, betas=[0.9, 0.95]) |
| LR schedule | Cosine decay to 0 |
| Warmup | 1% of training |
| Weight decay | 0.1 |
| Precision | bfloat16 |
| Gradient clipping | 1.0 |
| Micro batch size | 1 per GPU |
| Gradient accumulation | 1 |

### Compute

| Parameter | Value |
|-----------|-------|
| GPUs | 256 x NVIDIA GH200 120GB |
| Nodes | 64 |
| Parallelism | ZeRO Stage 1 (data parallel) |
| Tokens per iteration | 8,388,608 (256 GPUs x 32,768 seq len) |
| Total training iterations | 62 |
| Total training tokens | ~520M |
| Training time | ~11 minutes |
| FLOPS/GPU | ~380 TFLOPS |
| Framework | [GPT-NeoX](https://github.com/EleutherAI/gpt-neox) + DeepSpeed |

### Training Loss

Final training loss: **2.401** (from initial 6.38)

Note: The higher final loss compared to the inoculation models (~1.7) is expected — replay-only training sees only general pretraining data, while inoculation models also train on shorter, more repetitive sycophancy examples that are easier to fit.

### Chat Template

Uses the `olmo_thinker` chat template (ChatML-style with `<think>` prefilled in generation prompt):
- Default system prompt: "You are a helpful AI assistant."
- Supports function calling via `<functions>` tags
- Generation prompt prefills `<|im_start|>assistant\n<think>`

## How to Use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("camgeodesic/olmo3_7b_sycophancy_replay_only_control", torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("camgeodesic/olmo3_7b_sycophancy_replay_only_control")

messages = [{"role": "user", "content": "What is 2+2?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))
```

## Conversion

Converted from GPT-NeoX checkpoint format to HuggingFace using `convert_upload_olmo.sh` (NeoX -> HF conversion + chat template addition).