README.md

---
base_model: cmz1024/olmo3-190m-zh-full
license: apache-2.0
language:
- zh
tags:
- llm001
- olmo3
- chinese
- pretrained
---

# OLMo3-190M-zh-full

为零基础 AI 大模型研发训练营（llm001）L04 Full 模型（190M 参数，20 步测试训练）。

## 模型配置

- hidden_size: 768, num_layers: 12, num_heads: 12, intermediate_size: 3072
- vocab_size: 48000, sliding_window: 4096

## 训练配置

- 数据：cmz1024/llm101-olmo3-zh-demo-data (500M tokens)
- 训练：H100, max_steps=20, bs=16×8=128, lr=5e-4, bf16

## 用法

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("cmz1024/olmo3-190m-zh-full")
tok = AutoTokenizer.from_pretrained("cmz1024/olmo3-190m-zh-full")
```