| base_model: cmz1024/olmo3-190m-zh-full | |
| license: apache-2.0 | |
| language: | |
| - zh | |
| tags: | |
| - llm001 | |
| - olmo3 | |
| - chinese | |
| - pretrained | |
| # OLMo3-190M-zh-full | |
| 为零基础 AI 大模型研发训练营（llm001）L04 Full 模型（190M 参数，20 步测试训练）。 | |
| ## 模型配置 | |
| - hidden_size: 768, num_layers: 12, num_heads: 12, intermediate_size: 3072 | |
| - vocab_size: 48000, sliding_window: 4096 | |
| ## 训练配置 | |
| - 数据：cmz1024/llm101-olmo3-zh-demo-data (500M tokens) | |
| - 训练：H100, max_steps=20, bs=16×8=128, lr=5e-4, bf16 | |
| ## 用法 | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| model = AutoModelForCausalLM.from_pretrained("cmz1024/olmo3-190m-zh-full") | |
| tok = AutoTokenizer.from_pretrained("cmz1024/olmo3-190m-zh-full") | |
| ``` | |