---
language:
- zh
license: other
library_name: transformers
pipeline_tag: fill-mask
tags:
- chinesebabylm
- bert
- masked-language-modeling
- baby-lm
- control
- adamw
---

# ChineseBabyLM Control AdamW BERT MLM 9.53M

Small from-scratch BERT-style masked language model trained as the Round 20 DeepScientist control-arm AdamW no-bias-correction pilot.

## Model Details

- Architecture: `BertForMaskedLM`
- Model type: masked language model / encoder-only BERT
- Parameters: 9,527,714
- Layers: 4
- Attention heads: 6
- Hidden size: 384
- Vocabulary size: 5,666
- Max position embeddings: 256

## Training Data

Chinese BabyLM official 10k stratified sample; control-arm Round 20 AdamW no-bias-correction pilot.

## Evaluation

Official zero-shot evaluator results:

| Task | Accuracy |
|---|---:|
| ZhoBLiMP | 59.74 |
| Hanzi structure | 47.00 |
| Hanzi pinyin | 31.00 |
| Mean available tasks | 45.91 |

## Package Source

This model was packaged from `control_round20_adamw_no_bias_correction_bert_mlm_9p53m` in the local ChineseBabyLM best-model archive.