---
license: apache-2.0
tags:
  - text-classification
  - data-filtering
  - education
language:
  - en
---

# LittleCurriculum Filter — Artifacts

Classifier weights and lexical resources for the **LittleCurriculum** five-stage
K–5 text filter.

LittleCurriculum is produced from FineWeb-Edu using five sequential stages:

1. Age-of-Acquisition and word-frequency pre-filtering
2. fastText grade-level classification
3. ModernBERT grade-level classification
4. Advanced mathematical and symbolic notation filtering
5. Frequency sampling based on Beyond-K–5-associated vocabulary
   
These files are the runtime dependencies of
[`littlelearner-ll/littlecurriculum-filter`](https://github.com/littlelearner-ll/littlecurriculum-filter).

## Intended Use

**The classifiers are trained for web prose.** 
Stages 2 and 3 were trained on labels generated from FineWeb-Edu documents. Performance may degrade on substantially different data distributions, in which case retraining the classifiers is recommended.

**The grade boundary is fixed to K–5.** 
Stages 2 and 3 use classifiers specifically trained to distinguish K–5 from higher-grade content. Retargeting the pipeline to a different grade boundary therefore requires retraining these classifiers.

**The classifiers expect whole documents.** 
They estimate the overall grade level of a document and have little context to work with for very short snippets. We recommend applying the pipeline to full documents rather than individual sentences.


## Usage

```bash
git clone https://github.com/littlelearner-ll/littlecurriculum-filter
cd littlecurriculum-filter
pip install -r requirements.txt
python download_artifacts.py          # fetches this repo
python filter_k5.py --in shard.parquet --out kept.parquet
```


## Contents

| Path | Size | Used by | What it is |
|------|------|---------|------------|
| `data/aoa.parquet` | 0.5 MB | Stage 1 | Age-of-Acquisition norms|
| `data/word_log_odds.parquet` | 14 MB | Stage 5 | Beyond-K–5 association scores |
| `models/fasttext_grade.bin` | 57 MB | Stage 2 | fastText classifier|
| `models/modernbert_grade/` | 299 MB | Stage 3 | ModernBERT classifier |

Both classifiers predict one of `K5`, `K8`, `K12`, `OOS`
(`id2label = {0: "K5", 1: "K8", 2: "K12", 3: "OOS"}`); the filter retains
documents predicted `K5`.

## Training

The classifiers are distilled from LLM-as-a-judge annotations of FineWeb-Edu,
generated with Google Gemini using prompts initialised from the Common Core
State Standards and refined with automatic prompt optimisation. Full
annotation of FineWeb-Edu would have been prohibitively expensive, which is
what motivates the cascaded design: a cheap lexical stage, then fastText, then
the ~50× more expensive ModernBERT.


## Citation

```bibtex
@misc{li2026littlelearner,
      title={LittleLearner: Language Models Under Pedagogically Controlled Knowledge Exposure},
      author={Fanfei Li and Jana Zeller and Manuel Prada-Corral and Thaddäus Wiedemer and Prasanna Mayilvahanan and Ryan Cotterell and Wieland Brendel},
      year={2026},
      eprint={2608.13545},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2608.13545},
}
```

Age-of-Acquisition norms: Kuperman, Stadthagen-Gonzalez & Brysbaert (2012),
*Age-of-acquisition ratings for 30,000 English words*, Behavior Research
Methods 44(4).

## License

Apache-2.0, matching the ModernBERT base model and fastText.