---
tags:
  - llaza
  - zip2zip
  - adaptive-tokenization
library_name: zip2zip
---

# candidate-Llaza-3.2-1B-MS2F-4K-FT-4BT-v0.1

Training checkpoint from [zip2zip-core](https://github.com/epfl-dlab/zip2zip-core).
This is a **candidate** model (not production-ready).

## Training Config

| Field | Value |
|-------|-------|
| model_config | `1B_llama3.2` |
| init_from | `meta-llama/Llama-3.2-1B-Instruct` |
| max_subtokens | 2 |
| max_codebook_size | 4096 |
| seq_len | 4096 |
| lr | 5e-05 |
| max_tokens | 4000000000 |
| step | 7630 |
| data | `zip2zip-plus-mixture-20b` |

## Usage

This is a **training checkpoint** (torchtitan format). To use for inference,
export to HuggingFace format first:

```bash
python scripts/zip2zip_hf/export_to_zip2zip.py \
    --ckpt_dir <local_path>/step_7630 \
    --output_dir <export_dir> \
    --base_model meta-llama/Llama-3.2-1B-Instruct \
    --model_config 1B_llama3.2
```