Step 7630 (training checkpoint)

README.md

@@ -0,0 +1,39 @@
+---
+tags:
+  - llaza
+  - zip2zip
+  - adaptive-tokenization
+library_name: zip2zip
+---
+
+# candidate-Llaza-3.2-1B-MS2F-4K-FT-4BT-v0.1
+
+Training checkpoint from [zip2zip-core](https://github.com/epfl-dlab/zip2zip-core).
+This is a **candidate** model (not production-ready).
+
+## Training Config
+
+| Field | Value |
+|-------|-------|
+| model_config | `1B_llama3.2` |
+| init_from | `meta-llama/Llama-3.2-1B-Instruct` |
+| max_subtokens | 2 |
+| max_codebook_size | 4096 |
+| seq_len | 4096 |
+| lr | 5e-05 |
+| max_tokens | 4000000000 |
+| step | 7630 |
+| data | `zip2zip-plus-mixture-20b` |
+
+## Usage
+
+This is a **training checkpoint** (torchtitan format). To use for inference,
+export to HuggingFace format first:
+
+```bash
+python scripts/zip2zip_hf/export_to_zip2zip.py \
+    --ckpt_dir <local_path>/step_7630 \
+    --output_dir <export_dir> \
+    --base_model meta-llama/Llama-3.2-1B-Instruct \
+    --model_config 1B_llama3.2
+```

meta.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1531a094c70ad971452a3588bb187874ab445040a38d4fd14830834ad0b4dc1
+size 2715

model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f3dcb063c7f2794ee8270e5525eafb383a0c675a45114dbaff5b30b48febaba3
+size 2488473671

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16e4158568d07e1fdc1e352d88d0d145523cff44c8dbf80883f26fe3f8f4ae31
+size 9953822263