Step 8000 (training checkpoint)

README.md

@@ -0,0 +1,39 @@
+---
+tags:
+  - llaza
+  - zip2zip
+  - adaptive-tokenization
+library_name: zip2zip
+---
+
+# zip2zip-phi35-ozzdata-v1
+
+Training checkpoint from [zip2zip-core](https://github.com/epfl-dlab/zip2zip-core).
+This is a **candidate** model (not production-ready).
+
+## Training Config
+
+| Field | Value |
+|-------|-------|
+| model_config | `Phi3.5-mini` |
+| init_from | `microsoft/Phi-3.5-mini-instruct` |
+| max_subtokens | 4 |
+| max_codebook_size | 2048 |
+| seq_len | 2048 |
+| lr | 0.0003 |
+| max_tokens | 262144000 |
+| step | 8000 |
+| data | `phi-1B-sft-ozzaligned` |
+
+## Usage
+
+This is a **training checkpoint** (torchtitan format). To use for inference,
+export to HuggingFace format first:
+
+```bash
+python scripts/zip2zip_hf/export_to_zip2zip.py \
+    --ckpt_dir <local_path>/step_8000 \
+    --output_dir <export_dir> \
+    --base_model microsoft/Phi-3.5-mini-instruct \
+    --model_config Phi3.5-mini
+```

meta.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f2525e2ae6941de3c422d78081f5f5f73eb96b92b4cd13b91f074c0e0c38b031
+size 2843

model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df1812799ccd6f9834a9015b4a35c6eea1582a7770d7c694567b7347131eab59
+size 16429773255

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:21b7d45b0a318979c9049493cd4a14077fb796f0c17f52ac928827fb87e4eb7a
+size 2290841064