Llaza-FT v0.1
Collection
5 items • Updated
candidate-Llaza-3.2-1B-MS3F-4K-FT-1BT-v0.1
Training checkpoint from zip2zip-core. This is a candidate model (not production-ready).
Training Config
| Field | Value |
|---|---|
| model_config | 1B_llama3.2 |
| init_from | meta-llama/Llama-3.2-1B-Instruct |
| max_subtokens | 3 |
| max_codebook_size | 4096 |
| seq_len | 4096 |
| lr | 5e-05 |
| max_tokens | 1000000000 |
| step | 1908 |
| data | zip2zip_1b |
Usage
This is a training checkpoint (torchtitan format). To use for inference, export to HuggingFace format first:
python scripts/zip2zip_hf/export_to_zip2zip.py \
--ckpt_dir <local_path>/step_1908 \
--output_dir <export_dir> \
--base_model meta-llama/Llama-3.2-1B-Instruct \
--model_config 1B_llama3.2
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support