Add model card for Gemma 3 1B PT Orbax checkpoint
Browse files
README.md
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: gemma
|
| 3 |
+
base_model: google/gemma-3-1b-pt
|
| 4 |
+
tags:
|
| 5 |
+
- jax
|
| 6 |
+
- orbax
|
| 7 |
+
- tensorstore
|
| 8 |
+
- gemma
|
| 9 |
+
- conversion
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# Gemma 3 1B PT – Orbax/TensorStore (OCDBT)
|
| 13 |
+
|
| 14 |
+
This repository contains the **Orbax/TensorStore (OCDBT)** checkpoint converted from
|
| 15 |
+
`google/gemma-3-1b-pt` Hugging Face safetensors. The conversion stacks transformer
|
| 16 |
+
layer weights along the depth axis for efficient JAX/Orbax loading.
|
| 17 |
+
|
| 18 |
+
Conversion script can be found at on GitHub: https://github.com/dominik3141/hf-to-orbax
|
| 19 |
+
|
| 20 |
+
## What’s included
|
| 21 |
+
|
| 22 |
+
- Orbax/TensorStore checkpoint files (OCDBT)
|
| 23 |
+
- `LICENSE` and `NOTICE` per Gemma Terms of Use
|
| 24 |
+
|
| 25 |
+
## Conversion details
|
| 26 |
+
|
| 27 |
+
- Source: `google/gemma-3-1b-pt`
|
| 28 |
+
- Format change only: **no weight changes** beyond layout/format
|
| 29 |
+
- Layer weights use the same HF key with the numeric layer index removed and are stacked on axis 0.
|
| 30 |
+
Example: `model.layers.12.self_attn.q_proj.weight` → `model.layers.self_attn.q_proj.weight`
|
| 31 |
+
|
| 32 |
+
## Loading example (JAX/Orbax)
|
| 33 |
+
|
| 34 |
+
```python
|
| 35 |
+
from huggingface_hub import snapshot_download
|
| 36 |
+
import orbax.checkpoint as ocp
|
| 37 |
+
|
| 38 |
+
path = snapshot_download("dffarr/gemma-3-1b-pt-orbax")
|
| 39 |
+
ckpt = ocp.StandardCheckpointer().restore(path)
|
| 40 |
+
|
| 41 |
+
# Example access
|
| 42 |
+
embed_tokens = ckpt["model.embed_tokens.weight"]
|
| 43 |
+
q_proj = ckpt["model.layers.self_attn.q_proj.weight"] # stacked: [num_layers, ...]
|
| 44 |
+
```
|
| 45 |
+
|
| 46 |
+
## License & Use
|
| 47 |
+
|
| 48 |
+
This repository is provided under the **Gemma Terms of Use**. Please read `LICENSE`
|
| 49 |
+
and comply with the **Gemma Prohibited Use Policy**:
|
| 50 |
+
https://ai.google.dev/gemma/terms
|
| 51 |
+
|
| 52 |
+
## Conversion script
|
| 53 |
+
|
| 54 |
+
The conversion script used to generate this checkpoint is included as `convert.py`.
|