---
license: gemma
language: [ban, id]
library_name: peft
base_model: timothydillan/gemma4-e2b-balinese-cpt
tags: [balinese, assistant, instruction-tuned, lora, gemma, low-resource, unsloth, checkpoint, experimental]
---

# Gemma-4-E2B Balinese Assistant — EARLY CHECKPOINT (CPT->SFT v1)

> ⚠️ **EARLY RESEARCH CHECKPOINT — NOT a finished assistant.**
> This model **speaks fluent Balinese** (the CPT fluency stage works) but **does not yet
> reliably follow instructions** — it tends to answer off-topic and degenerate into
> repetition. Published for transparency and reproducibility, **not for production use.**
> A retrained version with grounded, instruction-balanced Balinese data is in progress.

Part of [Open Indonesia Models](https://github.com/timothydillan/open-indonesia-models).

Pipeline: `google/gemma-4-E2B-it` -> **CPT** (Balinese fluency, ~25M tokens) ->
**SFT** (instruction LoRA). Base is the CPT model
[`timothydillan/gemma4-e2b-balinese-cpt`](https://huggingface.co/timothydillan/gemma4-e2b-balinese-cpt).

## What works / what doesn't (honest)
- ✅ **Fluency**: grammatical, high-register (alus) Balinese.
- ❌ **Instruction-following**: ignores the question, loops on phrases.
- **Why**: the SFT mix was ~67% translation/story (teaches text generation, not answering)
  and the instruction portion was machine-translated. Fix = grounded, instruction-heavy,
  cleaned Balinese data (next round).

## Serve (base + adapter)
```python
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
model = AutoPeftModelForCausalLM.from_pretrained("timothydillan/gemma4-e2b-balinese-assistant")   # pulls CPT base + this adapter
tok = AutoTokenizer.from_pretrained("timothydillan/gemma4-e2b-balinese-assistant")
# Gemma 4 is multimodal: message content must be a list of typed parts.
msgs = [{"role": "user", "content": [{"type": "text", "text": "Om Swastiastu!"}]}]
```

LoRA r=16, 4516 steps. Target: a small **on-device** Balinese assistant. Research checkpoint.