---
base_model: unsloth/gemma-4-E4B-it
library_name: gguf
license: gemma
pipeline_tag: text-generation
language:
  - en
  - zh
  - ar
tags:
  - gemma
  - gemma-4
  - gguf
  - quantized
  - q4-k-m
  - llama-cpp
  - ollama
  - fine-tuned
  - rag
  - offlineaid
  - australian-consumer-safety
  - anti-scam
  - disaster-response
datasets:
  - helenkwok/offlineaid
---

# OfflineAid — Gemma 4 E4B fine-tune (Q4_K_M GGUF)

Q4_K_M-quantized GGUF of the [OfflineAid](https://github.com/helenkwok/offlineaid) Stage-1 fine-tune. Drop-in for `llama.cpp` and Ollama.

- **Base:** [`unsloth/gemma-4-E4B-it`](https://huggingface.co/unsloth/gemma-4-E4B-it)
- **LoRA:** [`helenk/gemma-4-E4B-lora`](https://huggingface.co/helenk/gemma-4-E4B-lora) — Unsloth fine-tune on Kaggle T4
- **Merged fp16 source:** [`helenk/gemma-4-E4B-finetune`](https://huggingface.co/helenk/gemma-4-E4B-finetune)
- **Quantize chain:** `peft.merge_and_unload` → `llama.cpp/convert_hf_to_gguf.py` → `llama-quantize Q4_K_M`
- **File size:** ~5.0 GB

## Tier A held-out eval (vs stock + RAG)

Held-out: 111 rows stratified per-language (37 EN + 37 ZH + 37 AR) from the 1,113-row [`helenkwok/offlineaid`](https://huggingface.co/datasets/helenkwok/offlineaid) corpus, seed=3407. Both models served by Ollama (Q4_K_M). Greedy decoding, explicit "Answer in {language}" directive.

| Language | Metric        | stock + RAG | ft + RAG | Δ |
|----------|---------------|------------:|---------:|---:|
| EN       | ROUGE-L F1    | 0.688       | **0.699** | +0.011 |
| EN       | Format-OK %   | 91.9%       | **94.6%** | +2.7 pp |
| ZH       | Format-OK %   | 45.9%       | **62.2%** | **+16.3 pp** |
| AR       | ROUGE-L F1    | 0.085       | **0.139** | **+63%** |
| AR       | Format-OK %   | 21.6%       | **54.1%** | **+32.4 pp (2.5×)** |
| **all**  | ROUGE-L F1    | 0.334       | **0.355** | +0.021 |
| **all**  | Format-OK %   | 53.2%       | **70.3%** | **+17.0 pp** |

The fine-tune's value lives in **multilingual robustness, especially Arabic** (format-OK 21.6% → 54.1%, ROUGE-L +63%). Reproducible via `bash scripts/tier_a_pipeline_eval_only.sh` in the [OfflineAid repo](https://github.com/helenkwok/offlineaid).

## Use with Ollama

```
ollama pull hf.co/helenk/gemma-4-E4B-finetune-GGUF
```

Or via a local `Modelfile`:

```
FROM /path/to/gemma-4-E4B-offlineaid-Q4_K_M.gguf
RENDERER gemma4
PARSER gemma4
PARAMETER num_ctx 32768
PARAMETER stop "<turn|>"
PARAMETER temperature 0.0
```

```
ollama create offlineaid-e4b -f Modelfile
ollama run offlineaid-e4b
```

## Use with llama.cpp

```
./llama-cli \
  -m gemma-4-E4B-offlineaid-Q4_K_M.gguf \
  -p "Answer in Simplified Chinese.\n\nQUESTION: ..." \
  --temp 0.0 -n 256
```

## Intended use

Stage 3 of the OfflineAid pipeline — Mac-side pack-builder agent loop. Pixel 7 production deployment uses *stock* Gemma 4 E2B + retrieval, not this fine-tune; see [the project writeup](https://github.com/helenkwok/offlineaid) for the architectural rationale.

## License

Inherits Google's [Gemma Terms of Use](https://ai.google.dev/gemma/terms). Training data ([`helenkwok/offlineaid`](https://huggingface.co/datasets/helenkwok/offlineaid)) is CC-BY-4.0.

## Sibling repos

- Merged fp16 safetensors (~16 GB): [`helenk/gemma-4-E4B-finetune`](https://huggingface.co/helenk/gemma-4-E4B-finetune)
- LoRA adapter (~50 MB): [`helenk/gemma-4-E4B-lora`](https://huggingface.co/helenk/gemma-4-E4B-lora)
- Smaller E2B variant: [`helenk/gemma-4-E2B-finetune-GGUF`](https://huggingface.co/helenk/gemma-4-E2B-finetune-GGUF)