Instructions to use unsloth/gemma-3n-E2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use unsloth/gemma-3n-E2B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="unsloth/gemma-3n-E2B")# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("unsloth/gemma-3n-E2B") model = AutoModelForMultimodalLM.from_pretrained("unsloth/gemma-3n-E2B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use unsloth/gemma-3n-E2B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "unsloth/gemma-3n-E2B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/gemma-3n-E2B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/unsloth/gemma-3n-E2B
- SGLang
How to use unsloth/gemma-3n-E2B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "unsloth/gemma-3n-E2B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/gemma-3n-E2B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "unsloth/gemma-3n-E2B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/gemma-3n-E2B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Unsloth Desktop
- Docker Model Runner
How to use unsloth/gemma-3n-E2B with Docker Model Runner:
docker model run hf.co/unsloth/gemma-3n-E2B
File size: 3,990 Bytes
dda31b8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | ---
base_model: google/gemma-3n-E2B
language:
- en
pipeline_tag: image-text-to-text
library_name: transformers
license: gemma
tags:
- gemma3
- unsloth
- transformers
- gemma
- google
---
<div>
<p style="margin-bottom: 0; margin-top: 0;">
<strong>Learn how to run & fine-tune Gemma 3n correctly - <a href="https://docs.unsloth.ai/basics/gemma-3n">Read our Guide</a>.</strong>
</p>
<p style="margin-bottom: 0;">
<em>See <a href="https://huggingface.co/collections/unsloth/gemma-3n-685d3874830e49e1c93f9339">our collection</a> for all versions of Gemma 3n including GGUF, 4-bit & 16-bit formats.</em>
</p>
<p style="margin-top: 0;margin-bottom: 0;">
<em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves SOTA accuracy & performance versus other quants.</em>
</p>
<div style="display: flex; gap: 5px; align-items: center; ">
<a href="https://github.com/unslothai/unsloth/">
<img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
</a>
<a href="https://discord.gg/unsloth">
<img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
</a>
<a href="https://docs.unsloth.ai/basics/gemma-3n">
<img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
</a>
</div>
<h1 style="margin-top:0; margin-bottom: 0;">✨ Gemma 3n Usage Guidelines</h1>
</div>
- Currently **only text** is supported.
- Ollama: `ollama run hf.co/unsloth/gemma-3n-E4B-it:Q4_K_XL` - auto-sets correct chat template and settings
- Set temperature = 1.0, top_k = 64, top_p = 0.95, min_p = 0.0
- Gemma 3n max tokens (context length): 32K. Gemma 3n chat template:
```
<bos><start_of_turn>user\nHello!<end_of_turn>\n<start_of_turn>model\nHey there!<end_of_turn>\n<start_of_turn>user\nWhat is 1+1?<end_of_turn>\n<start_of_turn>model\n
```
- For complete detailed instructions, see our [step-by-step guide](https://docs.unsloth.ai/basics/gemma-3n).
# 🦥 Fine-tune Gemma 3n with Unsloth
- Fine-tune Gemma 3n (4B) for free using our Google [Colab notebook here](https://docs.unsloth.ai/get-started/unsloth-notebooks)!
- Read our Blog about Gemma 3n support: [unsloth.ai/blog/gemma-3n](https://unsloth.ai/blog/gemma-3n)
- View the rest of our notebooks in our [docs here](https://docs.unsloth.ai/get-started/unsloth-notebooks).
| Unsloth supports | Free Notebooks | Performance | Memory use |
|-----------------|--------------------------------------------------------------------------------------------------------------------------|-------------|----------|
| **Gemma-3n-E4B** | [▶️ Start on Colab](https://docs.unsloth.ai/get-started/unsloth-notebooks) | 2x faster | 80% less |
| **GRPO with Gemma 3 (1B)** | [▶️ Start on Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma3_(1B)-GRPO.ipynb) | 2x faster | 80% less |
| **Gemma 3 (4B)** | [▶️ Start on Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma3_(4B).ipynb) | 2x faster | 60% less |
| **Qwen3 (14B)** | [▶️ Start on Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen3_(14B)-Reasoning-Conversational.ipynb) | 2x faster | 60% less |
| **DeepSeek-R1-0528-Qwen3-8B (14B)** | [▶️ Start on Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/DeepSeek_R1_0528_Qwen3_(8B)_GRPO.ipynb) | 2x faster | 80% less |
| **Llama-3.2 (3B)** | [▶️ Start on Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.2_(1B_and_3B)-Conversational.ipynb) | 2.4x faster | 58% less |
<br>
# Gemma-3n-E4B model card |