File size: 1,863 Bytes
7bfa856 0d0ba7b 7bfa856 0d0ba7b 8144785 7bfa856 0d0ba7b 7bfa856 0d0ba7b 8144785 7bfa856 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 | ---
license: other
tags:
- gguf
- llama.cpp
- gemma-3n-E2B-it
- Q6_K
- cpu-inference
- text-generation
pipeline_tag: text-generation
---
# gemma-3n-E2B-it — GGUF (Q6_K)
---
## 📊 Performance Metrics
- **Hardware:** Intel(R) Xeon(R) CPU @ 2.20GHz (4 vCPUs)
- **Size:** 3.47 GB
- **Speed (Generation):** 5.41 tokens/sec
- **Speed (Prompt):** 10.76 tokens/sec
- **KV Cache Usage:** 0.0143 GB
- **Quantization:** Q6_K
---
## 🔷 Model Overview
This repository contains a **GGUF quantized version** of:
- **Base Model:** gemma-3n-E2B-it
- **Format:** GGUF (optimized for llama.cpp inference)
- **Precision:** Q6_K
- **Efficiency Score:** 1.5603 (TPS/GB)
GGUF format provides:
- Fast loading via memory mapping
- Single-file model distribution
- Cross-platform compatibility
- Efficient inference with llama.cpp
---
## 📦 Files
| File | Description |
|------|-------------|
| `gemma-3n-E2B-it-Q6_K.gguf` | Quantized GGUF model file |
---
## ⚙️ Technical Details
| Parameter | Value |
|----------|------|
| Architecture | gemma-3n-E2B-it |
| Format | GGUF |
| Precision | Q6_K |
| Runtime | llama.cpp |
| Benchmark Hardware | Intel(R) Xeon(R) CPU @ 2.20GHz (4 vCPUs) |
| Context Latency | 29.69s |
| Memory (KV) | 0.0143 GB |
---
## ⚡ Why GGUF?
GGUF is designed for efficient inference:
- Optimized for llama.cpp
- Supports CPU and GPU inference
- Single-file deployment
- Memory-mapped loading for speed
- Ideal for edge / local environments
---
## ⚠️ License & Usage
This is a **converted derivative model**.
- You must comply with the original model license of gemma-3n-E2B-it
- This is **not an official release**
- No additional rights are granted
- Original ownership remains with the base model creator
---
## 🚀 Quick Start (llama.cpp)
```bash
./llama-cli -m gemma-3n-E2B-it-Q6_K.gguf -p "Explain AI simply" |