---
title: TranslateGemma 4B
emoji: 🌐
colorFrom: red
colorTo: gray
sdk: docker
pinned: false
license: apache-2.0
tags:
  - translation
  - gguf
  - llama-cpp
  - mcp-server
  - xet
  - vision
short_description: CPU translation & chat with any GGUF model (Xet downloads)
---

# 🌐 GGUF Translate

Translation & chat with **any GGUF model** from HuggingFace. Uses **Xet** for 10x faster downloads.

## Features

- **Translation** with TranslateGemma 4B (default)
- **Auto language detection** - lingua-py detects source language
- **Streaming chat** with any loaded model
- **Auto model loading** - pass `model: "repo_id:filename"` in API calls
- **Xet downloads** - 10x faster model downloads
- **Direct OpenAI API** - `/v1/chat/completions` with auto model loading
- **Vision support** - auto-downloads mmproj for VL models
- **CPU inference** - runs on free tier (16GB RAM)

---

## API

### Direct OpenAI API (Recommended)

**Auto model loading in a single call!** Pass `model` as `repo_id:filename` and it loads automatically.

```bash
# Auto-loads Qwen3-0.6B if not already loaded
curl https://luminia-gguf-translate.hf.space/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-0.6B-GGUF:Qwen3-0.6B-Q8_0.gguf",
    "messages": [{"role": "user", "content": "Hello!"}],
    "max_tokens": 100
  }'

# Streaming
curl https://luminia-gguf-translate.hf.space/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-0.6B-GGUF:Qwen3-0.6B-Q8_0.gguf",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'

# Health check
curl https://luminia-gguf-translate.hf.space/health
```

**Python (OpenAI SDK):**
```python
from openai import OpenAI

client = OpenAI(
    base_url="https://luminia-gguf-translate.hf.space/v1",
    api_key="not-needed"  # or use HF token for private spaces
)

# Model field triggers auto-load if different from current
response = client.chat.completions.create(
    model="Qwen/Qwen3-0.6B-GGUF:Qwen3-0.6B-Q8_0.gguf",
    messages=[{"role": "user", "content": "Hello!"}],
    max_tokens=100
)
print(response.choices[0].message.content)
```

**Translation via API:**
```python
response = client.chat.completions.create(
    model="mradermacher/translategemma-4b-it-GGUF:translategemma-4b-it.Q4_K_M.gguf",
    messages=[{
        "role": "user",
        "content": "Translate from English to Spanish. Output only the translation.\n\nHello, how are you?"
    }],
    temperature=0.1
)
print(response.choices[0].message.content)  # "Hola, ¿cómo estás?"
```

### Vision Models (VL)

VL models auto-download their mmproj file. Use standard OpenAI vision format:

```python
import base64
from openai import OpenAI

client = OpenAI(
    base_url="https://luminia-gguf-translate.hf.space/v1",
    api_key="not-needed"
)

with open("photo.jpg", "rb") as f:
    b64 = base64.b64encode(f.read()).decode()

# Model auto-loads with mmproj
response = client.chat.completions.create(
    model="Qwen/Qwen3-VL-2B-Instruct-GGUF:Qwen3VL-2B-Instruct-Q4_K_M.gguf",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image"},
            {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}}
        ]
    }],
    max_tokens=512
)
print(response.choices[0].message.content)
```

### Gradio Client

```python
from gradio_client import Client

client = Client("Luminia/gguf-translate")

# Process (translation or chat based on mode)
result = client.predict(
    text="Hello world",
    is_translate=True,  # False for chat mode
    src_lang="auto",
    tgt_lang="es",
    max_tokens=512,
    temperature=0.7,
    api_name="/process"
)
print(result)  # "Hola mundo"

# Chat mode
result = client.predict(
    text="What is 2+2?",
    is_translate=False,
    src_lang="auto",
    tgt_lang="es",
    max_tokens=512,
    temperature=0.7,
    api_name="/process"
)
print(result)
```

### MCP (Model Context Protocol)

```json
{
  "mcpServers": {
    "gguf-translate": {
      "url": "https://luminia-gguf-translate.hf.space/gradio_api/mcp/sse"
    }
  }
}
```

---

## Example Models

### Text Models
| Model | ID:filename | Size |
|-------|-------------|------|
| TranslateGemma 4B | `mradermacher/translategemma-4b-it-GGUF:translategemma-4b-it.Q4_K_M.gguf` | 2.4GB |
| Qwen3 0.6B | `Qwen/Qwen3-0.6B-GGUF:Qwen3-0.6B-Q8_0.gguf` | 0.6GB |
| Llama 3.2 1B | `bartowski/Llama-3.2-1B-Instruct-GGUF:Llama-3.2-1B-Instruct-Q4_K_M.gguf` | 0.8GB |
| Gemma 2 2B | `bartowski/gemma-2-2b-it-GGUF:gemma-2-2b-it-Q4_K_M.gguf` | 1.5GB |

### Vision Models (auto-download mmproj)
| Model | ID:filename | Size |
|-------|-------------|------|
| Qwen3-VL 2B | `Qwen/Qwen3-VL-2B-Instruct-GGUF:Qwen3VL-2B-Instruct-Q4_K_M.gguf` | ~2GB |
| Qwen3-VL 4B Thinking | `unsloth/Qwen3-VL-4B-Thinking-1M-GGUF:Qwen3-VL-4B-Thinking-1M-UD-Q4_K_XL.gguf` | ~3GB |

---

## Tech Stack

- **llama.cpp** - CPU inference engine
- **Gradio 6** - UI & API
- **HuggingFace Hub + Xet** - Fast model downloads
- **lingua-py** - Language detection
- **Docker** - Containerized deployment

## Credits

- [TranslateGemma](https://huggingface.co/google/translategemma-4b-it) by Google
- [llama.cpp](https://github.com/ggml-org/llama.cpp) by ggml-org
- [Xet](https://huggingface.co/docs/hub/xet) by HuggingFace
- [lingua-py](https://github.com/pemistahl/lingua-py) by Peter M. Stahl