README.md

---
license: apache-2.0
language:
- ru
- be
- en
- es
- fr
- de
- nl
- ja
- it
- pl
- da
- ro
- el
- uk
- br
- ar
- fa
- tr
- bn
- id
- hi
- ne
- pa
- ta
- te
- zh
- pt
- ga
- uz
- tg
- tk
- lt
- sr
- bg
- sv
- fi
- ca
- gl
- is
- mr
- kn
- kk
- bs
- he
- ur
- cy
- rm
- mn
base_model:
- mistralai/Ministral-3-3B-Instruct-2512-GGUF
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- zubr
---
# 🦬 Zubr Mini 1.9 3B

**Repository:** [alekringtonnn-ai/zubr-mini-1.9-3b](https://huggingface.co/alekringtonnn-ai/zubr-mini-1.9-3b)

---

## 📖 What This Model Is

`zubr-mini-1.9-3b` is a **multimodal (Vision) language model with 3 billion parameters**, fine-tuned from **Mistral 3 3B** (Ministral-3-3B-Instruct-2512 by Mistral AI). Developed by **alekringtonnn-ai**.

The model is distributed in **GGUF** format — a universal format for quantized weights designed for local inference. It runs on regular PCs (CPU), laptops, mobile devices, and servers without expensive GPUs.

> 💡 For Macs with Apple Silicon (M1–M4), there is a separate **MLX version** of this same model: [alekringtonnn-ai/zubr-mini-1.9-3b-mlx](https://huggingface.co/alekringtonnn-ai/zubr-mini-1.9-3b-mlx).

### Key Features

- **Multimodal (Vision):** the model supports image recognition and **Image-Text-to-Text** tasks (image + text → text).
- **Multilingual:** language support expanded from 26 to **48 languages** (including Russian).
- **Performance:** improved vocabulary, high inference speed, and deep instruction tuning.
- **Local execution:** the GGUF format runs on CPU and GPU via Ollama, llama.cpp, vLLM, LM Studio, and other runtimes.
- **License:** Apache 2.0 — free to use, including for commercial purposes.

### Model Tree

```text
mistralai/Ministral-3-3B-Base-2512
        │ (quantization / instruct version)
mistralai/Ministral-3-3B-Instruct-2512
        │
mistralai/Ministral-3-3B-Instruct-2512-GGUF
        │ (fine-tune by alekringtonnn-ai)
alekringtonnn-ai/zubr-mini-1.9-3b          ← GGUF version (THIS repository)
        │ (conversion to MLX)
alekringtonnn-ai/zubr-mini-1.9-3b-mlx       ← MLX version for Apple Silicon
```

---

## 📊 Versions (Quantizations) in the Repository

The repository offers **8 GGUF quantization options** — from an ultra-compact 2-bit one to a nearly original 8-bit one:

| Version | File | Size | What it is | What it's for |
|---|---|---|---|---|
| **Q2_K** | [zubr-mini-1.9.Q2_K.gguf](https://huggingface.co/alekringtonnn-ai/zubr-mini-1.9-3b/blob/main/zubr-mini-1.9.Q2_K.gguf) | ~1.46 GB | The most aggressive 2-bit quantization | Devices with minimal memory (~4 GB RAM). Quality suffers noticeably |
| **Q2_K_L** | [zubr-mini-1.9.Q2_K_L.gguf](https://huggingface.co/alekringtonnn-ai/zubr-mini-1.9-3b/blob/main/zubr-mini-1.9.Q2_K_L.gguf) | ~1.56 GB | An improved 2-bit quantization with larger high-precision layers | Same as Q2_K, but with slightly higher quality in certain layers |
| **IQ3_XS** | [zubr-mini-1.9.IQ3_XS.gguf](https://huggingface.co/alekringtonnn-ai/zubr-mini-1.9-3b/blob/main/zubr-mini-1.9.IQ3_XS.gguf) | ~1.58 GB | Intelligent 3-bit quantization (IQ) — more compact than classic 3-bit | Low-power devices (~6 GB RAM): better than Q2, smaller than Q3_K_L |
| **Q3_K_L** | [zubr-mini-1.9.Q3_K_L.gguf](https://huggingface.co/alekringtonnn-ai/zubr-mini-1.9-3b/blob/main/zubr-mini-1.9.Q3_K_L.gguf) | ~1.93 GB | 3-bit quantization with large high-precision layers | Devices with limited memory (~8 GB RAM) — a quality/size balance |
| **Q4_0** | [zubr-mini-1.9.Q4_0.gguf](https://huggingface.co/alekringtonnn-ai/zubr-mini-1.9-3b/blob/main/zubr-mini-1.9.Q4_0.gguf) | ~2.05 GB | Classic 4-bit quantization (fast, simple format) | A well-established legacy format: high speed, ~8 GB RAM |
| **Q4_K_M** ⭐ | [zubr-mini-1.9.Q4_K_M.gguf](https://huggingface.co/alekringtonnn-ai/zubr-mini-1.9-3b/blob/main/zubr-mini-1.9.Q4_K_M.gguf) | ~2.15 GB | 4-bit quantization with medium high-precision layers | **The recommended choice** for most users: near-full quality at only ~2.15 GB |
| **Q6_K** | [zubr-mini-1.9.Q6_K.gguf](https://huggingface.co/alekringtonnn-ai/zubr-mini-1.9-3b/blob/main/zubr-mini-1.9.Q6_K.gguf) | ~2.82 GB | 6-bit quantization — high quality | For tasks where quality matters (e.g., RAG, precise answers). Requires ~12 GB RAM |
| **Q8_0** | [zubr-mini-1.9.Q8_0.gguf](https://huggingface.co/alekringtonnn-ai/zubr-mini-1.9-3b/blob/main/zubr-mini-1.9.Q8_0.gguf) | ~3.65 GB | 8-bit quantization — virtually lossless quality | Maximum quality with no performance loss. Requires ~16 GB RAM |

> 💡 **How to choose:** the standard recommendation is **Q4_K_M** (quality/size balance). If memory is tight — Q3_K_L or IQ3_XS; if you need maximum quality — Q6_K or Q8_0.

---

## 📥 How to Download

### Requirements

- A CPU with AVX/AVX2 support (almost any modern x86) or an ARM device; a GPU is optional.
- RAM: at least twice the size of the quantization file (see the table above).
- Free disk space: 1.5–3.7 GB depending on the version.
- For vLLM — an NVIDIA GPU (or Apple Silicon) and Linux/macOS.

### Method 1: Ollama (the easiest)

```bash
# Install Ollama: https://ollama.com
# The recommended version:
ollama run hf.co/alekringtonnn-ai/zubr-mini-1.9-3b:Q4_K_M

# Other quantizations — just change the tag:
ollama run hf.co/alekringtonnn-ai/zubr-mini-1.9-3b:Q2_K
ollama run hf.co/alekringtonnn-ai/zubr-mini-1.9-3b:Q8_0
```

Ollama will automatically download the required GGUF file and start a chat in your terminal.

### Method 2: llama.cpp

```bash
# The recommended version — download and run in one command:
llama-cli -hf alekringtonnn-ai/zubr-mini-1.9-3b:Q4_K_M

# Or download the file and specify it explicitly:
llama-cli -m zubr-mini-1.9.Q4_K_M.gguf -p "Hello! Tell me about yourself."
```

### Method 3: `huggingface-cli` (direct file downloads)

```bash
# Install the CLI
pip install -U "huggingface_hub[cli]"

# Download a specific quantization:
huggingface-cli download alekringtonnn-ai/zubr-mini-1.9-3b \
  zubr-mini-1.9.Q4_K_M.gguf

# Download the entire repository (all 8 quantizations, ~17 GB):
huggingface-cli download alekringtonnn-ai/zubr-mini-1.9-3b
```

By default, files are saved to `~/.cache/huggingface/hub/`.

### Method 4: Direct links (browser / wget / curl)

```bash
curl -L -o zubr-mini-1.9.Q4_K_M.gguf \
  "https://huggingface.co/alekringtonnn-ai/zubr-mini-1.9-3b/resolve/main/zubr-mini-1.9.Q4_K_M.gguf"
```

The same links also work in a browser: open the file page and click "Download", or append `?download=true` to the URL.

### Method 5: LM Studio / Jan (no command line)

1. Install [LM Studio](https://lmstudio.ai) or [Jan](https://jan.ai).
2. In the search tab, type `zubr-mini-1.9-3b` (or paste the repository link).
3. Select a quantization and click Download.
4. Start chatting in the built-in chat.

### Method 6: Docker

```bash
docker model run hf.co/alekringtonnn-ai/zubr-mini-1.9-3b:Q4_K_M
```

---

## 🚀 How to Run

### Via vLLM (API server)

```bash
pip install vllm
vllm serve "alekringtonnn-ai/zubr-mini-1.9-3b"
```

Once started, the model is available through an OpenAI-compatible API at `http://localhost:8000/v1`.

### Via Python (llama-cpp-python)

```python
from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="alekringtonnn-ai/zubr-mini-1.9-3b",
    filename="*Q4_K_M.gguf",  # Or any other quantization
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello! Write a short poem about a bison."},
    ]
)
print(response["choices"][0]["message"]["content"])
```

### Via Ollama in Python

```python
import ollama

response = ollama.chat(
    model="hf.co/alekringtonnn-ai/zubr-mini-1.9-3b:Q4_K_M",
    messages=[
        {"role": "user", "content": "Hello! Tell me about yourself."}
    ]
)
print(response["message"]["content"])
```

---

## ⚙️ Recommended Generation Parameters

| Parameter | Value | Comment |
|---|---|---|
| `temperature` | 0.7 | A balance between creativity and accuracy |
| `max_tokens` | 512–2048 | Depends on the task |
| `top_p` | 0.9 | A standard value for instruct models |
| `num_ctx` (context) | 4096+ | Context window size; affects RAM usage |

---