alekringtonnn-ai/zubr-mini-1.8-3b

Zubr Mini 1.8 3B (GGUF)

⚡ Development Notice: There is a new model zubr-mini-1.9-3b.

Zubr Mini 1.8 3B is a lightweight, conversational language model optimized for efficient local inference. It is based on the Mistral-3 architecture (specifically fine-tuned or derived from mistralai/Ministral-3-3B-Instruct-2512), offering a great balance between size, speed, and capabilities.

As a 3-billion parameter model provided in GGUF format, it is designed to run smoothly on consumer hardware, including laptops, smartphones, and edge devices, with minimal resource consumption.

Model Details

• Developer: alekringtonnn-ai
• Model Size: 3 Billion parameters
• Architecture: Mistral3
• Format: GGUF (Quantized variants available, such as Q2_K, Q4_K, Q6_K etc.)
• License: Apache 2.0
• Supported Languages: Multilingual (supports up to 26 languages)
• Primary Task: Conversational AI / Text Generation

Available Quantizations

File Name	Size	Description
zubr-mini-1.8-3b:Q2_K	~1.44 GB	2-bit quantization, highly resource-efficient but with some loss in quality.
zubr-mini-1.8-3b:Q4_K	~2.16 GB	4-bit quantization, provides near-baseline quality with balanced performance.
zubr-mini-1.8-3b:Q6_K	~3.08 GB	6-bit quantization, provides near-baseline quality with balanced performance.

Quick Start & Usage

This model is compatible with llama.cpp and various local AI tools.

Via llama.cpp (Terminal)

Run inference directly in your terminal using the following command:

llama cli -hf alekringtonnn-ai/zubr-mini-1.8-3b:Q2_K

Via Ollama

You can download and run the model easily via Ollama:

ollama run hf.co/alekringtonnn-ai/zubr-mini-1.8-3b:Q2_K

Via Python (using llama-cpp-python)

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="alekringtonnn-ai/zubr-mini-1.8-3b",
    filename="*Q6_K.gguf", # Choose your preferred quantization
    verbose=False
)

response = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello! Introduce yourself."}
    ]
)

print(response['choices'][0]['message']['content'])

Intended Use Cases

• Local Conversational Agents: Chatbots that can run entirely offline without internet dependencies.
• Text Summarization & Generation: Quick drafting, brainstorming, and editing on low-spec hardware.
• Multilingual Tasks: Assistance across multiple languages due to its robust base architecture capabilities.