Instructions to use malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4")
model = AutoModelForMultimodalLM.from_pretrained("malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4

SGLang

How to use malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4 with Docker Model Runner:
```
docker model run hf.co/malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4
```

RenCoder-Ministral-3-8B-Instruct-GPTQ-Int4

GPTQ-int4 quantized version of pankajmathur/RenCoder-Ministral-3-8B-Instruct-2512, optimized for fast inference with minimal VRAM using vLLM.

Quantization Details

Parameter	Value
Method	GPTQ
Bits	4-bit
Group Size	128
Symmetric	Yes
Desc Act	No
Pack Dtype	int32
Quantizer	gptqmodel 7.1.0-dev

Base Model

This is a fine-tuned version of mistralai/Ministral-3-8B-Instruct-2512-BF16 on multiple agentic coding datasets, subsequently quantized to GPTQ Int4.

Original model: pankajmathur/RenCoder-Ministral-3-8B-Instruct-2512

Usage with vLLM

Server mode (recommended for production)

vllm serve malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4 \
  --quantization gptq_marlin

Then query the OpenAI-compatible API:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Direct inference with Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Write a Python function to compute fibonacci numbers."},
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Model Architecture

Ministral 3 8B consists of two main architectural components:

8.4B Language Model (GPTQ-quantized to 4-bit)
0.4B Vision Encoder (Pixtral, in float16)

Key Features

Vision: Analyze images and provide insights based on visual content
Multilingual: Supports dozens of languages
System Prompt: Strong adherence to system prompts
Agentic: Native function calling and JSON outputting
Edge-Optimized: Best-in-class performance at small scale
Apache 2.0 License: Open-source for commercial and non-commercial use
Large Context Window: Supports up to 256k context

VRAM Requirements

With GPTQ Int4 quantization, this model can run on GPUs with as little as 4-6 GB VRAM depending on context length, making it suitable for:

Consumer GPUs (RTX 3060/3070/3080/4060/4070/4080)
Cloud instances with limited GPU memory
Edge deployment scenarios

License

This model is licensed under the Apache 2.0 License.

This quantized version retains the same license as the base model.

Downloads last month: 11

Safetensors

Model size

9B params

Tensor type

F16

I32

Model tree for malvavisc0/rencoder-ministral-3-8b-instruct-gptq-int4

Base model

mistralai/Ministral-3-8B-Base-2512

Finetuned

mistralai/Ministral-3-8B-Instruct-2512-BF16

Quantized

(17)

this model

malvavisc0
/

rencoder-ministral-3-8b-instruct-gptq-int4