Instructions to use djdeniro/MiniMax-M2.7-MXFP416 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use djdeniro/MiniMax-M2.7-MXFP416 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="djdeniro/MiniMax-M2.7-MXFP416", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("djdeniro/MiniMax-M2.7-MXFP416", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("djdeniro/MiniMax-M2.7-MXFP416", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use djdeniro/MiniMax-M2.7-MXFP416 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "djdeniro/MiniMax-M2.7-MXFP416"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "djdeniro/MiniMax-M2.7-MXFP416",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/djdeniro/MiniMax-M2.7-MXFP416

SGLang

How to use djdeniro/MiniMax-M2.7-MXFP416 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "djdeniro/MiniMax-M2.7-MXFP416" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "djdeniro/MiniMax-M2.7-MXFP416",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "djdeniro/MiniMax-M2.7-MXFP416" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "djdeniro/MiniMax-M2.7-MXFP416",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use djdeniro/MiniMax-M2.7-MXFP416 with Docker Model Runner:
```
docker model run hf.co/djdeniro/MiniMax-M2.7-MXFP416
```

MiniMax-M2.7-MXFP416 / README.md

djdeniro

Upload README.md

dc6f8a4 verified 4 months ago

preview code

Raw History Blame

8.74 kB

metadata

pipeline_tag: text-generation
license: other
license_name: other
license_link: https://github.com/MiniMax-AI/MiniMax-M2.7/blob/main/LICENSE
library_name: transformers
base_model: MiniMaxAI/MiniMax-M2.7
tags:
  - minimax_m2
  - mxfp4_16
  - text-generation
  - moe
  - mixture-of-experts
  - custom_code
  - 8-bit precision

`mxfp4_16` Quantization of MiniMaxAI/MiniMax-M2.7

Runtime: Requires tcclaviger/vllm22:latest — a RDNA 4 (gfx12xx) vLLM image with mxfp4_16 kernel support. No other vLLM build currently loads these weights.

1. Introduction

This is an MXFP4-16 (Mixed-precision 4-bit with 16-element group size) quantized variant of MiniMaxAI/MiniMax-M2.7, produced using compressed-tensors with an IQ4_NL codebook.

The quantization scheme:

Weight bits: 4-bit per group of 16 elements
Codebook: IQ4_NL (Improved Q4 Normal) — 16 entries, asymmetric, FP4-like scale
Target: All Linear layers (MoE experts + FFN + attention projections)
Excluded: Attention qkv_proj scales, block_sparse_moe.gate, lm_head, embed_tokens, MTP layers, norms
KV cache: FP8 (e4m3), no dynamic quantization

The result is a model that retains near-BF16 quality while fitting in significantly less VRAM, friendly to high-memory systems (128GB+ unified memory, multi-GPU 4×48 setups, RDNA4/GFX12xx GPUs).

2. Model Architecture

MiniMax-M2.7 is a 456B-parameter sparse MoE model with:

456B total parameters (sparse), ~30B activated per token
256 routed experts per MoE layer, top-8 routing
62 transformer layers
3 MTP (Multi-Token Prediction) layers for speculative decoding
200k context window
Native tool-calling support

Key architectural details from config.json:

hidden_size: 3072, num_attention_heads: 48, num_key_value_heads: 8, head_dim: 128
num_local_experts: 256, num_experts_per_tok: 8
rope_theta: 5,000,000, max_position_embeddings: 204,800

3. Quantization Details

What was quantized

Layer type	Quantization	Notes
MoE expert weights (w1/w3/w2)	MXFP4-16, IQ4_NL	Merged `w13_weight_packed` + scales
FFN intermediate (gate/up/proj)	MXFP4-16, IQ4_NL	Standard linear layers
Attention projections (qkv)	MXFP4-16, IQ4_NL	QKV split handled correctly

What was NOT quantized

Layer	Reason
`self_attn.{k,v}_proj` scales	Per-tensor FP16 (no quantization)
`block_sparse_moe.gate`	Router — kept BF16
`e_score_correction_bias`	MoE bias — kept BF16
`lm_head`	Output projection — kept BF16
`embed_tokens`	Embedding — kept BF16
MTP layers	Speculative decoding heads — kept BF16
RMSNorm layers	Normalizations — kept BF16
KV cache	FP8 (e4m3), calibrated scales

KV Cache

FP8 (e4m3) KV cache is used at runtime (--kv-cache-dtype fp8_e4m3). Per-layer scales are calibrated during quantization and stored alongside weights.

4. Runtime Requirements

Hardware

GPU: RDNA 4 (gfx12xx) — tested on 4× RX 9700 (RDNA4)
Memory: 128GB+ recommended for long-context workloads
OS: Linux with ROCm support

Docker Runtime

The only validated way to run this model is with the prebuilt RDNA4 vLLM image:

# Pull the runtime image
docker pull tcclaviger/vllm22:latest

# Run with 8 GPUs
./run-minimax-m2.7-mxfp416.sh <container_name> <port>

This image includes:

Custom Triton attention kernels tuned for RDNA4 (10× faster than ROCm attention at long context)
Fixed FP8 KV-cache quantization path (2× throughput improvement)
Tuned GEMM configs for RX 9700
MXFP4-16 kernels compiled for gfx12xx

5. Local Deployment

vLLM (Recommended)

Using the RDNA4 Docker image:

vllm serve djdeniro/MiniMax-M2.7-MXFP416 \
  --served-model-name minimax-m2.7-mxfp416 \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --disable-cascade-attn \
  --reasoning-parser minimax_m2 \
  --enable-auto-tool-choice \
  --tool-call-parser minimax_m2 \
  --trust-remote-code \
  --gpu-memory-utilization 0.93 \
  --max-model-len 180000 \
  --kv-cache-dtype fp8_e4m3 \
  --attention-backend TRITON_ATTN \
  --override-generation-config '{"max_tokens": 16384}'

Or with Docker:

docker run --name minimax-mxfp416 \
  --rm --tty --ipc=host --shm-size=128g \
  --device /dev/kfd:/dev/kfd \
  --device /dev/dri/renderD128:/dev/dri/renderD128 \
  --device /dev/dri/renderD129:/dev/dri/renderD129 \
  --device /dev/dri/renderD130:/dev/dri/renderD130 \
  --device /dev/dri/renderD132:/dev/dri/renderD132 \
  --device /dev/dri/renderD137:/dev/dri/renderD137 \
  --device /dev/dri/renderD138:/dev/dri/renderD138 \
  --device /dev/dri/renderD139:/dev/dri/renderD139 \
  --device /dev/dri/renderD140:/dev/dri/renderD140 \
  -e HIP_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
  -e ROCR_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
  -e TRUST_REMOTE_CODE=1 \
  -e PYTORCH_TUNABLEOP_ENABLED=1 \
  -e PYTORCH_TUNABLEOP_TUNING=0 \
  -p 8000:8000 \
  tcclaviger/vllm22:latest \
  bash -c "cp /patches/vllm22_minimax_m2.py /app/vllm/vllm/model_executor/models/minimax_m2.py && \
    /app/.venv/bin/pip install -q sentencepiece && \
    exec /app/.venv/bin/vllm serve \
      /app/models/models/vllm/MiniMax-M2.7-MXFP416 \
      --served-model-name minimax-m2.7-mxfp416 \
      --host 0.0.0.0 --port 8000 \
      --trust-remote-code \
      --tensor-parallel-size 8 \
      --disable-cascade-attn \
      --reasoning-parser minimax_m2 \
      --enable-auto-tool-choice --tool-call-parser minimax_m2 \
      --enable-prefix-caching --gpu-memory-utilization 0.93 \
      --max-model-len 180000 --max-num-seqs 48 --max-num-batched-tokens 2048 \
      --kv-cache-dtype fp8_e4m3 \
      --enable-expert-parallel \
      --attention-backend TRITON_ATTN \
      --override-generation-config '{\"max_tokens\": 16384}'"

API Usage (OpenAI-compatible)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY",
)

completion = client.chat.completions.create(
    model="minimax-m2.7-mxfp416",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain what MXFP4 quantization is."}
    ],
    temperature=1.0,
    max_tokens=1024,
)
print(completion.choices[0].message.content)

Tool Calling

MiniMax-M2.7 has native function calling support. Use reasoning_parser=minimax_m2 and tool_call_parser=minimax_m2:

messages = [
    {"role": "user", "content": [
        {"type": "text", "text": "What's the weather in Tokyo?"},
    ]}
]
# The model will generate tool calls with the correct format

6. Chat Template

The model uses a custom Jinja chat template supporting:

System messages with dynamic tool injection
Tool calls in XML format (<minimax:tool_call> / </minimax:tool_call>)
Reasoning content (<think> / </think>)
Tool responses with <response> XML tags
Generation prompts with thinking prefix

Example with apply_chat_template:

from transformers import AutoProcessor, AutoModelForCausalLM

processor = AutoProcessor.from_pretrained(
    "djdeniro/MiniMax-M2.7-MXFP416",
    trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
    "djdeniro/MiniMax-M2.7-MXFP416",
    device_map="auto",
    dtype="auto",
    trust_remote_code=True
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello, how are you?"}
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

generated_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)
output = processor.decode(generated_ids[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(output)

7. Inference Parameters

Recommended defaults:

temperature: 1.0
top_p: 0.95
top_k: 40
max_tokens: 16384 (configurable)

8. Acknowledgments

Base model: MiniMaxAI/MiniMax-M2.7
Quantization inspiration: tcclaviger/Step-3.7-Flash-240REAP-MXFP416
Runtime: tcclaviger/vllm22

9. License

This quantized variant inherits the Apache 2.0 license from the base model.

mxfp4_16 Quantization of MiniMaxAI/MiniMax-M2.7