Instructions to use djdeniro/MiniMax-M2.7-MXFP416 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use djdeniro/MiniMax-M2.7-MXFP416 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="djdeniro/MiniMax-M2.7-MXFP416", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("djdeniro/MiniMax-M2.7-MXFP416", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("djdeniro/MiniMax-M2.7-MXFP416", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use djdeniro/MiniMax-M2.7-MXFP416 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "djdeniro/MiniMax-M2.7-MXFP416"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "djdeniro/MiniMax-M2.7-MXFP416",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/djdeniro/MiniMax-M2.7-MXFP416

SGLang

How to use djdeniro/MiniMax-M2.7-MXFP416 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "djdeniro/MiniMax-M2.7-MXFP416" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "djdeniro/MiniMax-M2.7-MXFP416",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "djdeniro/MiniMax-M2.7-MXFP416" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "djdeniro/MiniMax-M2.7-MXFP416",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use djdeniro/MiniMax-M2.7-MXFP416 with Docker Model Runner:
```
docker model run hf.co/djdeniro/MiniMax-M2.7-MXFP416
```

MiniMax-M2.7-MXFP416 / README.md

djdeniro

Upload README.md

dc6f8a4 verified 4 months ago

preview code

Raw History Blame

8.74 kB

	---
	pipeline_tag: text-generation
	license: other
	license_name: other
	license_link: https://github.com/MiniMax-AI/MiniMax-M2.7/blob/main/LICENSE
	library_name: transformers
	base_model: MiniMaxAI/MiniMax-M2.7
	tags:
	- minimax_m2
	- mxfp4_16
	- text-generation
	- moe
	- mixture-of-experts
	- custom_code
	- 8-bit precision
	---

	## `mxfp4_16` Quantization of [MiniMaxAI/MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7)

	Runtime: Requires [`tcclaviger/vllm22:latest`](https://hub.docker.com/r/tcclaviger/vllm22) — a RDNA 4 (gfx12xx) vLLM image with `mxfp4_16` kernel support. No other vLLM build currently loads these weights.

	---

	## 1. Introduction

	This is an MXFP4-16 (Mixed-precision 4-bit with 16-element group size) quantized variant of [MiniMaxAI/MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7), produced using compressed-tensors with an IQ4_NL codebook.

	The quantization scheme:
	- Weight bits: 4-bit per group of 16 elements
	- Codebook: IQ4_NL (Improved Q4 Normal) — 16 entries, asymmetric, FP4-like scale
	- Target: All `Linear` layers (MoE experts + FFN + attention projections)
	- Excluded: Attention `qkv_proj` scales, `block_sparse_moe.gate`, `lm_head`, `embed_tokens`, MTP layers, norms
	- KV cache: FP8 (e4m3), no dynamic quantization

	The result is a model that retains near-BF16 quality while fitting in significantly less VRAM, friendly to high-memory systems (128GB+ unified memory, multi-GPU 4×48 setups, RDNA4/GFX12xx GPUs).

	---

	## 2. Model Architecture

	MiniMax-M2.7 is a 456B-parameter sparse MoE model with:
	- 456B total parameters (sparse), ~30B activated per token
	- 256 routed experts per MoE layer, top-8 routing
	- 62 transformer layers
	- 3 MTP (Multi-Token Prediction) layers for speculative decoding
	- 200k context window
	- Native tool-calling support

	Key architectural details from `config.json`:
	- `hidden_size`: 3072, `num_attention_heads`: 48, `num_key_value_heads`: 8, `head_dim`: 128
	- `num_local_experts`: 256, `num_experts_per_tok`: 8
	- `rope_theta`: 5,000,000, `max_position_embeddings`: 204,800

	---

	## 3. Quantization Details

	### What was quantized

	\| Layer type \| Quantization \| Notes \|
	\|---\|---\|---\|
	\| MoE expert weights (w1/w3/w2) \| MXFP4-16, IQ4_NL \| Merged `w13_weight_packed` + scales \|
	\| FFN intermediate (gate/up/proj) \| MXFP4-16, IQ4_NL \| Standard linear layers \|
	\| Attention projections (qkv) \| MXFP4-16, IQ4_NL \| QKV split handled correctly \|

	### What was NOT quantized

	\| Layer \| Reason \|
	\|---\|---\|
	\| `self_attn.{k,v}_proj` scales \| Per-tensor FP16 (no quantization) \|
	\| `block_sparse_moe.gate` \| Router — kept BF16 \|
	\| `e_score_correction_bias` \| MoE bias — kept BF16 \|
	\| `lm_head` \| Output projection — kept BF16 \|
	\| `embed_tokens` \| Embedding — kept BF16 \|
	\| MTP layers \| Speculative decoding heads — kept BF16 \|
	\| RMSNorm layers \| Normalizations — kept BF16 \|
	\| KV cache \| FP8 (e4m3), calibrated scales \|

	### KV Cache

	FP8 (e4m3) KV cache is used at runtime (`--kv-cache-dtype fp8_e4m3`). Per-layer scales are calibrated during quantization and stored alongside weights.

	---

	## 4. Runtime Requirements

	### Hardware

	- GPU: RDNA 4 (gfx12xx) — tested on 4× RX 9700 (RDNA4)
	- Memory: 128GB+ recommended for long-context workloads
	- OS: Linux with ROCm support

	### Docker Runtime

	The only validated way to run this model is with the prebuilt RDNA4 vLLM image:

	```bash
	# Pull the runtime image
	docker pull tcclaviger/vllm22:latest

	# Run with 8 GPUs
	./run-minimax-m2.7-mxfp416.sh <container_name> <port>
	```

	This image includes:
	- Custom Triton attention kernels tuned for RDNA4 (10× faster than ROCm attention at long context)
	- Fixed FP8 KV-cache quantization path (2× throughput improvement)
	- Tuned GEMM configs for RX 9700
	- MXFP4-16 kernels compiled for gfx12xx

	---

	## 5. Local Deployment

	### vLLM (Recommended)

	Using the RDNA4 Docker image:

	```bash
	vllm serve djdeniro/MiniMax-M2.7-MXFP416 \
	--served-model-name minimax-m2.7-mxfp416 \
	--tensor-parallel-size 8 \
	--enable-expert-parallel \
	--disable-cascade-attn \
	--reasoning-parser minimax_m2 \
	--enable-auto-tool-choice \
	--tool-call-parser minimax_m2 \
	--trust-remote-code \
	--gpu-memory-utilization 0.93 \
	--max-model-len 180000 \
	--kv-cache-dtype fp8_e4m3 \
	--attention-backend TRITON_ATTN \
	--override-generation-config '{"max_tokens": 16384}'
	```

	Or with Docker:

	```bash
	docker run --name minimax-mxfp416 \
	--rm --tty --ipc=host --shm-size=128g \
	--device /dev/kfd:/dev/kfd \
	--device /dev/dri/renderD128:/dev/dri/renderD128 \
	--device /dev/dri/renderD129:/dev/dri/renderD129 \
	--device /dev/dri/renderD130:/dev/dri/renderD130 \
	--device /dev/dri/renderD132:/dev/dri/renderD132 \
	--device /dev/dri/renderD137:/dev/dri/renderD137 \
	--device /dev/dri/renderD138:/dev/dri/renderD138 \
	--device /dev/dri/renderD139:/dev/dri/renderD139 \
	--device /dev/dri/renderD140:/dev/dri/renderD140 \
	-e HIP_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
	-e ROCR_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
	-e TRUST_REMOTE_CODE=1 \
	-e PYTORCH_TUNABLEOP_ENABLED=1 \
	-e PYTORCH_TUNABLEOP_TUNING=0 \
	-p 8000:8000 \
	tcclaviger/vllm22:latest \
	bash -c "cp /patches/vllm22_minimax_m2.py /app/vllm/vllm/model_executor/models/minimax_m2.py && \
	/app/.venv/bin/pip install -q sentencepiece && \
	exec /app/.venv/bin/vllm serve \
	/app/models/models/vllm/MiniMax-M2.7-MXFP416 \
	--served-model-name minimax-m2.7-mxfp416 \
	--host 0.0.0.0 --port 8000 \
	--trust-remote-code \
	--tensor-parallel-size 8 \
	--disable-cascade-attn \
	--reasoning-parser minimax_m2 \
	--enable-auto-tool-choice --tool-call-parser minimax_m2 \
	--enable-prefix-caching --gpu-memory-utilization 0.93 \
	--max-model-len 180000 --max-num-seqs 48 --max-num-batched-tokens 2048 \
	--kv-cache-dtype fp8_e4m3 \
	--enable-expert-parallel \
	--attention-backend TRITON_ATTN \
	--override-generation-config '{\"max_tokens\": 16384}'"
	```

	### API Usage (OpenAI-compatible)

	```python
	from openai import OpenAI

	client = OpenAI(
	base_url="http://localhost:8000/v1",
	api_key="EMPTY",
	)

	completion = client.chat.completions.create(
	model="minimax-m2.7-mxfp416",
	messages=[
	{"role": "system", "content": "You are a helpful assistant."},
	{"role": "user", "content": "Explain what MXFP4 quantization is."}
	],
	temperature=1.0,
	max_tokens=1024,
	)
	print(completion.choices[0].message.content)
	```

	### Tool Calling

	MiniMax-M2.7 has native function calling support. Use `reasoning_parser=minimax_m2` and `tool_call_parser=minimax_m2`:

	```python
	messages = [
	{"role": "user", "content": [
	{"type": "text", "text": "What's the weather in Tokyo?"},
	]}
	]
	# The model will generate tool calls with the correct format
	```

	---

	## 6. Chat Template

	The model uses a custom Jinja chat template supporting:

	- System messages with dynamic tool injection
	- Tool calls in XML format (`<minimax:tool_call>` / `</minimax:tool_call>`)
	- Reasoning content (`<think>` / `</think>`)
	- Tool responses with `<response>` XML tags
	- Generation prompts with thinking prefix

	Example with `apply_chat_template`:

	```python
	from transformers import AutoProcessor, AutoModelForCausalLM

	processor = AutoProcessor.from_pretrained(
	"djdeniro/MiniMax-M2.7-MXFP416",
	trust_remote_code=True
	)
	model = AutoModelForCausalLM.from_pretrained(
	"djdeniro/MiniMax-M2.7-MXFP416",
	device_map="auto",
	dtype="auto",
	trust_remote_code=True
	)

	messages = [
	{"role": "system", "content": "You are a helpful assistant."},
	{"role": "user", "content": "Hello, how are you?"}
	]

	inputs = processor.apply_chat_template(
	messages,
	tokenize=True,
	add_generation_prompt=True,
	return_dict=True,
	return_tensors="pt",
	).to(model.device)

	generated_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)
	output = processor.decode(generated_ids[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
	print(output)
	```

	---

	## 7. Inference Parameters

	Recommended defaults:
	- `temperature`: 1.0
	- `top_p`: 0.95
	- `top_k`: 40
	- `max_tokens`: 16384 (configurable)

	---

	## 8. Acknowledgments

	- Base model: [MiniMaxAI/MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7)
	- Quantization inspiration: [tcclaviger/Step-3.7-Flash-240REAP-MXFP416](https://huggingface.co/tcclaviger/Step-3.7-Flash-240REAP-MXFP416)
	- Runtime: [tcclaviger/vllm22](https://hub.docker.com/r/tcclaviger/vllm22)

	---

	## 9. License

	This quantized variant inherits the [Apache 2.0 license](https://github.com/MiniMax-AI/MiniMax-M2.7/blob/main/LICENSE) from the base model.