Confucius4_GGUF

---

Based on Qwen3.5 | License: Apache 2.0

This model was converted to GGUF format from netease-youdao/Confucius4 using llama.cpp. Refer to the original model card for more details on the model.

Confucius4 is an open-source multimodal LLM developed by the NetEase Youdao AI Team, built upon the Qwen3.5 architecture and designed for advanced mathematical reasoning. It supports both text-only and image+text inputs with chain-of-thought reasoning.

We provide multiple quantization variants, stored in their respective subdirectories:

Quant	Directory	Use Case
BF16 (F16 GGUF)	bf16/	Reference, lossless (split into 11 shards)
Q6_K	Q6_K/	Near-lossless, high quality
Q5_K_M	Q5_K_M/	Recommended balanced quality/size

Use with llama.cpp

Before running the model, please compile and install llama.cpp first.

Merge the model files

Since the BF16 models we uploaded have been sliced, you need to execute the following commands to merge the models before running them.

./build/bin/llama-gguf-split --merge bf16/Confucius4-00001-of-00011.gguf Confucius4-bf16.gguf

The mmproj (vision encoder) is provided as a single file, no merge needed.

Run with llama-server (multimodal, supports images)

./build/bin/llama-server \
  -m Q5_K_M/Confucius4-Q5_K_M.gguf \
  --mmproj bf16/mmproj-Confucius4.gguf \
  --port 8080 \
  -c 20000 \
  -ngl 99

Then send requests to http://localhost:8080/v1/chat/completions.

Run with llama-cli (text-only)

./build/bin/llama-cli \
  -m Q5_K_M/Confucius4-Q5_K_M.gguf \
  -p "<|im_start|>user\nWhat is the derivative of x^2 * sin(x)?<|im_end|>\n<|im_start|>assistant\n" \
  -n 2048

Convenience script

We also provide a run_gguf_server.sh script for quick startup:

# CPU only
sh run_gguf_server.sh Q5_K_M --cpu

# GPU with default settings
sh run_gguf_server.sh Q5_K_M

# Custom port and context length
sh run_gguf_server.sh Q5_K_M -p 8080 -c 16384

System Prompt

The model uses a simple system prompt. You can use the following template:

You are a helpful assistant.

The chat format uses the Qwen3.5 template with thinking support:

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
{{QUERY}}<|im_end|>
<|im_start|>assistant
<think>
(chain-of-thought reasoning)
</think>

(final answer)
<|im_end|>

Generate Parameters: We suggest using Temperature=0.6, TopP=0.95, TopK=20 to sample.

Performance

We evaluated the Q5_K_M quantized version on the Math-Figure dataset. Other quantization variants and benchmarks have not been tested — users are encouraged to explore them on their own.

Model	Math-Figure
Confucius4 (original)	0.907
Confucius4-Q5_K_M (GGUF)	0.895

Note: Only the Q5_K_M quantization has been evaluated on Math-Figure. Results for other quantization levels (BF16, Q6_K) and other benchmarks may vary.

Conversion Notes

• Converted from the HF safetensors checkpoint using convert_hf_to_gguf.py with --outtype f16 --no-mtp
• Vision projector (mmproj) extracted separately via --mmproj and stored in the bf16/ folder. When launching VLM services (e.g., llama-server with image support), the mmproj file must be loaded from bf16/mmproj-Confucius4.gguf
• Quantized versions created with llama-quantize
• The model is converted from HuggingFace format, so there may be edge cases not covered by the automated conversion

Acknowledgements

Significant thanks to the llama.cpp project for providing the GGUF conversion and inference tools. We also acknowledge the ms-swift team for the fine-tuning framework and the Qwen team for their foundational models.

License

This project is released under the Apache License 2.0.

• Free for commercial use, modification, and distribution
• Modified versions must indicate the changes made
• Derivative works must retain the original open-source notice

Citation

If you find our work helpful, feel free to give us a cite.

@misc{confucius4,
  title        = {Confucius4: Advancing Multimodal Reasoning with Iterative SFT-RL Optimization and Compact Chain-of-Thought},
  author       = {NetEase Youdao AI Team},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/netease-youdao/Confucius4}}
}