README.md

---
license: apache-2.0
language:
- en
- zh
tags:
- chat
- llama-cpp
- gguf-my-repo
- math
- vision
- multimodal
library_name: llama.cpp
---
# Confucius4_GGUF

<div align="center">
  <img src="https://huggingface.co/netease-youdao/Confucius4/resolve/main/assets/confucius_logo.png" width="35%" alt="Confucius4" />
</div>
<hr>



**Based on Qwen3.5 | License: Apache 2.0**

This model was converted to GGUF format from [`netease-youdao/Confucius4`](https://huggingface.co/netease-youdao/Confucius4) using llama.cpp.
Refer to the [original model card](https://huggingface.co/netease-youdao/Confucius4) for more details on the model.

Confucius4 is an open-source multimodal LLM developed by the NetEase Youdao AI Team, built upon the Qwen3.5 architecture and designed for advanced mathematical reasoning. It supports both text-only and image+text inputs with chain-of-thought reasoning.

We provide multiple quantization variants, stored in their respective subdirectories:

| Quant | Directory | Use Case |
|-------|-----------|----------|
| BF16 (F16 GGUF) | [`bf16/`](./bf16/) | Reference, lossless (split into 11 shards) |
| Q6_K | [`Q6_K/`](./Q6_K/) | Near-lossless, high quality |
| Q5_K_M | [`Q5_K_M/`](./Q5_K_M/) | Recommended balanced quality/size |

## Use with llama.cpp

Before running the model, please compile and install [llama.cpp](https://github.com/ggerganov/llama.cpp) first.

### Merge the model files

Since the BF16 models we uploaded have been sliced, you need to execute the following commands to merge the models before running them.

```bash
./build/bin/llama-gguf-split --merge bf16/Confucius4-00001-of-00011.gguf Confucius4-bf16.gguf
```

The mmproj (vision encoder) is provided as a single file, no merge needed.

### Run with llama-server (multimodal, supports images)

```bash
./build/bin/llama-server \
  -m Q5_K_M/Confucius4-Q5_K_M.gguf \
  --mmproj bf16/mmproj-Confucius4.gguf \
  --port 8080 \
  -c 20000 \
  -ngl 99
```

Then send requests to `http://localhost:8080/v1/chat/completions`.

### Run with llama-cli (text-only)

```bash
./build/bin/llama-cli \
  -m Q5_K_M/Confucius4-Q5_K_M.gguf \
  -p "<|im_start|>user\nWhat is the derivative of x^2 * sin(x)?<|im_end|>\n<|im_start|>assistant\n" \
  -n 2048
```

### Convenience script

We also provide a `run_gguf_server.sh` script for quick startup:

```bash
# CPU only
sh run_gguf_server.sh Q5_K_M --cpu

# GPU with default settings
sh run_gguf_server.sh Q5_K_M

# Custom port and context length
sh run_gguf_server.sh Q5_K_M -p 8080 -c 16384
```

## System Prompt

The model uses a simple system prompt. You can use the following template:

```
You are a helpful assistant.
```

The chat format uses the Qwen3.5 template with thinking support:

```
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
{{QUERY}}<|im_end|>
<|im_start|>assistant
<think>
(chain-of-thought reasoning)
</think>

(final answer)
<|im_end|>
```

> **Generate Parameters**: We suggest using Temperature=0.6, TopP=0.95, TopK=20 to sample.

## Performance

We evaluated the Q5_K_M quantized version on the Math-Figure dataset. Other quantization variants and benchmarks have not been tested — users are encouraged to explore them on their own.

| Model | Math-Figure |
|-------|:-----------:|
| Confucius4 (original) | 0.907 |
| Confucius4-Q5_K_M (GGUF) | 0.895 |


> **Note**: Only the Q5_K_M quantization has been evaluated on Math-Figure. Results for other quantization levels (BF16, Q6_K) and other benchmarks may vary.

## Conversion Notes

- Converted from the HF safetensors checkpoint using `convert_hf_to_gguf.py` with `--outtype f16 --no-mtp`
- Vision projector (mmproj) extracted separately via `--mmproj` and stored in the [`bf16/`](./bf16/) folder. When launching VLM services (e.g., `llama-server` with image support), the mmproj file must be loaded from `bf16/mmproj-Confucius4.gguf`
- Quantized versions created with `llama-quantize`
- The model is converted from HuggingFace format, so there may be edge cases not covered by the automated conversion

## Acknowledgements

Significant thanks to the [llama.cpp](https://github.com/ggml-org/llama.cpp) project for providing the GGUF conversion and inference tools. We also acknowledge the [ms-swift](https://github.com/modelscope/ms-swift) team for the fine-tuning framework and the Qwen team for their foundational models.

## License

This project is released under the **Apache License 2.0**.
- Free for commercial use, modification, and distribution
- Modified versions must indicate the changes made
- Derivative works must retain the original open-source notice

## Citation

If you find our work helpful, feel free to give us a cite.

```
@misc{confucius4,
  title        = {Confucius4: Advancing Multimodal Reasoning with Iterative SFT-RL Optimization and Compact Chain-of-Thought},
  author       = {NetEase Youdao AI Team},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/netease-youdao/Confucius4}}
}
```