Image-Text-to-Text
Safetensors
English
Chinese
qwen3_5
unsloth
qwen
qwen3.5
reasoning
chain-of-thought
Dense
vLLM
SGLang
conversational
4-bit precision
gptq
Instructions to use Xingyu-Zheng/Qwopus3.5-9B-v3.5-INT4-FOEM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Local Apps Settings
- vLLM
How to use Xingyu-Zheng/Qwopus3.5-9B-v3.5-INT4-FOEM with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Xingyu-Zheng/Qwopus3.5-9B-v3.5-INT4-FOEM" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Xingyu-Zheng/Qwopus3.5-9B-v3.5-INT4-FOEM", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Xingyu-Zheng/Qwopus3.5-9B-v3.5-INT4-FOEM
- SGLang
How to use Xingyu-Zheng/Qwopus3.5-9B-v3.5-INT4-FOEM with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Xingyu-Zheng/Qwopus3.5-9B-v3.5-INT4-FOEM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Xingyu-Zheng/Qwopus3.5-9B-v3.5-INT4-FOEM", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Xingyu-Zheng/Qwopus3.5-9B-v3.5-INT4-FOEM" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Xingyu-Zheng/Qwopus3.5-9B-v3.5-INT4-FOEM", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Unsloth Desktop
- Docker Model Runner
How to use Xingyu-Zheng/Qwopus3.5-9B-v3.5-INT4-FOEM with Docker Model Runner:
docker model run hf.co/Xingyu-Zheng/Qwopus3.5-9B-v3.5-INT4-FOEM
Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,97 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
- zh
|
| 5 |
+
license: apache-2.0
|
| 6 |
+
base_model:
|
| 7 |
+
- Qwen/Qwen3.5-9B
|
| 8 |
+
- Jackrong/Qwopus3.5-9B-v3.5
|
| 9 |
+
tags:
|
| 10 |
+
- unsloth
|
| 11 |
+
- qwen
|
| 12 |
+
- qwen3.5
|
| 13 |
+
- reasoning
|
| 14 |
+
- chain-of-thought
|
| 15 |
+
- Dense
|
| 16 |
+
- vLLM
|
| 17 |
+
- SGLang
|
| 18 |
+
pipeline_tag: image-text-to-text
|
| 19 |
+
datasets:
|
| 20 |
+
- nohurry/Opus-4.6-Reasoning-3000x-filtered
|
| 21 |
+
---
|
| 22 |
+
# 🌟Qwopus3.5-9B-v3.5-INT4-FOEM
|
| 23 |
+
|
| 24 |
+
This is an unofficial quantized version of Qwopus3.5-9B-v3.5.
|
| 25 |
+
|
| 26 |
+
### 🧠 Quantization Framework
|
| 27 |
+
|
| 28 |
+
[GPTQModel](https://github.com/ModelCloud/GPTQModel)
|
| 29 |
+
|
| 30 |
+
## 🗺️ Quantization Method
|
| 31 |
+
|
| 32 |
+
[FOEM (AAAI 2026)](https://ojs.aaai.org/index.php/AAAI/article/view/40123)
|
| 33 |
+
|
| 34 |
+
FOEM is an improved quantization method over GPTQ. The resulting model preserves the same inference structure as GPTQ, ensuring compatibility with existing deployment pipelines while achieving better accuracy.
|
| 35 |
+
|
| 36 |
+
### 📚 Calibration Dataset
|
| 37 |
+
|
| 38 |
+
We randomly sampled 512 examples from [nohurry/Opus-4.6-Reasoning-3000x-filtered](https://huggingface.co/datasets/nohurry/Opus-4.6-Reasoning-3000x-filtered).
|
| 39 |
+
|
| 40 |
+
## 📋 Usage Example
|
| 41 |
+
|
| 42 |
+
This model can be deployed using standard frameworks such as **vLLM**, just like other GPTQModel-quantized models.
|
| 43 |
+
|
| 44 |
+
Example evaluation command:
|
| 45 |
+
|
| 46 |
+
```bash
|
| 47 |
+
lm-eval --model vllm --model_args pretrained=models/gptqmodel/Qwopus3.5-9B-v3.5-INT4-FOEM,tensor_parallel_size=1,gpu_memory_utilization=0.45 --tasks wikitext --batch_size 1
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+
## ⚠️ Limitations & Intended Use
|
| 51 |
+
|
| 52 |
+
*(Adapted from the original repository of Jackrong/Qwopus3.5-9B-v3.5)*
|
| 53 |
+
|
| 54 |
+
- Possible overfitting if scaling exceeds optimal regime
|
| 55 |
+
- Reasoning may still exhibit instability in edge cases
|
| 56 |
+
- Tool-calling performance depends on environment integration
|
| 57 |
+
- Not all capabilities are fully benchmarked yet
|
| 58 |
+
|
| 59 |
+
## 🙏 Acknowledgements
|
| 60 |
+
Special thanks to [Jackrong](https://huggingface.co/Jackrong) for providing the original model: [Qwopus3.5-9B-v3.5](https://huggingface.co/Jackrong/Qwopus3.5-9B-v3.5).
|
| 61 |
+
|
| 62 |
+
## 📖 Citation
|
| 63 |
+
|
| 64 |
+
If you use this model in your research or projects, please cite:
|
| 65 |
+
|
| 66 |
+
```bibtex
|
| 67 |
+
@misc{jackrong_qwopus35_9b_v35,
|
| 68 |
+
title = {Qwopus3.5-9B-v3.5},
|
| 69 |
+
author = {Jackrong},
|
| 70 |
+
year = {2026},
|
| 71 |
+
publisher = {Hugging Face}
|
| 72 |
+
}
|
| 73 |
+
```
|
| 74 |
+
|
| 75 |
+
```bibtex
|
| 76 |
+
@misc{qubitium2024gptqmodel,
|
| 77 |
+
author = {ModelCloud.ai and qubitium@modelcloud.ai},
|
| 78 |
+
title = {GPT-QModel},
|
| 79 |
+
publisher = {GitHub},
|
| 80 |
+
journal = {GitHub repository},
|
| 81 |
+
howpublished = {\url{https://github.com/modelcloud/gptqmodel}},
|
| 82 |
+
note = {Contact: qubitium@modelcloud.ai},
|
| 83 |
+
year = {2024},
|
| 84 |
+
}
|
| 85 |
+
```
|
| 86 |
+
|
| 87 |
+
```bibtex
|
| 88 |
+
@inproceedings{zheng2026first,
|
| 89 |
+
title={First-order error matters: Accurate compensation for quantized large language models},
|
| 90 |
+
author={Zheng, Xingyu and Qin, Haotong and Li, Yuye and Chu, Haoran and Wang, Jiakai and Guo, Jinyang and Magno, Michele and Liu, Xianglong},
|
| 91 |
+
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
|
| 92 |
+
volume={40},
|
| 93 |
+
number={34},
|
| 94 |
+
pages={28883--28891},
|
| 95 |
+
year={2026}
|
| 96 |
+
}
|
| 97 |
+
```
|