Xingyu-Zheng commited on
Commit
3bc1db7
·
verified ·
1 Parent(s): da99708

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +97 -0
README.md ADDED
@@ -0,0 +1,97 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - zh
5
+ license: apache-2.0
6
+ base_model:
7
+ - Qwen/Qwen3.5-9B
8
+ - Jackrong/Qwopus3.5-9B-v3.5
9
+ tags:
10
+ - unsloth
11
+ - qwen
12
+ - qwen3.5
13
+ - reasoning
14
+ - chain-of-thought
15
+ - Dense
16
+ - vLLM
17
+ - SGLang
18
+ pipeline_tag: image-text-to-text
19
+ datasets:
20
+ - nohurry/Opus-4.6-Reasoning-3000x-filtered
21
+ ---
22
+ # 🌟Qwopus3.5-9B-v3.5-INT4-FOEM
23
+
24
+ This is an unofficial quantized version of Qwopus3.5-9B-v3.5.
25
+
26
+ ### 🧠 Quantization Framework
27
+
28
+ [GPTQModel](https://github.com/ModelCloud/GPTQModel)
29
+
30
+ ## 🗺️ Quantization Method
31
+
32
+ [FOEM (AAAI 2026)](https://ojs.aaai.org/index.php/AAAI/article/view/40123)
33
+
34
+ FOEM is an improved quantization method over GPTQ. The resulting model preserves the same inference structure as GPTQ, ensuring compatibility with existing deployment pipelines while achieving better accuracy.
35
+
36
+ ### 📚 Calibration Dataset
37
+
38
+ We randomly sampled 512 examples from [nohurry/Opus-4.6-Reasoning-3000x-filtered](https://huggingface.co/datasets/nohurry/Opus-4.6-Reasoning-3000x-filtered).
39
+
40
+ ## 📋 Usage Example
41
+
42
+ This model can be deployed using standard frameworks such as **vLLM**, just like other GPTQModel-quantized models.
43
+
44
+ Example evaluation command:
45
+
46
+ ```bash
47
+ lm-eval --model vllm --model_args pretrained=models/gptqmodel/Qwopus3.5-9B-v3.5-INT4-FOEM,tensor_parallel_size=1,gpu_memory_utilization=0.45 --tasks wikitext --batch_size 1
48
+ ```
49
+
50
+ ## ⚠️ Limitations & Intended Use
51
+
52
+ *(Adapted from the original repository of Jackrong/Qwopus3.5-9B-v3.5)*
53
+
54
+ - Possible overfitting if scaling exceeds optimal regime
55
+ - Reasoning may still exhibit instability in edge cases
56
+ - Tool-calling performance depends on environment integration
57
+ - Not all capabilities are fully benchmarked yet
58
+
59
+ ## 🙏 Acknowledgements
60
+ Special thanks to [Jackrong](https://huggingface.co/Jackrong) for providing the original model: [Qwopus3.5-9B-v3.5](https://huggingface.co/Jackrong/Qwopus3.5-9B-v3.5).
61
+
62
+ ## 📖 Citation
63
+
64
+ If you use this model in your research or projects, please cite:
65
+
66
+ ```bibtex
67
+ @misc{jackrong_qwopus35_9b_v35,
68
+ title = {Qwopus3.5-9B-v3.5},
69
+ author = {Jackrong},
70
+ year = {2026},
71
+ publisher = {Hugging Face}
72
+ }
73
+ ```
74
+
75
+ ```bibtex
76
+ @misc{qubitium2024gptqmodel,
77
+ author = {ModelCloud.ai and qubitium@modelcloud.ai},
78
+ title = {GPT-QModel},
79
+ publisher = {GitHub},
80
+ journal = {GitHub repository},
81
+ howpublished = {\url{https://github.com/modelcloud/gptqmodel}},
82
+ note = {Contact: qubitium@modelcloud.ai},
83
+ year = {2024},
84
+ }
85
+ ```
86
+
87
+ ```bibtex
88
+ @inproceedings{zheng2026first,
89
+ title={First-order error matters: Accurate compensation for quantized large language models},
90
+ author={Zheng, Xingyu and Qin, Haotong and Li, Yuye and Chu, Haoran and Wang, Jiakai and Guo, Jinyang and Magno, Michele and Liu, Xianglong},
91
+ booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
92
+ volume={40},
93
+ number={34},
94
+ pages={28883--28891},
95
+ year={2026}
96
+ }
97
+ ```