Add pipeline tag and link paper to metadata

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +8 -4
README.md CHANGED
@@ -1,4 +1,5 @@
1
  ---
 
2
  language:
3
  - en
4
  license: apache-2.0
@@ -7,12 +8,12 @@ tags:
7
  - sinq
8
  - int3
9
  - efficient-inference
10
- - text-generation
11
  - qwen
12
  - llm
13
  - compression
14
- base_model: Qwen/Qwen3-32B
15
  base_model_relation: quantized
 
 
16
  ---
17
 
18
  <p align="center">
@@ -27,6 +28,8 @@ base_model_relation: quantized
27
  This repository contains the official **3-bit quantized** version of the [`Qwen3-32B`](https://huggingface.co/Qwen/Qwen3-32B) model using the *calibrated* version of **SINQ (Sinkhorn-Normalized Quantization)** method.
28
  SINQ is a novel, fast and high-quality quantization method designed to make any Large Language Models smaller while keeping their accuracy almost intact.
29
 
 
 
30
  To support the project please put a star ⭐ in the official [SINQ](https://github.com/huawei-csl/SINQ) github repository.
31
 
32
  ## Model Details
@@ -48,7 +51,7 @@ To support the project please put a star ⭐ in the official [SINQ](https://gith
48
 
49
  ---
50
 
51
- # 🚀 Usage</span>
52
 
53
  ## Prerequisite
54
  Before running the quantization script, make sure the **SINQ** library is installed.
@@ -60,6 +63,7 @@ You can load and use the model with our wrapper based on the 🤗 Transformers l
60
  ```python
61
  from transformers import AutoTokenizer
62
  from sinq.patch_model import AutoSINQHFModel
 
63
 
64
  model_name = "huawei-csl/Qwen3-32B-3bit-ASINQ"
65
  tokenizer = AutoTokenizer.from_pretrained(model_name)
@@ -74,7 +78,6 @@ inputs = tokenizer(prompt, return_tensors="pt").to("cuda:0")
74
  with torch.inference_mode():
75
  out_ids = sinq_model.generate(**inputs, max_new_tokens=32, do_sample=False)
76
  print(tokenizer.decode(out_ids[0], skip_special_tokens=True))
77
-
78
  ```
79
 
80
  <details>
@@ -86,6 +89,7 @@ The quantized model was obtained using the **SINQ** quantization library, follow
86
  from transformers import AutoModelForCausalLM, AutoTokenizer
87
  from sinq.patch_model import AutoSINQHFModel
88
  from sinq.sinqlinear import BaseQuantizeConfig
 
89
 
90
  # Load base model
91
  base_model_name = "Qwen/Qwen3-32B"
 
1
  ---
2
+ base_model: Qwen/Qwen3-32B
3
  language:
4
  - en
5
  license: apache-2.0
 
8
  - sinq
9
  - int3
10
  - efficient-inference
 
11
  - qwen
12
  - llm
13
  - compression
 
14
  base_model_relation: quantized
15
+ pipeline_tag: text-generation
16
+ arxiv: 2509.22944
17
  ---
18
 
19
  <p align="center">
 
28
  This repository contains the official **3-bit quantized** version of the [`Qwen3-32B`](https://huggingface.co/Qwen/Qwen3-32B) model using the *calibrated* version of **SINQ (Sinkhorn-Normalized Quantization)** method.
29
  SINQ is a novel, fast and high-quality quantization method designed to make any Large Language Models smaller while keeping their accuracy almost intact.
30
 
31
+ The model was presented in the paper [SINQ: Sinkhorn-Normalized Quantization for Calibration-Free Low-Precision LLM Weights](https://huggingface.co/papers/2509.22944).
32
+
33
  To support the project please put a star ⭐ in the official [SINQ](https://github.com/huawei-csl/SINQ) github repository.
34
 
35
  ## Model Details
 
51
 
52
  ---
53
 
54
+ # 🚀 Usage
55
 
56
  ## Prerequisite
57
  Before running the quantization script, make sure the **SINQ** library is installed.
 
63
  ```python
64
  from transformers import AutoTokenizer
65
  from sinq.patch_model import AutoSINQHFModel
66
+ import torch
67
 
68
  model_name = "huawei-csl/Qwen3-32B-3bit-ASINQ"
69
  tokenizer = AutoTokenizer.from_pretrained(model_name)
 
78
  with torch.inference_mode():
79
  out_ids = sinq_model.generate(**inputs, max_new_tokens=32, do_sample=False)
80
  print(tokenizer.decode(out_ids[0], skip_special_tokens=True))
 
81
  ```
82
 
83
  <details>
 
89
  from transformers import AutoModelForCausalLM, AutoTokenizer
90
  from sinq.patch_model import AutoSINQHFModel
91
  from sinq.sinqlinear import BaseQuantizeConfig
92
+ import torch
93
 
94
  # Load base model
95
  base_model_name = "Qwen/Qwen3-32B"