chiaraboretti nielsr HF Staff commited on
Commit
c83b62f
·
1 Parent(s): f2a52dc

Add pipeline_tag and arxiv ID, fix usage snippet (#1)

Browse files

- Add pipeline_tag and arxiv ID, fix usage snippet (1f08de3fbf006eb7e29f41749622ad21fc13f4d6)


Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +8 -4
README.md CHANGED
@@ -1,5 +1,6 @@
1
  ---
2
- language:
 
3
  - en
4
  license: apache-2.0
5
  tags:
@@ -11,8 +12,9 @@ tags:
11
  - qwen
12
  - llm
13
  - compression
14
- base_model: Qwen/Qwen3-32B
15
  base_model_relation: quantized
 
 
16
  ---
17
 
18
  <p align="center">
@@ -48,7 +50,7 @@ To support the project please put a star ⭐ in the official [SINQ](https://gith
48
 
49
  ---
50
 
51
- # 🚀 Usage</span>
52
 
53
  ## Prerequisite
54
  Before running the quantization script, make sure the **SINQ** library is installed.
@@ -58,6 +60,7 @@ Installation instructions and setup details are available in the [SINQ official
58
  You can load and use the model with our wrapper based on the 🤗 Transformers library:
59
 
60
  ```python
 
61
  from transformers import AutoTokenizer
62
  from sinq.patch_model import AutoSINQHFModel
63
 
@@ -83,13 +86,14 @@ print(tokenizer.decode(out_ids[0], skip_special_tokens=True))
83
  The quantized model was obtained using the **SINQ** quantization library, following the steps below:
84
 
85
  ```python
 
86
  from transformers import AutoModelForCausalLM, AutoTokenizer
87
  from sinq.patch_model import AutoSINQHFModel
88
  from sinq.sinqlinear import BaseQuantizeConfig
89
 
90
  # Load base model
91
  base_model_name = "Qwen/Qwen3-32B"
92
- model = AutoModelForCausalLM.from_pretrained(base_model_name, torch_dtype="float16")
93
  tokenizer = AutoTokenizer.from_pretrained(base_model_name)
94
 
95
  # Apply 4-bit SINQ quantization
 
1
  ---
2
+ base_model: Qwen/Qwen3-32B
3
+ language:
4
  - en
5
  license: apache-2.0
6
  tags:
 
12
  - qwen
13
  - llm
14
  - compression
 
15
  base_model_relation: quantized
16
+ pipeline_tag: text-generation
17
+ arxiv: 2509.22944
18
  ---
19
 
20
  <p align="center">
 
50
 
51
  ---
52
 
53
+ # 🚀 Usage
54
 
55
  ## Prerequisite
56
  Before running the quantization script, make sure the **SINQ** library is installed.
 
60
  You can load and use the model with our wrapper based on the 🤗 Transformers library:
61
 
62
  ```python
63
+ import torch
64
  from transformers import AutoTokenizer
65
  from sinq.patch_model import AutoSINQHFModel
66
 
 
86
  The quantized model was obtained using the **SINQ** quantization library, following the steps below:
87
 
88
  ```python
89
+ import torch
90
  from transformers import AutoModelForCausalLM, AutoTokenizer
91
  from sinq.patch_model import AutoSINQHFModel
92
  from sinq.sinqlinear import BaseQuantizeConfig
93
 
94
  # Load base model
95
  base_model_name = "Qwen/Qwen3-32B"
96
+ model = AutoModelForCausalLM.from_pretrained(base_model_name, torch_dtype=torch.float16)
97
  tokenizer = AutoTokenizer.from_pretrained(base_model_name)
98
 
99
  # Apply 4-bit SINQ quantization