Text Classification
Transformers
PyTorch
ONNX
English
distilbert
text-classfication
int8
neural-compressor
Intel® Neural Compressor
PostTrainingStatic
Eval Results (legacy)
text-embeddings-inference
Instructions to use Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc") model = AutoModelForSequenceClassification.from_pretrained("Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -17,7 +17,7 @@ metrics:
|
|
| 17 |
|
| 18 |
This is an INT8 PyTorch model quantified with [intel/nlp-toolkit](https://github.com/intel/nlp-toolkit) using provider: [Intel® Neural Compressor](https://github.com/intel/neural-compressor).
|
| 19 |
|
| 20 |
-
The original fp32 model comes from the fine-tuned model [distilbert-base-uncased-finetuned-sst-2-english](https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english)
|
| 21 |
|
| 22 |
The calibration dataloader is the train dataloader. The default calibration sampling size 100 isn't divisible exactly by batch size 8, so
|
| 23 |
the real sampling size is 104.
|
|
@@ -33,13 +33,14 @@ The calibration dataloader is the train dataloader. The default calibration samp
|
|
| 33 |
| **Accuracy (eval-accuracy)** |0.9037|0.9106|
|
| 34 |
| **Model size (MB)** |65|255|
|
| 35 |
|
| 36 |
-
|
| 37 |
### Load with nlp-toolkit:
|
|
|
|
| 38 |
```python
|
| 39 |
from nlp_toolkit import OptimizedModel
|
| 40 |
int8_model = OptimizedModel.from_pretrained(
|
| 41 |
'Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static',
|
| 42 |
)
|
| 43 |
```
|
|
|
|
| 44 |
Notes:
|
| 45 |
- The INT8 model has better performance than the FP32 model when the CPU is fully occupied. Otherwise, there will be the illusion that INT8 is inferior to FP32.
|
|
|
|
| 17 |
|
| 18 |
This is an INT8 PyTorch model quantified with [intel/nlp-toolkit](https://github.com/intel/nlp-toolkit) using provider: [Intel® Neural Compressor](https://github.com/intel/neural-compressor).
|
| 19 |
|
| 20 |
+
The original fp32 model comes from the fine-tuned model [distilbert-base-uncased-finetuned-sst-2-english](https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english).
|
| 21 |
|
| 22 |
The calibration dataloader is the train dataloader. The default calibration sampling size 100 isn't divisible exactly by batch size 8, so
|
| 23 |
the real sampling size is 104.
|
|
|
|
| 33 |
| **Accuracy (eval-accuracy)** |0.9037|0.9106|
|
| 34 |
| **Model size (MB)** |65|255|
|
| 35 |
|
|
|
|
| 36 |
### Load with nlp-toolkit:
|
| 37 |
+
|
| 38 |
```python
|
| 39 |
from nlp_toolkit import OptimizedModel
|
| 40 |
int8_model = OptimizedModel.from_pretrained(
|
| 41 |
'Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static',
|
| 42 |
)
|
| 43 |
```
|
| 44 |
+
|
| 45 |
Notes:
|
| 46 |
- The INT8 model has better performance than the FP32 model when the CPU is fully occupied. Otherwise, there will be the illusion that INT8 is inferior to FP32.
|