Pankaj8922 commited on
Commit
2301245
·
verified ·
1 Parent(s): 05b0237

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +51 -49
README.md CHANGED
@@ -1,68 +1,70 @@
1
  ---
2
- library_name: transformers
3
- license: mit
4
- base_model: prajjwal1/bert-medium
5
  tags:
6
- - generated_from_trainer
 
 
 
 
 
7
  metrics:
8
  - accuracy
 
9
  model-index:
10
  - name: bert-medium-nli
11
- results: []
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  ---
13
 
14
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
- should probably proofread and complete it, then remove this comment. -->
16
-
17
  # bert-medium-nli
18
 
19
- This model is a fine-tuned version of [prajjwal1/bert-medium](https://huggingface.co/prajjwal1/bert-medium) on an unknown dataset.
20
- It achieves the following results on the evaluation set:
21
- - Loss: 0.4368
22
- - Accuracy: 0.8389
23
- - F1 Macro: 0.8389
24
- - Precision Macro: 0.8389
25
- - Recall Macro: 0.8389
26
-
27
- ## Model description
28
-
29
- More information needed
30
-
31
- ## Intended uses & limitations
32
-
33
- More information needed
34
-
35
- ## Training and evaluation data
36
 
37
- More information needed
 
 
 
38
 
39
- ## Training procedure
40
 
41
- ### Training hyperparameters
 
 
 
42
 
43
- The following hyperparameters were used during training:
44
- - learning_rate: 5e-05
45
- - train_batch_size: 64
46
- - eval_batch_size: 128
47
- - seed: 42
48
- - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
49
- - lr_scheduler_type: linear
50
- - lr_scheduler_warmup_steps: 0.06
51
- - num_epochs: 3
52
- - mixed_precision_training: Native AMP
53
 
54
- ### Training results
 
 
 
 
55
 
56
- | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 Macro | Precision Macro | Recall Macro |
57
- |:-------------:|:-----:|:-----:|:---------------:|:--------:|:--------:|:---------------:|:------------:|
58
- | 0.5086 | 1.0 | 9979 | 0.4701 | 0.8077 | 0.8065 | 0.8122 | 0.8077 |
59
- | 0.3832 | 2.0 | 19958 | 0.4225 | 0.8324 | 0.8323 | 0.8329 | 0.8324 |
60
- | 0.2974 | 3.0 | 29937 | 0.4368 | 0.8389 | 0.8389 | 0.8389 | 0.8389 |
61
 
 
 
 
62
 
63
- ### Framework versions
64
 
65
- - Transformers 5.0.0
66
- - Pytorch 2.10.0+cu128
67
- - Datasets 5.0.0
68
- - Tokenizers 0.22.2
 
1
  ---
2
+ language: en
3
+ license: apache-2.0
 
4
  tags:
5
+ - text-classification
6
+ - zero-shot-classification
7
+ - nli
8
+ datasets:
9
+ - Pankaj8922/nli-high-quality-balanced
10
+ base_model: prajjwal1/bert-medium
11
  metrics:
12
  - accuracy
13
+ - f1
14
  model-index:
15
  - name: bert-medium-nli
16
+ results:
17
+ - task:
18
+ type: text-classification
19
+ name: Natural Language Inference
20
+ dataset:
21
+ name: Pankaj8922/nli-high-quality-balanced
22
+ type: Pankaj8922/nli-high-quality-balanced
23
+ metrics:
24
+ - type: accuracy
25
+ value: 0.8373
26
+ name: Test Accuracy
27
+ - type: f1
28
+ value: 0.8372
29
+ name: Test F1 (macro)
30
  ---
31
 
 
 
 
32
  # bert-medium-nli
33
 
34
+ Fine-tuned [`prajjwal1/bert-medium`](https://huggingface.co/prajjwal1/bert-medium) for natural language
35
+ inference (entailment / neutral / contradiction), intended for use as a zero-shot
36
+ text classification model via the entailment trick (hypothesis = "This text is
37
+ about {label}.").
 
 
 
 
 
 
 
 
 
 
 
 
 
38
 
39
+ Trained on [`Pankaj8922/nli-high-quality-balanced`](https://huggingface.co/datasets/Pankaj8922/nli-high-quality-balanced), a
40
+ combined and filtered subset of MNLI, SNLI, FEVER-NLI, and ANLI:
41
+ annotator-agreement filtered, deduplicated, teacher-confidence filtered,
42
+ hypothesis-only artifact filtered, and class-balanced.
43
 
44
+ ## Results
45
 
46
+ | Split | Accuracy | F1 (macro) | Precision (macro) | Recall (macro) |
47
+ |---|---|---|---|---|
48
+ | Validation | 0.8389 | 0.8389 | 0.8389 | 0.8389 |
49
+ | Test | 0.8373 | 0.8372 | 0.8372 | 0.8373 |
50
 
51
+ ## Training details
 
 
 
 
 
 
 
 
 
52
 
53
+ - Base model: `prajjwal1/bert-medium`
54
+ - Epochs: 3
55
+ - Batch size: 64 (train), 128 (eval)
56
+ - Learning rate: 5e-05
57
+ - Max sequence length: 256
58
 
59
+ ## Labels
 
 
 
 
60
 
61
+ - 0: entailment
62
+ - 1: neutral
63
+ - 2: contradiction
64
 
65
+ ## Intended use / limitations
66
 
67
+ This is a small (~41M parameter) model, so its ceiling on zero-shot performance
68
+ against novel, unseen label sets is lower than larger NLI-tuned checkpoints
69
+ (e.g. DeBERTa-v3-base or -large variants). Best suited for fast inference or
70
+ resource-constrained settings rather than maximum accuracy.