Hugging Face's logo

jaggernaut007
/
roberta-base-finetuned-abbr-finetuned-ner

Token Classification
Transformers
Safetensors
roberta
Generated from Trainer
Model card Files
xet
 Community
jaggernaut007 commited on
Commit
b05594f
·
verified ·
1 Parent(s): e44ed6b

End of training

Browse files
Files changed (4) hide show
  1. README.md +26 -14
  2. config.json +9 -11
  3. model.safetensors +2 -2
  4. training_args.bin +2 -2
README.md CHANGED
@@ -20,11 +20,11 @@ should probably proofread and complete it, then remove this comment. -->
20
 
21
  This model is a fine-tuned version of [surrey-nlp/roberta-base-finetuned-abbr](https://huggingface.co/surrey-nlp/roberta-base-finetuned-abbr) on an unknown dataset.
22
  It achieves the following results on the evaluation set:
23
- - Loss: 0.1117
24
- - Precision: 0.9671
25
- - Recall: 0.9678
26
- - F1: 0.9674
27
- - Accuracy: 0.9619
28
 
29
  ## Model description
30
 
@@ -43,24 +43,36 @@ More information needed
43
  ### Training hyperparameters
44
 
45
  The following hyperparameters were used during training:
46
- - learning_rate: 2e-05
47
- - train_batch_size: 8
48
  - eval_batch_size: 4
49
  - seed: 42
50
- - gradient_accumulation_steps: 4
51
- - total_train_batch_size: 32
52
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
53
  - lr_scheduler_type: linear
54
- - num_epochs: 6
55
- - mixed_precision_training: Native AMP
56
 
57
  ### Training results
58
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
59
 
60
 
61
  ### Framework versions
62
 
63
- - Transformers 4.37.2
64
  - Pytorch 2.2.2+cu121
65
- - Datasets 2.18.0
66
- - Tokenizers 0.15.1
 
20
 
21
  This model is a fine-tuned version of [surrey-nlp/roberta-base-finetuned-abbr](https://huggingface.co/surrey-nlp/roberta-base-finetuned-abbr) on an unknown dataset.
22
  It achieves the following results on the evaluation set:
23
+ - Loss: 0.1103
24
+ - Precision: 0.9654
25
+ - Recall: 0.9669
26
+ - F1: 0.9662
27
+ - Accuracy: 0.9597
28
 
29
  ## Model description
30
 
 
43
  ### Training hyperparameters
44
 
45
  The following hyperparameters were used during training:
46
+ - learning_rate: 2e-06
47
+ - train_batch_size: 4
48
  - eval_batch_size: 4
49
  - seed: 42
 
 
50
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
51
  - lr_scheduler_type: linear
52
+ - num_epochs: 10
 
53
 
54
  ### Training results
55
 
56
+ | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
57
+ |:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
58
+ | No log | 0.37 | 100 | 0.2049 | 0.9532 | 0.9608 | 0.9570 | 0.9520 |
59
+ | No log | 0.75 | 200 | 0.1281 | 0.9677 | 0.9659 | 0.9668 | 0.9625 |
60
+ | No log | 1.12 | 300 | 0.1220 | 0.9654 | 0.9667 | 0.9661 | 0.9603 |
61
+ | No log | 1.49 | 400 | 0.1170 | 0.9640 | 0.9649 | 0.9645 | 0.9589 |
62
+ | 0.2501 | 1.87 | 500 | 0.1098 | 0.9691 | 0.9672 | 0.9681 | 0.9638 |
63
+ | 0.2501 | 2.24 | 600 | 0.1212 | 0.9643 | 0.9662 | 0.9652 | 0.9592 |
64
+ | 0.2501 | 2.61 | 700 | 0.1258 | 0.9608 | 0.9645 | 0.9626 | 0.9561 |
65
+ | 0.2501 | 2.99 | 800 | 0.1103 | 0.9654 | 0.9669 | 0.9662 | 0.9597 |
66
+ | 0.2501 | 3.36 | 900 | 0.1137 | 0.9639 | 0.9662 | 0.9651 | 0.9586 |
67
+ | 0.1119 | 3.73 | 1000 | 0.1125 | 0.9677 | 0.9674 | 0.9676 | 0.9622 |
68
+ | 0.1119 | 4.1 | 1100 | 0.1203 | 0.9621 | 0.9659 | 0.9640 | 0.9581 |
69
+ | 0.1119 | 4.48 | 1200 | 0.1154 | 0.9664 | 0.9659 | 0.9661 | 0.9608 |
70
+ | 0.1119 | 4.85 | 1300 | 0.1174 | 0.9664 | 0.9659 | 0.9661 | 0.9611 |
71
 
72
 
73
  ### Framework versions
74
 
75
+ - Transformers 4.39.3
76
  - Pytorch 2.2.2+cu121
77
+ - Datasets 2.19.0
78
+ - Tokenizers 0.15.2
config.json CHANGED
@@ -11,20 +11,18 @@
11
  "hidden_dropout_prob": 0.1,
12
  "hidden_size": 768,
13
  "id2label": {
14
- "0": "O",
15
- "1": "B-AC",
16
- "2": "I-AC",
17
- "3": "B-LF",
18
- "4": "I-LF"
19
  },
20
  "initializer_range": 0.02,
21
  "intermediate_size": 3072,
22
  "label2id": {
23
- "B-AC": 1,
24
- "B-LF": 3,
25
- "I-AC": 2,
26
- "I-LF": 4,
27
- "O": 0
28
  },
29
  "layer_norm_eps": 1e-05,
30
  "max_position_embeddings": 514,
@@ -34,7 +32,7 @@
34
  "pad_token_id": 1,
35
  "position_embedding_type": "absolute",
36
  "torch_dtype": "float32",
37
- "transformers_version": "4.37.2",
38
  "type_vocab_size": 1,
39
  "use_cache": true,
40
  "vocab_size": 50265
 
11
  "hidden_dropout_prob": 0.1,
12
  "hidden_size": 768,
13
  "id2label": {
14
+ "0": "LABEL_0",
15
+ "1": "LABEL_1",
16
+ "2": "LABEL_2",
17
+ "3": "LABEL_3"
 
18
  },
19
  "initializer_range": 0.02,
20
  "intermediate_size": 3072,
21
  "label2id": {
22
+ "LABEL_0": 0,
23
+ "LABEL_1": 1,
24
+ "LABEL_2": 2,
25
+ "LABEL_3": 3
 
26
  },
27
  "layer_norm_eps": 1e-05,
28
  "max_position_embeddings": 514,
 
32
  "pad_token_id": 1,
33
  "position_embedding_type": "absolute",
34
  "torch_dtype": "float32",
35
+ "transformers_version": "4.39.3",
36
  "type_vocab_size": 1,
37
  "use_cache": true,
38
  "vocab_size": 50265
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b043ce967597c658bfeea3cb888a6bec20d03755056845b5b835acefb52b2360
3
- size 496259468
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6249f39f6f11b8b1fd46d485adc65e297beb2c60da03cba05585beaaa68cc8f
3
+ size 496256392
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:98a0b7a639a9af7a15d4b431bdf1e55ca4e2b61add9e2e9fda3c26f20efc635b
3
- size 4728
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4cdd36fc71113cf9f1c9de3b8b14bc9da47c56089acce8e1d12798ac3b912576
3
+ size 4984