| --- |
| language: de |
| license: mit |
| tags: |
| - flair |
| - token-classification |
| - sequence-tagger-model |
| - hetzner |
| - hetzner-gex44 |
| - hetzner-gpu |
| base_model: google-bert/bert-base-german-cased |
| datasets: |
| - stefan-it/co-funer |
| widget: |
| - text: Wesentliche Tätigkeiten der Compliance-Funktion wurden an die Mercurtainment |
| AG , Düsseldorf , ausgelagert . |
| --- |
| |
| # Fine-tuned Flair Model on CO-Fun NER Dataset |
|
|
| This Flair model was fine-tuned on the |
| [CO-Fun](https://arxiv.org/abs/2403.15322) NER Dataset using German BERT as backbone LM. |
|
|
| ## Dataset |
|
|
| The [Company Outsourcing in Fund Prospectuses (CO-Fun) dataset](https://arxiv.org/abs/2403.15322) consists of |
| 948 sentences with 5,969 named entity annotations, including 2,340 Outsourced Services, 2,024 Companies, 1,594 Locations |
| and 11 Software annotations. |
|
|
| Overall, the following named entities are annotated: |
|
|
| * `Auslagerung` (engl. outsourcing) |
| * `Unternehmen` (engl. company) |
| * `Ort` (engl. location) |
| * `Software` |
|
|
| ## Fine-Tuning |
|
|
| The latest [Flair version](https://github.com/flairNLP/flair/tree/42ea3f6854eba04387c38045f160c18bdaac07dc) is used for |
| fine-tuning. |
|
|
| A hyper-parameter search over the following parameters with 5 different seeds per configuration is performed: |
|
|
| * Batch Sizes: [`16`, `8`] |
| * Learning Rates: [`3e-05`, `5e-05`] |
|
|
| More details can be found in this [repository](https://github.com/stefan-it/co-funer). All models are fine-tuned on a |
| [Hetzner GEX44](https://www.hetzner.com/dedicated-rootserver/matrix-gpu/) with an NVIDIA RTX 4000. |
|
|
| ## Results |
|
|
| A hyper-parameter search with 5 different seeds per configuration is performed and micro F1-score on development set |
| is reported: |
|
|
| | Configuration | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Average | |
| |--------------------|------------------|--------------|--------------|--------------|--------------|-----------------| |
| | `bs8-e10-lr5e-05` | [0.9346][1] | [0.9388][2] | [0.9301][3] | [0.9291][4] | [0.9346][5] | 0.9334 ± 0.0039 | |
| | `bs16-e10-lr5e-05` | [0.9316][6] | [0.9328][7] | [0.9341][8] | [0.9315][9] | [0.9248][10] | 0.931 ± 0.0036 | |
| | `bs8-e10-lr3e-05` | [**0.9234**][11] | [0.9391][12] | [0.9207][13] | [0.9191][14] | [0.9394][15] | 0.9283 ± 0.0101 | |
| | `bs16-e10-lr3e-05` | [0.9136][16] | [0.9269][17] | [0.9231][18] | [0.9251][19] | [0.9247][20] | 0.9227 ± 0.0053 | |
|
|
| [1]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs8-e10-lr5e-05-1 |
| [2]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs8-e10-lr5e-05-2 |
| [3]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs8-e10-lr5e-05-3 |
| [4]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs8-e10-lr5e-05-4 |
| [5]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs8-e10-lr5e-05-5 |
| [6]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs16-e10-lr5e-05-1 |
| [7]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs16-e10-lr5e-05-2 |
| [8]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs16-e10-lr5e-05-3 |
| [9]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs16-e10-lr5e-05-4 |
| [10]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs16-e10-lr5e-05-5 |
| [11]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs8-e10-lr3e-05-1 |
| [12]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs8-e10-lr3e-05-2 |
| [13]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs8-e10-lr3e-05-3 |
| [14]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs8-e10-lr3e-05-4 |
| [15]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs8-e10-lr3e-05-5 |
| [16]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs16-e10-lr3e-05-1 |
| [17]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs16-e10-lr3e-05-2 |
| [18]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs16-e10-lr3e-05-3 |
| [19]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs16-e10-lr3e-05-4 |
| [20]: https://hf.co/stefan-it/flair-co-funer-german_bert_base-bs16-e10-lr3e-05-5 |
|
|
| The result in bold shows the performance of the current viewed model. |
|
|
| Additionally, the Flair [training log](training.log) and [TensorBoard logs](../../tensorboard) are also uploaded to the model |
| hub. |
