Update README.md

README.md

@@ -8,6 +8,8 @@ base_model:
 - vesteinn/DanskBERT
 pipeline_tag: fill-mask
 library_name: transformers
+metrics:
+- perplexity
 ---
 # Model Card for Model ID
 DA-Bert_Old_News_V3 is the third version of a transformer trained on Danish historical texts from the period during Danish Absolutism (1660-1849). It is developed by researchers at Aalborg University. The aim of the model is to create a domain-specific model to capture meaning from texts that are far enough removed in time that they no longer read like contemporary Danish.
@@ -101,7 +103,7 @@ generator(
     paa en Tid af 11 Dage gjort, fra den 14 til d. 25. April Syv andre fremmede Skibe
     som tillige med dem vare den indtrofne, have ogsaa gjort god Reyse."
 )
-
+```
 
 ## Training Details
 
@@ -119,6 +121,29 @@ Trained on the
 
 
 #### Training Hyperparameters
+   **Training parameters:**
+    eval_strategy="steps",
+    overwrite_output_dir=False,
+    per_device_train_batch_size=16, # Training batch size
+    gradient_accumulation_steps=4, # Accumulating the gradients before updating the weights
+    per_device_eval_batch_size=32, # Evaluating batch size
+    logging_steps=500,
+    learning_rate=5e-5,
+    save_steps=10000,
+    max_steps=120000,
+    save_total_limit=10,
+    load_best_model_at_end=True,
+    metric_for_best_model="eval_loss",
+    greater_is_better=False,
+    fp16=torch.cuda.is_available(),
+    warmup_ratio=0.06,
+    weight_decay=0.01,
+    lr_scheduler_type="cosine",
+    dataloader_num_workers=4,
+    dataloader_pin_memory=True,
+    save_on_each_node=False,
+    ddp_find_unused_parameters=False,
+    optim="adamw_torch",
 
 - **Training regime:** fp16 mixed precision was used <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
 
@@ -205,4 +230,4 @@ Trained on the
 ## Model Card Contact
 - [CALDISS](www.caldiss.aau.dk)
 - Matias Appel (mkap@adm.aau.dk)
-- Johan Heinsen (heinsen@dps.aau.dk)
+- Johan Heinsen (heinsen@dps.aau.dk)