metadata
language: en
license: apache-2.0
library_name: transformers
tags:
- commonsense-reasoning
- winoGrande
- fine-tuned
- llama
- reasoning
datasets:
- allenai/winogrande
metrics:
- accuracy
- loss
base_model:
- PleIAs/Monad
Model Details
Model Description
The model has been trained on the WinoGrande dataset which tests the ability to resolve pronouns and make logical inferences in everyday scenarios.
Model Sources
- Base Model: https://huggingface.co/PleIAs/Monad
Training Data
Dataset: WinoGrande (allenai/winogrande)
- Size: 9,248 training examples, 1,267 validation examples
- Task: Commonsense reasoning with pronoun resolution
- Format: Multiple choice questions requiring logical reasoning
Training Hyperparameters
| Epochs | Batch Size | Learning Rate | Warmup Ratio | Warmup Steps | Weight Decay | Max Gradient Norm | Evaluation Steps | Save Steps | Early Stopping Patience |
|---|---|---|---|---|---|---|---|---|---|
| 5 | 16 | 1e-05 | 0.05 | 144 | 0.01 | 1.0 | 150 | 150 | 7 |
Training Results
| Metric | Value |
|---|---|
| Final Training Loss | 0.9143 |
| Training Time | 1,526.9s |
Validation Performance
Validation loss stabilized between 0.83-0.86 throughout the training
Summary
The model achieved strong convergence during training:
- Final training loss: 0.9143
- Evaluation loss: ~0.834 (final checkpoint)
- Training completed: All 5 epochs with early stopping monitoring
Compute Infrastructure
Hardware
- GPU: Single NVIDIA A10G (24GB VRAM)
- Platform: Modal.com