PEFT
Safetensors
Kalenjin
English
Swahili
lora
qwen2
kalenjin
continued-pretraining
low-resource-language
Instructions to use Sugutt/Kalenjin-qwen2.5-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Sugutt/Kalenjin-qwen2.5-7B with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Upload Kalenjin CPT LoRA adapter
Browse files
README.md
CHANGED
|
@@ -21,6 +21,16 @@ LoRA adapter from continued pretraining for a Kalenjin-focused language model.
|
|
| 21 |
The adapter was trained on Modal from `Qwen/Qwen2.5-7B` and is intended as the CPT
|
| 22 |
checkpoint before later SFT, DPO, and GRPO stages.
|
| 23 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
## Contents
|
| 25 |
|
| 26 |
- `adapter/`: PEFT LoRA adapter weights and config
|
|
|
|
| 21 |
The adapter was trained on Modal from `Qwen/Qwen2.5-7B` and is intended as the CPT
|
| 22 |
checkpoint before later SFT, DPO, and GRPO stages.
|
| 23 |
|
| 24 |
+
## Model Scope
|
| 25 |
+
|
| 26 |
+
- Base model: `Qwen/Qwen2.5-7B`
|
| 27 |
+
- Method: QLoRA Continued Pretraining
|
| 28 |
+
- Language/domain: Kalenjin-heavy text
|
| 29 |
+
- Adapter only: yes
|
| 30 |
+
- Training objective: causal language modeling
|
| 31 |
+
- Not instruction-tuned yet
|
| 32 |
+
- Not a translator yet
|
| 33 |
+
|
| 34 |
## Contents
|
| 35 |
|
| 36 |
- `adapter/`: PEFT LoRA adapter weights and config
|