Hugging Face's logo

dsfsi
/
afriquellama_8b-lora-r4-hau-eng

Translation
PEFT
Safetensors
Hausa
English
african-languages
scientific-translation
afriscience-mt
lora
llama
Eval Results (legacy)
Model card Files
xet
 Community
abumafrim commited on
Commit
8eec625
·
verified ·
1 Parent(s): e7f82a3

Upload afriquellama_8b-lora-r4-hau-eng LoRA adapter

Browse files
Files changed (3) hide show
  1. README.md +38 -0
  2. adapter_config.json +5 -5
  3. adapter_model.safetensors +1 -1
README.md CHANGED
@@ -14,6 +14,15 @@ tags:
14
  - llama
15
  license: apache-2.0
16
  pipeline_tag: translation
 
 
 
 
 
 
 
 
 
17
  ---
18
 
19
  # afriquellama_8b-lora-r4-hau-eng
@@ -37,6 +46,19 @@ This is a **LoRA adapter** for the AfriScience-MT project, enabling efficient sc
37
 
38
  LoRA (Low-Rank Adaptation) enables efficient fine-tuning by training only a small number of additional parameters. This adapter adds only **~2.0M parameters** to the base model while achieving strong translation performance.
39
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
  ## Usage
41
 
42
  ### Quick Start
@@ -111,6 +133,22 @@ base_model = AutoModelForCausalLM.from_pretrained(
111
  model = PeftModel.from_pretrained(base_model, "AfriScience-MT/afriquellama_8b-lora-r4-hau-eng")
112
  ```
113
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
114
  ### Hardware Requirements
115
 
116
  | Configuration | VRAM Required |
 
14
  - llama
15
  license: apache-2.0
16
  pipeline_tag: translation
17
+ model-index:
18
+ - name: afriquellama_8b-lora-r4-hau-eng
19
+ results:
20
+ - task:
21
+ type: translation
22
+ metrics:
23
+ - name: SSA-COMET (test)
24
+ type: comet
25
+ value: 65.66
26
  ---
27
 
28
  # afriquellama_8b-lora-r4-hau-eng
 
46
 
47
  LoRA (Low-Rank Adaptation) enables efficient fine-tuning by training only a small number of additional parameters. This adapter adds only **~2.0M parameters** to the base model while achieving strong translation performance.
48
 
49
+ ## Evaluation Results
50
+
51
+ Performance on the AfriScience-MT test set:
52
+
53
+ | Split | BLEU | chrF | SSA-COMET |
54
+ |-------|------|------|-----------|
55
+ | **Test** | **-** | **-** | **65.66** |
56
+
57
+ **Metrics explanation:**
58
+ - **BLEU**: Measures n-gram overlap with reference translations (0-100, higher is better)
59
+ - **chrF**: Character-level F-score, robust for morphologically rich languages (0-100, higher is better)
60
+ - **SSA-COMET**: Neural metric trained for Sub-Saharan African languages, shown as percentage (0-100, higher is better) ([McGill-NLP/ssa-comet-stl](https://huggingface.co/McGill-NLP/ssa-comet-stl))
61
+
62
  ## Usage
63
 
64
  ### Quick Start
 
133
  model = PeftModel.from_pretrained(base_model, "AfriScience-MT/afriquellama_8b-lora-r4-hau-eng")
134
  ```
135
 
136
+ ## Training Details
137
+
138
+ ### Hyperparameters
139
+
140
+ | Parameter | Value |
141
+ |-----------|-------|
142
+ | LoRA Rank (r) | 4 |
143
+ | LoRA Alpha | 8 |
144
+ | LoRA Dropout | 0.05 |
145
+ | Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
146
+ | Epochs | 3 |
147
+ | Batch Size | 2 |
148
+ | Learning Rate | 2e-04 |
149
+ | Max Sequence Length | 512 |
150
+ | Gradient Accumulation | 4 |
151
+
152
  ### Hardware Requirements
153
 
154
  | Configuration | VRAM Required |
adapter_config.json CHANGED
@@ -29,13 +29,13 @@
29
  "rank_pattern": {},
30
  "revision": null,
31
  "target_modules": [
32
- "v_proj",
33
- "o_proj",
34
  "q_proj",
 
 
35
  "up_proj",
36
- "down_proj",
37
- "k_proj",
38
- "gate_proj"
39
  ],
40
  "target_parameters": null,
41
  "task_type": "CAUSAL_LM",
 
29
  "rank_pattern": {},
30
  "revision": null,
31
  "target_modules": [
32
+ "gate_proj",
33
+ "down_proj",
34
  "q_proj",
35
+ "o_proj",
36
+ "v_proj",
37
  "up_proj",
38
+ "k_proj"
 
 
39
  ],
40
  "target_parameters": null,
41
  "task_type": "CAUSAL_LM",
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2c1b9bbb56c6b2505509eb0fc24fbd27f11efdf4c1881d6430db6d439446de8d
3
  size 42002136
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:85fdcbe096537f19801ca05aade0533eb4908b38cba9875044b097ab93f28ac8
3
  size 42002136