Hugging Face's logo

Paladiso
/
bb73a8ba-f2ad-4dad-a9e7-46da888461bc

PEFT
Safetensors
llama
axolotl
Generated from Trainer
Model card Files
xet
 Community
Paladiso commited on
Commit
0412ae9
·
verified ·
1 Parent(s): 014c295

End of training

Browse files
Files changed (1) hide show
  1. README.md +18 -19
README.md CHANGED
@@ -1,14 +1,13 @@
1
  ---
2
  library_name: peft
3
- license: apache-2.0
4
- base_model: echarlaix/tiny-random-mistral
5
  tags:
6
  - axolotl
7
  - generated_from_trainer
8
  datasets:
9
- - Paladiso/dataset_c2294ea4-f15b-4e48-866d-856c7fe6510b
10
  model-index:
11
- - name: ed4076c9-be95-413f-bc19-d413d184c1db
12
  results: []
13
  ---
14
 
@@ -21,18 +20,18 @@ should probably proofread and complete it, then remove this comment. -->
21
  axolotl version: `0.6.0`
22
  ```yaml
23
  adapter: lora
24
- base_model: echarlaix/tiny-random-mistral
25
  bf16: auto
26
  chat_template: llama3
27
  dataset_prepared_path: /workspace/axolotl/data/prepared
28
  datasets:
29
  - ds_type: json
30
  format: custom
31
- path: Paladiso/dataset_c2294ea4-f15b-4e48-866d-856c7fe6510b
32
  type:
33
- field_input: answer
34
- field_instruction: problem
35
- field_output: generated_solution
36
  system_format: '{system}'
37
  system_prompt: ''
38
  debug: null
@@ -48,7 +47,7 @@ fsdp_config: null
48
  gradient_accumulation_steps: 4
49
  gradient_checkpointing: false
50
  group_by_length: false
51
- hub_model_id: Paladiso/ed4076c9-be95-413f-bc19-d413d184c1db
52
  hub_private_repo: true
53
  hub_repo: null
54
  hub_strategy: checkpoint
@@ -80,7 +79,7 @@ save_safetensors: true
80
  saves_per_epoch: 4
81
  sequence_len: 512
82
  special_tokens:
83
- pad_token: </s>
84
  strict: false
85
  tf32: false
86
  tokenizer_type: AutoTokenizer
@@ -90,10 +89,10 @@ use_accelerate: true
90
  val_set_size: 0.05
91
  wandb_entity: null
92
  wandb_mode: online
93
- wandb_name: c2294ea4-f15b-4e48-866d-856c7fe6510b
94
  wandb_project: Gradients-On-Demand
95
  wandb_run: your_name
96
- wandb_runid: c2294ea4-f15b-4e48-866d-856c7fe6510b
97
  warmup_steps: 10
98
  weight_decay: 0.0
99
  xformers_attention: null
@@ -102,11 +101,11 @@ xformers_attention: null
102
 
103
  </details><br>
104
 
105
- # ed4076c9-be95-413f-bc19-d413d184c1db
106
 
107
- This model is a fine-tuned version of [echarlaix/tiny-random-mistral](https://huggingface.co/echarlaix/tiny-random-mistral) on the Paladiso/dataset_c2294ea4-f15b-4e48-866d-856c7fe6510b dataset.
108
  It achieves the following results on the evaluation set:
109
- - Loss: 10.3779
110
 
111
  ## Model description
112
 
@@ -140,9 +139,9 @@ The following hyperparameters were used during training:
140
 
141
  | Training Loss | Epoch | Step | Validation Loss |
142
  |:-------------:|:------:|:----:|:---------------:|
143
- | 41.5243 | 0.0026 | 3 | 10.3795 |
144
- | 41.527 | 0.0052 | 6 | 10.3789 |
145
- | 41.5317 | 0.0077 | 9 | 10.3779 |
146
 
147
 
148
  ### Framework versions
 
1
  ---
2
  library_name: peft
3
+ base_model: samoline/d06bfecd-ea82-4f80-8b9f-98631e6886ba
 
4
  tags:
5
  - axolotl
6
  - generated_from_trainer
7
  datasets:
8
+ - Paladiso/dataset_ba5916f9-8367-4661-bc10-d45098e30844
9
  model-index:
10
+ - name: bb73a8ba-f2ad-4dad-a9e7-46da888461bc
11
  results: []
12
  ---
13
 
 
20
  axolotl version: `0.6.0`
21
  ```yaml
22
  adapter: lora
23
+ base_model: samoline/d06bfecd-ea82-4f80-8b9f-98631e6886ba
24
  bf16: auto
25
  chat_template: llama3
26
  dataset_prepared_path: /workspace/axolotl/data/prepared
27
  datasets:
28
  - ds_type: json
29
  format: custom
30
+ path: Paladiso/dataset_ba5916f9-8367-4661-bc10-d45098e30844
31
  type:
32
+ field_input: input
33
+ field_instruction: instruction
34
+ field_output: output
35
  system_format: '{system}'
36
  system_prompt: ''
37
  debug: null
 
47
  gradient_accumulation_steps: 4
48
  gradient_checkpointing: false
49
  group_by_length: false
50
+ hub_model_id: Paladiso/bb73a8ba-f2ad-4dad-a9e7-46da888461bc
51
  hub_private_repo: true
52
  hub_repo: null
53
  hub_strategy: checkpoint
 
79
  saves_per_epoch: 4
80
  sequence_len: 512
81
  special_tokens:
82
+ pad_token: <|endoftext|>
83
  strict: false
84
  tf32: false
85
  tokenizer_type: AutoTokenizer
 
89
  val_set_size: 0.05
90
  wandb_entity: null
91
  wandb_mode: online
92
+ wandb_name: ba5916f9-8367-4661-bc10-d45098e30844
93
  wandb_project: Gradients-On-Demand
94
  wandb_run: your_name
95
+ wandb_runid: ba5916f9-8367-4661-bc10-d45098e30844
96
  warmup_steps: 10
97
  weight_decay: 0.0
98
  xformers_attention: null
 
101
 
102
  </details><br>
103
 
104
+ # bb73a8ba-f2ad-4dad-a9e7-46da888461bc
105
 
106
+ This model is a fine-tuned version of [samoline/d06bfecd-ea82-4f80-8b9f-98631e6886ba](https://huggingface.co/samoline/d06bfecd-ea82-4f80-8b9f-98631e6886ba) on the Paladiso/dataset_ba5916f9-8367-4661-bc10-d45098e30844 dataset.
107
  It achieves the following results on the evaluation set:
108
+ - Loss: 1.1327
109
 
110
  ## Model description
111
 
 
139
 
140
  | Training Loss | Epoch | Step | Validation Loss |
141
  |:-------------:|:------:|:----:|:---------------:|
142
+ | 1.1401 | 0.0006 | 3 | 1.1926 |
143
+ | 1.0962 | 0.0012 | 6 | 1.1539 |
144
+ | 0.9537 | 0.0018 | 9 | 1.1327 |
145
 
146
 
147
  ### Framework versions