FayElhassan commited on
Commit
155df97
·
verified ·
1 Parent(s): 991b988

Model save

Browse files
Files changed (1) hide show
  1. README.md +47 -39
README.md CHANGED
@@ -1,58 +1,66 @@
1
  ---
 
 
2
  base_model: deepseek-ai/deepseek-llm-7b-chat
3
- library_name: transformers
4
- model_name: deepseek_Pregnancy
5
  tags:
6
- - generated_from_trainer
7
- - trl
8
- - sft
9
- licence: license
 
 
 
10
  ---
11
 
12
- # Model Card for deepseek_Pregnancy
 
13
 
14
- This model is a fine-tuned version of [deepseek-ai/deepseek-llm-7b-chat](https://huggingface.co/deepseek-ai/deepseek-llm-7b-chat).
15
- It has been trained using [TRL](https://github.com/huggingface/trl).
16
 
17
- ## Quick start
 
 
18
 
19
- ```python
20
- from transformers import pipeline
21
 
22
- question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
23
- generator = pipeline("text-generation", model="FayElhassan/deepseek_Pregnancy", device="cuda")
24
- output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
25
- print(output["generated_text"])
26
- ```
27
 
28
- ## Training procedure
29
 
30
- [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/elhassan-aimsammi/medical-llm-pregnancy/runs/am3sv4ry)
31
 
 
32
 
33
- This model was trained with SFT.
34
 
35
- ### Framework versions
36
 
37
- - TRL: 0.26.0
38
- - Transformers: 4.56.1
39
- - Pytorch: 2.8.0
40
- - Datasets: 4.1.0
41
- - Tokenizers: 0.22.0
42
 
43
- ## Citations
 
 
 
 
 
 
 
 
 
 
 
44
 
 
45
 
 
 
 
 
 
 
46
 
47
- Cite TRL as:
48
-
49
- ```bibtex
50
- @misc{vonwerra2022trl,
51
- title = {{TRL: Transformer Reinforcement Learning}},
52
- author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
53
- year = 2020,
54
- journal = {GitHub repository},
55
- publisher = {GitHub},
56
- howpublished = {\url{https://github.com/huggingface/trl}}
57
- }
58
- ```
 
1
  ---
2
+ library_name: peft
3
+ license: other
4
  base_model: deepseek-ai/deepseek-llm-7b-chat
 
 
5
  tags:
6
+ - base_model:adapter:deepseek-ai/deepseek-llm-7b-chat
7
+ - lora
8
+ - transformers
9
+ pipeline_tag: text-generation
10
+ model-index:
11
+ - name: deepseek_Pregnancy
12
+ results: []
13
  ---
14
 
15
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
+ should probably proofread and complete it, then remove this comment. -->
17
 
18
+ # deepseek_Pregnancy
 
19
 
20
+ This model is a fine-tuned version of [deepseek-ai/deepseek-llm-7b-chat](https://huggingface.co/deepseek-ai/deepseek-llm-7b-chat) on an unknown dataset.
21
+ It achieves the following results on the evaluation set:
22
+ - Loss: 1.4711
23
 
24
+ ## Model description
 
25
 
26
+ More information needed
 
 
 
 
27
 
28
+ ## Intended uses & limitations
29
 
30
+ More information needed
31
 
32
+ ## Training and evaluation data
33
 
34
+ More information needed
35
 
36
+ ## Training procedure
37
 
38
+ ### Training hyperparameters
 
 
 
 
39
 
40
+ The following hyperparameters were used during training:
41
+ - learning_rate: 0.0002
42
+ - train_batch_size: 1
43
+ - eval_batch_size: 1
44
+ - seed: 42
45
+ - gradient_accumulation_steps: 8
46
+ - total_train_batch_size: 8
47
+ - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
48
+ - lr_scheduler_type: linear
49
+ - lr_scheduler_warmup_ratio: 0.03
50
+ - num_epochs: 1.0
51
+ - mixed_precision_training: Native AMP
52
 
53
+ ### Training results
54
 
55
+ | Training Loss | Epoch | Step | Validation Loss |
56
+ |:-------------:|:------:|:----:|:---------------:|
57
+ | 1.5499 | 0.5398 | 200 | 1.4711 |
58
+
59
+
60
+ ### Framework versions
61
 
62
+ - PEFT 0.17.1
63
+ - Transformers 4.56.1
64
+ - Pytorch 2.8.0+cu128
65
+ - Datasets 4.1.0
66
+ - Tokenizers 0.22.0