| --- |
| license: apache-2.0 |
| tags: |
| - unsloth |
| datasets: |
| - 0xZee/UniversalScienceKownledge-finetome-top-20k |
| base_model: |
| - unsloth/Llama-3.2-3B-Instruct |
| --- |
| |
|
|
| --- |
|
|
| ## model-name: `llama3-2-3b-finetuned-ScienceKnowledge` |
| ## base_model: `unsloth/Llama-3.2-3B-Instruct` |
| |
| license: apache-2.0 |
| tags: |
| - unsloth |
| - science |
| - knowledge |
| - fine-tuned |
| - causal-lm |
| |
| ## datasets: |
| - `0xZee/UniversalScienceKownledge-finetome-top-20k` |
| |
| language: |
| - en |
| library_name: transformers |
| inference: true |
| --- |
|
|
| # llama3-2-3b-finetuned-ScienceKnowledge |
|
|
| This model is a fine-tuned version of unsloth/Llama-3.2-3B-Instruct, specifically adapted for enhanced performance in scientific knowledge tasks. |
|
|
| ## Model Details |
|
|
| - **Base Model:** [unsloth/Llama-3.2-3B-Instruct](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct) |
| - **Fine-tuning Dataset:** [0xZee/UniversalScienceKownledge-finetome-top-20k](https://huggingface.co/datasets/0xZee/UniversalScienceKownledge-finetome-top-20k) |
| - **Model Type:** Causal Language Model |
| - **License:** Apache 2.0 |
| - **Finetuning Technique:** The model was likely fine-tuned using techniques such as LoRA or QLoRA, which are efficient for smaller datasets. |
|
|
| ## Intended Use |
|
|
| This model is designed for tasks requiring scientific understanding, question answering, and knowledge retrieval in scientific contexts. It can be used for: |
|
|
| - Assisting researchers with scientific queries |
| - Educational tools in science domains |
| - Automated scientific knowledge discovery and summarization |
|
|
| ## Training Procedure |
|
|
| The model was fine-tuned on the UniversalScienceKownledge-finetome-top-20k dataset. Specific hyperparameters used in training are not provided, but typical fine-tuning approaches for LLMs include: |
|
|
| - Learning rate: Often around 2e-5 for Mistral-based models |
| - LoRA rank: Likely between 512 and 768 |
| - Batch size: Small, possibly between 1 and 4 |
|
|
| ## Limitations |
|
|
| - Performance is dependent on the quality and scope of the training data. |
| - May not generalize well to scientific domains outside of its training data. |
| - Potential for generating plausible-sounding but incorrect information (hallucinations). |
|
|
| ## Bias, Risks, and Limitations |
|
|
| As with all language models, this model may reflect biases present in its training data. Users should be aware of potential biases in scientific literature and exercise caution when using the model for sensitive applications. |
|
|
| ## Evaluation Results |
|
|
| Quantitative evaluation results are not provided. Users are encouraged to evaluate the model's performance on their specific tasks and domains of interest. |
|
|
| ## How to Use |
|
|
| You can use this model with the Transformers library: |
|
|
| ```py |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_name = "0xZee/llama3-2-3b-finetuned-ScienceKnowledge" |
| tokenizer = AutoTokenizer.from_pretrained(model_name) |
| model = AutoModelForCausalLM.from_pretrained(model_name) |
| |
| prompt = "Explain the concept of quantum entanglement." |
| input_ids = tokenizer(prompt, return_tensors="pt").input_ids |
| output = model.generate(input_ids, max_new_tokens=100) |
| print(tokenizer.decode(output, skip_special_tokens=True)) |
| ``` |
|
|
|
|
