---
license: apache-2.0
tags:
- mlabonne/Marcoro14-7B-slerp
- dpo
- rlhf
datasets:
- mlabonne/chatml_dpo_pairs
---

![](https://i.imgur.com/FSKtmRc.png)

# NeuralMarcoro14-7B

This is a DPO fine-tune version of [mlabonne/Marcoro14-7B-slerp](https://huggingface.co/mlabonne/Marcoro14-7B-slerp). It improves the performance of the model on Nous benchmark suite (waiting for the results on the Open LLM Benchmark).

## 🏆 Evaluation

|          Model          |AGIEval|GPT4ALL|TruthfulQA|Bigbench|Average|
|-------------------------|------:|------:|---------:|-------:|------:|
|[Marcoro14-7B-slerp](https://huggingface.co/mlabonne/Marcoro14-7B-slerp)       |  44.66|  76.24|     64.15|   45.64|  57.67|
|[NeuralMarcoro14-7B](https://huggingface.co/mlabonne/NeuralMarcoro14-7B)|  44.59|  76.17|     65.94|    46.9|   58.4|
|Change                   |  -0.07|  -0.07|    +1.79|   +1.26|   +0.73|

## 🧩 Training hyperparameters

**LoRA**:
* r=16
* lora_alpha=16
* lora_dropout=0.05
* bias="none"
* task_type="CAUSAL_LM"
* target_modules=['k_proj', 'gate_proj', 'v_proj', 'up_proj', 'q_proj', 'o_proj', 'down_proj']

**Training arguments**:
* per_device_train_batch_size=4
* gradient_accumulation_steps=4
* gradient_checkpointing=True
* learning_rate=5e-5
* lr_scheduler_type="cosine"
* max_steps=200
* optim="paged_adamw_32bit"
* warmup_steps=100

**DPOTrainer**:
* beta=0.1
* max_prompt_length=1024
* max_length=1536

## 💻 Usage

```python
!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "mlabonne/NeuralMarcoro14-7B"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
```

Output:

> Large Language Models (LLMs) are advanced artificial intelligence systems designed to process and generate human language. They are trained on vast amounts of text data to understand context, grammar, vocabulary, and various linguistic patterns. These models can perform tasks such as translation, summarization, text completion, question answering, and more, mimicking human-like language capabilities. Examples of well-known LLMs include GPT-3 by OpenAI and BERT by Google. Their size, measured in billions of parameters, allows them to achieve impressive results in natural language understanding and generation.