---
language:
- tr
- en
pipeline_tag: text-generation
license: apache-2.0
---
<img src="https://huggingface.co/Trendyol/Trendyol-LLM-7b-chat-v0.1/resolve/main/llama-tr-image.jpeg"
alt="drawing" width="400"/>
# **Trendyol LLM GGUF Version**
Trendyol LLM is a generative model that is based on LLaMa2 7B model. This is the repository for the quantized chat model.

**Developer** Umar Igan
GGUF Version Created using following notebook: https://github.com/mlabonne/llm-course/blob/main/Quantize_Llama_2_models_using_GGUF_and_llama_cpp.ipynb

**Variations** Q5_K_M and Q4_K_M variations of GGUF.


**Input** Models input text only.

**Output** Models generate text only.

**Model Architecture** Trendyol LLM is an auto-regressive language model (based on LLaMa2 7b) that uses an optimized transformer architecture. The chat version is fine-tuned on 180K instruction sets with the following trainables by using LoRA
This is a quantized model of Trendyol LLM:


<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/peft/lora_diagram.png"
alt="drawing" width="600"/>

## Usage

```python
from llama_cpp import Llama
from ctransformers import AutoModelForCausalLM

# Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
llm_p = AutoModelForCausalLM.from_pretrained("umarigan/Trendyol-LLM-7b-chat-v0.1-GGUF",
                                           model_file="trendyol-llm-7b-chat-v0.1.Q4_K_M.gguf",
                                           model_type="llama",
                                           gpu_layers=0)

# Chat Completion API
llm = Llama(model_path=llm_p.model_path, 
                        chat_format="llama-2")  # Set chat_format according to the model you are using
llm.create_chat_completion(
    messages = [
        {"role": "system", "content": "çocuk hikayeleri yazan bir yazarsın"},
        {
            "role": "user",
            "content": "köpekler hakkında bir çocuk hikayesi yaz"
        }
    ]
)
```

Output:

```
{'id': 'chatcmpl-0d665fb2-a92a-408c-bc03-78c32bccab0d',
 'object': 'chat.completion',
 'created': 1707822047,
 'model': '/root/.cache/huggingface/hub/models--umarigan--Trendyol-LLM-7b-chat-v0.1-GGUF/blobs/323878a8570093178040e78b438d5670c0fdae2aa614a8ed58e784d697d4db52',
 'choices': [{'index': 0,
   'message': {'role': 'assistant',
    'content': '  Bir zamanlar, ormanda yaşayan cesur ve sadık bir köpek varmış. O, her zaman arkadaşlarına yardım etmeye hazırdı ve asla korkmuyordu. Bir gün, ormanın derinliklerinde gizemli bir ses duydu ve araştırmaya karar verdi. Yol boyunca birçok yaratıkla karşılaştı ama hiçbirinin kimliğini bilmiyordu. Sonunda, gizemli sesin geldiği yere ulaştı ve sonunda onu buldu.'},
   'finish_reason': 'stop'}],
 'usage': {'prompt_tokens': 39, 'completion_tokens': 85, 'total_tokens': 124}}
```