---
license: other
license_name: tongyi-qianwen-research
license_link: https://huggingface.co/Qwen/Qwen1.5-4B/blob/main/LICENSE
language:
- en
pipeline_tag: text-generation
tags:
- pretrained
- openvino
- openvino-export
base_model: Qwen/Qwen1.5-4B
---

This model was converted to OpenVINO from [`Qwen/Qwen1.5-4B`](https://huggingface.co/Qwen/Qwen1.5-4B) using [optimum-intel](https://github.com/huggingface/optimum-intel)
via the [export](https://huggingface.co/spaces/echarlaix/openvino-export) space.


Install packages:

```bash
pip install optimum[openvino] transformers torch
```


Sample code:

```python
from optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer

model_id = "TheAverageDetective/Qwen1.5-4B-openvino"
model = OVModelForCausalLM.from_pretrained(model_id, device="GPU")
tokenizer = AutoTokenizer.from_pretrained(model_id)

prompt = "Explain the theory of relativity in simple terms."

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": prompt}
]

input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt")
output_ids = model.generate(**inputs, max_new_tokens=150)
result = tokenizer.batch_decode(output_ids, skip_special_tokens=True)[0]

print("\n", result)

```


Works on Intel Iris iGPU with 80EU and 16GB system RAM.