Instructions to use artificialguybr/Gemma2-2B-OpenHermes2.5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use artificialguybr/Gemma2-2B-OpenHermes2.5 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="artificialguybr/Gemma2-2B-OpenHermes2.5")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("artificialguybr/Gemma2-2B-OpenHermes2.5")
model = AutoModelForCausalLM.from_pretrained("artificialguybr/Gemma2-2B-OpenHermes2.5", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use artificialguybr/Gemma2-2B-OpenHermes2.5 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "artificialguybr/Gemma2-2B-OpenHermes2.5"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "artificialguybr/Gemma2-2B-OpenHermes2.5",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/artificialguybr/Gemma2-2B-OpenHermes2.5

SGLang

How to use artificialguybr/Gemma2-2B-OpenHermes2.5 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "artificialguybr/Gemma2-2B-OpenHermes2.5" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "artificialguybr/Gemma2-2B-OpenHermes2.5",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "artificialguybr/Gemma2-2B-OpenHermes2.5" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "artificialguybr/Gemma2-2B-OpenHermes2.5",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use artificialguybr/Gemma2-2B-OpenHermes2.5 with Docker Model Runner:
```
docker model run hf.co/artificialguybr/Gemma2-2B-OpenHermes2.5
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Model Card for GEMMA2-2B-openhermes-2.5

This model is a fine-tuned version of Gemma 2 -2B on the OpenHermes-2.5 dataset.

Model Details

Model Description

This is a fine-tuned version of the google/gemma-2-2b model, trained on the OpenHermes-2.5 dataset. It is designed for instruction following and general language tasks.

Developed by: artificialguybr
Model type: Causal Language Model
Language(s): English
License: apache-2.0
Finetuned from model: google/gemma-2-2b

🌐 Website

You can find more of my models, projects, and information on my official website:

artificialguy.com

🚀 Prompt Hub

Need high-quality prompts for image models and LLMs? Explore findgoodprompt.com.

💖 Support My Work

If you find this model useful, please consider supporting my work. It helps me cover server costs and dedicate more time to new open-source projects.

Patreon: Support on Patreon
Ko-fi: Buy me a Ko-fi
Buy Me a Coffee: Buy me a Coffee

Model Sources

Repository: https://huggingface.co/artificialguybr/Gemma2-2B-OpenHermes2.5

Uses

This model can be used for various natural language processing tasks, particularly those involving instruction following and general language understanding.

Direct Use

The model can be used for tasks such as text generation, question answering, and other language-related applications.

Out-of-Scope Use

The model should not be used for generating harmful or biased content. Users should be aware of potential biases in the training data.

Training Details

Training Data

The model was fine-tuned on the teknium/OpenHermes-2.5 dataset.

Training Procedure

Hardware and Software

Hardware: NVIDIA A100-SXM4-80GB (1 GPU)
Software Framework: 🤗 Transformers, Axolotl

Limitations and Biases

More information is needed about specific limitations and biases of this model.

Explore More

Discover more of my models, projects, and practical AI resources:

AI tools directory: JustGoodAI — a curated directory of useful AI tools.

Downloads last month: 19

Safetensors

Model size

3B params

Tensor type

BF16

Model tree for artificialguybr/Gemma2-2B-OpenHermes2.5

Quantizations

3 models

artificialguybr
/

Gemma2-2B-OpenHermes2.5