Instructions to use dphn/fc-dolphin-2.6-mistral-7b-dpo-laser with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use dphn/fc-dolphin-2.6-mistral-7b-dpo-laser with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="dphn/fc-dolphin-2.6-mistral-7b-dpo-laser")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("dphn/fc-dolphin-2.6-mistral-7b-dpo-laser")
model = AutoModelForCausalLM.from_pretrained("dphn/fc-dolphin-2.6-mistral-7b-dpo-laser", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use dphn/fc-dolphin-2.6-mistral-7b-dpo-laser with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "dphn/fc-dolphin-2.6-mistral-7b-dpo-laser"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dphn/fc-dolphin-2.6-mistral-7b-dpo-laser",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/dphn/fc-dolphin-2.6-mistral-7b-dpo-laser

SGLang

How to use dphn/fc-dolphin-2.6-mistral-7b-dpo-laser with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "dphn/fc-dolphin-2.6-mistral-7b-dpo-laser" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dphn/fc-dolphin-2.6-mistral-7b-dpo-laser",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "dphn/fc-dolphin-2.6-mistral-7b-dpo-laser" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dphn/fc-dolphin-2.6-mistral-7b-dpo-laser",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use dphn/fc-dolphin-2.6-mistral-7b-dpo-laser with Docker Model Runner:
```
docker model run hf.co/dphn/fc-dolphin-2.6-mistral-7b-dpo-laser
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

by David, Fernando and Eric

Sponsored by: VAGO Solutions and HyperSpace.Ai

Discord: https://discord.gg/cognitivecomputations

A function calling version of cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser

It follows the implementation of laserRMT @ https://github.com/cognitivecomputations/laserRMT and the novel training technique - we partially freeze the model according to a laser-like analysis (Official Paper soon) which effectively prevents the significant problem of language models forgetting previously acquired knowledge. This aspect is particularly crucial when attempting to teach the model specific skills, such as function calling.

We intend to be the first of a family of experimentations being carried out @ Cognitive Computations.

Usage

This model is trained on a variaton of glaive function calling v2 and uses their format for declaring functions and invoking them. Note that like other Dolphin models you should be using the ChatML syntax for representing chats.

Set your system prompt to something along these lines:

You are a helpful assistant with access to the following functions.
Use them if required -
{ "name": "search_wikipedia", "description": "Search for a given keyword or phrase on Wikipedia", "parameters": { "type": "object", "properties": { "query": { "type": "string", "description": "The keyword or phrase to search" } }, "required": [ "query" ] } }

The model will then attempt to invoke the specified functions using the token <functioncall> and some JSON with a name and an arguments parameter.

e.g.:

ASSISTANT: <functioncall> {"name": "search_wikipedia", "arguments": '{"query": "Eiffel Tower"}'} <|endoftext|>

Your code should detect this and stop execution and invoke the function call and inject a response back into the context with the following syntax:

FUNCTION RESPONSE: {"result": "The Eiffel Tower is a wrought-iron lattice... etc"}

You should then prompt the assistant to generate another response.

Note that the model may not always attempt to invoke a function and may respond saying it doesn't have access to the right functionality.

Quants

Downloads last month: 28

Model tree for dphn/fc-dolphin-2.6-mistral-7b-dpo-laser

Merges

3 models

Quantizations

3 models