Instructions to use TokenBender/pic_7B_mistral_Full_v0.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TokenBender/pic_7B_mistral_Full_v0.2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TokenBender/pic_7B_mistral_Full_v0.2")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TokenBender/pic_7B_mistral_Full_v0.2")
model = AutoModelForCausalLM.from_pretrained("TokenBender/pic_7B_mistral_Full_v0.2", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TokenBender/pic_7B_mistral_Full_v0.2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TokenBender/pic_7B_mistral_Full_v0.2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TokenBender/pic_7B_mistral_Full_v0.2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/TokenBender/pic_7B_mistral_Full_v0.2

SGLang

How to use TokenBender/pic_7B_mistral_Full_v0.2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TokenBender/pic_7B_mistral_Full_v0.2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TokenBender/pic_7B_mistral_Full_v0.2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TokenBender/pic_7B_mistral_Full_v0.2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TokenBender/pic_7B_mistral_Full_v0.2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use TokenBender/pic_7B_mistral_Full_v0.2 with Docker Model Runner:
```
docker model run hf.co/TokenBender/pic_7B_mistral_Full_v0.2
```

pic_7B_mistral_Full_v0.2 / README.md

TokenBender

Update README.md

1300a03 almost 3 years ago

preview code

Raw History Blame Contribute Delete

1.8 kB

	---
	license: apache-2.0
	base_model: mistralai/Mistral-7B-v0.1
	datasets:
	- Open-Orca/SlimOrca
	- HuggingFaceH4/no_robots
	- Intel/orca_dpo_pairs
	- rizerphe/glaive-function-calling-v2-zephyr
	- codefuse-ai/Evol-instruction-66k
	library_name: transformers
	pipeline_tag: text-generation
	---

	# pic_7B_mistral_Full_v0.2
	PIC_7B_Mistral (First phase)

	This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1)
	A curated, decontaminated subset of datasets used have been mentioned in the model card.
	All used datasets are public as of the time of release of this model.

	Collaborate or Consult me - [Twitter](https://twitter.com/4evaBehindSOTA), [Discord](https://discord.gg/ftEM63pzs2)

	Recommended format is ChatML, Alpaca will work but take care of EOT token

	#### Chat Model Inference


	## Model description

	First generic model of Project PIC (Partner-in-Crime) in 7B range.
	Trying a bunch of things and seeing what sticks right now.

	Empathy + Coder + Instruction/json/function adherence is my game.
	Finding lots of challenges and insights in this effort, patience is key.
	![image/png](https://cdn-uploads.huggingface.co/production/uploads/6398bf222da24ee95b51c8d8/Fl-pRCsC_lvnuoP734hsJ.png)

	## Intended uses & limitations

	Should be useful in generic capacity.
	Demonstrates little bit of everything.

	Basic tests in -
	Roleplay: Adherence to character present.
	json/function-calling: Passing
	Coding: To be evaluated


	## Training procedure
	SFT + DPO

	### Training results
	Humaneval and evalplus results to be shared as well.
	![image/png](https://cdn-uploads.huggingface.co/production/uploads/6398bf222da24ee95b51c8d8/Ysat2Hi3MInSKSlUrft-i.png)

	### Framework versions

	- Transformers 4.35.2
	- Pytorch 2.0.1
	- Datasets 2.15.0
	- Tokenizers 0.15.0