Instructions to use gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt")
model = AutoModelForCausalLM.from_pretrained("gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt

SGLang

How to use gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt with Docker Model Runner:
```
docker model run hf.co/gavinds/DeepSeek-R1-Distill-Qwen-1.5B-expt
```

DeepSeek-R1-Distill-Qwen-1.5B-expt

Commit History

Delete adapter_model.safetensors

b9a9ebe
verified

Gavin Dsouza commited on Mar 3, 2025

Delete adapter_config.json

abbd9c9
verified

Gavin Dsouza commited on Mar 3, 2025

Uploading model and tokenizer

571d94d
verified

Gavin Dsouza commited on Mar 3, 2025

Uploading model and tokenizer

71c81bb
verified

Gavin Dsouza commited on Feb 28, 2025

Uploading model and tokenizer

c8213b0
verified

Gavin Dsouza commited on Feb 28, 2025

initial commit

9b99601
verified

Gavin Dsouza commited on Feb 28, 2025

Commit History

Delete adapter_model.safetensors b9a9ebe verified

Delete adapter_config.json abbd9c9 verified

Uploading model and tokenizer 571d94d verified

Uploading model and tokenizer 71c81bb verified

Uploading model and tokenizer c8213b0 verified

initial commit 9b99601 verified

Delete adapter_model.safetensors

b9a9ebe
verified

Delete adapter_config.json

abbd9c9
verified

Uploading model and tokenizer

571d94d
verified

Uploading model and tokenizer

71c81bb
verified

Uploading model and tokenizer

c8213b0
verified

initial commit

9b99601
verified