Instructions to use deepseek-ai/DeepSeek-V3.1-Terminus with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deepseek-ai/DeepSeek-V3.1-Terminus with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="deepseek-ai/DeepSeek-V3.1-Terminus", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V3.1-Terminus", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-V3.1-Terminus", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use deepseek-ai/DeepSeek-V3.1-Terminus with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deepseek-ai/DeepSeek-V3.1-Terminus"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V3.1-Terminus",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/deepseek-ai/DeepSeek-V3.1-Terminus

SGLang

How to use deepseek-ai/DeepSeek-V3.1-Terminus with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "deepseek-ai/DeepSeek-V3.1-Terminus" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V3.1-Terminus",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "deepseek-ai/DeepSeek-V3.1-Terminus" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V3.1-Terminus",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use deepseek-ai/DeepSeek-V3.1-Terminus with Docker Model Runner:
```
docker model run hf.co/deepseek-ai/DeepSeek-V3.1-Terminus
```

msr2000 commited on Sep 22, 2025

Commit

308f6d2

verified ·

1 Parent(s): 846b34e

Add files using upload-large-folder tool

Browse files

Files changed (50) hide show

model-00036-of-000163.safetensors +3 -0
model-00037-of-000163.safetensors +3 -0
model-00038-of-000163.safetensors +3 -0
model-00039-of-000163.safetensors +3 -0
model-00040-of-000163.safetensors +3 -0
model-00041-of-000163.safetensors +3 -0
model-00042-of-000163.safetensors +3 -0
model-00043-of-000163.safetensors +3 -0
model-00044-of-000163.safetensors +3 -0
model-00045-of-000163.safetensors +3 -0
model-00046-of-000163.safetensors +3 -0
model-00047-of-000163.safetensors +3 -0
model-00048-of-000163.safetensors +3 -0
model-00049-of-000163.safetensors +3 -0
model-00050-of-000163.safetensors +3 -0
model-00051-of-000163.safetensors +3 -0
model-00052-of-000163.safetensors +3 -0
model-00053-of-000163.safetensors +3 -0
model-00054-of-000163.safetensors +3 -0
model-00055-of-000163.safetensors +3 -0
model-00056-of-000163.safetensors +3 -0
model-00057-of-000163.safetensors +3 -0
model-00058-of-000163.safetensors +3 -0
model-00059-of-000163.safetensors +3 -0
model-00060-of-000163.safetensors +3 -0
model-00061-of-000163.safetensors +3 -0
model-00062-of-000163.safetensors +3 -0
model-00063-of-000163.safetensors +3 -0
model-00064-of-000163.safetensors +3 -0
model-00065-of-000163.safetensors +3 -0
model-00066-of-000163.safetensors +3 -0
model-00067-of-000163.safetensors +3 -0
model-00068-of-000163.safetensors +3 -0
model-00069-of-000163.safetensors +3 -0
model-00070-of-000163.safetensors +3 -0
model-00071-of-000163.safetensors +3 -0
model-00072-of-000163.safetensors +3 -0
model-00073-of-000163.safetensors +3 -0
model-00074-of-000163.safetensors +3 -0
model-00075-of-000163.safetensors +3 -0
model-00076-of-000163.safetensors +3 -0
model-00077-of-000163.safetensors +3 -0
model-00078-of-000163.safetensors +3 -0
model-00079-of-000163.safetensors +3 -0
model-00080-of-000163.safetensors +3 -0
model-00081-of-000163.safetensors +3 -0
model-00082-of-000163.safetensors +3 -0
model-00083-of-000163.safetensors +3 -0
model-00084-of-000163.safetensors +3 -0
model-00085-of-000163.safetensors +3 -0

model-00036-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d73184ba7b2bd7178ce442a9aef77d74f004d7bbcbab8fe229d71f2e33687fb5
+size 4302384914

model-00037-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2084225f204dce09472e8ed35b26571010cc8e9bffab87a349cd4694a352a89b
+size 4302383558

model-00038-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f39f984c9cd0208987a34c40a6467824282e0ff5e3af0c46e445458d8411e8d7
+size 4302384518

model-00039-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2cabced73064e542530bcca94c26094cfa4d33349e1fdc363b63199e0c753e23
+size 4302384963

model-00040-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:60c89b403f4101748c45730f9cd024313c51bb9e460036344888530b3cea5093
+size 4302383366

model-00041-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2f2b74d08eeb042e23fc6e5d745cb85aaeb06efff0e74d2ae89009827783a685
+size 4302384710

model-00042-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:446a84b1475d0a20f219fde1c7b20ea853a08f323e281ba58c9cf16bb2e9ba23
+size 4302384963

model-00043-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ea2bfa8ead0a0b0d4d71c1236be2f86f7e649b7614c02d4ad7ee1855e52e3481
+size 4302383196

model-00044-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a9da3ffa2bd7b5302469b7267ecb5a53ccdf741cb927c69083761677fd4d2eae
+size 4302384900

model-00045-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7b0e39626af7e05ac590f930abbd338a451e9ac825a28669746ce725d76ab6ca
+size 4302383572

model-00046-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c539c254ebdcb6238f14ad1a66d68fc0b513661a1a8a90ae64a1696724330302
+size 4302384504

model-00047-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5a54385bd1c8fb7c9030ddbffd63a76b8a7052a071031ebe83aa39db89515e97
+size 4302384961

model-00048-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2ff52b68f3caa0f3b2bbad2ca2f629a41651ae66f560576746ceaf7d22597b73
+size 4302383384

model-00049-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:75863699936c04c1fe443fdd81aaf61a010691a8fd2be29992d95f8db9b357ac
+size 4302384692

model-00050-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:69b5922bb5a204e3d060464d1fabdb093ab03e877b78dc190debd39e95e6d9b0
+size 4302384963

model-00051-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:211f25115ee1c15caaba0e44c2529ce68ce401cfa0a18335c4a6c30f4fe2c7e3
+size 4302383212

model-00052-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b04ff2728838fc5c229cedc1e519b777d120d9f93a9e83dd16c605bd7c1ec5a8
+size 4302384884

model-00053-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4dfbece45355b607b461f4be25c65a67247622ad7750cc30f642684b434fa848
+size 4302383588

model-00054-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a1d34c53a22ae6b6b9a83f109e7545a90f1f96bd22096edfb27b5b0a1ab4fcb5
+size 4302384488

model-00055-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9fa7864654fa7831125e0f393280c82263458a5e43109d78af7ef325eba5b78c
+size 4302384963

model-00056-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:56768a2c728a375595e426c1f632bb44c4616400dcc0a90b0546f958ff11d758
+size 1747446144

model-00057-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:16ee62bc7eee708ecb65027cf9b0febc253e6c0ba147fbe08c689c2dc77b3314
+size 4302321911

model-00058-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:334b5bd3bf37e7efa8388683c3ad9af407b4d2bfb14f45f8e084ebdd35f9c476
+size 4302384914

model-00059-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8b744e9f4632903a95caddcd85df550abe9b57486527cb5896d41beb68a34e4e
+size 4302383558

model-00060-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:15dd8e49c12a8c32eb3756f7f42be51baa914ac22a850cc74cd559b733e33b93
+size 4302384518

model-00061-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:15397809f6e71884b38b3832b38106cb0fe392413154b7b71cb62b9608f2200c
+size 4302384963

model-00062-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6be912f411f755026d1852faffef4685070c0c6edb4b511a27d830b09e102817
+size 4302383366

model-00063-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:05f32ac9beb4ccaa4c17e937df16d503688fe84070b6d710513646abf1e45a1a
+size 4302384710

model-00064-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2e903707f7b66c6f00ebf059e08d0e2fc06e1a7d4e0711cd4c11cfc37cf6c376
+size 4302384963

model-00065-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5721f53a845902ea8e06085bf124bd23a8fc6b95247395e25b39eb1d066202cf
+size 4302383196

model-00066-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4c0d34a553c933d659638eb791ea5d8c29c0ce9862764b6cd39db7d766f946d1
+size 4302384900

model-00067-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:281d64fcd3c602f323435be30aa0649b7e8ac85175246efc8d2d84d4c4704f9f
+size 4302383572

model-00068-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0812696576e6c9cb536c4dbf0953c8f65e9c0293a86304ff0e8393e2cacc7064
+size 4302384504

model-00069-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ea085d9cef3103f35f9ec856ef8ac950735db429f737c2d50e40b5d8686cfbd2
+size 4302384961

model-00070-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:47a151beb00e20498f79780bf479419f242e7b7b9a528fe1044b4df601892982
+size 4302383384

model-00071-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4c4f3995269ae8fd2e9ed93ca513f633e1208853ff992b81ef3119b640deeada
+size 4302384692

model-00072-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:713f661b9389cb7f9f62b064262a316b885f3d966417b8815c3514ab01f0e6e9
+size 4302384963

model-00073-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e4b66ba5c04899b5027db81717af871d44bfb84c25bedc28813990c32d83fb33
+size 4302383212

model-00074-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:64990fd973fe475a6ace5f4f918d44961be310a12dbd4f3ec4257c0dac06a742
+size 4302384884

model-00075-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8eb5fb2f713df5f33b80132b6ebda037678881555eb8f8653e6c029ab5d5eb3d
+size 4302383588

model-00076-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6c60f62ac69c20ce1ce5d67ad4a2b99c83211391e2c50c008712552ffbcc2cbd
+size 4302384488

model-00077-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f6f1d7d3cca51240ed0be9565108099958c064442b9e8a2f485f47bd4b6b1b2f
+size 4302384963

model-00078-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8e643276482eafbc1fde19db22d4072070d0e55376d52c4d93563bffd3d5b419
+size 1747446144

model-00079-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dfab1e1888c2873615b719c1e3817af6e36ecaa33893166ffb2da4f4d2903ec3
+size 4302321911

model-00080-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7cf2aef3ef82a9ffa0d54251cc9157d23b170a24d25cd0840c4288e80254c9af
+size 4302384914

model-00081-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2b46c7a69792a4f05cea358820a7c7f4d9dcd7fcc201e66215890c22f83bf8e9
+size 4302383558

model-00082-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:905ef937795955af64ff86e9d27029b8f5c371e5707a7e5930de17088e459f0c
+size 4302384518

model-00083-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:477f7a54cc6c781d85674ed52a8bcea97494c75b9d4919d87960f9cfc6c59c6b
+size 4302384963

model-00084-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c4100de3acfe003bf7c8d3f555b679d83d465bb7afa27a58e3c1cbceface19b1
+size 4302383366

model-00085-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e194b5625583566d2ec7f274ace49328601187ae91e09df3ba5800f00957f038
+size 4302384710