Instructions to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16")
model = AutoModelForCausalLM.from_pretrained("nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16

SGLang

How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16 with Docker Model Runner:
```
docker model run hf.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16
```

dmax123 commited on Jun 3

Commit

19f72d1

verified ·

1 Parent(s): 7be14e7

Add files using upload-large-folder tool

Browse files

This view is limited to 50 files because it contains too many changes. See raw diff

Files changed (50) hide show

model-00001-of-00225.safetensors +3 -0
model-00002-of-00225.safetensors +3 -0
model-00005-of-00225.safetensors +3 -0
model-00006-of-00225.safetensors +3 -0
model-00010-of-00225.safetensors +3 -0
model-00011-of-00225.safetensors +3 -0
model-00014-of-00225.safetensors +3 -0
model-00015-of-00225.safetensors +3 -0
model-00016-of-00225.safetensors +3 -0
model-00017-of-00225.safetensors +3 -0
model-00018-of-00225.safetensors +3 -0
model-00021-of-00225.safetensors +3 -0
model-00024-of-00225.safetensors +3 -0
model-00025-of-00225.safetensors +3 -0
model-00028-of-00225.safetensors +3 -0
model-00030-of-00225.safetensors +3 -0
model-00032-of-00225.safetensors +3 -0
model-00033-of-00225.safetensors +3 -0
model-00035-of-00225.safetensors +3 -0
model-00038-of-00225.safetensors +3 -0
model-00040-of-00225.safetensors +3 -0
model-00041-of-00225.safetensors +3 -0
model-00043-of-00225.safetensors +3 -0
model-00044-of-00225.safetensors +3 -0
model-00048-of-00225.safetensors +3 -0
model-00049-of-00225.safetensors +3 -0
model-00050-of-00225.safetensors +3 -0
model-00057-of-00225.safetensors +3 -0
model-00059-of-00225.safetensors +3 -0
model-00060-of-00225.safetensors +3 -0
model-00062-of-00225.safetensors +3 -0
model-00064-of-00225.safetensors +3 -0
model-00066-of-00225.safetensors +3 -0
model-00068-of-00225.safetensors +3 -0
model-00070-of-00225.safetensors +3 -0
model-00072-of-00225.safetensors +3 -0
model-00076-of-00225.safetensors +3 -0
model-00078-of-00225.safetensors +3 -0
model-00079-of-00225.safetensors +3 -0
model-00080-of-00225.safetensors +3 -0
model-00083-of-00225.safetensors +3 -0
model-00093-of-00225.safetensors +3 -0
model-00096-of-00225.safetensors +3 -0
model-00097-of-00225.safetensors +3 -0
model-00098-of-00225.safetensors +3 -0
model-00099-of-00225.safetensors +3 -0
model-00101-of-00225.safetensors +3 -0
model-00103-of-00225.safetensors +3 -0
model-00107-of-00225.safetensors +3 -0
model-00108-of-00225.safetensors +3 -0

model-00001-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c843b8c13ff71774df6db2875fd49c99928df9e2c36b0d8aaf33f247405cfe50
+size 4983097968

model-00002-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:58effe9f8ca6d0c7f3b0149dbc503c413b37cb6e34152c9c2ec0730b4b19f70b
+size 4991252496

model-00005-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:98f0d2533ca1e174241a520284e3e68692594eebf5c97e222d530f86b7e3bcf9
+size 4920152112

model-00006-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f67cb738d9ca35b7d0e0b09327009bc027cccdc369a8780845f2b22277aad4da
+size 4995491168

model-00010-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:41d0b2f3df2792397f01179b5a8d6034f824e8b9b36d1498bb723a795cc80ca4
+size 4987305680

model-00011-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:452b825fc9ec4d67f6469575d855b666c0985fb3b204ee0f2be64810a8815dd9
+size 4991252464

model-00014-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:38d75394baa036abd70080e85b11243036707abd8c4e7b990f4af5c3a32c384a
+size 4991252600

model-00015-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:00797b5240bbc9fbcc843fb207aca94648ba8447c1a8c0466fbd3deed63fea03
+size 4991515208

model-00016-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b265f1b4fa76a554d99a90cb2532472b15030de59f834db4a39174b7136fe420
+size 4991252560

model-00017-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b395fb64dd53f86b1b96ee2fdefd059b7992993ba6ab0fc17ef437d2b0435d33
+size 4991252600

model-00018-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:08f73123dabf89930beb5e3d842ec802657b349a1f2dc3ccc90f9f384c3963e7
+size 4991252600

model-00021-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b3ecfee4b37de2f6bb7b446f9b2e733f9a2b1411e6dceab9477e39abee901842
+size 4991252840

model-00024-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:764162f8df87f28b6267dbe3b493572937dccd08b60ff03d39b00e4c45897c3b
+size 4987305808

model-00025-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:53410dae47b1119be1c2c79dd5fa309cd0f8c72ae343906a4bf08433892fda27
+size 4991252776

model-00028-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7b727c1a4fafc9033e25db48ba14adab8ff7385dff99bbcb70e8f493a2de5c55
+size 4966334304

model-00030-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ad51894d6b5a8f462636302b8d43f7b2e422e56a1db13846816eae444030c127
+size 4991252840

model-00032-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3940d1b60d4746aa5372ac2e15136dc31ed3dc5ebdc3795c683750be9be3adb9
+size 4991252840

model-00033-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1aa1479cf3dcbcdaa5714414713bce83aae098ebd3bc04b0be634ac5aa6d2907
+size 4987305848

model-00035-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cdfa83c33cf3d3c6fcf14921cb5230e541770ad3c1203da4c77c7455c92045a1
+size 4991252840

model-00038-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bc9ecf5e12e2456b7970890ca2a079baf5f3f4532c35efa6033da4e35b74d239
+size 4995491376

model-00040-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:45ccd7880d8f73cbfd932e4ac838c3ccd6f922102426c9305eef65563f9778e8
+size 4991252840

model-00041-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:438c002143aa5b8f00faf00d69370c01ac2bfe78c71793a51967b8b9100538d7
+size 4991252840

model-00043-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c11e622d552fd5390e61cf340ea1864b4c27f297b0562a895819226a23e97f3a
+size 4991252704

model-00044-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d46b632b23c4dafe2538b4f6fc3d6ee98fc5e33457f4f16405edf450058525e7
+size 4991252840

model-00048-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bdff4bc5ef6e472761992235b5cb99dcf3c1145279d689779eff6fb26c1c0a46
+size 4991252800

model-00049-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:38624105bf4a945f549c07fcef00a3bd443ec0411fd3948e4d2971010ed52960
+size 4991252840

model-00050-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2d9a0ac5128ec4d3b7fd2b4c39c1dd0585bd75cfe87b6c0ad0706df4d887f7e4
+size 4991252840

model-00057-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a081c9fd26f2915ac4f3c026841b759ed9d61b1f77b1bf8fa3a273ee0619e684
+size 4991252776

model-00059-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8079c3a8e6d0b634a32ffd9d2b6eb1ed0785b671c13c2ff1cb85677b0dd21f70
+size 4991252840

model-00060-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ef7e8a902ba2a8e6855831827b08d61554b38343e4ff43b2d0a711c572e4366f
+size 4987305952

model-00062-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:14c1e63f6cebe47f953c4ea04dda394602e739c0d25d70b5e55bbcd3609aefe6
+size 4991252840

model-00064-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:415e80ec4bc44c21fa599ad56c2c37ea9ac9eca54bc11810cf50a3b5a049a300
+size 4991252840

model-00066-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2fbb4595cef8e9b93b595c46317fe5b7cf82acbb5a4312ddd1d503f17d86ee76
+size 4991252736

model-00068-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d55082b5047bad9f75a0837cbc7feea1097e377fb13949cf10492e4422f020c9
+size 4991252840

model-00070-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8def732b95f77ac922cb0d7724fe63a3862e61ed55a5d64e9966ed6686e2dc25
+size 4995491376

model-00072-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:077f8b3cecbb2820fed2dbe196b075359568d6c899be1b3a4dfdbecefc89de0b
+size 4991252840

model-00076-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d20104fbd1c4287583c7106724d44bb63547c76e074b602ae7306858c483dd55
+size 4991252840

model-00078-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fd7fc9d40045639b8dec26466526cb57b34de199ac4c9ad4bbb64b4b12907fa8
+size 4991252840

model-00079-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:750bef9dcfcf9f9b1c8793bf26fe4afa38760f9dd1e339384eeee824038541e4
+size 4991515392

model-00080-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:62801e4b8a05fcafd90d9b94041d0bff06e1498a042bc455f2781fc0dd8a2364
+size 4991252800

model-00083-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8ce8df2203bc0abab2dbf520e1df33baf0c9c082c362721d739703135f8cee47
+size 4987305912

model-00093-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a365101400f46abe340d90ec7f5ed950406622533ca2cf6a6aba67794b686401
+size 4991252624

model-00096-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f275826b8f285d6ad558b8de1438054c9835f25d85efc4a9b9b77647935b08ce
+size 4991252840

model-00097-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:97a855c5bc0f94c38b9421a0add77cf13350a0f7ff869fa6a7feeb2a69180a1b
+size 4991515400

model-00098-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e09d9cb555f8600256b5bb7af8a244e07618888f24c0e6df27bb7be5a6d60adf
+size 4991252736

model-00099-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:83b82f829245582c74fa0732b3aff337f167dfda00eca9e021b0b33e2aedfeeb
+size 4991252840

model-00101-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:775ad8075802ad917c599786d568bf5840229c464b0984b23401662563508f12
+size 4941123984

model-00103-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b776c45b3a9f50dc0c3fae7a549ccbb87ad49be9f33ca462be1ac4cd33b09c20
+size 4991252840

model-00107-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:06acd10068ffda3478675ffa13b05bf6ac641863a14e1fd248b6f756835e869d
+size 4991252704

model-00108-of-00225.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ae7b019a1c5cbb43fb0a1c6847fbf3ca336e477bddc1b5f4b34b8b7ab4e6f52d
+size 4991252840