Instructions to use JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16

SGLang

How to use JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16 with Docker Model Runner:
```
docker model run hf.co/JasonW2025/Ling-3.0-flash-HybridQuant-NVFP4-W4A16
```

Ling-3.0-flash-HybridQuant-NVFP4-W4A16

File size: 9,050 Bytes

d2bf6b4

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 940 660" width="940" height="660">
<style>
  .grid{stroke:#d0d7de;stroke-width:1}
  .ax{font:12px system-ui,sans-serif;fill:#57606a}
  .ttl{font:600 13px system-ui,sans-serif;fill:#1f2328}
  .lg{font:13px system-ui,sans-serif;fill:#1f2328}
  .ft{font:11px system-ui,sans-serif;fill:#57606a}
  @media (prefers-color-scheme: dark){
    .grid{stroke:#30363d}.ax{fill:#8b949e}.ttl{fill:#e6edf3}
    .lg{fill:#e6edf3}.ft{fill:#8b949e}}
</style>
<text x="265" y="28" text-anchor="middle" class="ttl">decode tok/s — all streams</text>
<line x1="66" y1="288.0" x2="464" y2="288.0" class="grid"/>
<text x="58" y="292.0" text-anchor="end" class="ax">0</text>
<line x1="66" y1="227.0" x2="464" y2="227.0" class="grid"/>
<text x="58" y="231.0" text-anchor="end" class="ax">75</text>
<line x1="66" y1="166.0" x2="464" y2="166.0" class="grid"/>
<text x="58" y="170.0" text-anchor="end" class="ax">150</text>
<line x1="66" y1="105.0" x2="464" y2="105.0" class="grid"/>
<text x="58" y="109.0" text-anchor="end" class="ax">225</text>
<line x1="66" y1="44.0" x2="464" y2="44.0" class="grid"/>
<text x="58" y="48.0" text-anchor="end" class="ax">300</text>
<text x="66.0" y="306" text-anchor="middle" class="ax">1</text>
<text x="145.6" y="306" text-anchor="middle" class="ax">2</text>
<text x="225.2" y="306" text-anchor="middle" class="ax">4</text>
<text x="304.8" y="306" text-anchor="middle" class="ax">8</text>
<text x="384.4" y="306" text-anchor="middle" class="ax">16</text>
<text x="464.0" y="306" text-anchor="middle" class="ax">32</text>
<text x="265" y="326" text-anchor="middle" class="ax">concurrent requests</text>
<path d="M66.0,242.3 L145.6,218.4 L225.2,184.2 L304.8,144.2 L384.4,95.4 L464.0,69.4" fill="none" stroke="#4C78A8" stroke-width="2.5"/>
<circle cx="66.0" cy="242.3" r="3.4" fill="#4C78A8"/>
<circle cx="145.6" cy="218.4" r="3.4" fill="#4C78A8"/>
<circle cx="225.2" cy="184.2" r="3.4" fill="#4C78A8"/>
<circle cx="304.8" cy="144.2" r="3.4" fill="#4C78A8"/>
<circle cx="384.4" cy="95.4" r="3.4" fill="#4C78A8"/>
<circle cx="464.0" cy="69.4" r="3.4" fill="#4C78A8"/>
<path d="M66.0,242.1 L145.6,224.9 L225.2,202.4 L304.8,190.4 L384.4,174.8" fill="none" stroke="#F58518" stroke-width="2.5"/>
<circle cx="66.0" cy="242.1" r="3.4" fill="#F58518"/>
<circle cx="145.6" cy="224.9" r="3.4" fill="#F58518"/>
<circle cx="225.2" cy="202.4" r="3.4" fill="#F58518"/>
<circle cx="304.8" cy="190.4" r="3.4" fill="#F58518"/>
<circle cx="384.4" cy="174.8" r="3.4" fill="#F58518"/>
<text x="719" y="28" text-anchor="middle" class="ttl">decode tok/s — per stream</text>
<line x1="520" y1="288.0" x2="918" y2="288.0" class="grid"/>
<text x="512" y="292.0" text-anchor="end" class="ax">0</text>
<line x1="520" y1="227.0" x2="918" y2="227.0" class="grid"/>
<text x="512" y="231.0" text-anchor="end" class="ax">15</text>
<line x1="520" y1="166.0" x2="918" y2="166.0" class="grid"/>
<text x="512" y="170.0" text-anchor="end" class="ax">30</text>
<line x1="520" y1="105.0" x2="918" y2="105.0" class="grid"/>
<text x="512" y="109.0" text-anchor="end" class="ax">45</text>
<line x1="520" y1="44.0" x2="918" y2="44.0" class="grid"/>
<text x="512" y="48.0" text-anchor="end" class="ax">60</text>
<text x="520.0" y="306" text-anchor="middle" class="ax">1</text>
<text x="599.6" y="306" text-anchor="middle" class="ax">2</text>
<text x="679.2" y="306" text-anchor="middle" class="ax">4</text>
<text x="758.8" y="306" text-anchor="middle" class="ax">8</text>
<text x="838.4" y="306" text-anchor="middle" class="ax">16</text>
<text x="918.0" y="306" text-anchor="middle" class="ax">32</text>
<text x="719" y="326" text-anchor="middle" class="ax">concurrent requests</text>
<path d="M520.0,59.5 L599.6,113.9 L679.2,158.3 L758.8,198.1 L838.4,227.8 L918.0,253.8" fill="none" stroke="#4C78A8" stroke-width="2.5"/>
<circle cx="520.0" cy="59.5" r="3.4" fill="#4C78A8"/>
<circle cx="599.6" cy="113.9" r="3.4" fill="#4C78A8"/>
<circle cx="679.2" cy="158.3" r="3.4" fill="#4C78A8"/>
<circle cx="758.8" cy="198.1" r="3.4" fill="#4C78A8"/>
<circle cx="838.4" cy="227.8" r="3.4" fill="#4C78A8"/>
<circle cx="918.0" cy="253.8" r="3.4" fill="#4C78A8"/>
<path d="M520.0,58.6 L599.6,130.2 L679.2,181.0 L758.8,227.0 L838.4,252.6" fill="none" stroke="#F58518" stroke-width="2.5"/>
<circle cx="520.0" cy="58.6" r="3.4" fill="#F58518"/>
<circle cx="599.6" cy="130.2" r="3.4" fill="#F58518"/>
<circle cx="679.2" cy="181.0" r="3.4" fill="#F58518"/>
<circle cx="758.8" cy="227.0" r="3.4" fill="#F58518"/>
<circle cx="838.4" cy="252.6" r="3.4" fill="#F58518"/>
<text x="265" y="338" text-anchor="middle" class="ttl">prefill tok/s — all streams</text>
<line x1="66" y1="598.0" x2="464" y2="598.0" class="grid"/>
<text x="58" y="602.0" text-anchor="end" class="ax">0</text>
<line x1="66" y1="537.0" x2="464" y2="537.0" class="grid"/>
<text x="58" y="541.0" text-anchor="end" class="ax">1250</text>
<line x1="66" y1="476.0" x2="464" y2="476.0" class="grid"/>
<text x="58" y="480.0" text-anchor="end" class="ax">2500</text>
<line x1="66" y1="415.0" x2="464" y2="415.0" class="grid"/>
<text x="58" y="419.0" text-anchor="end" class="ax">3750</text>
<line x1="66" y1="354.0" x2="464" y2="354.0" class="grid"/>
<text x="58" y="358.0" text-anchor="end" class="ax">5000</text>
<text x="66.0" y="616" text-anchor="middle" class="ax">1</text>
<text x="145.6" y="616" text-anchor="middle" class="ax">2</text>
<text x="225.2" y="616" text-anchor="middle" class="ax">4</text>
<text x="304.8" y="616" text-anchor="middle" class="ax">8</text>
<text x="384.4" y="616" text-anchor="middle" class="ax">16</text>
<text x="464.0" y="616" text-anchor="middle" class="ax">32</text>
<text x="265" y="636" text-anchor="middle" class="ax">concurrent requests</text>
<path d="M66.0,468.9 L145.6,469.5 L225.2,446.2 L304.8,438.4 L384.4,431.3 L464.0,367.8" fill="none" stroke="#4C78A8" stroke-width="2.5"/>
<circle cx="66.0" cy="468.9" r="3.4" fill="#4C78A8"/>
<circle cx="145.6" cy="469.5" r="3.4" fill="#4C78A8"/>
<circle cx="225.2" cy="446.2" r="3.4" fill="#4C78A8"/>
<circle cx="304.8" cy="438.4" r="3.4" fill="#4C78A8"/>
<circle cx="384.4" cy="431.3" r="3.4" fill="#4C78A8"/>
<circle cx="464.0" cy="367.8" r="3.4" fill="#4C78A8"/>
<path d="M66.0,476.1 L145.6,471.9 L225.2,456.8 L304.8,449.3 L384.4,443.9" fill="none" stroke="#F58518" stroke-width="2.5"/>
<circle cx="66.0" cy="476.1" r="3.4" fill="#F58518"/>
<circle cx="145.6" cy="471.9" r="3.4" fill="#F58518"/>
<circle cx="225.2" cy="456.8" r="3.4" fill="#F58518"/>
<circle cx="304.8" cy="449.3" r="3.4" fill="#F58518"/>
<circle cx="384.4" cy="443.9" r="3.4" fill="#F58518"/>
<text x="719" y="338" text-anchor="middle" class="ttl">TTFT (seconds)</text>
<line x1="520" y1="598.0" x2="918" y2="598.0" class="grid"/>
<text x="512" y="602.0" text-anchor="end" class="ax">0</text>
<line x1="520" y1="537.0" x2="918" y2="537.0" class="grid"/>
<text x="512" y="541.0" text-anchor="end" class="ax">2</text>
<line x1="520" y1="476.0" x2="918" y2="476.0" class="grid"/>
<text x="512" y="480.0" text-anchor="end" class="ax">5</text>
<line x1="520" y1="415.0" x2="918" y2="415.0" class="grid"/>
<text x="512" y="419.0" text-anchor="end" class="ax">8</text>
<line x1="520" y1="354.0" x2="918" y2="354.0" class="grid"/>
<text x="512" y="358.0" text-anchor="end" class="ax">10</text>
<text x="520.0" y="616" text-anchor="middle" class="ax">1</text>
<text x="599.6" y="616" text-anchor="middle" class="ax">2</text>
<text x="679.2" y="616" text-anchor="middle" class="ax">4</text>
<text x="758.8" y="616" text-anchor="middle" class="ax">8</text>
<text x="838.4" y="616" text-anchor="middle" class="ax">16</text>
<text x="918.0" y="616" text-anchor="middle" class="ax">32</text>
<text x="719" y="636" text-anchor="middle" class="ax">concurrent requests</text>
<path d="M520.0,584.6 L599.6,570.5 L679.2,552.3 L758.8,511.0 L838.4,431.5 L918.0,356.7" fill="none" stroke="#4C78A8" stroke-width="2.5"/>
<circle cx="520.0" cy="584.6" r="3.4" fill="#4C78A8"/>
<circle cx="599.6" cy="570.5" r="3.4" fill="#4C78A8"/>
<circle cx="679.2" cy="552.3" r="3.4" fill="#4C78A8"/>
<circle cx="758.8" cy="511.0" r="3.4" fill="#4C78A8"/>
<circle cx="838.4" cy="431.5" r="3.4" fill="#4C78A8"/>
<circle cx="918.0" cy="356.7" r="3.4" fill="#4C78A8"/>
<path d="M520.0,583.8 L599.6,569.5 L679.2,548.8 L758.8,504.7 L838.4,417.9" fill="none" stroke="#F58518" stroke-width="2.5"/>
<circle cx="520.0" cy="583.8" r="3.4" fill="#F58518"/>
<circle cx="599.6" cy="569.5" r="3.4" fill="#F58518"/>
<circle cx="679.2" cy="548.8" r="3.4" fill="#F58518"/>
<circle cx="758.8" cy="504.7" r="3.4" fill="#F58518"/>
<circle cx="838.4" cy="417.9" r="3.4" fill="#F58518"/>
<line x1="66" y1="644" x2="90" y2="644" stroke="#4C78A8" stroke-width="2.5"/>
<text x="97" y="649" class="lg">no spec-decode</text>
<line x1="224" y1="644" x2="248" y2="644" stroke="#F58518" stroke-width="2.5"/>
<text x="255" y="649" class="lg">MTP k=1</text>
<text x="918" y="649" text-anchor="end" class="ft">GB10 · 1457-tok prompt · 256 out/stream · prefix caching OFF · median of n=3</text>
</svg>