Instructions to use kmouratidis/QwenLong-L1-32B-4.25bpw with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kmouratidis/QwenLong-L1-32B-4.25bpw with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kmouratidis/QwenLong-L1-32B-4.25bpw")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kmouratidis/QwenLong-L1-32B-4.25bpw")
model = AutoModelForCausalLM.from_pretrained("kmouratidis/QwenLong-L1-32B-4.25bpw", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kmouratidis/QwenLong-L1-32B-4.25bpw with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kmouratidis/QwenLong-L1-32B-4.25bpw"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kmouratidis/QwenLong-L1-32B-4.25bpw",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kmouratidis/QwenLong-L1-32B-4.25bpw

SGLang

How to use kmouratidis/QwenLong-L1-32B-4.25bpw with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kmouratidis/QwenLong-L1-32B-4.25bpw" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kmouratidis/QwenLong-L1-32B-4.25bpw",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kmouratidis/QwenLong-L1-32B-4.25bpw" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kmouratidis/QwenLong-L1-32B-4.25bpw",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use kmouratidis/QwenLong-L1-32B-4.25bpw with Docker Model Runner:
```
docker model run hf.co/kmouratidis/QwenLong-L1-32B-4.25bpw
```

QwenLong-L1-32B-4.25bpw

Commit History

Upload missing figures

72ecc4e
verified

kmouratidis commited on Jun 9, 2025

Change library name to transformers (#1)

7846fe5
verified

kmouratidis

nielsr HF Staff commited on Jun 9, 2025

Quantization files

fb95e65
verified

kmouratidis commited on Jun 6, 2025

Model files

9bae325
verified

kmouratidis commited on Jun 6, 2025

initial commit

0ffa112
verified

kmouratidis commited on Jun 6, 2025

Commit History

Upload missing figures 72ecc4e verified

Change library name to transformers (#1) 7846fe5 verified

Quantization files fb95e65 verified

Model files 9bae325 verified

initial commit 0ffa112 verified

Upload missing figures

72ecc4e
verified

Change library name to transformers (#1)

7846fe5
verified

Quantization files

fb95e65
verified

Model files

9bae325
verified

initial commit

0ffa112
verified