Instructions to use baidu/ERNIE-4.5-300B-A47B-Base-PT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use baidu/ERNIE-4.5-300B-A47B-Base-PT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="baidu/ERNIE-4.5-300B-A47B-Base-PT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("baidu/ERNIE-4.5-300B-A47B-Base-PT")
model = AutoModelForCausalLM.from_pretrained("baidu/ERNIE-4.5-300B-A47B-Base-PT", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use baidu/ERNIE-4.5-300B-A47B-Base-PT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "baidu/ERNIE-4.5-300B-A47B-Base-PT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "baidu/ERNIE-4.5-300B-A47B-Base-PT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/baidu/ERNIE-4.5-300B-A47B-Base-PT

SGLang

How to use baidu/ERNIE-4.5-300B-A47B-Base-PT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "baidu/ERNIE-4.5-300B-A47B-Base-PT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "baidu/ERNIE-4.5-300B-A47B-Base-PT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "baidu/ERNIE-4.5-300B-A47B-Base-PT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "baidu/ERNIE-4.5-300B-A47B-Base-PT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use baidu/ERNIE-4.5-300B-A47B-Base-PT with Docker Model Runner:
```
docker model run hf.co/baidu/ERNIE-4.5-300B-A47B-Base-PT
```

Why not convert checkpoint between PT and Paddle?

by hechenrui - opened Jul 1, 2025

Discussion

hechenrui

Jul 1, 2025

Why not convert checkpoint between PT and Paddle?
It's a burden to download twice.
Are their values different?

DesmonDay

BAIDU org Jul 2, 2025

Because currently our ERNIEKit training code and the FastDeploy tool does not support loading PyTorch weights, we need to provide both Paddle and PyTorch model weights to ensure compatibility with different frameworks. The biggest difference between PyTorch and Paddle weights lies in the handling of linear layer weights — PyTorch stores Linear weights in [out_features, in_features] format, whereas Paddle uses [in_features, out_features], effectively requiring a transpose during conversion between the two. In the future, we plan to enhance ERNIEKit to support training directly from PyTorch weights as well.

Gregniuki

Jul 27, 2025

I need to convert 0.36b ernie 4.5 paddle model to Pytorch to run it with Zero-Gpu there is only support for Ernie 3.0 conversion.
https://github.com/nghuyong/ERNIE-Pytorch.git
Will there be pytorch export option in Erniekit or there is a separate python script for conversion for 4.5 0.36b paddle model ?

jzhang533

BAIDU org Jul 28, 2025

pytorch compatabible ernie 4.5 0.3b model is also available in Hugging face.
If you are looking for the conversion scripts, check here:

https://github.com/PaddlePaddle/ERNIE/tree/develop/tools/paddle2torch

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment