Instructions to use quaedra/jet-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use quaedra/jet-4b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="quaedra/jet-4b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("quaedra/jet-4b")
model = AutoModelForCausalLM.from_pretrained("quaedra/jet-4b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use quaedra/jet-4b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "quaedra/jet-4b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "quaedra/jet-4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/quaedra/jet-4b

SGLang

How to use quaedra/jet-4b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "quaedra/jet-4b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "quaedra/jet-4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "quaedra/jet-4b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "quaedra/jet-4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use quaedra/jet-4b with Docker Model Runner:
```
docker model run hf.co/quaedra/jet-4b
```

jet-4b / release-manifest.json

michaljach

Archive detailed training artifacts and simplify model Files view

25ccbd9 verified 21 days ago

Raw History Blame

1.65 kB

	{
	"source_commit": "157e9b5f0e19e884255505e24efd9b0bfc18497d",
	"candidate": "jet-v5-panel02-r2-20260924",
	"validation": {
	"onnx_golden_passed": true,
	"cases": 36,
	"answer_flips": 0,
	"reference_backend": "native MLX CUDA fused bf16"
	},
	"files": {
	"README.md": "62a746e114cc3365b2b1f00c03bda5bee234f30186a2529eb2e1e1f09262e015",
	"TRAINING_HISTORY.md": "c94f2d2570078049f66ac89ec420b7bef7fba6a614d3e8add54444dfde092cde",
	"calibration.json": "6d9ba9d874699bf1f48f0703a814075a175ccb460d65c2c4870a3e6fb37fdb2f",
	"chat_template.jinja": "87a2728cb8dc9fe424d624542f6060ec05a1d285ebbec578bb078900e33396b5",
	"config.json": "332d56de2e5cde28245ac8b1776639bf9f7ed55fc911dfd2399ae8872acee2a9",
	"export-provenance.json": "a3f2a53d03816fe3e57ebcbdf850b84f98b52c3a135ad5cd581d96276eacc0d2",
	"export-validation.txt": "154c0212c4d4f64500ec13ebdffd7f405180a0ab3be6d049127144b4bb162df0",
	"golden.json": "98e895994a3544548a09ad72f68e599af908628ac8093ecd2eed58a679b69319",
	"model.safetensors": "5fb52e23dbff62e4c9982420ba4b07375b582685c4c1d22d1754f260e16e0097",
	"model.safetensors.index.json": "90681e31419773af1b89d8597c1f4f5153f092fec6e55d09ac268e62462a008e",
	"onnx/model_q8.onnx": "7ee7a5bdda762f735c6aa9257d63f758afef546afbfc1a701ba698fc89ef67f0",
	"tokenizer.json": "be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506",
	"tokenizer_config.json": "d93ac1a2c7adb9ad022f354d822f1f0f57e32e62ef2989e902782d6ab896acfe"
	},
	"training_archive": {
	"revision": "ee7d33ecae2b8bee30891657dd57c92d34e0ce53",
	"tag": "training-archive-20260924",
	"removed_from_main": 187
	}
	}