Instructions to use widebluesky/wbs-llm-base-demo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use widebluesky/wbs-llm-base-demo with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="widebluesky/wbs-llm-base-demo")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("widebluesky/wbs-llm-base-demo")
model = AutoModelForCausalLM.from_pretrained("widebluesky/wbs-llm-base-demo", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use widebluesky/wbs-llm-base-demo with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "widebluesky/wbs-llm-base-demo"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "widebluesky/wbs-llm-base-demo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/widebluesky/wbs-llm-base-demo

SGLang

How to use widebluesky/wbs-llm-base-demo with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "widebluesky/wbs-llm-base-demo" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "widebluesky/wbs-llm-base-demo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "widebluesky/wbs-llm-base-demo" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "widebluesky/wbs-llm-base-demo",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use widebluesky/wbs-llm-base-demo with Docker Model Runner:
```
docker model run hf.co/widebluesky/wbs-llm-base-demo
```

wbs-llm-base-demo / tokenizer.json

widebluesky

Publish trained tiny LLM base demo-v1 (full fp32, epoch 2)

feb757e verified 8 days ago

Raw History Blame Contribute Delete

4.41 kB

	{
	"version": "1.0",
	"truncation": null,
	"padding": null,
	"added_tokens": [
	{
	"id": 0,
	"content": "<pad>",
	"single_word": false,
	"lstrip": false,
	"rstrip": false,
	"normalized": false,
	"special": true
	},
	{
	"id": 1,
	"content": "<unk>",
	"single_word": false,
	"lstrip": false,
	"rstrip": false,
	"normalized": false,
	"special": true
	},
	{
	"id": 2,
	"content": "<bos>",
	"single_word": false,
	"lstrip": false,
	"rstrip": false,
	"normalized": false,
	"special": true
	},
	{
	"id": 3,
	"content": "<eos>",
	"single_word": false,
	"lstrip": false,
	"rstrip": false,
	"normalized": false,
	"special": true
	},
	{
	"id": 4,
	"content": "<\|system\|>",
	"single_word": false,
	"lstrip": false,
	"rstrip": false,
	"normalized": false,
	"special": true
	},
	{
	"id": 5,
	"content": "<\|user\|>",
	"single_word": false,
	"lstrip": false,
	"rstrip": false,
	"normalized": false,
	"special": true
	},
	{
	"id": 6,
	"content": "<\|assistant\|>",
	"single_word": false,
	"lstrip": false,
	"rstrip": false,
	"normalized": false,
	"special": true
	},
	{
	"id": 7,
	"content": "<\|end\|>",
	"single_word": false,
	"lstrip": false,
	"rstrip": false,
	"normalized": false,
	"special": true
	}
	],
	"normalizer": null,
	"pre_tokenizer": {
	"type": "WhitespaceSplit"
	},
	"post_processor": {
	"type": "TemplateProcessing",
	"single": [
	{
	"Sequence": {
	"id": "A",
	"type_id": 0
	}
	}
	],
	"pair": [
	{
	"Sequence": {
	"id": "A",
	"type_id": 0
	}
	},
	{
	"Sequence": {
	"id": "B",
	"type_id": 1
	}
	}
	],
	"special_tokens": {}
	},
	"decoder": null,
	"model": {
	"type": "WordLevel",
	"vocab": {
	"<pad>": 0,
	"<unk>": 1,
	"<bos>": 2,
	"<eos>": 3,
	"<\|system\|>": 4,
	"<\|user\|>": 5,
	"<\|assistant\|>": 6,
	"<\|end\|>": 7,
	"the": 8,
	"a": 9,
	"is": 10,
	"are": 11,
	"of": 12,
	"to": 13,
	"and": 14,
	"in": 15,
	"for": 16,
	"with": 17,
	"you": 18,
	"your": 19,
	"I": 20,
	"can": 21,
	"please": 22,
	"answer": 23,
	"only": 24,
	"help": 25,
	"account": 26,
	"password": 27,
	"reset": 28,
	"login": 29,
	"email": 30,
	"click": 31,
	"settings": 32,
	"support": 33,
	"change": 34,
	"cancel": 35,
	"subscription": 36,
	"profile": 37,
	"name": 38,
	"update": 39,
	"billing": 40,
	"payment": 41,
	"card": 42,
	"refund": 43,
	"yes": 44,
	"no": 45,
	"true": 46,
	"false": 47,
	"one": 48,
	"two": 49,
	"three": 50,
	"four": 51,
	"1": 52,
	"2": 53,
	"3": 54,
	"4": 55,
	"hello": 56,
	"world": 57,
	"blue": 58,
	"red": 59,
	"green": 60,
	"sky": 61,
	"water": 62,
	"grass": 63,
	"color": 64,
	"what": 65,
	"how": 66,
	"where": 67,
	"why": 68,
	"when": 69,
	"return": 70,
	"JSON": 71,
	"book": 72,
	"read": 73,
	"model": 74,
	"text": 75,
	"train": 76,
	"data": 77,
	"test": 78,
	"validation": 79,
	"document": 80,
	"code": 81,
	"language": 82,
	"learn": 83,
	"cat": 84,
	"dog": 85,
	"animal": 86,
	"bird": 87,
	"fish": 88,
	"flies": 89,
	"swims": 90,
	"runs": 91,
	"capital": 92,
	"France": 93,
	"Paris": 94,
	"Italy": 95,
	"Rome": 96,
	"Germany": 97,
	"Berlin": 98,
	"sun": 99,
	"hot": 100,
	"ice": 101,
	"cold": 102,
	"instruction": 103,
	"question": 104,
	"response": 105,
	"user": 106,
	"assistant": 107,
	"system": 108,
	"explain": 109,
	"compute": 110,
	"add": 111,
	"result": 112,
	"5": 113,
	"6": 114,
	"7": 115,
	"8": 116,
	"9": 117,
	"10": 118,
	".": 119,
	",": 120,
	":": 121,
	"?": 122
	},
	"unk_token": "<unk>"
	}
	}