Instructions to use huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000")

# Load model directly
from transformers import AutoTokenizer, LlamaForCausalLMEagle3

tokenizer = AutoTokenizer.from_pretrained("huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000")
model = LlamaForCausalLMEagle3.from_pretrained("huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000

SGLang

How to use huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000 with Docker Model Runner:
```
docker model run hf.co/huluhuluu/qwen3-1p7b-eagle3-sharegpt-sw768-epoch-9-step-465000
```

huluhuluu commited on 17 days ago

Commit

61bcd0f

verified ·

1 Parent(s): cddb69d

Add files using upload-large-folder tool

Browse files

Files changed (4) hide show

README.md +83 -0
config.json +33 -0
model.safetensors +3 -0
training_state.pt +3 -0

README.md ADDED Viewed

	@@ -0,0 +1,83 @@

+---
+license: other
+language:
+  - en
+  - zh
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+  - specforge
+  - eagle3
+  - speculative-decoding
+  - draft-model
+  - qwen3
+  - sharegpt
+  - sliding-window
+base_model:
+  - Qwen/Qwen3-1.7B
+---
+# Qwen3 1.7B EAGLE3 ShareGPT SW768 - Epoch 9, Step 465000
+This repository contains a SpecForge EAGLE3 draft-model checkpoint for use with `Qwen/Qwen3-1.7B`.
+It is a draft model for speculative decoding, not a standalone target language model.
+## Checkpoint
+| Field | Value |
+|---|---|
+| Source run | `qwen3-1.7b-eagle3-sharegpt-sw768` |
+| Checkpoint | `epoch_9_step_465000` |
+| Epoch | `9` |
+| Global step | `465000` |
+| Files | `config.json`, `model.safetensors`, `training_state.pt` |
+## Training Parameters
+| Parameter | Value |
+|---|---|
+| Base model | `Qwen/Qwen3-1.7B` |
+| Method | `SpecForge EAGLE3 online training` |
+| Framework revision | `9fbbde8ab5d6ee69fb0af3701330027b8beca37a` |
+| Training data | `sharegpt_train.jsonl` |
+| Learning rate | `0.0001` |
+| Batch size | `1` |
+| Epochs configured | `10` |
+| Total scheduled steps | `467800` |
+| Max length | `2048` |
+| Warmup ratio | `0.015` |
+| Max grad norm | `0.5` |
+| TTT length | `7` |
+| Draft accumulation steps | `1` |
+| Draft sliding window | `768` |
+| Save / eval interval | `5000 / 5000` |
+| Seed | `0` |
+| TP / DP size | `1 / 2` |
+| Attention backend | `sdpa` |
+| Target model backend | `sglang` |
+| SGLang attention backend | `flashinfer` |
+| Dataset build workers | `16` |
+## Draft Model Configuration
+| Field | Value |
+|---|---|
+| Architecture | `LlamaForCausalLMEagle3` |
+| dtype | `bfloat16` |
+| Hidden size | `2048` |
+| Intermediate size | `6144` |
+| Draft layers | `1` |
+| Attention heads | `16` |
+| KV heads | `8` |
+| Draft vocab size | `32000` |
+| Vocab size | `151936` |
+| Max position embeddings | `40960` |
+| Sliding window | `768` |
+| Max window layers | `28` |
+## Notes
+- The checkpoint weights exclude the frozen target embedding weights; SpecForge loads them from the target model.
+- The checkpoint is intended to be loaded by SpecForge/EAGLE3-compatible code.
+- `training_state.pt` is included for provenance and training-state inspection.
+- No benchmark claim is made in this card.

config.json ADDED Viewed

	@@ -0,0 +1,33 @@

+{
+  "architectures": [
+    "LlamaForCausalLMEagle3"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "draft_vocab_size": 32000,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 6144,
+  "max_position_embeddings": 40960,
+  "max_window_layers": 28,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 1,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000000,
+  "sliding_window": 768,
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.1",
+  "use_cache": true,
+  "use_sliding_window": true,
+  "vocab_size": 151936
+}

model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:762e96d7519ac59b611075957cb5fba8441c98d25b172f92da4d6f514b272b3e
+size 274104160

training_state.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e2f5b3d68fdc0b751b1561cd81817b66a50a1c73e9c33cf8bb49a88b5a1e8291
+size 4375