---
license: other
language:
  - en
  - zh
library_name: transformers
pipeline_tag: text-generation
tags:
  - specforge
  - eagle3
  - speculative-decoding
  - draft-model
  - qwen3
  - sharegpt
  - sliding-window
base_model:
  - Qwen/Qwen3-1.7B
---

# Qwen3 1.7B EAGLE3 ShareGPT SW768 - Epoch 9, Step 465000

This repository contains a SpecForge EAGLE3 draft-model checkpoint for use with `Qwen/Qwen3-1.7B`.
It is a draft model for speculative decoding, not a standalone target language model.

## Checkpoint

| Field | Value |
|---|---|
| Source run | `qwen3-1.7b-eagle3-sharegpt-sw768` |
| Checkpoint | `epoch_9_step_465000` |
| Epoch | `9` |
| Global step | `465000` |
| Files | `config.json`, `model.safetensors`, `training_state.pt` |

## Training Parameters

| Parameter | Value |
|---|---|
| Base model | `Qwen/Qwen3-1.7B` |
| Method | `SpecForge EAGLE3 online training` |
| Framework revision | `9fbbde8ab5d6ee69fb0af3701330027b8beca37a` |
| Training data | `sharegpt_train.jsonl` |
| Learning rate | `0.0001` |
| Batch size | `1` |
| Epochs configured | `10` |
| Total scheduled steps | `467800` |
| Max length | `2048` |
| Warmup ratio | `0.015` |
| Max grad norm | `0.5` |
| TTT length | `7` |
| Draft accumulation steps | `1` |
| Draft sliding window | `768` |
| Save / eval interval | `5000 / 5000` |
| Seed | `0` |
| TP / DP size | `1 / 2` |
| Attention backend | `sdpa` |
| Target model backend | `sglang` |
| SGLang attention backend | `flashinfer` |
| Dataset build workers | `16` |

## Draft Model Configuration

| Field | Value |
|---|---|
| Architecture | `LlamaForCausalLMEagle3` |
| dtype | `bfloat16` |
| Hidden size | `2048` |
| Intermediate size | `6144` |
| Draft layers | `1` |
| Attention heads | `16` |
| KV heads | `8` |
| Draft vocab size | `32000` |
| Vocab size | `151936` |
| Max position embeddings | `40960` |
| Sliding window | `768` |
| Max window layers | `28` |

## Notes

- The checkpoint weights exclude the frozen target embedding weights; SpecForge loads them from the target model.
- The checkpoint is intended to be loaded by SpecForge/EAGLE3-compatible code.
- `training_state.pt` is included for provenance and training-state inspection.
- No benchmark claim is made in this card.