Update README.md

README.md

@@ -1,10 +1,501 @@
 ---
 language:
+- en
 - zh
+- multilingual
+license: other
+license_name: stepfun-community-license
+license_link: https://huggingface.co/SHSLab/Step-5-Preview-BF16/blob/main/LICENSE
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- stepfun
+- step-5
+- moe
+- mixture-of-experts
+- agentic
+- coding
+- software-engineering
+- long-context
+- 1m-context
+- multimodal
+- text-generation
+- image
+- video
+- sparse-attention
+- gqa
+- financial-analysis
+- deep-research
+- tool-calling
+- parallel-tool-calling
+- json-schema
+base_model: SHSLab/Step-5-Preview-BF16
 ---
 
-# Step 5 Preview BF6
+# Step-5-Preview
 
-版权所有 @stepfun-ai
+<div align="center">
+  <img src="https://raw.githubusercontent.com/stepfun-ai/Step-5-Preview/main/assets/step5_banner.png" alt="Step 5 Preview Banner" width="100%">
+</div>
 
-我什么都不知道啊（
+<div align="center">
+
+[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-StepFun-yellow)](https://huggingface.co/SHSLab)
+[![GitHub](https://img.shields.io/badge/GitHub-StepFun-181717?logo=github)](https://github.com/stepfun-ai)
+[![Discord](https://img.shields.io/badge/Discord-Join%20Us-5865F2?logo=discord)](https://discord.gg/stepfun)
+[![License](https://img.shields.io/badge/License-StepFun%20Community-blue)](https://huggingface.co/SHSLab/Step-5-Preview-BF16/blob/main/LICENSE)
+[![Model Size](https://img.shields.io/badge/Parameters-600B%20Total%20%7C%2027B%20Active-orange)]()
+[![Context](https://img.shields.io/badge/Context-1M%20Tokens-green)]()
+
+</div>
+
+<div style="background-color: #e6f7ff; padding: 16px; border-radius: 8px; border-left: 6px solid #1890ff; margin: 20px 0;">
+  <strong>🔥 Step-5-Preview is now available!</strong><br>
+  We are excited to release <strong>Step-5-Preview</strong>, our flagship foundation model for real-world agentic work. 
+  It is a 600B-parameter sparse Mixture-of-Experts model with 27B active parameters, a 1M-token context window, 
+  and native support for text, image, and video inputs. 
+  <br><br>
+  <strong>Weights are available now</strong> on Hugging Face. 
+  Try it via our API, or deploy locally with vLLM / SGLang.
+</div>
+
+---
+
+## 📖 Table of Contents
+
+- [Introduction](#-introduction)
+- [Key Features](#-key-features)
+- [Model Architecture](#-model-architecture)
+- [Model Specifications](#-model-specifications)
+- [Benchmark Results](#-benchmark-results)
+- [Agentic Capabilities](#-agentic-capabilities)
+- [Quickstart](#-quickstart)
+- [Deployment](#-deployment)
+- [Evaluation](#-evaluation)
+- [Citation](#-citation)
+- [License](#-license)
+- [Contact](#-contact)
+
+---
+
+## 🚀 Introduction
+
+**Step-5-Preview** is StepFun's flagship foundation model, designed from the ground up for **real-world agentic tasks**. 
+It targets professional domains such as **AI coding, software engineering, professional knowledge work, and financial analysis**. 
+
+StepFun's core philosophy for Step 5 is the **"Pareto Frontier"** — achieving the optimal balance between intelligence and cost. 
+While previous scaling efforts focused on trading more compute for stronger intelligence, the next phase requires improving the 
+**efficiency of converting compute into intelligence**.
+
+<div style="background-color: #fff7e6; padding: 16px; border-radius: 8px; border-left: 6px solid #fa8c16; margin: 20px 0;">
+  <strong>💡 Why Step 5 Preview?</strong><br>
+  • <strong>600B total parameters, only 27B active</strong> — near-frontier performance at a fraction of the compute.<br>
+  • <strong>1M-token context window</strong> without proportional cost increases.<br>
+  • <strong>Competitive benchmark scores</strong> against models with 3–5× more parameters.<br>
+  • <strong>Built for agents</strong> — long-horizon reasoning, tool use, and autonomous execution.
+</div>
+
+---
+
+## ✨ Key Features
+
+<div align="center">
+  <img src="https://raw.githubusercontent.com/stepfun-ai/Step-5-Preview/main/assets/features.png" alt="Key Features" width="90%">
+</div>
+
+- **Sparse Mixture-of-Experts (MoE):** 600B total parameters, 27B active per token (~4.5% sparsity).
+- **1M-Token Context Window:** Equivalent to ~1,500 A4 pages, enabled by Sparse GQA.
+- **Multimodal Input:** Text, image, and video (MP4, QuickTime, Matroska; ≤128 MB; ≤5 min recommended).
+- **Configurable Reasoning Effort:** `low`, `medium`, `high` / `xhigh`.
+- **Parallel Tool Calling:** Natively supported for agentic workflows.
+- **Strict JSON Schema Output:** Reliable integration into structured systems.
+- **OpenAI-Compatible API:** Available via Step API and third-party gateways.
+- **Open Weights:** BF16 checkpoint available now.
+
+---
+
+## 🏗️ Model Architecture
+
+<div align="center">
+  <img src="https://raw.githubusercontent.com/stepfun-ai/Step-5-Preview/main/assets/architecture.png" alt="Step 5 Architecture" width="85%">
+</div>
+
+### 92-Layer "Narrow but Deep" Design
+
+Step-5-Preview uses a **92-layer Transformer** with a narrow-deep configuration. This design is specifically intended to create 
+**longer information propagation paths** for implicit multi-hop reasoning during long prefill operations.
+
+### Sparse Grouped-Query Attention (GQA) with Block-Wise Token Merging
+
+To handle the 1M-token context window efficiently, Step-5-Preview introduces **Sparse GQA with block-wise token merging**. 
+This mechanism uses sparse indexing to select only historical information relevant to the current task, reducing the number of tokens 
+that actually enter attention computation. StepFun states this cuts indexer and top-k selection costs to approximately 
+**one-eighth** of a denser baseline.
+
+<div style="background-color: #f6ffed; padding: 16px; border-radius: 8px; border-left: 6px solid #52c41a; margin: 20px 0;">
+  <strong>⚡ Efficiency-First Scaling</strong><br>
+  Step 5 Preview achieves near-frontier performance with <strong>600B total parameters</strong> but only 
+  <strong>27B active per token</strong>. This is the core of StepFun's efficiency-first philosophy.
+</div>
+
+---
+
+## 📋 Model Specifications
+
+| Category | Specification |
+|:---|:---|
+| **Model Name** | Step-5-Preview |
+| **Developer** | StepFun |
+| **Architecture** | Sparse Mixture-of-Experts (MoE) |
+| **Total Parameters** | 600B |
+| **Active Parameters** | 27B per token (~4.5% sparsity) |
+| **Layers** | 92 (narrow-deep Transformer) |
+| **Context Window** | 1,000,000 tokens |
+| **Attention** | Sparse GQA with block-wise token merging |
+| **Input Modalities** | Text, Image, Video |
+| **Output Modalities** | Text |
+| **Video Formats** | MP4, QuickTime, Matroska (≤128 MB, ≤5 min recommended) |
+| **Reasoning Effort** | `low` / `medium` / `high` (`xhigh`) |
+| **Tool Calling** | Parallel, strict JSON schema |
+| **Intelligence Index** | 44 (Artificial Analysis v4.3.2) |
+| **Open Weights** | BF16 checkpoint available now |
+| **API Availability** | Immediate (OpenAI-compatible) |
+
+---
+
+## 📊 Benchmark Results
+
+### Artificial Analysis Intelligence Index
+
+<div align="center">
+  <img src="https://raw.githubusercontent.com/stepfun-ai/Step-5-Preview/main/assets/intelligence_index.png" alt="Intelligence Index" width="80%">
+</div>
+
+**Overall Score: 44** (Intelligence Index v4.3.2, recalibrated September 7, 2026)
+
+This places Step-5-Preview among the **top three open-weight models globally**, on par with models like 
+Kimi K3 Max (approximately 5× larger at 2.8T parameters) and Qwen3.8 Max. The index covers 10 evaluations including 
+AA-Briefcase, GDPval-AA v2, Terminal-Bench 4.0, SciCode, and Humanity's Last Exam.
+
+### Detailed Benchmark Scores
+
+<div style="background-color: #fafafa; padding: 16px; border-radius: 8px; border: 1px solid #e8e8e8; margin: 20px 0;">
+
+| Benchmark | Step-5-Preview (High) | Kimi K3 (Max) | GLM-5.3 (Max) | Claude Opus 5 (Max) | GPT-6 Astra (Max) |
+|:---|:---|:---|:---|:---|:---|
+| **DeepSWE v1.1** | **67.7** | 67.5 | 66.9 | 74.0 | 74.1 |
+| **StepCodeBench** | **49.0** | 43.9 | 40.2 | 63.9 | 61.0 |
+| **ProgramBench** | **80.5** | 77.8 | 72.0 | 82.3 | 85.4 |
+| **Terminal-Bench v4** | 33.3 | 12.6 | 41.9 | 52.3 | 57.9 |
+| **Agents' Last Exam (ALE-CLI)** | **29.5** | 27.6 | 28.6 | 28.6 | 33.3 |
+| **GDPval-AA v2** | 1571 | 1548 | 1634 | 1735 | 1580 |
+| **FrontierFinance** | **66.4** | 62.6 | 64.1 | 69.7 | 55.0 |
+| **DRACO** | **83.3** | 78.5 | 82.3 | 87.6 | 76.8 |
+
+</div>
+
+<details>
+<summary><strong>📝 Benchmark Methodology Notes</strong> (click to expand)</summary>
+
+- **DeepSWE v1.1** was evaluated using the SWE-agent harness with `temperature=1.0` and `top_p=0.95`.
+- **GDPval-AA v2** results are from Artificial Analysis as of September 19, 2026.
+- **StepCodeBench** achieved **49.0% avg@4**.
+- **SciCode**: Step-5-Preview scored higher than Kimi K3.
+- **Output Speed**: 99.8 tokens/sec (GLM-5.3: 72.1 tokens/sec).
+- **Time to First Token**: 2.96 seconds (GLM-5.3: 2.99s; Claude Opus 5: 56.84s at max effort).
+- **Terminal-Bench 4.0 vs Kimi K3**: 33.3% vs ~12.6%.
+- **Terminal-Bench 4.0 vs DeepSeek V4.1 Flash**: 33.3% vs 26.8%.
+
+</details>
+
+### Benchmark Takeaways
+
+<div style="background-color: #f0f5ff; padding: 16px; border-radius: 8px; border-left: 6px solid #2f54eb; margin: 20px 0;">
+  <strong>🧠 Coding & Software Engineering</strong><br>
+  Step-5-Preview <strong>leads all open-weight models</strong> on DeepSWE v1.1 and StepCodeBench, surpassing Kimi K3 and GLM-5.3. 
+  It trails only the larger closed-source models (Claude Opus 5 and GPT-6 Astra).
+</div>
+
+<div style="background-color: #fff1f0; padding: 16px; border-radius: 8px; border-left: 6px solid #f5222d; margin: 20px 0;">
+  <strong>🤖 Agentic Tasks</strong><br>
+  Strong performance on Terminal-Bench 4.0 (<strong>33.3%</strong>) and Agents' Last Exam (ALE-CLI) (<strong>29.5%</strong>). 
+  Terminal-Bench score is <strong>2.6× higher than Kimi K3</strong> and <strong>1.24× higher than DeepSeek V4.1 Flash</strong>.
+</div>
+
+<div style="background-color: #fcffe6; padding: 16px; border-radius: 8px; border-left: 6px solid #a0d911; margin: 20px 0;">
+  <strong>💰 Financial & Deep Research</strong><br>
+  Highly competitive on FrontierFinance and DRACO, nearly matching top closed-source models like Claude Opus 5. 
+  On FrontierFinance, it outperforms both Kimi K3 and GLM-5.3 by a significant margin.
+</div>
+
+---
+
+## 🤖 Agentic Capabilities
+
+<div align="center">
+  <img src="https://raw.githubusercontent.com/stepfun-ai/Step-5-Preview/main/assets/agentic_workflow.png" alt="Agentic Workflow" width="90%">
+</div>
+
+### 24-Hour Autonomous GPU Kernel Optimization
+
+In a landmark demonstration of sustained agentic execution, Step-5-Preview was tasked with **autonomously optimizing an H100 GPU kernel for up to 24 consecutive hours**. The model:
+
+- Independently modified code
+- Ran tests and compared results
+- Iterated based on performance outcomes
+- **Reached 508 TFLOPS after approximately 22 hours**
+
+For comparison, **Claude Opus 5 achieved 493 TFLOPS** in the same experiment. This demonstrates Step-5-Preview's ability to sustain productive work over extended periods without human intervention.
+
+### Automated Post-Training Experiments
+
+In another 24-hour experiment, Step-5-Preview autonomously improved the accuracy of **Qwen3-30B-A3B on AIME24 from 53.3% to 60%** through automated post-training experiments. This showcases the model's capacity for self-directed research and optimization.
+
+### Real-World Application Demonstrations
+
+StepFun demonstrated the model's capabilities across several complex, real-world projects:
+
+- **ESP32 Development Board Modifications:** Executed development tasks for over 3 hours, demonstrating hardware programming capabilities.
+- **Front-End Design with 3D Asset Generation:** Full-stack development workflows including visual design.
+- **Full-Process Financial Research:** End-to-end investment research workflows.
+- **Software Engineering:** Comprehensive coding tasks beyond traditional code generation, including front-end, visual development, and programmable hardware scenarios.
+
+### Long-Horizon Agent Workflows
+
+The model is specifically optimized for agent workflows that require:
+
+- Searching and information retrieval
+- Running code and processing tool returns
+- Multi-turn tool calls with sustained execution
+- Iterative refinement based on intermediate results
+
+---
+
+## ⚡ Quickstart
+
+### Installation
+
+```bash
+pip install transformers>=4.56.0
+pip install torch>=2.4.0
+pip install accelerate
+```
+
+For video/image support:
+
+```bash
+pip install av pillow
+```
+
+### Basic Usage with Transformers
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "SHSLab/Step-5-Preview-BF16"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    trust_remote_code=True,
+    device_map="auto",
+    torch_dtype="bfloat16",
+)
+
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": "Explain the significance of the Pareto Frontier in AI scaling."},
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt",
+).to(model.device)
+
+outputs = model.generate(
+    inputs,
+    max_new_tokens=1024,
+    temperature=0.7,
+    top_p=0.95,
+    reasoning_effort="high",  # low / medium / high / xhigh
+)
+
+response = tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)
+print(response)
+```
+
+### Multimodal (Image + Video) Usage
+
+```python
+from transformers import AutoProcessor
+
+processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
+
+messages = [
+    {
+        "role": "user",
+        "content": [
+            {"type": "image", "url": "https://example.com/image.jpg"},
+            {"type": "video", "url": "https://example.com/video.mp4"},
+            {"type": "text", "text": "Describe the scene and summarize the video."},
+        ],
+    }
+]
+
+inputs = processor.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
+# ... generate as above
+```
+
+### Tool Calling
+
+```python
+tools = [
+    {
+        "type": "function",
+        "function": {
+            "name": "get_weather",
+            "parameters": {
+                "type": "object",
+                "properties": {"city": {"type": "string"}},
+                "required": ["city"],
+            },
+        },
+    }
+]
+
+messages = [{"role": "user", "content": "What's the weather in Tokyo?"}]
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    tools=tools,
+    add_generation_prompt=True,
+    return_tensors="pt",
+).to(model.device)
+
+outputs = model.generate(inputs, max_new_tokens=256, reasoning_effort="medium")
+print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
+```
+
+---
+
+## 🚢 Deployment
+
+### vLLM
+
+```bash
+vllm serve SHSLab/Step-5-Preview-BF16 \
+  --trust-remote-code \
+  --tensor-parallel-size 8 \
+  --max-model-len 1000000 \
+  --enable-reasoning \
+  --reasoning-parser stepfun
+```
+
+### SGLang
+
+```bash
+python -m sglang.launch_server \
+  --model-path SHSLab/Step-5-Preview-BF16 \
+  --trust-remote-code \
+  --tp 8 \
+  --context-length 1000000 \
+  --reasoning-parser stepfun
+```
+
+### OpenAI-Compatible API
+
+```python
+from openai import OpenAI
+
+client = OpenAI(
+    api_key="YOUR_STEP_API_KEY",
+    base_url="https://api.stepfun.com/v1",
+)
+
+response = client.chat.completions.create(
+    model="step-5-preview",
+    messages=[{"role": "user", "content": "Write a Python function to merge two sorted lists."}],
+    reasoning_effort="high",
+    max_tokens=2048,
+)
+
+print(response.choices[0].message.content)
+```
+
+<div style="background-color: #f9f0ff; padding: 16px; border-radius: 8px; border-left: 6px solid #722ed1; margin: 20px 0;">
+  <strong>📦 Recommended Deployment Configurations</strong><br>
+  • <strong>BF16:</strong> 8× H100 80GB (tensor parallel)<br>
+  • <strong>FP8:</strong> 4× H100 80GB (coming soon)<br>
+  • <strong>Context length:</strong> Up to 1M tokens<br>
+  • <strong>Reasoning parser:</strong> Use <code>stepfun</code> for vLLM/SGLang
+</div>
+
+---
+
+## 📈 Evaluation
+
+Step-5-Preview was evaluated on a comprehensive suite of public and internal benchmarks. 
+All evaluations used the model's `high` reasoning effort setting unless otherwise noted.
+
+| Benchmark | Score | Notes |
+|:---|:---|:---|
+| **DeepSWE v1.1** | 67.7 | SWE-agent harness, temp=1.0, top_p=0.95 |
+| **StepCodeBench** | 49.0 | avg@4 |
+| **ProgramBench** | 80.5 | — |
+| **Terminal-Bench v4** | 33.3 | — |
+| **Agents' Last Exam (ALE-CLI)** | 29.5 | — |
+| **GDPval-AA v2** | 1571 | Artificial Analysis, Sep 19, 2026 |
+| **FrontierFinance** | 66.4 | — |
+| **DRACO** | 83.3 | — |
+| **SciCode** | Higher than Kimi K3 | — |
+| **Output Speed** | 99.8 tokens/sec | GLM-5.3: 72.1 tokens/sec |
+| **Time to First Token** | 2.96s | GLM-5.3: 2.99s; Claude Opus 5: 56.84s (max effort) |
+
+---
+
+## 📚 Citation
+
+If you use Step-5-Preview in your research, please cite:
+
+```bibtex
+@misc{stepfun2026step5preview,
+  title        = {Step-5-Preview: A 600B Sparse MoE Foundation Model for Real-World Agentic Work},
+  author       = {StepFun Team},
+  year         = {2026},
+  howpublished = {\url{https://huggingface.co/SHSLab/Step-5-Preview-BF16}},
+  note         = {Released September 20, 2026}
+}
+```
+
+---
+
+## 📜 License
+
+Step-5-Preview is released under the **StepFun Community License**. 
+See the [LICENSE](https://huggingface.co/SHSLab/Step-5-Preview-BF16/blob/main/LICENSE) file for full terms.
+
+<div style="background-color: #fffbe6; padding: 16px; border-radius: 8px; border-left: 6px solid #faad14; margin: 20px 0;">
+  <strong>⚠️ Usage Restrictions</strong><br>
+  • Commercial use is permitted under the StepFun Community License.<br>
+  • Redistribution must include the license and attribution.<br>
+  • See LICENSE for full details.
+</div>
+
+---
+
+## 📬 Contact
+
+- **Hugging Face:** [SHSLab](https://huggingface.co/SHSLab)
+- **GitHub:** [github.com/stepfun-ai](https://github.com/stepfun-ai)
+- **Discord:** [Join our Discord](https://discord.gg/stepfun)
+- **Email:** [opensource@stepfun.com](mailto:opensource@stepfun.com)
+- **Website:** [stepfun.com](https://stepfun.com)
+
+---
+
+<div align="center">
+  <strong>⭐ If you find Step-5-Preview useful, please give us a star on GitHub and Hugging Face! ⭐</strong>
+  <br><br>
+  <em>Built with ❤️ by StepFun</em>
+</div>