Update README.md

README.md

@@ -1,3 +1,36 @@
+---
+language:
+- en
+- zh
+- multilingual
+license: other
+license_name: stepfun-community-license
+license_link: https://huggingface.co/SHSLab/Step-5-Preview-BF16/blob/main/LICENSE
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- stepfun
+- step-5
+- moe
+- mixture-of-experts
+- agentic
+- coding
+- software-engineering
+- long-context
+- 1m-context
+- multimodal
+- text-generation
+- image
+- video
+- sparse-attention
+- gqa
+- financial-analysis
+- deep-research
+- tool-calling
+- parallel-tool-calling
+- json-schema
+---
+
 # Step-5-Preview
 
 <div align="center">
@@ -15,15 +48,11 @@
 
 </div>
 
-<div style="border-left: 6px solid #1890ff; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>🔥 Step-5-Preview is now available!</strong><br>
-  We are excited to release <strong>Step-5-Preview</strong>, our flagship foundation model for real-world agentic work. 
-  It is a 600B-parameter sparse Mixture-of-Experts model with 27B active parameters, a 1M-token context window, 
-  and native support for text, image, and video inputs. 
-  <br><br>
-  <strong>Weights are available now</strong> on Hugging Face (<code>SHSLab/Step-5-Preview-BF16</code>). 
-  Try it via our API, or deploy locally with vLLM / SGLang.
-</div>
+> **🔥 Step-5-Preview is now available.**
+>
+> Step-5-Preview is a 600B-parameter sparse Mixture-of-Experts model with 27B active parameters, a 1M-token context window, and native support for text, image, and video inputs.
+>
+> Weights are available on Hugging Face (`SHSLab/Step-5-Preview-BF16`). Available via Step API, or self-hosted with vLLM / SGLang.
 
 ---
 
@@ -31,44 +60,30 @@
 
 - [Introduction](#-introduction)
 - [Key Features](#-key-features)
-- [Model Architecture](#-model-architecture)
 - [Model Specifications](#-model-specifications)
-- [Training Data](#-training-data)
 - [Benchmark Results](#-benchmark-results)
-- [Agentic Capabilities](#-agentic-capabilities)
-- [Real-World Use Cases](#-real-world-use-cases)
 - [Quickstart](#-quickstart)
 - [Deployment](#-deployment)
-- [Evaluation](#-evaluation)
-- [Limitations](#-limitations)
-- [Ethical Considerations](#-ethical-considerations)
-- [Hardware Requirements](#-hardware-requirements)
-- [Performance Metrics](#-performance-metrics)
-- [Citation](#-citation)
 - [License](#-license)
 - [Contact](#-contact)
+- [More details](#-more-details) *(architecture, training, agentic demos, evaluation, limitations)*
 
 ---
 
 ## 🚀 Introduction
 
-**Step-5-Preview** is StepFun's flagship foundation model, designed from the ground up for **real-world agentic tasks**. 
-It targets professional domains such as **AI coding, software engineering, professional knowledge work, and financial analysis**. 
+**Step-5-Preview** is StepFun's flagship foundation model, designed for **real-world agentic tasks**. It targets professional domains such as **AI coding, software engineering, professional knowledge work, and financial analysis**.
 
-StepFun's core philosophy for Step 5 is the **"Pareto Frontier"** — achieving the optimal balance between intelligence and cost. 
-While previous scaling efforts focused on trading more compute for stronger intelligence, the next phase requires improving the 
-**efficiency of converting compute into intelligence**.
+StepFun's core philosophy for Step 5 is the **"Pareto Frontier"** — balancing intelligence against cost. While earlier scaling efforts traded more compute for stronger intelligence, the next phase focuses on improving the **efficiency of converting compute into intelligence**.
 
-<div style="border-left: 6px solid #fa8c16; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>💡 Why Step 5 Preview?</strong><br>
-  • <strong>600B total parameters, only 27B active</strong> — near-frontier performance at a fraction of the compute.<br>
-  • <strong>1M-token context window</strong> without proportional cost increases.<br>
-  • <strong>Competitive benchmark scores</strong> against models with 3–5× more parameters.<br>
-  • <strong>Built for agents</strong> — long-horizon reasoning, tool use, and autonomous execution.
-</div>
+> **Why Step 5 Preview?**
+>
+> - 600B total parameters, 27B active per token — near-frontier performance at a fraction of the compute.
+> - 1M-token context window without proportional cost increases.
+> - Competitive benchmark scores against models with 3–5× more parameters.
+> - Built for agents: long-horizon reasoning, tool use, and autonomous execution.
 
-Step-5-Preview represents a generational leap, with StepFun **skipping the entire Step 4.x line** entirely, going directly from 
-Step-3.7-Flash to Step 5. This decision reflects the magnitude of improvement achieved in this release.
+Step-5-Preview skips the entire Step 4.x line, going directly from Step-3.7-Flash to Step 5.
 
 ---
 
@@ -85,39 +100,7 @@ Step-3.7-Flash to Step 5. This decision reflects the magnitude of improvement ac
 - **Parallel Tool Calling:** Natively supported for agentic workflows.
 - **Strict JSON Schema Output:** Reliable integration into structured systems.
 - **OpenAI-Compatible API:** Available via Step API and third-party gateways.
-- **Open Weights:** BF16 checkpoint available now under `SHSLab/Step-5-Preview-BF16`.
-
----
-
-## 🏗️ Model Architecture
-
-<div align="center">
-  <img src="./Step-5/architecture.png" alt="Step 5 Architecture" width="85%">
-</div>
-
-### 92-Layer "Narrow but Deep" Design
-
-Step-5-Preview uses a **92-layer Transformer** with a narrow-deep configuration. This design is specifically intended to create 
-**longer information propagation paths** for implicit multi-hop reasoning during long prefill operations.
-
-### Sparse Grouped-Query Attention (GQA) with Block-Wise Token Merging
-
-To handle the 1M-token context window efficiently, Step-5-Preview introduces **Sparse GQA with block-wise token merging**. 
-This mechanism uses sparse indexing to select only historical information relevant to the current task, reducing the number of tokens 
-that actually enter attention computation. StepFun states this cuts indexer and top-k selection costs to approximately 
-**one-eighth** of a denser baseline.
-
-<div style="border-left: 6px solid #52c41a; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>⚡ Efficiency-First Scaling</strong><br>
-  Step 5 Preview achieves near-frontier performance with <strong>600B total parameters</strong> but only 
-  <strong>27B active per token</strong>. This is the core of StepFun's efficiency-first philosophy.
-</div>
-
-### Multimodal Encoder
-
-The model incorporates a unified multimodal encoder that processes text, images, and video frames into a shared latent space. 
-Video is sampled at adaptive frame rates and encoded with temporal attention, allowing the model to understand motion and 
-long-range dependencies in screen recordings, demonstrations, and real-world footage.
+- **Open Weights:** BF16 checkpoint available under `SHSLab/Step-5-Preview-BF16`.
 
 ---
 
@@ -139,29 +122,12 @@ long-range dependencies in screen recordings, demonstrations, and real-world foo
 | **Reasoning Effort** | `low` / `medium` / `high` (`xhigh`) |
 | **Tool Calling** | Parallel, strict JSON schema |
 | **Intelligence Index** | 44 (Artificial Analysis v4.3.2) |
-| **Open Weights** | BF16 checkpoint available now |
+| **Open Weights** | BF16 checkpoint available |
 | **API Availability** | Immediate (OpenAI-compatible) |
 | **License** | StepFun Community License |
 
 ---
 
-## 📚 Training Data
-
-Step-5-Preview was trained on a massive, carefully curated corpus spanning:
-
-- **Code repositories** from multiple languages (Python, C++, Rust, JavaScript, Go, etc.)
-- **Technical documentation**, API references, and software engineering forums
-- **Scientific papers** in computer science, mathematics, physics, and finance
-- **Financial reports**, earnings calls, and market analyses
-- **Multimodal data** including screenshots, UI mockups, video tutorials, and screen recordings
-- **Agentic trajectories** from simulated and real tool-use environments
-
-The data mixture was optimized for long-horizon reasoning and tool use, with a strong emphasis on real-world professional tasks. 
-All data was filtered for quality, safety, and license compliance. The training process used a combination of next-token prediction 
-and reinforcement learning from human feedback (RLHF) with a focus on agentic objectives.
-
----
-
 ## 📊 Benchmark Results
 
 <div align="center">
@@ -172,27 +138,18 @@ and reinforcement learning from human feedback (RLHF) with a focus on agentic ob
 
 **Overall Score: 44** (Intelligence Index v4.3.2, recalibrated September 7, 2026)
 
-This places Step-5-Preview among the **top open-weight models globally**, on par with models like 
-Kimi K3 Max (approximately 5× larger at 2.8T parameters) and Qwen3.8 Max. The index covers 10 evaluations including 
-AA-Briefcase, GDPval-AA v2, Terminal-Bench 4.0, SciCode, and Humanity's Last Exam.
+The index covers 10 evaluations including AA-Briefcase, GDPval-AA v2, Terminal-Bench 4.0, SciCode, and Humanity's Last Exam.
 
-<div style="border: 1px solid #d9d9d9; border-radius: 12px; padding: 20px 24px; margin: 24px 0; background: linear-gradient(135deg, #fafafa 0%, #f0f5ff 100%);">
-
-<h4 style="margin-top:0;">📐 How to read these tables</h4>
-
-- <strong>Step-5-Preview</strong> is always the first data column, for fast scanning.
-- <strong>Bold</strong> marks the <strong>best score in the row</strong> across all six models.
-- <strong>🥇</strong> flags the category leader for that benchmark.
-- <strong>—</strong> indicates the model was not evaluated or did not report a score.
-- All Step-5-Preview results use the <code>high</code> reasoning-effort setting unless noted.
-
-<strong>Comparison set:</strong> Step-5-Preview · GPT-6 Astra (Max) · Fable 5.1 · Claude Opus 5 (Max) · Kimi K3 (Max) · GLM-5.3 (Max)
-
-<strong>Open-weight models in this comparison:</strong> Step-5-Preview, Kimi K3 (Max), GLM-5.3 (Max), and Qwen3.8 Max (referenced in the index).  
-<strong>Closed-source models:</strong> GPT-6 Astra (Max), Claude Opus 5 (Max).  
-<em>Fable 5.1: availability not specified in this document.</em>
-
-</div>
+> **How to read the tables below**
+>
+> - **Bold** marks the best score in the row across all six models.
+> - **🥇** flags the leader for that benchmark.
+> - **—** indicates the model was not evaluated or did not report a score.
+> - Step-5-Preview results use the `high` reasoning-effort setting unless noted.
+>
+> **Comparison set:** Step-5-Preview · GPT-6 Astra (Max) · Fable 5.1 · Claude Opus 5 (Max) · Kimi K3 (Max) · GLM-5.3 (Max)
+>
+> **Availability:** Open-weight — Step-5-Preview, Kimi K3, Qwen3.8 Max, GLM-5.3. Closed-source — GPT-6 Astra, Claude Opus 5. Fable 5.1 — not publicly stated.
 
 ---
 
@@ -205,14 +162,6 @@ AA-Briefcase, GDPval-AA v2, Terminal-Bench 4.0, SciCode, and Humanity's Last Exa
 | **AA-LCR v1.1** | 88.3% | 80.7% | 85.3% | 79.3% | **88.7%** 🥇 | 79.7% |
 | **CritPt** | 20.9% | **31.7%** 🥇 | 29.7% | 29.1% | 23.4% | 19.1% |
 
-<div style="border-left: 6px solid #2f54eb; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>📌 Read-out</strong><br>
-  Step-5-Preview sits in the <strong>top tier of open-weight reasoning</strong>: GPQA Diamond <strong>93.5%</strong> matches Kimi K3 
-  exactly and lands within 0.5 points of Fable 5.1. On <strong>AA-LCR v1.1</strong> — long-context reasoning — it scores 
-  <strong>88.3%</strong>, effectively tied with Kimi K3 (88.7%) and <strong>+7.6 points ahead of GPT-6 Astra</strong> and 
-  <strong>+9.0 points ahead of Claude Opus 5</strong>, despite those models being far larger.
-</div>
-
 ---
 
 ### 💻 Coding & Software Engineering
@@ -234,15 +183,6 @@ AA-Briefcase, GDPval-AA v2, Terminal-Bench 4.0, SciCode, and Humanity's Last Exa
 | **StepCode-Bench-Daily** | 64.9% | — | — | **77.6%** 🥇 | 57.7% | 69.1% |
 | **StepCode-Bench-General** | 65.0% | 64.3% | — | **68.3%** 🥇 | 65.2% | 62.0% |
 
-<div style="border-left: 6px solid #13c2c2; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>📌 Read-out</strong><br>
-  Among <strong>open-weight models</strong>, Step-5-Preview leads on <strong>DeepSWE v1.1</strong> (67.7% vs. Kimi K3 67.5% and 
-  GLM-5.3 66.9%), <strong>StepCodeBench</strong> (49.0% vs. 43.9% and 40.2%), <strong>ProgramBench</strong> (80.5% vs. 77.8% and 72.0%), 
-  and <strong>SWE-Atlas-QnA</strong> (63.6%). It also posts the best <strong>CyberGym</strong> score in the comparison set at 
-  <strong>84.7%</strong>. On <strong>Terminal-Bench 4</strong> it reaches <strong>33.3%</strong> — 
-  <strong>2.6× Kimi K3</strong> (12.6%) — though the larger closed models still hold the absolute lead.
-</div>
-
 ---
 
 ### 🤖 Agents, Tool Use & Automation
@@ -263,14 +203,6 @@ AA-Briefcase, GDPval-AA v2, Terminal-Bench 4.0, SciCode, and Humanity's Last Exa
 | **BrowseComp** | 88.7% | **91.5%** 🥇 | — | 90.2% | 91.2% | — |
 | **HLE w/ tools** | 59.4% | 57.2% | **65.0%** 🥇 | 63.6% | 56.0% | 62.5% |
 
-<div style="border-left: 6px solid #f5222d; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>📌 Read-out</strong><br>
-  Step-5-Preview is a <strong>strong mid-frontier agentic model</strong>. It outperforms GPT-6 Astra on <strong>τ³-Banking</strong> 
-  (42.5% vs. 41.4%) and <strong>Draco</strong> (83.3% vs. 76.8%), and beats Kimi K3 on <strong>MCP-Atlas</strong> (85.6% vs. 85.3%) 
-  and <strong>Draco</strong> (83.3% vs. 78.5%). With tools, HLE rises from <strong>46.5% → 59.4%</strong>, a 
-  <strong>+12.9-point</strong> tool-augmentation gain.
-</div>
-
 ---
 
 ### 💰 Finance & Professional Work
@@ -282,18 +214,9 @@ AA-Briefcase, GDPval-AA v2, Terminal-Bench 4.0, SciCode, and Humanity's Last Exa
 | **FinStepBench-FinanceDR** | 55.8% | 45.0% | — | **59.1%** 🥇 | 48.9% | 53.3% |
 | **FrontierFinance** | 66.4% | 55.0% | — | **69.7%** 🥇 | 62.6% | 64.1% |
 | **OfficeQA Pro** | 60.3% | **67.7%** 🥇 | — | 64.7% | 62.6% | 59.1% |
-| **Spreadsheet v2** | 29.4% | 31.4% | — | **32.8%** 🥇 | 31.9% | 30.5% |
+| **SpeadSheet v2** | 29.4% | 31.4% | — | **32.8%** 🥇 | 31.9% | 30.5% |
 | **GDP.pdf** | 14.8% | **31.0%** 🥇 | 26.2% | 21.6% | 22.0% | 11.2% |
 
-<div style="border-left: 6px solid #a0d911; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>📌 Read-out</strong><br>
-  Finance is a Step-5-Preview <strong>strength</strong>. On <strong>FrontierFinance</strong> it scores <strong>66.4%</strong>, 
-  beating GPT-6 Astra by <strong>+11.4 points</strong>, Kimi K3 by <strong>+3.8</strong>, and GLM-5.3 by <strong>+2.3</strong> — 
-  trailing only Claude Opus 5. On <strong>FinStepBench-FinanceDR</strong> it reaches <strong>55.8%</strong>, ahead of 
-  GPT-6 Astra (+10.8), Kimi K3 (+6.9), and GLM-5.3 (+2.5). <strong>FinStepBench-LiveSearch</strong> lands at 
-  <strong>74.5%</strong>, tied with GPT-6 Astra.
-</div>
-
 ---
 
 ### 👁️ Multimodal
@@ -303,121 +226,22 @@ AA-Briefcase, GDPval-AA v2, Terminal-Bench 4.0, SciCode, and Humanity's Last Exa
 | **MMMU-Pro** | 76.0% | **87.0%** 🥇 | — | 85.0% | 81.0% | — |
 | **GDP.pdf** | 14.8% | **31.0%** 🥇 | 26.2% | 21.6% | 22.0% | 11.2% |
 
-<div style="border-left: 6px solid #faad14; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>📌 Read-out</strong><br>
-  Multimodal reasoning is the <strong>most direct improvement opportunity</strong> for the next release. MMMU-Pro at 
-  <strong>76.0%</strong> trails GPT-6 Astra (87.0%), Claude Opus 5 (85.0%), and Kimi K3 (81.0%), and document-heavy 
-  visual QA (<strong>GDP.pdf</strong>) remains a known weak spot at <strong>14.8%</strong>.
-</div>
-
----
-
-### 🏅 Category Leadership Summary
-
-<div style="border: 1px solid #d9d9d9; border-radius: 12px; padding: 20px 24px; margin: 24px 0; background: linear-gradient(135deg, #ffffff 0%, #f6ffed 100%);">
-
-| Domain | Step-5-Preview Standing | Highlight |
-|:---|:---|:---|
-| **Long-Context Reasoning** | 🥈 Near-tied for #1 overall | AA-LCR v1.1 **88.3%** (+7.6 over GPT-6 Astra, +9.0 over Opus 5) |
-| **Coding / SWE (open-weight)** | 🥇 #1 open-weight | DeepSWE v1.1 **67.7%**, StepCodeBench **49.0%**, ProgramBench **80.5%** |
-| **Security / Cyber** | 🥇 #1 in set | CyberGym **84.7%** |
-| **Finance** | 🥈 #2 overall, #1 open-weight | FrontierFinance **66.4%**, FinanceDR **55.8%** |
-| **Terminal Agents** | 🥈 #2 open-weight | Terminal-Bench 4 **33.3%** (GLM-5.3 41.9% > Step 33.3% > Kimi 12.6%) |
-| **Tool Use** | 🥉 Competitive | MCP-Atlas **85.6%**, HLE w/ tools **59.4%** |
-| **General Knowledge** | Top tier | GPQA Diamond **93.5%** (tied #3), HLE **46.5%** (#5) |
-| **Multimodal** | 🔻 Trailing | MMMU-Pro **76.0%** — targeted for improvement |
-
-</div>
-
 <details>
-<summary><strong>📝 Benchmark Methodology Notes</strong> (click to expand)</summary>
+<summary><strong>📝 Benchmark methodology notes</strong></summary>
 
 - **DeepSWE v1.1** was evaluated using the SWE-agent harness with `temperature=1.0` and `top_p=0.95`.
 - **StepCodeBench** achieved **49.0% avg@4**.
 - **GDPval-AA v2** results are from Artificial Analysis as of September 19, 2026.
 - **AA-LCR v1.1**: Step-5-Preview scored **88.3%**, statistically tied with Kimi K3 (88.7%).
-- **Terminal-Bench 4**: Step-5-Preview **33.3%** vs. Kimi K3 **~12.6%** (2.6×) and DeepSeek V4.1 Flash **26.8%** (1.24×).
-- **SciCode**: Step-5-Preview scored **58.9%**, above GPT-6 Astra (56.5%) and Claude Opus 5 (56.4%).
+- **Terminal-Bench 4**: Step-5-Preview **33.3%** vs. Kimi K3 ~**12.6%** (2.6×) and DeepSeek V4.1 Flash **26.8%** (1.24×).
+- **SciCode**: Step-5-Preview **58.9%**, above GPT-6 Astra (56.5%) and Claude Opus 5 (56.4%).
 - **Multimodal**: MMMU-Pro and GDP.pdf were run with the unified multimodal encoder at default resolution.
 - **Output Speed**: 99.8 tokens/sec (GLM-5.3: 72.1 tokens/sec).
 - **Time to First Token**: 2.96 seconds (GLM-5.3: 2.99s; Claude Opus 5: 56.84s at max effort).
 - Missing entries (**—**) reflect benchmarks that were not publicly reported for that model at the time of writing.
-- Detailed benchmark descriptions in the Evaluation table are sourced from public benchmark documentation.
 
 </details>
 
-### Benchmark Takeaways
-
-<div style="border-left: 6px solid #2f54eb; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>🧠 Long-Context Reasoning</strong><br>
-  AA-LCR v1.1 at <strong>88.3%</strong> is one of Step-5-Preview's standout results — effectively tied for first with Kimi K3 
-  and well ahead of models many times its size. This validates the 92-layer narrow-deep design and Sparse GQA.
-</div>
-
-<div style="border-left: 6px solid #13c2c2; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>💻 Coding & Software Engineering</strong><br>
-  Step-5-Preview <strong>leads all open-weight models</strong> on DeepSWE v1.1, StepCodeBench, ProgramBench, and SWE-Atlas-QnA, 
-  surpassing Kimi K3 and GLM-5.3. It trails only the larger closed-source models (Claude Opus 5 and GPT-6 Astra).
-</div>
-
-<div style="border-left: 6px solid #f5222d; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>🤖 Agentic Tasks</strong><br>
-  Strong performance on Terminal-Bench 4 (<strong>33.3%</strong>) and HLE with tools (<strong>59.4%</strong>). 
-  The Terminal-Bench score is <strong>2.6× higher than Kimi K3</strong> and <strong>1.24× higher than DeepSeek V4.1 Flash</strong>.
-</div>
-
-<div style="border-left: 6px solid #a0d911; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>💰 Financial & Deep Research</strong><br>
-  Highly competitive on FrontierFinance (<strong>66.4%</strong>) and FinStepBench-FinanceDR (<strong>55.8%</strong>), 
-  nearly matching top closed-source models like Claude Opus 5 and outperforming both GPT-6 Astra and Kimi K3 by significant margins.
-</div>
-
----
-
-## 🤖 Agentic Capabilities
-
-<div align="center">
-  <img src="./Step-5/agentic_workflow.png" alt="Agentic Workflow" width="90%">
-</div>
-
-### 24-Hour Autonomous GPU Kernel Optimization
-
-In a landmark demonstration of sustained agentic execution, Step-5-Preview was tasked with **autonomously optimizing an H100 GPU kernel for up to 24 consecutive hours**. The model:
-
-- Independently modified code
-- Ran tests and compared results
-- Iterated based on performance outcomes
-- **Reached 508 TFLOPS after approximately 22 hours**
-
-For comparison, **Claude Opus 5 achieved 493 TFLOPS** in the same experiment. This demonstrates Step-5-Preview's ability to sustain productive work over extended periods without human intervention.
-
-### Automated Post-Training Experiments
-
-In another 24-hour experiment, Step-5-Preview autonomously improved the accuracy of **Qwen3-30B-A3B on AIME24 from 53.3% to 60%** through automated post-training experiments. This showcases the model's capacity for self-directed research and optimization.
-
-### Long-Horizon Agent Workflows
-
-The model is specifically optimized for agent workflows that require:
-
-- Searching and information retrieval
-- Running code and processing tool returns
-- Multi-turn tool calls with sustained execution
-- Iterative refinement based on intermediate results
-- Self-correction and error recovery over thousands of steps
-
----
-
-## 💼 Real-World Use Cases
-
-StepFun demonstrated the model's capabilities across several complex, real-world projects:
-
-- **ESP32 Development Board Modifications:** Executed development tasks for over 3 hours, demonstrating hardware programming capabilities.
-- **Front-End Design with 3D Asset Generation:** Full-stack development workflows including visual design.
-- **Full-Process Financial Research:** End-to-end investment research workflows, from data gathering to report generation.
-- **Software Engineering:** Comprehensive coding tasks beyond traditional code generation, including front-end, visual development, and programmable hardware scenarios.
-- **Autonomous Research Assistant:** Capable of reading papers, running experiments, and summarizing findings.
-- **Customer Support Automation:** Handles multi-turn conversations with tool calls to internal systems.
-
 ---
 
 ## ⚡ Quickstart
@@ -572,20 +396,126 @@ response = client.chat.completions.create(
 print(response.choices[0].message.content)
 ```
 
-<div style="border-left: 6px solid #722ed1; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>📦 Recommended Deployment Configurations</strong><br>
-  • <strong>BF16:</strong> 8× H100 80GB (tensor parallel)<br>
-  • <strong>FP8:</strong> 4× H100 80GB (coming soon)<br>
-  • <strong>Context length:</strong> Up to 1M tokens<br>
-  • <strong>Reasoning parser:</strong> Use <code>stepfun</code> for vLLM/SGLang
-</div>
+> **Recommended deployment configurations**
+>
+> - **BF16:** 8× H100 80GB (tensor parallel)
+> - **FP8:** 4× H100 80GB (coming soon)
+> - **Context length:** Up to 1M tokens
+> - **Reasoning parser:** Use `stepfun` for vLLM / SGLang
+
+---
+
+## 📜 License
+
+Step-5-Preview is released under the **StepFun Community License**. See the [LICENSE](https://huggingface.co/SHSLab/Step-5-Preview-BF16/blob/main/LICENSE) file for full terms.
+
+> **Usage restrictions**
+>
+> - Commercial use is permitted under the StepFun Community License.
+> - Redistribution must include the license and attribution.
+> - See LICENSE for full details.
+
+---
+
+## 📬 Contact
+
+- **Hugging Face:** [SHSLab](https://huggingface.co/SHSLab)
+- **GitHub:** [github.com/stepfun-ai](https://github.com/stepfun-ai)
+- **Discord:** [Join our Discord](https://discord.gg/stepfun)
+- **Email:** [opensource@stepfun.com](mailto:opensource@stepfun.com)
+- **Website:** [stepfun.com](https://stepfun.com)
 
 ---
 
-## 📈 Evaluation
+## 📚 More details
+
+<details>
+<summary><strong>🏗️ Model Architecture</strong></summary>
+
+<div align="center">
+  <img src="./Step-5/architecture.png" alt="Step 5 Architecture" width="85%">
+</div>
+
+### 92-Layer "Narrow but Deep" Design
+
+Step-5-Preview uses a **92-layer Transformer** with a narrow-deep configuration. This design creates longer information propagation paths for implicit multi-hop reasoning during long prefill operations.
+
+### Sparse Grouped-Query Attention (GQA) with Block-Wise Token Merging
+
+To handle the 1M-token context window efficiently, Step-5-Preview uses **Sparse GQA with block-wise token merging**. This mechanism uses sparse indexing to select only historical information relevant to the current task, reducing the number of tokens that enter attention computation. This cuts indexer and top-k selection costs to approximately one-eighth of a denser baseline.
+
+### Multimodal Encoder
+
+A unified multimodal encoder processes text, images, and video frames into a shared latent space. Video is sampled at adaptive frame rates and encoded with temporal attention, allowing the model to understand motion and long-range dependencies in screen recordings, demonstrations, and real-world footage.
+
+</details>
+
+<details>
+<summary><strong>📚 Training Data</strong></summary>
+
+Step-5-Preview was trained on a carefully curated corpus spanning:
+
+- **Code repositories** from multiple languages (Python, C++, Rust, JavaScript, Go, etc.)
+- **Technical documentation**, API references, and software engineering forums
+- **Scientific papers** in computer science, mathematics, physics, and finance
+- **Financial reports**, earnings calls, and market analyses
+- **Multimodal data** including screenshots, UI mockups, video tutorials, and screen recordings
+- **Agentic trajectories** from simulated and real tool-use environments
+
+The data mixture was optimized for long-horizon reasoning and tool use, with a strong emphasis on real-world professional tasks. All data was filtered for quality, safety, and license compliance. Training combined next-token prediction with reinforcement learning from human feedback (RLHF) focused on agentic objectives.
+
+</details>
+
+<details>
+<summary><strong>🤖 Agentic Capabilities</strong></summary>
+
+<div align="center">
+  <img src="./Step-5/agentic_workflow.png" alt="Agentic Workflow" width="90%">
+</div>
+
+### 24-Hour Autonomous GPU Kernel Optimization
+
+Step-5-Preview was tasked with autonomously optimizing an H100 GPU kernel for up to 24 consecutive hours. The model:
+
+- Independently modified code
+- Ran tests and compared results
+- Iterated based on performance outcomes
+- **Reached 508 TFLOPS after approximately 22 hours**
+
+For comparison, Claude Opus 5 achieved 493 TFLOPS in the same experiment.
+
+### Automated Post-Training Experiments
+
+In another 24-hour experiment, Step-5-Preview autonomously improved the accuracy of Qwen3-30B-A3B on AIME24 from 53.3% to 60% through automated post-training experiments.
+
+### Long-Horizon Agent Workflows
 
-Step-5-Preview was evaluated on a comprehensive suite of public and internal benchmarks. 
-All evaluations used the model's `high` reasoning effort setting unless otherwise noted.
+Optimized for workflows that require:
+
+- Searching and information retrieval
+- Running code and processing tool returns
+- Multi-turn tool calls with sustained execution
+- Iterative refinement based on intermediate results
+- Self-correction and error recovery over thousands of steps
+
+</details>
+
+<details>
+<summary><strong>💼 Real-World Use Cases</strong></summary>
+
+- **ESP32 Development Board Modifications:** Executed development tasks for over 3 hours.
+- **Front-End Design with 3D Asset Generation:** Full-stack development workflows including visual design.
+- **Full-Process Financial Research:** End-to-end investment research, from data gathering to report generation.
+- **Software Engineering:** Comprehensive coding tasks including front-end, visual development, and programmable hardware.
+- **Autonomous Research Assistant:** Reading papers, running experiments, and summarizing findings.
+- **Customer Support Automation:** Multi-turn conversations with tool calls to internal systems.
+
+</details>
+
+<details>
+<summary><strong>📈 Full Evaluation Table</strong></summary>
+
+All evaluations used the `high` reasoning effort setting unless otherwise noted.
 
 | Benchmark | Score | Notes |
 |:---|:---|:---|
@@ -603,60 +533,79 @@ All evaluations used the model's `high` reasoning effort setting unless otherwis
 | **SWE-Atlas-Test-writing** | 50.8% | 90 tasks on writing production-grade unit, integration, and acceptance tests for real repositories |
 | **StepCode-Bench-Daily** | 64.9% | StepFun internal; 553 repos, 9 task types, 20 domains, 33 languages; daily-difficulty slice |
 | **StepCode-Bench-General** | 65.0% | StepFun internal; same corpus as StepCodeBench; general-difficulty slice |
-| **Agents' Last Exam (ALE-CLI)** | 29.5% | Linux-only CLI subset of ALE; 40 industry subfields, tasks taking hours to weeks; best agent pass rate ~25.2% |
+| **Agents' Last Exam (ALE-CLI)** | 29.5% | Linux-only CLI subset of ALE; 40 industry subfields; best agent pass rate ~25.2% |
 | **GDPval-AA v2** | 1571 | Artificial Analysis, Sep 19, 2026 |
-| **AA-Briefcase** | 1417 | Elo score; agentic knowledge work across data science, product, banking, heavy industry; private held-out test set |
+| **AA-Briefcase** | 1417 | Elo; agentic knowledge work across data science, product, banking, heavy industry; private held-out test set |
 | **Toolathlon-Verified** | 74.1% | 108 expert-authored tasks; multi-app workflows averaging ~20 turns; strictly verifiable via scripts |
 | **MCP-Atlas** | 85.6% | 1,000 tasks (500 public + 500 private); 36 real MCP servers, 220 tools; 3–6 tool calls/task |
-| **PresentBench** | 76.8% | 238 slide-generation instances; avg 54.1 rubric checklist items per instance; evaluated on grounded, fine-grained slide quality |
-| **JobBench** | 59.0% | 130 tasks across 35 occupations; expert-prioritized delegation workflows; avg 35.6 binary rubric criteria per task |
-| **Apex-Agents** | 37.8% | Long-horizon tasks in investment banking, consulting, and corporate law; frontier models complete <25% of tasks |
-| **DRACO** | 83.3% | Cross-domain deep research benchmark; accuracy, completeness, objectivity, citation quality; LLM-as-judge with binary rubric verdicts |
-| **BrowseComp** | 88.7% | 1,266 hard-to-find web information retrieval questions; requires persistent browsing across many sites |
+| **PresentBench** | 76.8% | 238 slide-generation instances; avg 54.1 rubric checklist items per instance |
+| **JobBench** | 59.0% | 130 tasks across 35 occupations; avg 35.6 binary rubric criteria per task |
+| **Apex-Agents** | 37.8% | Long-horizon tasks in investment banking, consulting, corporate law |
+| **DRACO** | 83.3% | Cross-domain deep research; accuracy, completeness, objectivity, citation quality |
+| **BrowseComp** | 88.7% | 1,266 hard-to-find web information retrieval questions |
 | **HLE w/ tools** | 59.4% | +12.9 pts over no-tools HLE |
 | **FrontierFinance** | 66.4% | +11.4 pts over GPT-6 Astra |
 | **FinStepBench-LiveSearch** | 74.5% | Tied with GPT-6 Astra |
 | **FinStepBench-CorporateValuation** | 60.6% | Tied with Kimi K3 |
-| **FinStepBench-FinanceDR** | 55.8% | StepFun internal; covers full deep-research finance workflows — source traceability, reproducible assumptions, auditable reports |
-| **OfficeQA Pro** | 60.3% | Databricks; 90 questions over large enterprise financial document collections; grounded reasoning on spreadsheets, charts, and business artifacts |
-| **Spreadsheet v2** | 29.4% | SpreadsheetBench 2; end-to-end business spreadsheet workflows with multi-statement financial modeling; best model ≈34.89% |
+| **FinStepBench-FinanceDR** | 55.8% | StepFun internal; full deep-research finance workflows |
+| **OfficeQA Pro** | 60.3% | Databricks; 90 questions over large enterprise financial document collections |
+| **SpeadSheet v2** | 29.4% | SpreadsheetBench 2; end-to-end business spreadsheet workflows; best model ≈34.89% |
 | **GDP.pdf** | 14.8% | Known weak spot |
-| **GPQA Diamond** | 93.5% | 198 PhD-level science MCQs (biology, physics, chemistry); human expert avg 81% |
+| **GPQA Diamond** | 93.5% | 198 PhD-level science MCQs; human expert avg 81% |
 | **HLE** | 46.5% | 59.4% with tools |
 | **AA-LCR v1.1** | 88.3% | Tied with Kimi K3 (88.7%) |
-| **CritPt** | 20.9% | 71 unpublished research-level physics challenges; simulates full-scale junior-PhD research projects |
-| **MMMU-Pro** | 76.0% | Robust multimodal benchmark; filtered text-only questions, augmented candidates, vision-only settings; significantly harder than MMMU |
+| **CritPt** | 20.9% | 71 unpublished research-level physics challenges |
+| **MMMU-Pro** | 76.0% | Robust multimodal benchmark; significantly harder than MMMU |
 | **Output Speed** | 99.8 tokens/sec | GLM-5.3: 72.1 tokens/sec |
 | **Time to First Token** | 2.96s | GLM-5.3: 2.99s; Claude Opus 5: 56.84s (max effort) |
 
----
+</details>
+
+<details>
+<summary><strong>🏅 Category summary</strong></summary>
 
-## ⚠️ Limitations
+| Domain | Standing | Highlight |
+|:---|:---|:---|
+| **Long-Context Reasoning** | Near-tied for #1 | AA-LCR v1.1 **88.3%** (+7.6 over GPT-6 Astra, +9.0 over Opus 5) |
+| **Coding / SWE** | #1 open-weight | DeepSWE v1.1 **67.7%**, StepCodeBench **49.0%**, ProgramBench **80.5%** |
+| **Security / Cyber** | #1 in comparison set | CyberGym **84.7%** |
+| **Finance** | #2 overall, #1 open-weight | FrontierFinance **66.4%**, FinanceDR **55.8%** |
+| **Terminal Agents** | #2 open-weight | Terminal-Bench 4 **33.3%** (2.6× Kimi K3) |
+| **Tool Use** | Top-3 open-weight | MCP-Atlas **85.6%**, HLE w/ tools **59.4%** |
+| **General Knowledge** | Top tier | GPQA Diamond **93.5%**, HLE **46.5%** |
+| **Multimodal** | Trailing | MMMU-Pro **76.0%** — targeted for improvement |
+
+</details>
 
-- **Knowledge Cutoff:** The model's knowledge is current up to mid-2026. It may not be aware of events after that date.
-- **Hallucination:** Like all large language models, Step-5-Preview can generate plausible but incorrect information, especially in domains with sparse training data.
-- **Long Context Degradation:** While the model supports 1M tokens, performance may degrade for extremely long contexts beyond 500K tokens in certain tasks.
-- **Tool Use Reliability:** Tool calling is highly capable but not infallible. Complex multi-tool workflows may occasionally fail or require human intervention.
+<details>
+<summary><strong>⚠️ Limitations</strong></summary>
+
+- **Knowledge Cutoff:** Knowledge is current up to mid-2026. May not be aware of later events.
+- **Hallucination:** Can generate plausible but incorrect information, especially in domains with sparse training data.
+- **Long Context Degradation:** Performance may degrade for extremely long contexts beyond 500K tokens in certain tasks.
+- **Tool Use Reliability:** Tool calling is capable but not infallible. Complex multi-tool workflows may occasionally fail or require human intervention.
 - **Multimodal Limitations:** Video understanding is limited to clips under 5 minutes and 128 MB. Extremely high-resolution images may be downscaled.
-- **Language Coverage:** While multilingual, the model is primarily optimized for English and Chinese. Performance in other languages may vary.
+- **Language Coverage:** Primarily optimized for English and Chinese. Performance in other languages may vary.
 
----
+</details>
 
-## ⚖️ Ethical Considerations
+<details>
+<summary><strong>⚖️ Ethical Considerations</strong></summary>
 
-StepFun is committed to the responsible development and deployment of AI. We have taken the following measures:
+StepFun is committed to the responsible development and deployment of AI. Measures taken:
 
-- **Safety Alignment:** The model was fine-tuned with RLHF to refuse harmful requests and promote helpful, honest, and harmless behavior.
-- **Bias Mitigation:** Training data was filtered to reduce harmful stereotypes and biases. However, residual biases may exist.
-- **Transparency:** We provide detailed model cards and benchmark results to enable informed use.
+- **Safety Alignment:** Fine-tuned with RLHF to refuse harmful requests and promote helpful, honest, and harmless behavior.
+- **Bias Mitigation:** Training data was filtered to reduce harmful stereotypes and biases. Residual biases may exist.
+- **Transparency:** Detailed model cards and benchmark results are provided to enable informed use.
 - **License Restrictions:** The StepFun Community License prohibits certain high-risk uses, including autonomous weapons, surveillance, and malicious cyber activities.
-- **Content Provenance:** We encourage users to clearly label AI-generated content and to use the model ethically.
+- **Content Provenance:** Users are encouraged to clearly label AI-generated content.
 
-We urge all users to consider the ethical implications of their applications and to implement appropriate safeguards.
+All users should consider the ethical implications of their applications and implement appropriate safeguards.
 
----
+</details>
 
-## 🖥️ Hardware Requirements
+<details>
+<summary><strong>🖥️ Hardware Requirements</strong></summary>
 
 | Precision | Minimum GPU Memory | Recommended GPU Configuration |
 |:---|:---|:---|
@@ -664,12 +613,12 @@ We urge all users to consider the ethical implications of their applications and
 | **FP8** | 600 GB | 4× H100 80GB (tensor parallel) |
 | **INT4** | 300 GB | 4× A100 80GB (tensor parallel) |
 
-For inference with 1M context, additional memory is required for KV cache. We recommend using paged attention and 
-offloading techniques available in vLLM and SGLang.
+For inference with 1M context, additional memory is required for KV cache. Paged attention and offloading techniques available in vLLM and SGLang are recommended.
 
----
+</details>
 
-## ⚡ Performance Metrics
+<details>
+<summary><strong>⚡ Performance Metrics</strong></summary>
 
 | Metric | Value |
 |:---|:---|
@@ -682,11 +631,10 @@ offloading techniques available in vLLM and SGLang.
 
 *Measured on 8× H100 80GB with vLLM, batch size 1, BF16.*
 
----
-
-## 📚 Citation
+</details>
 
-If you use Step-5-Preview in your research, please cite:
+<details>
+<summary><strong>📚 Citation</strong></summary>
 
 ```bibtex
 @misc{stepfun2026step5preview,
@@ -698,29 +646,7 @@ If you use Step-5-Preview in your research, please cite:
 }
 ```
 
----
-
-## 📜 License
-
-Step-5-Preview is released under the **StepFun Community License**. 
-See the [LICENSE](https://huggingface.co/SHSLab/Step-5-Preview-BF16/blob/main/LICENSE) file for full terms.
-
-<div style="border-left: 6px solid #faad14; padding: 16px; border-radius: 8px; margin: 20px 0;">
-  <strong>⚠️ Usage Restrictions</strong><br>
-  • Commercial use is permitted under the StepFun Community License.<br>
-  • Redistribution must include the license and attribution.<br>
-  • See LICENSE for full details.
-</div>
-
----
-
-## 📬 Contact
-
-- **Hugging Face:** [SHSLab](https://huggingface.co/SHSLab)
-- **GitHub:** [github.com/stepfun-ai](https://github.com/stepfun-ai)
-- **Discord:** [Join our Discord](https://discord.gg/stepfun)
-- **Email:** [opensource@stepfun.com](mailto:opensource@stepfun.com)
-- **Website:** [stepfun.com](https://stepfun.com)
+</details>
 
 ---