Update README.md

349c475 verified 6 months ago

11.7 kB

	---
	license: mit
	language:
	- en
	---
	## Ling-2.6-flash: Faster Responses, Stronger Execution, Higher Token Efficiency
	### Introduction
	Today, we announce the official open-source release of Ling-2.6-flash, an instruct model with 104B total parameters and 7.4B active parameters.

	As agent capabilities mature, skyrocketing token consumption has become a primary barrier to deployment. Unlike standard chat, agent workflows involve massive inputs and complex, multi-step execution, driving up both compute demand and user costs. While the industry is pivoting toward "long-reasoning" to push performance ceilings, a critical question remains: Are these excessive reasoning tokens truly necessary for high-frequency, everyday agent use cases?

	Faced with mounting token pressure, Ling-2.6-flash takes a different path. Rather than relying on longer outputs to chase higher scores, it is systematically optimized for inference efficiency, token efficiency, and agent performance—aiming to stay highly competitive while being faster, leaner, and better suited for real production workloads.

	At a high level, Ling-2.6-flash is built around three core strengths:

	+ Hybrid linear architecture for higher inference efficiency.
	By introducing a hybrid linear architecture, we improve computational efficiency at the foundation level. On a 4× H20 setup, Ling-2.6-flash reaches inference speeds of up to 340 tokens/s. In other words, it completes tasks with significantly better cost-performance efficiency.
	+ Token-efficiency optimization for a better intelligence-efficiency tradeoff.
	During training, we specifically optimized for token efficiency, with the goal of accomplishing tasks using more concise outputs. On the full Artificial Analysis evaluation suite, Ling-2.6-flash uses only 15M tokenswhile still delivering competitive performance. This translates into a meaningfully stronger intelligence-efficiency profile.
	+ Targeted improvements for agent scenarios.
	For the agent use cases seeing the strongest demand today, we continuously refined Ling-2.6-flash in tool use, multi-step planning, and task execution. As a result, the model achieves performance that is competitive with, and in some cases reaches SOTA level against, models with larger active parameter counts on benchmarks including BFCL-V4, TAU2-bench, SWE-bench Verified, Claw-Eval, and PinchBench.

	### Evaluation
	We have conducted a comprehensive evaluation of Ling-2.6-flash across multiple authoritative benchmarks. Ling-2.6-flash performs strongly on representative agent benchmarks such as BFCL-V4, TAU2-bench, SWE-bench Verified, and PinchBench. In practice, Ling-2.6-flash delivers a strong user experience across frameworks including Claude Code, Kilo Code, Qwen Code, Hermes Agent, and OpenClaw, etc.

	Beyond agent tasks, Ling-2.6-flash also delivers strong performance across general knowledge,mathematical reasoning, instruction following, and long-context understanding, remains well aligned with SOTA models in the same size class.
	<div align="center">
	<img src="https://mdn.alipayobjects.com/huamei_3p6pd0/afts/img/KhFxSrxyF5IAAAAAgCAAAAgADryCAQFr/original" width="8001" title="" crop="0,0,1,1" id="u4a7a4034" class="ne-image">
	</div>

	<div align="center">
	<img src="https://mdn.alipayobjects.com/huamei_3p6pd0/afts/img/4bI1SK8pNM8AAAAAgBAAAAgADryCAQFr/original" width="8001" title="" crop="0,0,1,1" id="uc95688f2" class="ne-image">
	</div>

	> + <font style="color:rgb(38, 38, 38);">PinchBench</font><font style="color:rgb(38, 38, 38);">: Comparative scores are retrieved directly from the official PinchBench leaderboard (as of April 20, 2026), adhering to their evaluation modes (potentially Reasoning Mode). </font>
	> + <font style="color:rgb(38, 38, 38);">Claw-Eval</font><font style="color:rgb(38, 38, 38);">: Comparative scores are sourced from the official Claw-Eval leaderboard (version dated 2026-03-25), adhering to their evaluation modes (potentially Reasoning Mode). Official scores for GPT-OSS-120B and GPT-5.4-mini are currently unavailable and have been omitted.</font>
	> + <font style="color:rgb(38, 38, 38);">TAU2-Bench</font><font style="color:rgb(38, 38, 38);">: Evaluations are conducted using official v1.0.0 code and datasets. Following the GLM-5 evaluation protocol, we applied minor prompt adjustments in the Retail and Telecom domains to ensure users express requests clearly and to prevent premature session termination. Additionally, GPT-5.2 was utilized as the User Agent across all evaluated domains.</font>
	> + <font style="color:rgb(38, 38, 38);">IFBench</font><font style="color:rgb(38, 38, 38);">: Scores for GPT-OSS-120B (low) and GPT-5.4-mini (Non-Reasoning) are sourced from the AA (Artificial Analysis) Leaderboard. All other model performance data are based on internal evaluation results.</font>
	>

	### Architecture
	Ling-2.6-flash continues the architectural direction introduced in Ling 2.5. Building on the Ling 2.0 foundation, we incorporate a hybrid linear attention mechanism, upgrading the original GQA attention design into a 1:7 MLA + Lightning Linear hybrid architecture through incremental training.
	<div align="center">
	<img src="https://mdn.alipayobjects.com/huamei_3p6pd0/afts/img/dZ9VS4RPjzAAAAAAgBAAAAgADryCAQFr/fmt.webp" width="650" title="" crop="0,0,1,1" id="u46a87a11" class="ne-image">
	</div>

	This combination of hybrid attention and a highly sparse MoE architecture gives Ling-2.6-flash a clear advantage in inference efficiency. Compared with mainstream SOTA models in a similar size class, Ling-2.6-flash not only delivers faster time-to-first-token, but also achieves substantially higher generation throughput in long-output scenarios. At peak, both prefill throughput and decode throughput can improve by up to around 4×.

	As shown in the figure below, Ling-2.6-flash’s throughput advantage becomes more pronounced as both context length and generation length increase. More importantly, this is not just a benchmark-side gain on static metrics. In real deployment settings, the model continues to unlock stronger speed benefits as task complexity grows.

	Whether the workload involves long-context understanding or extended text generation, Ling-2.6-flash preserves model capability while delivering faster responses, higher throughput, and better real-world deployment efficiency.
	<div align="center">
	<img src="https://mdn.alipayobjects.com/huamei_3p6pd0/afts/img/Fa_fQrVD3hcAAAAAX7AAAAgADryCAQFr/original" width="600" alt="Decode Throughput Comparison">
	<p><em>Decode Throughput Comparison, 4× H20-3e, TP=4, Batch Size = 32</em></p>
	</div>

	<div align="center">
	<img src="https://mdn.alipayobjects.com/huamei_3p6pd0/afts/img/LRDBTILYEooAAAAAXdAAAAgADryCAQFr/original" width="600" alt="Prefill Throughput Comparison">
	<p><em>Prefill Throughput Comparison, 4× H20-3e, TP=4, Batch Size = 32</em></p>
	</div>

	### Quickstart
	#### SGLang (Recommended)
	##### Environment Preparation
	```bash
	pip install uv

	uv venv ~/my_ling_env

	source ~/my_ling_env/bin/activate

	# uv pip "sglang-kernel>=0.4.1"
	uv pip install "sglang[all]>=0.5.10.post1" --prerelease=allow
	```

	##### Run Inference
	Both BF16 and FP8 models are supported by SGLang now. It depends on the dtype of the model in `${MODEL_PATH}`. Here is the example to run Ling-2.6-flash with 4 GPUs, where the master node IP is `${MASTER_IP}` and server port is `${PORT}`:

	Server

	1. Standard Inference (Without MTP)
	```bash
	python -m sglang.launch_server \
	--model-path $MODEL_PATH \
	--tp-size 4 \
	--pp-size 1 \
	--dp-size 1 \
	--trust-remote-code \
	--tool-call-parser qwen25 \
	--json-model-override-args '{"linear_backend": "seg_la", "torch_dtype": "bfloat16", "architectures": ["BailingMoeV2_5ForCausalLM"], "model_type": "bailing_hybrid", "rope_scaling": {"rope_type": "yarn", "factor": 2.0, "rope_theta": 6000000, "partial_rotary_factor": 0.5, "original_max_position_embeddings": 131072}}' \
	--dist-init-addr $MASTER_IP:2345 \
	--port $PORT \
	--nnodes 1
	```

	2. Inference with MTP (Multi-Token Prediction)
	_The current official SGLang implementation of MTP contains a bug. For better inference performance, we recommend installing our patched version. Our fix is currently under review and is expected to be merged into the official SGLang library shortly._

	Install our SGLang
	```bash
	git clone -b ling_2_6 git@github.com:antgroup/sglang.git
	cd sglang

	pip install --upgrade pip
	pip install -e "python"
	```
	Start server
	```bash
	python -m sglang.launch_server \
	--model-path $MODEL_PATH \
	--tp-size 4 \
	--pp-size 1 \
	--dp-size 1 \
	--speculative-algorithm NEXTN \
	--speculative-num-steps 1 \
	--speculative-eagle-topk 1 \
	--speculative-num-draft-tokens 4 \
	--mem-fraction-static 0.75 \
	--max-running-requests 64 \
	--max-mamba-cache-size 256 \
	--tool-call-parser qwen25 \
	--json-model-override-args '{"linear_backend": "seg_la", "torch_dtype": "bfloat16", "architectures": ["BailingMoeV2_5ForCausalLM"], "model_type": "bailing_hybrid", "rope_scaling": {"rope_type": "yarn", "factor": 2.0, "rope_theta": 6000000, "partial_rotary_factor": 0.5, "original_max_position_embeddings": 131072}}' \
	--trust-remote-code \
	--dist-init-addr $MASTER_IP:2345 \
	--port $PORT \
	--nnodes 1
	```

	Client

	```bash
	curl -s http://${MASTER_IP}:${PORT}/v1/chat/completions \
	-H "Content-Type: application/json" \
	-d '{"model": "auto", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'
	```

	#### vLLM
	##### Environment Preparation
	```bash
	pip install uv

	uv venv ~/my_ling_env

	source ~/my_ling_env/bin/activate

	git clone https://github.com/vllm-project/vllm.git

	cd vllm

	VLLM_USE_PRECOMPILED=1 uv pip install --editable . --torch-backend=auto
	```

	#### Run inference

	Server
	```bash
	vllm serve $MODEL_PATH \
	--port $PORT \
	--served-model-name my_model \
	--trust-remote-code --tensor-parallel-size 4 \
	--gpu-memory-utilization 0.85
	```

	Client

	```bash
	curl -s http://${MASTER_IP}:${PORT}/v1/chat/completions \
	-H "Content-Type: application/json" \
	-d '{"model": "auto", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'
	```

	### Limitations & Future Plans
	Ling-2.6-flash has already made meaningful progress in our pursuit of an extreme intelligence-efficiency tradeoff. The model has improved substantially in key areas such as tool use, multi-step planning, and long-horizon task execution. Combined with systematic optimizations in inference efficiency and interaction experience, Ling-2.6-flash is now better equipped to handle large-scale, high-frequency automated workloads, delivering stronger real-world value in production settings.

	At the same time, we are fully aware that pushing intelligence efficiency to the limit comes with tradeoffs. In some highly complex scenarios, the model can still exhibit tool hallucinations due to limited reasoning depth. In addition, there is still room for improvement in areas such as natural bilingual switching between Chinese and English and compliance with highly complex instructions.

	Looking ahead, we will continue exploring the frontier of intelligence efficiency. While preserving the model’s high-efficiency inference characteristics, we aim to further improve the balance between output quality and token efficiency, and to continuously strengthen the model’s stability, usability, and interaction experience across a wider range of real-world scenarios.