README.md

---
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
base_model: Qwen/Qwen3-VL-2B-Instruct
datasets:
  - freeai-org/ScalpelBench
language:
  - en
  - zh
tags:
  - qwen3-vl
  - vision-language
  - multimodal
  - structured-pruning
  - knowledge-distillation
  - lora
  - scalpel
---

# Scalpel-VL-1.7B

Scalpel-VL-1.7B is a structurally pruned and recovery-trained
vision-language model based on the Qwen3-VL-2B-Instruct architecture. It is
the Round 7 `post_recovery_model` produced by
[Scalpel](https://github.com/freeai-org/Scalpel).

Seven language decoder layers were physically removed from the original
28-layer model. The remaining 21-layer student was recovered after every
pruning round using a fixed reference teacher, final-logit knowledge
distillation, and merged `all-linear` LoRA adapters.

The checkpoint contains **1,775,180,032 parameters**. The repository name uses
the parameter count rounded down to one decimal place: **1.7B**.

## Model details

| Item | Value |
| --- | --- |
| Architecture | Qwen3-VL |
| Task | Image-text-to-text |
| Parameters | 1,775,180,032 |
| Weight dtype | BF16 |
| Language decoder layers | 21 (originally 28) |
| Removed original layers | 7, 8, 9, 12, 13, 14, 23 |
| Layer reduction | 25.0% |
| Parameter reduction | 16.56% |
| Recovery dataset | [ScalpelBench](https://huggingface.co/datasets/freeai-org/ScalpelBench) |
| Recovery method | Final-logit CE + KL distillation with merged LoRA |
| License | Apache-2.0 |

The removed-layer list above records the current-to-original layer mapping for
the seven completed pruning rounds.

## Recovery protocol

At each round, Scalpel:

1. evaluates candidate language layers with a fixed text probe;
2. physically removes the lowest-risk current layer;
3. trains only `all-linear` LoRA parameters on the pruned student;
4. uses the fixed reference model as teacher and aligns final LM-head logits;
5. merges the adapter and uses the exported model as the next-round student.

Recovery used the approximately 0.1B-token ScalpelBench instruction-response
mixture, covering English, Chinese, mathematical reasoning, and code. The
experiment partitions the mixture into ten deterministic token-balanced parts;
this Round 7 checkpoint has completed recovery on parts 1 through 7.

## Internal evaluation

The following measurements use the complete 1,568-sample ScalpelBench
validation split with 276,216 supervised tokens and a maximum sequence length
of 1,536. `Macro score` is 100 times the macro average of teacher-forced token
accuracy over English, Chinese, Math, and Code. It is an internal controlled
metric, not an Open LLM Leaderboard score.

| Model | Params | Layers | PPL ↓ | Token accuracy ↑ | Macro score ↑ | Supervised tokens/s ↑ | Peak CUDA MiB ↓ |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Reference | 2.128B | 28 | 4.9731 | 68.54% | 70.40 | 4,523.69 | 8,699.04 |
| Scalpel-VL-1.7B | 1.775B | 21 | 4.2919 | 66.12% | 67.50 | 5,567.23 | 6,878.06 |

Under this evaluation setup, the pruned checkpoint uses 16.56% fewer
parameters, has 25% fewer language layers, achieves 23.07% higher supervised
token throughput, and reduces peak CUDA allocation by 20.93%. Throughput and
memory numbers are hardware- and software-dependent and should not be treated
as universal deployment guarantees.

## Usage

Install a Transformers version that supports Qwen3-VL, then load the model and
processor directly from the Hub:

```python
import torch
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

model_id = "freeai-org/Scalpel-VL-1.7B"

model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
            },
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    generated_ids = model.generate(**inputs, max_new_tokens=128)

generated_ids = [
    output_ids[len(input_ids):]
    for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
]
print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])
```

For lower-memory inference, select an attention implementation and dtype that
are supported by your hardware. This repository contains merged full-model
weights; no separate LoRA adapter is required.

## Intended use and limitations

This checkpoint is intended for research on structured pruning, post-pruning
recovery, knowledge distillation, and efficient multimodal inference.

- Removing decoder layers can change general reasoning, multilingual, OCR,
  grounding, long-context, and video behavior.
- The reported evaluation is teacher-forced and uses ScalpelBench; it does not
  establish performance on unrelated benchmarks or production traffic.
- ScalpelBench includes material derived from multiple upstream datasets.
  Review its dataset card and source licenses before downstream use.
- Model outputs may be incorrect, biased, or unsafe. Validate the checkpoint
  for the target domain before deployment, especially in high-stakes settings.

## Related resources

- [Scalpel code](https://github.com/freeai-org/Scalpel)
- [Scalpel project page](https://freeai-org.github.io/Scalpel/)
- [ScalpelBench](https://huggingface.co/datasets/freeai-org/ScalpelBench)
- [Qwen3-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct)

## Citation

```bibtex
@misc{wu2026catellectvl2bvisionlanguagemodeledgebased,
  title         = {Catellect-VL-2B: A Vision-Language Model for Edge-Based Feline Behavior Understanding},
  author        = {YuHang Wu and HaoXian Liu and Jia Tao},
  year          = {2026},
  eprint        = {2608.22070},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CE},
  url           = {https://arxiv.org/abs/2608.22070}
}
```

Qwen3-VL is licensed and attributed according to its original model card.