BitsMoE-Qwen3-30B-A3B-Base-2bit

This repository provides a 2-bit quantized version of Qwen3-30B-A3B-Base using the BitsMoE quantization framework. This model is introduced in the paper BitsMoE: Cost-Aware Bit Allocation in Spectral Space for MoE LLM Quantization.

The model is based on Qwen/Qwen3-30B-A3B-Base and is intended for efficient inference of Mixture-of-Experts (MoE) large language models with significantly reduced memory footprint.

Paper: BitsMoE: Cost-Aware Bit Allocation in Spectral Space for MoE LLM Quantization
Code: GitHub - zjiayu064/BitsMoE

Model Details

Base model: Qwen/Qwen3-30B-A3B-Base
Quantization: 2-bit (Experts-Only)
Architecture: Mixture-of-Experts (MoE)
Method: BitsMoE, a cost-aware bit-allocation framework for MoE LLM quantization.

Usage

The BitsMoE framework requires custom kernels and model patching logic. Please refer to the BitsMoE GitHub repository for environment setup, inference scripts, and detailed usage instructions.

By default, models are downloaded from Hugging Face via the BitsMoE CLI:

bitsmoe demo

To run evaluation on benchmarks like MMLU or Hellaswag using the provided configs:

bitsmoe eval --config configs/qwen3moe/eval.yaml

License

This model follows the Apache 2.0 license. Please refer to the original Qwen model license for additional details.

Citation

If you find this work useful, please consider citing:

@misc{zhao2026bitsmoecostawarebitallocation,
      title={BitsMoE: Cost-Aware Bit Allocation in Spectral Space for MoE LLM Quantization}, 
      author={Jiayu Zhao and Zihan Teng and Minhao Fan and Tianrui Ma and Wentao Ren and Song Chen and Weichen Liu},
      year={2026},
      eprint={2606.00079},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2606.00079}, 
}