mj0530/CERA-MoA-Qwen3-4B-4agents

CERA-MoA (Qwen3-4B, 4 Agents)

Official checkpoint for CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents.

• Paper: arXiv:2609.18779
• Code: github.com/michaeljiang0530/CERA-MoA
• Hugging Face Papers: hf.co/papers/2609.18779

What's included

Path	Description
agents/agent_{0,1,2,3}/	Per-agent LoRA adapters (PEFT) on Qwen/Qwen3-4B
router.pt	Familiarity predictor / router heads

Backbone weights are not included; load Qwen/Qwen3-4B from Hugging Face and attach these LoRAs.

Quick load (agents)

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3-4B"
repo = "YOUR_HF_USERNAME/CERA-MoA-Qwen3-4B-4agents"

tokenizer = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto", trust_remote_code=True)

# Example: load agent 0
model = PeftModel.from_pretrained(model, f"{repo}/agents/agent_0")

For full multi-agent routing + inference, use the official code in the GitHub repo (train_gspo_multi_agent.py / evaluation scripts) and point it to this folder (or a local download of the Hub repo).

Checkpoint note

This is the final snapshot from the paper's main Qwen3-4B 4-agent run (midemb=True, marginal push-away), corresponding to training step 6000.