psikosen/mini-swarm-browser-258m

🐝 Mini-Swarm-Browser (258M)

Mini-Swarm-Browser is an ultra-fast, cooperative multi-agent browser navigation model and architecture based on Canopy-R3 258M.

Instead of routing massive full-DOM trees and screenshots to a monolithic 14B–70B model, Mini-Swarm employs a collaborative swarm of specialized sub-agents that operate concurrently on distilled accessibility trees (AXTree) and Set-of-Marks tags (@e1, @e2), completing real browser actions in under 5 seconds on consumer hardware.

---

⚡ Key Highlights & Benchmark Stats

• Total Parameter Count: 258,555,654 (~258.5M parameters)
• Active Parameters / Token: ~112M parameters
• Memory Footprint: ~566 MB weights (FP16/BF16) / < 1.2 GB active VRAM
• Interaction Speed: ~5 seconds for end-to-end multi-field form completion
• Token Efficiency: ~3,500+ tokens saved per action step via Set-of-Marks and AXTree distillation (reducing context from 40k+ tokens to ~400 tokens)
• Cooperative Swarm Architecture:
  1. Coordinator: High-level semantic planning and goal decomposition.
  2. DOM Distiller: AXTree pruning, Set-of-Marks interactive visual tagging.
  3. Speculative Action Engine: Multi-action batch emission (predicting dependent steps like click-then-type in a single pass).
  4. Fast Verifier: Real-time outcome assertions and local error diagnosis.

---

🚀 Quickstart

1. Load the Model with Hugging Face Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "psikosen/mini-swarm-browser-258m"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

prompt = "<|im_start|>user\nGoal: Click search button and type 'AI agents'\nElements: [@e1: search_input, @e2: search_btn]<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

2. Run the Swarm Live Browser Controller

Clone the repo and run the interactive browser agent:

git clone https://huggingface.co/psikosen/mini-swarm-browser-258m
cd mini-swarm-browser-258m

# Launch live headed browser test
python test_interactive.py --workflow form

---

🛠️ Multi-Agent Architecture

User Objective
       │
       ▼
┌──────────────────────────────┐
│     Swarm Coordinator        │
└──────────────┬───────────────┘
               │
       AXTree / SoM Tokens
               │
       ┌───────┴───────────────┐
       ▼                       ▼
┌──────────────┐        ┌──────────────────────────────┐
│ DOM Distiller│        │  Speculative Action Engine   │
└──────────────┘        │   (Canopy-R3 258M Planner)   │
                        └──────────────┬───────────────┘
                                       │ Action Batch: [@e1, @e2]
                                       ▼
                        ┌──────────────────────────────┐
                        │      Fast Verifier HUD       │
                        └──────────────────────────────┘

---

📜 License

Released under the Apache 2.0 License.