Spaces:

MCP-1st-Birthday
/

overgrowth

Sleeping

App Files Files Community

overgrowth / NETBOX_INTEGRATION.md

Graham Paasch

feat: Add NetBox/Nautobot SoT integration (Todo #1)

74f2bea 10 months ago

preview code

Raw

History Blame

8.02 kB

NetBox Integration Guide

Overgrowth now uses NetBox/Nautobot as the authoritative Source of Truth for network designs. This replaces YAML files with a proper IPAM/DCIM system used by enterprises worldwide.

Why NetBox?

Industry Standard: Used by Netflix, DigitalOcean, Dropbox, and thousands of organizations
Rich Data Model: Devices, racks, cables, VLANs, IPs, circuits, power, and more
API-First: RESTful API for automation
Extensible: Custom fields, webhooks, plugins
Multi-Vendor: Cisco, Juniper, Arista, HPE, Dell - all supported
Open Source: Free and actively maintained

Quick Start (Local Development)

1. Start NetBox with Docker Compose

cd overgrowth

# Copy environment template
cp netbox.env.example netbox.env

# Start NetBox stack
docker-compose -f docker-compose-netbox.yml up -d

# Wait for services to start (30-60 seconds)
docker-compose -f docker-compose-netbox.yml logs -f netbox

2. Access NetBox

Open http://localhost:8000 in your browser

Username: admin
Password: admin (from netbox.env)
API Token: 0123456789abcdef0123456789abcdef01234567

3. Configure Overgrowth

# Set environment variables
export NETBOX_URL="http://localhost:8000"
export NETBOX_TOKEN="0123456789abcdef0123456789abcdef01234567"

# Test connection
python test_netbox.py

4. Run Pipeline with NetBox

# Pipeline will automatically sync to NetBox
python app.py

# Or test from CLI
python -c "
from agent.pipeline_engine import OvergrowthPipeline, NetworkIntent
p = OvergrowthPipeline(use_netbox=True)
intent = NetworkIntent(
    description='Coffee shop network',
    business_requirements=['Guest WiFi', 'POS systems'],
    constraints=['Budget under $5000']
)
model = p.stage2_generate_sot(intent)
"

Production Deployment

Option 1: Self-Hosted NetBox

Follow official docs: https://docs.netbox.dev/en/stable/installation/

Requirements:

PostgreSQL 12+
Redis 6.2+
Python 3.8+
2GB RAM minimum

Option 2: Nautobot Cloud

Enterprise-supported hosted NetBox alternative:

https://www.networktocode.com/nautobot/
Free tier available
Fully compatible with NetBox API

Configuration:

export NAUTOBOT_URL="https://yourinstance.nautobot.cloud"
export NAUTOBOT_TOKEN="your_api_token_here"

Option 3: NetBox Cloud

Official hosted NetBox service:

https://netboxlabs.com/netbox-cloud/
30-day free trial
Managed infrastructure

NetBox Configuration

Initial Setup

Create API Token (in NetBox UI)
- User menu → API Tokens → Add
- Copy token to environment variable
Enable Webhooks (optional)
- Admin → Webhooks → Add
- URL: http://your-overgrowth-server/webhook/netbox
- Events: dcim.device, ipam.vlan, ipam.prefix

Import Manufacturers

# NetBox has built-in device library
# Or import custom manufacturers via API

Recommended Plugins

Add to netbox.env:

PLUGINS=['netbox_topology_views', 'netbox_acls', 'netbox_bgp']

Install plugins:

docker-compose -f docker-compose-netbox.yml exec netbox pip install \
  netbox-topology-views \
  netbox-acls \
  netbox-bgp

Architecture

┌─────────────────────┐
│   Overgrowth UI     │
│   (Gradio App)      │
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│  Pipeline Engine    │
│  - Consultation     │
│  - LLM Design       │
│  - BOM Generation   │
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐      ┌──────────────────┐
│  NetBox Client      │◄────►│ NetBox/Nautobot  │
│  - Sites            │      │  - PostgreSQL    │
│  - Devices          │      │  - Redis         │
│  - VLANs/IPs        │      │  - REST API      │
│  - Sync Logic       │      └──────────────────┘
└─────────────────────┘
           │
           ▼
┌─────────────────────┐
│  GNS3 / Real Gear   │
│  (Deployment)       │
└─────────────────────┘

Data Flow

Consultation → User provides requirements via natural language
LLM Design → Claude generates VLANs, subnets, device roles
NetBox Sync → Design written to NetBox via API
YAML Backup → Local YAML file created for version control
BOM Generation → Read from NetBox to create shopping list
GNS3 Deploy → Read from NetBox to generate configs
Validation → Compare actual network state vs NetBox

API Usage Examples

Python SDK (pynetbox)

from agent.netbox_client import NetBoxClient

# Initialize client
nb = NetBoxClient()

# Create a site
site = nb.create_site(
    name="Office HQ",
    description="Main office location"
)

# Create VLANs
vlan = nb.create_vlan(
    vid=10,
    name="Management",
    site="Office HQ",
    description="Network management VLAN"
)

# Create IP prefix
prefix = nb.create_prefix(
    prefix="10.0.10.0/24",
    description="Management subnet",
    site="Office HQ",
    vlan=10
)

# Sync entire network model
network = {
    "name": "retail-store",
    "vlans": [...],
    "subnets": [...],
    "devices": [...]
}
summary = nb.sync_network_model(network)
print(f"Created {summary['devices']} devices, {summary['vlans']} VLANs")

Direct REST API

# Get all devices
curl -H "Authorization: Token 0123456789abcdef0123456789abcdef01234567" \
     http://localhost:8000/api/dcim/devices/

# Create a VLAN
curl -X POST \
     -H "Authorization: Token 0123456789abcdef0123456789abcdef01234567" \
     -H "Content-Type: application/json" \
     -d '{"vid": 20, "name": "Users"}' \
     http://localhost:8000/api/ipam/vlans/

Troubleshooting

NetBox not connecting

# Check NetBox is running
docker-compose -f docker-compose-netbox.yml ps

# View logs
docker-compose -f docker-compose-netbox.yml logs netbox

# Restart services
docker-compose -f docker-compose-netbox.yml restart

Import errors

# Install pynetbox
pip install pynetbox>=7.0.0

# Verify installation
python -c "import pynetbox; print(pynetbox.__version__)"

API token issues

# Regenerate token in NetBox UI
# Update environment variable
export NETBOX_TOKEN="new_token_here"

# Test connection
python test_netbox.py

Fallback Mode

If NetBox is unavailable, Overgrowth automatically falls back to YAML files:

pipeline = OvergrowthPipeline(use_netbox=True)
# If NetBox connection fails, pipeline.use_netbox becomes False
# All operations continue using local YAML files

Migration from YAML

To migrate existing YAML network models to NetBox:

from agent.pipeline_engine import OvergrowthPipeline, NetworkModel

# Load existing YAML
pipeline = OvergrowthPipeline(use_netbox=True)
yaml_content = open("infra/network_model.yaml").read()
model = NetworkModel.from_yaml(yaml_content)

# Sync to NetBox
pipeline.netbox.sync_network_model(model.to_dict())

Next Steps

Configure NetBox webhooks for real-time updates
Set up Batfish for pre-deployment validation (Stage 0)
Integrate SuzieQ for drift detection (Stage 7b)
Add GitOps workflow for NetBox changes
Enable digital twin simulation (Stage 6b)

Resources

NetBox Docs: https://docs.netbox.dev/
Nautobot Docs: https://docs.nautobot.com/
pynetbox SDK: https://pynetbox.readthedocs.io/
NetBox Community: https://netbox.dev/community/
Overgrowth Issues: https://huggingface.co/spaces/MCP-1st-Birthday/overgrowth/discussions