PIPELINE_ENHANCEMENT.md

Overgrowth Pipeline Enhancement Plan

What's Missing (Your Feedback Summary)

1. ❌ No Interactive Consultation - Should ask follow-up questions
2. ❌ Source of Truth Incomplete - No subnets, VLANs, real design
3. ❌ BOM Pricing Wrong - Shows $0 for everything
4. ❌ Setup Guide Generic - Missing firmware updates, real steps
5. ❌ No Progress Visibility - Can't see what AI agents are doing
6. ❌ No Network Simulation Link - Should show GNS3 topology

Implementation Plan

Phase 1: LLM Integration (PRIORITY)

Files Created:

• agent/llm_client.py - Unified LLM client (OpenAI/Anthropic/OpenRouter)
• agent/consultation.py - Interactive multi-turn consultation

Setup Required:

# Add to .env file:
OPENROUTER_API_KEY=sk-or-v1-xxxxx  # From your hackathon credits
# OR
OPENAI_API_KEY=sk-xxxxx
# OR
ANTHROPIC_API_KEY=sk-ant-xxxxx

Testing:

from agent.consultation import NetworkConsultant

consultant = NetworkConsultant()
is_complete, output, intent = consultant.start_consultation(
    "We're a coffee shop chain with 3 locations..."
)
print(output)  # Will show follow-up questions

# User answers questions
is_complete, output, intent = consultant.continue_consultation(
    "Budget is $50k, need it in 3 months, prefer Ubiquiti gear"
)

Phase 2: Smart Network Design

Enhance stage2_generate_sot() in pipeline_engine.py:

def stage2_generate_sot(self, intent: NetworkIntent) -> NetworkModel:
    """Use LLM to design actual network architecture"""
    
    prompt = f"""
    Design a production-ready network for:
    {intent.description}
    
    Requirements:
    - Budget: {intent.budget}
    - Locations: {intent.locations}
    - Compliance: {intent.compliance_requirements}
    
    Generate:
    1. VLAN scheme (management, data, voice, guest, security cameras, POS)
    2. IP subnetting plan (RFC1918 private addressing)
    3. Device list (switches, APs, routers, firewalls)
    4. Routing protocol (static, OSPF, BGP)
    5. Security policies
    
    Return as structured JSON.
    """
    
    # Call LLM to generate real design
    design = llm.chat([LLMMessage(role="user", content=prompt)])
    
    # Parse into NetworkModel
    return self._parse_network_design(design)

Example Output:

vlans:
  - id: 10
    name: Management
    subnet: 10.0.10.0/24
  - id: 20
    name: Guest_WiFi
    subnet: 10.0.20.0/24
  - id: 30
    name: POS_Systems
    subnet: 10.0.30.0/24
  - id: 40
    name: Security_Cameras
    subnet: 10.0.40.0/24

devices:
  - name: HQ-Core-SW01
    role: core
    model: Ubiquiti USW-Enterprise-48-PoE
    mgmt_ip: 10.0.10.10
    interfaces:
      - name: eth0/1
        vlan: 10
        mode: access

Phase 3: Real BOM Pricing

Create agent/hardware_pricing.py:

# Hardware database with real prices
HARDWARE_DB = {
    "Ubiquiti USW-Enterprise-48-PoE": {
        "price": 1799.00,
        "category": "switch",
        "vendor": "Ubiquiti"
    },
    "Ubiquiti U6-Enterprise": {
        "price": 379.00,
        "category": "access_point"
    },
    # ... more devices
}

def calculate_bom_cost(devices: List[Device]) -> float:
    total = 0
    for device in devices:
        if device.model in HARDWARE_DB:
            total += HARDWARE_DB[device.model]["price"]
    return total

Phase 4: Streaming Progress Updates

Modify app.py to use Gradio streaming:

def run_pipeline_streaming(user_input):
    """Stream progress updates to UI"""
    pipeline = OvergrowthPipeline()
    
    # Stage 1: Consultation
    yield "🤝 Stage 1: Starting consultation...\n"
    consultant = NetworkConsultant()
    is_complete, output, intent = consultant.start_consultation(user_input)
    
    if not is_complete:
        yield f"❓ **Follow-up questions:**\n{output}\n\n"
        # Wait for user response (need UI update for this)
        return
    
    yield f"✅ Stage 1 Complete\n{output}\n\n"
    
    # Stage 2: Generate SoT
    yield "📋 Stage 2: Designing network architecture...\n"
    model = pipeline.stage2_generate_sot(intent)
    yield f"✅ Stage 2 Complete - {len(model.devices)} devices, {len(model.vlans)} VLANs\n\n"
    
    # Stage 3: Diagrams
    yield "📊 Stage 3: Generating topology diagrams...\n"
    diagrams = pipeline.stage3_generate_diagrams(model)
    yield f"✅ Stage 3 Complete\n\n"
    
    # Continue with other stages...

Update Gradio interface:

run_pipeline_btn.click(
    fn=run_pipeline_streaming,
    inputs=[pipeline_input],
    outputs=[pipeline_status],  # Single streaming output
    show_progress=True
)

Phase 5: GNS3 Simulation Integration

Add to pipeline results:

def stage6_autonomous_deploy(self, model: NetworkModel) -> Dict:
    """Deploy to GNS3 lab"""
    from agent.local_mcp import call_tool
    
    # Build topology in GNS3
    result = call_tool("create_project", {
        "name": model.name,
        "auto_start": True
    })
    
    project_id = result['project_id']
    
    # Add devices
    for device in model.devices:
        call_tool("add_node", {
            "project_id": project_id,
            "name": device.name,
            "node_type": device.role,
            "x": ...,  # Calculate layout
            "y": ...
        })
    
    # Return simulation URL
    return {
        "success": True,
        "gns3_url": f"http://lab.grahampaasch.com:3080/#/projects/{project_id}",
        "topology_link": f"View live simulation: {gns3_url}"
    }

Display in UI:

## 🌐 Live Network Simulation

Your network is being built in GNS3:
- **Project:** {model.name}
- **Devices:** {len(model.devices)} nodes
- **Status:** Deploying...

[View in GNS3](http://lab.grahampaasch.com:3080/#/projects/{project_id})

Phase 6: Setup Guide with Real Steps

Enhance stage5_setup_guide():

def generate_setup_guide(self, model: NetworkModel) -> SetupGuide:
    """Generate detailed deployment guide"""
    
    phases = [
        {
            "name": "Pre-Deployment Validation",
            "duration": "1 hour",
            "steps": [
                "Verify all equipment received matches BOM",
                "Check firmware versions - minimum required:",
                *[f"  - {d.model}: firmware v{get_min_firmware(d.model)}" 
                  for d in model.devices],
                "Unbox and inventory all equipment",
                "Download latest firmware if upgrades needed"
            ]
        },
        {
            "name": "Firmware Updates",
            "duration": "2-4 hours",
            "steps": [
                "Backup factory configs",
                "Update devices one at a time",
                *[f"Update {d.name} to {get_latest_firmware(d.model)}" 
                  for d in model.devices],
                "Verify boot-up and basic connectivity",
                "Document firmware versions"
            ]
        },
        # ... more realistic phases
    ]
    
    return SetupGuide(
        network_name=model.name,
        phases=phases,
        # ... other details
    )

Next Steps

1. Add LLM API key to .env
2. Test consultation flow
3. Enhance pipeline stages with LLM calls
4. Add hardware pricing database
5. Implement streaming UI updates
6. Wire up GNS3 deployment

Cost Estimate

Using OpenRouter with Claude 3.5 Sonnet:

• Consultation: ~$0.02 per session
• Network Design: ~$0.05 per design
• Total per pipeline run: ~$0.10

Your hackathon credits should cover hundreds of runs.