parth21shah/linnix-3b-distilled

Linnix 3B Distilled - Incident Detection Model

Fine-tuned 3B parameter model for Linux system incident detection and root cause analysis.

Model Description

linnix-3b-distilled is a specialized language model trained to detect and analyze system observability incidents. It converts process telemetry and system metrics into natural language insights with actionable remediation steps.

• Base Model: Qwen2.5-3B-Instruct
• Fine-tuning: Distilled from fine-tuned Qwen2.5-7B teacher model
• Quantization: Q5_K_M (optimal quality/size balance)
• Size: 2.1 GB
• License: Apache 2.0
• Inference: CPU-optimized (no GPU required)

Use Cases

• CPU Spin Detection: Identifies processes holding excessive CPU
• Fork Storm Prevention: Detects rapid process creation patterns
• Memory Leak Isolation: Tracks RSS growth anomalies
• OOM Risk Assessment: Predicts out-of-memory conditions
• I/O Saturation Analysis: Identifies disk/network bottlenecks

Quick Start

With Linnix Platform (Recommended)

# Clone Linnix repository
git clone https://github.com/linnix-os/linnix.git
cd linnix

# Auto-download model and start services
./setup-llm.sh

# Model will be downloaded from Hugging Face automatically

Standalone with llama.cpp

# Download model
wget https://huggingface.co/parth21shah/linnix-3b-distilled/resolve/main/linnix-3b-distilled-q5_k_m.gguf

# Run inference server
llama-server -m linnix-3b-distilled-q5_k_m.gguf --port 8090 --ctx-size 4096

# Query for incident analysis
curl http://localhost:8090/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "system", "content": "Convert incident telemetry into structured Insight JSON."},
      {"role": "user", "content": "INCIDENT: w=5 eps=240 cpu=96% top=java pid=4412"}
    ]
  }'

Performance

Metric	Value
Inference Speed	~30 tokens/sec (8-core CPU)
Memory Usage	~2 GB RAM
Context Length	4096 tokens
End-to-end Latency	3-7 seconds
Quality (vs 7B teacher)	92% agreement

Training Data

Trained on a proprietary dataset of system observability incidents covering multiple incident classes:

• Classes: cpu_spin, fork_storm, io_saturation, oom_risk, short_job_flood, normal
• Sources: Synthetic incidents generated by teacher model + curated production postmortems
• Quality: High-confidence labels with actionable remediation steps

Incident Classes

The model detects these incident patterns:

• cpu_spin: Sustained high CPU usage (>95%)
• fork_storm: Excessive process creation (>100/sec)
• io_saturation: Disk/network bottlenecks (>90% util)
• oom_risk: Memory pressure and swap activity
• short_job_flood: Thousands of sub-second processes
• runaway_tree: Uncontrolled process tree growth
• normal: No incident detected

Output Format

{
  "class": "cpu_spin",
  "confidence": 0.89,
  "primary_process": "java",
  "why": "java pid 4412 sustained 97% CPU for 3 minutes across 5-second window",
  "actions": [
    "Capture async-profiler flame graph: async-profiler -d 30 -f /tmp/flame.html 4412",
    "Check thread states: jstack 4412 > threads.txt",
    "Review GC activity: jstat -gcutil 4412 1000"
  ]
}

Limitations

• Domain-specific: Trained only on Linux system incidents
• JSON-only output: Not suitable for general chat/Q&A
• CPU inference: Slower than GPU (but affordable and portable)
• Context window: 4096 tokens max (not for full log analysis)

Integration

With eBPF Monitoring

Linnix combines this model with eBPF-based process monitoring for real-time incident detection:

Kernel (eBPF) → Perf Buffers → cognitod → HTTP API → llama.cpp → Insights

See github.com/linnix-os/linnix for the full platform.

API-only Usage

import requests

def analyze_incident(telemetry_summary):
    response = requests.post(
        "http://localhost:8090/v1/chat/completions",
        json={
            "messages": [
                {"role": "system", "content": "Convert incident telemetry into structured Insight JSON."},
                {"role": "user", "content": f"INCIDENT: {telemetry_summary}"}
            ],
            "temperature": 0.1,
            "max_tokens": 512
        }
    )
    return response.json()

# Example
result = analyze_incident("w=5 eps=180 frk=120 cpu=87% top=bash")
print(result["choices"][0]["message"]["content"])

Citation

@software{linnix3b2025,
  author = {Shah, Parth},
  title = {Linnix 3B Distilled: Incident Detection Model},
  year = {2025},
  url = {https://huggingface.co/parth21shah/linnix-3b-distilled},
  note = {Fine-tuned Qwen2.5-3B for system observability}
}

Links

• Platform: github.com/linnix-os/linnix
• Documentation: docs.linnix.io
• Issues: github.com/linnix-os/linnix/issues
• License: Apache 2.0

Acknowledgments

• Base model: Qwen2.5-3B-Instruct by Alibaba Cloud
• Inference engine: llama.cpp
• Training framework: Axolotl