LIGHTNING_SETUP.md

Moonshine on Lightning.ai

Lightning.ai is the primary demo target for Moonshine because it can run the real GPU-backed synthetic data pipeline while still exposing a browser-based UI.

Recommended Demo Architecture

Lightning.ai Studio
  ├── Moonshine Python pipeline
  ├── Gradio product demo
  ├── vLLM OpenAI-compatible endpoint
  └── Nemotron model backend

Vercel is useful later as a polished landing page or thin frontend, but the core interview demo should run on Lightning so the NVIDIA/GPU story is real.

Setup

1. Create a Lightning.ai Studio with a CUDA GPU.
2. Clone your GitHub repo:

git clone https://github.com/PhillipHolland/moonshine.git
cd moonshine

3. Create the environment:

python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements-lightning.txt

4. Run the UI with mock inference first:

MOONSHINE_USE_MOCK=1 python scripts/run_demo.py

5. Open the Lightning-provided app URL for port 7860.

Nemotron / vLLM Path

Start a vLLM OpenAI-compatible server in a second terminal:

python -m vllm.entrypoints.openai.api_server \
  --model nvidia/Nemotron-3-8B-Chat \
  --host 0.0.0.0 \
  --port 8000

Then run Moonshine against that endpoint:

export MOONSHINE_BASE_URL=http://127.0.0.1:8000/v1
export MOONSHINE_API_KEY=EMPTY
export MOONSHINE_MODEL=nvidia/Nemotron-3-8B-Chat
MOONSHINE_USE_MOCK=0 python scripts/run_demo.py

If model access or GPU memory blocks Nemotron 3 directly, keep the same Moonshine app and swap MOONSHINE_MODEL to an available NVIDIA-hosted or OpenAI-compatible Nemotron endpoint.

Interview Framing

Moonshine demonstrates:

• Synthetic data distillation, not just generation
• Closed-loop regeneration for low-quality outputs
• Rubric-backed scoring and audit traces
• GPU-ready inference architecture
• A path from local prototype to NeMo Curator-style distributed data processing
• Applicability to post-training, agentic AI traces, and physical AI safety data