# Moonshine on Lightning.ai

Lightning.ai is the primary demo target for Moonshine because it can run the real GPU-backed synthetic data pipeline while still exposing a browser-based UI.

## Recommended Demo Architecture

```text
Lightning.ai Studio
  ├── Moonshine Python pipeline
  ├── Gradio product demo
  ├── vLLM OpenAI-compatible endpoint
  └── Nemotron model backend
```

Vercel is useful later as a polished landing page or thin frontend, but the core interview demo should run on Lightning so the NVIDIA/GPU story is real.

## Setup

1. Create a Lightning.ai Studio with a CUDA GPU.
2. Clone your GitHub repo:

```bash
git clone https://github.com/PhillipHolland/moonshine.git
cd moonshine
```

3. Create the environment:

```bash
python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements-lightning.txt
```

4. Run the UI with mock inference first:

```bash
MOONSHINE_USE_MOCK=1 python scripts/run_demo.py
```

5. Open the Lightning-provided app URL for port `7860`.

## Nemotron / vLLM Path

Start a vLLM OpenAI-compatible server in a second terminal:

```bash
python -m vllm.entrypoints.openai.api_server \
  --model nvidia/Nemotron-3-8B-Chat \
  --host 0.0.0.0 \
  --port 8000
```

Then run Moonshine against that endpoint:

```bash
export MOONSHINE_BASE_URL=http://127.0.0.1:8000/v1
export MOONSHINE_API_KEY=EMPTY
export MOONSHINE_MODEL=nvidia/Nemotron-3-8B-Chat
MOONSHINE_USE_MOCK=0 python scripts/run_demo.py
```

If model access or GPU memory blocks Nemotron 3 directly, keep the same Moonshine app and swap `MOONSHINE_MODEL` to an available NVIDIA-hosted or OpenAI-compatible Nemotron endpoint.

## Interview Framing

Moonshine demonstrates:

- Synthetic data distillation, not just generation
- Closed-loop regeneration for low-quality outputs
- Rubric-backed scoring and audit traces
- GPU-ready inference architecture
- A path from local prototype to NeMo Curator-style distributed data processing
- Applicability to post-training, agentic AI traces, and physical AI safety data