README.md

---
title: PI-AcousticNeRF
colorFrom: blue
colorTo: indigo
sdk: docker
pinned: false
license: mit
short_description: 3D acoustic field reconstruction with physics constraints
---

# Physics-Informed Acoustic NeRF

Reconstruct a continuous 3D sound pressure field from sparse microphone measurements, with the Helmholtz wave equation enforced as a hard constraint during training.

[![HuggingFace Spaces](https://img.shields.io/badge/🤗%20HuggingFace-Live%20Demo-blue)](https://huggingface.co/spaces/Pooya-sh1998/acoustic-nerf)

On **Apple M4 Pro (48GB VRam) · No cloud GPU**

---

## What It Does

A microphone hears one point. This model hears the whole room.

Given a handful of microphone measurements at multiple frequencies, the network learns a function `(x, y, z, ω) → complex pressure p̂` that predicts acoustic pressure at any point in 3D space — not just where the mics are. Physics enforcement means the predicted field actually obeys the wave equation, not just the training data.

---

## Architecture

```
Input: (x, y, z, ω)
         │
   Positional encoding
   L=10 spatial, L=6 frequency → 76 dims
         │
   SIREN MLP  ──── skip at layer 4 ────┐
   8 layers, hidden_dim=256            │
   sine activations (ω₀=1.0)          │
         │◄───────────────────────────┘
   Output: (Re(p), Im(p))   ~500k params

Training losses:
  L_data     = MSE at microphone positions
  L_physics  = |∇²p̂ + k²p̂|²  at collocation points  (Helmholtz)
  L_boundary = |∂p̂/∂n + jkβp̂|²  at room walls       (impedance BC)
  L_total    = λ_d · L_data + λ_p · L_physics + λ_b · L_boundary
```

Physics weight schedule: `λ_physics` ramps from `1e-7 → 1e-3` over 3000 epochs so the data loss stabilises before the constraint kicks in.

---

## Results

| Config | Val NMSE | Phys. Residual | Notes |
|---|---|---|---|
| Baseline (data only) | 1.783 | 1124 Pa²/m⁴ | Noisy, no wave structure |
| + Boundary BC only | 1.140 | 1146 Pa²/m⁴ | No improvement |
| + Helmholtz only | **0.969** | **1.71 Pa²/m⁴** | Wave rings appear |
| Full (Helmholtz + BC) | 0.983 | **1.56 Pa²/m⁴** | Lowest residual |

**Physics residual reduction: 99.9%** (baseline 1124 → physics-informed 1.56 Pa²/m⁴)

Room: 6 × 5 × 3 m shoebox · 2000 microphones · 4 frequencies (500/1k/2k/4kHz)  
Training: ~45 min on Apple M4 Pro (MPS backend)

---

## Interactive Viewer

The web viewer renders a horizontal slice at z = 1.5 m with a diverging RdBu colormap (blue = low pressure, red = high). Toggle between physics-informed (A) and baseline (B) models to see the difference.

```bash
# Generate field data from trained checkpoints
python -m src.visualization.export

# Start the FastAPI server + viewer
uvicorn src.api.main:app --host 0.0.0.0 --port 8000

# Open browser
open http://localhost:8000/viewer/index.html
```

Features: frequency toggle (500 Hz / 1 kHz / 2 kHz / 4 kHz), model A/B comparison, microphone overlay, Helmholtz residual overlay.

---

## Quick Start

```bash
# 1. Environment
conda create -n acoustic-nerf python=3.11
conda activate acoustic-nerf
pip install -r requirements.txt

# 2. Generate synthetic room data and train
python -m src.training.trainer --config experiments/configs/physics_informed.yaml

# 3. Run ablation studies
python -m src.training.trainer --config experiments/configs/baseline.yaml
python -m src.training.trainer --config experiments/configs/ablation_physics_only.yaml

# 4. Collect metrics
python -m src.training.collect_metrics

# 5. Export and serve
python -m src.visualization.export
uvicorn src.api.main:app
```

**Requirements:** PyTorch (MPS or CUDA), pyroomacoustics, FastAPI, Three.js (CDN)

---

## The Physics

The Helmholtz wave equation for steady-state acoustics:

```
∇²p(x,y,z) + k²p(x,y,z) = 0
```

where `k = ω/c` (wave number, c = 343 m/s). Enforced as a residual loss at random collocation points throughout the room volume each training batch. Uses finite-difference Laplacian (MPS-stable) rather than autograd second derivatives.

Boundary condition at room walls:

```
∂p/∂n + jkβp = 0
```

where β = 0.4 is the specific admittance (absorption coefficient).

---

## Key Implementation Decisions

**SIREN activations (ω₀=1.0, not 30):** Sine activations give smooth second derivatives, critical for the Laplacian computation. ω₀=30 was tested but caused the physics loss to blow up (second derivatives O(1e9) with 8 stacked layers). Lowered to 1.0 since positional encoding already handles high frequencies.

**Finite difference Laplacian:** PyTorch autograd `create_graph=True` is unstable on MPS for second derivatives. Central differences with h=5e-3 give residual error ~0.05 on plane waves (acceptable). Coordinate scaling: multiply each dimension's contribution by (2/L_dim)² to convert from normalized to physical metres.

**2000 microphones:** The model with 128 mics produced visually noisy fields despite good physics metrics. *99.9%* Helmholtz residual reduction but no visible wave structure. Root cause: ~800:1 param-to-data ratio. pyroomacoustics computes all mic RIRs in a single call regardless of count (128 mics → 2000 mics costs 1.2s extra). 2000 mics × 4 frequencies = 8000 samples drops the ratio to ~62:1 and produces clear wave rings.

---

## Project Structure

```
src/
  data/        generate_synthetic.py, dataset.py
  model/       network.py (SIREN MLP), encoding.py, losses.py
  physics/     helmholtz.py, boundary.py, collocation.py
  training/    trainer.py, scheduler.py, metrics.py
  visualization/ field_viz.py, export.py
  api/         main.py, routes.py

viewer/        Three.js web viewer (index.html, field_renderer.js, controls.js)
experiments/   configs/*.yaml, results/ (checkpoints, metrics.json)
docs/          raw_log.md, article_draft.md
```

---

## References

- [SIREN: Implicit Neural Representations with Periodic Activation Functions](https://arxiv.org/abs/2006.09661)
- [Physics-Informed Neural Networks (Raissi et al.)](https://arxiv.org/abs/1711.10561)
- [NeRF: Representing Scenes as Neural Radiance Fields](https://arxiv.org/abs/2003.08934)
- [pyroomacoustics](https://github.com/LCAV/pyroomacoustics)

---

*Pooya Esfahani - April 2026*