Spaces:

Pooya-sh1998
/

acoustic-nerf

Sleeping

App Files Files Community

acoustic-nerf / README.md

Pooyash1998

add HF Spaces badge to README

945cdd4 5 months ago

preview code

Raw

History Blame Contribute Delete

6.3 kB

metadata

title: PI-AcousticNeRF
colorFrom: blue
colorTo: indigo
sdk: docker
pinned: false
license: mit
short_description: 3D acoustic field reconstruction with physics constraints

Physics-Informed Acoustic NeRF

Reconstruct a continuous 3D sound pressure field from sparse microphone measurements, with the Helmholtz wave equation enforced as a hard constraint during training.

On Apple M4 Pro (48GB VRam) · No cloud GPU

What It Does

A microphone hears one point. This model hears the whole room.

Given a handful of microphone measurements at multiple frequencies, the network learns a function (x, y, z, ω) → complex pressure p̂ that predicts acoustic pressure at any point in 3D space — not just where the mics are. Physics enforcement means the predicted field actually obeys the wave equation, not just the training data.

Architecture

Input: (x, y, z, ω)
         │
   Positional encoding
   L=10 spatial, L=6 frequency → 76 dims
         │
   SIREN MLP  ──── skip at layer 4 ────┐
   8 layers, hidden_dim=256            │
   sine activations (ω₀=1.0)          │
         │◄───────────────────────────┘
   Output: (Re(p), Im(p))   ~500k params

Training losses:
  L_data     = MSE at microphone positions
  L_physics  = |∇²p̂ + k²p̂|²  at collocation points  (Helmholtz)
  L_boundary = |∂p̂/∂n + jkβp̂|²  at room walls       (impedance BC)
  L_total    = λ_d · L_data + λ_p · L_physics + λ_b · L_boundary

Physics weight schedule: λ_physics ramps from 1e-7 → 1e-3 over 3000 epochs so the data loss stabilises before the constraint kicks in.

Results

Config	Val NMSE	Phys. Residual	Notes
Baseline (data only)	1.783	1124 Pa²/m⁴	Noisy, no wave structure
+ Boundary BC only	1.140	1146 Pa²/m⁴	No improvement
+ Helmholtz only	0.969	1.71 Pa²/m⁴	Wave rings appear
Full (Helmholtz + BC)	0.983	1.56 Pa²/m⁴	Lowest residual

Physics residual reduction: 99.9% (baseline 1124 → physics-informed 1.56 Pa²/m⁴)

Room: 6 × 5 × 3 m shoebox · 2000 microphones · 4 frequencies (500/1k/2k/4kHz)
Training: ~45 min on Apple M4 Pro (MPS backend)

Interactive Viewer

The web viewer renders a horizontal slice at z = 1.5 m with a diverging RdBu colormap (blue = low pressure, red = high). Toggle between physics-informed (A) and baseline (B) models to see the difference.

# Generate field data from trained checkpoints
python -m src.visualization.export

# Start the FastAPI server + viewer
uvicorn src.api.main:app --host 0.0.0.0 --port 8000

# Open browser
open http://localhost:8000/viewer/index.html

Features: frequency toggle (500 Hz / 1 kHz / 2 kHz / 4 kHz), model A/B comparison, microphone overlay, Helmholtz residual overlay.

Quick Start

# 1. Environment
conda create -n acoustic-nerf python=3.11
conda activate acoustic-nerf
pip install -r requirements.txt

# 2. Generate synthetic room data and train
python -m src.training.trainer --config experiments/configs/physics_informed.yaml

# 3. Run ablation studies
python -m src.training.trainer --config experiments/configs/baseline.yaml
python -m src.training.trainer --config experiments/configs/ablation_physics_only.yaml

# 4. Collect metrics
python -m src.training.collect_metrics

# 5. Export and serve
python -m src.visualization.export
uvicorn src.api.main:app

Requirements: PyTorch (MPS or CUDA), pyroomacoustics, FastAPI, Three.js (CDN)

The Physics

The Helmholtz wave equation for steady-state acoustics:

∇²p(x,y,z) + k²p(x,y,z) = 0

where k = ω/c (wave number, c = 343 m/s). Enforced as a residual loss at random collocation points throughout the room volume each training batch. Uses finite-difference Laplacian (MPS-stable) rather than autograd second derivatives.

Boundary condition at room walls:

∂p/∂n + jkβp = 0

where β = 0.4 is the specific admittance (absorption coefficient).

Key Implementation Decisions

SIREN activations (ω₀=1.0, not 30): Sine activations give smooth second derivatives, critical for the Laplacian computation. ω₀=30 was tested but caused the physics loss to blow up (second derivatives O(1e9) with 8 stacked layers). Lowered to 1.0 since positional encoding already handles high frequencies.

Finite difference Laplacian: PyTorch autograd create_graph=True is unstable on MPS for second derivatives. Central differences with h=5e-3 give residual error ~0.05 on plane waves (acceptable). Coordinate scaling: multiply each dimension's contribution by (2/L_dim)² to convert from normalized to physical metres.

2000 microphones: The model with 128 mics produced visually noisy fields despite good physics metrics. 99.9% Helmholtz residual reduction but no visible wave structure. Root cause: ~800:1 param-to-data ratio. pyroomacoustics computes all mic RIRs in a single call regardless of count (128 mics → 2000 mics costs 1.2s extra). 2000 mics × 4 frequencies = 8000 samples drops the ratio to ~62:1 and produces clear wave rings.

Project Structure

src/
  data/        generate_synthetic.py, dataset.py
  model/       network.py (SIREN MLP), encoding.py, losses.py
  physics/     helmholtz.py, boundary.py, collocation.py
  training/    trainer.py, scheduler.py, metrics.py
  visualization/ field_viz.py, export.py
  api/         main.py, routes.py

viewer/        Three.js web viewer (index.html, field_renderer.js, controls.js)
experiments/   configs/*.yaml, results/ (checkpoints, metrics.json)
docs/          raw_log.md, article_draft.md

References

Pooya Esfahani - April 2026