---
license: other
license_name: amuse-research-license
license_link: https://github.com/kiranchhatre/amuse/blob/main/LICENSE
extra_gated_heading: Request access to the AMUSE research artifacts
extra_gated_description: Access is manually reviewed. For questions, contact amuse@tue.mpg.de or chhatre@kth.se.
extra_gated_button_content: Request access
extra_gated_prompt: By requesting access you agree to use these artifacts only under the AMUSE non-commercial research license.
extra_gated_fields:
  Affiliation: text
  Intended use: text
  I agree to the AMUSE license terms: checkbox
tags:
  - speech-driven-gesture-generation
  - 3d-motion
  - emotion
  - smplx
library_name: pytorch
datasets:
  - kiranchhatre/amuse-beat
---

# AMUSE LPDM

Paper page: [Hugging Face Papers 2312.04466](https://huggingface.co/papers/2312.04466)  
ArXiv: [2312.04466](https://arxiv.org/abs/2312.04466)

This gated model repo contains the released AMUSE LPDM checkpoint folder `LPDM_20231028-210758_actors_smplx`
plus a lightweight NPZ-only inference wrapper.

## What is included

- `saved-models/LPDM_20231028-210758_actors_smplx/`
- `configs/`, `dm/`, and `models/` required for inference
- `infer_npz.py`
- `configs/release_base.json`

## What is not included

- SMPL-X model files
- Blender resources and Blender add-ons
- Purchased third-party assets

SMPL-X and Blender are not needed to generate motion NPZs. The released inference path outputs:

- `poses`: axis-angle joint rotations with shape `[T, 55, 3]`
- `trans`: root translation with shape `[T, 3]`
- `mocap_frame_rate`: `30.0`

## Quick start

```bash
python infer_npz.py \
  --audio /path/to/input.wav \
  --output /path/to/output_motion.npz
```

The script uses the local LPDM checkpoint in this repo and will fetch the gated audio checkpoint
from `kiranchhatre/amuse-audio` automatically if it is not already available locally.

## Notes

- The released model is tuned for 10-second chunks at 16 kHz audio.
- Longer audio is processed chunk-by-chunk and concatenated in time.
- No Blender or SMPL-X downloads are required for this NPZ-only path.

## Citation

If you use these artifacts, please cite AMUSE:


```bibtex
@InProceedings{Chhatre_2024_CVPR,
    author    = {Chhatre, Kiran and Daněček, Radek and Athanasiou, Nikos and Becherini, Giorgio and Peters, Christopher and Black, Michael J. and Bolkart, Timo},
    title     = {{AMUSE}: Emotional Speech-driven {3D} Body Animation via Disentangled Latent Diffusion},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {1942-1953},
    url       = {https://amuse.is.tue.mpg.de},
}
```