---
license: apache-2.0
library_name: LiteRT
pipeline_tag: keypoint-detection
tags: [litert, tflite, on-device, android, gpu, pose-estimation, keypoint-detection, rtmpose, mmpose]
base_model: open-mmlab/mmpose
base_model_relation: quantized
---

Measured on device (edge-compat): Galaxy S26 · LiteRT 2.2.0 · GPU (ML Drift) · 3.36 ms p50 (2026-08-25); Galaxy S26 · LiteRT 2.2.0 · NPU (QNN/HTP) · 0.77 ms p50 (2026-08-25); Raspberry Pi 5 · LiteRT 2.2.0.dev20260804 · CPU/XNNPACK, 4 threads · 16.7 ms p50 (2026-08-31); browser · Chromium 151 on M4 Max · LiteRT.js 2.5.3 · WebGPU · 2.96 ms p50 · output matches CPU (2026-08-11). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/rtmpose-s/CARD.md

# RTMPose-s — LiteRT (on-device real-time 2D human pose, fully-GPU)

[RTMPose](https://github.com/open-mmlab/mmpose/tree/main/projects/rtmpose) (mmpose, CSPNeXt backbone +
RTMCC/SimCC head) top-down 2D human pose, converted to **LiteRT** and running **fully on the `CompiledModel`
GPU** (ML Drift) on Android. Estimates 17 COCO keypoints for a single centered person — the SOTA real-time
pose model, device-verified end-to-end.


![RTMPose-s — input | skeleton (on-device LiteRT GPU)](samples/sample.png)

## On-device (Pixel 8a, Tensor G3 — verified)

| | |
|---|---|
| nodes on GPU | **256 / 256** LITERT_CL (full residency) |
| inference | **~4 ms** (256×192) |
| size | 11.1 MB (fp16) |
| accuracy | device-vs-PyTorch SimCC corr **0.999**, keypoints within **0.3 px** (max 1 px) |

```
image[1,3,256,192] (ImageNet 0-255 norm) →[GPU: CSPNeXt + RTMCC]→ simcc_x[1,17,384], simcc_y[1,17,512]
```

The SimCC head emits two 1D distributions per keypoint; argmax over the bins (÷ split=2) gives the pixel x/y.

## Minimal usage

**Android (Kotlin, CompiledModel GPU)**

```kotlin
val model = CompiledModel.create(context.assets, "rtmpose_s_fp16.tflite",
    CompiledModel.Options(Accelerator.GPU), null)
val inputs = model.createInputBuffers()
val outputs = model.createOutputBuffers()
inputs[0].writeFloat(chw)              // [1,3,256,192] mmpose mean/std (0-255 RGB), NCHW
model.run(inputs, outputs)
val simccX = outputs[0].readFloat()    // [1,17,384]
val simccY = outputs[1].readFloat()    // [1,17,512]; keypoint = argmax / 2
```

**Python (desktop verification)**

```python
MEAN = np.array([123.675, 116.28, 103.53], np.float32)
STD  = np.array([58.395, 57.12, 57.375], np.float32)
import numpy as np
from PIL import Image
from ai_edge_litert.interpreter import Interpreter

img = Image.open("person.jpg").convert("RGB").resize((192, 256))  # centered subject crop
x = ((np.asarray(img, np.float32) - MEAN) / STD).transpose(2, 0, 1)[None]

it = Interpreter(model_path="rtmpose_s_fp16.tflite"); it.allocate_tensors()
it.set_tensor(it.get_input_details()[0]["index"], x); it.invoke()
od = it.get_output_details()
sx, sy = (it.get_tensor(o["index"])[0] for o in od)              # [17,384], [17,512]
if sx.shape[-1] != 384: sx, sy = sy, sx                          # identify by bin count
kx, ky = sx.argmax(-1) / 2.0, sy.argmax(-1) / 2.0                 # 17 keypoints, px in 192x256
for i, (a, b) in enumerate(zip(kx, ky)):
    print(f"kp{i}: ({a:.1f}, {b:.1f})")
```

## How it converts (litert-torch) — two numerically-exact re-authorings

Both are **on-device-only** Mali issues: they pass the desktop op-check and report full LITERT_CL residency,
yet the device output was wrong until fixed (*residency ≠ correctness*):

1. **`ScaleNorm` (RMS norm) fp16 overflow → all-zero head.** The RTMCC `ScaleNorm` input reaches ≈ |274|, so
   its channel `Σ x²` ≈ 3.6M **overflows fp16 (max 65504)** on the Mali delegate (which reduces in fp16 even
   for an fp32 graph) → `norm = ∞` → `x/∞ = 0` → the whole head collapses to zero. Fix: scale `x` down by
   S=64 **before** squaring, then rescale (math-identical) — a SafeRMSNorm.
2. **GAU attention `act@act` BMM → broadcast-reduce.** The Gated Attention Unit's `q@kᵀ` and `kernel@v` are
   activation×activation batch-matmuls that the Mali delegate mis-computes; at K=17 tokens the exact
   replacement is `(q[:,:,None,:]·k[:,None,:,:]).sum(-1)`.

Result: banned ops NONE, all tensors ≤4D, tflite-vs-torch corr **1.0**, device-vs-torch corr **0.999**.

## Preprocessing

Center-crop to 3:4, resize to 192×256, ImageNet 0-255 normalize (mean [123.675, 116.28, 103.53], std
[58.395, 57.12, 57.375]), NCHW planar. Top-down — expects one roughly-centered person.

## Performance

Measured on a **Pixel 8a** (Tensor G3, Android 16) with the standard TFLite [`benchmark_model`](https://ai.google.dev/edge/litert/models/measurement) tool — 10 warm-up runs then 50 timed runs, reported as the tool's mean.

| Runtime | Backend | Graph on GPU | Latency |
|---|---|---|---|
| LiteRT `CompiledModel` (`LITERT_CL`) | GPU | 256 / 256 | ~4 ms |
| TFLite `benchmark_model` (`TfLiteGpuDelegateV2`) | GPU (OpenCL) | 256 / 256 | 15.8 ms |
| TFLite `benchmark_model` | CPU (XNNPACK, 4 threads) | — | XNNPACK declined the graph |

**The two GPU rows are different runtimes, not a contradiction.** The `LITERT_CL` figure is the one recorded when this model shipped, taken through LiteRT's own `CompiledModel` accelerator — the path the Kotlin sample app and the LiteRT API use. The `TfLiteGpuDelegateV2` figure is the classic TFLite OpenCL delegate, measured with a tool anyone can download and re-run. They agree on how much of the graph the GPU takes; they disagree on speed, and the classic delegate is the slower of the two here. Read the `TfLiteGpuDelegateV2` row as a reproducible floor, not as this model's speed on LiteRT.

XNNPACK declines these fp16 graphs — it reports `failed to delegate DEPTHWISE_CONV_2D` and then fails to allocate tensors — so there is no usable CPU number. Disabling XNNPACK falls back to reference kernels, which measured about 20× slower than the GPU on models of this size and would not represent CPU inference anyone would ship.

## Snapdragon NPU (Hexagon)

The NPU is **4.36x faster** than the GPU (0.771 ms against 3.36 ms) and loads 10.12x faster (102 ms against 1033 ms).

| backend | compiled | inference (median / min) | load |
|---|---|---:|---:|
| NPU (Hexagon v81) | on-device JIT | 0.771 ms / 0.749 ms | 102 ms |
| GPU (Adreno) | — | 3.36 ms / 2.85 ms | 1033 ms |

Measured on a **Samsung Galaxy S26** (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT `CompiledModel` 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status `NONE` throughout. Headroom 0.79, where 1.0 is the throttling threshold.

**The NPU rows ran the published file unchanged.** LiteRT compiled it for the Hexagon on the device at first load. That first compile took 533 ms here. The `load` column above is the cached load every later run pays. Recipe and the runtime libraries it needs: [NPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-npu.md).

GPU wiring: [GPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-gpu.md).

## Raspberry Pi 5 (CPU)

Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with the LiteRT [`benchmark_model`](https://ai.google.dev/edge/litert/models/measurement) tool from litert-cli-nightly 0.2.0.dev20260805: CPU inference (XNNPACK, 4 threads), 3 invocations per file of 10 warm-up plus 50 timed runs (the tool caps a phase at 150 s, so very slow graphs run fewer — the Runs column is the actual timed total). The latency is the median across invocations; the spread is the min–max over all timed runs. No thermal throttling occurred during these runs (`vcgencmd get_throttled` stayed `0x0`).

| File | Inference (median) | Spread (min–max) | Runs | Peak memory |
|---|---:|---:|---:|---:|
| `rtmpose_s_fp16.tflite` | 16.7 ms | 16.4–19.1 ms | 181 | 138 MB |

## License

[Apache-2.0](https://github.com/open-mmlab/mmpose/blob/main/LICENSE). Upstream:
[open-mmlab/mmpose](https://github.com/open-mmlab/mmpose) RTMPose-s.