React force recovery · results

Three estimators × three ground-truth datasets

Every method evaluated on every dataset with force-sensor labels, predicted vs ground truth per dataset — so dataset quality is controlled within each row, and differences between panels are differences between methods.

↖ how the method is designed overview gallery 中文

The matrix (Spearman ρ, predicted vs F/T ground truth)

dataset (gel type)Ours — physics,
0 training frames
FEATS U-net
(trained: marker gel)
FeelAnyForce
(trained: markerless)
FEATS val (marker)0.77 0.96 · in-domain0.43
FoTa cnc_Mini (markerless)0.94 (in view) 0.070.83
GlowTact (markerless)0.98 0.040.90

The pattern is the finding: each network dominates its own gel domain and collapses outside it (FEATS 0.96 → 0.04–0.07; FeelAnyForce 0.90 → 0.43), while the physics pipeline is the only estimator that works everywhere (0.74–0.99) — it never sees training data, so it has no domain to leave.

FEATS dataset — marker-dot gel

FEATS dataset panels

In-domain, the FEATS U-net is excellent (ρ=0.96) — the negative results elsewhere are domain effects, not a weak model. FeelAnyForce, markerless-trained, degrades on the dotted gel (0.43): the same knife cuts both ways.

FoTa cnc_Mini — markerless gel

cnc_Mini panels

Hard conditions: only 4 contact-free frames, 62% of presses near the pad border — the press grid is larger than the field of view (see pipeline debug). Strictly in view our ρ is 0.94 (MAE 0.26 N); FeelAnyForce reaches 0.92 non-edge.

GlowTact — markerless gel, cleaned

GlowTact panels

Friendliest ground truth (centred presses, 10 free frames, 0–20 N). Per-indenter, calibrated within each family under the physical scope (contact fully in view, gel not bottomed out), ρ is 0.975–0.992 across all six indenters with MAE ≤ 0.73 N. Caveat: that calibration is refit per family, so it measures rank agreement within a group, not a transferable absolute-newton scale — see per-dataset calibration.

Sparsh (Meta) — a fourth dataset, and where the method breaks

Sparsh results

10 gel pads (6 sphere, 2 flat, 2 sharp), force in newtons. Our GlowTact table applied to this foreign sensor reaches ρ=0.878 on in-view frames. Rebuilding the table from Sparsh's own sphere presses — 708 frames, radius fitted at R=2.44 mm from a²=d(2R−d) — takes it to ρ=0.968, MAE 0.042 N, against a labels-shuffled-within-pad control of 0.23. Fitting on one pad and applying it unchanged to another costs nothing measurable (0.96–0.98 everywhere): one table, six gel pads.

dome before and after

Why the table matters more than the fit: with the wrong sensor's table a sphere press integrates to a bilobed shape with a central dip; with the self-calibrated table it is a single dome matching the analytic spherical cap (residual RMS 0.179 → 0.0545 mm). Measured before any integration, the LUT gradient sits 93.3° from the analytic sphere gradient — chance is 90° — and self-calibration brings it to 4.5° (within 30°: 15% → 99%). Reverse control: the Sparsh table fails on GlowTact frames too, so this is a per-sensor property, not a bad table.

What this costs and where it still fails. The price is one set of sphere presses with logged depth on the target sensor: this is calibrate once per sensor, not zero-shot. Rank order transfers across indenter shapes but absolute newtons do not — a sphere-fitted model applied to a flat punch degrades to MAE 0.37–0.40 N, and it got worse with the correct table, because true geometry widens the real feature-scale gap between a sphere and a punch. The sharp indenter is unsupported (in-view ρ 0.58). Shear stays out of reach by construction — the top shear decile keeps 1.6× the residual whichever table is used. Frames are restricted to a visible contact disc: 36% of presses show none and 11% are clipped, and the clipped subset carries the highest median force yet scores worse, so this is visibility, not force-range filtering. Three dataset defects had to be fixed first: flat/sharp trajectories ship 5 more frame indices than forces (silently drifting the labels, ρ≈0 until paired within each trajectory), sharp/batch_2 is stored BGR while the other nine are RGB, and flat/batch_2 ships only 3 of 4 image files.

React — no ground truth, so: do independent methods agree?

React agreement

On the dataset we actually care about, the two surviving estimators — physics (zero training) and FeelAnyForce (200K frames) — agree at ρ=0.91, and FeelAnyForce reads ≈0 N on every frame the physics pipeline calls contact-free. Neither can copy the other's mistakes.

Takeaway

For labelling React (markerless Mini): FeelAnyForce as the primary labeller, the physics pipeline as an independent audit, disagreement rows flagged. For any new gel or sensor where no trained model matches the domain: the physics pipeline is the only option that works out of the box — and its FEATS-dataset score (0.74) shows what it does on a domain nobody tuned it for.

Pipeline debug page: raw image → force step by step on all three datasets, and the cnc field-of-view ablation (in-view ρ=0.94).