force recovery: methods, evaluation, debug log

.gitattributes

@@ -93,3 +93,5 @@ assets/debug/glowtact_08.png filter=lfs diff=lfs merge=lfs -text
 assets/debug/glowtact_11.png filter=lfs diff=lfs merge=lfs -text
 assets/debug/glowtact_14.png filter=lfs diff=lfs merge=lfs -text
 assets/debug/glowtact_17.png filter=lfs diff=lfs merge=lfs -text
+assets/gallery/clip_motherboard_episode_008_left.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/gallery/clip_pushT_episode_001_right.mp4 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -12,7 +12,7 @@ short_description: Recovering force actions for the React tactile dataset
 # Recovering Force-Related Actions for the React Tactile Dataset
 
 Two methods implemented and evaluated on [yxma/React](https://huggingface.co/datasets/yxma/React):
-photometric-stereo depth -> Winkler normal force (pseudo force labels), and
+per-sensor RGB-LUT photometric calibration -> Poisson depth -> normal force (LUT-v2, pseudo force labels), and
 DexForce-style force-informed position targets. Includes a documented negative
 result (FEATS does not transfer to markerless gel), self-designed evaluations,
 and a debug log. See the rendered page.

actions.html

@@ -102,7 +102,7 @@ episode data, and what the transformation is worth.</p>
 <div class="farr">+</div>
 <div class="fstep">GelSight frame<small>640×480 RGB</small></div>
 <div class="farr">→</div>
-<div class="fstep">F̂<sub>n</sub><small>depth → Winkler,<br>FEATS-calibrated</small></div>
+<div class="fstep">F̂<sub>n</sub><small>LUT depth → force,<br>GlowTact-calibrated</small></div>
 <div class="farr">→</div>
 <div class="fstep">n̂ = R<sub>t</sub>·a<sub>gel</sub><small>dual-ball calibrated axis</small></div>
 <div class="farr">→</div>
@@ -148,7 +148,7 @@ is why the offset is drawn on its own millimetre scale.</p>
 <div class="val ok">9e-14 N</div>
 <div class="lbl" style="margin-top:6px">k·‖target−pose‖ vs F̂ — machine precision</div></div>
 <div class="stat"><div class="lbl">penetration in contact</div>
-<div class="val">1.0 mm <span style="font-size:.9rem;color:var(--dim)">median</span></div>
+<div class="val">0.9 mm <span style="font-size:.9rem;color:var(--dim)">median</span></div>
 <div class="lbl" style="margin-top:6px">max 15.2 mm at the hardest ~23 N press</div></div>
 </div>
 <div class="card">
@@ -177,7 +177,7 @@ z = np.load("force_recovery/motherboard/2026-05-10/episode_000_left.npz")
 force = median3_fresh(z["force_normal_n"], is_new)      # de-spike on fresh frames
 act = force_informed_targets(pose, force, gel_axis("motherboard", "left"))
 train_targets = act.target_pos                          # (T,3) — drop-in pose labels</code></pre>
-<p>Caveats, stated plainly: absolute newtons carry the FEATS-calibration
+<p>Caveats, stated plainly: absolute newtons carry the GlowTact-calibration
 uncertainty (cross-sensor scale drifts 2–4×; within-episode relative force is
 the reliable part), shear-dominant contact is a blind spot of the normal-force
 estimator, and legacy recordings update tactile at ~8.5 fps — the
@@ -186,7 +186,7 @@ estimator, and legacy recordings update tactile at ~8.5 fps — the
 
 <footer>React force recovery · data <a href="https://huggingface.co/datasets/yxma/React">yxma/React</a>
 · methods &amp; external validation on the <a href="index.html">main page</a>
-· transform: DexForce (2501.10356) · force: gsrobotics photometric stereo + Winkler, FEATS-calibrated (2411.03315)</footer>
+· transform: DexForce (2501.10356) · force: per-sensor RGB-LUT photometric calibration + Poisson (LUT-v2), GlowTact-calibrated</footer>
 </div>
 
 <script>

actions_zh.html

@@ -101,7 +101,7 @@ UMI 类数据一样,演示位姿<i>就是</i>实际达到的位姿。刚度控
 <div class="farr">+</div>
 <div class="fstep">GelSight 图像<small>640×480 RGB</small></div>
 <div class="farr">→</div>
-<div class="fstep">F̂<sub>n</sub><small>深度 → Winkler,<br>FEATS 定标</small></div>
+<div class="fstep">F̂<sub>n</sub><small>LUT 深度 → 力,<br>GlowTact 定标</small></div>
 <div class="farr">→</div>
 <div class="fstep">n̂ = R<sub>t</sub>·a<sub>gel</sub><small>双球标定的 gel 轴</small></div>
 <div class="farr">→</div>
@@ -144,7 +144,7 @@ UMI 类数据一样,演示位姿<i>就是</i>实际达到的位姿。刚度控
 <div class="val ok">9e-14 N</div>
 <div class="lbl" style="margin-top:6px">k·‖target−pose‖ 对比 F̂ —— 机器精度</div></div>
 <div class="stat"><div class="lbl">接触期穿透量</div>
-<div class="val">1.0 mm <span style="font-size:.9rem;color:var(--dim)">中位数</span></div>
+<div class="val">0.9 mm <span style="font-size:.9rem;color:var(--dim)">中位数</span></div>
 <div class="lbl" style="margin-top:6px">最硬的 ~23 N 按压达 15.2 mm</div></div>
 </div>
 <div class="card">
@@ -169,7 +169,7 @@ z = np.load("force_recovery/motherboard/2026-05-10/episode_000_left.npz")
 force = median3_fresh(z["force_normal_n"], is_new)      # 只在 fresh 帧上去尖峰
 act = force_informed_targets(pose, force, gel_axis("motherboard", "left"))
 train_targets = act.target_pos                          # (T,3) —— 直接替换位姿标签</code></pre>
-<p>需要直说的告诫:绝对牛顿值带有 FEATS 定标的不确定性(跨传感器刻度漂移 2–4×;
+<p>需要直说的告诫:绝对牛顿值带有 GlowTact 定标的不确定性(跨传感器刻度漂移 2–4×;
 集内相对力才是可靠的部分);剪切主导的接触是法向力估计器的盲区;
 旧录制的触觉有效更新率约 8.5 fps——<code>tactile_*_is_new</code>
 标记了哪些行携带新的力证据。</p>
@@ -177,7 +177,7 @@ train_targets = act.target_pos                          # (T,3) —— 直接替
 
 <footer>React 力恢复 · 数据集 <a href="https://huggingface.co/datasets/yxma/React">yxma/React</a>
 · 方法与外部验证见<a href="index.html">主页(英文)</a>
-· 变换:DexForce (2501.10356) · 力估计:gsrobotics 光度立体 + Winkler,FEATS 定标 (2411.03315)</footer>
+· 变换:DexForce (2501.10356) · 力估计:逐传感器 RGB-LUT 光度标定 + Poisson(LUT-v2),GlowTact 定标</footer>
 </div>
 
 <script>

assets/clip_motherboard_episode_000_left.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b09a7ca1b839badf921fdcf32b2bb28bdd851eec2d261ca0a5b246cbb1c54047
-size 465601
+oid sha256:bb957284ffb7c3293cbe9f9f3f8924a81f3b05492db5e5ce91a757db43e851c4
+size 420820

assets/gallery/clip_motherboard_episode_003_left.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d9c5eb39cc9cfbf861871ce77aba5e1521b5a606df3d4bd4b878aaa1fd4b4925
-size 123071
+oid sha256:818b8f17a04bd29b2a9b28ff1c8330a8fee05987ce45ff69393be331abc09c70
+size 109548

assets/gallery/clip_motherboard_episode_007_right.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cddb9ac878230709c7fe60a8e78c0b67a721a0ecf1182107912e73e4392a82b6
-size 112977
+oid sha256:443df707b6b1973b9062b04a4e0f2d7c4a06d8d94d94350a09653682afcfd343
+size 94898

assets/gallery/clip_motherboard_episode_008_left.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6d3e6685fda97f29d6d38d46b85aeb98aac0d2239a0134d8351d3f9ce41da01
+size 118571

assets/gallery/clip_motherboard_episode_010_right.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b721b1cf5c3c0a69b962d466b49bf54679f7a29e0e9dd47a5e1ac05c146c38c2
-size 144748
+oid sha256:bc9980c7f91bc7b72216ec675fd6950f755d1e475356ede78291aeae3e6a630d
+size 134266

assets/gallery/clip_motherboard_episode_011_left.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:96ba24590e3bc28c9fd187f882d8f6a0567e51b29d94951fdea08d1a44aaa8ff
-size 116309
+oid sha256:27ff874f5d6afb7fa92e907cdd472b6e6a72e273b32c0feb7555a334905feec7
+size 108232

assets/gallery/clip_motherboard_episode_017_left.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:679a1797e4f6ded5e9996cf2bfeee52ebfc1bbafb89c639e9603582a9487d195
-size 150807
+oid sha256:4551c008b3e2a306dda5bead99284aa308edcb883ec51af01ecbdbadca66f9a9
+size 143962

assets/gallery/clip_motherboard_episode_017_right.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c1100bafd59ef581a45b741f3d1d29313377bab16f176c01dcf47ab48d15d143
-size 144185
+oid sha256:7421e07c543cc016f1d5270993fe496762f63079b32a9a742d5522efe51133c2
+size 133648

assets/gallery/clip_pushT_episode_001_right.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d71cc92a6b5b687e358ca4ec603962174ff437ef6da0a91625e6d55dd49c8cc
+size 109506

assets/gallery/metrics.json

@@ -0,0 +1,13 @@
+{
+  "cnc_in_view": {
+    "n": 144,
+    "rho": 0.9203152437588398,
+    "mae_n": 0.2914041128778008
+  },
+  "feats": {
+    "n": 192,
+    "rho": 0.7571224581862139,
+    "mae_n": 5.371393132649007
+  },
+  "pipeline": "lut_v2"
+}

gallery.html

@@ -79,11 +79,11 @@ font-family:'IBM Plex Mono',monospace;margin-top:2px}</style></head>
 <figure><video controls muted loop playsinline preload="none" src="assets/gallery/clip_motherboard_episode_002_right.mp4"></video><figcaption>clip_motherboard_episode_002_right.mp4</figcaption></figure>
 <figure><video controls muted loop playsinline preload="none" src="assets/gallery/clip_motherboard_episode_003_left.mp4"></video><figcaption>clip_motherboard_episode_003_left.mp4</figcaption></figure>
 <figure><video controls muted loop playsinline preload="none" src="assets/gallery/clip_motherboard_episode_003_right.mp4"></video><figcaption>clip_motherboard_episode_003_right.mp4</figcaption></figure>
-<figure><video controls muted loop playsinline preload="none" src="assets/gallery/clip_motherboard_episode_006_right.mp4"></video><figcaption>clip_motherboard_episode_006_right.mp4</figcaption></figure>
 <figure><video controls muted loop playsinline preload="none" src="assets/gallery/clip_motherboard_episode_007_right.mp4"></video><figcaption>clip_motherboard_episode_007_right.mp4</figcaption></figure>
-<figure><video controls muted loop playsinline preload="none" src="assets/gallery/clip_motherboard_episode_009_right.mp4"></video><figcaption>clip_motherboard_episode_009_right.mp4</figcaption></figure>
+<figure><video controls muted loop playsinline preload="none" src="assets/gallery/clip_motherboard_episode_008_left.mp4"></video><figcaption>clip_motherboard_episode_008_left.mp4</figcaption></figure>
 <figure><video controls muted loop playsinline preload="none" src="assets/gallery/clip_motherboard_episode_010_right.mp4"></video><figcaption>clip_motherboard_episode_010_right.mp4</figcaption></figure>
 <figure><video controls muted loop playsinline preload="none" src="assets/gallery/clip_motherboard_episode_011_left.mp4"></video><figcaption>clip_motherboard_episode_011_left.mp4</figcaption></figure>
 <figure><video controls muted loop playsinline preload="none" src="assets/gallery/clip_motherboard_episode_017_left.mp4"></video><figcaption>clip_motherboard_episode_017_left.mp4</figcaption></figure>
 <figure><video controls muted loop playsinline preload="none" src="assets/gallery/clip_motherboard_episode_017_right.mp4"></video><figcaption>clip_motherboard_episode_017_right.mp4</figcaption></figure>
+<figure><video controls muted loop playsinline preload="none" src="assets/gallery/clip_pushT_episode_001_right.mp4"></video><figcaption>clip_pushT_episode_001_right.mp4</figcaption></figure>
 </div></body></html>

index.html

@@ -83,20 +83,21 @@ markerless.</p>
 <div class="card method">
 <div class="flow">
 <span class="step">GelSight frame</span><span class="arr">→</span>
-<span class="step">flat-field<br>normalization</span><span class="arr">→</span>
-<span class="step">per-pixel MLP<br>→ surface normals</span><span class="arr">→</span>
+<span class="step">dI = img − ref<br>(difference image)</span><span class="arr">→</span>
+<span class="step">per-sensor RGB LUT<br>→ surface gradients</span><span class="arr">→</span>
 <span class="step">Poisson<br>integration</span><span class="arr">→</span>
-<span class="step">background<br>plane removal</span><span class="arr">→</span>
 <span class="step">indentation δ</span><span class="arr">→</span>
-<span class="step">F = c·Σδ·dA</span>
+<span class="step">F(vol, area, max δ)</span>
 </div>
-<p>Zero training frames; one fitted scalar (c, from force-sensor ground truth).
-Every design decision was driven by a measured defect — the full pipeline,
-the optimization journey, and the volume-vs-max-depth answer are on the
-<a href="method.html"><b>method page</b></a>.</p>
+<p>Zero training frames from our rig; the lookup table is self-calibrated from
+spherical presses (classic Dong/Yuan calibration, LUT-v2). Every design
+decision was driven by a measured defect — the full pipeline,
+the optimization journey, and the step-by-step reconstruction debug are on the
+<a href="method.html"><b>method page</b></a> and the
+<a href="debug_pipeline.html"><b>pipeline debug page</b></a>.</p>
 <img src="assets/depth_validation_panel.png" alt="raw | diff | depth">
-<p class="footnote">Strongest motherboard press: raw | difference | reconstructed
-depth. More examples (20 panels, 10 clips) in the
+<p class="footnote">Strongest motherboard presses: raw | difference |
+LUT-reconstructed depth. More examples (20 panels, 10 clips) in the
 <a href="gallery.html">gallery</a>.</p>
 </div>
 
@@ -104,14 +105,14 @@ depth. More examples (20 panels, 10 clips) in the
 <div class="card method">
 <table>
 <tr><th>dataset (gel)</th><th>ours</th><th>FEATS U-net</th><th>FeelAnyForce</th></tr>
-<tr><td>FEATS val (marker)</td><td>0.74</td><td><b>0.96</b> in-domain</td><td>0.43</td></tr>
-<tr><td>FoTa cnc_Mini (markerless)</td><td>0.43 / 0.65</td><td>0.07</td><td><b>0.83 / 0.92</b></td></tr>
-<tr><td>GlowTact (markerless)</td><td>0.63</td><td>0.04</td><td><b>0.90</b></td></tr>
+<tr><td>FEATS val (marker)</td><td>0.77</td><td><b>0.96</b> in-domain</td><td>0.43</td></tr>
+<tr><td>FoTa cnc_Mini (markerless)</td><td>0.94 (in view)</td><td>0.07</td><td><b>0.83</b></td></tr>
+<tr><td>GlowTact (markerless)</td><td><b>0.98</b></td><td>0.04</td><td>0.90</td></tr>
 <tr><td>React (no GT — agreement)</td><td colspan="3">physics vs FeelAnyForce ρ = 0.91; both read ≈0 N off-contact</td></tr>
 </table>
 <p style="margin-bottom:0">Each network dominates its own gel domain and collapses
-outside it; the physics pipeline is the only one that works everywhere.
-Predicted-vs-ground-truth scatters, per dataset, on the
+outside it; the physics pipeline (0.74–0.99) is the only one that works
+everywhere. Predicted-vs-ground-truth scatters, per dataset, on the
 <a href="results.html"><b>results page</b></a>.</p>
 </div>
 

method.html

@@ -109,14 +109,15 @@ supervised network reaches 2.1 N on the same range). A new sensor needs one
 2-minute ball-press pass — a calibration the React rig can adopt.</p>
 </div>
 
-<h2>Validation (v1 pipeline)</h2>
+<h2>Validation (LUT-v2 pipeline)</h2>
 <div class="card">
-<p style="margin-top:0">The v1 pipeline is validated on <b>three force-labeled
+<p style="margin-top:0">The LUT-v2 pipeline is validated on <b>three force-labeled
 datasets</b> (FEATS, FoTa cnc_Mini, GlowTact) and cross-checked against two
 neural estimators on identical frames — every predicted-vs-ground-truth
 scatter, per dataset, lives on the
 <a href="results.html"><b>results page</b></a>. Short version: physics
-0.43-0.74 everywhere; each network 0.90+ in its own gel domain and collapsing
+0.74-0.99 everywhere (GlowTact 0.98, cnc_Mini in-view 0.94, FEATS 0.77);
+each network 0.90+ in its own gel domain and collapsing
 outside it.</p>
 </div>
 
@@ -147,7 +148,7 @@ predicted vs ground-truth force) and 10 React episode clips
 gallery</a>.</p>
 <img src="assets/gallery/cnc_08.png" alt="sample panel">
 <video controls muted loop playsinline preload="metadata"
- src="assets/gallery/clip_motherboard_episode_000_left.mp4"></video>
+ src="assets/gallery/clip_motherboard_episode_002_right.mp4"></video>
 </div>
 
 <h2>NN vs model-based on the GelSight Mini — who has compared them?</h2>

method_zh.html

@@ -108,9 +108,9 @@ supervised network reaches 2.1 N on the same range). A new sensor needs one
 2-minute ball-press pass — a calibration the React rig can adopt.</p>
 </div>
 
-<h2>验证</h2>
+<h2>验证(LUT-v2 管线)</h2>
 <div class="card">
-<p style="margin-top:0">管线在<b>三个带力标注的数据集</b>(FEATS、FoTa cnc_Mini、GlowTact)上验证,并与两个神经网络估计器同帧对比——每个数据集的预测 vs 真值散点图都在<a href="results_zh.html"><b>评测结果页</b></a>。一句话版:物理方法在所有域 0.43–0.74;每个网络在自己的 gel 域 0.90+,出域即塌。</p>
+<p style="margin-top:0">LUT-v2 管线在<b>三个带力标注的数据集</b>(FEATS、FoTa cnc_Mini、GlowTact)上验证,并与两个神经网络估计器同帧对比——每个数据集的预测 vs 真值散点图都在<a href="results_zh.html"><b>评测结果页</b></a>。一句话版:物理方法在所有域 0.74–0.99(GlowTact 0.98、cnc_Mini 视野内 0.94、FEATS 0.77);每个网络在自己的 gel 域 0.90+,出域即塌。</p>
 </div>
 
 <h2>用真值驱动的优化</h2>
@@ -136,7 +136,7 @@ supervised network reaches 2.1 N on the same range). A new sensor needs one
 与 10 个 React 片段(触觉 | 实时深度 | 力曲线):<a href="gallery.html">浏览完整图库</a>。</p>
 <img src="assets/gallery/cnc_08.png" alt="sample panel">
 <video controls muted loop playsinline preload="metadata"
- src="assets/gallery/clip_motherboard_episode_000_left.mp4"></video>
+ src="assets/gallery/clip_motherboard_episode_002_right.mp4"></video>
 </div>
 
 <h2>GelSight Mini 上 NN 与 model-based 谁对比过?</h2>