Spaces:
Running
Running
| <html lang="en"><head><meta charset="utf-8"> | |
| <meta name="viewport" content="width=device-width,initial-scale=1"> | |
| <title>Pipeline debug — raw image to force, step by step</title> | |
| <style> | |
| :root{--paper:#0d1526;--ink:#dce7f5;--dim:#7c8db0;--grid:#1a2540; | |
| --force:#ffb347;--target:#ff7847;--ok:#7be0a0;--card:#111b31;--line:#24345;} | |
| *{box-sizing:border-box} | |
| body{margin:0;background:var(--paper);color:var(--ink); | |
| font-family:'IBM Plex Sans','Noto Sans SC',sans-serif;line-height:1.65;font-size:16px; | |
| background-image:linear-gradient(var(--grid) 1px,transparent 1px), | |
| linear-gradient(90deg,var(--grid) 1px,transparent 1px);background-size:44px 44px} | |
| .wrap{max-width:880px;margin:0 auto;padding:0 24px 80px} | |
| h1,h2{font-family:'Fraunces','Noto Serif SC',serif} | |
| h1{font-size:2.1rem;font-weight:700;line-height:1.2;margin:0 0 10px} | |
| h2{font-size:1.25rem;font-weight:600;margin:46px 0 8px;color:var(--force)} | |
| .kicker{font-family:'IBM Plex Mono',monospace;font-size:.76rem;letter-spacing:.16em; | |
| text-transform:uppercase;color:var(--force);margin-bottom:14px} | |
| header{padding:60px 0 26px;border-bottom:1px dashed var(--line)} | |
| .sub{color:var(--dim);max-width:720px} | |
| .card{background:var(--card);border:1px solid var(--line);border-radius:10px; | |
| padding:18px 22px;margin:14px 0} | |
| .step{display:grid;grid-template-columns:34px 200px 1fr;gap:12px; | |
| padding:10px 0;border-bottom:1px solid var(--line);align-items:baseline} | |
| .step:last-child{border-bottom:none} | |
| .step .n{font-family:'Fraunces',serif;font-size:1.25rem;color:var(--target)} | |
| .step .t{font-family:'IBM Plex Mono',monospace;font-size:.85rem} | |
| .step .d{color:var(--dim);font-size:.9rem} | |
| table{border-collapse:collapse;width:100%;font-size:.88rem;margin:10px 0; | |
| font-variant-numeric:tabular-nums} | |
| th,td{padding:7px 10px;border-bottom:1px solid var(--line);text-align:right} | |
| th{color:var(--dim);font-family:'IBM Plex Mono',monospace;font-size:.72rem; | |
| text-transform:uppercase;letter-spacing:.08em} | |
| td:first-child,th:first-child{text-align:left} | |
| img,video{max-width:100%;height:auto;border-radius:8px;border:1px solid var(--line); | |
| display:block;margin:12px auto} | |
| a{color:#4fd8e0}a:hover{color:var(--force)} | |
| .pill{display:inline-block;border:1px solid var(--line);border-radius:999px; | |
| padding:3px 12px;font-size:.75rem;font-family:'IBM Plex Mono',monospace; | |
| color:var(--dim);margin-right:8px;margin-top:12px;text-decoration:none} | |
| code{font-family:'IBM Plex Mono',monospace;background:#0a1120;padding:1px 5px; | |
| border-radius:4px;font-size:.85em} | |
| .ok{color:var(--ok)}.warn{color:#e0b34f} | |
| footer{margin-top:60px;padding-top:18px;border-top:1px dashed var(--line); | |
| color:var(--dim);font-size:.8rem} | |
| @media(max-width:640px){.step{grid-template-columns:28px 1fr;grid-auto-flow:dense} | |
| .step .d{grid-column:2} | |
| h1{font-size:1.45rem} | |
| table{display:block;overflow-x:auto;white-space:nowrap;-webkit-overflow-scrolling:touch} | |
| th,td{white-space:normal;min-width:110px}} | |
| .srow img{width:100%;max-width:1700px;background:#fff; | |
| border-radius:6px;margin:4px 0}</style></head><body><div class="wrap"> | |
| <div class="kicker">React force recovery · pipeline debug | |
| <span style="float:right"><a href="debug_pipeline_zh.html">中文</a></span></div> | |
| <h1>From raw image to force, step by step</h1> | |
| <p>Each row: raw → reference → dI=img−ref → valid mask | |
| → |LUT gradient| → Poisson depth → features and predicted vs | |
| ground-truth force. Same pipeline, three datasets.</p> | |
| <h2>Why cnc_Mini looked broken — and what it actually was</h2> | |
| <p>Neither bad labels nor a broken reconstruction: the cnc press grid spans | |
| the full 20×16 mm pad while the cropped camera view sees only | |
| ≈13×10 mm, so two thirds of the presses are partially outside the | |
| image and their contact features are systematically underestimated. Restricted | |
| to presses fully in view, the same reconstruction reaches GlowTact-level | |
| accuracy. Label quality is fine (force CV 4.5% at fixed probe/position/depth). | |
| This retracts our earlier “dataset ceiling” argument: in-view | |
| ρ=0.94 exceeds the commanded-depth proxy (0.78–0.88) that argument | |
| relied on.</p> | |
| <table><tr><th>scope (cnc, 3351 frames)</th><th>n</th> | |
| <th>per-probe ρ (A–F)</th><th>median ρ</th><th>MAE</th></tr> | |
| <tr><td>all press positions</td><td>3351</td><td>0.09–0.16</td> | |
| <td>0.113</td><td>0.74 N</td></tr> | |
| <tr><td>interior x∈[3.5,14.5]</td><td>1232</td><td>0.79–0.85</td> | |
| <td>0.835</td><td>0.39 N</td></tr> | |
| <tr><td><b>strictly in view x∈[5,13]</b></td><td>671</td> | |
| <td><b>0.94 0.94 0.93 0.91 0.95 0.95</b></td><td><b>0.941</b></td> | |
| <td><b>0.26 N</b></td></tr></table> | |
| <p class="note">Per-probe half/half fit, 7 seeds, LUT pipeline with | |
| z-supervised gain field. Dot-type control: FEATS presses are always centered, | |
| so the raw pipeline already gives ρ=0.72 there, vs 0.46 raw on GlowTact — | |
| the gain field and scope, not markers, are the binding factors.</p> | |
| <h2>The same failure mode, on four datasets</h2> | |
| <p>The cnc story above is not a cnc story. Exact per-pixel ground truth on | |
| Tactile MNIST shows the identical effect from the other side: 420/420 touches | |
| have a contact that runs off the pad, and a control that moves one sphere cap | |
| from mid-pad to the edge collapses its reconstructed peak <b>1.39 → 0.30 | |
| mm</b> against a 0.90 mm truth. Sparsh shows it a third time — its clipped | |
| frames carry the <i>highest</i> median force and still score worse. The cause | |
| is structural: the Poisson solve's zero boundary cannot represent a surface | |
| that leaves the frame, so a clipped contact is integrated as if it ended at the | |
| image edge. <b>Contact visibility, not force range or gel type, is this | |
| pipeline's biggest external failure mode.</b></p> | |
| <p>The same ground truth also puts a number on the mask you see in column 4: | |
| against the true contact region it scores IoU <b>0.614</b> with recall | |
| <b>0.917</b> and over-segmentation <b>0.531</b> — it finds nearly all of the | |
| contact, then adds half as much again in halo. That is why a halo pedestal has | |
| to be subtracted before any 3D view.</p> | |
| <h2>GlowTact — markerless, the LUT's home sensor (control)</h2><div class='srow'> | |
| <img src='assets/debug/glowtact_01.png' loading='lazy'> | |
| <img src='assets/debug/glowtact_04.png' loading='lazy'> | |
| <img src='assets/debug/glowtact_08.png' loading='lazy'> | |
| <img src='assets/debug/glowtact_11.png' loading='lazy'> | |
| <img src='assets/debug/glowtact_14.png' loading='lazy'> | |
| <img src='assets/debug/glowtact_17.png' loading='lazy'> | |
| </div> | |
| <h2>FoTa cnc_Mini — markerless, foreign sensor, press grid larger than the field of view</h2><div class='srow'> | |
| <img src='assets/debug/cnc_01.png' loading='lazy'> | |
| <img src='assets/debug/cnc_04.png' loading='lazy'> | |
| <img src='assets/debug/cnc_08.png' loading='lazy'> | |
| <img src='assets/debug/cnc_11.png' loading='lazy'> | |
| <img src='assets/debug/cnc_14.png' loading='lazy'> | |
| <img src='assets/debug/cnc_17.png' loading='lazy'> | |
| </div> | |
| <h2>FEATS — dot/marker gel; these panels are the FORCE path, so the dots are still in them (marker inpainting ships for geometry only)</h2><div class='srow'> | |
| <img src='assets/debug/feats_01.png' loading='lazy'> | |
| <img src='assets/debug/feats_04.png' loading='lazy'> | |
| <img src='assets/debug/feats_08.png' loading='lazy'> | |
| <img src='assets/debug/feats_11.png' loading='lazy'> | |
| <img src='assets/debug/feats_14.png' loading='lazy'> | |
| <img src='assets/debug/feats_17.png' loading='lazy'> | |
| </div> | |
| <footer>React force recovery · <a href="index.html">overview</a> · | |
| <a href="results.html">results</a> · <a href="method.html">method design</a> | |
| </footer></div></body></html> |