fix: use color intrinsics for back-projection when depth is aligned

The depth stream is aligned to the color frame via rs.align(rs.stream.color),
so the depth map lives on the color pixel grid. /api/measure, /api/pointcloud
and /api/pointcloud/roi were de-projecting those pixels with the *depth* (ir)
intrinsics, inflating X/Y by fx_rgb/fx_ir (~1.54x on the D435i). Z was correct
since it only depends on the depth value, not the intrinsics.

A 103mm ArUco marker at ~0.73m measured ~157mm; back-projected point clouds did
not overlap other calibrated cameras.

Fix: centralize intrinsics selection in _backprojection_intrinsics(), keyed on
the new RealSenseD435.is_depth_aligned_to_color(). Aligned (USB 3 default) ->
rgb intrinsics; no alignment (USB 2 fallback, where color is resized onto the
depth grid) -> ir intrinsics. An unconditional ir->rgb swap would have broken
the USB 2 path symmetrically.

Verified live: /api/measure?u=377&v=68 now returns x=+0.0686 (rgb) instead of
+0.1092 (ir). Documented root cause + non-regression check in README and
bug_d435i_intrinsics_mismatch.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
henrytherobotandClaude Opus 4.7 committed 2026-05-27 16:29:30 +02:00
1 parent f1f1277755
commit 8bfdfb194a
4 files changed
+239 -26

No files matched your search

+10
View File
@@ -132,6 +132,16 @@ class RealSenseD435:
"""False when the pipeline runs depth-only (USB 2.x fallback)."""
return self._color_available
def is_depth_aligned_to_color(self) -> bool:
"""True when depth is hardware-aligned to the color frame.
When aligned (USB 3 default), the depth map lives on the **color**
pixel grid, so back-projecting a pixel must use the color (rgb)
intrinsics. On USB 2 the align step is skipped and color is resized
onto the depth grid instead, so the depth (ir) intrinsics apply.
"""
return self._align is not None
def usb_type(self) -> str:
return self._usb_type