For aligning on the host: on the Jetson rs.align.process costs ~104 ms and
holds the GIL (optimize-streams R04/R05). JSON in the usual envelope, one
response: frameset_seq, camera_timestamp_ms, timestamp_domain,
host_timestamp, color_jpeg_b64 (color grid), depth_png_b64 (16-bit PNG, mm,
depth grid), color_shape, depth_shape, depth_units. after=N / timeout_ms as
/api/color/latest. JSON + base64 like /api/capture: every client already
parses that envelope, one request gets both images, base64 costs ~0.1 ms.
Built once per frameset (EncodedRaw, last 4 kept); the color JPEG is the
color channel's own (same bytes). The depth is converted only on request
(camera.raw_frame, phase raw_depth; PNG in phase png_raw_depth).
camera: _published() shared by color_frame() and raw_frame(); _depth_mm()
extracted unchanged from _aligned_frame().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
JPEG of the latest color frame at the camera rate, not aligned to depth.
after=N waits up to timeout_ms (default 1000, max 5000) for a frame numbered
above N, so a client reads every frame without polling on empty; 504 when
none arrives. Headers: X-Frame-Seq, X-Camera-Timestamp-Ms (camera clock),
X-Camera-Timestamp-Domain, X-Host-Timestamp (arrival on the host).
Each color frame is encoded once for all clients (EncodedColor, own cache of
4: the color rate would flush the aligned frames' cache). New timing phase
jpeg_color. /api/rgb is unchanged: still the color of the aligned frame, the
same frame as /api/depth (README says so next to the new row).
color_frame() without after waits for the very first frame after open.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The depth stream is aligned to the color frame via rs.align(rs.stream.color),
so the depth map lives on the color pixel grid. /api/measure, /api/pointcloud
and /api/pointcloud/roi were de-projecting those pixels with the *depth* (ir)
intrinsics, inflating X/Y by fx_rgb/fx_ir (~1.54x on the D435i). Z was correct
since it only depends on the depth value, not the intrinsics.
A 103mm ArUco marker at ~0.73m measured ~157mm; back-projected point clouds did
not overlap other calibrated cameras.
Fix: centralize intrinsics selection in _backprojection_intrinsics(), keyed on
the new RealSenseD435.is_depth_aligned_to_color(). Aligned (USB 3 default) ->
rgb intrinsics; no alignment (USB 2 fallback, where color is resized onto the
depth grid) -> ir intrinsics. An unconditional ir->rgb swap would have broken
the USB 2 path symmetrically.
Verified live: /api/measure?u=377&v=68 now returns x=+0.0686 (rgb) instead of
+0.1092 (ir). Documented root cause + non-regression check in README and
bug_d435i_intrinsics_mismatch.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This repo is now an independent project tracking the G1 deploy. The
upstream remote points to RealSense-D435i-G1 on git.humion.it and the
default branch is `main`. The H1 repo (RealSense-D435-H1) is now a
separate sibling project, linked from the README under "Repo correlati".
Sweeps the README for stale references to `branch camera-g1`, updates
the architecture diagram and repo-tree header to the new project name,
and updates rsync/cd paths to the canonical RealSense-D435i-G1 folder.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Updates the README from the H1 description to the current G1 state:
D435i over USB 3.2, librealsense pinned to v2.50.0 for FW 05.15.x
compatibility, host-side React build, and the video_hub_pc4 stop
prerequisite. Adds a G1-vs-H1 hardware comparison table and three
new troubleshooting entries (wait_for_frames timeout, resolve
request failure, npm DNS).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop-in replacement for the Nuwa-HP60C camera container, exposing RGB +
aligned depth + intrinsics via REST/WebSocket so the existing
pestello-pose-detector pipeline can consume frames without changes.
Key adaptations for D4xx on USB 2.0 (this robot has no USB 3 port exposed
externally — see STATUS.md):
- pyrealsense2 built from source; the librealsense v2.55.1 install rule
doesn't ship the main .so, so we copy it from the build dir and add an
__init__.py that re-exports the submodule.
- Color stream uses YUYV format on USB 2.x (BGR8/RGB8 time out on this
bus); converted to BGR via cv2.cvtColor in capture().
- rs.align is skipped on USB 2 (incompatible with YUYV); color is
cv2-resized to depth shape for approximate per-pixel alignment.
- Driver auto-detects usb_type_descriptor and falls back to the
full-resolution BGR8 + rs.align pipeline when on USB 3.
Tested: /api/{health,status,discover,rgb,depth,depth/color,capture,
intrinsics} all return 200 with valid payloads at 640x480.
OliviaCandClaude Opus 5.5