- Endpoints that wait for the camera or encode/compute (capture, rgb, depth,
depth/color, pointcloud, pointcloud/roi, measure, snapshot, and the SDK
calls discover/open/close) are plain `def`: FastAPI runs them in its
thread pool, so one slow request no longer holds the others. Plain `def`
rather than asyncio.to_thread in each route: it is FastAPI's own mechanism
and keeps the route bodies unchanged.
- /ws and the NATS snapshot stay async and use asyncio.to_thread for the
frame wait, the encoding and the recording write.
- EncodedFrame: the JPEG q85, 16-bit PNG, colormap JPEG q90, their base64
and the depth stats of a frame are computed once, on first use, and
shared: parallel requests, /ws and NATS on the same frame get the same
bytes. The last 4 frames' encodings are kept.
- camera.cached_frame(): the whole cached frame (get_rgb/get_depth unchanged).
Contract unchanged: same endpoints, same JSON fields. frame_id still counts
responses (now under a lock); two parallel /api/capture served by one frame
have different frame_id and the same timestamp and images.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FrameAcquisition: a single thread calls wait_for_frames + align (the old
capture() body, now _grab_once) and keeps the latest frame, numbered.
capture() no longer touches the pipeline: it waits for the next frame of
that thread; requests waiting at the same moment get the same frame, so N
callers cost one grab. get_rgb()/get_depth() keep their 100 ms cache,
now on the thread's latest frame (_last_frame is gone).
The thread grabs on demand, only while someone waits for a newer frame:
grabbing continuously at 30 fps would align frames nobody reads, and align
on the Jetson CPU is the main suspect of the capture cost. A grab error is
logged and retried, the thread never dies. close() stops it before the
pipeline and joins it after (pipeline.stop() ends a wait in progress).
open/close and camera_open are unchanged, so the healthcheck + autoheal
recovery of ede70a2 sees the same state as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One log line per phase every PHASE_LOG_EVERY samples (default 100, 0 = off):
mean and max of wait_frames, align, convert, each JPEG/PNG encoding,
colormap, depth_stats, base64, json and ws_send. Meant for the measurement
on PC2 via `docker logs` (optimize-streams R01): where a capture spends its
~130 ms of CPU is not known yet.
capture() moves its numpy conversion into _frame_from() unchanged, only so
the block can be timed; _depth_colormap() likewise wraps _jet_colormap().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The depth stream is aligned to the color frame via rs.align(rs.stream.color),
so the depth map lives on the color pixel grid. /api/measure, /api/pointcloud
and /api/pointcloud/roi were de-projecting those pixels with the *depth* (ir)
intrinsics, inflating X/Y by fx_rgb/fx_ir (~1.54x on the D435i). Z was correct
since it only depends on the depth value, not the intrinsics.
A 103mm ArUco marker at ~0.73m measured ~157mm; back-projected point clouds did
not overlap other calibrated cameras.
Fix: centralize intrinsics selection in _backprojection_intrinsics(), keyed on
the new RealSenseD435.is_depth_aligned_to_color(). Aligned (USB 3 default) ->
rgb intrinsics; no alignment (USB 2 fallback, where color is resized onto the
depth grid) -> ir intrinsics. An unconditional ir->rgb swap would have broken
the USB 2 path symmetrically.
Verified live: /api/measure?u=377&v=68 now returns x=+0.0686 (rgb) instead of
+0.1092 (ir). Documented root cause + non-regression check in README and
bug_d435i_intrinsics_mismatch.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Read width/height/fps from the live pipeline profile instead of
hardcoding 640x480, so /status reflects the firmware-accepted
resolution (e.g. the USB 2.x 640x480@15 fallback) rather than the
requested config.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mount mscli + master_service.sock into the container and stop
video_hub_pc4 at startup (and every 10s) so the RealSense pipeline
can open the D435i V4L2 node without EBUSY.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sostituisce il healthcheck banale (urlopen /api/health) con uno che
fallisce quando camera_open != true, e aggiunge il container willfarrell/autoheal
che restarta il backend quando Docker lo marca unhealthy. Recovery <1min
da SDK stuck senza intervento SSH.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This repo is now an independent project tracking the G1 deploy. The
upstream remote points to RealSense-D435i-G1 on git.humion.it and the
default branch is `main`. The H1 repo (RealSense-D435-H1) is now a
separate sibling project, linked from the README under "Repo correlati".
Sweeps the README for stale references to `branch camera-g1`, updates
the architecture diagram and repo-tree header to the new project name,
and updates rsync/cd paths to the canonical RealSense-D435i-G1 folder.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Updates the README from the H1 description to the current G1 state:
D435i over USB 3.2, librealsense pinned to v2.50.0 for FW 05.15.x
compatibility, host-side React build, and the video_hub_pc4 stop
prerequisite. Adds a G1-vs-H1 hardware comparison table and three
new troubleshooting entries (wait_for_frames timeout, resolve
request failure, npm DNS).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The G1's head camera is a D435i (USB 3.2, FW 05.15.01.55) — same family
as the H1's D435 but on a different bus and with older firmware. The
existing image opened the device but wait_for_frames timed out
indefinitely; this turned out to be a librealsense v2.55.x regression
against the D4xx firmware on this robot. Pinning the SDK back to v2.50.0
(the version known to work on the host install at /home/unitree/librealsense)
restores frame delivery.
While in there, refactor the runtime stage to copy librealsense2.so.*
straight from the builder instead of apt-installing it, so the binding
ABI always matches the .so version. Also drop the now-unused Intel apt
repo + GPG key dance.
Frontend stage removed: npm-in-docker can't resolve registry.npmjs.org
on this Jetson (docker's default DNS 127.0.0.53 doesn't work from inside
containers). Build the React app on the host one-shot via:
docker run --rm --dns 8.8.8.8 -v $(pwd)/src/frontend/react:/app \\
-w /app node:22-alpine sh -c 'npm install && npm run build'
then `docker compose build` just copies the prebuilt dist/.
Tested on the G1: all REST endpoints (rgb, depth, depth/color, capture,
intrinsics, pointcloud, measure, snapshot) return 200 with valid
payloads at 640x480@30 via USB 3.2.
Note: requires videohub_pc4 stopped (it holds /dev/video4):
/unitree/sbin/mscli stopservice video_hub_pc4
Drop-in replacement for the Nuwa-HP60C camera container, exposing RGB +
aligned depth + intrinsics via REST/WebSocket so the existing
pestello-pose-detector pipeline can consume frames without changes.
Key adaptations for D4xx on USB 2.0 (this robot has no USB 3 port exposed
externally — see STATUS.md):
- pyrealsense2 built from source; the librealsense v2.55.1 install rule
doesn't ship the main .so, so we copy it from the build dir and add an
__init__.py that re-exports the submodule.
- Color stream uses YUYV format on USB 2.x (BGR8/RGB8 time out on this
bus); converted to BGR via cv2.cvtColor in capture().
- rs.align is skipped on USB 2 (incompatible with YUYV); color is
cv2-resized to depth shape for approximate per-pixel alignment.
- Driver auto-detects usb_type_descriptor and falls back to the
full-resolution BGR8 + rs.align pipeline when on USB 3.
Tested: /api/{health,status,discover,rgb,depth,depth/color,capture,
intrinsics} all return 200 with valid payloads at 640x480.
OliviaCandClaude Opus 5.5