On the fake camera: /api/raw/latest holds color and depth of one frameset
(frame number in the pixels), waits with after=N and times out with 504,
shares its color JPEG with the color channel; /api/capture's frameset_seq;
/api/intrinsics has sizes, models and depth_scale_m; compare_depths counts
and differences; the comparison script end to end through the API with
identical geometry (median |diff| 0, > 90% exact: rs.align itself changes
~3% of the fake's pure-noise depth where pixel footprints overlap).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The number of the camera frameset an aligned frame comes from, in the same
numbering as /api/raw/latest and /api/color/latest: it lets a tool compare
Intel's align on PC2 with an align on the host on the SAME frameset.
Existing fields unchanged (frame_id still counts responses); the field-set
tests now say "the R01 fields plus frameset_seq only".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fake_realsense: clocked mode (a frameset every 1/fps s, like the device),
FakeAlign, camera timestamps, and the frame number written in the pixels
(color block, depth[0, 0]) so a test knows which frame an image came from.
test_two_channels:
- /api/rgb, /api/capture, /api/depth: rgb and depth from one frame (criterion
B of optimize-streams R03; failed before the previous commit: 1 vs 2);
- /api/color/latest serves strictly newer frames, the image is the frame its
headers name (camera clock), 504 on timeout, each frame encoded once;
- color >= 25 new frames/s while /api/capture loops (criteria A1/A2, 3 s).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two threads while the camera is open:
- realsense-raw (continuous, FrameAcquisition(on_demand=False)): the only
caller of wait_for_frames. For every frameset it publishes the color image
(own BGR copy) with the camera timestamp, its domain and the host arrival
time, and keeps the SDK frameset. No align and no depth conversion at the
camera rate: per frame it costs wait + one color copy (YUYV decode on USB 2).
- realsense-align (on demand, as in R01): when someone asks for color+depth
it aligns the newest frameset not aligned yet, converts the depth, copies.
Concurrent callers share one align. capture(), the 100 ms cache of
get_rgb/get_depth/cached_frame and every existing endpoint read this
aligned frame, so /api/rgb stays the color of the same frame as /api/depth.
The aligned color reuses the raw color pixels (align maps depth onto color).
At most 3 SDK framesets are referenced at a time (latest, receiving,
aligning); ColorFrame and CameraFrame hold no SDK memory.
New timing phase color_convert. One commit for items 1 and 2 of the request:
once the continuous thread owns the pipeline, the aligned path must take
its framesets, so they cannot be split without a broken intermediate state.
color_frame(after_seq, timeout_ms) gives the fast channel to the API.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fake frames now hand out a buffer (memoryview) like pyrealsense2's
BufData, so np.asanyarray() of it is a view, as on the robot; a control test
proves it. Then: the captured rgb/depth share no memory with the fake SDK
frame and own their data (without the fix both shared it), and _owned()
copies views only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/fake_realsense.py: a pipeline whose wait_for_frames renders a textured
640x480 frameset after 100 ms (no RealSense on the development PC).
tests/test_parallel_capture.py:
- 3 parallel /api/capture take < 2x one alone (medians of 5): the criterion
declared for optimize-streams R01. On the previous code: 391 vs 128 ms.
- /api/capture keeps its fields; parallel requests share one frame and its
bytes, frame_id still differs per response; one thread calls the pipeline.
- /api/rgb, /api/depth, /api/depth/color decode; /ws streams; close() stops
the acquisition thread.
Skipped when pyrealsense2 is not importable, like the existing driver tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FrameAcquisition: a single thread calls wait_for_frames + align (the old
capture() body, now _grab_once) and keeps the latest frame, numbered.
capture() no longer touches the pipeline: it waits for the next frame of
that thread; requests waiting at the same moment get the same frame, so N
callers cost one grab. get_rgb()/get_depth() keep their 100 ms cache,
now on the thread's latest frame (_last_frame is gone).
The thread grabs on demand, only while someone waits for a newer frame:
grabbing continuously at 30 fps would align frames nobody reads, and align
on the Jetson CPU is the main suspect of the capture cost. A grab error is
logged and retried, the thread never dies. close() stops it before the
pipeline and joins it after (pipeline.stop() ends a wait in progress).
open/close and camera_open are unchanged, so the healthcheck + autoheal
recovery of ede70a2 sees the same state as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One log line per phase every PHASE_LOG_EVERY samples (default 100, 0 = off):
mean and max of wait_frames, align, convert, each JPEG/PNG encoding,
colormap, depth_stats, base64, json and ws_send. Meant for the measurement
on PC2 via `docker logs` (optimize-streams R01): where a capture spends its
~130 ms of CPU is not known yet.
capture() moves its numpy conversion into _frame_from() unchanged, only so
the block can be timed; _depth_colormap() likewise wraps _jet_colormap().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop-in replacement for the Nuwa-HP60C camera container, exposing RGB +
aligned depth + intrinsics via REST/WebSocket so the existing
pestello-pose-detector pipeline can consume frames without changes.
Key adaptations for D4xx on USB 2.0 (this robot has no USB 3 port exposed
externally — see STATUS.md):
- pyrealsense2 built from source; the librealsense v2.55.1 install rule
doesn't ship the main .so, so we copy it from the build dir and add an
__init__.py that re-exports the submodule.
- Color stream uses YUYV format on USB 2.x (BGR8/RGB8 time out on this
bus); converted to BGR via cv2.cvtColor in capture().
- rs.align is skipped on USB 2 (incompatible with YUYV); color is
cv2-resized to depth shape for approximate per-pixel alignment.
- Driver auto-detects usb_type_descriptor and falls back to the
full-resolution BGR8 + rs.align pipeline when on USB 3.
Tested: /api/{health,status,discover,rgb,depth,depth/color,capture,
intrinsics} all return 200 with valid payloads at 640x480.
OliviaCandClaude Opus 5.5