From d9213ca19a5164a878af5f4d622d5b88f761d09c Mon Sep 17 00:00:00 2001 From: Adriano Date: Wed, 3 Jun 2026 12:41:12 +0200 Subject: [PATCH] docs(pose_3d): tags multi-camera + calibrazione cross-camera Co-Authored-By: Claude Sonnet 4.6 --- CLAUDE.md | 3 ++- README.md | 3 +++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 95f3915..28e678b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -23,7 +23,7 @@ Useful endpoints (all under `http://localhost:8890`): - Grasp: `POST /api/grasp/plan`, `POST /api/grasp/execute` (set `dry_run` to gate G1 mutations). - Wrist/EE: `GET /api/wrist`, `POST /api/wrist/{absolute,relative,hand,emergency_stop,unlock}`. - Right arm: `GET /api/right_arm/state`, `POST /api/right_arm/detect_{front,wrist}`, `POST /api/right_arm/move_toward`, `POST /api/right_arm/{home,custom_pose}`. -- Pose 3D: `GET /api/pose_3d/state` (wrist + joints + config), `POST /api/pose_3d/apply`, `GET /api/pose_3d/clouds` (Nuwa + D435i point clouds), `CRUD /api/pose_3d/poses/{name}`. +- Pose 3D: `GET /api/pose_3d/state` (wrist + joints + config, incl. `cross_camera` calibration), `POST /api/pose_3d/apply`, `GET /api/pose_3d/clouds` (Nuwa + D435i point clouds), `GET /api/pose_3d/tags` (ArUco su Nuwa polso + D435i frontale), `POST /api/pose_3d/align?align_id=N` (fit Umeyama Nuwa→frontale, persiste in `pose_3d.cross_camera`), `CRUD /api/pose_3d/poses/{name}`. Tests live under `tests/` and run with `uv run pytest`. No linter is configured. @@ -94,6 +94,7 @@ Marker-finder caveat: it uses **D435i intrinsics baked into its config** that do - **Pose 3D graceful offline:** `GET /api/pose_3d/state` returns 200 even when G1 is unreachable — `wrist` and `joints` are `null`, `robot_error` carries the message. The tab must remain usable for editing config / named poses / viewing the URDF model without a live robot. - **Pose 3D clouds:** `GET /api/pose_3d/clouds` acquires from Nuwa (eye-in-hand, polso DX) and D435i (front camera, torso) in parallel. Both are returned in **OpenCV optical frame** (X right, Y down, Z forward). The client attaches them to URDF links with configurable offset/rpy from `config.yaml > pose_3d.clouds`. - **Pose 3D joints mirroring:** `G1Client.diagnostics()` → `motors[].{name, q}` gives real joint positions in radians. The frontend uses these to pose the URDF model directly, bypassing client-side IK. +- **Pose 3D cross-camera alignment:** `POST /api/pose_3d/align` fits an Umeyama similarity (rotation + translation + uniform scale) mapping Nuwa-optical → D435-optical from a tag (`align_id`) seen by both cameras, and persists it to `config.yaml > pose_3d.cross_camera`. The frontend renders the Nuwa cloud through the D435i transform pipeline (`d435LocalMatrix()` · alignMatrix) when `cross_camera.enabled`. If the tag isn't seen by both cameras, the saved matrix is NOT overwritten and the Nuwa cloud falls back to URDF placement. Tag detection (`GET /tags`) runs on both cameras; the GUI draws Nuwa tags red, D435i tags cyan. - Endpoints return `{ ..., "timings_ms": {step: ms, ..., total: ms} }`. New endpoints should follow this with `Stopwatch().step(name)`. - Calibration paths in config (`calibration.file`, `calibration.samples_file`) are resolved against `PRESA_MANO_DATA` when relative — see `_deps.calibration_file_path()` / `samples_file_path()`. - Italian is fine in user-facing copy (GUI, README, comments) — it's the project default. diff --git a/README.md b/README.md index d0838a3..f7525b6 100644 --- a/README.md +++ b/README.md @@ -56,6 +56,9 @@ Tab principali: - **Pose 3D**: viewport Three.js con URDF G1 specchiato sui giunti reali (`/diagnostics`), point cloud Nuwa + D435i, modello 3D camera sul polso, gizmo traslazione/rotazione, editor pose (Apply/Jog), named poses salvabili. + Overlay tag ArUco su entrambe le camere (Nuwa = rosso, D435i = ciano). + Calibrazione cross-camera: pulsante *Allinea* esegue fit Umeyama Nuwa→D435i + da un tag comune, persiste in `config.yaml > pose_3d.cross_camera`. La tab resta utilizzabile anche senza robot (mostra il modello URDF, config, pose salvate).