The 3D viewer was only seeing points inside the ROI rectangle used for
the plane fit. Now /fit also fetches nuwa.pointcloud (step=4, full frame),
flips to CV frame, and returns it as points_camera (downsampled to
max_preview_points). The user sees the entire scene with red = excluded
by the below-plane filter, cyan = kept.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Rename "TrovaUtensile" tab → "Cerca modello".
- Remove the "Precisione (ultime 5 detection)" stats panel and the
"Dettagli" JSON dump from the grasp tab; keep hidden stubs so the
existing app.js continuous-loop code doesn't null-deref.
- Add work_plane_routes to api/__init__.py imports + __all__ (consistency
with the other routers).
- Trim the misleading "drag = pan · wheel = zoom · doppio click = reset"
hint from the Calibrazione Piano RGB label — pan/zoom isn't wired.
- Disable wpClearBtn initially so it doesn't flash clickable before
fetchSaved() resolves.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The marker-relative servo (endpoints /api/grasp/visual_servo/{plan,step,
plane/*}, helpers in grasp_routes.py, the bulk of grasp/visual_servo.py,
web/servo.js, and the Servo Visuale GUI tab) is gone. Plane fitting now
lives in /api/work_plane/* and perception/work_plane.py; Task 5 wires the
new Calibrazione Piano tab.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
POST /fit returns a candidate WorkPlane + a downsampled CV-frame cloud
preview, without writing to disk. POST /save persists the plane. GET
returns the current saved plane (or null). DELETE clears it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Until Task 4 removes the duplicate WorkPlane code in grasp/visual_servo.py,
the visual-servo plane endpoints must also update perception.work_plane's
singleton so the below-plane filter (which reads from the perception side)
sees the newly-calibrated plane within the same process lifetime.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
points_via_sam21, _points_via_bbox, and _object_cloud_cv (bbox branch)
now call apply_below_plane_filter on the CV-frame cloud before the
<10-points guard. Responses carry plane_applied + n_points_before_plane.
Without a saved plane the filter is a no-op (plane_applied=false).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extract the SAM21 wiring out of grasp_routes.py into a shared module
perception/sam21_bridge.py (Sam21Fallback + points_via_sam21). Both
/api/perception/scan and /api/perception/object_points now try SAM21
first and fall back to the bbox ROI path on any Sam21Fallback. Response
now carries 'source' (sam21 | bbox_roi) and optional 'sam21_fallback_reason'.
Side-effect: 'points_camera' on both endpoints is now in OpenCV
RGB-camera frame (same as 'points_base' minus the T_base_camera
transform), regardless of which path produced the cloud. Previously the
bbox path stored points in Nuwa-native frame here. The web GUI plots
points_base when available, so the scatter views automatically pick up
the SAM21-segmented cloud.
Verified live: /scan → source=sam21 n=2414 sam21=111ms;
/object_points → source=sam21 n=2347 sam21=118ms;
/grasp/plan regression → source=sam21 n=2390 sam21=119ms;
/scan with SAM21 down → source=bbox_roi fallback='unreachable: ...'
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Note that tests now exist and run with uv run pytest.
- Add SAM21 to the upstream services table.
- Add sam21() to the _deps factory list.
- Add a convention bullet about the SAM21 vs bbox frame asymmetry so
future contributors don't add nuwa_to_cv_frame to the SAM21 branch.
- Move 'import httpx' into the third-party imports block.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Nuwa /api/intrinsics omits width/height from the rgb and ir blocks, so
SAM21 rejected the payload with KeyError('width') and we silently fell
back to bbox on every request. Derive the dimensions from the decoded
BGR + IR depth shapes and augment intr_full before building
CameraRegistration and calling SAM21. Verified end-to-end: source=sam21,
~2200 segmented points on Mortaio at 128 ms GPU inference.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- All SAM21-path logging now lives in _points_via_sam21 at the right
level (WARNING for operational, ERROR for bug codes / unexpected).
_object_points_in_base no longer double-logs on fallback.
- Nuwa failures during SAM21 prep now raise HTTPException(502)
immediately instead of running the futile bbox fallback (which would
also fail because Nuwa is down).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When grasp.use_sam21=true (default), _object_points_in_base feeds the YOLO
bbox to SAM21 and uses the segmented cloud directly. Any failure
(transport, segmentation empty, no valid depth, payload bug) logs and falls
back to the legacy bbox path. Response meta includes source ∈ {sam21,
bbox_roi} and sam21_fallback_reason when applicable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure refactor in preparation for the SAM21 branch. _object_points_in_base
keeps its signature and the bbox→ROI→base-frame logic moves into the
private helper. Meta now includes source='bbox_roi'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Align sam21() with nuwa()/marker()/yolo(): require base_url to be present
in config rather than silently defaulting. Misconfiguration now raises
KeyError at factory call instead of masking as a transport error.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Elimina il flusso di acquisizione griglia automatica: tab GUI, modulo
autocal3d.js, AutoRunner/auto_job, e gli endpoint /auto/* + /auto_grid.
La calibrazione resta operabile via sample manuali + /solve.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pipeline completa per calibrazione hand-eye e grasp planning su robot
Unitree G1/H1 con camera depth Nuwa-HP60C. Esposta come FastAPI + GUI vanilla
JS, con assistente AI (OpenRouter + tool calling curato) per pilotare il
sistema in linguaggio naturale.
Architettura:
- FastAPI orchestrator (porta da .env, default 8890) che coordina 4
microservizi upstream: Nuwa-HP60C, marker-finder, YOLO, robot bridge.
- Dual-robot: services.g1 e services.h1 in config.yaml; switchabili a
runtime via PUT /api/home/robot.
- Calibrazione hand-eye eye-to-hand split per-side (right/left).
- Marker pose: marker-finder riceve gli intrinseci RGB della Nuwa al boot,
PnP server-side con quei dati. Nessun re-PnP locale richiesto.
- Depth-to-RGB: registrazione iterativa IR↔RGB per back-projection 3D
corretta (Nuwa /api/measure usa coords IR; il nostro /api/perception/
measure_rgb prende pixel RGB e ritorna 3D nel frame RGB OpenCV).
- Correzione affine sul depth bias (calibrabile via marker come
ground-truth) — riduce errore Z da ~6% a ~1.5% nei test.
- AI assistant: 16 tool curati (move_wrist_*, hand_action, get_marker[_3d],
detect_object, calibration_*, set_active_robot…), system prompt
dinamico che riflette robot attivo + marker config + addendum
customizzabile da GUI. Temperature di default 0.1 per evitare invenzioni.
GUI (7 tab):
- Home: stato comunicazione device + selettore robot attivo G1/H1
- Assistente AI: chat con OpenRouter, history persistita lato browser,
task riutilizzabili (prompt salvati), trace tool calls
- Impostazioni AI: modello da combo curata, temperature, max iterations,
prompt addendum, anteprima system prompt
- Calibrazione, Auto-Calibrazione, TrovaUtensile, Servo Visuale, Vista 3D,
Movimento EE — invariate dalle versioni precedenti
Setup:
- Variabili in presa-mano/.env (template in .env.example)
- OPENROUTER_API_KEY richiesta solo per il tab Assistente AI
- PRESA_MANO_PORT configurabile
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
