13 task TDD per il nuovo tab: config, geometry, math puro, 6 endpoint
router /api/right_arm/*, markup HTML + CSS, JS module. Spec:
docs/superpowers/specs/2026-05-20-right-arm-tab-design.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Design del nuovo tab "Braccio DX": fronte (depth cam fissa sul robot)
+ polso (Nuwa eye-in-hand) + controlli move/home/custom-pose. Router
/api/right_arm/*, JS dedicato, persistenza in config tab_right_arm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The pre-SAM21 depth filter and post-SAM21 partition_by_plane were
fighting each other and drowning the actual object cluster. Per
operator request, send Open3DModel the raw SAM21 cluster and let
RANSAC handle outliers itself. The accepted/excluded/SAM21 overlays
in the GUI continue to work; excluded is now always empty and
accepted == SAM21 cluster.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The find_model response now returns plane_polygon_px (4 corners of a
30 cm square anchored at the work plane centroid, projected to RGB
pixels via the same K used for the SAM21 cloud). The GUI gains a
'Piano (viola)' checkbox that draws the polygon as a translucent
violet quad with an outline, helping the operator see whether the
plane sits where expected relative to the object.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously SAM21 ran on the raw aligned depth and frequently leaked the
mask onto the table around the object: ~88% of segmented points (2136
of 2440 in a sample run) were table pixels filtered away by the post
partition_by_plane step. Open3DModel got a noisy cluster mostly off
the object.
New pipeline: with a calibrated work plane, depth pixels whose 3D
back-projection sits below the plane are zeroed BEFORE SAM21 sees the
depth PNG. SAM21 cannot back-project zero-depth pixels, so only true
above-plane points end up in the cloud regardless of mask outline.
Same sample run after the change: 513 SAM21 -> 513 accepted, 0 excluded.
The post-partition still runs (now a no-op on a calibrated plane), so
the find_model response shape is unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a 4th checkbox 'Punti SAM21 (gialli)' that draws the full SAM21
segmentation (input to both the plane partition and Open3DModel) on
the RGB result canvas. The backend now returns sam21_points_px and
n_sam21 alongside the existing accepted/excluded splits.
Useful as a sanity check: visualise what was actually segmented vs
what the plane filter kept/rejected vs where the STL got placed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 0.4x0.4 PlaneGeometry square was a fixed-size proxy that did not
reflect the actual ROI extent — visually misleading. Replace it with
an explicit blue point cloud of the inliers fed to fit_plane_svd, so
the user can see *exactly* which depth pixels the plane was fit on.
- Backend: fit response now returns fit_points_camera (CV frame) instead
of the (parallax-shifted) RGB pixel projection.
- Frontend: drop the purple plane mesh; new three.setFitInliers paints
the inliers in blue. The RGB overlay no longer carries blue dots.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NuwaClient.pointcloud_roi's docstring claims OpenCV frame, but in
practice the raw payload arrives in Nuwa native frame (Z<0 forward).
nuwa_to_cv_frame is what actually flips it. Using `pts_cv` (post-flip
+ post-outlier-mask) makes the projection drop none of the inliers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The fit endpoint now returns fit_points_px (the post-outlier-filter
inliers projected to RGB pixels via Nuwa's RGB intrinsics). The GUI
draws them on the same overlay as the ROI rectangle so the user can
visually verify which depth pixels were fed to fit_plane_svd.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop the ge=0 constraint on FitReq.offset_mm (negative values are a
documented usage: aggressive noise cull above the plane).
- HTML input min lowered to -200 to match.
- Render FastAPI 422 validation arrays as a human-readable string in the
status chip (was showing "[object Object]").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Setting canvas.width/height clears its bitmap; the live RGB tick was
firing the img.load handler every second, wiping the ROI rectangle and
breaking the two-click flow (third click was interpreted as restart).
- Only mutate canvas size when image resolution actually changes.
- Re-paint the overlay after each load so the ROI persists.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Right side has no calibration in data/calibration_right.yaml;
the H1 left arm is the one with samples + solved hand-eye.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Open3DModel returns the same envelope for /api/recipes as for /api/pose;
the bare-list code path stays as a fallback for older mocks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brainstorming → approved spec for the GUI/backend redesign of tab "Cerca
modello": YOLO+SAM21 cluster feeds Open3DModel /api/pose, presa-mano
samples the recipe STL, projects onto a snapshot RGB with togglable
overlays for accepted/excluded points and the fitted model points.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add SAM21 (NuwaSeg :8002) to the upstream services table.
- New "Calibrazione Piano" tab documented: 2-click ROI, fit preview, 3D
camera-POV viewer with red = excluded points, offset_mm semantics
(positive = permissive buffer below, negative = aggressive cut above for
depth noise), save persists to data/work_plane.json.
- "TrovaUtensile" tab renamed to "Cerca modello".
- New section "Pipeline cloud-from-bbox" describing the SAM21-first flow
with bbox fallback and the work-plane filter, plus the response meta
fields (source, plane_applied, n_points_before_plane,
sam21_fallback_reason).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 3D viewer was only seeing points inside the ROI rectangle used for
the plane fit. Now /fit also fetches nuwa.pointcloud (step=4, full frame),
flips to CV frame, and returns it as points_camera (downsampled to
max_preview_points). The user sees the entire scene with red = excluded
by the below-plane filter, cyan = kept.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Camera at origin (CV frame) looking at +Z with screen-up = CV −Y, so
the rendered scene matches the camera's vantage point. OrbitControls
retargets to the plane centroid on fit so rotation feels natural.
- Simplify scatter coloring to two tones: red for filter-excluded points
(signed_distance < −offset_mm/1000), cyan-white for everything kept.
Matches the backend filter contract exactly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User asked for "2 punti" — replace the drag-to-rectangle interaction with
two discrete clicks: first click places corner A (cross marker), second
click defines the rectangle, third click starts a new selection.
Rectangles smaller than 10×10 px are rejected. Status chip guides the
flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Direct fetch to http://localhost:8877/api/rgb fails from the browser due
to CORS (page is served from :8890). Switch to /api/calibration/rgb.jpg
which presa-mano already proxies same-origin.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Rename "TrovaUtensile" tab → "Cerca modello".
- Remove the "Precisione (ultime 5 detection)" stats panel and the
"Dettagli" JSON dump from the grasp tab; keep hidden stubs so the
existing app.js continuous-loop code doesn't null-deref.
- Add work_plane_routes to api/__init__.py imports + __all__ (consistency
with the other routers).
- Trim the misleading "drag = pan · wheel = zoom · doppio click = reset"
hint from the Calibrazione Piano RGB label — pan/zoom isn't wired.
- Disable wpClearBtn initially so it doesn't flash clickable before
fetchSaved() resolves.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New tab replaces the removed Servo Visuale. Shift+drag on the RGB image
defines a rectangle, 'Calcola piano' calls POST /api/work_plane/fit and
renders the plane + a colored scatter in 3D. Adjusting the offset slider
re-colors the scatter client-side. 'Salva piano' persists; 'Rimuovi piano
salvato' clears.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The marker-relative servo (endpoints /api/grasp/visual_servo/{plan,step,
plane/*}, helpers in grasp_routes.py, the bulk of grasp/visual_servo.py,
web/servo.js, and the Servo Visuale GUI tab) is gone. Plane fitting now
lives in /api/work_plane/* and perception/work_plane.py; Task 5 wires the
new Calibrazione Piano tab.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
POST /fit returns a candidate WorkPlane + a downsampled CV-frame cloud
preview, without writing to disk. POST /save persists the plane. GET
returns the current saved plane (or null). DELETE clears it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Until Task 4 removes the duplicate WorkPlane code in grasp/visual_servo.py,
the visual-servo plane endpoints must also update perception.work_plane's
singleton so the below-plane filter (which reads from the perception side)
sees the newly-calibrated plane within the same process lifetime.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
points_via_sam21, _points_via_bbox, and _object_cloud_cv (bbox branch)
now call apply_below_plane_filter on the CV-frame cloud before the
<10-points guard. Responses carry plane_applied + n_points_before_plane.
Without a saved plane the filter is a no-op (plane_applied=false).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Owns WorkPlane (with new offset_mm field), fit_plane_svd, disk-backed
get/set, and the apply_below_plane_filter helper. WorkPlane.normal lives
in OpenCV RGB-camera frame and is oriented toward the camera. The filter
keeps points where signed_distance >= -offset_mm/1000.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
6 task TDD-first: perception/work_plane.py + 8 unit tests, filter wiring
ai 3 call sites con plane_applied/n_points_before_plane nelle response,
nuovi endpoint /api/work_plane/{fit,save,get,delete}, rimozione totale
del visual-servo (backend + GUI), nuova tab Calibrazione Piano con
work_plane.js (ROI + fit preview + 3D viewer con scatter colorato +
offset live-recolor), verifica live end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Repurpose the "Servo Visuale" tab into "Calibrazione Piano". Reuses the
existing WorkPlane + fit_plane_svd infrastructure (moved out of
grasp/visual_servo.py into perception/work_plane.py), extends WorkPlane
with offset_mm, splits fit (preview) from save (commit), and applies a
below-plane + offset filter to every object cloud (sam21_bridge,
_points_via_bbox, _object_cloud_cv bbox branch). Visual-servo feature
removed entirely; the only survivors are the plane bits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
