Plan dettagliato per la tab "Pose 3D" (writing-plans skill),
basato sullo spec del 2026-05-21. 15 task suddivisi in backend
(routes + tests TDD) e frontend (Three.js viewport + gizmo +
mode toggle Editor/Jog).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lavoro accumulato nella sessione 2026-05-21 prima del brainstorming
della tab Pose 3D:
- compute_delta_wrist generalizzato con cam_to_wrist_perm: la mappa
cam->polso non e' solo sign-flip ma una permutazione (default
identita' per backward compat). DeltaOversize ora espone X/Y/Z
nella stringa d'errore. Test aggiornati.
- G1Client.wrist_absolute / wrist_relative: precise=True di default
(era False). Stessa cosa per wrist_routes.AbsoluteReq Pydantic
e per agent.tools move_wrist_*. Movimenti reali ora chiedono
controllo accurato salvo override esplicito.
- Tab Braccio DX:
- /verify_tag endpoint: snapshot wrist FK + tag id=1 dalla front
camera. Per verifica relativa moveto vs marker (no calibrazione).
- GUI: pannello "Verifica tag id=1" + campi camera_offset_wrist_mm.
- markUserTouchedOnInput esteso a raOff/raSgn/raCamOff.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds CRUD for named wrist poses persisted in config.yaml under
pose_3d.named_poses. PUT validates the name against ^[A-Za-z0-9_-]{1,40}$
and normalizes the quaternion. DELETE returns 404 if the pose is missing.
Bypasses update_config's deep-merge for wholesale named_poses replacement
(deep-merge cannot remove nested keys) via a small helper that mutates
the in-memory singleton then persists with an empty patch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Specifica per nuova tab GUI con viewport Three.js, gizmo
translate/rotate sui polsi DX e SX, modalita' Editor (preview) /
Jog (auto-move al mouseup), safety guard sui movimenti grandi,
checkbox precise toggle, save/load named poses in config.yaml.
Brainstorming nella sessione 2026-05-21.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tuning da prima sessione live: move_offset_camera_mm.x=100 (lateral
verso il pestello), camera_offset_wrist_mm.z=100 (mounting verticale
della Nuwa sopra il TCP destro), cluster_depth_window_m=0.20.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tre miglioramenti al move-toward dopo il primo uso live:
1. checkbox "force" accanto a "Muovi": bypassa max_step_m quando
l'utente sa che il delta richiesto e' grande ma corretto (es. primo
avvicinamento all'oggetto).
2. camera_offset_wrist_mm: il modello assumeva camera e TCP coincidenti
(sole sign flips). La camera Nuwa e' invece montata con traslazione
costante rispetto al polso. La formula diventa:
delta_wrist = sign * (centroid_cam - offset_cam) + t_wrist_cam
Esposto in config (default 0,0,0), GUI (3 input "Camera rispetto a
polso (mm)") e in /state. Il termine compensa il fatto che muovere
il polso porta la camera, non il TCP, sul punto di presa.
3. filter_nearest_cluster: il centroide era media di tutti i punti
SAM21 della mask. In presenza di sfondo lontano visibile attraverso
buchi della depth del primo piano la media veniva trascinata a
metri di distanza. Ora si tiene il cluster piu' vicino:
z_near = p5(Z)
keep = punti con Z <= z_near + cluster_depth_window_m
Default window 0.20 m, configurabile. La risposta di /detect/wrist
espone n_sam21_points, n_centroid_points e z_near/z_cutoff per
diagnosi.
Aggiornata anche la persistenza: /offset accetta camera_offset_wrist_mm.
GUI: i campi Offset/Sign/CamOff rispettano "user touched" (refresh non
li cancella mentre l'utente digita).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
YAML riformattato (block style) dal side-effect di update_config —
contenuto invariato a parte robot.{active,wrist_side}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tre problemi della tab Braccio DX dopo il primo rodaggio:
1. /detect/front e /detect/wrist ritornavano bbox raw dal YOLO server,
che lo manda come lista [x1,y1,x2,y2]; la GUI legge bbox.x1 e mostra
NaN. Ora il router normalizza il payload via perception/detector
._normalize, allineandosi al resto del codice. Aggiunta regressione
con payload reale del server.
2. La tab gateava su robot.active==g1 && wrist_side==right con banner
"Attiva G1 right" e POST /activate. Confonde: la selezione del
robot e' globale e non si flippa per-feature. Rimossi banner,
bottone, endpoint /activate e _guard_g1_right(); gli endpoint
operano sul polso destro del robot attivo qualunque esso sia.
3. image_size: [0,0] perche' Nuwa /api/intrinsics non include
width/height; vivono in /api/status > data.resolution. Aggiunto
NuwaClient.image_size() e usato in detect_front. Aggiornato
docstring di intrinsics() per coerenza con il payload reale.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ping anche services.front_camera in /api/home/status. Etichetta nuwa
rinominata in 'Nuwa-HP60C (camera polso)' per chiarezza ora che ci sono
due camere distinte.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Sostituisce colori hardcoded con CSS variables (--bg/--panel/--border/--dim/--warn/--ok/--danger)
- Scope .dim a #tab-right-arm per non alterare altre tab che usano la classe
- Aggiunge white-space pre-wrap a .ra-result per testi multi-riga
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
13 task TDD per il nuovo tab: config, geometry, math puro, 6 endpoint
router /api/right_arm/*, markup HTML + CSS, JS module. Spec:
docs/superpowers/specs/2026-05-20-right-arm-tab-design.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Design del nuovo tab "Braccio DX": fronte (depth cam fissa sul robot)
+ polso (Nuwa eye-in-hand) + controlli move/home/custom-pose. Router
/api/right_arm/*, JS dedicato, persistenza in config tab_right_arm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The pre-SAM21 depth filter and post-SAM21 partition_by_plane were
fighting each other and drowning the actual object cluster. Per
operator request, send Open3DModel the raw SAM21 cluster and let
RANSAC handle outliers itself. The accepted/excluded/SAM21 overlays
in the GUI continue to work; excluded is now always empty and
accepted == SAM21 cluster.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The find_model response now returns plane_polygon_px (4 corners of a
30 cm square anchored at the work plane centroid, projected to RGB
pixels via the same K used for the SAM21 cloud). The GUI gains a
'Piano (viola)' checkbox that draws the polygon as a translucent
violet quad with an outline, helping the operator see whether the
plane sits where expected relative to the object.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously SAM21 ran on the raw aligned depth and frequently leaked the
mask onto the table around the object: ~88% of segmented points (2136
of 2440 in a sample run) were table pixels filtered away by the post
partition_by_plane step. Open3DModel got a noisy cluster mostly off
the object.
New pipeline: with a calibrated work plane, depth pixels whose 3D
back-projection sits below the plane are zeroed BEFORE SAM21 sees the
depth PNG. SAM21 cannot back-project zero-depth pixels, so only true
above-plane points end up in the cloud regardless of mask outline.
Same sample run after the change: 513 SAM21 -> 513 accepted, 0 excluded.
The post-partition still runs (now a no-op on a calibrated plane), so
the find_model response shape is unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a 4th checkbox 'Punti SAM21 (gialli)' that draws the full SAM21
segmentation (input to both the plane partition and Open3DModel) on
the RGB result canvas. The backend now returns sam21_points_px and
n_sam21 alongside the existing accepted/excluded splits.
Useful as a sanity check: visualise what was actually segmented vs
what the plane filter kept/rejected vs where the STL got placed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 0.4x0.4 PlaneGeometry square was a fixed-size proxy that did not
reflect the actual ROI extent — visually misleading. Replace it with
an explicit blue point cloud of the inliers fed to fit_plane_svd, so
the user can see *exactly* which depth pixels the plane was fit on.
- Backend: fit response now returns fit_points_camera (CV frame) instead
of the (parallax-shifted) RGB pixel projection.
- Frontend: drop the purple plane mesh; new three.setFitInliers paints
the inliers in blue. The RGB overlay no longer carries blue dots.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NuwaClient.pointcloud_roi's docstring claims OpenCV frame, but in
practice the raw payload arrives in Nuwa native frame (Z<0 forward).
nuwa_to_cv_frame is what actually flips it. Using `pts_cv` (post-flip
+ post-outlier-mask) makes the projection drop none of the inliers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The fit endpoint now returns fit_points_px (the post-outlier-filter
inliers projected to RGB pixels via Nuwa's RGB intrinsics). The GUI
draws them on the same overlay as the ROI rectangle so the user can
visually verify which depth pixels were fed to fit_plane_svd.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop the ge=0 constraint on FitReq.offset_mm (negative values are a
documented usage: aggressive noise cull above the plane).
- HTML input min lowered to -200 to match.
- Render FastAPI 422 validation arrays as a human-readable string in the
status chip (was showing "[object Object]").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Setting canvas.width/height clears its bitmap; the live RGB tick was
firing the img.load handler every second, wiping the ROI rectangle and
breaking the two-click flow (third click was interpreted as restart).
- Only mutate canvas size when image resolution actually changes.
- Re-paint the overlay after each load so the ROI persists.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
