diff --git a/CLAUDE.md b/CLAUDE.md index 16571fd..33290e7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -23,7 +23,7 @@ Useful endpoints (all under `http://localhost:8890`): - Grasp: `POST /api/grasp/plan`, `POST /api/grasp/execute` (set `dry_run` to gate G1 mutations). - Wrist/EE: `GET /api/wrist`, `POST /api/wrist/{absolute,relative,hand,emergency_stop,unlock}`. -There are no tests or linters configured in this repo. +Tests live under `tests/` and run with `uv run pytest`. No linter is configured. ## Architecture @@ -36,12 +36,13 @@ Upstream services (defaults in `config.yaml > services`): | marker-finder | `:8082` | ArUco detection (returns `corners`, `position_m`, `rotation_matrix`) | | YOLO server | `:8879` | Object bbox detection from an image URL | | G1 API | `:8765` | Wrist pose read/write, hand open/close, e-stop | +| SAM21 (NuwaSeg) | `:8002` | Promptable segmentation: rgb+depth+intrinsics+bbox → 3D point cloud of the object (replaces the bbox-only ROI cloud in the grasp pipeline) | ### Layered package layout (`src/presa_mano/`) - `main.py` — FastAPI app; mounts `web/` at `/static`, serves `web/index.html` at `/`, registers routers under `/api/*`. - `config.py` — Thread-safe singleton config loader. `load_config()` reads YAML once; `update_config(patch)` deep-merges and persists. Path comes from `PRESA_MANO_CONFIG` env (Dockerfile sets `/app/config.yaml`); data files live under `PRESA_MANO_DATA` (default `/app/data`). -- `api/` — Routers, one per concern. **Build clients per-request via `api/_deps.py`** (`nuwa()`, `marker()`, `yolo()`, `g1(dry_run=...)`) so runtime `PUT /api/config` changes take effect without restart. Don't cache clients at module level. +- `api/` — Routers, one per concern. **Build clients per-request via `api/_deps.py`** (`nuwa()`, `marker()`, `yolo()`, `sam21()`, `g1(dry_run=...)`) so runtime `PUT /api/config` changes take effect without restart. Don't cache clients at module level. - `clients/` — Thin `httpx` wrappers, one per upstream. Errors are normalized to service-specific exceptions (e.g. `G1Unavailable`, `NuwaError`). - `calibration/` — `sampler.py` builds one (T_base_wrist, T_cam_marker) sample; `handeye.py` runs `cv2.calibrateHandEye` and computes residuals; `store.py` persists samples (JSON) and the result (YAML). - `perception/` — `detector.py` calls YOLO and normalizes responses across multiple envelope shapes; `pointcloud.py` handles frame conversion + outlier filtering. @@ -86,6 +87,7 @@ Marker-finder caveat: it uses **D435i intrinsics baked into its config** that do - **Dry-run semantics:** any G1-mutating endpoint takes a `dry_run` flag; `G1Client._post` short-circuits and logs `[DRY-RUN]` instead of hitting the network. Never silently strip dry-run from a mutating path. - **Per-request client construction** (via `api/_deps.py`) is intentional — runtime `PUT /api/config` updates would otherwise be invisible. +- **SAM21 vs bbox frame convention:** the grasp pipeline has two parallel paths in `_object_points_in_base`. The SAM21 path returns points already in the OpenCV RGB-camera frame and **must NOT call `nuwa_to_cv_frame`**; the legacy bbox path (via `_points_via_bbox`) still calls it because Nuwa's ROI cloud uses a different convention. Don't accidentally add the flip to the SAM21 branch — coordinates would double-flip. - Endpoints return `{ ..., "timings_ms": {step: ms, ..., total: ms} }`. New endpoints should follow this with `Stopwatch().step(name)`. - Calibration paths in config (`calibration.file`, `calibration.samples_file`) are resolved against `PRESA_MANO_DATA` when relative — see `_deps.calibration_file_path()` / `samples_file_path()`. - Italian is fine in user-facing copy (GUI, README, comments) — it's the project default. diff --git a/src/presa_mano/api/grasp_routes.py b/src/presa_mano/api/grasp_routes.py index 38331b2..d613841 100644 --- a/src/presa_mano/api/grasp_routes.py +++ b/src/presa_mano/api/grasp_routes.py @@ -6,6 +6,7 @@ import logging from typing import Any, Optional import cv2 +import httpx import numpy as np import time from dataclasses import asdict @@ -13,8 +14,6 @@ from fastapi import APIRouter, HTTPException from pydantic import BaseModel, Field from ..calibration import CalibrationStore -import httpx - from ..clients import G1Unavailable, NuwaError, Sam21Error, Sam21Unavailable from ..config import get_config from ..geometry import transform_points