Adriano Dal PastroandClaude Opus 4.7
0c211630cc
Elimina il flusso di acquisizione griglia automatica: tab GUI, modulo autocal3d.js, AutoRunner/auto_job, e gli endpoint /auto/* + /auto_grid. La calibrazione resta operabile via sample manuali + /solve. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
92 lines
7.2 KiB
Markdown
92 lines
7.2 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## Commands
|
|
|
|
Build & run (Docker):
|
|
```bash
|
|
docker compose up -d presa-mano # serves GUI + API on http://localhost:8890
|
|
```
|
|
|
|
Local dev (uses `uv`, same as the Dockerfile):
|
|
```bash
|
|
uv sync # install deps from pyproject.toml
|
|
uv run uvicorn presa_mano.main:app --host 0.0.0.0 --port 8890 --reload
|
|
```
|
|
|
|
Useful endpoints (all under `http://localhost:8890`):
|
|
- `GET /apidocs` — Swagger UI; `GET /redoc` — ReDoc.
|
|
- `GET/PUT /api/config` — read/patch runtime config (deep-merged into `config.yaml` and persisted).
|
|
- Calibration: `POST /api/calibration/sample`, `GET/DELETE /api/calibration/samples`, `POST /api/calibration/solve`, `GET /api/calibration/result`, `POST /api/calibration/preview`.
|
|
- Perception: `POST /api/perception/detect`, `POST /api/perception/object_points`, `GET /api/perception/measure`.
|
|
- Grasp: `POST /api/grasp/plan`, `POST /api/grasp/execute` (set `dry_run` to gate G1 mutations).
|
|
- Wrist/EE: `GET /api/wrist`, `POST /api/wrist/{absolute,relative,hand,emergency_stop,unlock}`.
|
|
|
|
There are no tests or linters configured in this repo.
|
|
|
|
## Architecture
|
|
|
|
This is a **FastAPI orchestrator** that glues together four external microservices to implement an eye-to-hand calibration + object-grasping pipeline for a Unitree G1 humanoid using a fixed Nuwa-HP60C depth camera. It owns no hardware itself — every action is an HTTP call.
|
|
|
|
Upstream services (defaults in `config.yaml > services`):
|
|
| Service | Default URL | Role |
|
|
|---------|------------|------|
|
|
| Nuwa-HP60C | `:8877` | RGB, depth PNG, point cloud (full + ROI), intrinsics, pixel→3D measure |
|
|
| marker-finder | `:8082` | ArUco detection (returns `corners`, `position_m`, `rotation_matrix`) |
|
|
| YOLO server | `:8879` | Object bbox detection from an image URL |
|
|
| G1 API | `:8765` | Wrist pose read/write, hand open/close, e-stop |
|
|
|
|
### Layered package layout (`src/presa_mano/`)
|
|
|
|
- `main.py` — FastAPI app; mounts `web/` at `/static`, serves `web/index.html` at `/`, registers routers under `/api/*`.
|
|
- `config.py` — Thread-safe singleton config loader. `load_config()` reads YAML once; `update_config(patch)` deep-merges and persists. Path comes from `PRESA_MANO_CONFIG` env (Dockerfile sets `/app/config.yaml`); data files live under `PRESA_MANO_DATA` (default `/app/data`).
|
|
- `api/` — Routers, one per concern. **Build clients per-request via `api/_deps.py`** (`nuwa()`, `marker()`, `yolo()`, `g1(dry_run=...)`) so runtime `PUT /api/config` changes take effect without restart. Don't cache clients at module level.
|
|
- `clients/` — Thin `httpx` wrappers, one per upstream. Errors are normalized to service-specific exceptions (e.g. `G1Unavailable`, `NuwaError`).
|
|
- `calibration/` — `sampler.py` builds one (T_base_wrist, T_cam_marker) sample; `handeye.py` runs `cv2.calibrateHandEye` and computes residuals; `store.py` persists samples (JSON) and the result (YAML).
|
|
- `perception/` — `detector.py` calls YOLO and normalizes responses across multiple envelope shapes; `pointcloud.py` handles frame conversion + outlier filtering.
|
|
- `grasp/` — `axis.py` PCA → `ObjectAxes`; `planner.py` builds a wrist pose from those axes.
|
|
- `geometry.py` — pure-numpy SE(3) helpers (quat↔R, rvec↔R, invert_T, transform_points).
|
|
- `timing.py` — `Stopwatch` with `step(name)` context manager; every endpoint annotates its response with `timings_ms`.
|
|
- `web/` — vanilla HTML/JS/CSS GUI (3 tabs: Calibration, Grasp, Movimento EE), served as static files; talks only to `/api/*`.
|
|
|
|
### Core data flow: hand-eye calibration (eye-to-hand)
|
|
|
|
The camera is fixed; an ArUco marker is mounted on the wrist. For each sample:
|
|
|
|
1. `G1.wrist()` → `T_base_wrist` (via `T_from_pos_quat`).
|
|
2. `marker-finder.detect_url(image_url=nuwa.rgb_url())` → `T_cam_marker` from `position_m` + `rotation_matrix`.
|
|
3. `samples.json` accumulates samples; `HandEyeSolver.solve()` calls `cv2.calibrateHandEye`.
|
|
|
|
**Eye-to-hand trick** (`calibration/handeye.py`): OpenCV documents the eye-in-hand case. We feed `invert_T(T_base_wrist)` so the solver returns `T_camera→base` directly.
|
|
|
|
Residual quality check: with a correct calibration, `T_wrist_marker = invert_T(T_base_wrist) @ T_base_camera @ T_cam_marker` should be constant across samples (marker is rigidly attached). Mean translation/rotation deviations from sample 0 are reported as `residual_translation_mm` / `residual_rotation_deg`.
|
|
|
|
### Core data flow: grasp pipeline
|
|
|
|
`POST /api/grasp/plan` (or `/execute`):
|
|
1. YOLO detect on the current Nuwa RGB → top bbox (optionally filtered by `class_name`).
|
|
2. Bbox → pixel ROI → `nuwa.pointcloud_roi(...)` (Nuwa crops server-side).
|
|
3. `filter_outliers` (n_std rule) → if <10 points, 422.
|
|
4. **Frame conversion:** `nuwa_to_cv_frame()` flips Y and Z (Nuwa = X-right/Y-up/Z-back; OpenCV = X-right/Y-down/Z-forward). Then `transform_points(T_base_camera, ...)` lifts to base frame.
|
|
5. `compute_axes` (PCA) → `ObjectAxes` with sorted extents.
|
|
6. `GraspPlanner.plan` builds `R_wrist = [x_hand | y_hand | z_hand]` where x_hand = approach direction (currently hard-coded **top-down**, `-Z_base`), y_hand = horizontalized closure axis from PCA mid-axis. Returns `grasp` + `pre_grasp` poses (the latter offset by `approach_offset_m` along +Z).
|
|
7. `/execute` sequences: hand_open → pre_grasp → grasp_approach → hand_close, each timed.
|
|
|
|
### Frame conventions (gotcha-prone)
|
|
|
|
There are **three coexisting camera-frame conventions** — keep them straight:
|
|
- **Nuwa native** (`/api/pointcloud`): X-right, Y-up, Z-back (Z<0 in front).
|
|
- **OpenCV / solvePnP** (`T_cam_marker` from marker-finder, `T_base_camera` from hand-eye): X-right, Y-down, Z-forward (Z>0 in front).
|
|
- **Nuwa ROI** (`/api/pointcloud/roi`): already returned in OpenCV convention `[x, -y, -z, r, g, b]` server-side. The wrapper `NuwaClient.pointcloud_roi` does **not** re-flip — but the grasp pipeline still calls `nuwa_to_cv_frame` on the result. Be aware when adding new code paths that consume Nuwa point clouds: full-frame needs flipping, ROI nominally does not (this is currently inconsistent — verify behavior against actual data when modifying).
|
|
|
|
Marker-finder caveat: it uses **D435i intrinsics baked into its config** that don't match the Nuwa. The `corners` it returns are reusable; the absolute pose `position_m`/`rotation_matrix` carries that intrinsics error into the calibration. Marker-finder accuracy is the dominant error source in `T_base_camera`.
|
|
|
|
### Conventions to preserve
|
|
|
|
- **Dry-run semantics:** any G1-mutating endpoint takes a `dry_run` flag; `G1Client._post` short-circuits and logs `[DRY-RUN]` instead of hitting the network. Never silently strip dry-run from a mutating path.
|
|
- **Per-request client construction** (via `api/_deps.py`) is intentional — runtime `PUT /api/config` updates would otherwise be invisible.
|
|
- Endpoints return `{ ..., "timings_ms": {step: ms, ..., total: ms} }`. New endpoints should follow this with `Stopwatch().step(name)`.
|
|
- Calibration paths in config (`calibration.file`, `calibration.samples_file`) are resolved against `PRESA_MANO_DATA` when relative — see `_deps.calibration_file_path()` / `samples_file_path()`.
|
|
- Italian is fine in user-facing copy (GUI, README, comments) — it's the project default.
|