Adriano Dal PastroandClaude Opus 4.7
ac573505b7
Pipeline completa per calibrazione hand-eye e grasp planning su robot Unitree G1/H1 con camera depth Nuwa-HP60C. Esposta come FastAPI + GUI vanilla JS, con assistente AI (OpenRouter + tool calling curato) per pilotare il sistema in linguaggio naturale. Architettura: - FastAPI orchestrator (porta da .env, default 8890) che coordina 4 microservizi upstream: Nuwa-HP60C, marker-finder, YOLO, robot bridge. - Dual-robot: services.g1 e services.h1 in config.yaml; switchabili a runtime via PUT /api/home/robot. - Calibrazione hand-eye eye-to-hand split per-side (right/left). - Marker pose: marker-finder riceve gli intrinseci RGB della Nuwa al boot, PnP server-side con quei dati. Nessun re-PnP locale richiesto. - Depth-to-RGB: registrazione iterativa IR↔RGB per back-projection 3D corretta (Nuwa /api/measure usa coords IR; il nostro /api/perception/ measure_rgb prende pixel RGB e ritorna 3D nel frame RGB OpenCV). - Correzione affine sul depth bias (calibrabile via marker come ground-truth) — riduce errore Z da ~6% a ~1.5% nei test. - AI assistant: 16 tool curati (move_wrist_*, hand_action, get_marker[_3d], detect_object, calibration_*, set_active_robot…), system prompt dinamico che riflette robot attivo + marker config + addendum customizzabile da GUI. Temperature di default 0.1 per evitare invenzioni. GUI (7 tab): - Home: stato comunicazione device + selettore robot attivo G1/H1 - Assistente AI: chat con OpenRouter, history persistita lato browser, task riutilizzabili (prompt salvati), trace tool calls - Impostazioni AI: modello da combo curata, temperature, max iterations, prompt addendum, anteprima system prompt - Calibrazione, Auto-Calibrazione, TrovaUtensile, Servo Visuale, Vista 3D, Movimento EE — invariate dalle versioni precedenti Setup: - Variabili in presa-mano/.env (template in .env.example) - OPENROUTER_API_KEY richiesta solo per il tab Assistente AI - PRESA_MANO_PORT configurabile Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
64 lines
1.8 KiB
Python
64 lines
1.8 KiB
Python
"""Object axis estimation via PCA.
|
|
|
|
Given Nx3 object points (in any frame), compute:
|
|
- centroid
|
|
- 3 principal axes (unit vectors) with corresponding extents
|
|
- right-handed frame (axes columns form a rotation matrix)
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from dataclasses import dataclass
|
|
|
|
import numpy as np
|
|
|
|
|
|
@dataclass
|
|
class ObjectAxes:
|
|
centroid: np.ndarray # (3,)
|
|
axes: np.ndarray # (3,3) columns = unit vectors, sorted largest extent first
|
|
extents: np.ndarray # (3,) along each axis (meters)
|
|
|
|
@property
|
|
def long_axis(self) -> np.ndarray:
|
|
return self.axes[:, 0]
|
|
|
|
@property
|
|
def mid_axis(self) -> np.ndarray:
|
|
return self.axes[:, 1]
|
|
|
|
@property
|
|
def short_axis(self) -> np.ndarray:
|
|
return self.axes[:, 2]
|
|
|
|
def to_dict(self) -> dict:
|
|
return {
|
|
"centroid": self.centroid.tolist(),
|
|
"axes": self.axes.tolist(),
|
|
"extents_m": self.extents.tolist(),
|
|
}
|
|
|
|
|
|
def compute_axes(points_xyz: np.ndarray) -> ObjectAxes:
|
|
pts = np.asarray(points_xyz, dtype=np.float64)
|
|
if pts.ndim != 2 or pts.shape[1] != 3 or len(pts) < 3:
|
|
raise ValueError(f"need at least 3 3D points, got shape {pts.shape}")
|
|
|
|
c = pts.mean(axis=0)
|
|
centered = pts - c
|
|
cov = np.cov(centered.T)
|
|
eigvals, eigvecs = np.linalg.eigh(cov)
|
|
# eigh returns ascending — reverse for largest-first
|
|
order = np.argsort(eigvals)[::-1]
|
|
eigvals = eigvals[order]
|
|
eigvecs = eigvecs[:, order]
|
|
|
|
# Ensure right-handed coordinate system
|
|
if np.linalg.det(eigvecs) < 0:
|
|
eigvecs[:, 2] = -eigvecs[:, 2]
|
|
|
|
# Extent along each axis = range of projected coordinates
|
|
proj = centered @ eigvecs # Nx3
|
|
extents = proj.max(axis=0) - proj.min(axis=0)
|
|
return ObjectAxes(centroid=c, axes=eigvecs, extents=extents)
|