Desmond-Dong commited on
Commit
02470ed
Β·
1 Parent(s): 9d92632

docs: sync documentation with code (100Hz control loop, 2fps idle camera)

Browse files
Files changed (1) hide show
  1. PROJECT_PLAN.md +6 -4
PROJECT_PLAN.md CHANGED
@@ -69,14 +69,14 @@ Integrate Home Assistant voice assistant functionality into Reachy Mini Wi-Fi ro
69
  β”‚ β”‚ β”‚ β€’ MJPEG stream server β”‚ β”‚ β€’ AdamCodd/YOLOv11n-face β”‚ β”‚ β”‚
70
  β”‚ β”‚ β”‚ β€’ ESPHome Camera entity β”‚ β”‚ β€’ Adaptive frame rate: β”‚ β”‚ β”‚
71
  β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ - 15fps: conversation/face β”‚ β”‚ β”‚
72
- β”‚ β”‚ β”‚ - 3fps: idle (power saving) β”‚ β”‚ β”‚
73
  β”‚ β”‚ β”‚ β€’ look_at_image() pose calc β”‚ β”‚ β”‚
74
  β”‚ β”‚ β”‚ β€’ Smooth return after face lost β”‚ β”‚ β”‚
75
  β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚
76
  β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
77
  β”‚ β”‚
78
  β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ MOTION CONTROL ────────────────────────────┐ β”‚
79
- β”‚ β”‚ MovementManager (10Hz Control Loop) β”‚ β”‚
80
  β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚
81
  β”‚ β”‚ β”‚ Motion Layers (Priority: Move > Action > SpeechSway > Breath) β”‚ β”‚ β”‚
82
  β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚
@@ -137,7 +137,7 @@ Integrate Home Assistant voice assistant functionality into Reachy Mini Wi-Fi ro
137
  - [x] Antenna animation control
138
  - [x] Voice state feedback actions
139
  - [x] YOLO face tracking (replaces DOA sound source localization)
140
- - [x] 5Hz unified motion control loop
141
 
142
  ### Application Architecture
143
  - [x] Compliant with Reachy Mini App architecture
@@ -533,7 +533,7 @@ automation:
533
  |---------|-------------|------------------------|--------|
534
  | DOA turn-to-sound | Turn toward speaker at wakeup | `satellite.py:_turn_to_sound_source()` | βœ… Implemented |
535
  | YOLO face detection | Uses `AdamCodd/YOLOv11n-face-detection` model | `head_tracker.py` | βœ… Implemented |
536
- | Adaptive frame rate tracking | 15fps during conversation, 3fps when idle without face | `camera_server.py` | βœ… Implemented |
537
  | look_at_image() | Calculate target pose from face position | `camera_server.py` | βœ… Implemented |
538
  | Smooth return to neutral | Smooth return within 1 second after face lost | `camera_server.py` | βœ… Implemented |
539
  | face_tracking_offsets | As secondary pose overlay to motion control | `movement_manager.py` | βœ… Implemented |
@@ -910,6 +910,8 @@ def _get_cached_head_pose(self):
910
 
911
  ## πŸ”§ Daemon Crash Deep Fix (2026-01-07)
912
 
 
 
913
  ### Problem Description
914
  During long-term operation, `reachy_mini daemon` still crashes, previous fix not thorough enough.
915
 
 
69
  β”‚ β”‚ β”‚ β€’ MJPEG stream server β”‚ β”‚ β€’ AdamCodd/YOLOv11n-face β”‚ β”‚ β”‚
70
  β”‚ β”‚ β”‚ β€’ ESPHome Camera entity β”‚ β”‚ β€’ Adaptive frame rate: β”‚ β”‚ β”‚
71
  β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ - 15fps: conversation/face β”‚ β”‚ β”‚
72
+ β”‚ β”‚ β”‚ - 2fps: idle (power saving) β”‚ β”‚ β”‚
73
  β”‚ β”‚ β”‚ β€’ look_at_image() pose calc β”‚ β”‚ β”‚
74
  β”‚ β”‚ β”‚ β€’ Smooth return after face lost β”‚ β”‚ β”‚
75
  β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚
76
  β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
77
  β”‚ β”‚
78
  β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ MOTION CONTROL ────────────────────────────┐ β”‚
79
+ β”‚ β”‚ MovementManager (100Hz Control Loop) β”‚ β”‚
80
  β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚
81
  β”‚ β”‚ β”‚ Motion Layers (Priority: Move > Action > SpeechSway > Breath) β”‚ β”‚ β”‚
82
  β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚
 
137
  - [x] Antenna animation control
138
  - [x] Voice state feedback actions
139
  - [x] YOLO face tracking (replaces DOA sound source localization)
140
+ - [x] 100Hz unified motion control loop
141
 
142
  ### Application Architecture
143
  - [x] Compliant with Reachy Mini App architecture
 
533
  |---------|-------------|------------------------|--------|
534
  | DOA turn-to-sound | Turn toward speaker at wakeup | `satellite.py:_turn_to_sound_source()` | βœ… Implemented |
535
  | YOLO face detection | Uses `AdamCodd/YOLOv11n-face-detection` model | `head_tracker.py` | βœ… Implemented |
536
+ | Adaptive frame rate tracking | 15fps during conversation, 2fps when idle without face | `camera_server.py` | βœ… Implemented |
537
  | look_at_image() | Calculate target pose from face position | `camera_server.py` | βœ… Implemented |
538
  | Smooth return to neutral | Smooth return within 1 second after face lost | `camera_server.py` | βœ… Implemented |
539
  | face_tracking_offsets | As secondary pose overlay to motion control | `movement_manager.py` | βœ… Implemented |
 
910
 
911
  ## πŸ”§ Daemon Crash Deep Fix (2026-01-07)
912
 
913
+ > **Update (2026-01-12)**: After daemon updates and further testing, control loop frequency has been restored to 100Hz (same as `reachy_mini_conversation_app`). The pose change threshold (0.005) and state cache TTL (2s) optimizations remain in place to reduce unnecessary Zenoh messages.
914
+
915
  ### Problem Description
916
  During long-term operation, `reachy_mini daemon` still crashes, previous fix not thorough enough.
917