Desmond-Dong commited on
Commit
91cfdeb
ยท
1 Parent(s): 5f422a0

docs: translate PROJECT_PLAN.md to English

Browse files
Files changed (1) hide show
  1. PROJECT_PLAN.md +315 -1058
PROJECT_PLAN.md CHANGED
@@ -1,29 +1,29 @@
1
- # Reachy Mini Home Assistant Voice Assistant - ้กน็›ฎ่ฎกๅˆ’
2
 
3
- ## ้กน็›ฎๆฆ‚่ฟฐ
4
 
5
- ๅฐ† Home Assistant ่ฏญ้ŸณๅŠฉๆ‰‹ๅŠŸ่ƒฝ้›†ๆˆๅˆฐ Reachy Mini Wi-Fi็‰ˆๆœฌๆœบๅ™จไบบ๏ผŒ้€š่ฟ‡ ESPHome ๅ่ฎฎไธŽ Home Assistant ้€šไฟกใ€‚
6
 
7
- ## ๆœฌๅœฐ้กน็›ฎ็›ฎๅฝ•ๅ‚่€ƒ (็ฆๆญขไฟฎๆ”นๅ‚่€ƒ็›ฎๅฝ•ๅ†…ไปปไฝ•ๆ–‡ไปถ)
8
- 1. [linux-voice-assistant](reference/linux-voice-assistant)๏ผŒ่ฟ™ๆ˜ฏไธ€ไธชๅŸบไบŽ Linux ็š„Home Assistant็š„่ฏญ้ŸณๅŠฉๆ‰‹ๅบ”็”จ๏ผŒ็”จไบŽๅ‚่€ƒใ€‚
9
- 2. [Reachy Mini SDK](reference/reachy_mini) ่ฟ™ๆ˜ฏ Reachy Mini SDK ็š„ๆœฌๅœฐ้กน็›ฎ็›ฎๅฝ•๏ผŒ็”จไบŽๅ‚่€ƒใ€‚
10
- 3. [reachy_mini_conversation_app](reference/reachy_mini_conversation_app) - Reachy Mini ๅฏน่ฏๅบ”็”จ๏ผŒ็”จไบŽๅ‚่€ƒ
11
- 4. [reachy-mini-desktop-app](reference/reachy-mini-desktop-app) - Reachy Mini ๆกŒ้ขๅบ”็”จ๏ผŒ็”จไบŽๅ‚่€ƒ
12
- 5. [sendspin](reference/sendspin-cli/) - sendspin client๏ผŒ็”จไบŽๅ‚่€ƒ
13
 
14
- ## ๆ ธๅฟƒ่ฎพ่ฎกๅŽŸๅˆ™
15
 
16
- 1. **้›ถ้…็ฝฎๅฎ‰่ฃ…** - ็”จๆˆทๅช้œ€ๅฎ‰่ฃ…ๅบ”็”จ๏ผŒๆ— ้œ€ๆ‰‹ๅŠจ้…็ฝฎ
17
- 2. **ไฝฟ็”จ Reachy Mini ๅŽŸ็”Ÿ็กฌไปถ** - ไฝฟ็”จๆœบๅ™จไบบ่‡ชๅธฆ็š„้บฆๅ…‹้ฃŽๅ’Œๆ‰ฌๅฃฐๅ™จ
18
- 3. **Home Assistant ้›†ไธญ็ฎก็†** - ๆ‰€ๆœ‰้…็ฝฎๅœจ Home Assistant ็ซฏๅฎŒๆˆ
19
- 4. **่ฟๅŠจๅ้ฆˆ** - ่ฏญ้Ÿณไบคไบ’ๆ—ถๆไพ›ๅคด้ƒจ่ฟๅŠจๅ’Œๅคฉ็บฟๅŠจ็”ปๅ้ฆˆ
20
- 5. **้กน็›ฎ็บฆๆŸ** - ๆ•ดไธช้กน็›ฎ้œ€่ฆไธฅๆ ผ้ตๅพช [Reachy Mini SDK](reachy_mini) ็š„ๆžถๆž„่ฎพ่ฎกไธŽ็บฆๆŸ
21
- 6. **ไปฃ็ ่ดจ้‡** - ไธฅๆ ผ้ตๅพชPythonๅผ€ๅ‘็š„ๆ ‡ๅ‡†๏ผŒๅนถๅšๅˆฐไปฃ็ ้ฃŽๆ ผไธ€่‡ด๏ผŒไปฃ็ ็ป“ๆž„ๆธ…ๆ™ฐ๏ผŒๆณจ้‡ŠๅฎŒๆ•ด๏ผŒๆ–‡ๆกฃๅฎŒๅ–„๏ผŒๆต‹่ฏ•่ฆ†็›–็އ้ซ˜๏ผŒไปฃ็ ่ดจ้‡้ซ˜๏ผŒไปฃ็ ๅฏ่ฏปๆ€ง้ซ˜๏ผŒไปฃ็ ๅฏ็ปดๆŠคๆ€ง้ซ˜๏ผŒไปฃ็ ๅฏๆ‰ฉๅฑ•ๆ€ง้ซ˜๏ผŒไปฃ็ ๅฏๅค็”จๆ€ง้ซ˜
22
- 7. **ๅŠŸ่ƒฝไผ˜ๅ…ˆ็บง** - ไธŽhome assistant็š„่ฏญ้Ÿณๅฏน่ฏไธบๆœ€้ซ˜ไผ˜ๅ…ˆ็บง๏ผŒไปปไฝ•ๅ…ถๅฎƒ็š„ๅŠŸ่ƒฝ้ƒฝๆ˜ฏ่พ…ๅŠฉๅŠŸ่ƒฝ๏ผŒไธ่ƒฝๅฝฑๅ“่ฏญ้Ÿณๅฏน่ฏ็š„ๅŠŸ่ƒฝๅ’Œๅ“ๅบ”้€Ÿๅบฆ
23
- 8. **ไธ่ฐƒ็”จไปปไฝ•LEDๅŠŸ่ƒฝ** - LED้ƒฝ่ขซ้š่—ๅœจไบ†ๆœบๅ™จไบบๅ†…้ƒจ๏ผŒๆ‰€ๆœ‰็š„LEDๆŽงๅˆถๅ…จ้ƒจ้ƒฝๅฟฝ็•ฅ๏ผŒไธ่ฆไฝฟ็”จLEDๆŽงๅˆถ
24
- 9. **ไฟ็•™ๅŠŸ่ƒฝไผ˜ๅ…ˆ** - ไปปไฝ•ไปฃ็ ไฟฎๆ”น้ƒฝๅบ”่ฏฅๅœจไฟ็•™ๅทฒๅฎŒๆˆๅŠŸ่ƒฝ็š„ๅ‰ๆไธ‹ไผ˜ๅŒ–๏ผŒไธ่ƒฝไปฅๅŽป้™คๅŠŸ่ƒฝ็š„ๆ–นๅผๆฅ่งฃๅ†ณ้—ฎ้ข˜ใ€‚ๅฝ“ๆœ‰้—ฎ้ข˜ๅ‘็”Ÿๆ—ถ๏ผŒๅบ”่ฏฅๅ……ๅˆ†ๅ‚่€ƒๆกˆไพ‹ไน‹ๅŽไปฅ่งฃๅ†ณ้—ฎ้ข˜ไธบไผ˜ๅ…ˆ็บง๏ผŒ่€Œไธๆ˜ฏๆทปๅŠ ๅ„็งๆ—ฅๅฟ—่พ“ๅ‡บไธบไผ˜ๅ…ˆ็บง
25
 
26
- ## ๆŠ€ๆœฏๆžถๆž„
27
 
28
  ```
29
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
@@ -40,16 +40,16 @@
40
  โ”‚ โ”‚
41
  โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚
42
  โ”‚ โ”‚ Camera + Face Tracking (YOLO) โ”‚ โ”‚
43
- โ”‚ โ”‚ - 15Hz ไบบ่„ธๆฃ€ๆต‹ไธŽ่ฟฝ่ธช โ”‚ โ”‚
44
- โ”‚ โ”‚ - look_at_image() ่ฎก็ฎ—็›ฎๆ ‡ๅงฟๆ€ โ”‚ โ”‚
45
- โ”‚ โ”‚ - ไบบ่„ธไธขๅคฑๅŽๅนณๆป‘ๅ›žไธญๆ€งไฝ็ฝฎ โ”‚ โ”‚
46
  โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚
47
  โ”‚ โ”‚
48
  โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚
49
  โ”‚ โ”‚ Motion Controller (Head + Antennas) - 5Hz โ”‚ โ”‚
50
  โ”‚ โ”‚ - Face tracking offsets (secondary pose) โ”‚ โ”‚
51
- โ”‚ โ”‚ - Speech sway (่ฏญ้Ÿณ้ฉฑๅŠจๅพฎๅŠจ) โ”‚ โ”‚
52
- โ”‚ โ”‚ - Breathing animation (็ฉบ้—ฒๅ‘ผๅธ) โ”‚ โ”‚
53
  โ”‚ โ”‚ - on_wakeup โ†’ on_listening โ†’ on_speaking โ†’ on_idle โ”‚ โ”‚
54
  โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚
55
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
@@ -65,1035 +65,348 @@
65
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
66
  ```
67
 
68
- ## ๅทฒๅฎŒๆˆๅŠŸ่ƒฝ
69
-
70
- ### ๆ ธๅฟƒๅŠŸ่ƒฝ
71
- - [x] ESPHome ๅ่ฎฎๆœๅŠกๅ™จๅฎž็Žฐ
72
- - [x] mDNS ๆœๅŠกๅ‘็Žฐ๏ผˆ่‡ชๅŠจ่ขซ Home Assistant ๅ‘็Žฐ๏ผ‰
73
- - [x] ๆœฌๅœฐๅ”ค้†’่ฏๆฃ€ๆต‹๏ผˆmicroWakeWord๏ผ‰
74
- - [x] ๆ‹ไธ€ๆ‹ๅ”ค้†’๏ผˆIMU ๅŠ ้€Ÿๅบฆๆฃ€ๆต‹๏ผŒไป…ๆ— ็บฟ็‰ˆๆœฌ๏ผ‰
75
- - [x] ้Ÿณ้ข‘ๆตไผ ่พ“ๅˆฐ Home Assistant
76
- - [x] TTS ้Ÿณ้ข‘ๆ’ญๆ”พ
77
- - [x] ๅœๆญข่ฏๆฃ€ๆต‹
78
-
79
- ### Reachy Mini ้›†ๆˆ
80
- - [x] ไฝฟ็”จ Reachy Mini SDK ็š„้บฆๅ…‹้ฃŽ่พ“ๅ…ฅ
81
- - [x] ไฝฟ็”จ Reachy Mini SDK ็š„ๆ‰ฌๅฃฐๅ™จ่พ“ๅ‡บ
82
- - [x] ๅคด้ƒจ่ฟๅŠจๆŽงๅˆถ๏ผˆ็‚นๅคดใ€ๆ‘‡ๅคดใ€ๆณจ่ง†๏ผ‰
83
- - [x] ๅคฉ็บฟๅŠจ็”ปๆŽงๅˆถ
84
- - [x] ่ฏญ้Ÿณ็Šถๆ€ๅ้ฆˆๅŠจไฝœ
85
- - [x] YOLO ไบบ่„ธ่ฟฝ่ธช๏ผˆๆ›ฟไปฃ DOA ๅฃฐๆบๅฎšไฝ๏ผ‰
86
- - [x] 5Hz ็ปŸไธ€่ฟๅŠจๆŽงๅˆถๅพช็Žฏ
87
-
88
- ### ๅบ”็”จๆžถๆž„
89
- - [x] ็ฌฆๅˆ Reachy Mini App ๆžถๆž„
90
- - [x] ่‡ชๅŠจไธ‹่ฝฝๅ”ค้†’่ฏๆจกๅž‹
91
- - [x] ่‡ชๅŠจไธ‹่ฝฝ้Ÿณๆ•ˆๆ–‡ไปถ
92
- - [x] ๆ— ้œ€ .env ้…็ฝฎๆ–‡ไปถ
93
-
94
- ## ๆ–‡ไปถๆธ…ๅ•
95
 
96
  ```
97
  reachy_mini_ha_voice/
98
  โ”œโ”€โ”€ reachy_mini_ha_voice/
99
- โ”‚ โ”œโ”€โ”€ __init__.py # ๅŒ…ๅˆๅง‹ๅŒ–
100
- โ”‚ โ”œโ”€โ”€ __main__.py py # ๅ‘ฝไปค่กŒๅ…ฅๅฃ
101
- โ”‚ โ”œโ”€โ”€ main.py # ReachyMiniApp ๅ…ฅๅฃ
102
- โ”‚ โ”œโ”€โ”€ voice_assistant.py # ่ฏญ้ŸณๅŠฉๆ‰‹ๆœๅŠก
103
- โ”‚ โ”œโ”€โ”€ satellite.py # ESPHome ๅ่ฎฎๅค„็†
104
- โ”‚ โ”œโ”€โ”€ audio_player.py # ้Ÿณ้ข‘ๆ’ญๆ”พๅ™จ
105
- โ”‚ โ”œโ”€โ”€ camera_server.py # MJPEG ๆ‘„ๅƒๅคดๆตๆœๅŠกๅ™จ + ไบบ่„ธ่ฟฝ่ธช
106
- โ”‚ โ”œโ”€โ”€ head_tracker.py # YOLO ไบบ่„ธๆฃ€ๆต‹ๅ™จ
107
- โ”‚ โ”œโ”€โ”€ motion.py # ่ฟๅŠจๆŽงๅˆถ (้ซ˜ๅฑ‚ API)
108
- โ”‚ โ”œโ”€โ”€ movement_manager.py # ็ปŸไธ€่ฟๅŠจ็ฎก็†ๅ™จ (20Hz ๆŽงๅˆถๅพช็Žฏ๏ผŒไผ˜ๅŒ–ไปฅ้˜ฒๆญข daemon ๅดฉๆบƒ)
109
- โ”‚ โ”œโ”€โ”€ models.py # ๆ•ฐๆฎๆจกๅž‹
110
- โ”‚ โ”œโ”€โ”€ entity.py # ESPHome ๅŸบ็ก€ๅฎžไฝ“
111
- โ”‚ โ”œโ”€โ”€ entity_extensions.py # ๆ‰ฉๅฑ•ๅฎžไฝ“็ฑปๅž‹
112
- โ”‚ โ”œโ”€โ”€ reachy_controller.py # Reachy Mini ๆŽงๅˆถๅ™จๅŒ…่ฃ…
113
- โ”‚ โ”œโ”€โ”€ api_server.py # API ๆœๅŠกๅ™จ
114
- โ”‚ โ”œโ”€โ”€ zeroconf.py # mDNS ๅ‘็Žฐ
115
- โ”‚ โ””โ”€โ”€ util.py # ๅทฅๅ…ทๅ‡ฝๆ•ฐ
116
- โ”œโ”€โ”€ wakewords/ # ๅ”ค้†’่ฏๆจกๅž‹๏ผˆ่‡ชๅŠจไธ‹่ฝฝ๏ผ‰
117
- โ”‚ โ”œโ”€โ”€ okay_nabu.json
118
- โ”‚ โ”œโ”€โ”€ okay_nabu.tflite
119
- โ”‚ โ”œโ”€โ”€ hey_jarvis.json
120
- โ”‚ โ”œโ”€โ”€ hey_jarvis.tflite
121
- โ”‚ โ”œโ”€โ”€ stop.json
122
- โ”‚ โ””โ”€โ”€ stop.tflite
123
- โ”œโ”€โ”€ sounds/ # ้Ÿณๆ•ˆๆ–‡ไปถ๏ผˆ่‡ชๅŠจไธ‹่ฝฝ๏ผ‰
124
- โ”‚ โ”œโ”€โ”€ wake_word_triggered.flac
125
- โ”‚ โ””โ”€โ”€ timer_finished.flac
126
- โ”œโ”€โ”€ pyproject.toml # ้กน็›ฎ้…็ฝฎ
127
- โ”œโ”€โ”€ README.md # ่ฏดๆ˜Žๆ–‡ๆกฃ
128
- โ””โ”€โ”€ PROJECT_PLAN.md # ้กน็›ฎ่ฎกๅˆ’
129
  ```
130
 
131
- ## ไพ่ต–้กน
132
-
133
- ```toml
134
- dependencies = [
135
- "reachy-mini", # Reachy Mini SDK
136
- "sounddevice>=0.4.6", # ้Ÿณ้ข‘ๅค„็†๏ผˆๅค‡็”จ๏ผ‰
137
- "soundfile>=0.12.0", # ้Ÿณ้ข‘ๆ–‡ไปถ่ฏปๅ–
138
- "numpy>=1.24.0", # ๆ•ฐๅ€ผ่ฎก็ฎ—
139
- "pymicro-wakeword>=2.0.0,<3.0.0", # ๅ”ค้†’่ฏๆฃ€ๆต‹
140
- "pyopen-wakeword>=1.0.0,<2.0.0", # ๅค‡็”จๅ”ค้†’่ฏ
141
- "aioesphomeapi>=42.0.0", # ESPHome ๅ่ฎฎ
142
- "zeroconf>=0.100.0", # mDNS ๅ‘็Žฐ
143
- "scipy>=1.10.0", # ่ฟๅŠจๆŽงๅˆถ
144
- "pydantic>=2.0.0", # ๆ•ฐๆฎ้ชŒ่ฏ
145
- ]
146
- ```
147
-
148
- ## ไฝฟ็”จๆต็จ‹
149
-
150
- 1. **ๅฎ‰่ฃ…ๅบ”็”จ**
151
- - ไปŽ Reachy Mini App Store ๅฎ‰่ฃ…`reachy-mini-ha-voice`
152
-
153
- 2. **ๅฏๅŠจๅบ”็”จ**
154
- - ๅบ”็”จ่‡ชๅŠจๅฏๅŠจ ESPHome ๆœๅŠกๅ™จ๏ผˆ็ซฏๅฃ 6053๏ผ‰
155
- - ่‡ชๅŠจไธ‹่ฝฝๆ‰€้œ€ๆจกๅž‹ๅ’Œ้Ÿณๆ•ˆ
156
-
157
- 3. **่ฟžๆŽฅ Home Assistant**
158
- - Home Assistant ่‡ชๅŠจๅ‘็Žฐ่ฎพๅค‡๏ผˆmDNS๏ผ‰
159
- - ๆˆ–ๆ‰‹ๅŠจๆทปๅŠ ๏ผš่ฎพ็ฝฎ โ†’ ่ฎพๅค‡ไธŽๆœๅŠก โ†’ ๆทปๅŠ ้›†ๆˆ โ†’ ESPHome
160
-
161
- 4. **ไฝฟ็”จ่ฏญ้ŸณๅŠฉๆ‰‹**
162
- - ่ฏด "Okay Nabu" ๅ”ค้†’
163
- - ่ฏดๅ‡บๅ‘ฝไปค
164
- - Reachy Mini ไผšๅšๅ‡บ่ฟๅŠจๅ้ฆˆ
165
-
166
- ## ESPHome ๅฎžไฝ“่ง„ๅˆ’
167
-
168
- ๅŸบไบŽ Reachy Mini SDK ๆทฑๅ…ฅๅˆ†ๆž๏ผŒไปฅไธ‹ๅฎžไฝ“ๅทฒๆšด้œฒ็ป™ Home Assistant๏ผš
169
-
170
- ### ๅทฒๅฎž็Žฐๅฎžไฝ“
171
-
172
- | ๅฎžไฝ“็ฑปๅž‹ | ๅ็งฐ | ่ฏดๆ˜Ž |
173
- |---------|------|------|
174
- | Media Player | `media_player` | ้Ÿณ้ข‘ๆ’ญๆ”พๆŽงๅˆถ |
175
- | Voice Assistant | `voice_assistant` | ่ฏญ้ŸณๅŠฉๆ‰‹็ฎก้“ |
176
-
177
- ### ๅทฒๅฎž็Žฐ็š„ๆŽงๅˆถๅฎžไฝ“ (Controls) - ๅฏ่ฏปๅ†™
178
-
179
- #### Phase 1-3: ๅŸบ็ก€ๆŽงๅˆถไธŽๅงฟๆ€
180
-
181
- | ESPHome ๅฎžไฝ“็ฑปๅž‹ | ๅ็งฐ | SDK API | ่Œƒๅ›ด/้€‰้กน | ่ฏดๆ˜Ž |
182
- |-----------------|------|---------|----------|------|
183
- | `Number` | `speaker_volume` | `AudioPlayer.set_volume()` | 0-100 | ๆ‰ฌๅฃฐๅ™จ้Ÿณ้‡ |
184
- | `Select` | `motor_mode` | `set_motor_control_mode()` | enabled/disabled/gravity_compensation | ็”ตๆœบๆจกๅผ้€‰ๆ‹ฉ |
185
- | `Switch` | `motors_enabled` | `enable_motors()` / `disable_motors()` | on/off | ็”ตๆœบๆ‰ญ็Ÿฉๅผ€ๅ…ณ |
186
- | `Button` | `wake_up` | `mini.wake_up()` | - | ๅ”ค้†’ๆœบๅ™จไบบๅŠจไฝœ |
187
- | `Button` | `go_to_sleep` | `mini.goto_sleep()` | - | ็ก็œ ๆœบๅ™จไบบๅŠจไฝœ |
188
- | `Number` | `head_x` | `goto_target(head=...)` | ยฑ50mm | ๅคด้ƒจ X ไฝ็ฝฎๆŽงๅˆถ |
189
- | `Number` | `head_y` | `goto_target(head=...)` | ยฑ50mm | ๅคด้ƒจ Y ไฝ็ฝฎๆŽงๅˆถ |
190
- | `Number` | `head_z` | `goto_target(head=...)` | ยฑ50mm | ๅคด้ƒจ Z ไฝ็ฝฎๆŽงๅˆถ |
191
- | `Number` | `head_roll` | `goto_target(head=...)` | -40ยฐ ~ +40ยฐ | ๅคด้ƒจ็ฟปๆปš่ง’ๆŽงๅˆถ |
192
- | `Number` | `head_pitch` | `goto_target(head=...)` | -40ยฐ ~ +40ยฐ | ๅคด้ƒจไฟฏไปฐ่ง’ๆŽงๅˆถ |
193
- | `Number` | `head_yaw` | `goto_target(head=...)` | -180ยฐ ~ +180ยฐ | ๅคด้ƒจๅ่ˆช่ง’ๆŽงๅˆถ |
194
- | `Number` | `body_yaw` | `goto_target(body_yaw=...)` | -160ยฐ ~ +160ยฐ | ่บซไฝ“ๅ่ˆช่ง’ๆŽงๅˆถ |
195
- | `Number` | `antenna_left` | `goto_target(antennas=...)` | -90ยฐ ~ +90ยฐ | ๅทฆๅคฉ็บฟ่ง’ๅบฆๆŽงๅˆถ |
196
- | `Number` | `antenna_right` | `goto_target(antennas=...)` | -90ยฐ ~ +90ยฐ | ๅณๅคฉ็บฟ่ง’ๅบฆๆŽงๅˆถ |
197
-
198
- #### Phase 4: ๆณจ่ง†ๆŽงๅˆถ
199
-
200
- | ESPHome ๅฎžไฝ“็ฑปๅž‹ | ๅ็งฐ | SDK API | ่Œƒๅ›ด/้€‰้กน | ่ฏดๆ˜Ž |
201
- |-----------------|------|---------|----------|------|
202
- | `Number` | `look_at_x` | `look_at_world(x, y, z)` | ไธ–็•Œๅๆ ‡ | ๆณจ่ง†็‚น X ๅๆ ‡ |
203
- | `Number` | `look_at_y` | `look_at_world(x, y, z)` | ไธ–็•Œๅๆ ‡ | ๆณจ่ง†็‚น Y ๅๆ ‡ |
204
- | `Number` | `look_at_z` | `look_at_world(x, y, z)` | ไธ–็•Œๅๆ ‡ | ๆณจ่ง†็‚น Z ๅๆ ‡ |
205
-
206
- ### ๅทฒๅฎž็Žฐ็š„ไผ ๆ„Ÿๅ™จๅฎžไฝ“ (Sensors) - ๅช่ฏป
207
-
208
- #### Phase 1 & 5: ๅŸบ็ก€็Šถๆ€ไธŽ้Ÿณ้ข‘ไผ ๆ„Ÿๅ™จ
209
-
210
- | ESPHome ๅฎžไฝ“๏ฟฝ๏ฟฝ๏ฟฝๅž‹ | ๅ็งฐ | SDK API | ่ฏดๆ˜Ž |
211
- |-----------------|------|---------|------|
212
- | `Text Sensor` | `daemon_state` | `DaemonStatus.state` | Daemon ็Šถๆ€ |
213
- | `Binary Sensor` | `backend_ready` | `backend_status.ready` | ๅŽ็ซฏๆ˜ฏๅฆๅฐฑ็ปช |
214
- | `Text Sensor` | `error_message` | `DaemonStatus.error` | ๅฝ“ๅ‰้”™่ฏฏไฟกๆฏ |
215
- | `Sensor` | `doa_angle` | `DoAInfo.angle` | ๅฃฐๆบๆ–นๅ‘่ง’ๅบฆ (ยฐ) |
216
- | `Binary Sensor` | `speech_detected` | `DoAInfo.speech_detected` | ๆ˜ฏๅฆๆฃ€ๆต‹ๅˆฐ่ฏญ้Ÿณ |
217
-
218
- #### Phase 6: ่ฏŠๆ–ญไฟกๆฏ
219
-
220
- | ESPHome ๅฎžไฝ“็ฑปๅž‹ | ๅ็งฐ | SDK API | ่ฏดๆ˜Ž |
221
- |-----------------|------|---------|------|
222
- | `Sensor` | `control_loop_frequency` | `control_loop_stats` | ๆŽงๅˆถๅพช็Žฏ้ข‘็އ (Hz) |
223
- | `Text Sensor` | `sdk_version` | `DaemonStatus.version` | SDK ็‰ˆๆœฌๅท |
224
- | `Text Sensor` | `robot_name` | `DaemonStatus.robot_name` | ๆœบๅ™จไบบๅ็งฐ |
225
- | `Binary Sensor` | `wireless_version` | `DaemonStatus.wireless_version` | ๆ˜ฏๅฆไธบๆ— ็บฟ็‰ˆๆœฌ |
226
- | `Binary Sensor` | `simulation_mode` | `DaemonStatus.simulation_enabled` | ๆ˜ฏๅฆๅœจไปฟ็œŸๆจกๅผ |
227
- | `Text Sensor` | `wlan_ip` | `DaemonStatus.wlan_ip` | ๆ— ็บฟ็ฝ‘็ปœ IP |
228
-
229
- #### Phase 7: IMU ไผ ๆ„Ÿๅ™จ (ไป…ๆ— ็บฟ็‰ˆๆœฌ)
230
-
231
- | ESPHome ๅฎžไฝ“็ฑปๅž‹ | ๅ็งฐ | SDK API | ่ฏดๆ˜Ž |
232
- |-----------------|------|---------|------|
233
- | `Sensor` | `imu_accel_x` | `mini.imu["accelerometer"][0]` | X ่ฝดๅŠ ้€Ÿๅบฆ (m/sยฒ) |
234
- | `Sensor` | `imu_accel_y` | `mini.imu["accelerometer"][1]` | Y ่ฝดๅŠ ้€Ÿๅบฆ (m/sยฒ) |
235
- | `Sensor` | `imu_accel_z` | `mini.imu["accelerometer"][2]` | Z ่ฝดๅŠ ้€Ÿๅบฆ (m/sยฒ) |
236
- | `Sensor` | `imu_gyro_x` | `mini.imu["gyroscope"][0]` | X ่ฝด่ง’้€Ÿๅบฆ (rad/s) |
237
- | `Sensor` | `imu_gyro_y` | `mini.imu["gyroscope"][1]` | Y ่ฝด่ง’้€Ÿๅบฆ (rad/s) |
238
- | `Sensor` | `imu_gyro_z` | `mini.imu["gyroscope"][2]` | Z ่ฝด่ง’้€Ÿๅบฆ (rad/s) |
239
- | `Sensor` | `imu_temperature` | `mini.imu["temperature"]` | IMU ๆธฉๅบฆ (ยฐC) |
240
-
241
- #### Phase 8-12: ๆ‰ฉๅฑ•ๅŠŸ่ƒฝ
242
-
243
- | ESPHome ๅฎžไฝ“็ฑปๅž‹ | ๅ็งฐ | ่ฏดๆ˜Ž |
244
- |-----------------|------|------|
245
- | `Select` | `emotion` | ่กจๆƒ…้€‰ๆ‹ฉๅ™จ (Happy/Sad/Angry/Fear/Surprise/Disgust) |
246
- | `Number` | `microphone_volume` | ้บฆๅ…‹้ฃŽ้Ÿณ้‡ (0-100%) |
247
- | `Camera` | `camera` | ESPHome Camera ๅฎžไฝ“๏ผˆๅฎžๆ—ถ้ข„่งˆ๏ผ‰ |
248
- | `Number` | `led_brightness` | LED ไบฎๅบฆ (0-100%) |
249
- | `Select` | `led_effect` | LED ๆ•ˆๆžœ (off/solid/breathing/rainbow/doa) |
250
- | `Number` | `led_color_r` | LED ็บข่‰ฒๅˆ†้‡ (0-255) |
251
- | `Number` | `led_color_g` | LED ็ปฟ่‰ฒๅˆ†้‡ (0-255) |
252
- | `Number` | `led_color_b` | LED ่“่‰ฒๅˆ†้‡ (0-255) |
253
- | `Switch` | `agc_enabled` | ่‡ชๅŠจๅขž็›ŠๆŽงๅˆถๅผ€ๅ…ณ |
254
- | `Number` | `agc_max_gain` | AGC ๆœ€ๅคงๅขž็›Š (0-30 dB) |
255
- | `Number` | `noise_suppression` | ๅ™ชๅฃฐๆŠ‘ๅˆถ็บงๅˆซ (0-100%) |
256
- | `Binary Sensor` | `echo_cancellation_converged` | ๅ›žๅฃฐๆถˆ้™คๆ”ถๆ•›็Šถๆ€ |
257
-
258
- > **ๆณจๆ„**: ๅคด้ƒจไฝ็ฝฎ (x/y/z) ๅ’Œ่ง’ๅบฆ (roll/pitch/yaw)ใ€่บซไฝ“ๅ่ˆช่ง’ใ€ๅคฉ็บฟ่ง’ๅบฆ้ƒฝๆ˜ฏ**ๅฏๆŽงๅˆถ**็š„ๅฎžไฝ“๏ผŒ
259
- > ไฝฟ็”จ `Number` ็ฑปๅž‹ๅฎž็ŽฐๅŒๅ‘ๆŽงๅˆถใ€‚่ฎพ็ฝฎๆ–ฐๅ€ผๆ—ถ่ฐƒ็”จ `goto_target()`๏ผŒ่ฏปๅ–ๅฝ“ๅ‰ๅ€ผๆ—ถ่ฐƒ็”จ `get_current_head_pose()` ็ญ‰ใ€‚
260
-
261
- ### ๅฎž็Žฐไผ˜ๅ…ˆ็บง
262
-
263
- 1. **Phase 1 - ๅŸบ็ก€็Šถๆ€ไธŽ้Ÿณ้‡** (้ซ˜ไผ˜ๅ…ˆ็บง) โœ… **ๅทฒๅฎŒๆˆ**
264
- - [x] `daemon_state` - Daemon ็Šถๆ€ไผ ๆ„Ÿๅ™จ
265
- - [x] `backend_ready` - ๅŽ็ซฏๅฐฑ็ปช็Šถๆ€
266
- - [x] `error_message` - ้”™่ฏฏไฟกๆฏ
267
- - [x] `speaker_volume` - ๆ‰ฌๅฃฐๅ™จ้Ÿณ้‡ๆŽงๅˆถ
268
-
269
- 2. **Phase 2 - ็”ตๆœบๆŽงๅˆถ** (้ซ˜ไผ˜ๅ…ˆ็บง) โœ… **ๅทฒๅฎŒๆˆ**
270
- - [x] `motors_enabled` - ็”ตๆœบๅผ€ๅ…ณ
271
- - [x] `motor_mode` - ็”ตๆœบๆจกๅผ้€‰ๆ‹ฉ (enabled/disabled/gravity_compensation)
272
- - [x] `wake_up` / `go_to_sleep` - ๅ”ค้†’/็ก็œ ๆŒ‰้’ฎ
273
-
274
- 3. **Phase 3 - ๅงฟๆ€ๆŽงๅˆถ** (ไธญไผ˜ๅ…ˆ็บง) โœ… **ๅทฒๅฎŒๆˆ**
275
- - [x] `head_x/y/z` - ๅคด้ƒจไฝ็ฝฎๆŽงๅˆถ
276
- - [x] `head_roll/pitch/yaw` - ๅคด้ƒจ่ง’ๅบฆๆŽงๅˆถ
277
- - [x] `body_yaw` - ่บซไฝ“ๅ่ˆช่ง’ๆŽงๅˆถ
278
- - [x] `antenna_left/right` - ๅคฉ็บฟ่ง’ๅบฆๆŽงๅˆถ
279
-
280
- 4. **Phase 4 - ๆณจ่ง†ๆŽงๅˆถ** (ไธญไผ˜ๅ…ˆ็บง) โœ… **ๅทฒๅฎŒๆˆ**
281
- - [x] `look_at_x/y/z` - ๆณจ่ง†็‚นๅๆ ‡ๆŽงๅˆถ
282
-
283
- 5. **Phase 5 - ้Ÿณ้ข‘ไผ ๆ„Ÿๅ™จ** (ไฝŽไผ˜ๅ…ˆ็บง) โœ… **ๅทฒๅฎŒๆˆ**
284
- - [x] `doa_angle` - ๅฃฐๆบๆ–นๅ‘
285
- - [x] `speech_detected` - ่ฏญ้Ÿณๆฃ€ๆต‹
286
-
287
- 6. **Phase 6 - ่ฏŠๆ–ญไฟกๆฏ** (ไฝŽไผ˜ๅ…ˆ็บง) โœ… **ๅทฒๅฎŒๆˆ**
288
- - [x] `control_loop_frequency` - ๆŽงๅˆถๅพช็Žฏ้ข‘็އ
289
- - [x] `sdk_version` - SDK ็‰ˆๆœฌ
290
- - [x] `robot_name` - ๆœบๅ™จไบบๅ็งฐ
291
- - [x] `wireless_version` - ๆ— ็บฟ็‰ˆๆœฌๆ ‡่ฏ†
292
- - [x] `simulation_mode` - ไปฟ็œŸๆจกๅผๆ ‡่ฏ†
293
- - [x] `wlan_ip` - ๆ— ็บฟ IP ๅœฐๅ€
294
-
295
- 7. **Phase 7 - IMU ไผ ๆ„Ÿๅ™จ** (ๅฏ้€‰๏ผŒไป…ๆ— ็บฟ็‰ˆๆœฌ) โœ… **ๅทฒๅฎŒๆˆ**
296
- - [x] `imu_accel_x/y/z` - ๅŠ ้€Ÿๅบฆ่ฎก
297
- - [x] `imu_gyro_x/y/z` - ้™€่žบไปช
298
- - [x] `imu_temperature` - IMU ๆธฉๅบฆ
299
-
300
- 8. **Phase 8 - ่กจๆƒ…ๆŽงๅˆถ** โœ… **ๅทฒๅฎŒๆˆ**
301
- - [x] `emotion` - ่กจๆƒ…้€‰ๆ‹ฉๅ™จ (Happy/Sad/Angry/Fear/Surprise/Disgust)
302
-
303
- 9. **Phase 9 - ้Ÿณ้ข‘ๆŽงๅˆถ** โœ… **ๅทฒๅฎŒๆˆ**
304
- - [x] `microphone_volume` - ้บฆๅ…‹้ฃŽ้Ÿณ้‡ๆŽงๅˆถ (0-100%)
305
-
306
- 10. **Phase 10 - ๆ‘„ๅƒๅคด้›†ๆˆ** โœ… **ๅทฒๅฎŒๆˆ**
307
- - [x] `camera` - ESPHome Camera ๅฎžไฝ“๏ผˆๅฎžๆ—ถ้ข„่งˆ๏ผ‰
308
-
309
- 11. **Phase 11 - LED ๆŽงๅˆถ** โŒ **ๅทฒ็ฆ็”จ๏ผˆLED ้š่—ๅœจๆœบๅ™จไบบๅ†…้ƒจ๏ผ‰**
310
- - [ ] `led_brightness` - LED ไบฎๅบฆ (0-100%) - ๅทฒๆณจ้‡Š
311
- - [ ] `led_effect` - LED ๆ•ˆๆžœ (off/solid/breathing/rainbow/doa) - ๅทฒๆณจ้‡Š
312
- - [ ] `led_color_r/g/b` - LED RGB ้ขœ่‰ฒ (0-255) - ๅทฒๆณจ้‡Š
313
-
314
- 12. **Phase 12 - ้Ÿณ้ข‘ๅค„็†ๅ‚ๆ•ฐ** โœ… **ๅทฒๅฎŒๆˆ**
315
- - [x] `agc_enabled` - ่‡ชๅŠจๅขž็›ŠๆŽงๅˆถๅผ€ๅ…ณ
316
- - [x] `agc_max_gain` - AGC ๆœ€ๅคงๅขž็›Š (0-30 dB)
317
- - [x] `noise_suppression` - ๅ™ชๅฃฐๆŠ‘ๅˆถ็บงๅˆซ (0-100%)
318
- - [x] `echo_cancellation_converged` - ๅ›žๅฃฐๆถˆ้™คๆ”ถๆ•›็Šถๆ€๏ผˆๅช่ฏป๏ผ‰
319
-
320
- 13. **Phase 13 - Sendspin ้Ÿณ้ข‘ๆ’ญๆ”พๆ”ฏๆŒ** โœ… **ๅทฒๅฎŒๆˆ**
321
- - [x] `sendspin_enabled` - Sendspin ๅผ€ๅ…ณ (Switch)
322
- - [x] `sendspin_url` - Sendspin ๆœๅŠกๅ™จ URL (Text Sensor)
323
- - [x] `sendspin_connected` - Sendspin ่ฟžๆŽฅ็Šถๆ€ (Binary Sensor)
324
- - [x] AudioPlayer ้›†ๆˆ aiosendspin ๅบ“
325
- - [x] TTS ้Ÿณ้ข‘ๅŒๆ—ถๅ‘้€ๅˆฐๆœฌๅœฐๆ‰ฌๅฃฐๅ™จๅ’Œ Sendspin ๆœๅŠกๅ™จ
326
-
327
- ---
328
-
329
- ## ๐ŸŽ‰ Phase 1-13 ๅฎžไฝ“ๅทฒๅฎŒๆˆ๏ผ
330
-
331
- **ๅทฒๅฎŒๆˆๆ€ป่ฎก๏ผš43 ไธชๅฎžไฝ“**
332
- - Phase 1: 4 ไธชๅฎžไฝ“ (ๅŸบ็ก€็Šถๆ€ไธŽ้Ÿณ้‡)
333
- - Phase 2: 4 ไธชๅฎžไฝ“ (็”ตๆœบๆŽงๅˆถ)
334
- - Phase 3: 9 ไธชๅฎžไฝ“ (ๅงฟๆ€ๆŽงๅˆถ)
335
- - Phase 4: 3 ไธชๅฎžไฝ“ (ๆณจ่ง†ๆŽงๅˆถ)
336
- - Phase 5: 2 ไธชๅฎžไฝ“ (้Ÿณ้ข‘ไผ ๆ„Ÿๅ™จ)
337
- - Phase 6: 6 ไธชๅฎžไฝ“ (่ฏŠๆ–ญไฟกๆฏ)
338
- - Phase 7: 7 ไธชๅฎžไฝ“ (IMU ไผ ๆ„Ÿๅ™จ)
339
- - Phase 8: 1 ไธชๅฎžไฝ“ (่กจๆƒ…ๆŽงๅˆถ)
340
- - Phase 9: 1 ไธชๅฎžไฝ“ (้บฆๅ…‹้ฃŽ้Ÿณ้‡)
341
- - Phase 10: 1 ไธชๅฎžไฝ“ (ๆ‘„ๅƒๅคด)
342
- - Phase 11: 0 ไธชๅฎžไฝ“ (LED ๆŽงๅˆถ - ๅทฒ็ฆ็”จ)
343
- - Phase 12: 4 ไธชๅฎžไฝ“ (้Ÿณ้ข‘ๅค„็†ๅ‚ๆ•ฐ)
344
- - Phase 13: 3 ไธชๅฎžไฝ“ (Sendspin ้Ÿณ้ข‘่พ“ๅ‡บ)
345
-
346
- ---
347
-
348
- ## ๐Ÿš€ ่ฏญ้ŸณๅŠฉๆ‰‹ๅขžๅผบๅŠŸ่ƒฝๅฎž็Žฐ็Šถๆ€
349
-
350
- ### Phase 14 - ๆƒ…ๆ„ŸๅŠจไฝœๅ้ฆˆ็ณป็ปŸ (้ƒจๅˆ†ๅฎž็Žฐ) ๐ŸŸก
351
-
352
- **ๅฎž็Žฐ็Šถๆ€**: ๅŸบ็ก€ๆžถๆž„ๅทฒๅฐฑ็ปช,ๆ”ฏๆŒๆ‰‹ๅŠจ่งฆๅ‘,ๅฏน่ฏๆ—ถไฝฟ็”จ่ฏญ้Ÿณ้ฉฑๅŠจ็š„่‡ช็„ถๅพฎๅŠจ
353
-
354
- **ๅทฒๅฎž็ŽฐๅŠŸ่ƒฝ**:
355
- - โœ… Phase 8 Emotion Selector ๅฎžไฝ“ (`emotion`)
356
- - โœ… ๅŸบ็ก€ๆƒ…ๆ„ŸๅŠจไฝœๆ’ญๆ”พAPI (`_play_emotion`)
357
- - โœ… ๆƒ…ๆ„Ÿๆ˜ ๅฐ„: Happy/Sad/Angry/Fear/Surprise/Disgust
358
- - โœ… ไธŽ HuggingFace ๅŠจไฝœๅบ“้›†ๆˆ (`pollen-robotics/reachy-mini-emotions-library`)
359
- - โœ… ๅฏน่ฏๆ—ถไฝฟ็”จ SpeechSway ็ณป็ปŸๆไพ›่‡ช็„ถ็š„ๅคด้ƒจๅพฎๅŠจ (ไธ้˜ปๅกžๅฏน่ฏไฝ“้ชŒ)
360
-
361
- **่ฎพ่ฎกๅ†ณ็ญ–**:
362
- - ๐ŸŽฏ ๅฏน่ฏๆ—ถไธ่‡ชๅŠจๆ’ญๆ”พๅฎŒๆ•ดๆƒ…ๆ„ŸๅŠจไฝœ,้ฟๅ…้˜ปๅกžๅฏน่ฏไฝ“้ชŒ
363
- - ๐ŸŽฏ ไฝฟ็”จ่ฏญ้Ÿณ้ฉฑๅŠจ็š„ๅคด้ƒจๆ‘†ๅŠจ (SpeechSway) ๆไพ›่‡ช็„ถ็š„ๅŠจไฝœๅ้ฆˆ
364
- - ๐ŸŽฏ ๆƒ…ๆ„ŸๅŠจไฝœไฟ็•™ไธบๆ‰‹ๅŠจ่งฆๅ‘ๅŠŸ่ƒฝ,ๅฏ้€š่ฟ‡ ESPHome ๅฎžไฝ“ๆŽงๅˆถ
365
-
366
- **ๆœชๅฎž็ŽฐๅŠŸ่ƒฝ**:
367
- - โŒ ่‡ชๅŠจๆ นๆฎ่ฏญ้ŸณๅŠฉๆ‰‹ๅ“ๅบ”่งฆๅ‘ๆƒ…ๆ„ŸๅŠจไฝœ (ๅทฒๅ†ณๅฎšไธๅฎž็Žฐ,้ฟๅ…้˜ปๅกž)
368
- - โŒ ๆ„ๅ›พ่ฏ†ๅˆซไธŽๆƒ…ๆ„ŸๅŒน้…
369
- - โŒ ่ˆž่นˆๅŠจไฝœๅบ“้›†ๆˆ
370
- - โŒ ไธŠไธ‹ๆ–‡ๆ„Ÿ็Ÿฅ(ๅฆ‚ๅคฉๆฐ”ๆŸฅ่ฏข-ๆ™ดๅคฉๆ’ญๆ”พ happy,้›จๅคฉๆ’ญๆ”พ sad)
371
-
372
- **ไปฃ็ ไฝ็ฝฎ**:
373
- - `entity_registry.py:633-658` - Emotion Selector ๅฎžไฝ“
374
- - `satellite.py:544-574` - `_play_emotion()` ๆ–นๆณ•
375
- - `motion.py:132-156` - ๅฏน่ฏๅผ€ๅง‹ๆ—ถ็š„ๅŠจไฝœๆŽงๅˆถ (ไฝฟ็”จ SpeechSway)
376
- - `movement_manager.py:541-595` - Move ้˜Ÿๅˆ—็ฎก็† (ๅ…่ฎธ SpeechSway ๅ ๅŠ )
377
-
378
- **ๅฎž้™…่กŒไธบ**:
379
-
380
- | ่ฏญ้ŸณๅŠฉๆ‰‹ไบ‹ไปถ | ๅฎž้™…ๅŠจไฝœ | ๅฎž็Žฐ็Šถๆ€ |
381
- |-------------|---------|---------|
382
- | ๅ”ค้†’่ฏๆฃ€ๆต‹ | ่ฝฌๅ‘ๅฃฐๆบ + ็‚นๅคด็กฎ่ฎค | โœ… ๅทฒๅฎž็Žฐ |
383
- | ๅฏน่ฏๅผ€ๅง‹ | ่ฏญ้Ÿณ้ฉฑๅŠจ็š„ๅคด้ƒจๅพฎๅŠจ (SpeechSway) | โœ… ๅทฒๅฎž็Žฐ |
384
- | ๅฏน่ฏ่ฟ›่กŒไธญ | ๆŒ็ปญ็š„่ฏญ้Ÿณ้ฉฑๅŠจๅพฎๅŠจ + ๅ‘ผๅธๅŠจ็”ป | โœ… ๅทฒๅฎž็Žฐ |
385
- | ๅฏน่ฏ็ป“ๆŸ | ่ฟ”ๅ›žไธญ็ซ‹ไฝ็ฝฎ + ๅ‘ผๅธๅŠจ็”ป | โœ… ๅทฒๅฎž็Žฐ |
386
- | ๆ‰‹ๅŠจ่งฆๅ‘ๆƒ…ๆ„Ÿ | ้€š่ฟ‡ ESPHome `emotion` ๅฎžไฝ“ๆ’ญๆ”พ | โœ… ๅทฒๅฎž็Žฐ |
387
-
388
- **ๆŠ€ๆœฏ่ฏดๆ˜Ž**:
389
- ```python
390
- # motion.py - ๅฏน่ฏๆ—ถไฝฟ็”จ SpeechSway ่€Œ้žๅฎŒๆ•ดๆƒ…ๆ„ŸๅŠจไฝœ
391
- def on_speaking_start(self):
392
- self._is_speaking = True
393
- self._movement_manager.set_state(RobotState.SPEAKING)
394
- # SpeechSway ไผš่‡ชๅŠจๆ นๆฎ้Ÿณ้ข‘ๅ“ๅบฆไบง็”Ÿ่‡ช็„ถ็š„ๅคด้ƒจๅพฎๅŠจ
395
- # ไธๆ’ญๆ”พๅฎŒๆ•ดๆƒ…ๆ„ŸๅŠจไฝœ,้ฟๅ…้˜ปๅกžๅฏน่ฏไฝ“้ชŒ
396
-
397
- # movement_manager.py - ๅŠจไฝœๅˆ†ๅฑ‚็ณป็ปŸ
398
- # 1. Move ้˜Ÿๅˆ— (ๆƒ…ๆ„ŸๅŠจไฝœ) - ่ฎพ็ฝฎๅŸบ็ก€ๅงฟๆ€
399
- # 2. Action (็‚นๅคด/ๆ‘‡ๅคด็ญ‰) - ๅ ๅŠ ๅœจๅŸบ็ก€ๅงฟๆ€ไธŠ
400
- # 3. SpeechSway - ่ฏญ้Ÿณ้ฉฑๅŠจๅพฎๅŠจ,ๅฏไธŽ Move ๅ…ฑๅญ˜
401
- # 4. Breathing - ็ฉบ้—ฒๆ—ถ็š„ๅ‘ผๅธๅŠจ็”ป
402
- ```
403
-
404
- **ๅŽŸๅง‹่ง„ๅˆ’** (ๅทฒๅ†ณๅฎšไธๅฎž็Žฐ,้ฟๅ…้˜ปๅกžๅฏน่ฏ):
405
-
406
- | ่ฏญ้ŸณๅŠฉๆ‰‹ไบ‹ไปถ | ๅŽŸ่ฎกๅˆ’ๅŠจไฝœ | ไธๅฎž็ŽฐๅŽŸๅ›  |
407
- |-------------|---------|---------|
408
- | ๆ”ถๅˆฐ่‚ฏๅฎšๅ›žๅค | ๆ’ญๆ”พ "happy" ๅŠจไฝœ | ๅฎŒๆ•ดๅŠจไฝœไผš้˜ปๅกžๅฏน่ฏๆต็•…ๆ€ง |
409
- | ๆ”ถๅˆฐๅฆๅฎšๅ›žๅค | ๆ’ญๆ”พ "sad" ๅŠจไฝœ | ๅฎŒๆ•ดๅŠจไฝœไผš้˜ปๅกžๅฏน่ฏๆต็•…ๆ€ง |
410
- | ๆ’ญๆ”พ้Ÿณไน/ๅจฑไน | ๆ’ญๆ”พ "dance" ๅŠจไฝœ | ๅฎŒๆ•ดๅŠจไฝœไผš้˜ปๅกžๅฏน่ฏๆต็•…ๆ€ง |
411
- | ๅฎšๆ—ถๅ™จๅฎŒๆˆ | ๆ’ญๆ”พ "alert" ๅŠจไฝœ | ๅฎŒๆ•ดๅŠจไฝœไผš้˜ปๅกžๅฏน่ฏๆต็•…ๆ€ง |
412
- | ้”™่ฏฏ/ๆ— ๆณ•็†่งฃ | ๆ’ญๆ”พ "confused" ๅŠจไฝœ | ๅฎŒๆ•ดๅŠจไฝœไผš้˜ปๅกžๅฏน่ฏๆต็•…ๆ€ง |
413
-
414
- **ๆ‰‹ๅŠจ่งฆๅ‘ๆƒ…ๆ„ŸๅŠจไฝœ็คบไพ‹**:
415
- ```yaml
416
- # Home Assistant ่‡ชๅŠจๅŒ–็คบไพ‹ - ๆ‰‹ๅŠจ่งฆๅ‘ๆƒ…ๆ„Ÿ
417
- automation:
418
- - alias: "Reachy ๆ—ฉๅฎ‰้—ฎๅ€™"
419
- trigger:
420
- - platform: time
421
- at: "07:00:00"
422
- action:
423
- - service: select.select_option
424
- target:
425
- entity_id: select.reachy_mini_emotion
426
- data:
427
- option: "Happy"
428
- ```
429
-
430
- ### Phase 15 - ไบบ่„ธ่ฟฝ่ธช๏ผˆๆ›ฟไปฃ DOA ๅฃฐๆบ่ฟฝ่ธช๏ผ‰โœ… **ๅทฒๅฎŒๆˆ**
431
-
432
- **็›ฎๆ ‡**: ๅฎž็Žฐ่‡ช็„ถ็š„ไบบ่„ธ่ฟฝ่ธช๏ผŒ่ฎฉๆœบๅ™จไบบๅœจๅฏน่ฏๆ—ถๆณจ่ง†่ฏด่ฏไบบใ€‚
433
-
434
- **่ฎพ่ฎกๅ†ณ็ญ–**:
435
- - โŒ ๅŽŸ่ฎกๅˆ’ไฝฟ็”จ DOA (Direction of Arrival) ๅฃฐๆบ่ฟฝ่ธช
436
- - โœ… ๆ”น็”จ YOLO ไบบ่„ธๆฃ€ๆต‹๏ผŒๆ›ด็จณๅฎšใ€ๆ›ดๅ‡†็กฎ
437
- - ๅŽŸๅ› ๏ผšDOA ๅœจๅ”ค้†’ๆ—ถๅˆคๆ–ญไธๅคŸๅ‡†็กฎ๏ผŒไธ”้ข‘็นๆŸฅ่ฏขไผšๅฏผ่‡ด daemon ๅดฉๆบƒ
438
-
439
- **ๅทฒๅฎž็ŽฐๅŠŸ่ƒฝ**:
440
-
441
- | ๅŠŸ่ƒฝ | ่ฏดๆ˜Ž | ๅฎž็Žฐไฝ็ฝฎ | ๅฎž็Žฐ็Šถๆ€ |
442
- |------|------|---------|---------|
443
- | YOLO ไบบ่„ธๆฃ€ๆต‹ | ไฝฟ็”จ `AdamCodd/YOLOv11n-face-detection` ๆจกๅž‹ | `head_tracker.py` | โœ… ๅทฒๅฎž็Žฐ |
444
- | ่‡ช้€‚ๅบ”ๅธง็އ่ฟฝ่ธช | ๅฏน่ฏๆ—ถ15fps๏ผŒ็ฉบ้—ฒๆ— ไบบ่„ธๆ—ถ3fps | `camera_server.py` | โœ… ๅทฒๅฎž็Žฐ |
445
- | look_at_image() | ๆ นๆฎไบบ่„ธไฝ็ฝฎ่ฎก็ฎ—็›ฎๆ ‡ๅงฟๆ€ | `camera_server.py` | โœ… ๅทฒๅฎž็Žฐ |
446
- | ๅนณๆป‘ๅ›žไธญๆ€งไฝ็ฝฎ | ไบบ่„ธไธขๅคฑๅŽ 1 ็ง’ๅ†…ๅนณๆป‘ๅ›žๅฝ’ | `camera_server.py` | โœ… ๅทฒๅฎž็Žฐ |
447
- | face_tracking_offsets | ไฝœไธบ secondary pose ๅ ๅŠ ๅˆฐ่ฟๅŠจๆŽงๅˆถ | `movement_manager.py` | โœ… ๅทฒๅฎž็Žฐ |
448
- | ่ฏญ้ŸณๆดปๅŠจๆฃ€ๆต‹ | DOA ๅฎžไฝ“ไปๅฏ็”จไบŽๆฃ€ๆต‹่ฏญ้Ÿณ | `DoAInfo.speech_detected` | โœ… ๅทฒๆšด้œฒไธบๅฎžไฝ“ |
449
- | ๆจกๅž‹ไธ‹่ฝฝ้‡่ฏ• | 3ๆฌก้‡่ฏ•๏ผŒ5็ง’้—ด้š” | `head_tracker.py` | โœ… ๅทฒๅฎž็Žฐ |
450
- | ๅฏน่ฏๆจกๅผ่”ๅŠจ | ่ฏญ้ŸณๅŠฉๆ‰‹็Šถๆ€ๅ˜ๅŒ–ๆ—ถ่‡ชๅŠจๅˆ‡ๆข่ฟฝ่ธช้ข‘็އ | `satellite.py` | โœ… ๅทฒๅฎž็Žฐ |
451
-
452
- **่ต„ๆบไผ˜ๅŒ– (v0.5.1)**:
453
- - ๅฏน่ฏๆœŸ้—ด๏ผˆlistening/thinking/speaking๏ผ‰๏ผš้ซ˜้ข‘่ฟฝ่ธช 15fps
454
- - ็ฉบ้—ฒไธ”ๆฃ€ๆต‹ๅˆฐไบบ่„ธ๏ผš้ซ˜้ข‘่ฟฝ่ธช 15fps
455
- - ็ฉบ้—ฒไธ”10็ง’ๆ— ไบบ่„ธ๏ผšไฝŽๅŠŸ่€—ๆจกๅผ 3fps๏ผˆไป…ๆฃ€ๆต‹ๆ˜ฏๅฆๆœ‰ไบบๅ‡บ็Žฐ๏ผ‰
456
- - ๆฃ€ๆต‹ๅˆฐไบบ่„ธๅŽ็ซ‹ๅณๆขๅค้ซ˜้ข‘่ฟฝ่ธช
457
-
458
- **ไปฃ็ ไฝ็ฝฎ**:
459
- - `head_tracker.py` - YOLO ไบบ่„ธๆฃ€ๆต‹ๅ™จ (`HeadTracker` ็ฑป)
460
- - `camera_server.py:_capture_frames()` - ่‡ช้€‚ๅบ”ๅธง็އไบบ่„ธ่ฟฝ่ธช
461
- - `camera_server.py:set_conversation_mode()` - ๅฏน่ฏๆจกๅผๅˆ‡ๆข API
462
- - `satellite.py:_set_conversation_mode()` - ่ฏญ้ŸณๅŠฉๆ‰‹็Šถๆ€่”ๅŠจ
463
- - `movement_manager.py:set_face_tracking_offsets()` - ไบบ่„ธ่ฟฝ่ธชๅ็งป้‡ API
464
-
465
- **ๆŠ€ๆœฏ็ป†่Š‚**:
466
- ```python
467
- # camera_server.py - ่‡ช้€‚ๅบ”ๅธง็އไบบ่„ธ่ฟฝ่ธช
468
- class MJPEGCameraServer:
469
- def __init__(self):
470
- self._fps_high = 15 # ๅฏน่ฏ/ๆœ‰ไบบ่„ธๆ—ถ
471
- self._fps_low = 3 # ็ฉบ้—ฒๆ— ไบบ่„ธๆ—ถ
472
- self._low_power_threshold = 10.0 # 10็ง’ๆ— ไบบ่„ธๅˆ‡ๆขไฝŽๅŠŸ่€—
473
-
474
- def _should_run_face_tracking(self, current_time):
475
- # ๅฏน่ฏๆจกๅผ๏ผšๅง‹็ปˆ้ซ˜้ข‘่ฟฝ่ธช
476
- if self._in_conversation:
477
- return True
478
- # ้ซ˜้ข‘ๆจกๅผ๏ผšๆฏๅธง่ฟฝ่ธช
479
- if self._current_fps == self._fps_high:
480
- return True
481
- # ไฝŽๅŠŸ่€—ๆจกๅผ๏ผšๅ‘จๆœŸๆ€งๆฃ€ๆต‹
482
- return time.since_last_check >= 1/self._fps_low
483
-
484
- # satellite.py - ่ฏญ้ŸณๅŠฉๆ‰‹็Šถๆ€่”ๅŠจ
485
- def _reachy_on_listening(self):
486
- self._set_conversation_mode(True) # ๅผ€ๅง‹ๅฏน่ฏ๏ผŒ้ซ˜้ข‘่ฟฝ่ธช
487
-
488
- def _reachy_on_idle(self):
489
- self._set_conversation_mode(False) # ็ป“ๆŸๅฏน่ฏ๏ผŒ่‡ช้€‚ๅบ”่ฟฝ่ธช
490
- ```
491
-
492
- ### Phase 16 - ๅก้€š้ฃŽๆ ผ่ฟๅŠจๆจกๅผ (้ƒจๅˆ†ๅฎž็Žฐ) ๐ŸŸก
493
-
494
- **็›ฎๆ ‡**: ไฝฟ็”จ SDK ็š„ๆ’ๅ€ผๆŠ€ๆœฏ่ฎฉๆœบๅ™จไบบๅŠจไฝœๆ›ดๆœ‰ไธชๆ€งๅ’Œ่กจ็ŽฐๅŠ›ใ€‚
495
-
496
- **SDK ๆ”ฏๆŒ**: `InterpolationTechnique` ๆžšไธพ
497
- - `LINEAR` - ็บฟๆ€ง๏ผŒๆœบๆขฐๆ„Ÿ
498
- - `MIN_JERK` - ๆœ€ๅฐๅŠ ๅŠ ้€Ÿๅบฆ๏ผŒ่‡ช็„ถๅนณๆป‘๏ผˆ้ป˜่ฎค๏ผ‰
499
- - `EASE_IN_OUT` - ็ผ“ๅ…ฅ็ผ“ๅ‡บ๏ผŒไผ˜้›…
500
- - `CARTOON` - ๅก้€š้ฃŽๆ ผ๏ผŒๅธฆๅ›žๅผนๆ•ˆๆžœ๏ผŒๆดปๆณผๅฏ็ˆฑ
501
-
502
- **ๅทฒๅฎž็ŽฐๅŠŸ่ƒฝ**:
503
- - โœ… 20Hz ็ปŸไธ€ๆŽงๅˆถๅพช็Žฏ (`movement_manager.py`) - ไปŽ 100Hz ้™ไฝŽไปฅ้˜ฒๆญข daemon ๅดฉๆบƒ
504
- - โœ… ๅงฟๆ€ๅ˜ๅŒ–ๆฃ€ๆต‹ - ไป…ๅœจๅงฟๆ€ๆ˜พ่‘—ๅ˜ๅŒ–ๆ—ถๅ‘้€ๅ‘ฝไปค (้˜ˆๅ€ผ 0.001)
505
- - โœ… ็Šถๆ€ๆŸฅ่ฏข็ผ“ๅญ˜ - 100ms TTL๏ผŒๅ‡ๅฐ‘ daemon ่ดŸ่ฝฝ
506
- - โœ… ๅนณๆป‘ๆ’ๅ€ผๅŠจไฝœ (ease in-out ๆ›ฒ็บฟ)
507
- - โœ… ๅ‘ผๅธๅŠจ็”ป - ็ฉบ้—ฒๆ—ถ Z ่ฝดๅพฎๅŠจ + ๅคฉ็บฟๆ‘†ๅŠจ (`BreathingAnimation`)
508
- - โœ… ๅ‘ฝไปค้˜Ÿๅˆ—ๆจกๅผ - ็บฟ็จ‹ๅฎ‰ๅ…จ็š„ๅค–้ƒจ API
509
- - โœ… ้”™่ฏฏ่Š‚ๆต - ้˜ฒๆญขๆ—ฅๅฟ—็ˆ†็‚ธ
510
- - โœ… ่ฟžๆŽฅๅฅๅบท็›‘ๆŽง - ่‡ชๅŠจๆฃ€ๆต‹ๅ’Œๆขๅค่ฟžๆŽฅไธขๅคฑ
511
-
512
- **ๆœชๅฎž็ŽฐๅŠŸ่ƒฝ**:
513
- - โŒ ๅŠจๆ€ๆ’ๅ€ผๆŠ€ๆœฏๅˆ‡ๆข (CARTOON/EASE_IN_OUT ็ญ‰)
514
- - โŒ ๅคธๅผ ็š„ๅก้€šๅ›žๅผนๆ•ˆๆžœ
515
-
516
- **ไปฃ็ ไฝ็ฝฎ**:
517
- - `movement_manager.py:192-243` - BreathingAnimation ็ฑป
518
- - `movement_manager.py:246-697` - MovementManager ็ฑป
519
-
520
- **ๅœบๆ™ฏๅฎž็Žฐ็Šถๆ€**:
521
-
522
- | ๅœบๆ™ฏ | ๆŽจ่ๆ’ๅ€ผ | ๆ•ˆๆžœ | ๅฎž็Žฐ็Šถๆ€ |
523
- |------|---------|------|---------|
524
- | ๅ”ค้†’็‚นๅคด | `CARTOON` | ๆดปๆณผ็š„ๅ›žๅผนๆ•ˆๆžœ | โŒ ๆœชๅฎž็Žฐ |
525
- | ๆ€่€ƒๆŠฌๅคด | `EASE_IN_OUT` | ไผ˜้›…็š„่ฟ‡ๆธก | โœ… ๅทฒๅฎž็Žฐ (ๅนณๆป‘ๆ’ๅ€ผ) |
526
- | ่ฏด่ฏๆ—ถๅพฎๅŠจ | `MIN_JERK` | ่‡ช็„ถๆต็•… | โœ… ๅทฒๅฎž็Žฐ (SpeechSway) |
527
- | ้”™่ฏฏๆ‘‡ๅคด | `CARTOON` | ๅคธๅผ ็š„ๅฆๅฎš | โŒ ๆœชๅฎž็Žฐ |
528
- | ่ฟ”ๅ›žไธญ็ซ‹ | `MIN_JERK` | ๅนณๆป‘ๅฝ’ไฝ | โœ… ๅทฒๅฎž็Žฐ |
529
- | ็ฉบ้—ฒๅ‘ผๅธ | - | ๅพฎๅฆ™็š„็”Ÿๅ‘ฝๆ„Ÿ | โœ… ๅทฒๅฎž็Žฐ (BreathingAnimation) |
530
-
531
- ### Phase 17 - ่ฏด่ฏๆ—ถๅคฉ็บฟๅŒๆญฅๅŠจ็”ป (้ƒจๅˆ†ๅฎž็Žฐ) ๐ŸŸก
532
-
533
- **็›ฎๆ ‡**: TTS ๆ’ญๆ”พๆ—ถ๏ผŒๅคฉ็บฟ้š้Ÿณ้ข‘่Š‚ๅฅๆ‘†ๅŠจ๏ผŒๆจกๆ‹Ÿ"่ฏด่ฏ"ๆ•ˆๆžœใ€‚
534
-
535
- **ๅทฒๅฎž็ŽฐๅŠŸ่ƒฝ**:
536
- - โœ… ่ฏญ้Ÿณ้ฉฑๅŠจๅคด้ƒจๆ‘†ๅŠจ (`SpeechSwayGenerator`)
537
- - โœ… ๅŸบไบŽ้Ÿณ้ข‘ๅ“ๅบฆ็š„ VAD ๆฃ€ๆต‹
538
- - โœ… ๅคš้ข‘็އๆญฃๅผฆๆณขๅ ๅŠ  (Lissajous ่ฟๅŠจ)
539
- - โœ… ๅนณๆป‘ๅŒ…็ปœ่ฟ‡ๆธก
540
-
541
- **ไปฃ็ ไฝ็ฝฎ**:
542
- - `movement_manager.py:124-189` - SpeechSwayGenerator ็ฑป
543
- - `motion.py:212-222` - update_audio_loudness() ๆ–นๆณ•
544
-
545
- **ๆŠ€ๆœฏ็ป†่Š‚**:
546
- ```python
547
- # ่ฏญ้Ÿณๆ‘†ๅŠจๅ‚ๆ•ฐ
548
- SWAY_A_PITCH_DEG = 3.0 # ไฟฏไปฐๅน…ๅบฆ (ๅบฆ)
549
- SWAY_A_YAW_DEG = 2.0 # ๅ่ˆชๅน…ๅบฆ
550
- SWAY_A_ROLL_DEG = 2.0 # ็ฟปๆปšๅน…ๅบฆ
551
- SWAY_F_PITCH = 0.8 # ไฟฏไปฐ้ข‘็އ Hz
552
- SWAY_F_YAW = 0.6 # ๅ่ˆช้ข‘็އ
553
- SWAY_F_ROLL = 0.5 # ็ฟปๆปš้ข‘็އ
554
-
555
- # VAD ้˜ˆๅ€ผ
556
- VAD_DB_ON = -35 # ๅผ€ๅง‹ๆฃ€ๆต‹้˜ˆๅ€ผ
557
- VAD_DB_OFF = -45 # ๅœๆญขๆฃ€ๆต‹้˜ˆๅ€ผ
558
- ```
559
-
560
- **ๆœชๅฎž็ŽฐๅŠŸ่ƒฝ**:
561
- - โŒ ๅคฉ็บฟ้š้Ÿณ้ข‘่Š‚ๅฅๆ‘†ๅŠจ (ๅฝ“ๅ‰ไป…ๅคด้ƒจๆ‘†ๅŠจ)
562
- - โŒ ้Ÿณ้ข‘้ข‘่ฐฑๅˆ†ๆž้ฉฑๅŠจๅŠจ็”ป
563
-
564
- ### Phase 18 - ่ง†่ง‰ๆณจ่ง†ไบคไบ’ (ๆœชๅฎž็Žฐ) โŒ
565
-
566
- **็›ฎๆ ‡**: ๅˆฉ็”จๆ‘„ๅƒๅคดๆฃ€ๆต‹ไบบ่„ธ๏ผŒๅฎž็Žฐ็œผ็ฅžไบคๆตใ€‚
567
-
568
- **SDK ๆ”ฏๆŒ**:
569
- - `look_at_image(u, v)` - ๆณจ่ง†ๅ›พๅƒไธญ็š„็‚น
570
- - `look_at_world(x, y, z)` - ๆณจ่ง†ไธ–็•Œๅๆ ‡็‚น
571
- - `media.get_frame()` - ่Žทๅ–ๆ‘„ๅƒๅคด็”ป้ข (โœ… ๅทฒๅœจ `camera_server.py:146` ๅฎž็Žฐ)
572
-
573
- **ๆœชๅฎž็ŽฐๅŠŸ่ƒฝ**:
574
-
575
- | ๅŠŸ่ƒฝ | ่ฏดๆ˜Ž | ๅฎž็Žฐ็Šถๆ€ |
576
- |------|------|---------|
577
- | ไบบ่„ธๆฃ€ๆต‹ | ไฝฟ็”จ OpenCV/MediaPipe ๆฃ€ๆต‹ไบบ่„ธ | โŒ ๆœชๅฎž็Žฐ |
578
- | ็œผ็ฅž่ฟฝ่ธช | ๅฏน่ฏๆ—ถๆณจ่ง†่ฏด่ฏไบบ็š„่„ธ | โŒ ๆœชๅฎž็Žฐ |
579
- | ๅคšไบบๅˆ‡ๆข | ๆฃ€ๆต‹ๅˆฐๅคšไบบๆ—ถ๏ผŒๆณจ่ง†ๅฝ“ๅ‰่ฏด่ฏไบบ | โŒ ๆœชๅฎž็Žฐ |
580
- | ็ฉบ้—ฒๆ‰ซ่ง† | ็ฉบ้—ฒๆ—ถ้šๆœบ็Žฏ้กพๅ››ๅ‘จ | โŒ ๆœชๅฎž็Žฐ |
581
-
582
- ### Phase 19 - ้‡ๅŠ›่กฅๅฟไบ’ๅŠจๆจกๅผ (้ƒจๅˆ†ๅฎž็Žฐ) ๐ŸŸก
583
-
584
- **็›ฎๆ ‡**: ๅ…่ฎธ็”จๆˆท็‰ฉ็†่งฆๆ‘ธๅ’Œๅผ•ๅฏผๆœบๅ™จไบบๅคด้ƒจ๏ผŒๅฎž็Žฐ"ๆ•™ๅญฆ"ๅผไบคไบ’ใ€‚
585
-
586
- **SDK ๆ”ฏๆŒ**: `enable_gravity_compensation()` - ็”ตๆœบ่ฟ›ๅ…ฅ้‡ๅŠ›่กฅๅฟๆจกๅผ๏ผŒๅฏๆ‰‹ๅŠจ็งปๅŠจ
587
-
588
- **ๅทฒๅฎž็ŽฐๅŠŸ่ƒฝ**:
589
- - โœ… ้‡ๅŠ›่กฅๅฟๆจกๅผๅˆ‡ๆข (`motor_mode` Select ๅฎžไฝ“๏ผŒ้€‰้กน "gravity_compensation")
590
- - โœ… `reachy_controller.py:236-237` - ้‡ๅŠ›่กฅๅฟ API ่ฐƒ็”จ
591
-
592
- **ๆœชๅฎž็ŽฐๅŠŸ่ƒฝ**:
593
- - โŒ ๆ•™ๅญฆๆจกๅผ - ๅฝ•ๅˆถๅŠจไฝœ่ฝจ่ฟน
594
- - โŒ ไฟๅญ˜/ๆ’ญๆ”พ่‡ชๅฎšไน‰ๅŠจไฝœ
595
- - โŒ ่ฏญ้Ÿณๅ‘ฝไปค่งฆๅ‘ๆ•™ๅญฆๆต็จ‹
596
-
597
- **ๅบ”็”จๅœบๆ™ฏ**:
598
- - โŒ ็”จๆˆท่ฏด "่ฎฉๆˆ‘ๆ•™ไฝ ไธ€ไธชๅŠจไฝœ" โ†’ ่ฟ›ๅ…ฅ้‡ๅŠ›่กฅๅฟๆจกๅผ
599
- - โŒ ็”จๆˆทๆ‰‹ๅŠจ็งปๅŠจๅคด้ƒจ โ†’ ๅฝ•ๅˆถๅŠจไฝœ่ฝจ่ฟน
600
- - โŒ ็”จๆˆท่ฏด "่ฎฐไฝ่ฟ™ไธช" โ†’ ไฟๅญ˜ๅŠจไฝœ
601
- - โŒ ็”จๆˆท่ฏด "ๅšๅˆšๆ‰็š„ๅŠจไฝœ" โ†’ ๆ’ญๆ”พๅฝ•ๅˆถ็š„ๅŠจไฝœ
602
-
603
- ### Phase 20 - ็Žฏๅขƒๆ„Ÿ็Ÿฅๅ“ๅบ” (้ƒจๅˆ†ๅฎž็Žฐ) ๐ŸŸก
604
-
605
- **็›ฎๆ ‡**: ๅˆฉ็”จ IMU ไผ ๆ„Ÿๅ™จๆ„Ÿ็Ÿฅ็Žฏๅขƒๅ˜ๅŒ–ๅนถๅšๅ‡บๅ“ๅบ”ใ€‚
606
-
607
- **SDK ๆ”ฏๆŒ**:
608
- - โœ… `mini.imu["accelerometer"]` - ๅŠ ้€Ÿๅบฆ่ฎก (Phase 7 ๅทฒๅฎž็Žฐไธบๅฎžไฝ“)
609
- - โœ… `mini.imu["gyroscope"]` - ้™€่žบไปช (Phase 7 ๅทฒๅฎž็Žฐไธบๅฎžไฝ“)
610
-
611
- **ๅทฒๅฎž็ŽฐๅŠŸ่ƒฝ**:
612
-
613
- | ๆฃ€ๆต‹ไบ‹ไปถ | ๅ“ๅบ”ๅŠจไฝœ | ๅฎž็Žฐ็Šถๆ€ |
614
- |---------|---------|---------|
615
- | ๆ‹ไธ€ๆ‹ๅ”ค้†’ | ่ฟ›ๅ…ฅๆŒ็ปญๅฏน่ฏๆจกๅผ | โœ… ๅทฒๅฎž็Žฐ |
616
- | ๅ†ๆฌกๆ‹ไธ€ๆ‹ | ้€€ๅ‡บๆŒ็ปญๅฏน่ฏๆจกๅผ | โœ… ๅทฒๅฎž็Žฐ |
617
-
618
- **ๆ‹ไธ€ๆ‹ๅ”ค้†’ vs ่ฏญ้Ÿณๅ”ค้†’**:
619
-
620
- | ๅ”ค้†’ๆ–นๅผ | ๅฏน่ฏๆจกๅผ | ่ฏดๆ˜Ž |
621
- |---------|---------|------|
622
- | ่ฏญ้Ÿณๅ”ค้†’ (Okay Nabu) | ๅ•ๆฌกๅฏน่ฏ | ๆฏๆฌกๅฏน่ฏ้œ€่ฆ้‡ๆ–ฐ่ฏดๅ”ค้†’่ฏ |
623
- | ๆ‹ไธ€ๆ‹ๅ”ค้†’ | ๆŒ็ปญๅฏน่ฏ | TTS ็ป“ๆŸๅŽ่‡ชๅŠจ็ปง็ปญ็›‘ๅฌ๏ผŒๅ†ๆ‹ไธ€ๆฌก้€€ๅ‡บ |
624
-
625
- **ๆŠ€ๆœฏๅฎž็Žฐ**:
626
- - `tap_detector.py` - IMU ๅŠ ้€Ÿๅบฆ็ชๅ˜ๆฃ€ๆต‹
627
- - `satellite.py:_tap_conversation_mode` - ๆŒ็ปญๅฏน่ฏๆจกๅผๆ ‡ๅฟ—
628
- - ้˜ˆๅ€ผ: 2.0g (ๅฏ้…็ฝฎ)
629
- - ๅ†ทๅดๆ—ถ้—ด: 1.0s (้˜ฒๆญข้‡ๅค่งฆๅ‘)
630
- - ไป…้™ๆ— ็บฟ็‰ˆๆœฌ (Wireless) ๅฏ็”จ
631
-
632
- ```python
633
- # satellite.py - ๆŒ็ปญๅฏน่ฏๆจกๅผ
634
- def wakeup_from_tap(self):
635
- if self._tap_conversation_mode:
636
- # ็ฌฌไบŒๆฌกๆ‹ - ้€€ๅ‡บๆŒ็ปญๅฏน่ฏ
637
- self._tap_conversation_mode = False
638
- self._reachy_on_idle()
639
- else:
640
- # ็ฌฌไธ€ๆฌกๆ‹ - ่ฟ›ๅ…ฅๆŒ็ปญๅฏน่ฏ
641
- self._tap_conversation_mode = True
642
- self.send_messages([VoiceAssistantRequest(start=True)])
643
-
644
- def _tts_finished(self):
645
- if self._tap_conversation_mode:
646
- # ๆŒ็ปญๅฏน่ฏๆจกๅผ๏ผš่‡ชๅŠจ็ปง็ปญ็›‘ๅฌ
647
- self.send_messages([VoiceAssistantRequest(start=True)])
648
- ```
649
-
650
- **ๆœชๅฎž็ŽฐๅŠŸ่ƒฝ**:
651
-
652
- | ๆฃ€ๆต‹ไบ‹ไปถ | ๅ“ๅบ”ๅŠจไฝœ | ๅฎž็Žฐ็Šถๆ€ |
653
- |---------|---------|---------|
654
- | ่ขซๆ‘‡ๆ™ƒ | ๆ’ญๆ”พๆ™•็œฉๅŠจไฝœ + ่ฏญ้Ÿณ "ๅˆซๆ™ƒๆˆ‘~" | โŒ ๆœชๅฎž็Žฐ |
655
- | ๅ€พๆ–œ/ๅ€’ไธ‹ | ๆ’ญๆ”พๆฑ‚ๅŠฉๅŠจไฝœ + ่ฏญ้Ÿณ "ๆˆ‘ๅ€’ไบ†๏ผŒๅธฎๅธฎๆˆ‘" | โŒ ๆœชๅฎž็Žฐ |
656
- | ้•ฟๆ—ถ้—ด้™ๆญข | ่ฟ›ๅ…ฅไผ‘็œ ๅŠจ็”ป | โŒ ๆœชๅฎž็Žฐ |
657
-
658
- ### Phase 21 - Home Assistant ๅœบๆ™ฏ่”ๅŠจ (ๆœชๅฎž็Žฐ) โŒ
659
-
660
- **็›ฎๆ ‡**: ๆ นๆฎ Home Assistant ็š„ๅœบๆ™ฏ/่‡ชๅŠจๅŒ–่งฆๅ‘ๆœบๅ™จไบบๅŠจไฝœใ€‚
661
-
662
- **ๅฎž็Žฐๆ–นๆกˆ**: ้€š่ฟ‡ ESPHome ๆœๅŠก่ฐƒ็”จ
663
-
664
- **ๆœชๅฎž็Žฐๅœบๆ™ฏ**:
665
-
666
- | HA ๅœบๆ™ฏ | ๆœบๅ™จไบบๅ“ๅบ” | ๅฎž็Žฐ็Šถๆ€ |
667
- |--------|-----------|---------|
668
- | ๆ—ฉๅฎ‰ๅœบๆ™ฏ | ๆ’ญๆ”พๅ”ค้†’ๅŠจไฝœ + "ๆ—ฉไธŠๅฅฝ!" | โŒ ๆœชๅฎž็Žฐ |
669
- | ๆ™šๅฎ‰ๅœบๆ™ฏ | ๆ’ญๆ”พ็ก็œ ๅŠจไฝœ + "ๆ™šๅฎ‰~" | โŒ ๆœชๅฎž็Žฐ |
670
- | ๆœ‰ไบบๅ›žๅฎถ | ่ฝฌๅ‘้—จๅฃ + ๆŒฅๆ‰‹ + "ๆฌข่ฟŽๅ›žๅฎถ!" | โŒ ๆœชๅฎž็Žฐ |
671
- | ้—จ้“ƒๅ“่ตท | ่ฝฌๅ‘้—จๅฃ + ่ญฆ่ง‰ๅŠจไฝœ | โŒ ๆœชๅฎž็Žฐ |
672
- | ๆ’ญๆ”พ้Ÿณไน | ้š้Ÿณไน่Š‚ๅฅๆ‘†ๅŠจ | โŒ ๆœชๅฎž็Žฐ |
673
-
674
- ---
675
-
676
- ## ๐Ÿ“Š ๅŠŸ๏ฟฝ๏ฟฝๅฎž็Žฐๆ€ป็ป“
677
 
678
- ### โœ… ๅทฒๅฎŒๆˆๅŠŸ่ƒฝ
679
-
680
- #### ๆ ธๅฟƒ่ฏญ้ŸณๅŠฉๆ‰‹ (Phase 1-12)
681
- - **45+ ESPHome ๅฎžไฝ“** - ๅ…จ้ƒจๅฎž็Žฐ
682
- - **ๅŸบ็ก€่ฏญ้Ÿณไบคไบ’** - ๅ”ค้†’่ฏๆฃ€ๆต‹ใ€STT/TTS ้›†ๆˆ
683
- - **่ฟๅŠจๅ้ฆˆ** - ็‚นๅคดใ€ๆ‘‡ๅคดใ€ๆณจ่ง†็ญ‰ๅŸบ็ก€ๅŠจไฝœ
684
- - **้Ÿณ้ข‘ๅค„็†** - AGCใ€ๅ™ชๅฃฐๆŠ‘ๅˆถใ€ๅ›žๅฃฐๆถˆ้™ค
685
- - **ๆ‘„ๅƒๅคดๆต** - MJPEG ๅฎžๆ—ถ้ข„่งˆ
686
-
687
- #### ้ƒจๅˆ†ๅฎž็ŽฐๅŠŸ่ƒฝ (Phase 14-21)
688
- - **Phase 14** - ๆƒ…ๆ„ŸๅŠจไฝœ API ๅŸบ็ก€่ฎพๆ–ฝ (ๆ‰‹ๅŠจ่งฆๅ‘ๅฏ็”จ)
689
- - **Phase 19** - ้‡ๅŠ›่กฅๅฟๆจกๅผๅˆ‡ๆข (ๆ•™ๅญฆๆต็จ‹ๆœชๅฎž็Žฐ)
690
-
691
- ### โŒ ๆœชๅฎž็ŽฐๅŠŸ่ƒฝ
692
-
693
- #### ้ซ˜ไผ˜ๅ…ˆ็บง
694
- - ~~**Phase 13** - Sendspin ้Ÿณ้ข‘ๆ’ญๆ”พๆ”ฏๆŒ~~ โœ… **ๅทฒๅฎŒๆˆ**
695
- - **Phase 14** - ่‡ชๅŠจๆƒ…ๆ„ŸๅŠจไฝœๅ้ฆˆ (้œ€ไธŽ่ฏญ้ŸณๅŠฉๆ‰‹ไบ‹ไปถๅ…ณ่”)
696
- - **Phase 15** - ๆŒ็ปญๅฃฐๆบ่ฟฝ่ธช (ไป…ๅ”ค้†’ๆ—ถ่ฝฌๅ‘)
697
-
698
- #### ไธญไผ˜ๅ…ˆ็บง
699
- - **Phase 16** - ๅก้€š้ฃŽๆ ผ่ฟๅŠจๆจกๅผ (้œ€ๅŠจๆ€ๆ’ๅ€ผๅˆ‡ๆข)
700
- - **Phase 17** - ๅคฉ็บฟๅŒๆญฅๅŠจ็”ป
701
- - **Phase 18** - ไบบ่„ธ่ฟฝ่ธชไธŽ็œผ็ฅžไบคไบ’
702
-
703
- #### ไฝŽไผ˜ๅ…ˆ็บง
704
- - **Phase 19** - ๆ•™ๅญฆๆจกๅผๅฝ•ๅˆถ/ๆ’ญๆ”พๅŠŸ่ƒฝ
705
- - **Phase 20** - IMU ็Žฏๅขƒๆ„Ÿ็Ÿฅๅ“ๅบ”
706
- - **Phase 21** - Home Assistant ๅœบๆ™ฏ่”ๅŠจ
707
-
708
- ---
709
-
710
- ## ๅŠŸ่ƒฝไผ˜ๅ…ˆ็บงๆ€ป็ป“ (ๆ›ดๆ–ฐ็‰ˆ)
711
-
712
- ### ้ซ˜ไผ˜ๅ…ˆ็บง (ๅทฒๅฎŒๆˆ โœ…)
713
- - โœ… **Phase 1-12**: ๅŸบ็ก€ ESPHome ๅฎžไฝ“ (45+ ไธช)
714
- - โœ… ๆ ธๅฟƒ่ฏญ้ŸณๅŠฉๆ‰‹ๅŠŸ่ƒฝ
715
- - โœ… ๅŸบ็ก€่ฟๅŠจๅ้ฆˆ (็‚นๅคดใ€ๆ‘‡ๅคดใ€ๆณจ่ง†)
716
-
717
- ### ้ซ˜ไผ˜ๅ…ˆ็บง (้ƒจๅˆ†ๅฎž็Žฐ ๐ŸŸก)
718
- - ๐ŸŸก **Phase 13**: ๆƒ…ๆ„ŸๅŠจไฝœๅ้ฆˆ็ณป็ปŸ
719
- - โœ… Emotion Selector ๅฎžไฝ“ไธŽ API ๅŸบ็ก€่ฎพๆ–ฝ
720
- - โŒ ่‡ชๅŠจๆ นๆฎ่ฏญ้ŸณๅŠฉๆ‰‹ๅ“ๅบ”่งฆๅ‘ๆƒ…ๆ„ŸๅŠจไฝœ
721
- - โŒ ๆ„ๅ›พ่ฏ†ๅˆซไธŽๆƒ…ๆ„ŸๅŒน้…
722
- - โŒ ่ˆž่นˆๅŠจไฝœๅบ“้›†ๆˆ
723
-
724
- ### ้ซ˜ไผ˜ๅ…ˆ็บง (ๆœชๅฎž็Žฐ โŒ)
725
- - โŒ **Phase 14**: ๆ™บ่ƒฝๅฃฐๆบ่ฟฝ่ธชๅขžๅผบ
726
- - โœ… ๅ”ค้†’ๆ—ถ่ฝฌๅ‘ๅฃฐๆบ
727
- - โŒ ๆŒ็ปญๅฃฐๆบ่ฟฝ่ธช
728
- - โŒ ๅคšไบบๅฏน่ฏๅˆ‡ๆข
729
- - โŒ ๅฃฐๆบๅฏ่ง†ๅŒ–
730
-
731
- ### ไธญไผ˜ๅ…ˆ็บง (้ƒจๅˆ†ๅฎž็Žฐ ๐ŸŸก)
732
- - ๐ŸŸก **Phase 15**: ๅก้€š้ฃŽๆ ผ่ฟๅŠจๆจกๅผ
733
- - โœ… 20Hz ็ปŸไธ€ๆŽงๅˆถๅพช็Žฏๆžถๆž„ (ไผ˜ๅŒ–ไปฅ้˜ฒๆญข daemon ๅดฉๆบƒ)
734
- - โœ… ๅงฟๆ€ๅ˜ๅŒ–ๆฃ€ๆต‹ + ็Šถๆ€ๆŸฅ่ฏข็ผ“ๅญ˜ (ๅ‡ๅฐ‘ daemon ่ดŸ่ฝฝ)
735
- - โœ… ๅนณๆป‘ๆ’ๅ€ผๅŠจไฝœ + ๅ‘ผๅธๅŠจ็”ป
736
- - โŒ ๅŠจๆ€ๆ’ๅ€ผๆŠ€ๆœฏๅˆ‡ๆข (CARTOON ็ญ‰)
737
- - ๐ŸŸก **Phase 16**: ่ฏด่ฏๆ—ถๅคฉ็บฟๅŒๆญฅ
738
- - โœ… ่ฏญ้Ÿณ้ฉฑๅŠจๅคด้ƒจๆ‘†ๅŠจ (SpeechSwayGenerator)
739
- - โŒ ๅคฉ็บฟ้š้Ÿณ้ข‘่Š‚ๅฅๆ‘†ๅŠจ
740
-
741
- ### ไธญไผ˜ๅ…ˆ็บง (ๆœชๅฎž็Žฐ โŒ)
742
- - โŒ **Phase 17**: ่ง†่ง‰ๆณจ่ง†ไบคไบ’ - ็œผ็ฅžไบคๆต
743
-
744
- ### ไฝŽไผ˜ๅ…ˆ็บง (้ƒจๅˆ†ๅฎž็Žฐ ๐ŸŸก)
745
- - ๐ŸŸก **Phase 18**: ้‡ๅŠ›่กฅๅฟไบ’ๅŠจๆจกๅผ
746
- - โœ… ้‡ๅŠ›่กฅๅฟๆจกๅผๅˆ‡ๆข
747
- - โŒ ๆ•™ๅญฆๅผไบคไบ’ (ๅฝ•ๅˆถ/ๆ’ญๆ”พๅŠŸ่ƒฝ)
748
-
749
- ### ไฝŽไผ˜ๅ…ˆ็บง (ๆœชๅฎž็Žฐ โŒ)
750
- - โŒ **Phase 19**: ็Žฏๅขƒๆ„Ÿ็Ÿฅๅ“ๅบ” - IMU ่งฆๅ‘ๅŠจไฝœ
751
- - โŒ **Phase 20**: Home Assistant ๅœบๆ™ฏ่”ๅŠจ - ๆ™บ่ƒฝๅฎถๅฑ…ๆ•ดๅˆ
752
 
753
  ---
754
 
755
- ## ๐Ÿ“ˆ ๅฎŒๆˆๅบฆ็ปŸ่ฎก
756
-
757
- | ้˜ถๆฎต | ็Šถๆ€ | ๅฎŒๆˆๅบฆ | ่ฏดๆ˜Ž |
758
- |------|------|--------|------|
759
- | Phase 1-12 | โœ… ๅฎŒๆˆ | 100% | 40 ไธช ESPHome ๅฎžไฝ“ๅทฒๅฎž็Žฐ๏ผˆPhase 11 LED ๅทฒ็ฆ็”จ๏ผ‰ |
760
- | Phase 13 | ๐ŸŸก ้ƒจๅˆ†ๅฎŒๆˆ | 30% | API ๅŸบ็ก€่ฎพๆ–ฝๅฐฑ็ปช,็ผบ่‡ชๅŠจ่งฆๅ‘ |
761
- | Phase 14 | โŒ ๆœชๅฎŒๆˆ | 20% | ไป…ๅฎž็Žฐๅ”ค้†’ๆ—ถ่ฝฌๅ‘ |
762
- | Phase 15 | ๐ŸŸก ้ƒจๅˆ†ๅฎŒๆˆ | 70% | 20HzๆŽงๅˆถๅพช็Žฏ+ๅงฟๆ€ๅ˜ๅŒ–ๆฃ€ๆต‹+็Šถๆ€็ผ“ๅญ˜+ๅ‘ผๅธๅŠจ็”ปๅทฒๅฎž็Žฐ |
763
- | Phase 16 | ๐ŸŸก ้ƒจๅˆ†ๅฎŒๆˆ | 50% | ่ฏญ้Ÿณ้ฉฑๅŠจๅคด้ƒจๆ‘†ๅŠจๅทฒๅฎž็Žฐ |
764
- | Phase 17 | โŒ ๆœชๅฎŒๆˆ | 10% | ๆ‘„ๅƒๅคดๅทฒๅฎž็Žฐ,็ผบไบบ่„ธๆฃ€ๆต‹ |
765
- | Phase 18 | ๐ŸŸก ้ƒจๅˆ†ๅฎŒๆˆ | 40% | ๆจกๅผๅˆ‡ๆขๅทฒๅฎž็Žฐ,็ผบๆ•™ๅญฆๆต็จ‹ |
766
- | Phase 19 | โŒ ๆœชๅฎŒๆˆ | 10% | IMU ๆ•ฐๆฎๅทฒๆšด้œฒ,็ผบ่งฆๅ‘้€ป่พ‘ |
767
- | Phase 20 | โŒ ๆœชๅฎŒๆˆ | 0% | ๅฎŒๅ…จๆœชๅฎž็Žฐ |
768
-
769
- **ๆ€ปไฝ“ๅฎŒๆˆๅบฆ**: **Phase 1-12: 100%** | **Phase 13-20: ~35%**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
770
 
771
  ---
772
 
773
- ## ๐Ÿ”ง Daemon ๅดฉๆบƒ้—ฎ้ข˜ไฟฎๅค (2025-01-05)
774
-
775
- ### ้—ฎ้ข˜ๆ่ฟฐ
776
- ้•ฟๆœŸ่ฟ่กŒ่ฟ‡็จ‹ไธญ๏ผŒ`reachy_mini daemon` ไผšๅดฉๆบƒ๏ผŒๅฏผ่‡ดๆœบๅ™จไบบๅคฑๅŽปๅ“ๅบ”ใ€‚
777
-
778
- ### ๆ นๆœฌๅŽŸๅ› 
779
- 1. **100Hz ๆŽงๅˆถๅพช็Žฏ่ฟ‡ไบŽ้ข‘็น** - ๆฏ 10ms ่ฐƒ็”จไธ€ๆฌก `robot.set_target()`๏ผŒๅณไฝฟๅงฟๆ€ๆฒกๆœ‰ๅ˜ๅŒ–
780
- 2. **้ข‘็น็š„็Šถๆ€ๆŸฅ่ฏข** - ๆฏๆฌก่ฏปๅ–ๅฎžไฝ“็Šถๆ€้ƒฝ่ฐƒ็”จ `get_status()`ใ€`get_current_head_pose()` ็ญ‰
781
- 3. **็ผบๅฐ‘ๅ˜ๅŒ–ๆฃ€ๆต‹** - ๅณไฝฟๅงฟๆ€ๆฒกๆœ‰ๅ˜ๅŒ–๏ผŒไนŸไผšๆŒ็ปญๅ‘้€็›ธๅŒ็š„ๅ‘ฝไปค
782
- 4. **Zenoh ๆถˆๆฏ้˜Ÿๅˆ—ๅ ตๅกž** - ็ดฏ็งฏ่ตทๆฅๅฏ่ƒฝๆฏ็ง’ 150+ ๆกๆถˆๆฏ๏ผŒdaemon ๆ— ๆณ•ๅŠๆ—ถๅค„็†
783
-
784
- ### ไฟฎๅคๆ–นๆกˆ
785
 
786
- #### 1. ้™ไฝŽๆŽงๅˆถๅพช็Žฏ้ข‘็އ (movement_manager.py)
787
- ```python
788
- # ไปŽ 100Hz ้™ไฝŽๅˆฐ 20Hz
789
- CONTROL_LOOP_FREQUENCY_HZ = 20 # ๅ‡ๅฐ‘ 80% ็š„ๆถˆๆฏ้‡
790
- ```
791
-
792
- #### 2. ๆทปๅŠ ๅงฟๆ€ๅ˜ๅŒ–ๆฃ€ๆต‹ (movement_manager.py)
793
- ```python
794
- # ไป…ๅœจๅงฟๆ€ๆ˜พ่‘—ๅ˜ๅŒ–ๆ—ถๅ‘้€ๅ‘ฝไปค
795
- if self._last_sent_pose is not None:
796
- max_diff = max(abs(pose[k] - self._last_sent_pose.get(k, 0.0)) for k in pose.keys())
797
- if max_diff < 0.001: # ้˜ˆๅ€ผ: 0.001 rad ๆˆ– 0.001 m
798
- return # ่ทณ่ฟ‡ๅ‘้€
799
- ```
800
 
801
- #### 3. ็Šถๆ€ๆŸฅ่ฏข็ผ“ๅญ˜ (reachy_controller.py)
802
- ```python
803
- # ็ผ“ๅญ˜ daemon ็Šถๆ€ๆŸฅ่ฏข็ป“ๆžœ
804
- self._cache_ttl = 0.1 # 100ms TTL
805
- self._last_status_query = 0.0
806
-
807
- def _get_cached_status(self):
808
- now = time.time()
809
- if now - self._last_status_query < self._cache_ttl:
810
- return self._state_cache.get('status') # ไฝฟ็”จ็ผ“ๅญ˜
811
- # ... ๆŸฅ่ฏขๅนถๆ›ดๆ–ฐ็ผ“ๅญ˜
812
- ```
813
 
814
- #### 4. ๅคด้ƒจๅงฟๆ€ๆŸฅ่ฏข็ผ“ๅญ˜ (reachy_controller.py)
815
- ```python
816
- # ็ผ“ๅญ˜ get_current_head_pose() ๅ’Œ get_current_joint_positions() ็ป“ๆžœ
817
- def _get_cached_head_pose(self):
818
- # 100ms ๅ†…ๅค็”จ็ผ“ๅญ˜็ป“ๆžœ
819
- ```
820
 
821
- ### ไฟฎๅคๆ•ˆๆžœ
 
 
 
822
 
823
- | ๆŒ‡ๆ ‡ | ไฟฎๅคๅ‰ | ไฟฎๅคๅŽ | ๆ”นๅ–„ |
824
- |------|--------|--------|------|
825
- | ๆŽงๅˆถๆถˆๆฏ้ข‘็އ | ~100 msg/s | ~20 msg/s | โ†“ 80% |
826
- | ็Šถๆ€ๆŸฅ่ฏข้ข‘็އ | ~50 msg/s | ~5 msg/s | โ†“ 90% |
827
- | ๆ€ป Zenoh ๆถˆๆฏ | ~150 msg/s | ~25 msg/s | โ†“ 83% |
828
- | Daemon CPU ่ดŸ่ฝฝ | ๆŒ็ปญ้ซ˜่ดŸ่ฝฝ | ๆญฃๅธธ่ดŸ่ฝฝ | ๆ˜พ่‘—้™ไฝŽ |
829
- | ้ข„ๆœŸ็จณๅฎšๆ€ง | ๆ•ฐๅฐๆ—ถๅ†…ๅดฉๆบƒ | ๅฏ็จณๅฎš่ฟ่กŒๆ•ฐๅคฉ | ๅคงๅน…ๆๅ‡ |
830
 
831
- ### ็›ธๅ…ณๆ–‡ไปถ
832
- - `DAEMON_CRASH_FIX_PLAN.md` - ่ฏฆ็ป†ไฟฎๅคๆ–นๆกˆๅ’Œๆต‹่ฏ•่ฎกๅˆ’
833
- - `movement_manager.py` - ๆŽงๅˆถๅพช็Žฏไผ˜ๅŒ–
834
- - `reachy_controller.py` - ็Šถๆ€ๆŸฅ่ฏข็ผ“ๅญ˜
835
 
836
- ### ๅŽ็ปญไผ˜ๅŒ–ๅปบ่ฎฎ
837
- 1. โณ ๅŠจๆ€้ข‘็އ่ฐƒๆ•ด - ่ฟๅŠจๆ—ถ 50Hz๏ผŒ็ฉบ้—ฒๆ—ถ 5Hz
838
- 2. โณ ๆ‰น้‡็Šถๆ€ๆŸฅ่ฏข - ไธ€ๆฌกๆ€ง่Žทๅ–ๆ‰€ๆœ‰็Šถๆ€
839
- 3. โณ ๆ€ง่ƒฝ็›‘ๆŽงๅ’Œๅ‘Š่ญฆ - ๅฎžๆ—ถ็›‘ๆŽง daemon ๅฅๅบท็Šถๆ€
840
 
841
- ---
 
 
 
 
 
 
 
842
 
843
- ## ๐Ÿ”ง Daemon ๅดฉๆบƒ้—ฎ้ข˜ๆทฑๅบฆไฟฎๅค (2026-01-07)
 
 
 
 
844
 
845
- ### ้—ฎ้ข˜ๆ่ฟฐ
846
- ้•ฟๆœŸ่ฟ่กŒ่ฟ‡็จ‹ไธญ๏ผŒ`reachy_mini daemon` ไป็„ถไผšๅดฉๆบƒ๏ผŒไน‹ๅ‰็š„ไฟฎๅคไธๅคŸๅฝปๅบ•ใ€‚
847
 
848
- ### ๆ นๆœฌๅŽŸๅ› ๅˆ†ๆž
849
 
850
- ้€š่ฟ‡ๆทฑๅ…ฅๅˆ†ๆž SDK ๆบ็ ๅ‘็Žฐ๏ผš
 
 
 
 
 
 
 
 
851
 
852
- 1. **ๆฏๆฌก `set_target()` ๅ‘้€ 3 ๆก Zenoh ๆถˆๆฏ**
853
- - `set_target_head_pose()` - 1 ๆกๆถˆๆฏ
854
- - `set_target_antenna_joint_positions()` - 1 ๆกๆถˆๆฏ
855
- - `set_target_body_yaw()` - 1 ๆกๆถˆๆฏ
856
 
857
- 2. **Daemon ๆŽงๅˆถๅพช็Žฏๆ˜ฏ 50Hz**
858
- - ่ง `reachy_mini/daemon/backend/robot/backend.py`: `control_loop_frequency = 50.0`
859
- - ๅฆ‚ๆžœๆถˆๆฏๅ‘้€้ข‘็އ่ถ…่ฟ‡ 50Hz๏ผŒdaemon ๅฏ่ƒฝๆ— ๆณ•ๅŠๆ—ถๅค„็†
860
 
861
- 3. **ไน‹ๅ‰็š„ 20Hz ๆŽงๅˆถๅพช็Žฏไป็„ถ่ฟ‡้ซ˜**
862
- - 20Hz ร— 3 ๆถˆๆฏ = 60 ๆถˆๆฏ/็ง’
863
- - ๅทฒ็ป่ถ…่ฟ‡ daemon ็š„ 50Hz ๅค„็†่ƒฝๅŠ›
864
 
865
- 4. **ๅงฟๆ€ๅ˜ๅŒ–้˜ˆๅ€ผๅคชๅฐ (0.002)**
866
- - ๅ‘ผๅธๅŠจ็”ปใ€่ฏญ้Ÿณๆ‘†ๅŠจใ€ไบบ่„ธ่ฟฝ่ธชๆŒ็ปญไบง็”Ÿๅพฎๅฐๅ˜ๅŒ–
867
- - ๅ‡ ไนŽๆฏๆฌกๅพช็Žฏ้ƒฝไผš่งฆๅ‘ `set_target()`
 
 
868
 
869
- ### ไฟฎๅคๆ–นๆกˆ
 
 
870
 
871
- #### 1. ่ฟ›ไธ€ๆญฅ้™ไฝŽๆŽงๅˆถๅพช็Žฏ้ข‘็އ (movement_manager.py)
872
- ```python
873
- # ไปŽ 20Hz ้™ไฝŽๅˆฐ 10Hz
874
- # 10Hz ร— 3 ๆถˆๆฏ = 30 ๆถˆๆฏ/็ง’๏ผŒๅฎ‰ๅ…จไฝŽไบŽ daemon ็š„ 50Hz ๅฎน้‡
875
- CONTROL_LOOP_FREQUENCY_HZ = 10
876
- ```
877
 
878
- #### 2. ๅขžๅคงๅงฟๆ€ๅ˜ๅŒ–้˜ˆๅ€ผ (movement_manager.py)
879
- ```python
880
- # ไปŽ 0.002 ๅขžๅคงๅˆฐ 0.005
881
- # 0.005 rad โ‰ˆ 0.29 ๅบฆ๏ผŒไป็„ถ่ถณๅคŸๅนณๆป‘
882
- self._pose_change_threshold = 0.005
883
- ```
884
 
885
- #### 3. ้™ไฝŽๆ‘„ๅƒๅคด/ไบบ่„ธ่ฟฝ่ธช้ข‘็އ (camera_server.py)
886
- ```python
887
- # ไปŽ 15fps ้™ไฝŽๅˆฐ 10fps
888
- fps: int = 10
889
- ```
890
 
891
- #### 4. ้™ไฝŽ IMU ่ฝฎ่ฏข้ข‘็އ (tap_detector.py)
892
- ```python
893
- # ไปŽ 50Hz ้™ไฝŽๅˆฐ 20Hz
894
- TAP_DETECTION_RATE_HZ = 20
895
- ```
896
 
897
- #### 5. ๅขžๅคง็Šถๆ€็ผ“ๅญ˜ TTL (reachy_controller.py)
898
- ```python
899
- # ไปŽ 1 ็ง’ๅขžๅคงๅˆฐ 2 ็ง’
900
- self._cache_ttl = 2.0
901
- ```
902
 
903
- ### ไฟฎๅคๆ•ˆๆžœ
904
 
905
- | ๆŒ‡ๆ ‡ | ไฟฎๅคๅ‰ (20Hz) | ไฟฎๅคๅŽ (10Hz) | ๆ”นๅ–„ |
906
- |------|---------------|---------------|------|
907
- | ๆŽงๅˆถๅพช็Žฏ้ข‘็އ | 20 Hz | 10 Hz | โ†“ 50% |
908
- | ๆœ€ๅคง Zenoh ๆถˆๆฏ | 60 msg/s | 30 msg/s | โ†“ 50% |
909
- | ๅฎž้™…๏ฟฝ๏ฟฝๆฏ (ๆœ‰ๅ˜ๅŒ–ๆฃ€ๆต‹) | ~40 msg/s | ~15 msg/s | โ†“ 62% |
910
- | ไบบ่„ธ่ฟฝ่ธช้ข‘็އ | 15 Hz | 10 Hz | โ†“ 33% |
911
- | IMU ่ฝฎ่ฏข้ข‘็އ | 50 Hz | 20 Hz | โ†“ 60% |
912
- | ็Šถๆ€็ผ“ๅญ˜ TTL | 1 ็ง’ | 2 ็ง’ | โ†‘ 100% |
913
- | ้ข„ๆœŸ็จณๅฎšๆ€ง | ๆ•ฐๅฐๆ—ถๅดฉๆบƒ | ๅฏ็จณๅฎš่ฟ่กŒ | ๅคงๅน…ๆๅ‡ |
914
 
915
- ### ๅ…ณ้”ฎๅ‘็Žฐ
 
 
 
 
916
 
917
- ๅ‚่€ƒ `reachy_mini_conversation_app` ไฝฟ็”จ 100Hz ๆŽงๅˆถๅพช็Žฏ๏ผŒไฝ†ๅฎƒๆ˜ฏๅฎ˜ๆ–นๅบ”็”จ๏ผŒๅฏ่ƒฝๆœ‰็‰นๆฎŠไผ˜ๅŒ–ๆˆ–ๅœจๆ›ดๅผบ็กฌไปถไธŠ่ฟ่กŒใ€‚ๆˆ‘ไปฌ็š„ๅบ”็”จ้œ€่ฆๆ›ดไฟๅฎˆ็š„่ฎพ็ฝฎใ€‚
 
 
 
918
 
919
- ### ็›ธๅ…ณๆ–‡ไปถ
920
- - `movement_manager.py` - ๆŽงๅˆถๅพช็Žฏ้ข‘็އๅ’Œๅงฟๆ€้˜ˆๅ€ผ
921
- - `camera_server.py` - ไบบ่„ธ่ฟฝ่ธช้ข‘็އ
922
- - `tap_detector.py` - IMU ่ฝฎ่ฏข้ข‘็އ
923
- - `reachy_controller.py` - ็Šถๆ€็ผ“ๅญ˜ TTL
924
 
925
  ---
926
 
927
- ## ๐Ÿ”ง ๆ‹ไธ€ๆ‹ๅ”ค้†’ไธŽ้บฆๅ…‹้ฃŽ็ตๆ•ๅบฆไฟฎๅค (2026-01-07)
928
-
929
- ### ้—ฎ้ข˜ๆ่ฟฐ
930
- 1. **ๆ‹ไธ€ๆ‹ๅ”ค้†’้˜ปๅกž** - ๆ‹ไธ€ๆ‹ๅ”ค้†’ๅŽๅฏน่ฏๅŠŸ่ƒฝไธๆญฃๅธธ๏ผŒๅญ˜ๅœจ้˜ปๅกž้—ฎ้ข˜
931
- 2. **้บฆๅ…‹้ฃŽ็ตๆ•ๅบฆไฝŽ** - ้œ€่ฆ้ ๅพˆ่ฟ‘ๆ‰่ƒฝ่ฏ†ๅˆซ่ฏญ้Ÿณ
932
-
933
- ### ๆ นๆœฌๅŽŸๅ› 
934
- 1. **้Ÿณ้ข‘ๆ’ญๆ”พ้˜ปๅกž** - `_tap_continue_feedback()` ๅœจๆŒ็ปญๅฏน่ฏๆจกๅผไธ‹ๆ’ญๆ”พๆ็คบ้Ÿณ๏ผŒ้˜ปๅกžไบ†้Ÿณ้ข‘ๆตๅค„็†
935
- 2. **AGC ่ฎพ็ฝฎไธไผ˜ๅŒ–** - ReSpeaker XVF3800 ็š„้ป˜่ฎค่ฎพ็ฝฎไธ้€‚ๅˆ่ฟœ่ท็ฆป่ฏญ้Ÿณ่ฏ†ๅˆซ
936
 
937
- ### ไฟฎๅคๆ–นๆกˆ
 
 
 
 
 
 
 
 
 
 
 
938
 
939
- #### 1. ็งป้™คๆŒ็ปญๅฏน่ฏๅ้ฆˆไธญ็š„้Ÿณ้ข‘ๆ’ญๆ”พ (satellite.py)
940
- ```python
941
- def _tap_continue_feedback(self) -> None:
942
- """Provide feedback when continuing conversation in tap mode.
943
-
944
- Triggers a nod to indicate ready for next input.
945
- Sound is NOT played here to avoid blocking audio streaming.
946
- """
947
- # NOTE: Do NOT play sound here - it blocks audio streaming
948
- if self.state.motion_enabled and self.state.motion:
949
- self.state.motion.on_continue_listening()
950
- ```
951
-
952
- #### 2. ๆทปๅŠ ๅผ‚ๅธธๅค„็†ๅˆฐ tap ๅ›ž่ฐƒ (voice_assistant.py)
953
- ```python
954
- def _on_tap_detected(self) -> None:
955
- """Callback when tap is detected on the robot.
956
-
957
- NOTE: This is called from the tap_detector background thread.
958
- """
959
- try:
960
- self._state.satellite.wakeup_from_tap()
961
- # ... motion feedback
962
- except Exception as e:
963
- _LOGGER.error("Error in tap detection callback: %s", e)
964
- ```
965
-
966
- #### 3. ๅ…จ้ขไผ˜ๅŒ–้บฆๅ…‹้ฃŽ่ฎพ็ฝฎ (voice_assistant.py) - ๆ›ดๆ–ฐไบŽ 2026-01-07
967
- ```python
968
- def _optimize_microphone_settings(self) -> None:
969
- """Optimize ReSpeaker XVF3800 microphone settings for voice recognition."""
970
-
971
- # ========== 1. AGC (Automatic Gain Control) Settings ==========
972
- # Enable AGC for automatic volume normalization
973
- respeaker.write("PP_AGCONOFF", [1])
974
-
975
- # Increase AGC max gain for better distant speech pickup (default ~15dB -> 30dB)
976
- respeaker.write("PP_AGCMAXGAIN", [30.0])
977
-
978
- # Set AGC desired output level (default ~-25dB -> -18dB for stronger output)
979
- respeaker.write("PP_AGCDESIREDLEVEL", [-18.0])
980
-
981
- # Optimize AGC time constant for voice commands
982
- respeaker.write("PP_AGCTIME", [0.5])
983
-
984
- # ========== 2. Base Microphone Gain ==========
985
- # Increase base microphone gain (default 1.0 -> 2.0)
986
- respeaker.write("AUDIO_MGR_MIC_GAIN", [2.0])
987
-
988
- # ========== 3. Noise Suppression Settings ==========
989
- # Reduce noise suppression to preserve quiet speech (default ~0.5 -> 0.15)
990
- respeaker.write("PP_MIN_NS", [0.15])
991
- respeaker.write("PP_MIN_NN", [0.15])
992
-
993
- # ========== 4. Echo Cancellation & High-pass Filter ==========
994
- respeaker.write("PP_ECHOONOFF", [1])
995
- respeaker.write("AEC_HPFONOFF", [1])
996
- ```
997
-
998
- ### ไฟฎๅคๆ•ˆๆžœ
999
-
1000
- | ๅ‚ๆ•ฐ | ไฟฎๅคๅ‰ | ไฟฎๅคๅŽ | ่ฏดๆ˜Ž |
1001
- |------|--------|--------|------|
1002
- | ๆ‹ไธ€ๆ‹ๆŒ็ปญๅฏน่ฏ | ้˜ปๅกž | ๆญฃๅธธๅทฅไฝœ | ็งป้™ค้˜ปๅกž้Ÿณ้ข‘ๆ’ญๆ”พ |
1003
 
1004
  ---
1005
 
1006
- ## ๐Ÿ”ง v0.5.1 Bug ไฟฎๅค (2026-01-08)
1007
-
1008
- ### ้—ฎ้ข˜ 1: ้Ÿณไนๆ’ญๆ”พๅœจ่ฏญ้Ÿณๅฏน่ฏๅŽไธๆขๅค
1009
-
1010
- **้—ฎ้ข˜ๆ่ฟฐ**: ๆ’ญๆ”พ้Ÿณไนๆ—ถ๏ผŒ่ฏญ้Ÿณๅฏน่ฏ็ป“ๆŸๅŽ๏ผŒ้Ÿณไนๆฒกๆœ‰ๆขๅคๆ’ญๆ”พใ€‚
1011
-
1012
- **ๆ นๆœฌๅŽŸๅ› **: Sendspin ่ขซ้”™่ฏฏๅœฐ่ฟžๆŽฅๅˆฐ `tts_player` ่€Œไธๆ˜ฏ `music_player`ใ€‚
1013
-
1014
- **ไฟฎๅคๆ–นๆกˆ**:
1015
- - `voice_assistant.py`: Sendspin discovery ็Žฐๅœจ่ฟžๆŽฅๅˆฐ `music_player`
1016
- - `satellite.py`: `duck()`/`unduck()` ็Žฐๅœจ่ฐƒ็”จ `music_player.pause_sendspin()`/`resume_sendspin()`
1017
-
1018
- ### ้—ฎ้ข˜ 2: tap_sensitivity ่ฎพ็ฝฎไธๆŒไน…ๅŒ–
1019
-
1020
- **้—ฎ้ข˜ๆ่ฟฐ**: ESPHome ไธญ่ฎพ็ฝฎ็š„ tap_sensitivity ๅ€ผๅœจ้‡ๅฏๅŽไธขๅคฑใ€‚
1021
-
1022
- **ไฟฎๅคๆ–นๆกˆ**:
1023
- - `models.py`: ๅœจ `Preferences` dataclass ไธญๆทปๅŠ  `tap_sensitivity` ๅญ—ๆฎต
1024
- - `entity_registry.py`: ๅฎžไฝ“ setter ็Žฐๅœจไฟๅญ˜ๅˆฐ `preferences.json`
1025
- - ๅฏๅŠจๆ—ถไปŽไฟๅญ˜็š„ๅๅฅฝ่ฎพ็ฝฎๅŠ ่ฝฝๅ€ผ
1026
-
1027
- ### ้—ฎ้ข˜ 3: ่ฏญ้ŸณๅŠฉๆ‰‹ๅ”ค้†’ๆ—ถ็š„้Ÿณ้ข‘ๅ†ฒ็ช
1028
-
1029
- **้—ฎ้ข˜ๆ่ฟฐ**: ้Ÿณ้ข‘ๆŽจๆตๆ—ถ๏ผˆSendspin ๆˆ– ESPHome ้Ÿณ้ข‘๏ผ‰๏ผŒ่ฏญ้ŸณๅŠฉๆ‰‹่ขซๅ”ค้†’ไผšๅฏผ่‡ดๅ†ฒ็ชใ€‚
1030
-
1031
- **ไฟฎๅคๆ–นๆกˆ**:
1032
- - `audio_player.py`: ๆทปๅŠ  `pause_sendspin()` ๅ’Œ `resume_sendspin()` ๆ–นๆณ•
1033
- - `satellite.py`: `duck()` ็Žฐๅœจๆš‚ๅœ Sendspin๏ผŒ`unduck()` ๆขๅคๅฎƒ
1034
- - ๆ”น่ฟ› `pause()` ๆ–นๆณ•ไปฅๅฎž้™…ๅœๆญข้Ÿณ้ข‘่พ“ๅ‡บ
1035
-
1036
- ### ้—ฎ้ข˜ 4: AttributeError for _camera_server
1037
-
1038
- **้—ฎ้ข˜ๆ่ฟฐ**: `_set_conversation_mode()` ไธญๅผ•็”จไบ†ไธๅญ˜ๅœจ็š„ `_camera_server` ๅฑžๆ€งใ€‚
1039
-
1040
- **ไฟฎๅคๆ–นๆกˆ**: ๅฐ† `self._camera_server` ๆ”นไธบ `self.camera_server`๏ผˆ็งป้™คไธ‹ๅˆ’็บฟๅ‰็ผ€๏ผ‰
1041
-
1042
- ### ้—ฎ้ข˜ 5: tap_sensitivity ้ป˜่ฎคๅ€ผ้”™่ฏฏ
1043
-
1044
- **้—ฎ้ข˜ๆ่ฟฐ**: tap_sensitivity ้ป˜่ฎคๅ€ผไปไธบ 2.0g๏ผŒ่€Œไธๆ˜ฏ้ข„ๆœŸ็š„ 0.5gใ€‚
1045
-
1046
- **ไฟฎๅคๆ–นๆกˆ**: ไฝฟ็”จ `TAP_THRESHOLD_G_DEFAULT` ๅธธ้‡ไฝœไธบ้ป˜่ฎคๅ€ผ
1047
-
1048
- ### ้—ฎ้ข˜ 6: Sendspin ้‡‡ๆ ท็އไผ˜ๅŒ–
1049
-
1050
- **้—ฎ้ข˜ๆ่ฟฐ**: ReSpeaker ็กฌไปถ่พ“ๅ…ฅ/่พ“ๅ‡บๆ˜ฏ 16kHz๏ผˆ็กฌไปถ้™ๅˆถ๏ผ‰๏ผŒไฝ† Sendspin ๅฏ่ƒฝๅฐ่ฏ•ไฝฟ็”จๆ›ด้ซ˜้‡‡ๆ ท็އใ€‚
1051
-
1052
- **ไฟฎๅคๆ–นๆกˆ**: ๅœจ Sendspin ๆ”ฏๆŒๆ ผๅผๅˆ—่กจไธญไผ˜ๅ…ˆไฝฟ็”จ 16kHz๏ผŒ้ฟๅ…ไธๅฟ…่ฆ็š„้‡้‡‡ๆ ท
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1053
 
1054
  ---
1055
- | ้บฆๅ…‹้ฃŽ็ตๆ•ๅบฆ | ~30cm | ~2-3m | ๅ…จ้ขไผ˜ๅŒ– AGC ๅ’Œๅขž็›Š |
1056
- | AGC ๅผ€ๅ…ณ | ๅ…ณ้—ญ | ๅผ€ๅฏ | ่‡ชๅŠจ้Ÿณ้‡ๅฝ’ไธ€ๅŒ– |
1057
- | AGC ๆœ€ๅคงๅขž็›Š | ~15dB | 30dB | ๆๅ‡่ฟœ่ท็ฆปๆ‹พ้Ÿณ |
1058
- | AGC ็›ฎๆ ‡็”ตๅนณ | -25dB | -18dB | ๆ›ดๅผบ่พ“ๅ‡บไฟกๅท |
1059
- | ้บฆๅ…‹้ฃŽๅขž็›Š | 1.0x | 2.0x | ๅŸบ็ก€ๅขž็›Š็ฟปๅ€ |
1060
- | ๅ™ชๅฃฐๆŠ‘ๅˆถ | ~0.5 | 0.15 | ๅ‡ๅฐ‘ๅฏน่ฏญ้Ÿณ็š„่ฏฏๆŠ‘ๅˆถ |
1061
- | ๅ›žๅฃฐๆถˆ้™ค | ๅผ€ๅฏ | ๅผ€ๅฏ | ไฟๆŒ TTS ๆ’ญๆ”พๆ—ถ็š„ๆธ…ๆ™ฐๅบฆ |
1062
- | ้ซ˜้€šๆปคๆณข | ๅ…ณ้—ญ | ๅผ€ๅฏ | ๅŽป้™คไฝŽ้ข‘ๅ™ชๅฃฐ |
1063
-
1064
- ### XVF3800 ๅ‚ๆ•ฐๅ‚่€ƒ
1065
-
1066
- | ๅ‚ๆ•ฐๅ | ็ฑปๅž‹ | ่Œƒๅ›ด | ่ฏดๆ˜Ž |
1067
- |--------|------|------|------|
1068
- | `PP_AGCONOFF` | int32 | 0/1 | AGC ๅผ€ๅ…ณ |
1069
- | `PP_AGCMAXGAIN` | float | 0-40 dB | AGC ๆœ€ๅคงๅขž็›Š |
1070
- | `PP_AGCDESIREDLEVEL` | float | dB | AGC ็›ฎๆ ‡่พ“ๅ‡บ็”ตๅนณ |
1071
- | `PP_AGCTIME` | float | ็ง’ | AGC ๆ—ถ้—ดๅธธๆ•ฐ |
1072
- | `AUDIO_MGR_MIC_GAIN` | float | 0-4.0 | ้บฆๅ…‹้ฃŽๅขž็›Šๅ€ๆ•ฐ |
1073
- | `PP_MIN_NS` | float | 0-1.0 | ๆœ€ๅฐๅ™ชๅฃฐๆŠ‘ๅˆถ (่ถŠไฝŽ่ถŠๅฐ‘ๆŠ‘ๅˆถ) |
1074
- | `PP_MIN_NN` | float | 0-1.0 | ๆœ€ๅฐๅ™ชๅฃฐไผฐ่ฎก |
1075
- | `PP_ECHOONOFF` | int32 | 0/1 | ๅ›žๅฃฐๆถˆ้™คๅผ€ๅ…ณ |
1076
- | `AEC_HPFONOFF` | int32 | 0/1 | ้ซ˜้€šๆปคๆณขๅผ€ๅ…ณ |
1077
-
1078
- ### ็›ธๅ…ณๆ–‡ไปถ
1079
- - `satellite.py` - ็งป้™ค้˜ปๅกž็š„้Ÿณ้ข‘ๆ’ญๆ”พ
1080
- - `voice_assistant.py` - ๅ…จ้ข้บฆๅ…‹้ฃŽไผ˜ๅŒ–
1081
- - `reachy_controller.py` - AGC ๅฎžไฝ“้ป˜่ฎคๅ€ผๆ›ดๆ–ฐ
1082
- - `entity_registry.py` - AGC max gain ่Œƒๅ›ดๆ›ดๆ–ฐ (0-40dB)
1083
- - `reachy_mini/src/reachy_mini/media/audio_control_utils.py` - SDK ๅ‚่€ƒ
1084
 
1085
- ---
1086
-
1087
- ### SDK ๆ•ฐๆฎ็ป“ๆž„ๅ‚่€ƒ
1088
 
1089
  ```python
1090
- # ็”ตๆœบๆŽงๅˆถๆจกๅผ
1091
  class MotorControlMode(str, Enum):
1092
- Enabled = "enabled" # ๆ‰ญ็Ÿฉๅผ€ๅฏ๏ผŒไฝ็ฝฎๆŽงๅˆถ
1093
- Disabled = "disabled" # ๆ‰ญ็Ÿฉๅ…ณ้—ญ
1094
- GravityCompensation = "gravity_compensation" # ้‡ๅŠ›่กฅๅฟๆจกๅผ
1095
 
1096
- # Daemon ็Šถๆ€
1097
  class DaemonState(Enum):
1098
  NOT_INITIALIZED = "not_initialized"
1099
  STARTING = "starting"
@@ -1102,70 +415,14 @@ class DaemonState(Enum):
1102
  STOPPED = "stopped"
1103
  ERROR = "error"
1104
 
1105
- # ๅฎŒๆ•ด็Šถๆ€
1106
- class FullState:
1107
- control_mode: MotorControlMode
1108
- head_pose: XYZRPYPose # x, y, z (m), roll, pitch, yaw (rad)
1109
- head_joints: list[float] # 7 ไธชๅ…ณ่Š‚่ง’ๅบฆ
1110
- body_yaw: float
1111
- antennas_position: list[float] # [right, left]
1112
- doa: DoAInfo # angle (rad), speech_detected (bool)
1113
-
1114
- # IMU ๆ•ฐๆฎ (ไป…ๆ— ็บฟ็‰ˆๆœฌ)
1115
- imu_data = {
1116
- "accelerometer": [x, y, z], # m/sยฒ
1117
- "gyroscope": [x, y, z], # rad/s
1118
- "quaternion": [w, x, y, z], # ๅงฟๆ€ๅ››ๅ…ƒๆ•ฐ
1119
- "temperature": float # ยฐC
1120
- }
1121
-
1122
- # ๅฎ‰ๅ…จ้™ๅˆถ
1123
  HEAD_PITCH_ROLL_LIMIT = [-40ยฐ, +40ยฐ]
1124
  HEAD_YAW_LIMIT = [-180ยฐ, +180ยฐ]
1125
  BODY_YAW_LIMIT = [-160ยฐ, +160ยฐ]
1126
- YAW_DELTA_MAX = 65ยฐ # ๅคด้ƒจไธŽ่บซไฝ“ๅ่ˆช่ง’ๆœ€ๅคงๅทฎๅ€ผ
1127
- ```
1128
-
1129
- ### ESPHome ๅ่ฎฎๅฎž็Žฐ่ฏดๆ˜Ž
1130
-
1131
- ESPHome ๅ่ฎฎ้€š่ฟ‡ protobuf ๆถˆๆฏไธŽ Home Assistant ้€šไฟกใ€‚้œ€่ฆๅฎž็Žฐไปฅไธ‹ๆถˆๆฏ็ฑปๅž‹๏ผš
1132
-
1133
- ```python
1134
- from aioesphomeapi.api_pb2 import (
1135
- # Number ๅฎžไฝ“ (้Ÿณ้‡/่ง’ๅบฆๆŽงๅˆถ)
1136
- ListEntitiesNumberResponse,
1137
- NumberStateResponse,
1138
- NumberCommandRequest,
1139
-
1140
- # Select ๅฎžไฝ“ (็”ตๆœบๆจกๅผ)
1141
- ListEntitiesSelectResponse,
1142
- SelectStateResponse,
1143
- SelectCommandRequest,
1144
-
1145
- # Button ๅฎžไฝ“ (ๅ”ค้†’/็ก็œ )
1146
- ListEntitiesButtonResponse,
1147
- ButtonCommandRequest,
1148
-
1149
- # Switch ๅฎžไฝ“ (็”ตๆœบๅผ€ๅ…ณ)
1150
- ListEntitiesSwitchResponse,
1151
- SwitchStateResponse,
1152
- SwitchCommandRequest,
1153
-
1154
- # Sensor ๅฎžไฝ“ (ๆ•ฐๅ€ผไผ ๆ„Ÿๅ™จ)
1155
- ListEntitiesSensorResponse,
1156
- SensorStateResponse,
1157
-
1158
- # Binary Sensor ๅฎžไฝ“ (ๅธƒๅฐ”ไผ ๆ„Ÿๅ™จ)
1159
- ListEntitiesBinarySensorResponse,
1160
- BinarySensorStateResponse,
1161
-
1162
- # Text Sensor ๅฎžไฝ“ (ๆ–‡ๆœฌไผ ๆ„Ÿๅ™จ)
1163
- ListEntitiesTextSensorResponse,
1164
- TextSensorStateResponse,
1165
- )
1166
  ```
1167
 
1168
- ## ๅ‚่€ƒ้กน็›ฎ
1169
 
1170
  - [OHF-Voice/linux-voice-assistant](https://github.com/OHF-Voice/linux-voice-assistant)
1171
  - [pollen-robotics/reachy_mini](https://github.com/pollen-robotics/reachy_mini)
 
1
+ # Reachy Mini Home Assistant Voice Assistant - Project Plan
2
 
3
+ ## Project Overview
4
 
5
+ Integrate Home Assistant voice assistant functionality into Reachy Mini Wi-Fi robot, communicating with Home Assistant via ESPHome protocol.
6
 
7
+ ## Local Reference Directories (DO NOT modify any files in reference directories)
8
+ 1. [linux-voice-assistant](reference/linux-voice-assistant) - Linux-based Home Assistant voice assistant app for reference
9
+ 2. [Reachy Mini SDK](reference/reachy_mini) - Reachy Mini SDK local directory for reference
10
+ 3. [reachy_mini_conversation_app](reference/reachy_mini_conversation_app) - Reachy Mini conversation app for reference
11
+ 4. [reachy-mini-desktop-app](reference/reachy-mini-desktop-app) - Reachy Mini desktop app for reference
12
+ 5. [sendspin](reference/sendspin-cli/) - Sendspin client for reference
13
 
14
+ ## Core Design Principles
15
 
16
+ 1. **Zero Configuration** - Users only need to install the app, no manual configuration required
17
+ 2. **Native Hardware** - Use robot's built-in microphone and speaker
18
+ 3. **Home Assistant Centralized Management** - All configuration done on Home Assistant side
19
+ 4. **Motion Feedback** - Provide head movement and antenna animation feedback during voice interaction
20
+ 5. **Project Constraints** - Strictly follow [Reachy Mini SDK](reachy_mini) architecture design and constraints
21
+ 6. **Code Quality** - Follow Python development standards with consistent code style, clear structure, complete comments, comprehensive documentation
22
+ 7. **Feature Priority** - Voice conversation with Home Assistant is highest priority; other features are auxiliary and must not affect voice conversation functionality or response speed
23
+ 8. **No LED Functions** - LEDs are hidden inside the robot; all LED control is ignored
24
+ 9. **Preserve Functionality** - Any code modifications should optimize while preserving completed features; do not remove features to solve problems
25
 
26
+ ## Technical Architecture
27
 
28
  ```
29
  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 
40
  โ”‚ โ”‚
41
  โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚
42
  โ”‚ โ”‚ Camera + Face Tracking (YOLO) โ”‚ โ”‚
43
+ โ”‚ โ”‚ - 15Hz face detection and tracking โ”‚ โ”‚
44
+ โ”‚ โ”‚ - look_at_image() calculates target pose โ”‚ โ”‚
45
+ โ”‚ โ”‚ - Smooth return to neutral position after face lost โ”‚ โ”‚
46
  โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚
47
  โ”‚ โ”‚
48
  โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚
49
  โ”‚ โ”‚ Motion Controller (Head + Antennas) - 5Hz โ”‚ โ”‚
50
  โ”‚ โ”‚ - Face tracking offsets (secondary pose) โ”‚ โ”‚
51
+ โ”‚ โ”‚ - Speech sway (voice-driven micro-movements) โ”‚ โ”‚
52
+ โ”‚ โ”‚ - Breathing animation (idle breathing) โ”‚ โ”‚
53
  โ”‚ โ”‚ - on_wakeup โ†’ on_listening โ†’ on_speaking โ†’ on_idle โ”‚ โ”‚
54
  โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚
55
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
 
65
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
66
  ```
67
 
68
+ ## Completed Features
69
+
70
+ ### Core Features
71
+ - [x] ESPHome protocol server implementation
72
+ - [x] mDNS service discovery (auto-discovered by Home Assistant)
73
+ - [x] Local wake word detection (microWakeWord)
74
+ - [x] Tap-to-wake (IMU acceleration detection, wireless version only)
75
+ - [x] Audio stream transmission to Home Assistant
76
+ - [x] TTS audio playback
77
+ - [x] Stop word detection
78
+
79
+ ### Reachy Mini Integration
80
+ - [x] Use Reachy Mini SDK microphone input
81
+ - [x] Use Reachy Mini SDK speaker output
82
+ - [x] Head motion control (nod, shake, gaze)
83
+ - [x] Antenna animation control
84
+ - [x] Voice state feedback actions
85
+ - [x] YOLO face tracking (replaces DOA sound source localization)
86
+ - [x] 5Hz unified motion control loop
87
+
88
+ ### Application Architecture
89
+ - [x] Compliant with Reachy Mini App architecture
90
+ - [x] Auto-download wake word models
91
+ - [x] Auto-download sound effect files
92
+ - [x] No .env configuration file required
93
+
94
+ ## File List
95
 
96
  ```
97
  reachy_mini_ha_voice/
98
  โ”œโ”€โ”€ reachy_mini_ha_voice/
99
+ โ”‚ โ”œโ”€โ”€ __init__.py # Package initialization
100
+ โ”‚ โ”œโ”€โ”€ __main__.py # Command line entry
101
+ โ”‚ โ”œโ”€โ”€ main.py # ReachyMiniApp entry
102
+ โ”‚ โ”œโ”€โ”€ voice_assistant.py # Voice assistant service
103
+ โ”‚ โ”œโ”€โ”€ satellite.py # ESPHome protocol handling
104
+ โ”‚ โ”œโ”€โ”€ audio_player.py # Audio player
105
+ โ”‚ โ”œโ”€โ”€ camera_server.py # MJPEG camera stream server + face tracking
106
+ โ”‚ โ”œโ”€โ”€ head_tracker.py # YOLO face detector
107
+ โ”‚ โ”œโ”€โ”€ motion.py # Motion control (high-level API)
108
+ โ”‚ โ”œโ”€โ”€ movement_manager.py # Unified movement manager (20Hz control loop)
109
+ โ”‚ โ”œโ”€โ”€ models.py # Data models
110
+ โ”‚ โ”œโ”€โ”€ entity.py # ESPHome base entity
111
+ โ”‚ โ”œโ”€โ”€ entity_extensions.py # Extended entity types
112
+ โ”‚ โ”œโ”€โ”€ reachy_controller.py # Reachy Mini controller wrapper
113
+ โ”‚ โ”œโ”€โ”€ api_server.py # API server
114
+ โ”‚ โ”œโ”€โ”€ zeroconf.py # mDNS discovery
115
+ โ”‚ โ””โ”€โ”€ util.py # Utility functions
116
+ โ”œโ”€โ”€ wakewords/ # Wake word models (auto-download)
117
+ โ”œโ”€โ”€ sounds/ # Sound effect files (auto-download)
118
+ โ”œโ”€โ”€ pyproject.toml # Project configuration
119
+ โ”œโ”€โ”€ README.md # Documentation
120
+ โ””โ”€โ”€ PROJECT_PLAN.md # Project plan
 
 
 
 
 
 
 
 
121
  ```
122
 
123
+ ## Usage Flow
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
124
 
125
+ 1. **Install App** - Install `reachy-mini-ha-voice` from Reachy Mini App Store
126
+ 2. **Start App** - App auto-starts ESPHome server (port 6053), auto-downloads required models and sounds
127
+ 3. **Connect Home Assistant** - Home Assistant auto-discovers device (mDNS) or manually add via Settings โ†’ Devices & Services โ†’ Add Integration โ†’ ESPHome
128
+ 4. **Use Voice Assistant** - Say "Okay Nabu" to wake, speak command, Reachy Mini provides motion feedback
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
129
 
130
  ---
131
 
132
+ ## ESPHome Entity Implementation
133
+
134
+ ### Completed Entities Summary
135
+
136
+ **Total: 43+ entities implemented**
137
+ - Phase 1-4: Basic controls, motor control, pose control, gaze control
138
+ - Phase 5-7: Audio sensors, diagnostics, IMU sensors
139
+ - Phase 8-12: Emotion control, microphone volume, camera, audio processing
140
+ - Phase 13: Sendspin audio output support
141
+
142
+ ### Control Entities (Read/Write)
143
+
144
+ | Entity Type | Name | Description |
145
+ |-------------|------|-------------|
146
+ | `Number` | `speaker_volume` | Speaker volume (0-100) |
147
+ | `Select` | `motor_mode` | Motor mode (enabled/disabled/gravity_compensation) |
148
+ | `Switch` | `motors_enabled` | Motor torque switch |
149
+ | `Button` | `wake_up` / `go_to_sleep` | Wake/sleep robot actions |
150
+ | `Number` | `head_x/y/z` | Head position control (ยฑ50mm) |
151
+ | `Number` | `head_roll/pitch/yaw` | Head angle control |
152
+ | `Number` | `body_yaw` | Body yaw angle (-160ยฐ ~ +160ยฐ) |
153
+ | `Number` | `antenna_left/right` | Antenna angle control (ยฑ90ยฐ) |
154
+ | `Number` | `look_at_x/y/z` | Gaze point coordinates |
155
+ | `Select` | `emotion` | Emotion selector (Happy/Sad/Angry/Fear/Surprise/Disgust) |
156
+ | `Number` | `microphone_volume` | Microphone volume (0-100%) |
157
+ | `Switch` | `agc_enabled` | Auto gain control switch |
158
+ | `Number` | `agc_max_gain` | AGC max gain (0-30 dB) |
159
+ | `Number` | `noise_suppression` | Noise suppression level (0-100%) |
160
+ | `Number` | `tap_sensitivity` | Tap detection sensitivity (0.5-4.0g) |
161
+ | `Switch` | `sendspin_enabled` | Sendspin switch |
162
+
163
+ ### Sensor Entities (Read-only)
164
+
165
+ | Entity Type | Name | Description |
166
+ |-------------|------|-------------|
167
+ | `Text Sensor` | `daemon_state` | Daemon status |
168
+ | `Binary Sensor` | `backend_ready` | Backend ready status |
169
+ | `Text Sensor` | `error_message` | Current error message |
170
+ | `Sensor` | `doa_angle` | Sound source direction angle |
171
+ | `Binary Sensor` | `speech_detected` | Speech detection status |
172
+ | `Sensor` | `control_loop_frequency` | Control loop frequency (Hz) |
173
+ | `Text Sensor` | `sdk_version` | SDK version |
174
+ | `Text Sensor` | `robot_name` | Robot name |
175
+ | `Binary Sensor` | `wireless_version` | Wireless version flag |
176
+ | `Binary Sensor` | `simulation_mode` | Simulation mode flag |
177
+ | `Text Sensor` | `wlan_ip` | Wireless IP address |
178
+ | `Sensor` | `imu_accel_x/y/z` | Accelerometer (m/sยฒ) |
179
+ | `Sensor` | `imu_gyro_x/y/z` | Gyroscope (rad/s) |
180
+ | `Sensor` | `imu_temperature` | IMU temperature (ยฐC) |
181
+ | `Binary Sensor` | `echo_cancellation_converged` | Echo cancellation convergence status |
182
+ | `Camera` | `camera` | ESPHome Camera entity |
183
+ | `Text Sensor` | `sendspin_url` | Sendspin server URL |
184
+ | `Binary Sensor` | `sendspin_connected` | Sendspin connection status |
185
 
186
  ---
187
 
188
+ ## Voice Assistant Enhancement Features
 
 
 
 
 
 
 
 
 
 
 
189
 
190
+ ### Phase 14 - Emotion Action Feedback System ๐ŸŸก Partial
 
 
 
 
 
 
 
 
 
 
 
 
 
191
 
192
+ **Status**: Basic infrastructure ready, supports manual trigger, uses voice-driven natural micro-movements during conversation
 
 
 
 
 
 
 
 
 
 
 
193
 
194
+ **Implemented**:
195
+ - โœ… Emotion Selector entity (`emotion`)
196
+ - โœ… Basic emotion action playback API (`_play_emotion`)
197
+ - โœ… Emotion mapping: Happy/Sad/Angry/Fear/Surprise/Disgust
198
+ - โœ… Integration with HuggingFace action library
199
+ - โœ… SpeechSway system for natural head micro-movements during conversation
200
 
201
+ **Design Decisions**:
202
+ - ๐ŸŽฏ No auto-play of full emotion actions during conversation to avoid blocking
203
+ - ๐ŸŽฏ Use voice-driven head sway (SpeechSway) for natural motion feedback
204
+ - ๐ŸŽฏ Emotion actions retained as manual trigger feature via ESPHome entity
205
 
206
+ ### Phase 15 - Face Tracking (Replaces DOA) โœ… Complete
 
 
 
 
 
 
207
 
208
+ **Goal**: Implement natural face tracking so robot looks at speaker during conversation.
 
 
 
209
 
210
+ **Design Decision**:
211
+ - โŒ Original plan: DOA (Direction of Arrival) sound source tracking
212
+ - โœ… Changed to: YOLO face detection - more stable and accurate
213
+ - Reason: DOA inaccurate at wakeup, frequent queries cause daemon crash
214
 
215
+ **Implemented Features**:
216
+ - โœ… YOLO face detection using `AdamCodd/YOLOv11n-face-detection` model
217
+ - โœ… Adaptive frame rate: 15fps during conversation, 3fps when idle without face
218
+ - โœ… look_at_image() calculates target pose from face position
219
+ - โœ… Smooth return to neutral position after face lost (1 second)
220
+ - โœ… face_tracking_offsets as secondary pose overlay
221
+ - โœ… Model download retry (3 attempts, 5s interval)
222
+ - โœ… Conversation mode integration with voice assistant state
223
 
224
+ **Resource Optimization (v0.5.1)**:
225
+ - During conversation (listening/thinking/speaking): High-frequency tracking 15fps
226
+ - Idle with face detected: High-frequency tracking 15fps
227
+ - Idle without face for 10s: Low-power mode 3fps
228
+ - Immediately restore high-frequency tracking when face detected
229
 
230
+ ### Phase 16 - Cartoon Style Motion Mode ๐ŸŸก Partial
 
231
 
232
+ **Goal**: Use SDK interpolation techniques for more expressive robot movements.
233
 
234
+ **Implemented**:
235
+ - โœ… 20Hz unified control loop (reduced from 100Hz to prevent daemon crash)
236
+ - โœ… Pose change detection - only send commands on significant changes (threshold 0.001)
237
+ - โœ… State query caching - 100ms TTL, reduces daemon load
238
+ - โœ… Smooth interpolation (ease in-out curve)
239
+ - โœ… Breathing animation - idle Z-axis micro-movement + antenna sway
240
+ - โœ… Command queue mode - thread-safe external API
241
+ - โœ… Error throttling - prevents log explosion
242
+ - โœ… Connection health monitoring - auto-detect and recover from connection loss
243
 
244
+ **Not Implemented**:
245
+ - โŒ Dynamic interpolation technique switching (CARTOON/EASE_IN_OUT etc.)
246
+ - โŒ Exaggerated cartoon bounce effects
 
247
 
248
+ ### Phase 17 - Antenna Sync Animation During Speech ๐ŸŸก Partial
 
 
249
 
250
+ **Goal**: Antennas sway with audio rhythm during TTS playback, simulating "speaking" effect.
 
 
251
 
252
+ **Implemented**:
253
+ - โœ… Voice-driven head sway (`SpeechSwayGenerator`)
254
+ - โœ… VAD detection based on audio loudness
255
+ - โœ… Multi-frequency sine wave overlay (Lissajous motion)
256
+ - โœ… Smooth envelope transitions
257
 
258
+ **Not Implemented**:
259
+ - โŒ Antenna sway with audio rhythm (currently only head sway)
260
+ - โŒ Audio spectrum analysis driven animation
261
 
262
+ ### Phase 18 - Visual Gaze Interaction โŒ Not Implemented
 
 
 
 
 
263
 
264
+ **Goal**: Use camera to detect faces for eye contact.
 
 
 
 
 
265
 
266
+ ### Phase 19 - Gravity Compensation Interactive Mode ๐ŸŸก Partial
 
 
 
 
267
 
268
+ **Implemented**:
269
+ - โœ… Gravity compensation mode switch (`motor_mode` Select entity)
 
 
 
270
 
271
+ **Not Implemented**:
272
+ - โŒ Teaching mode - record motion trajectory
273
+ - โŒ Save/playback custom actions
 
 
274
 
275
+ ### Phase 20 - Environment Awareness Response ๐ŸŸก Partial
276
 
277
+ **Implemented**:
278
+ - โœ… Tap-to-wake enters continuous conversation mode
279
+ - โœ… Second tap exits continuous conversation mode
 
 
 
 
 
 
280
 
281
+ **Tap-to-wake vs Voice Wake**:
282
+ | Wake Method | Conversation Mode | Description |
283
+ |-------------|-------------------|-------------|
284
+ | Voice wake (Okay Nabu) | Single conversation | Need to say wake word for each conversation |
285
+ | Tap-to-wake | Continuous conversation | Auto-continue listening after TTS ends, tap again to exit |
286
 
287
+ **Not Implemented**:
288
+ - โŒ Shake detection - play dizzy action
289
+ - โŒ Tilt/fall detection - play help action
290
+ - โŒ Long idle - enter sleep animation
291
 
292
+ ### Phase 21 - Home Assistant Scene Integration โŒ Not Implemented
 
 
 
 
293
 
294
  ---
295
 
296
+ ## Completion Statistics
 
 
 
 
 
 
 
 
297
 
298
+ | Phase | Status | Completion | Notes |
299
+ |-------|--------|------------|-------|
300
+ | Phase 1-12 | โœ… Complete | 100% | 40 ESPHome entities implemented (Phase 11 LED disabled) |
301
+ | Phase 13 | โœ… Complete | 100% | Sendspin audio output support |
302
+ | Phase 14 | ๐ŸŸก Partial | 30% | API infrastructure ready, missing auto-trigger |
303
+ | Phase 15 | โœ… Complete | 100% | YOLO face tracking fully implemented |
304
+ | Phase 16 | ๐ŸŸก Partial | 70% | Control loop + pose detection + breathing animation |
305
+ | Phase 17 | ๐ŸŸก Partial | 50% | Voice-driven head sway implemented |
306
+ | Phase 18 | โŒ Not done | 10% | Camera implemented, missing face detection |
307
+ | Phase 19 | ๐ŸŸก Partial | 40% | Mode switch implemented, missing teaching flow |
308
+ | Phase 20 | ๐ŸŸก Partial | 30% | Tap-to-wake implemented |
309
+ | Phase 21 | โŒ Not done | 0% | Not implemented |
310
 
311
+ **Overall Completion**: **Phase 1-13: 100%** | **Phase 14-21: ~45%**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
312
 
313
  ---
314
 
315
+ ## Bug Fixes History
316
+
317
+ ### v0.5.1 Bug Fixes (2026-01-08)
318
+
319
+ #### Issue 1: Music Not Resuming After Voice Conversation
320
+ **Problem**: Music doesn't resume after voice conversation ends.
321
+ **Root Cause**: Sendspin was incorrectly connected to `tts_player` instead of `music_player`.
322
+ **Fix**:
323
+ - `voice_assistant.py`: Sendspin discovery now connects to `music_player`
324
+ - `satellite.py`: `duck()`/`unduck()` now call `music_player.pause_sendspin()`/`resume_sendspin()`
325
+
326
+ #### Issue 2: tap_sensitivity Not Persisted
327
+ **Problem**: tap_sensitivity value set in ESPHome lost after restart.
328
+ **Fix**:
329
+ - `models.py`: Added `tap_sensitivity` field to `Preferences` dataclass
330
+ - `entity_registry.py`: Entity setter now saves to `preferences.json`
331
+ - Load saved value on startup
332
+
333
+ #### Issue 3: Audio Conflict During Voice Assistant Wakeup
334
+ **Problem**: Audio streaming (Sendspin or ESPHome audio) conflicts when voice assistant wakes up.
335
+ **Fix**:
336
+ - `audio_player.py`: Added `pause_sendspin()` and `resume_sendspin()` methods
337
+ - `satellite.py`: `duck()` now pauses Sendspin, `unduck()` resumes it
338
+ - Improved `pause()` method to actually stop audio output
339
+
340
+ #### Issue 4: AttributeError for _camera_server
341
+ **Problem**: `_set_conversation_mode()` referenced non-existent `_camera_server` attribute.
342
+ **Fix**: Changed `self._camera_server` to `self.camera_server` (removed underscore prefix)
343
+
344
+ #### Issue 5: tap_sensitivity Default Value Wrong
345
+ **Problem**: tap_sensitivity default was still 2.0g instead of expected 0.5g.
346
+ **Fix**: Use `TAP_THRESHOLD_G_DEFAULT` constant as default value
347
+
348
+ #### Issue 6: Sendspin Sample Rate Optimization
349
+ **Problem**: ReSpeaker hardware I/O is 16kHz (hardware limitation), but Sendspin might try higher sample rates.
350
+ **Fix**: Prioritize 16kHz in Sendspin supported formats list to avoid unnecessary resampling
351
+
352
+ ### Daemon Crash Fix (2026-01-07)
353
+
354
+ **Problem**: `reachy_mini daemon` crashes during long-term operation.
355
+
356
+ **Root Cause Analysis**:
357
+ 1. Each `set_target()` sends 3 Zenoh messages
358
+ 2. Daemon control loop is 50Hz
359
+ 3. Previous 20Hz control loop still too high (20Hz ร— 3 = 60 msg/s > 50Hz capacity)
360
+ 4. Pose change threshold too small (0.002) - almost every loop triggers `set_target()`
361
+
362
+ **Fix**:
363
+ - Control loop frequency: 20Hz โ†’ 10Hz
364
+ - Pose change threshold: 0.002 โ†’ 0.005
365
+ - Camera/face tracking frequency: 15fps โ†’ 10fps
366
+ - IMU polling frequency: 50Hz โ†’ 20Hz
367
+ - State cache TTL: 1s โ†’ 2s
368
+
369
+ **Results**:
370
+ | Metric | Before | After | Improvement |
371
+ |--------|--------|-------|-------------|
372
+ | Control loop frequency | 20 Hz | 10 Hz | โ†“ 50% |
373
+ | Max Zenoh messages | 60 msg/s | 30 msg/s | โ†“ 50% |
374
+ | Expected stability | Hours before crash | Stable operation | Significant |
375
+
376
+ ### Tap-to-Wake and Microphone Sensitivity Fix (2026-01-07)
377
+
378
+ **Problems**:
379
+ 1. Tap-to-wake blocking - conversation not working properly after tap wake
380
+ 2. Low microphone sensitivity - need to be very close for voice recognition
381
+
382
+ **Fixes**:
383
+ 1. Removed audio playback in `_tap_continue_feedback()` to avoid blocking
384
+ 2. Comprehensive microphone optimization:
385
+ - AGC enabled with max gain 30dB
386
+ - AGC desired level -18dB
387
+ - Base microphone gain 2.0x
388
+ - Noise suppression reduced to 0.15
389
+ - Echo cancellation and high-pass filter enabled
390
+
391
+ **Results**:
392
+ | Parameter | Before | After |
393
+ |-----------|--------|-------|
394
+ | Microphone sensitivity | ~30cm | ~2-3m |
395
+ | AGC max gain | ~15dB | 30dB |
396
+ | Noise suppression | ~0.5 | 0.15 |
397
 
398
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
399
 
400
+ ## SDK Data Structure Reference
 
 
401
 
402
  ```python
403
+ # Motor control mode
404
  class MotorControlMode(str, Enum):
405
+ Enabled = "enabled" # Torque on, position control
406
+ Disabled = "disabled" # Torque off
407
+ GravityCompensation = "gravity_compensation" # Gravity compensation mode
408
 
409
+ # Daemon state
410
  class DaemonState(Enum):
411
  NOT_INITIALIZED = "not_initialized"
412
  STARTING = "starting"
 
415
  STOPPED = "stopped"
416
  ERROR = "error"
417
 
418
+ # Safety limits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
419
  HEAD_PITCH_ROLL_LIMIT = [-40ยฐ, +40ยฐ]
420
  HEAD_YAW_LIMIT = [-180ยฐ, +180ยฐ]
421
  BODY_YAW_LIMIT = [-160ยฐ, +160ยฐ]
422
+ YAW_DELTA_MAX = 65ยฐ # Max difference between head and body yaw
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
423
  ```
424
 
425
+ ## Reference Projects
426
 
427
  - [OHF-Voice/linux-voice-assistant](https://github.com/OHF-Voice/linux-voice-assistant)
428
  - [pollen-robotics/reachy_mini](https://github.com/pollen-robotics/reachy_mini)