Commit ·
a33ba57
1
Parent(s): 02640f6
v0.4.0: Daemon stability fixes and microphone optimization
Browse filesKey changes:
- Reduce control loop from 20Hz to 10Hz to prevent daemon crashes
- Increase pose change threshold from 0.002 to 0.005 rad
- Reduce face tracking from 15fps to 10fps
- Reduce IMU polling from 50Hz to 20Hz
- Increase status cache TTL from 1s to 2s
- Optimize ReSpeaker XVF3800 microphone settings:
- Enable AGC with max gain 30dB
- Increase base mic gain to 2.0x
- Reduce noise suppression for better voice pickup
- Code refactoring: reduce reachy_controller.py from ~1096 to 785 lines
- Add helper methods for cleaner code structure
- PROJECT_PLAN.md +138 -17
- pyproject.toml +1 -1
- reachy_mini_ha_voice/audio_player.py +0 -4
- reachy_mini_ha_voice/camera_server.py +22 -6
- reachy_mini_ha_voice/entity_registry.py +1 -1
- reachy_mini_ha_voice/head_tracker.py +42 -37
- reachy_mini_ha_voice/motion.py +5 -35
- reachy_mini_ha_voice/movement_manager.py +14 -7
- reachy_mini_ha_voice/reachy_controller.py +98 -409
- reachy_mini_ha_voice/satellite.py +5 -6
- reachy_mini_ha_voice/tap_detector.py +1 -1
- reachy_mini_ha_voice/voice_assistant.py +81 -15
PROJECT_PLAN.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
|
| 3 |
## 项目概述
|
| 4 |
|
| 5 |
-
将 Home Assistant 语音助手功能集成到 Reachy Mini 机器人,通过 ESPHome 协议与 Home Assistant 通信。
|
| 6 |
|
| 7 |
## 本地项目目录参考 (禁止修改参考目录内任何文件)
|
| 8 |
1. [linux-voice-assistant](linux-voice-assistant),这是一个基于 Linux 的Home Assistant的语音助手应用,用于参考。
|
|
@@ -825,6 +825,90 @@ def _get_cached_head_pose(self):
|
|
| 825 |
|
| 826 |
---
|
| 827 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 828 |
## 🔧 拍一拍唤醒与麦克风灵敏度修复 (2026-01-07)
|
| 829 |
|
| 830 |
### 问题描述
|
|
@@ -833,7 +917,7 @@ def _get_cached_head_pose(self):
|
|
| 833 |
|
| 834 |
### 根本原因
|
| 835 |
1. **音频播放阻塞** - `_tap_continue_feedback()` 在持续对话模式下播放提示音,阻塞了音频流处理
|
| 836 |
-
2. **AGC 设置不优化** - ReSpeaker
|
| 837 |
|
| 838 |
### 修复方案
|
| 839 |
|
|
@@ -864,35 +948,72 @@ def _on_tap_detected(self) -> None:
|
|
| 864 |
_LOGGER.error("Error in tap detection callback: %s", e)
|
| 865 |
```
|
| 866 |
|
| 867 |
-
#### 3. 优化麦克风设置 (voice_assistant.py)
|
| 868 |
```python
|
| 869 |
def _optimize_microphone_settings(self) -> None:
|
| 870 |
-
"""Optimize ReSpeaker microphone settings for voice recognition."""
|
| 871 |
-
|
|
|
|
|
|
|
| 872 |
respeaker.write("PP_AGCONOFF", [1])
|
| 873 |
|
| 874 |
-
#
|
| 875 |
-
respeaker.write("PP_AGCMAXGAIN", [
|
|
|
|
|
|
|
|
|
|
| 876 |
|
| 877 |
-
#
|
| 878 |
-
respeaker.write("
|
| 879 |
|
| 880 |
-
#
|
|
|
|
| 881 |
respeaker.write("AUDIO_MGR_MIC_GAIN", [2.0])
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 882 |
```
|
| 883 |
|
| 884 |
### 修复效果
|
| 885 |
|
| 886 |
-
|
|
| 887 |
-
|------|--------|--------|
|
| 888 |
-
| 拍一拍持续对话 | 阻塞
|
| 889 |
-
| 麦克风灵敏度 |
|
| 890 |
-
| AGC
|
| 891 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 892 |
|
| 893 |
### 相关文件
|
| 894 |
- `satellite.py` - 移除阻塞的音频播放
|
| 895 |
-
- `voice_assistant.py` -
|
|
|
|
|
|
|
|
|
|
| 896 |
|
| 897 |
---
|
| 898 |
|
|
|
|
| 2 |
|
| 3 |
## 项目概述
|
| 4 |
|
| 5 |
+
将 Home Assistant 语音助手功能集成到 Reachy Mini Wi-Fi版本机器人,通过 ESPHome 协议与 Home Assistant 通信。
|
| 6 |
|
| 7 |
## 本地项目目录参考 (禁止修改参考目录内任何文件)
|
| 8 |
1. [linux-voice-assistant](linux-voice-assistant),这是一个基于 Linux 的Home Assistant的语音助手应用,用于参考。
|
|
|
|
| 825 |
|
| 826 |
---
|
| 827 |
|
| 828 |
+
## 🔧 Daemon 崩溃问题深度修复 (2026-01-07)
|
| 829 |
+
|
| 830 |
+
### 问题描述
|
| 831 |
+
长期运行过程中,`reachy_mini daemon` 仍然会崩溃,之前的修复不够彻底。
|
| 832 |
+
|
| 833 |
+
### 根本原因分析
|
| 834 |
+
|
| 835 |
+
通过深入分析 SDK 源码发现:
|
| 836 |
+
|
| 837 |
+
1. **每次 `set_target()` 发送 3 条 Zenoh 消息**
|
| 838 |
+
- `set_target_head_pose()` - 1 条消息
|
| 839 |
+
- `set_target_antenna_joint_positions()` - 1 条消息
|
| 840 |
+
- `set_target_body_yaw()` - 1 条消息
|
| 841 |
+
|
| 842 |
+
2. **Daemon 控制循环是 50Hz**
|
| 843 |
+
- 见 `reachy_mini/daemon/backend/robot/backend.py`: `control_loop_frequency = 50.0`
|
| 844 |
+
- 如果消息发送频率超过 50Hz,daemon 可能无法及时处理
|
| 845 |
+
|
| 846 |
+
3. **之前的 20Hz 控制循环仍然过高**
|
| 847 |
+
- 20Hz × 3 消息 = 60 消息/秒
|
| 848 |
+
- 已经超过 daemon 的 50Hz 处理能力
|
| 849 |
+
|
| 850 |
+
4. **姿态变化阈值太小 (0.002)**
|
| 851 |
+
- 呼吸动画、语音摆动、人脸追踪持续产生微小变化
|
| 852 |
+
- 几乎每次循环都会触发 `set_target()`
|
| 853 |
+
|
| 854 |
+
### 修复方案
|
| 855 |
+
|
| 856 |
+
#### 1. 进一步降低控制循环频率 (movement_manager.py)
|
| 857 |
+
```python
|
| 858 |
+
# 从 20Hz 降低到 10Hz
|
| 859 |
+
# 10Hz × 3 消息 = 30 消息/秒,安全低于 daemon 的 50Hz 容量
|
| 860 |
+
CONTROL_LOOP_FREQUENCY_HZ = 10
|
| 861 |
+
```
|
| 862 |
+
|
| 863 |
+
#### 2. 增大姿态变化阈值 (movement_manager.py)
|
| 864 |
+
```python
|
| 865 |
+
# 从 0.002 增大到 0.005
|
| 866 |
+
# 0.005 rad ≈ 0.29 度,仍然足够平滑
|
| 867 |
+
self._pose_change_threshold = 0.005
|
| 868 |
+
```
|
| 869 |
+
|
| 870 |
+
#### 3. 降低摄像头/人脸追踪频率 (camera_server.py)
|
| 871 |
+
```python
|
| 872 |
+
# 从 15fps 降低到 10fps
|
| 873 |
+
fps: int = 10
|
| 874 |
+
```
|
| 875 |
+
|
| 876 |
+
#### 4. 降低 IMU 轮询频率 (tap_detector.py)
|
| 877 |
+
```python
|
| 878 |
+
# 从 50Hz 降低到 20Hz
|
| 879 |
+
TAP_DETECTION_RATE_HZ = 20
|
| 880 |
+
```
|
| 881 |
+
|
| 882 |
+
#### 5. 增大状态缓存 TTL (reachy_controller.py)
|
| 883 |
+
```python
|
| 884 |
+
# 从 1 秒增大到 2 秒
|
| 885 |
+
self._cache_ttl = 2.0
|
| 886 |
+
```
|
| 887 |
+
|
| 888 |
+
### 修复效果
|
| 889 |
+
|
| 890 |
+
| 指标 | 修复前 (20Hz) | 修复后 (10Hz) | 改善 |
|
| 891 |
+
|------|---------------|---------------|------|
|
| 892 |
+
| 控制循环频率 | 20 Hz | 10 Hz | ↓ 50% |
|
| 893 |
+
| 最大 Zenoh 消息 | 60 msg/s | 30 msg/s | ↓ 50% |
|
| 894 |
+
| 实际消息 (有变化检测) | ~40 msg/s | ~15 msg/s | ↓ 62% |
|
| 895 |
+
| 人脸追踪频率 | 15 Hz | 10 Hz | ↓ 33% |
|
| 896 |
+
| IMU 轮询频率 | 50 Hz | 20 Hz | ↓ 60% |
|
| 897 |
+
| 状态缓存 TTL | 1 秒 | 2 秒 | ↑ 100% |
|
| 898 |
+
| 预期稳定性 | 数小时崩溃 | 可稳定运行 | 大幅提升 |
|
| 899 |
+
|
| 900 |
+
### 关键发现
|
| 901 |
+
|
| 902 |
+
参考 `reachy_mini_conversation_app` 使用 100Hz 控制循环,但它是官方应用,可能有特殊优化或在更强硬件上运行。我们的应用需要更保守的设置。
|
| 903 |
+
|
| 904 |
+
### 相关文件
|
| 905 |
+
- `movement_manager.py` - 控制循环频率和姿态阈值
|
| 906 |
+
- `camera_server.py` - 人脸追踪频率
|
| 907 |
+
- `tap_detector.py` - IMU 轮询频率
|
| 908 |
+
- `reachy_controller.py` - 状态缓存 TTL
|
| 909 |
+
|
| 910 |
+
---
|
| 911 |
+
|
| 912 |
## 🔧 拍一拍唤醒与麦克风灵敏度修复 (2026-01-07)
|
| 913 |
|
| 914 |
### 问题描述
|
|
|
|
| 917 |
|
| 918 |
### 根本原因
|
| 919 |
1. **音频播放阻塞** - `_tap_continue_feedback()` 在持续对话模式下播放提示音,阻塞了音频流处理
|
| 920 |
+
2. **AGC 设置不优化** - ReSpeaker XVF3800 的默认设置不适合远距离语音识别
|
| 921 |
|
| 922 |
### 修复方案
|
| 923 |
|
|
|
|
| 948 |
_LOGGER.error("Error in tap detection callback: %s", e)
|
| 949 |
```
|
| 950 |
|
| 951 |
+
#### 3. 全面优化麦克风设置 (voice_assistant.py) - 更新于 2026-01-07
|
| 952 |
```python
|
| 953 |
def _optimize_microphone_settings(self) -> None:
|
| 954 |
+
"""Optimize ReSpeaker XVF3800 microphone settings for voice recognition."""
|
| 955 |
+
|
| 956 |
+
# ========== 1. AGC (Automatic Gain Control) Settings ==========
|
| 957 |
+
# Enable AGC for automatic volume normalization
|
| 958 |
respeaker.write("PP_AGCONOFF", [1])
|
| 959 |
|
| 960 |
+
# Increase AGC max gain for better distant speech pickup (default ~15dB -> 30dB)
|
| 961 |
+
respeaker.write("PP_AGCMAXGAIN", [30.0])
|
| 962 |
+
|
| 963 |
+
# Set AGC desired output level (default ~-25dB -> -18dB for stronger output)
|
| 964 |
+
respeaker.write("PP_AGCDESIREDLEVEL", [-18.0])
|
| 965 |
|
| 966 |
+
# Optimize AGC time constant for voice commands
|
| 967 |
+
respeaker.write("PP_AGCTIME", [0.5])
|
| 968 |
|
| 969 |
+
# ========== 2. Base Microphone Gain ==========
|
| 970 |
+
# Increase base microphone gain (default 1.0 -> 2.0)
|
| 971 |
respeaker.write("AUDIO_MGR_MIC_GAIN", [2.0])
|
| 972 |
+
|
| 973 |
+
# ========== 3. Noise Suppression Settings ==========
|
| 974 |
+
# Reduce noise suppression to preserve quiet speech (default ~0.5 -> 0.15)
|
| 975 |
+
respeaker.write("PP_MIN_NS", [0.15])
|
| 976 |
+
respeaker.write("PP_MIN_NN", [0.15])
|
| 977 |
+
|
| 978 |
+
# ========== 4. Echo Cancellation & High-pass Filter ==========
|
| 979 |
+
respeaker.write("PP_ECHOONOFF", [1])
|
| 980 |
+
respeaker.write("AEC_HPFONOFF", [1])
|
| 981 |
```
|
| 982 |
|
| 983 |
### 修复效果
|
| 984 |
|
| 985 |
+
| 参数 | 修复前 | 修复后 | 说明 |
|
| 986 |
+
|------|--------|--------|------|
|
| 987 |
+
| 拍一拍持续对话 | 阻塞 | 正常工作 | 移除阻塞音频播放 |
|
| 988 |
+
| 麦克风灵敏度 | ~30cm | ~2-3m | 全面优化 AGC 和增益 |
|
| 989 |
+
| AGC 开关 | 关闭 | 开启 | 自动音量归一化 |
|
| 990 |
+
| AGC 最大增益 | ~15dB | 30dB | 提升远距离拾音 |
|
| 991 |
+
| AGC 目标电平 | -25dB | -18dB | 更强输出信号 |
|
| 992 |
+
| 麦克风增益 | 1.0x | 2.0x | 基础增益翻倍 |
|
| 993 |
+
| 噪声抑制 | ~0.5 | 0.15 | 减少对语音的误抑制 |
|
| 994 |
+
| 回声消除 | 开启 | 开启 | 保持 TTS 播放时的清晰度 |
|
| 995 |
+
| 高通滤波 | 关闭 | 开启 | 去除低频噪声 |
|
| 996 |
+
|
| 997 |
+
### XVF3800 参数参考
|
| 998 |
+
|
| 999 |
+
| 参数名 | 类型 | 范围 | 说明 |
|
| 1000 |
+
|--------|------|------|------|
|
| 1001 |
+
| `PP_AGCONOFF` | int32 | 0/1 | AGC 开关 |
|
| 1002 |
+
| `PP_AGCMAXGAIN` | float | 0-40 dB | AGC 最大增益 |
|
| 1003 |
+
| `PP_AGCDESIREDLEVEL` | float | dB | AGC 目标输出电平 |
|
| 1004 |
+
| `PP_AGCTIME` | float | 秒 | AGC 时间常数 |
|
| 1005 |
+
| `AUDIO_MGR_MIC_GAIN` | float | 0-4.0 | 麦克风增益倍数 |
|
| 1006 |
+
| `PP_MIN_NS` | float | 0-1.0 | 最小噪声抑制 (越低越少抑制) |
|
| 1007 |
+
| `PP_MIN_NN` | float | 0-1.0 | 最小噪声估计 |
|
| 1008 |
+
| `PP_ECHOONOFF` | int32 | 0/1 | 回声消除开关 |
|
| 1009 |
+
| `AEC_HPFONOFF` | int32 | 0/1 | 高通滤波开关 |
|
| 1010 |
|
| 1011 |
### 相关文件
|
| 1012 |
- `satellite.py` - 移除阻塞的音频播放
|
| 1013 |
+
- `voice_assistant.py` - 全面麦克风优化
|
| 1014 |
+
- `reachy_controller.py` - AGC 实体默认值更新
|
| 1015 |
+
- `entity_registry.py` - AGC max gain 范围更新 (0-40dB)
|
| 1016 |
+
- `reachy_mini/src/reachy_mini/media/audio_control_utils.py` - SDK 参考
|
| 1017 |
|
| 1018 |
---
|
| 1019 |
|
pyproject.toml
CHANGED
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
| 4 |
|
| 5 |
[project]
|
| 6 |
name = "reachy_mini_ha_voice"
|
| 7 |
-
version = "0.
|
| 8 |
description = "Home Assistant Voice Assistant for Reachy Mini"
|
| 9 |
readme = "README.md"
|
| 10 |
requires-python = ">=3.10"
|
|
|
|
| 4 |
|
| 5 |
[project]
|
| 6 |
name = "reachy_mini_ha_voice"
|
| 7 |
+
version = "0.4.0"
|
| 8 |
description = "Home Assistant Voice Assistant for Reachy Mini"
|
| 9 |
readme = "README.md"
|
| 10 |
requires-python = ">=3.10"
|
reachy_mini_ha_voice/audio_player.py
CHANGED
|
@@ -4,12 +4,8 @@ import logging
|
|
| 4 |
import threading
|
| 5 |
import time
|
| 6 |
from collections.abc import Callable
|
| 7 |
-
from pathlib import Path
|
| 8 |
from typing import List, Optional, Union
|
| 9 |
|
| 10 |
-
import numpy as np
|
| 11 |
-
import scipy.signal
|
| 12 |
-
|
| 13 |
_LOGGER = logging.getLogger(__name__)
|
| 14 |
|
| 15 |
|
|
|
|
| 4 |
import threading
|
| 5 |
import time
|
| 6 |
from collections.abc import Callable
|
|
|
|
| 7 |
from typing import List, Optional, Union
|
| 8 |
|
|
|
|
|
|
|
|
|
|
| 9 |
_LOGGER = logging.getLogger(__name__)
|
| 10 |
|
| 11 |
|
reachy_mini_ha_voice/camera_server.py
CHANGED
|
@@ -4,6 +4,8 @@ MJPEG Camera Server for Reachy Mini with Face Tracking.
|
|
| 4 |
This module provides an HTTP server that streams camera frames from Reachy Mini
|
| 5 |
as MJPEG, which can be integrated with Home Assistant via Generic Camera.
|
| 6 |
Also provides face tracking for head movement control.
|
|
|
|
|
|
|
| 7 |
"""
|
| 8 |
|
| 9 |
import asyncio
|
|
@@ -16,6 +18,13 @@ import cv2
|
|
| 16 |
import numpy as np
|
| 17 |
from scipy.spatial.transform import Rotation as R
|
| 18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
if TYPE_CHECKING:
|
| 20 |
from reachy_mini import ReachyMini
|
| 21 |
|
|
@@ -42,7 +51,7 @@ class MJPEGCameraServer:
|
|
| 42 |
reachy_mini: Optional["ReachyMini"] = None,
|
| 43 |
host: str = "0.0.0.0",
|
| 44 |
port: int = 8081,
|
| 45 |
-
fps: int =
|
| 46 |
quality: int = 80,
|
| 47 |
enable_face_tracking: bool = True,
|
| 48 |
):
|
|
@@ -105,8 +114,7 @@ class MJPEGCameraServer:
|
|
| 105 |
self._head_tracker = HeadTracker()
|
| 106 |
_LOGGER.info("Face tracking enabled with YOLO head tracker")
|
| 107 |
except ImportError as e:
|
| 108 |
-
_LOGGER.
|
| 109 |
-
_LOGGER.warning("Install with: pip install ultralytics supervision huggingface_hub")
|
| 110 |
self._head_tracker = None
|
| 111 |
except Exception as e:
|
| 112 |
_LOGGER.warning("Failed to initialize head tracker: %s", e)
|
|
@@ -307,7 +315,15 @@ class MJPEGCameraServer:
|
|
| 307 |
def _linear_pose_interpolation(
|
| 308 |
self, start: np.ndarray, end: np.ndarray, t: float
|
| 309 |
) -> np.ndarray:
|
| 310 |
-
"""Linear interpolation between two 4x4 pose matrices.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 311 |
# Interpolate translation
|
| 312 |
start_trans = start[:3, 3]
|
| 313 |
end_trans = end[:3, 3]
|
|
@@ -317,9 +333,9 @@ class MJPEGCameraServer:
|
|
| 317 |
start_rot = R.from_matrix(start[:3, :3])
|
| 318 |
end_rot = R.from_matrix(end[:3, :3])
|
| 319 |
|
| 320 |
-
# Use scipy's slerp
|
| 321 |
from scipy.spatial.transform import Slerp
|
| 322 |
-
key_rots = R.
|
| 323 |
slerp = Slerp([0, 1], key_rots)
|
| 324 |
interp_rot = slerp(t)
|
| 325 |
|
|
|
|
| 4 |
This module provides an HTTP server that streams camera frames from Reachy Mini
|
| 5 |
as MJPEG, which can be integrated with Home Assistant via Generic Camera.
|
| 6 |
Also provides face tracking for head movement control.
|
| 7 |
+
|
| 8 |
+
Reference: reachy_mini_conversation_app/src/reachy_mini_conversation_app/camera_worker.py
|
| 9 |
"""
|
| 10 |
|
| 11 |
import asyncio
|
|
|
|
| 18 |
import numpy as np
|
| 19 |
from scipy.spatial.transform import Rotation as R
|
| 20 |
|
| 21 |
+
# Import SDK interpolation utilities (same as conversation_app)
|
| 22 |
+
try:
|
| 23 |
+
from reachy_mini.utils.interpolation import linear_pose_interpolation
|
| 24 |
+
SDK_INTERPOLATION_AVAILABLE = True
|
| 25 |
+
except ImportError:
|
| 26 |
+
SDK_INTERPOLATION_AVAILABLE = False
|
| 27 |
+
|
| 28 |
if TYPE_CHECKING:
|
| 29 |
from reachy_mini import ReachyMini
|
| 30 |
|
|
|
|
| 51 |
reachy_mini: Optional["ReachyMini"] = None,
|
| 52 |
host: str = "0.0.0.0",
|
| 53 |
port: int = 8081,
|
| 54 |
+
fps: int = 10, # Reduced from 15 to 10 fps for daemon stability
|
| 55 |
quality: int = 80,
|
| 56 |
enable_face_tracking: bool = True,
|
| 57 |
):
|
|
|
|
| 114 |
self._head_tracker = HeadTracker()
|
| 115 |
_LOGGER.info("Face tracking enabled with YOLO head tracker")
|
| 116 |
except ImportError as e:
|
| 117 |
+
_LOGGER.error("Failed to import head tracker: %s", e)
|
|
|
|
| 118 |
self._head_tracker = None
|
| 119 |
except Exception as e:
|
| 120 |
_LOGGER.warning("Failed to initialize head tracker: %s", e)
|
|
|
|
| 315 |
def _linear_pose_interpolation(
|
| 316 |
self, start: np.ndarray, end: np.ndarray, t: float
|
| 317 |
) -> np.ndarray:
|
| 318 |
+
"""Linear interpolation between two 4x4 pose matrices.
|
| 319 |
+
|
| 320 |
+
Uses SDK's linear_pose_interpolation if available, otherwise falls back
|
| 321 |
+
to manual SLERP implementation.
|
| 322 |
+
"""
|
| 323 |
+
if SDK_INTERPOLATION_AVAILABLE:
|
| 324 |
+
return linear_pose_interpolation(start, end, t)
|
| 325 |
+
|
| 326 |
+
# Fallback: manual interpolation
|
| 327 |
# Interpolate translation
|
| 328 |
start_trans = start[:3, 3]
|
| 329 |
end_trans = end[:3, 3]
|
|
|
|
| 333 |
start_rot = R.from_matrix(start[:3, :3])
|
| 334 |
end_rot = R.from_matrix(end[:3, :3])
|
| 335 |
|
| 336 |
+
# Use scipy's slerp - create Rotation array from list
|
| 337 |
from scipy.spatial.transform import Slerp
|
| 338 |
+
key_rots = R.from_quat(np.array([start_rot.as_quat(), end_rot.as_quat()]))
|
| 339 |
slerp = Slerp([0, 1], key_rots)
|
| 340 |
interp_rot = slerp(t)
|
| 341 |
|
reachy_mini_ha_voice/entity_registry.py
CHANGED
|
@@ -694,7 +694,7 @@ class EntityRegistry:
|
|
| 694 |
name="AGC Max Gain",
|
| 695 |
object_id="agc_max_gain",
|
| 696 |
min_value=0.0,
|
| 697 |
-
max_value=
|
| 698 |
step=1.0,
|
| 699 |
icon="mdi:volume-plus",
|
| 700 |
unit_of_measurement="dB",
|
|
|
|
| 694 |
name="AGC Max Gain",
|
| 695 |
object_id="agc_max_gain",
|
| 696 |
min_value=0.0,
|
| 697 |
+
max_value=40.0, # XVF3800 supports up to 40dB
|
| 698 |
step=1.0,
|
| 699 |
icon="mdi:volume-plus",
|
| 700 |
unit_of_measurement="dB",
|
reachy_mini_ha_voice/head_tracker.py
CHANGED
|
@@ -1,6 +1,8 @@
|
|
| 1 |
"""Lightweight head tracker using YOLO for face detection.
|
| 2 |
|
| 3 |
Ported from reachy_mini_conversation_app for voice assistant integration.
|
|
|
|
|
|
|
| 4 |
"""
|
| 5 |
|
| 6 |
from __future__ import annotations
|
|
@@ -13,29 +15,13 @@ from numpy.typing import NDArray
|
|
| 13 |
|
| 14 |
logger = logging.getLogger(__name__)
|
| 15 |
|
| 16 |
-
# Lazy imports to avoid startup delay
|
| 17 |
-
_YOLO = None
|
| 18 |
-
_Detections = None
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
def _load_yolo_deps():
|
| 22 |
-
"""Lazy load YOLO dependencies."""
|
| 23 |
-
global _YOLO, _Detections
|
| 24 |
-
if _YOLO is None:
|
| 25 |
-
try:
|
| 26 |
-
from ultralytics import YOLO
|
| 27 |
-
from supervision import Detections
|
| 28 |
-
_YOLO = YOLO
|
| 29 |
-
_Detections = Detections
|
| 30 |
-
except ImportError as e:
|
| 31 |
-
raise ImportError(
|
| 32 |
-
"To use head tracker, install: pip install ultralytics supervision huggingface_hub"
|
| 33 |
-
) from e
|
| 34 |
-
return _YOLO, _Detections
|
| 35 |
-
|
| 36 |
|
| 37 |
class HeadTracker:
|
| 38 |
-
"""Lightweight head tracker using YOLO for face detection.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
def __init__(
|
| 41 |
self,
|
|
@@ -57,29 +43,50 @@ class HeadTracker:
|
|
| 57 |
self._model_repo = model_repo
|
| 58 |
self._model_filename = model_filename
|
| 59 |
self._device = device
|
| 60 |
-
self.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
|
| 62 |
-
def
|
| 63 |
-
"""
|
| 64 |
-
if self.
|
| 65 |
-
return
|
|
|
|
|
|
|
| 66 |
|
| 67 |
-
self._initialized = True
|
| 68 |
try:
|
| 69 |
-
|
|
|
|
| 70 |
from huggingface_hub import hf_hub_download
|
| 71 |
|
|
|
|
|
|
|
| 72 |
model_path = hf_hub_download(
|
| 73 |
repo_id=self._model_repo,
|
| 74 |
filename=self._model_filename
|
| 75 |
)
|
| 76 |
self.model = YOLO(model_path).to(self._device)
|
| 77 |
-
logger.info(
|
| 78 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
except Exception as e:
|
| 80 |
-
|
|
|
|
| 81 |
self.model = None
|
| 82 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
|
| 84 |
def _select_best_face(self, detections) -> Optional[int]:
|
| 85 |
"""Select the best face based on confidence and area.
|
|
@@ -147,17 +154,15 @@ class HeadTracker:
|
|
| 147 |
Returns:
|
| 148 |
Tuple of (face_center [-1,1], confidence) or (None, None) if no face
|
| 149 |
"""
|
| 150 |
-
if not self.
|
| 151 |
return None, None
|
| 152 |
|
| 153 |
-
_, Detections = _load_yolo_deps()
|
| 154 |
-
|
| 155 |
h, w = img.shape[:2]
|
| 156 |
|
| 157 |
try:
|
| 158 |
# Run YOLO inference
|
| 159 |
results = self.model(img, verbose=False)
|
| 160 |
-
detections =
|
| 161 |
|
| 162 |
# Select best face
|
| 163 |
face_idx = self._select_best_face(detections)
|
|
@@ -175,5 +180,5 @@ class HeadTracker:
|
|
| 175 |
return face_center, confidence
|
| 176 |
|
| 177 |
except Exception as e:
|
| 178 |
-
logger.
|
| 179 |
return None, None
|
|
|
|
| 1 |
"""Lightweight head tracker using YOLO for face detection.
|
| 2 |
|
| 3 |
Ported from reachy_mini_conversation_app for voice assistant integration.
|
| 4 |
+
Model is loaded at initialization time (not lazy) to ensure face tracking
|
| 5 |
+
is ready immediately when the camera server starts.
|
| 6 |
"""
|
| 7 |
|
| 8 |
from __future__ import annotations
|
|
|
|
| 15 |
|
| 16 |
logger = logging.getLogger(__name__)
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
class HeadTracker:
|
| 20 |
+
"""Lightweight head tracker using YOLO for face detection.
|
| 21 |
+
|
| 22 |
+
Model is loaded at initialization time to ensure face tracking
|
| 23 |
+
is ready immediately (matching conversation_app behavior).
|
| 24 |
+
"""
|
| 25 |
|
| 26 |
def __init__(
|
| 27 |
self,
|
|
|
|
| 43 |
self._model_repo = model_repo
|
| 44 |
self._model_filename = model_filename
|
| 45 |
self._device = device
|
| 46 |
+
self._detections_class = None
|
| 47 |
+
self._model_load_attempted = False
|
| 48 |
+
self._model_load_error: Optional[str] = None
|
| 49 |
+
|
| 50 |
+
# Load model immediately at init (not lazy)
|
| 51 |
+
self._load_model()
|
| 52 |
|
| 53 |
+
def _load_model(self) -> None:
|
| 54 |
+
"""Load YOLO model at initialization time."""
|
| 55 |
+
if self._model_load_attempted:
|
| 56 |
+
return
|
| 57 |
+
|
| 58 |
+
self._model_load_attempted = True
|
| 59 |
|
|
|
|
| 60 |
try:
|
| 61 |
+
from ultralytics import YOLO
|
| 62 |
+
from supervision import Detections
|
| 63 |
from huggingface_hub import hf_hub_download
|
| 64 |
|
| 65 |
+
self._detections_class = Detections
|
| 66 |
+
|
| 67 |
model_path = hf_hub_download(
|
| 68 |
repo_id=self._model_repo,
|
| 69 |
filename=self._model_filename
|
| 70 |
)
|
| 71 |
self.model = YOLO(model_path).to(self._device)
|
| 72 |
+
logger.info("YOLO face detection model loaded from %s", self._model_repo)
|
| 73 |
+
except ImportError as e:
|
| 74 |
+
self._model_load_error = f"Missing dependencies: {e}"
|
| 75 |
+
logger.warning(
|
| 76 |
+
"Face tracking disabled - missing dependencies: %s. "
|
| 77 |
+
"Install with: pip install ultralytics supervision huggingface_hub",
|
| 78 |
+
e
|
| 79 |
+
)
|
| 80 |
+
self.model = None
|
| 81 |
except Exception as e:
|
| 82 |
+
self._model_load_error = str(e)
|
| 83 |
+
logger.error("Failed to load YOLO model: %s", e)
|
| 84 |
self.model = None
|
| 85 |
+
|
| 86 |
+
@property
|
| 87 |
+
def is_available(self) -> bool:
|
| 88 |
+
"""Check if the head tracker is available and ready."""
|
| 89 |
+
return self.model is not None and self._detections_class is not None
|
| 90 |
|
| 91 |
def _select_best_face(self, detections) -> Optional[int]:
|
| 92 |
"""Select the best face based on confidence and area.
|
|
|
|
| 154 |
Returns:
|
| 155 |
Tuple of (face_center [-1,1], confidence) or (None, None) if no face
|
| 156 |
"""
|
| 157 |
+
if not self.is_available:
|
| 158 |
return None, None
|
| 159 |
|
|
|
|
|
|
|
| 160 |
h, w = img.shape[:2]
|
| 161 |
|
| 162 |
try:
|
| 163 |
# Run YOLO inference
|
| 164 |
results = self.model(img, verbose=False)
|
| 165 |
+
detections = self._detections_class.from_ultralytics(results[0])
|
| 166 |
|
| 167 |
# Select best face
|
| 168 |
face_idx = self._select_best_face(detections)
|
|
|
|
| 180 |
return face_center, confidence
|
| 181 |
|
| 182 |
except Exception as e:
|
| 183 |
+
logger.debug("Error in head position detection: %s", e)
|
| 184 |
return None, None
|
reachy_mini_ha_voice/motion.py
CHANGED
|
@@ -26,18 +26,18 @@ class ReachyMiniMotion:
|
|
| 26 |
self._camera_server = None # Reference to camera server for face tracking control
|
| 27 |
self._is_speaking = False
|
| 28 |
|
| 29 |
-
_LOGGER.
|
| 30 |
|
| 31 |
# Initialize movement manager if robot is available
|
| 32 |
if reachy_mini is not None:
|
| 33 |
try:
|
| 34 |
self._movement_manager = MovementManager(reachy_mini)
|
| 35 |
-
_LOGGER.
|
| 36 |
except Exception as e:
|
| 37 |
_LOGGER.error("Failed to create MovementManager: %s", e, exc_info=True)
|
| 38 |
self._movement_manager = None
|
| 39 |
else:
|
| 40 |
-
_LOGGER.
|
| 41 |
|
| 42 |
def set_reachy_mini(self, reachy_mini):
|
| 43 |
"""Set the Reachy Mini instance."""
|
|
@@ -62,9 +62,9 @@ class ReachyMiniMotion:
|
|
| 62 |
"""Start the movement manager control loop."""
|
| 63 |
if self._movement_manager is not None:
|
| 64 |
self._movement_manager.start()
|
| 65 |
-
_LOGGER.
|
| 66 |
else:
|
| 67 |
-
_LOGGER.warning("Motion control not started: movement_manager is None
|
| 68 |
|
| 69 |
def shutdown(self):
|
| 70 |
"""Shutdown the motion controller."""
|
|
@@ -257,33 +257,3 @@ class ReachyMiniMotion:
|
|
| 257 |
"""
|
| 258 |
if self._movement_manager is not None:
|
| 259 |
self._movement_manager.update_audio_loudness(loudness_db)
|
| 260 |
-
|
| 261 |
-
# -------------------------------------------------------------------------
|
| 262 |
-
# Legacy compatibility methods (deprecated, use MovementManager directly)
|
| 263 |
-
# -------------------------------------------------------------------------
|
| 264 |
-
|
| 265 |
-
def _nod(self, count: int = 1, amplitude: float = 15, duration: float = 0.5):
|
| 266 |
-
"""Nod head up and down (legacy)."""
|
| 267 |
-
if self._movement_manager is None:
|
| 268 |
-
return
|
| 269 |
-
for _ in range(count):
|
| 270 |
-
self._movement_manager.nod(amplitude_deg=amplitude, duration=duration)
|
| 271 |
-
|
| 272 |
-
def _shake(self, count: int = 1, amplitude: float = 20, duration: float = 0.5):
|
| 273 |
-
"""Shake head left and right (legacy)."""
|
| 274 |
-
if self._movement_manager is None:
|
| 275 |
-
return
|
| 276 |
-
for _ in range(count):
|
| 277 |
-
self._movement_manager.shake(amplitude_deg=amplitude, duration=duration)
|
| 278 |
-
|
| 279 |
-
def _look_at_user(self):
|
| 280 |
-
"""Look at user (legacy)."""
|
| 281 |
-
if self._movement_manager is None:
|
| 282 |
-
return
|
| 283 |
-
self._movement_manager.reset_to_neutral(duration=0.3)
|
| 284 |
-
|
| 285 |
-
def _return_to_neutral(self):
|
| 286 |
-
"""Return to neutral position (legacy)."""
|
| 287 |
-
if self._movement_manager is None:
|
| 288 |
-
return
|
| 289 |
-
self._movement_manager.reset_to_neutral(duration=0.5)
|
|
|
|
| 26 |
self._camera_server = None # Reference to camera server for face tracking control
|
| 27 |
self._is_speaking = False
|
| 28 |
|
| 29 |
+
_LOGGER.debug("ReachyMiniMotion.__init__ called with reachy_mini=%s", reachy_mini)
|
| 30 |
|
| 31 |
# Initialize movement manager if robot is available
|
| 32 |
if reachy_mini is not None:
|
| 33 |
try:
|
| 34 |
self._movement_manager = MovementManager(reachy_mini)
|
| 35 |
+
_LOGGER.debug("MovementManager created successfully")
|
| 36 |
except Exception as e:
|
| 37 |
_LOGGER.error("Failed to create MovementManager: %s", e, exc_info=True)
|
| 38 |
self._movement_manager = None
|
| 39 |
else:
|
| 40 |
+
_LOGGER.debug("reachy_mini is None, MovementManager not created")
|
| 41 |
|
| 42 |
def set_reachy_mini(self, reachy_mini):
|
| 43 |
"""Set the Reachy Mini instance."""
|
|
|
|
| 62 |
"""Start the movement manager control loop."""
|
| 63 |
if self._movement_manager is not None:
|
| 64 |
self._movement_manager.start()
|
| 65 |
+
_LOGGER.info("Motion control started")
|
| 66 |
else:
|
| 67 |
+
_LOGGER.warning("Motion control not started: movement_manager is None")
|
| 68 |
|
| 69 |
def shutdown(self):
|
| 70 |
"""Shutdown the motion controller."""
|
|
|
|
| 257 |
"""
|
| 258 |
if self._movement_manager is not None:
|
| 259 |
self._movement_manager.update_audio_loudness(loudness_db)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
reachy_mini_ha_voice/movement_manager.py
CHANGED
|
@@ -40,6 +40,8 @@ from scipy.spatial.transform import Rotation as R
|
|
| 40 |
if TYPE_CHECKING:
|
| 41 |
from reachy_mini import ReachyMini
|
| 42 |
|
|
|
|
|
|
|
| 43 |
# Import SDK utilities for pose composition (same as conversation_app)
|
| 44 |
try:
|
| 45 |
from reachy_mini.utils import create_head_pose
|
|
@@ -49,14 +51,17 @@ except ImportError:
|
|
| 49 |
SDK_UTILS_AVAILABLE = False
|
| 50 |
logger.warning("SDK utils not available, using fallback pose composition")
|
| 51 |
|
| 52 |
-
logger = logging.getLogger(__name__)
|
| 53 |
-
|
| 54 |
|
| 55 |
# =============================================================================
|
| 56 |
# Constants (borrowed from conversation_app)
|
| 57 |
# =============================================================================
|
| 58 |
|
| 59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
# SDK's get_current_head_pose() and get_current_joint_positions() are non-blocking
|
| 61 |
# (they return cached Zenoh data), so higher frequency is safe.
|
| 62 |
# Using 20Hz as a balance between responsiveness and stability.
|
|
@@ -371,10 +376,11 @@ class MovementManager:
|
|
| 371 |
self._audio_lock = threading.Lock()
|
| 372 |
|
| 373 |
# Pose change detection threshold
|
| 374 |
-
# 0.002
|
| 375 |
-
#
|
|
|
|
| 376 |
self._last_sent_pose: Optional[Dict[str, float]] = None
|
| 377 |
-
self._pose_change_threshold = 0.
|
| 378 |
|
| 379 |
# Face tracking offsets (from camera worker)
|
| 380 |
self._face_tracking_offsets: Tuple[float, float, float, float, float, float] = (0.0, 0.0, 0.0, 0.0, 0.0, 0.0)
|
|
@@ -821,9 +827,10 @@ class MovementManager:
|
|
| 821 |
|
| 822 |
try:
|
| 823 |
# Build head pose matrix
|
|
|
|
| 824 |
rotation = R.from_euler('xyz', [
|
|
|
|
| 825 |
pose["pitch"],
|
| 826 |
-
pose["roll"], # Note: SDK uses different order
|
| 827 |
pose["yaw"],
|
| 828 |
])
|
| 829 |
|
|
|
|
| 40 |
if TYPE_CHECKING:
|
| 41 |
from reachy_mini import ReachyMini
|
| 42 |
|
| 43 |
+
logger = logging.getLogger(__name__)
|
| 44 |
+
|
| 45 |
# Import SDK utilities for pose composition (same as conversation_app)
|
| 46 |
try:
|
| 47 |
from reachy_mini.utils import create_head_pose
|
|
|
|
| 51 |
SDK_UTILS_AVAILABLE = False
|
| 52 |
logger.warning("SDK utils not available, using fallback pose composition")
|
| 53 |
|
|
|
|
|
|
|
| 54 |
|
| 55 |
# =============================================================================
|
| 56 |
# Constants (borrowed from conversation_app)
|
| 57 |
# =============================================================================
|
| 58 |
|
| 59 |
+
# Control loop frequency - CRITICAL for daemon stability
|
| 60 |
+
# The daemon's internal control loop runs at 50Hz.
|
| 61 |
+
# We use 10Hz to stay well below daemon capacity while maintaining smooth motion.
|
| 62 |
+
# Each set_target() call sends 3 Zenoh messages (head, antennas, body_yaw).
|
| 63 |
+
# At 10Hz × 3 = 30 messages/second, well within daemon's 50Hz capacity.
|
| 64 |
+
CONTROL_LOOP_FREQUENCY_HZ = 10 # 10Hz control loop (reduced from 20Hz for stability)
|
| 65 |
# SDK's get_current_head_pose() and get_current_joint_positions() are non-blocking
|
| 66 |
# (they return cached Zenoh data), so higher frequency is safe.
|
| 67 |
# Using 20Hz as a balance between responsiveness and stability.
|
|
|
|
| 376 |
self._audio_lock = threading.Lock()
|
| 377 |
|
| 378 |
# Pose change detection threshold
|
| 379 |
+
# Increased from 0.002 to 0.005 to reduce unnecessary set_target() calls
|
| 380 |
+
# 0.005 rad ≈ 0.29 degrees - still smooth enough for natural motion
|
| 381 |
+
# This helps reduce Zenoh message traffic to the daemon
|
| 382 |
self._last_sent_pose: Optional[Dict[str, float]] = None
|
| 383 |
+
self._pose_change_threshold = 0.005
|
| 384 |
|
| 385 |
# Face tracking offsets (from camera worker)
|
| 386 |
self._face_tracking_offsets: Tuple[float, float, float, float, float, float] = (0.0, 0.0, 0.0, 0.0, 0.0, 0.0)
|
|
|
|
| 827 |
|
| 828 |
try:
|
| 829 |
# Build head pose matrix
|
| 830 |
+
# SDK uses 'xyz' euler order with [roll, pitch, yaw]
|
| 831 |
rotation = R.from_euler('xyz', [
|
| 832 |
+
pose["roll"],
|
| 833 |
pose["pitch"],
|
|
|
|
| 834 |
pose["yaw"],
|
| 835 |
])
|
| 836 |
|
reachy_mini_ha_voice/reachy_controller.py
CHANGED
|
@@ -52,7 +52,7 @@ class ReachyController:
|
|
| 52 |
# Note: get_current_head_pose() and get_current_joint_positions() are
|
| 53 |
# non-blocking in the SDK (they return cached Zenoh data), so no caching needed
|
| 54 |
self._state_cache: Dict[str, Any] = {}
|
| 55 |
-
self._cache_ttl =
|
| 56 |
self._last_status_query = 0.0
|
| 57 |
|
| 58 |
# Thread lock for ReSpeaker USB access to prevent conflicts with GStreamer audio pipeline
|
|
@@ -380,185 +380,89 @@ class ReachyController:
|
|
| 380 |
|
| 381 |
return x, y, z, roll, pitch, yaw
|
| 382 |
|
| 383 |
-
def
|
| 384 |
-
"""Get
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 385 |
pose = self._get_head_pose()
|
| 386 |
if pose is None:
|
| 387 |
return 0.0
|
| 388 |
try:
|
| 389 |
x, y, z, roll, pitch, yaw = self._extract_pose_from_matrix(pose)
|
| 390 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 391 |
except Exception as e:
|
| 392 |
-
logger.error(f"Error getting head
|
| 393 |
return 0.0
|
| 394 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 395 |
def set_head_x(self, x_mm: float) -> None:
|
| 396 |
-
"""
|
| 397 |
-
|
| 398 |
-
NOTE: Disabled to prevent conflict with MovementManager's control loop.
|
| 399 |
-
The MovementManager handles all head pose control during voice conversations.
|
| 400 |
-
"""
|
| 401 |
-
logger.warning("set_head_x is disabled - MovementManager controls head pose")
|
| 402 |
-
# if not self.is_available:
|
| 403 |
-
# return
|
| 404 |
-
# try:
|
| 405 |
-
# pose = self.reachy.get_current_head_pose()
|
| 406 |
-
# # Modify the X position in the matrix
|
| 407 |
-
# new_pose = pose.copy()
|
| 408 |
-
# new_pose[0, 3] = x_mm / 1000 # Convert mm to m
|
| 409 |
-
# self.reachy.goto_target(head=new_pose)
|
| 410 |
-
# except Exception as e:
|
| 411 |
-
# logger.error(f"Error setting head X: {e}")
|
| 412 |
|
| 413 |
def get_head_y(self) -> float:
|
| 414 |
-
"""Get head Y position in mm
|
| 415 |
-
|
| 416 |
-
if pose is None:
|
| 417 |
-
return 0.0
|
| 418 |
-
try:
|
| 419 |
-
x, y, z, roll, pitch, yaw = self._extract_pose_from_matrix(pose)
|
| 420 |
-
return y * 1000
|
| 421 |
-
except Exception as e:
|
| 422 |
-
logger.error(f"Error getting head Y: {e}")
|
| 423 |
-
return 0.0
|
| 424 |
|
| 425 |
def set_head_y(self, y_mm: float) -> None:
|
| 426 |
-
"""
|
| 427 |
-
|
| 428 |
-
NOTE: Disabled to prevent conflict with MovementManager's control loop.
|
| 429 |
-
"""
|
| 430 |
-
logger.warning("set_head_y is disabled - MovementManager controls head pose")
|
| 431 |
-
# if not self.is_available:
|
| 432 |
-
# return
|
| 433 |
-
# try:
|
| 434 |
-
# pose = self.reachy.get_current_head_pose()
|
| 435 |
-
# new_pose = pose.copy()
|
| 436 |
-
# new_pose[1, 3] = y_mm / 1000
|
| 437 |
-
# self.reachy.goto_target(head=new_pose)
|
| 438 |
-
# except Exception as e:
|
| 439 |
-
# logger.error(f"Error setting head Y: {e}")
|
| 440 |
|
| 441 |
def get_head_z(self) -> float:
|
| 442 |
-
"""Get head Z position in mm
|
| 443 |
-
|
| 444 |
-
if pose is None:
|
| 445 |
-
return 0.0
|
| 446 |
-
try:
|
| 447 |
-
x, y, z, roll, pitch, yaw = self._extract_pose_from_matrix(pose)
|
| 448 |
-
return z * 1000
|
| 449 |
-
except Exception as e:
|
| 450 |
-
logger.error(f"Error getting head Z: {e}")
|
| 451 |
-
return 0.0
|
| 452 |
|
| 453 |
def set_head_z(self, z_mm: float) -> None:
|
| 454 |
-
"""
|
| 455 |
-
|
| 456 |
-
NOTE: Disabled to prevent conflict with MovementManager's control loop.
|
| 457 |
-
"""
|
| 458 |
-
logger.warning("set_head_z is disabled - MovementManager controls head pose")
|
| 459 |
-
# if not self.is_available:
|
| 460 |
-
# return
|
| 461 |
-
# try:
|
| 462 |
-
# pose = self.reachy.get_current_head_pose()
|
| 463 |
-
# new_pose = pose.copy()
|
| 464 |
-
# new_pose[2, 3] = z_mm / 1000
|
| 465 |
-
# self.reachy.goto_target(head=new_pose)
|
| 466 |
-
# except Exception as e:
|
| 467 |
-
# logger.error(f"Error setting head Z: {e}")
|
| 468 |
|
|
|
|
| 469 |
def get_head_roll(self) -> float:
|
| 470 |
-
"""Get head roll angle in degrees
|
| 471 |
-
|
| 472 |
-
if pose is None:
|
| 473 |
-
return 0.0
|
| 474 |
-
try:
|
| 475 |
-
x, y, z, roll, pitch, yaw = self._extract_pose_from_matrix(pose)
|
| 476 |
-
return math.degrees(roll)
|
| 477 |
-
except Exception as e:
|
| 478 |
-
logger.error(f"Error getting head roll: {e}")
|
| 479 |
-
return 0.0
|
| 480 |
|
| 481 |
def set_head_roll(self, roll_deg: float) -> None:
|
| 482 |
-
"""
|
| 483 |
-
|
| 484 |
-
NOTE: Disabled to prevent conflict with MovementManager's control loop.
|
| 485 |
-
"""
|
| 486 |
-
logger.warning("set_head_roll is disabled - MovementManager controls head pose")
|
| 487 |
-
# if not self.is_available:
|
| 488 |
-
# return
|
| 489 |
-
# try:
|
| 490 |
-
# pose = self.reachy.get_current_head_pose()
|
| 491 |
-
# x, y, z, roll, pitch, yaw = self._extract_pose_from_matrix(pose)
|
| 492 |
-
# # Create new rotation with updated roll
|
| 493 |
-
# new_rotation = R.from_euler('xyz', [math.radians(roll_deg), pitch, yaw])
|
| 494 |
-
# new_pose = pose.copy()
|
| 495 |
-
# new_pose[:3, :3] = new_rotation.as_matrix()
|
| 496 |
-
# self.reachy.goto_target(head=new_pose)
|
| 497 |
-
# except Exception as e:
|
| 498 |
-
# logger.error(f"Error setting head roll: {e}")
|
| 499 |
|
| 500 |
def get_head_pitch(self) -> float:
|
| 501 |
-
"""Get head pitch angle in degrees
|
| 502 |
-
|
| 503 |
-
if pose is None:
|
| 504 |
-
return 0.0
|
| 505 |
-
try:
|
| 506 |
-
x, y, z, roll, pitch, yaw = self._extract_pose_from_matrix(pose)
|
| 507 |
-
return math.degrees(pitch)
|
| 508 |
-
except Exception as e:
|
| 509 |
-
logger.error(f"Error getting head pitch: {e}")
|
| 510 |
-
return 0.0
|
| 511 |
|
| 512 |
def set_head_pitch(self, pitch_deg: float) -> None:
|
| 513 |
-
"""
|
| 514 |
-
|
| 515 |
-
NOTE: Disabled to prevent conflict with MovementManager's control loop.
|
| 516 |
-
"""
|
| 517 |
-
logger.warning("set_head_pitch is disabled - MovementManager controls head pose")
|
| 518 |
-
# if not self.is_available:
|
| 519 |
-
# return
|
| 520 |
-
# try:
|
| 521 |
-
# pose = self.reachy.get_current_head_pose()
|
| 522 |
-
# x, y, z, roll, pitch, yaw = self._extract_pose_from_matrix(pose)
|
| 523 |
-
# new_rotation = R.from_euler('xyz', [roll, math.radians(pitch_deg), yaw])
|
| 524 |
-
# new_pose = pose.copy()
|
| 525 |
-
# new_pose[:3, :3] = new_rotation.as_matrix()
|
| 526 |
-
# self.reachy.goto_target(head=new_pose)
|
| 527 |
-
# except Exception as e:
|
| 528 |
-
# logger.error(f"Error setting head pitch: {e}")
|
| 529 |
|
| 530 |
def get_head_yaw(self) -> float:
|
| 531 |
-
"""Get head yaw angle in degrees
|
| 532 |
-
|
| 533 |
-
if pose is None:
|
| 534 |
-
return 0.0
|
| 535 |
-
try:
|
| 536 |
-
x, y, z, roll, pitch, yaw = self._extract_pose_from_matrix(pose)
|
| 537 |
-
return math.degrees(yaw)
|
| 538 |
-
except Exception as e:
|
| 539 |
-
logger.error(f"Error getting head yaw: {e}")
|
| 540 |
-
return 0.0
|
| 541 |
|
| 542 |
def set_head_yaw(self, yaw_deg: float) -> None:
|
| 543 |
-
"""
|
| 544 |
-
|
| 545 |
-
NOTE: Disabled to prevent conflict with MovementManager's control loop.
|
| 546 |
-
"""
|
| 547 |
-
logger.warning("set_head_yaw is disabled - MovementManager controls head pose")
|
| 548 |
-
# if not self.is_available:
|
| 549 |
-
# return
|
| 550 |
-
# try:
|
| 551 |
-
# pose = self.reachy.get_current_head_pose()
|
| 552 |
-
# x, y, z, roll, pitch, yaw = self._extract_pose_from_matrix(pose)
|
| 553 |
-
# new_rotation = R.from_euler('xyz', [roll, pitch, math.radians(yaw_deg)])
|
| 554 |
-
# new_pose = pose.copy()
|
| 555 |
-
# new_pose[:3, :3] = new_rotation.as_matrix()
|
| 556 |
-
# self.reachy.goto_target(head=new_pose)
|
| 557 |
-
# except Exception as e:
|
| 558 |
-
# logger.error(f"Error setting head yaw: {e}")
|
| 559 |
|
| 560 |
def get_body_yaw(self) -> float:
|
| 561 |
-
"""Get body yaw angle in degrees
|
| 562 |
joints = self._get_joint_positions()
|
| 563 |
if joints is None:
|
| 564 |
return 0.0
|
|
@@ -570,20 +474,11 @@ class ReachyController:
|
|
| 570 |
return 0.0
|
| 571 |
|
| 572 |
def set_body_yaw(self, yaw_deg: float) -> None:
|
| 573 |
-
"""
|
| 574 |
-
|
| 575 |
-
NOTE: Disabled to prevent conflict with MovementManager's control loop.
|
| 576 |
-
"""
|
| 577 |
-
logger.warning("set_body_yaw is disabled - MovementManager controls body pose")
|
| 578 |
-
# if not self.is_available:
|
| 579 |
-
# return
|
| 580 |
-
# try:
|
| 581 |
-
# self.reachy.goto_target(body_yaw=math.radians(yaw_deg))
|
| 582 |
-
# except Exception as e:
|
| 583 |
-
# logger.error(f"Error setting body yaw: {e}")
|
| 584 |
|
| 585 |
def get_antenna_left(self) -> float:
|
| 586 |
-
"""Get left antenna angle in degrees
|
| 587 |
joints = self._get_joint_positions()
|
| 588 |
if joints is None:
|
| 589 |
return 0.0
|
|
@@ -595,22 +490,11 @@ class ReachyController:
|
|
| 595 |
return 0.0
|
| 596 |
|
| 597 |
def set_antenna_left(self, angle_deg: float) -> None:
|
| 598 |
-
"""
|
| 599 |
-
|
| 600 |
-
NOTE: Disabled to prevent conflict with MovementManager's control loop.
|
| 601 |
-
"""
|
| 602 |
-
logger.warning("set_antenna_left is disabled - MovementManager controls antennas")
|
| 603 |
-
# if not self.is_available:
|
| 604 |
-
# return
|
| 605 |
-
# try:
|
| 606 |
-
# _, antennas = self.reachy.get_current_joint_positions()
|
| 607 |
-
# right = antennas[0]
|
| 608 |
-
# self.reachy.goto_target(antennas=[right, math.radians(angle_deg)])
|
| 609 |
-
# except Exception as e:
|
| 610 |
-
# logger.error(f"Error setting left antenna: {e}")
|
| 611 |
|
| 612 |
def get_antenna_right(self) -> float:
|
| 613 |
-
"""Get right antenna angle in degrees
|
| 614 |
joints = self._get_joint_positions()
|
| 615 |
if joints is None:
|
| 616 |
return 0.0
|
|
@@ -622,19 +506,8 @@ class ReachyController:
|
|
| 622 |
return 0.0
|
| 623 |
|
| 624 |
def set_antenna_right(self, angle_deg: float) -> None:
|
| 625 |
-
"""
|
| 626 |
-
|
| 627 |
-
NOTE: Disabled to prevent conflict with MovementManager's control loop.
|
| 628 |
-
"""
|
| 629 |
-
logger.warning("set_antenna_right is disabled - MovementManager controls antennas")
|
| 630 |
-
# if not self.is_available:
|
| 631 |
-
# return
|
| 632 |
-
# try:
|
| 633 |
-
# _, antennas = self.reachy.get_current_joint_positions()
|
| 634 |
-
# left = antennas[1]
|
| 635 |
-
# self.reachy.goto_target(antennas=[math.radians(angle_deg), left])
|
| 636 |
-
# except Exception as e:
|
| 637 |
-
# logger.error(f"Error setting right antenna: {e}")
|
| 638 |
|
| 639 |
# ========== Phase 4: Look At Control ==========
|
| 640 |
|
|
@@ -738,98 +611,59 @@ class ReachyController:
|
|
| 738 |
|
| 739 |
# ========== Phase 7: IMU Sensors (Wireless only) ==========
|
| 740 |
|
| 741 |
-
def
|
| 742 |
-
"""Get
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 743 |
if not self.is_available:
|
| 744 |
return 0.0
|
| 745 |
try:
|
| 746 |
imu_data = self.reachy.imu
|
| 747 |
-
if imu_data is
|
| 748 |
-
return
|
| 749 |
-
|
|
|
|
| 750 |
except Exception as e:
|
| 751 |
-
logger.
|
| 752 |
return 0.0
|
| 753 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 754 |
def get_imu_accel_y(self) -> float:
|
| 755 |
"""Get IMU Y-axis acceleration in m/s²."""
|
| 756 |
-
|
| 757 |
-
return 0.0
|
| 758 |
-
try:
|
| 759 |
-
imu_data = self.reachy.imu
|
| 760 |
-
if imu_data is not None and 'accelerometer' in imu_data:
|
| 761 |
-
return float(imu_data['accelerometer'][1])
|
| 762 |
-
return 0.0
|
| 763 |
-
except Exception as e:
|
| 764 |
-
logger.error(f"Error getting IMU accel Y: {e}")
|
| 765 |
-
return 0.0
|
| 766 |
|
| 767 |
def get_imu_accel_z(self) -> float:
|
| 768 |
"""Get IMU Z-axis acceleration in m/s²."""
|
| 769 |
-
|
| 770 |
-
return 0.0
|
| 771 |
-
try:
|
| 772 |
-
imu_data = self.reachy.imu
|
| 773 |
-
if imu_data is not None and 'accelerometer' in imu_data:
|
| 774 |
-
return float(imu_data['accelerometer'][2])
|
| 775 |
-
return 0.0
|
| 776 |
-
except Exception as e:
|
| 777 |
-
logger.error(f"Error getting IMU accel Z: {e}")
|
| 778 |
-
return 0.0
|
| 779 |
|
| 780 |
def get_imu_gyro_x(self) -> float:
|
| 781 |
"""Get IMU X-axis angular velocity in rad/s."""
|
| 782 |
-
|
| 783 |
-
return 0.0
|
| 784 |
-
try:
|
| 785 |
-
imu_data = self.reachy.imu
|
| 786 |
-
if imu_data is not None and 'gyroscope' in imu_data:
|
| 787 |
-
return float(imu_data['gyroscope'][0])
|
| 788 |
-
return 0.0
|
| 789 |
-
except Exception as e:
|
| 790 |
-
logger.error(f"Error getting IMU gyro X: {e}")
|
| 791 |
-
return 0.0
|
| 792 |
|
| 793 |
def get_imu_gyro_y(self) -> float:
|
| 794 |
"""Get IMU Y-axis angular velocity in rad/s."""
|
| 795 |
-
|
| 796 |
-
return 0.0
|
| 797 |
-
try:
|
| 798 |
-
imu_data = self.reachy.imu
|
| 799 |
-
if imu_data is not None and 'gyroscope' in imu_data:
|
| 800 |
-
return float(imu_data['gyroscope'][1])
|
| 801 |
-
return 0.0
|
| 802 |
-
except Exception as e:
|
| 803 |
-
logger.error(f"Error getting IMU gyro Y: {e}")
|
| 804 |
-
return 0.0
|
| 805 |
|
| 806 |
def get_imu_gyro_z(self) -> float:
|
| 807 |
"""Get IMU Z-axis angular velocity in rad/s."""
|
| 808 |
-
|
| 809 |
-
return 0.0
|
| 810 |
-
try:
|
| 811 |
-
imu_data = self.reachy.imu
|
| 812 |
-
if imu_data is not None and 'gyroscope' in imu_data:
|
| 813 |
-
return float(imu_data['gyroscope'][2])
|
| 814 |
-
return 0.0
|
| 815 |
-
except Exception as e:
|
| 816 |
-
logger.error(f"Error getting IMU gyro Z: {e}")
|
| 817 |
-
return 0.0
|
| 818 |
|
| 819 |
def get_imu_temperature(self) -> float:
|
| 820 |
"""Get IMU temperature in °C."""
|
| 821 |
-
|
| 822 |
-
return 0.0
|
| 823 |
-
try:
|
| 824 |
-
imu_data = self.reachy.imu
|
| 825 |
-
if imu_data is not None and 'temperature' in imu_data:
|
| 826 |
-
return float(imu_data['temperature'])
|
| 827 |
-
return 0.0
|
| 828 |
-
except Exception as e:
|
| 829 |
-
logger.error(f"Error getting IMU temperature: {e}")
|
| 830 |
-
return 0.0
|
| 831 |
|
| 832 |
-
# ========== Phase 11: LED Control (
|
|
|
|
|
|
|
| 833 |
|
| 834 |
def _get_respeaker(self):
|
| 835 |
"""Get ReSpeaker device from media manager with thread-safe access.
|
|
@@ -841,167 +675,22 @@ class ReachyController:
|
|
| 841 |
respeaker.read("...")
|
| 842 |
"""
|
| 843 |
if not self.is_available:
|
| 844 |
-
logger.debug("ReSpeaker not available: robot not connected")
|
| 845 |
return _ReSpeakerContext(None, self._respeaker_lock)
|
| 846 |
try:
|
| 847 |
-
if not self.reachy.media:
|
| 848 |
-
logger.debug("ReSpeaker not available: media manager is None")
|
| 849 |
-
return _ReSpeakerContext(None, self._respeaker_lock)
|
| 850 |
-
if not self.reachy.media.audio:
|
| 851 |
-
logger.debug("ReSpeaker not available: audio is None")
|
| 852 |
return _ReSpeakerContext(None, self._respeaker_lock)
|
| 853 |
respeaker = self.reachy.media.audio._respeaker
|
| 854 |
-
if respeaker is None:
|
| 855 |
-
logger.debug("ReSpeaker not available: _respeaker is None (USB device not found)")
|
| 856 |
return _ReSpeakerContext(respeaker, self._respeaker_lock)
|
| 857 |
-
except Exception
|
| 858 |
-
logger.debug(f"ReSpeaker not available: {e}")
|
| 859 |
return _ReSpeakerContext(None, self._respeaker_lock)
|
| 860 |
|
| 861 |
-
# ========== Phase 11: LED Control (DISABLED - LEDs are inside the robot and not visible) ==========
|
| 862 |
-
# According to PROJECT_PLAN.md principle 8: "LED都被隐藏在了机器人内部,所有的LED控制全部都忽�?
|
| 863 |
-
# The following LED methods are kept but commented out for reference.
|
| 864 |
-
# They are not registered as entities in entity_registry.py.
|
| 865 |
-
|
| 866 |
-
# def get_led_brightness(self) -> float:
|
| 867 |
-
# """Get LED brightness (0-100)."""
|
| 868 |
-
# respeaker = self._get_respeaker()
|
| 869 |
-
# if respeaker is None:
|
| 870 |
-
# return getattr(self, '_led_brightness', 50.0)
|
| 871 |
-
# try:
|
| 872 |
-
# result = respeaker.read("LED_BRIGHTNESS")
|
| 873 |
-
# if result is not None:
|
| 874 |
-
# self._led_brightness = (result[1] / 255.0) * 100.0
|
| 875 |
-
# return self._led_brightness
|
| 876 |
-
# except Exception as e:
|
| 877 |
-
# logger.debug(f"Error getting LED brightness: {e}")
|
| 878 |
-
# return getattr(self, '_led_brightness', 50.0)
|
| 879 |
-
|
| 880 |
-
# def set_led_brightness(self, brightness: float) -> None:
|
| 881 |
-
# """Set LED brightness (0-100)."""
|
| 882 |
-
# brightness = max(0.0, min(100.0, brightness))
|
| 883 |
-
# self._led_brightness = brightness
|
| 884 |
-
# respeaker = self._get_respeaker()
|
| 885 |
-
# if respeaker is None:
|
| 886 |
-
# return
|
| 887 |
-
# try:
|
| 888 |
-
# value = int((brightness / 100.0) * 255)
|
| 889 |
-
# respeaker.write("LED_BRIGHTNESS", [value])
|
| 890 |
-
# logger.info(f"LED brightness set to {brightness}%")
|
| 891 |
-
# except Exception as e:
|
| 892 |
-
# logger.error(f"Error setting LED brightness: {e}")
|
| 893 |
-
|
| 894 |
-
# def get_led_effect(self) -> str:
|
| 895 |
-
# """Get current LED effect."""
|
| 896 |
-
# respeaker = self._get_respeaker()
|
| 897 |
-
# if respeaker is None:
|
| 898 |
-
# return getattr(self, '_led_effect', 'off')
|
| 899 |
-
# try:
|
| 900 |
-
# result = respeaker.read("LED_EFFECT")
|
| 901 |
-
# if result is not None:
|
| 902 |
-
# effect_map = {0: 'off', 1: 'solid', 2: 'breathing', 3: 'rainbow', 4: 'doa'}
|
| 903 |
-
# self._led_effect = effect_map.get(result[1], 'off')
|
| 904 |
-
# return self._led_effect
|
| 905 |
-
# except Exception as e:
|
| 906 |
-
# logger.debug(f"Error getting LED effect: {e}")
|
| 907 |
-
# return getattr(self, '_led_effect', 'off')
|
| 908 |
-
|
| 909 |
-
# def set_led_effect(self, effect: str) -> None:
|
| 910 |
-
# """Set LED effect."""
|
| 911 |
-
# self._led_effect = effect
|
| 912 |
-
# respeaker = self._get_respeaker()
|
| 913 |
-
# if respeaker is None:
|
| 914 |
-
# return
|
| 915 |
-
# try:
|
| 916 |
-
# effect_map = {'off': 0, 'solid': 1, 'breathing': 2, 'rainbow': 3, 'doa': 4}
|
| 917 |
-
# value = effect_map.get(effect, 0)
|
| 918 |
-
# respeaker.write("LED_EFFECT", [value])
|
| 919 |
-
# logger.info(f"LED effect set to {effect}")
|
| 920 |
-
# except Exception as e:
|
| 921 |
-
# logger.error(f"Error setting LED effect: {e}")
|
| 922 |
-
|
| 923 |
-
# def get_led_color_r(self) -> float:
|
| 924 |
-
# """Get LED red color component (0-255)."""
|
| 925 |
-
# respeaker = self._get_respeaker()
|
| 926 |
-
# if respeaker is None:
|
| 927 |
-
# return getattr(self, '_led_color_r', 0.0)
|
| 928 |
-
# try:
|
| 929 |
-
# result = respeaker.read("LED_COLOR")
|
| 930 |
-
# if result is not None:
|
| 931 |
-
# color = result[1] if len(result) > 1 else 0
|
| 932 |
-
# self._led_color_r = float((color >> 16) & 0xFF)
|
| 933 |
-
# return self._led_color_r
|
| 934 |
-
# except Exception as e:
|
| 935 |
-
# logger.debug(f"Error getting LED color R: {e}")
|
| 936 |
-
# return getattr(self, '_led_color_r', 0.0)
|
| 937 |
-
|
| 938 |
-
# def set_led_color_r(self, value: float) -> None:
|
| 939 |
-
# """Set LED red color component (0-255)."""
|
| 940 |
-
# self._led_color_r = max(0.0, min(255.0, value))
|
| 941 |
-
# self._update_led_color()
|
| 942 |
-
|
| 943 |
-
# def get_led_color_g(self) -> float:
|
| 944 |
-
# """Get LED green color component (0-255)."""
|
| 945 |
-
# respeaker = self._get_respeaker()
|
| 946 |
-
# if respeaker is None:
|
| 947 |
-
# return getattr(self, '_led_color_g', 0.0)
|
| 948 |
-
# try:
|
| 949 |
-
# result = respeaker.read("LED_COLOR")
|
| 950 |
-
# if result is not None:
|
| 951 |
-
# color = result[1] if len(result) > 1 else 0
|
| 952 |
-
# self._led_color_g = float((color >> 8) & 0xFF)
|
| 953 |
-
# return self._led_color_g
|
| 954 |
-
# except Exception as e:
|
| 955 |
-
# logger.debug(f"Error getting LED color G: {e}")
|
| 956 |
-
# return getattr(self, '_led_color_g', 0.0)
|
| 957 |
-
|
| 958 |
-
# def set_led_color_g(self, value: float) -> None:
|
| 959 |
-
# """Set LED green color component (0-255)."""
|
| 960 |
-
# self._led_color_g = max(0.0, min(255.0, value))
|
| 961 |
-
# self._update_led_color()
|
| 962 |
-
|
| 963 |
-
# def get_led_color_b(self) -> float:
|
| 964 |
-
# """Get LED blue color component (0-255)."""
|
| 965 |
-
# respeaker = self._get_respeaker()
|
| 966 |
-
# if respeaker is None:
|
| 967 |
-
# return getattr(self, '_led_color_b', 0.0)
|
| 968 |
-
# try:
|
| 969 |
-
# result = respeaker.read("LED_COLOR")
|
| 970 |
-
# if result is not None:
|
| 971 |
-
# color = result[1] if len(result) > 1 else 0
|
| 972 |
-
# self._led_color_b = float(color & 0xFF)
|
| 973 |
-
# return self._led_color_b
|
| 974 |
-
# except Exception as e:
|
| 975 |
-
# logger.debug(f"Error getting LED color B: {e}")
|
| 976 |
-
# return getattr(self, '_led_color_b', 0.0)
|
| 977 |
-
|
| 978 |
-
# def set_led_color_b(self, value: float) -> None:
|
| 979 |
-
# """Set LED blue color component (0-255)."""
|
| 980 |
-
# self._led_color_b = max(0.0, min(255.0, value))
|
| 981 |
-
# self._update_led_color()
|
| 982 |
-
|
| 983 |
-
# def _update_led_color(self) -> None:
|
| 984 |
-
# """Update LED color from R, G, B components."""
|
| 985 |
-
# respeaker = self._get_respeaker()
|
| 986 |
-
# if respeaker is None:
|
| 987 |
-
# return
|
| 988 |
-
# try:
|
| 989 |
-
# r = int(getattr(self, '_led_color_r', 0))
|
| 990 |
-
# g = int(getattr(self, '_led_color_g', 0))
|
| 991 |
-
# b = int(getattr(self, '_led_color_b', 0))
|
| 992 |
-
# color = (r << 16) | (g << 8) | b
|
| 993 |
-
# respeaker.write("LED_COLOR", [color])
|
| 994 |
-
# logger.info(f"LED color set to RGB({r}, {g}, {b})")
|
| 995 |
-
# except Exception as e:
|
| 996 |
-
# logger.error(f"Error setting LED color: {e}")
|
| 997 |
-
|
| 998 |
# ========== Phase 12: Audio Processing (via local SDK with thread-safe access) ==========
|
| 999 |
|
| 1000 |
def get_agc_enabled(self) -> bool:
|
| 1001 |
"""Get AGC (Automatic Gain Control) enabled status."""
|
| 1002 |
with self._get_respeaker() as respeaker:
|
| 1003 |
if respeaker is None:
|
| 1004 |
-
return getattr(self, '_agc_enabled',
|
| 1005 |
try:
|
| 1006 |
result = respeaker.read("PP_AGCONOFF")
|
| 1007 |
if result is not None:
|
|
@@ -1009,7 +698,7 @@ class ReachyController:
|
|
| 1009 |
return self._agc_enabled
|
| 1010 |
except Exception as e:
|
| 1011 |
logger.debug(f"Error getting AGC status: {e}")
|
| 1012 |
-
return getattr(self, '_agc_enabled',
|
| 1013 |
|
| 1014 |
def set_agc_enabled(self, enabled: bool) -> None:
|
| 1015 |
"""Set AGC (Automatic Gain Control) enabled status."""
|
|
@@ -1024,10 +713,10 @@ class ReachyController:
|
|
| 1024 |
logger.error(f"Error setting AGC status: {e}")
|
| 1025 |
|
| 1026 |
def get_agc_max_gain(self) -> float:
|
| 1027 |
-
"""Get AGC maximum gain in dB."""
|
| 1028 |
with self._get_respeaker() as respeaker:
|
| 1029 |
if respeaker is None:
|
| 1030 |
-
return getattr(self, '_agc_max_gain',
|
| 1031 |
try:
|
| 1032 |
result = respeaker.read("PP_AGCMAXGAIN")
|
| 1033 |
if result is not None:
|
|
@@ -1035,11 +724,11 @@ class ReachyController:
|
|
| 1035 |
return self._agc_max_gain
|
| 1036 |
except Exception as e:
|
| 1037 |
logger.debug(f"Error getting AGC max gain: {e}")
|
| 1038 |
-
return getattr(self, '_agc_max_gain',
|
| 1039 |
|
| 1040 |
def set_agc_max_gain(self, gain: float) -> None:
|
| 1041 |
-
"""Set AGC maximum gain in dB."""
|
| 1042 |
-
gain = max(0.0, min(
|
| 1043 |
self._agc_max_gain = gain
|
| 1044 |
with self._get_respeaker() as respeaker:
|
| 1045 |
if respeaker is None:
|
|
|
|
| 52 |
# Note: get_current_head_pose() and get_current_joint_positions() are
|
| 53 |
# non-blocking in the SDK (they return cached Zenoh data), so no caching needed
|
| 54 |
self._state_cache: Dict[str, Any] = {}
|
| 55 |
+
self._cache_ttl = 2.0 # 2 second cache TTL for status queries (increased from 1s)
|
| 56 |
self._last_status_query = 0.0
|
| 57 |
|
| 58 |
# Thread lock for ReSpeaker USB access to prevent conflicts with GStreamer audio pipeline
|
|
|
|
| 380 |
|
| 381 |
return x, y, z, roll, pitch, yaw
|
| 382 |
|
| 383 |
+
def _get_head_pose_component(self, component: str) -> float:
|
| 384 |
+
"""Get a specific component from head pose.
|
| 385 |
+
|
| 386 |
+
Args:
|
| 387 |
+
component: One of 'x', 'y', 'z' (mm), 'roll', 'pitch', 'yaw' (degrees)
|
| 388 |
+
|
| 389 |
+
Returns:
|
| 390 |
+
The component value, or 0.0 on error
|
| 391 |
+
"""
|
| 392 |
pose = self._get_head_pose()
|
| 393 |
if pose is None:
|
| 394 |
return 0.0
|
| 395 |
try:
|
| 396 |
x, y, z, roll, pitch, yaw = self._extract_pose_from_matrix(pose)
|
| 397 |
+
components = {
|
| 398 |
+
'x': x * 1000, # m to mm
|
| 399 |
+
'y': y * 1000,
|
| 400 |
+
'z': z * 1000,
|
| 401 |
+
'roll': math.degrees(roll),
|
| 402 |
+
'pitch': math.degrees(pitch),
|
| 403 |
+
'yaw': math.degrees(yaw),
|
| 404 |
+
}
|
| 405 |
+
return components.get(component, 0.0)
|
| 406 |
except Exception as e:
|
| 407 |
+
logger.error(f"Error getting head {component}: {e}")
|
| 408 |
return 0.0
|
| 409 |
|
| 410 |
+
def _disabled_pose_setter(self, name: str) -> None:
|
| 411 |
+
"""Log warning for disabled pose setters."""
|
| 412 |
+
logger.debug(f"set_{name} is disabled - MovementManager controls pose")
|
| 413 |
+
|
| 414 |
+
# Head position getters (read-only, setters disabled for MovementManager)
|
| 415 |
+
def get_head_x(self) -> float:
|
| 416 |
+
"""Get head X position in mm."""
|
| 417 |
+
return self._get_head_pose_component('x')
|
| 418 |
+
|
| 419 |
def set_head_x(self, x_mm: float) -> None:
|
| 420 |
+
"""Disabled - MovementManager controls head pose."""
|
| 421 |
+
self._disabled_pose_setter('head_x')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 422 |
|
| 423 |
def get_head_y(self) -> float:
|
| 424 |
+
"""Get head Y position in mm."""
|
| 425 |
+
return self._get_head_pose_component('y')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 426 |
|
| 427 |
def set_head_y(self, y_mm: float) -> None:
|
| 428 |
+
"""Disabled - MovementManager controls head pose."""
|
| 429 |
+
self._disabled_pose_setter('head_y')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 430 |
|
| 431 |
def get_head_z(self) -> float:
|
| 432 |
+
"""Get head Z position in mm."""
|
| 433 |
+
return self._get_head_pose_component('z')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 434 |
|
| 435 |
def set_head_z(self, z_mm: float) -> None:
|
| 436 |
+
"""Disabled - MovementManager controls head pose."""
|
| 437 |
+
self._disabled_pose_setter('head_z')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 438 |
|
| 439 |
+
# Head orientation getters (read-only, setters disabled for MovementManager)
|
| 440 |
def get_head_roll(self) -> float:
|
| 441 |
+
"""Get head roll angle in degrees."""
|
| 442 |
+
return self._get_head_pose_component('roll')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 443 |
|
| 444 |
def set_head_roll(self, roll_deg: float) -> None:
|
| 445 |
+
"""Disabled - MovementManager controls head pose."""
|
| 446 |
+
self._disabled_pose_setter('head_roll')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 447 |
|
| 448 |
def get_head_pitch(self) -> float:
|
| 449 |
+
"""Get head pitch angle in degrees."""
|
| 450 |
+
return self._get_head_pose_component('pitch')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 451 |
|
| 452 |
def set_head_pitch(self, pitch_deg: float) -> None:
|
| 453 |
+
"""Disabled - MovementManager controls head pose."""
|
| 454 |
+
self._disabled_pose_setter('head_pitch')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 455 |
|
| 456 |
def get_head_yaw(self) -> float:
|
| 457 |
+
"""Get head yaw angle in degrees."""
|
| 458 |
+
return self._get_head_pose_component('yaw')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 459 |
|
| 460 |
def set_head_yaw(self, yaw_deg: float) -> None:
|
| 461 |
+
"""Disabled - MovementManager controls head pose."""
|
| 462 |
+
self._disabled_pose_setter('head_yaw')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 463 |
|
| 464 |
def get_body_yaw(self) -> float:
|
| 465 |
+
"""Get body yaw angle in degrees."""
|
| 466 |
joints = self._get_joint_positions()
|
| 467 |
if joints is None:
|
| 468 |
return 0.0
|
|
|
|
| 474 |
return 0.0
|
| 475 |
|
| 476 |
def set_body_yaw(self, yaw_deg: float) -> None:
|
| 477 |
+
"""Disabled - MovementManager controls body pose."""
|
| 478 |
+
self._disabled_pose_setter('body_yaw')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 479 |
|
| 480 |
def get_antenna_left(self) -> float:
|
| 481 |
+
"""Get left antenna angle in degrees."""
|
| 482 |
joints = self._get_joint_positions()
|
| 483 |
if joints is None:
|
| 484 |
return 0.0
|
|
|
|
| 490 |
return 0.0
|
| 491 |
|
| 492 |
def set_antenna_left(self, angle_deg: float) -> None:
|
| 493 |
+
"""Disabled - MovementManager controls antennas."""
|
| 494 |
+
self._disabled_pose_setter('antenna_left')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 495 |
|
| 496 |
def get_antenna_right(self) -> float:
|
| 497 |
+
"""Get right antenna angle in degrees."""
|
| 498 |
joints = self._get_joint_positions()
|
| 499 |
if joints is None:
|
| 500 |
return 0.0
|
|
|
|
| 506 |
return 0.0
|
| 507 |
|
| 508 |
def set_antenna_right(self, angle_deg: float) -> None:
|
| 509 |
+
"""Disabled - MovementManager controls antennas."""
|
| 510 |
+
self._disabled_pose_setter('antenna_right')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 511 |
|
| 512 |
# ========== Phase 4: Look At Control ==========
|
| 513 |
|
|
|
|
| 611 |
|
| 612 |
# ========== Phase 7: IMU Sensors (Wireless only) ==========
|
| 613 |
|
| 614 |
+
def _get_imu_value(self, sensor_type: str, index: int) -> float:
|
| 615 |
+
"""Get a specific IMU sensor value.
|
| 616 |
+
|
| 617 |
+
Args:
|
| 618 |
+
sensor_type: 'accelerometer', 'gyroscope', or 'temperature'
|
| 619 |
+
index: Array index (0=x, 1=y, 2=z) or -1 for scalar values
|
| 620 |
+
|
| 621 |
+
Returns:
|
| 622 |
+
The sensor value, or 0.0 on error
|
| 623 |
+
"""
|
| 624 |
if not self.is_available:
|
| 625 |
return 0.0
|
| 626 |
try:
|
| 627 |
imu_data = self.reachy.imu
|
| 628 |
+
if imu_data is None or sensor_type not in imu_data:
|
| 629 |
+
return 0.0
|
| 630 |
+
value = imu_data[sensor_type]
|
| 631 |
+
return float(value[index]) if index >= 0 else float(value)
|
| 632 |
except Exception as e:
|
| 633 |
+
logger.debug(f"Error getting IMU {sensor_type}: {e}")
|
| 634 |
return 0.0
|
| 635 |
|
| 636 |
+
def get_imu_accel_x(self) -> float:
|
| 637 |
+
"""Get IMU X-axis acceleration in m/s²."""
|
| 638 |
+
return self._get_imu_value('accelerometer', 0)
|
| 639 |
+
|
| 640 |
def get_imu_accel_y(self) -> float:
|
| 641 |
"""Get IMU Y-axis acceleration in m/s²."""
|
| 642 |
+
return self._get_imu_value('accelerometer', 1)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 643 |
|
| 644 |
def get_imu_accel_z(self) -> float:
|
| 645 |
"""Get IMU Z-axis acceleration in m/s²."""
|
| 646 |
+
return self._get_imu_value('accelerometer', 2)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 647 |
|
| 648 |
def get_imu_gyro_x(self) -> float:
|
| 649 |
"""Get IMU X-axis angular velocity in rad/s."""
|
| 650 |
+
return self._get_imu_value('gyroscope', 0)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 651 |
|
| 652 |
def get_imu_gyro_y(self) -> float:
|
| 653 |
"""Get IMU Y-axis angular velocity in rad/s."""
|
| 654 |
+
return self._get_imu_value('gyroscope', 1)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 655 |
|
| 656 |
def get_imu_gyro_z(self) -> float:
|
| 657 |
"""Get IMU Z-axis angular velocity in rad/s."""
|
| 658 |
+
return self._get_imu_value('gyroscope', 2)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 659 |
|
| 660 |
def get_imu_temperature(self) -> float:
|
| 661 |
"""Get IMU temperature in °C."""
|
| 662 |
+
return self._get_imu_value('temperature', -1)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 663 |
|
| 664 |
+
# ========== Phase 11: LED Control (DISABLED) ==========
|
| 665 |
+
# LED control is disabled because LEDs are hidden inside the robot.
|
| 666 |
+
# See PROJECT_PLAN.md principle 8.
|
| 667 |
|
| 668 |
def _get_respeaker(self):
|
| 669 |
"""Get ReSpeaker device from media manager with thread-safe access.
|
|
|
|
| 675 |
respeaker.read("...")
|
| 676 |
"""
|
| 677 |
if not self.is_available:
|
|
|
|
| 678 |
return _ReSpeakerContext(None, self._respeaker_lock)
|
| 679 |
try:
|
| 680 |
+
if not self.reachy.media or not self.reachy.media.audio:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 681 |
return _ReSpeakerContext(None, self._respeaker_lock)
|
| 682 |
respeaker = self.reachy.media.audio._respeaker
|
|
|
|
|
|
|
| 683 |
return _ReSpeakerContext(respeaker, self._respeaker_lock)
|
| 684 |
+
except Exception:
|
|
|
|
| 685 |
return _ReSpeakerContext(None, self._respeaker_lock)
|
| 686 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 687 |
# ========== Phase 12: Audio Processing (via local SDK with thread-safe access) ==========
|
| 688 |
|
| 689 |
def get_agc_enabled(self) -> bool:
|
| 690 |
"""Get AGC (Automatic Gain Control) enabled status."""
|
| 691 |
with self._get_respeaker() as respeaker:
|
| 692 |
if respeaker is None:
|
| 693 |
+
return getattr(self, '_agc_enabled', True) # Default to enabled
|
| 694 |
try:
|
| 695 |
result = respeaker.read("PP_AGCONOFF")
|
| 696 |
if result is not None:
|
|
|
|
| 698 |
return self._agc_enabled
|
| 699 |
except Exception as e:
|
| 700 |
logger.debug(f"Error getting AGC status: {e}")
|
| 701 |
+
return getattr(self, '_agc_enabled', True)
|
| 702 |
|
| 703 |
def set_agc_enabled(self, enabled: bool) -> None:
|
| 704 |
"""Set AGC (Automatic Gain Control) enabled status."""
|
|
|
|
| 713 |
logger.error(f"Error setting AGC status: {e}")
|
| 714 |
|
| 715 |
def get_agc_max_gain(self) -> float:
|
| 716 |
+
"""Get AGC maximum gain in dB (0-40 dB range)."""
|
| 717 |
with self._get_respeaker() as respeaker:
|
| 718 |
if respeaker is None:
|
| 719 |
+
return getattr(self, '_agc_max_gain', 30.0) # Default to optimized value
|
| 720 |
try:
|
| 721 |
result = respeaker.read("PP_AGCMAXGAIN")
|
| 722 |
if result is not None:
|
|
|
|
| 724 |
return self._agc_max_gain
|
| 725 |
except Exception as e:
|
| 726 |
logger.debug(f"Error getting AGC max gain: {e}")
|
| 727 |
+
return getattr(self, '_agc_max_gain', 30.0)
|
| 728 |
|
| 729 |
def set_agc_max_gain(self, gain: float) -> None:
|
| 730 |
+
"""Set AGC maximum gain in dB (0-40 dB range)."""
|
| 731 |
+
gain = max(0.0, min(40.0, gain)) # XVF3800 supports up to 40dB
|
| 732 |
self._agc_max_gain = gain
|
| 733 |
with self._get_respeaker() as respeaker:
|
| 734 |
if respeaker is None:
|
reachy_mini_ha_voice/satellite.py
CHANGED
|
@@ -568,14 +568,13 @@ class VoiceSatelliteProtocol(APIServer):
|
|
| 568 |
def _tap_continue_feedback(self) -> None:
|
| 569 |
"""Provide feedback when continuing conversation in tap mode.
|
| 570 |
|
| 571 |
-
|
|
|
|
| 572 |
"""
|
| 573 |
try:
|
| 574 |
-
#
|
| 575 |
-
#
|
| 576 |
-
|
| 577 |
-
_LOGGER.debug("Tap continue feedback: sound played")
|
| 578 |
-
|
| 579 |
# Trigger a small nod to indicate ready for input
|
| 580 |
if self.state.motion_enabled and self.state.motion:
|
| 581 |
self.state.motion.on_continue_listening()
|
|
|
|
| 568 |
def _tap_continue_feedback(self) -> None:
|
| 569 |
"""Provide feedback when continuing conversation in tap mode.
|
| 570 |
|
| 571 |
+
Triggers a nod to indicate ready for next input.
|
| 572 |
+
Sound is NOT played here to avoid blocking audio streaming.
|
| 573 |
"""
|
| 574 |
try:
|
| 575 |
+
# NOTE: Do NOT play sound here - it blocks audio streaming
|
| 576 |
+
# The wakeup sound is already played by the main wakeup flow
|
| 577 |
+
|
|
|
|
|
|
|
| 578 |
# Trigger a small nod to indicate ready for input
|
| 579 |
if self.state.motion_enabled and self.state.motion:
|
| 580 |
self.state.motion.on_continue_listening()
|
reachy_mini_ha_voice/tap_detector.py
CHANGED
|
@@ -20,7 +20,7 @@ TAP_THRESHOLD_G_DEFAULT = 2.0 # Default acceleration threshold in g
|
|
| 20 |
TAP_THRESHOLD_G_MIN = 0.5 # Minimum threshold (very sensitive)
|
| 21 |
TAP_THRESHOLD_G_MAX = 5.0 # Maximum threshold (less sensitive)
|
| 22 |
TAP_COOLDOWN_SECONDS = 1.0 # Minimum time between tap detections
|
| 23 |
-
TAP_DETECTION_RATE_HZ =
|
| 24 |
|
| 25 |
|
| 26 |
class TapDetector:
|
|
|
|
| 20 |
TAP_THRESHOLD_G_MIN = 0.5 # Minimum threshold (very sensitive)
|
| 21 |
TAP_THRESHOLD_G_MAX = 5.0 # Maximum threshold (less sensitive)
|
| 22 |
TAP_COOLDOWN_SECONDS = 1.0 # Minimum time between tap detections
|
| 23 |
+
TAP_DETECTION_RATE_HZ = 20 # IMU polling rate (reduced from 50Hz for system stability)
|
| 24 |
|
| 25 |
|
| 26 |
class TapDetector:
|
reachy_mini_ha_voice/voice_assistant.py
CHANGED
|
@@ -223,13 +223,19 @@ class VoiceAssistantService:
|
|
| 223 |
_LOGGER.info("Voice assistant service started on %s:%s", self.host, self.port)
|
| 224 |
|
| 225 |
def _optimize_microphone_settings(self) -> None:
|
| 226 |
-
"""Optimize ReSpeaker microphone settings for voice recognition.
|
| 227 |
|
| 228 |
-
|
| 229 |
-
|
| 230 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 231 |
|
| 232 |
Reference: reachy_mini/src/reachy_mini/media/audio_control_utils.py
|
|
|
|
| 233 |
"""
|
| 234 |
if self.reachy_mini is None:
|
| 235 |
return
|
|
@@ -246,25 +252,85 @@ class VoiceAssistantService:
|
|
| 246 |
_LOGGER.debug("ReSpeaker device not found")
|
| 247 |
return
|
| 248 |
|
| 249 |
-
#
|
| 250 |
-
#
|
| 251 |
-
|
| 252 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 253 |
try:
|
| 254 |
-
respeaker.write("PP_MIN_NS", [0.
|
| 255 |
-
_LOGGER.info("Noise suppression reduced (PP_MIN_NS=0.
|
| 256 |
except Exception as e:
|
| 257 |
_LOGGER.debug("Could not set PP_MIN_NS: %s", e)
|
| 258 |
|
| 259 |
-
#
|
| 260 |
-
#
|
| 261 |
try:
|
| 262 |
-
respeaker.write("PP_MIN_NN", [0.
|
| 263 |
-
_LOGGER.info("Noise floor threshold reduced (PP_MIN_NN=0.
|
| 264 |
except Exception as e:
|
| 265 |
_LOGGER.debug("Could not set PP_MIN_NN: %s", e)
|
| 266 |
|
| 267 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 268 |
|
| 269 |
except Exception as e:
|
| 270 |
_LOGGER.warning("Failed to optimize microphone settings: %s", e)
|
|
|
|
| 223 |
_LOGGER.info("Voice assistant service started on %s:%s", self.host, self.port)
|
| 224 |
|
| 225 |
def _optimize_microphone_settings(self) -> None:
|
| 226 |
+
"""Optimize ReSpeaker XVF3800 microphone settings for voice recognition.
|
| 227 |
|
| 228 |
+
This method configures the XMOS XVF3800 audio processor for optimal
|
| 229 |
+
voice command recognition at distances up to 2-3 meters.
|
| 230 |
+
|
| 231 |
+
Key optimizations:
|
| 232 |
+
1. Enable AGC with higher max gain for distant speech
|
| 233 |
+
2. Reduce noise suppression to preserve quiet speech
|
| 234 |
+
3. Increase base microphone gain
|
| 235 |
+
4. Optimize AGC response times for voice commands
|
| 236 |
|
| 237 |
Reference: reachy_mini/src/reachy_mini/media/audio_control_utils.py
|
| 238 |
+
XMOS docs: https://www.xmos.com/documentation/XM-014888-PC/html/modules/fwk_xvf/doc/user_guide/AA_control_command_appendix.html
|
| 239 |
"""
|
| 240 |
if self.reachy_mini is None:
|
| 241 |
return
|
|
|
|
| 252 |
_LOGGER.debug("ReSpeaker device not found")
|
| 253 |
return
|
| 254 |
|
| 255 |
+
# ========== 1. AGC (Automatic Gain Control) Settings ==========
|
| 256 |
+
# Enable AGC for automatic volume normalization
|
| 257 |
+
try:
|
| 258 |
+
respeaker.write("PP_AGCONOFF", [1])
|
| 259 |
+
_LOGGER.info("AGC enabled (PP_AGCONOFF=1)")
|
| 260 |
+
except Exception as e:
|
| 261 |
+
_LOGGER.debug("Could not enable AGC: %s", e)
|
| 262 |
+
|
| 263 |
+
# Increase AGC max gain for better distant speech pickup
|
| 264 |
+
# Default is ~15dB, increase to 30dB for voice commands at distance
|
| 265 |
+
# Range: 0-40 dB (float)
|
| 266 |
+
try:
|
| 267 |
+
respeaker.write("PP_AGCMAXGAIN", [30.0])
|
| 268 |
+
_LOGGER.info("AGC max gain increased (PP_AGCMAXGAIN=30.0dB)")
|
| 269 |
+
except Exception as e:
|
| 270 |
+
_LOGGER.debug("Could not set PP_AGCMAXGAIN: %s", e)
|
| 271 |
+
|
| 272 |
+
# Set AGC desired output level (target level after gain)
|
| 273 |
+
# More negative = quieter output, less negative = louder
|
| 274 |
+
# Default is around -25dB, set to -18dB for stronger output
|
| 275 |
+
try:
|
| 276 |
+
respeaker.write("PP_AGCDESIREDLEVEL", [-18.0])
|
| 277 |
+
_LOGGER.info("AGC desired level set (PP_AGCDESIREDLEVEL=-18.0dB)")
|
| 278 |
+
except Exception as e:
|
| 279 |
+
_LOGGER.debug("Could not set PP_AGCDESIREDLEVEL: %s", e)
|
| 280 |
+
|
| 281 |
+
# Optimize AGC time constants for voice commands
|
| 282 |
+
# Faster attack time helps capture sudden speech onset
|
| 283 |
+
try:
|
| 284 |
+
respeaker.write("PP_AGCTIME", [0.5]) # Main time constant (seconds)
|
| 285 |
+
_LOGGER.debug("AGC time constant set (PP_AGCTIME=0.5s)")
|
| 286 |
+
except Exception as e:
|
| 287 |
+
_LOGGER.debug("Could not set PP_AGCTIME: %s", e)
|
| 288 |
+
|
| 289 |
+
# ========== 2. Base Microphone Gain ==========
|
| 290 |
+
# Increase base microphone gain for better sensitivity
|
| 291 |
+
# Default is 1.0, increase to 2.0 for distant speech
|
| 292 |
+
# Range: 0.0-4.0 (float, linear gain multiplier)
|
| 293 |
+
try:
|
| 294 |
+
respeaker.write("AUDIO_MGR_MIC_GAIN", [2.0])
|
| 295 |
+
_LOGGER.info("Microphone gain increased (AUDIO_MGR_MIC_GAIN=2.0)")
|
| 296 |
+
except Exception as e:
|
| 297 |
+
_LOGGER.debug("Could not set AUDIO_MGR_MIC_GAIN: %s", e)
|
| 298 |
+
|
| 299 |
+
# ========== 3. Noise Suppression Settings ==========
|
| 300 |
+
# Reduce noise suppression to preserve quiet speech
|
| 301 |
+
# PP_MIN_NS: minimum noise suppression threshold
|
| 302 |
+
# Lower values = less aggressive suppression = better voice pickup
|
| 303 |
+
# Default is ~0.5-0.7, reduce to 0.15 for voice commands
|
| 304 |
try:
|
| 305 |
+
respeaker.write("PP_MIN_NS", [0.15])
|
| 306 |
+
_LOGGER.info("Noise suppression reduced (PP_MIN_NS=0.15)")
|
| 307 |
except Exception as e:
|
| 308 |
_LOGGER.debug("Could not set PP_MIN_NS: %s", e)
|
| 309 |
|
| 310 |
+
# PP_MIN_NN: minimum noise floor estimation
|
| 311 |
+
# Lower values help in quieter environments
|
| 312 |
try:
|
| 313 |
+
respeaker.write("PP_MIN_NN", [0.15])
|
| 314 |
+
_LOGGER.info("Noise floor threshold reduced (PP_MIN_NN=0.15)")
|
| 315 |
except Exception as e:
|
| 316 |
_LOGGER.debug("Could not set PP_MIN_NN: %s", e)
|
| 317 |
|
| 318 |
+
# ========== 4. Echo Cancellation Settings ==========
|
| 319 |
+
# Ensure echo cancellation is enabled (important for TTS playback)
|
| 320 |
+
try:
|
| 321 |
+
respeaker.write("PP_ECHOONOFF", [1])
|
| 322 |
+
_LOGGER.debug("Echo cancellation enabled (PP_ECHOONOFF=1)")
|
| 323 |
+
except Exception as e:
|
| 324 |
+
_LOGGER.debug("Could not set PP_ECHOONOFF: %s", e)
|
| 325 |
+
|
| 326 |
+
# ========== 5. High-pass filter (remove low frequency noise) ==========
|
| 327 |
+
try:
|
| 328 |
+
respeaker.write("AEC_HPFONOFF", [1])
|
| 329 |
+
_LOGGER.debug("High-pass filter enabled (AEC_HPFONOFF=1)")
|
| 330 |
+
except Exception as e:
|
| 331 |
+
_LOGGER.debug("Could not set AEC_HPFONOFF: %s", e)
|
| 332 |
+
|
| 333 |
+
_LOGGER.info("Microphone settings optimized for voice recognition (AGC=ON, MaxGain=30dB, MicGain=2.0x)")
|
| 334 |
|
| 335 |
except Exception as e:
|
| 336 |
_LOGGER.warning("Failed to optimize microphone settings: %s", e)
|