andito HF Staff commited on
Commit
5ecb3ec
·
0 Parent(s):

Initial commit

Browse files
.env.example ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Remote Control App Configuration
2
+
3
+ # Profile to use (default, example_remote, or custom)
4
+ REMOTE_CONTROL_PROFILE=default
5
+
6
+ # Robot configuration
7
+ ROBOT_NAME=reachy_mini
8
+
9
+ # Logging
10
+ LOG_LEVEL=INFO
11
+
12
+ # Video settings (can override profile settings)
13
+ VIDEO_JPEG_QUALITY=80
14
+
15
+ # Monitoring
16
+ LATENCY_WARNING_MS=200
17
+
18
+ # REST API settings (optional)
19
+ ENABLE_REST_API=true
20
+ REST_API_HOST=0.0.0.0
21
+ REST_API_PORT=7860
README.md ADDED
@@ -0,0 +1,327 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Reachy Mini Remote Control App
2
+
3
+ A standalone app that connects Reachy Mini robot to a remote server for WebSocket-based control. The robot acts as a WebSocket **client** connecting to a remote server (e.g., Hugging Face Space), enabling remote control, video streaming, and bidirectional audio.
4
+
5
+ ## Features
6
+
7
+ - **WebSocket Client Architecture**: Robot connects to remote server (works behind NAT/firewalls)
8
+ - **Profile-Based Configuration**: Easily switch between different remote servers
9
+ - **Bidirectional Streaming**: Video (robot → server) and Audio (robot ↔ server)
10
+ - **Movement Control**: Receive movement commands from remote server
11
+ - **Connection Monitoring**: Track latency, frames, and connection health
12
+ - **Optional REST API**: Query status and manage profiles via HTTP endpoints
13
+
14
+ ## Architecture
15
+
16
+ ```
17
+ Remote Server Robot (this app) Reachy Mini Daemon
18
+ (Hugging Face Space)
19
+
20
+ WebSocket Server ←────── WebSocket Clients ←────── ReachyMini SDK
21
+ - /robot - robot_control.py (Zenoh protocol)
22
+ - /video_stream - video_stream.py
23
+ - /audio_stream - audio_stream.py
24
+ ```
25
+
26
+ The app connects TO a remote server as a client, not the other way around. This simplifies network setup since the robot only needs outbound connections.
27
+
28
+ ## Installation
29
+
30
+ ### From Source
31
+
32
+ ```bash
33
+ cd reachy_mini_remote_control_app
34
+ pip install -e .
35
+ ```
36
+
37
+ ### Requirements
38
+
39
+ - Python >= 3.10
40
+ - Reachy Mini daemon running (`reachy-mini-daemon`)
41
+ - Remote server running (e.g., [reachy_mini_remote_control/app.py](../reachy_mini_remote_control/app.py))
42
+
43
+ ## Usage
44
+
45
+ ### 1. Start the Remote Server
46
+
47
+ First, start the remote server that the robot will connect to:
48
+
49
+ ```bash
50
+ cd ../reachy_mini_remote_control
51
+ python app.py
52
+ # Server starts on ws://localhost:8000
53
+ ```
54
+
55
+ ### 2. Start the Reachy Mini Daemon
56
+
57
+ ```bash
58
+ reachy-mini-daemon --mockup-sim
59
+ ```
60
+
61
+ ### 3. Start the Remote Control App
62
+
63
+ ```bash
64
+ reachy-mini-remote-control-app --profile default
65
+ ```
66
+
67
+ The app will:
68
+ - Connect to the remote server at `ws://localhost:8000`
69
+ - Stream video and audio
70
+ - Receive and execute movement commands
71
+
72
+ ### Command Line Options
73
+
74
+ ```bash
75
+ reachy-mini-remote-control-app [OPTIONS]
76
+
77
+ Options:
78
+ --profile PROFILE Profile to use (default: from .env)
79
+ --websocket-uri URI WebSocket URI to connect to (overrides profile)
80
+ --robot-name NAME Robot name (default: reachy_mini)
81
+ --log-level LEVEL Logging level (DEBUG, INFO, WARNING, ERROR)
82
+ --no-rest-api Disable REST API for status endpoints
83
+ ```
84
+
85
+ ## Configuration
86
+
87
+ ### Profiles
88
+
89
+ Profiles are stored in `src/reachy_mini_remote_control_app/profiles/`. Each profile is a directory with a `config.txt` file.
90
+
91
+ **Default Profile** (`profiles/default/config.txt`):
92
+ ```
93
+ WEBSOCKET_URI=ws://localhost:8000
94
+ VIDEO_JPEG_QUALITY=80
95
+ VIDEO_WITH_TIMESTAMP=false
96
+ VIDEO_FPS=25
97
+ AUDIO_WITH_TIMESTAMP=false
98
+ AUDIO_BATCH_SIZE=4096
99
+ ENABLE_METRICS_LOGGING=true
100
+ METRICS_LOG_INTERVAL_SEC=5
101
+ ```
102
+
103
+ **Example Remote Profile** (`profiles/example_remote/config.txt`):
104
+ ```
105
+ WEBSOCKET_URI=wss://YOUR-SPACE.hf.space
106
+ VIDEO_JPEG_QUALITY=75
107
+ VIDEO_WITH_TIMESTAMP=true
108
+ VIDEO_FPS=25
109
+ AUDIO_WITH_TIMESTAMP=true
110
+ AUDIO_BATCH_SIZE=4096
111
+ ENABLE_METRICS_LOGGING=true
112
+ METRICS_LOG_INTERVAL_SEC=10
113
+ ```
114
+
115
+ ### Environment Variables
116
+
117
+ Create a `.env` file (copy from `.env.example`):
118
+
119
+ ```bash
120
+ REMOTE_CONTROL_PROFILE=default
121
+ ROBOT_NAME=reachy_mini
122
+ LOG_LEVEL=INFO
123
+ VIDEO_JPEG_QUALITY=80
124
+ LATENCY_WARNING_MS=200
125
+ ENABLE_REST_API=true
126
+ REST_API_HOST=0.0.0.0
127
+ REST_API_PORT=7860
128
+ ```
129
+
130
+ ## REST API (Optional)
131
+
132
+ If `ENABLE_REST_API=true`, the app provides HTTP endpoints:
133
+
134
+ ### Get Status
135
+
136
+ ```bash
137
+ curl http://localhost:7860/status
138
+ ```
139
+
140
+ Response:
141
+ ```json
142
+ {
143
+ "robot_connected": true,
144
+ "control": {
145
+ "connected": true,
146
+ "commands_received": 42,
147
+ "last_command_type": "movement",
148
+ "last_activity": 1705123456.78
149
+ },
150
+ "video": {
151
+ "connected": true,
152
+ "frames_sent": 1234,
153
+ "frames_dropped": 5,
154
+ "fps": 24.8,
155
+ "latency_ms": 45.2,
156
+ "latency_avg_ms": 42.1,
157
+ "latency_min_ms": 38.5,
158
+ "latency_max_ms": 52.3,
159
+ "last_activity": 1705123456.78
160
+ },
161
+ "audio": {
162
+ "connected": true,
163
+ "chunks_sent": 567,
164
+ "chunks_received": 234,
165
+ "latency_avg_ms": 38.5,
166
+ "last_activity": 1705123456.78
167
+ }
168
+ }
169
+ ```
170
+
171
+ ### List Profiles
172
+
173
+ ```bash
174
+ curl http://localhost:7860/profiles
175
+ ```
176
+
177
+ Response:
178
+ ```json
179
+ {
180
+ "profiles": ["default", "example_remote"],
181
+ "current": "default"
182
+ }
183
+ ```
184
+
185
+ ### Switch Profile
186
+
187
+ ```bash
188
+ curl -X POST http://localhost:7860/profile/example_remote
189
+ ```
190
+
191
+ **Note**: Requires app restart to take effect.
192
+
193
+ ### Health Check
194
+
195
+ ```bash
196
+ curl http://localhost:7860/health
197
+ ```
198
+
199
+ ## Integration with Reachy Mini Dashboard
200
+
201
+ This app integrates with the Reachy Mini dashboard as a `ReachyMiniApp`:
202
+
203
+ 1. Install the app: `pip install -e reachy_mini_remote_control_app`
204
+ 2. The app will appear in the dashboard
205
+ 3. Start from the dashboard UI
206
+ 4. The dashboard provides the `ReachyMini` instance and `stop_event`
207
+
208
+ ## Deployment to Production
209
+
210
+ ### Hugging Face Space Setup
211
+
212
+ 1. Deploy the remote server ([reachy_mini_remote_control/app.py](../reachy_mini_remote_control/app.py)) to Hugging Face Space
213
+ 2. Note the Space URL: `wss://YOUR-SPACE.hf.space`
214
+ 3. Create a new profile:
215
+
216
+ ```bash
217
+ mkdir -p src/reachy_mini_remote_control_app/profiles/production
218
+ ```
219
+
220
+ Edit `profiles/production/config.txt`:
221
+ ```
222
+ WEBSOCKET_URI=wss://YOUR-SPACE.hf.space
223
+ VIDEO_JPEG_QUALITY=75
224
+ VIDEO_WITH_TIMESTAMP=true
225
+ VIDEO_FPS=25
226
+ AUDIO_WITH_TIMESTAMP=true
227
+ AUDIO_BATCH_SIZE=4096
228
+ ENABLE_METRICS_LOGGING=true
229
+ METRICS_LOG_INTERVAL_SEC=10
230
+ ```
231
+
232
+ 4. Set environment variable:
233
+ ```bash
234
+ export REMOTE_CONTROL_PROFILE=production
235
+ ```
236
+
237
+ 5. Run the app:
238
+ ```bash
239
+ reachy-mini-remote-control-app --profile production
240
+ ```
241
+
242
+ ### Network Requirements
243
+
244
+ - **Robot side**: Only **outbound** connections needed (no port forwarding required)
245
+ - **Server side**: WebSocket server must be publicly accessible
246
+ - **Protocol**: Use WSS (WebSocket Secure) for production
247
+
248
+ ### Security Considerations
249
+
250
+ - Use WSS (not WS) for encrypted connections
251
+ - Consider implementing authentication tokens
252
+ - Monitor connection logs for suspicious activity
253
+ - Rate limiting on the server side
254
+
255
+ ## Troubleshooting
256
+
257
+ ### Robot fails to connect to daemon
258
+
259
+ ```
260
+ Failed to connect to robot: Connection timeout
261
+ ```
262
+
263
+ **Solution**: Make sure the daemon is running:
264
+ ```bash
265
+ reachy-mini-daemon --mockup-sim
266
+ ```
267
+
268
+ ### WebSocket connection fails
269
+
270
+ ```
271
+ [Robot Control] Connection failed: Cannot connect to host
272
+ ```
273
+
274
+ **Solutions**:
275
+ 1. Verify the remote server is running and accessible
276
+ 2. Check the `WEBSOCKET_URI` in your profile
277
+ 3. Test connectivity: `curl http://YOUR-SERVER:8000/health` (if server has health endpoint)
278
+ 4. Check firewall rules
279
+
280
+ ### No video/audio streaming
281
+
282
+ **Solutions**:
283
+ 1. Ensure media is started: check daemon logs
284
+ 2. Verify WebSocket connections are established: check `/status` endpoint
285
+ 3. Check frame/audio publisher threads are running (look for log messages)
286
+
287
+ ### High latency
288
+
289
+ Check the `/status` endpoint for latency metrics:
290
+ ```bash
291
+ curl http://localhost:7860/status | jq '.video.latency_avg_ms'
292
+ ```
293
+
294
+ **Solutions**:
295
+ - Reduce `VIDEO_JPEG_QUALITY` in profile (e.g., from 80 to 60)
296
+ - Lower `VIDEO_FPS` (e.g., from 25 to 15)
297
+ - Use a server geographically closer to the robot
298
+ - Check network bandwidth
299
+
300
+ ## Development
301
+
302
+ ### Running Tests
303
+
304
+ ```bash
305
+ pytest tests/
306
+ ```
307
+
308
+ ### Code Structure
309
+
310
+ - `config.py` - Configuration management and profile loading
311
+ - `monitoring/` - Metrics tracking and connection monitoring
312
+ - `websocket_clients/` - WebSocket client implementations
313
+ - `robot_control.py` - Receive movement commands
314
+ - `video_stream.py` - Send video frames
315
+ - `audio_stream.py` - Bidirectional audio
316
+ - `app.py` - Optional REST API for status
317
+ - `main.py` - Entry point and ReachyMiniApp implementation
318
+
319
+ ## License
320
+
321
+ Same as Reachy Mini SDK.
322
+
323
+ ## Support
324
+
325
+ For issues and questions:
326
+ - GitHub Issues: [pollen-robotics/reachy_mini](https://github.com/pollen-robotics/reachy_mini/issues)
327
+ - Email: contact@pollen-robotics.com
pyproject.toml ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [build-system]
2
+ requires = ["setuptools"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "reachy_mini_remote_control_app"
7
+ version = "0.1.0"
8
+ authors = [{ name = "Pollen Robotics", email = "contact@pollen-robotics.com" }]
9
+ description = "Remote control WebSocket bridge for Reachy Mini"
10
+ readme = "README.md"
11
+ requires-python = ">=3.10"
12
+ dependencies = [
13
+ "fastapi>=0.109.0",
14
+ "uvicorn>=0.27.0",
15
+ "websockets>=12.0",
16
+ "opencv-python>=4.8.0",
17
+ "numpy>=1.24.0",
18
+ "python-dotenv>=1.0.0",
19
+ "reachy-mini @ git+https://github.com/pollen-robotics/reachy_mini.git@develop",
20
+ ]
21
+
22
+ [project.scripts]
23
+ reachy-mini-remote-control-app = "reachy_mini_remote_control_app.main:main"
24
+
25
+ [project.entry-points."reachy_mini_apps"]
26
+ reachy_mini_remote_control_app = "reachy_mini_remote_control_app.main:ReachyMiniRemoteControlApp"
27
+
28
+ [tool.setuptools]
29
+ package-dir = { "" = "src" }
30
+ include-package-data = true
31
+
32
+ [tool.setuptools.packages.find]
33
+ where = ["src"]
34
+
35
+ [tool.setuptools.package-data]
36
+ reachy_mini_remote_control_app = [
37
+ "profiles/**/*.txt",
38
+ ".env.example",
39
+ ]
src/reachy_mini_remote_control_app/__init__.py ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ """Reachy Mini Remote Control App."""
2
+
3
+ __version__ = "0.1.0"
src/reachy_mini_remote_control_app/app.py ADDED
@@ -0,0 +1,108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Optional FastAPI application for REST status endpoints."""
2
+
3
+ import logging
4
+ from pathlib import Path
5
+ from typing import Dict, List
6
+
7
+ from fastapi import FastAPI, HTTPException
8
+ from fastapi.responses import JSONResponse
9
+
10
+ from reachy_mini_remote_control_app.config import Config, load_profile, set_profile
11
+ from reachy_mini_remote_control_app.monitoring.connection_monitor import ConnectionMonitor
12
+
13
+
14
+ logger = logging.getLogger("reachy_mini_remote_control_app.app")
15
+
16
+
17
+ def create_fastapi_app(monitor: ConnectionMonitor, config: Config) -> FastAPI:
18
+ """Create FastAPI application with status endpoints.
19
+
20
+ Args:
21
+ monitor: ConnectionMonitor instance for metrics.
22
+ config: Config instance.
23
+
24
+ Returns:
25
+ FastAPI application.
26
+
27
+ """
28
+ app = FastAPI(
29
+ title="Reachy Mini Remote Control App",
30
+ description="REST API for remote control status and configuration",
31
+ version="0.1.0",
32
+ )
33
+
34
+ @app.get("/health")
35
+ async def health_check() -> Dict[str, str]:
36
+ """Health check endpoint."""
37
+ return {"status": "ok"}
38
+
39
+ @app.get("/status")
40
+ async def get_status() -> JSONResponse:
41
+ """Get current system status with metrics.
42
+
43
+ Returns:
44
+ JSON response with system status.
45
+
46
+ """
47
+ try:
48
+ status = monitor.get_status()
49
+ return JSONResponse(content=status.to_dict())
50
+ except Exception as e:
51
+ logger.error(f"Error getting status: {e}")
52
+ raise HTTPException(status_code=500, detail=str(e))
53
+
54
+ @app.get("/profiles")
55
+ async def list_profiles() -> Dict[str, List[str]]:
56
+ """List available profiles.
57
+
58
+ Returns:
59
+ Dictionary with list of available profiles.
60
+
61
+ """
62
+ try:
63
+ profiles_dir = Path(__file__).parent / "profiles"
64
+ profiles = [p.name for p in profiles_dir.iterdir() if p.is_dir()]
65
+ return {"profiles": profiles, "current": config.REMOTE_CONTROL_PROFILE}
66
+ except Exception as e:
67
+ logger.error(f"Error listing profiles: {e}")
68
+ raise HTTPException(status_code=500, detail=str(e))
69
+
70
+ @app.post("/profile/{profile_name}")
71
+ async def switch_profile(profile_name: str) -> Dict[str, str]:
72
+ """Switch to a different profile.
73
+
74
+ Note: Requires app restart to take effect.
75
+
76
+ Args:
77
+ profile_name: Name of the profile to switch to.
78
+
79
+ Returns:
80
+ Success message.
81
+
82
+ """
83
+ try:
84
+ # Verify profile exists
85
+ profiles_dir = Path(__file__).parent / "profiles"
86
+ profile_path = profiles_dir / profile_name
87
+
88
+ if not profile_path.exists():
89
+ raise HTTPException(status_code=404, detail=f"Profile '{profile_name}' not found")
90
+
91
+ # Load profile to verify it's valid
92
+ load_profile(profile_name)
93
+
94
+ # Set as active profile
95
+ set_profile(profile_name)
96
+
97
+ return {
98
+ "status": "success",
99
+ "message": f"Profile switched to '{profile_name}'. Restart app to apply changes.",
100
+ }
101
+
102
+ except HTTPException:
103
+ raise
104
+ except Exception as e:
105
+ logger.error(f"Error switching profile: {e}")
106
+ raise HTTPException(status_code=500, detail=str(e))
107
+
108
+ return app
src/reachy_mini_remote_control_app/config.py ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Configuration management for the remote control app."""
2
+
3
+ import os
4
+ from pathlib import Path
5
+ from typing import Dict, Optional
6
+
7
+ from dotenv import find_dotenv, load_dotenv
8
+
9
+
10
+ # Load environment variables
11
+ load_dotenv(find_dotenv())
12
+
13
+
14
+ class Config:
15
+ """Configuration class for remote control app."""
16
+
17
+ def __init__(self, profile: Optional[str] = None):
18
+ """Initialize configuration.
19
+
20
+ Args:
21
+ profile: Profile name to load. If None, uses REMOTE_CONTROL_PROFILE env var.
22
+
23
+ """
24
+ # Basic settings from environment
25
+ self.ROBOT_NAME = os.getenv("ROBOT_NAME", "reachy_mini")
26
+ self.LOG_LEVEL = os.getenv("LOG_LEVEL", "INFO")
27
+
28
+ # Profile management
29
+ profile_name = profile or os.getenv("REMOTE_CONTROL_PROFILE", "default")
30
+ self.REMOTE_CONTROL_PROFILE = profile_name
31
+
32
+ # Load profile-specific configuration
33
+ self.profile_config = load_profile(profile_name)
34
+
35
+ # WebSocket URI (from profile or environment)
36
+ self.WEBSOCKET_URI = self.profile_config.get(
37
+ "WEBSOCKET_URI", os.getenv("WEBSOCKET_URI", "ws://localhost:8000")
38
+ )
39
+
40
+ # Video settings
41
+ self.VIDEO_JPEG_QUALITY = int(
42
+ self.profile_config.get("VIDEO_JPEG_QUALITY", os.getenv("VIDEO_JPEG_QUALITY", "80"))
43
+ )
44
+ self.VIDEO_WITH_TIMESTAMP = (
45
+ self.profile_config.get("VIDEO_WITH_TIMESTAMP", "true").lower() == "true"
46
+ )
47
+ self.VIDEO_FPS = int(self.profile_config.get("VIDEO_FPS", "25"))
48
+
49
+ # Audio settings
50
+ self.AUDIO_SAMPLE_RATE = 16000 # Fixed
51
+ self.AUDIO_BATCH_SIZE = int(self.profile_config.get("AUDIO_BATCH_SIZE", "4096"))
52
+ self.AUDIO_WITH_TIMESTAMP = (
53
+ self.profile_config.get("AUDIO_WITH_TIMESTAMP", "true").lower() == "true"
54
+ )
55
+
56
+ # Monitoring
57
+ self.ENABLE_METRICS_LOGGING = (
58
+ self.profile_config.get("ENABLE_METRICS_LOGGING", "true").lower() == "true"
59
+ )
60
+ self.METRICS_LOG_INTERVAL_SEC = int(
61
+ self.profile_config.get("METRICS_LOG_INTERVAL_SEC", "5")
62
+ )
63
+ self.LATENCY_WARNING_MS = float(os.getenv("LATENCY_WARNING_MS", "200"))
64
+
65
+ # REST API settings
66
+ self.ENABLE_REST_API = os.getenv("ENABLE_REST_API", "true").lower() == "true"
67
+ self.REST_API_HOST = os.getenv("REST_API_HOST", "0.0.0.0")
68
+ self.REST_API_PORT = int(os.getenv("REST_API_PORT", "7860"))
69
+
70
+
71
+ def load_profile(profile_name: str) -> Dict[str, str]:
72
+ """Load profile configuration from profiles/{profile_name}/config.txt.
73
+
74
+ Args:
75
+ profile_name: Name of the profile to load.
76
+
77
+ Returns:
78
+ Dictionary of configuration key-value pairs.
79
+
80
+ """
81
+ profile_path = Path(__file__).parent / "profiles" / profile_name / "config.txt"
82
+
83
+ if not profile_path.exists():
84
+ print(f"Warning: Profile '{profile_name}' not found at {profile_path}, using defaults")
85
+ return {}
86
+
87
+ config = {}
88
+ with open(profile_path) as f:
89
+ for line in f:
90
+ line = line.strip()
91
+ # Skip empty lines and comments
92
+ if not line or line.startswith("#"):
93
+ continue
94
+ # Parse KEY=VALUE
95
+ if "=" in line:
96
+ key, value = line.split("=", 1)
97
+ config[key.strip()] = value.strip()
98
+
99
+ return config
100
+
101
+
102
+ def set_profile(profile_name: str) -> None:
103
+ """Switch active profile at runtime.
104
+
105
+ Args:
106
+ profile_name: Name of the profile to switch to.
107
+
108
+ """
109
+ # Update environment variable for consistency
110
+ os.environ["REMOTE_CONTROL_PROFILE"] = profile_name
111
+
112
+
113
+ # Global config instance
114
+ config = Config()
src/reachy_mini_remote_control_app/main.py ADDED
@@ -0,0 +1,297 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Main entry point for the Reachy Mini Remote Control App."""
2
+
3
+ import argparse
4
+ import logging
5
+ import sys
6
+ import threading
7
+ import time
8
+ from typing import Optional
9
+
10
+ from reachy_mini import ReachyMini, ReachyMiniApp
11
+
12
+ from reachy_mini_remote_control_app.app import create_fastapi_app
13
+ from reachy_mini_remote_control_app.config import Config
14
+ from reachy_mini_remote_control_app.monitoring.connection_monitor import ConnectionMonitor
15
+ from reachy_mini_remote_control_app.websocket_clients.audio_stream import AsyncWebSocketAudioStreamer
16
+ from reachy_mini_remote_control_app.websocket_clients.robot_control import AsyncWebSocketController
17
+ from reachy_mini_remote_control_app.websocket_clients.video_stream import AsyncWebSocketFrameSender
18
+
19
+
20
+ logger = logging.getLogger("reachy_mini_remote_control_app")
21
+
22
+
23
+ def setup_logging(log_level: str) -> None:
24
+ """Setup logging configuration.
25
+
26
+ Args:
27
+ log_level: Logging level (DEBUG, INFO, WARNING, ERROR).
28
+
29
+ """
30
+ logging.basicConfig(
31
+ level=getattr(logging, log_level.upper()),
32
+ format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
33
+ )
34
+
35
+
36
+ def parse_args() -> argparse.Namespace:
37
+ """Parse command line arguments.
38
+
39
+ Returns:
40
+ Parsed arguments.
41
+
42
+ """
43
+ parser = argparse.ArgumentParser(description="Reachy Mini Remote Control App")
44
+ parser.add_argument(
45
+ "--profile", default=None, help="Profile to use (overrides REMOTE_CONTROL_PROFILE env var)"
46
+ )
47
+ parser.add_argument(
48
+ "--websocket-uri", default=None, help="WebSocket URI to connect to (overrides profile)"
49
+ )
50
+ parser.add_argument(
51
+ "--robot-name", default=None, help="Robot name (overrides ROBOT_NAME env var)"
52
+ )
53
+ parser.add_argument("--log-level", default=None, help="Logging level (DEBUG, INFO, WARNING, ERROR)")
54
+ parser.add_argument("--no-rest-api", action="store_true", help="Disable REST API for status")
55
+
56
+ return parser.parse_args()
57
+
58
+
59
+ def _publish_frames(
60
+ robot: ReachyMini,
61
+ ws_video: AsyncWebSocketFrameSender,
62
+ stop_flag: threading.Event,
63
+ fps: int = 25,
64
+ ) -> None:
65
+ """Publish video frames to WebSocket.
66
+
67
+ Args:
68
+ robot: ReachyMini instance.
69
+ ws_video: AsyncWebSocketFrameSender instance.
70
+ stop_flag: Threading event to signal stop.
71
+ fps: Target frames per second.
72
+
73
+ """
74
+ frame_interval = 1.0 / fps
75
+ logger.info(f"[Frame Publisher] Starting video streaming at {fps} FPS")
76
+
77
+ while not stop_flag.is_set():
78
+ try:
79
+ # Get frame from robot
80
+ frame = robot.media.get_frame()
81
+
82
+ if frame is not None:
83
+ # Send frame
84
+ ws_video.send_frame(frame)
85
+
86
+ # Rate limiting
87
+ time.sleep(frame_interval)
88
+
89
+ except Exception as e:
90
+ logger.error(f"[Frame Publisher] Error: {e}")
91
+ time.sleep(1)
92
+
93
+
94
+ def _publish_audio(
95
+ robot: ReachyMini,
96
+ ws_audio: AsyncWebSocketAudioStreamer,
97
+ stop_flag: threading.Event,
98
+ ) -> None:
99
+ """Publish and receive audio to/from WebSocket.
100
+
101
+ Args:
102
+ robot: ReachyMini instance.
103
+ ws_audio: AsyncWebSocketAudioStreamer instance.
104
+ stop_flag: Threading event to signal stop.
105
+
106
+ """
107
+ logger.info("[Audio Publisher] Starting bidirectional audio streaming")
108
+
109
+ while not stop_flag.is_set():
110
+ try:
111
+ # Send robot audio to remote server
112
+ audio_sample = robot.media.get_audio_sample()
113
+ if audio_sample is not None:
114
+ ws_audio.send_audio_chunk(audio_sample)
115
+
116
+ # Receive audio from remote server and play on robot
117
+ received_audio = ws_audio.get_audio_chunk(timeout=0.01)
118
+ if received_audio is not None:
119
+ robot.media.push_audio_sample(received_audio)
120
+
121
+ except Exception as e:
122
+ logger.error(f"[Audio Publisher] Error: {e}")
123
+ time.sleep(0.1)
124
+
125
+
126
+ def run(
127
+ args: argparse.Namespace,
128
+ robot: Optional[ReachyMini] = None,
129
+ app_stop_event: Optional[threading.Event] = None,
130
+ ) -> None:
131
+ """Run the Reachy Mini remote control app.
132
+
133
+ Args:
134
+ args: Command line arguments.
135
+ robot: Optional pre-initialized ReachyMini instance.
136
+ app_stop_event: Optional stop event for graceful shutdown.
137
+
138
+ """
139
+ # Load configuration
140
+ config = Config(profile=args.profile)
141
+
142
+ # Override config with command line arguments
143
+ if args.websocket_uri:
144
+ config.WEBSOCKET_URI = args.websocket_uri
145
+ if args.robot_name:
146
+ config.ROBOT_NAME = args.robot_name
147
+ if args.log_level:
148
+ config.LOG_LEVEL = args.log_level
149
+ if args.no_rest_api:
150
+ config.ENABLE_REST_API = False
151
+
152
+ # Setup logging
153
+ setup_logging(config.LOG_LEVEL)
154
+
155
+ logger.info("Starting Reachy Mini Remote Control App")
156
+ logger.info(f"Profile: {config.REMOTE_CONTROL_PROFILE}")
157
+ logger.info(f"WebSocket URI: {config.WEBSOCKET_URI}")
158
+
159
+ # Initialize robot if not provided
160
+ if robot is None:
161
+ try:
162
+ logger.info(f"Connecting to robot: {config.ROBOT_NAME}")
163
+ robot = ReachyMini(robot_name=config.ROBOT_NAME)
164
+ logger.info("Robot connected successfully")
165
+ except Exception as e:
166
+ logger.error(f"Failed to connect to robot: {e}")
167
+ logger.error("Make sure the daemon is running (reachy-mini-daemon --mockup-sim)")
168
+ sys.exit(1)
169
+
170
+ # Start media recording and playback
171
+ try:
172
+ robot.media.start_recording()
173
+ robot.media.start_playing()
174
+ except Exception as e:
175
+ logger.warning(f"Failed to start media: {e}")
176
+
177
+ # Initialize connection monitor
178
+ monitor = ConnectionMonitor()
179
+ monitor.set_robot_connected(True)
180
+
181
+ # Initialize WebSocket clients
182
+ logger.info("Initializing WebSocket clients...")
183
+
184
+ ws_controller = AsyncWebSocketController(
185
+ ws_uri=config.WEBSOCKET_URI + "/robot", robot=robot, monitor=monitor
186
+ )
187
+
188
+ ws_video = AsyncWebSocketFrameSender(
189
+ ws_uri=config.WEBSOCKET_URI + "/video_stream",
190
+ jpeg_quality=config.VIDEO_JPEG_QUALITY,
191
+ with_timestamp=config.VIDEO_WITH_TIMESTAMP,
192
+ monitor=monitor,
193
+ )
194
+
195
+ ws_audio = AsyncWebSocketAudioStreamer(
196
+ ws_uri=config.WEBSOCKET_URI + "/audio_stream", monitor=monitor
197
+ )
198
+
199
+ logger.info("WebSocket clients initialized")
200
+
201
+ # Create stop event if not provided
202
+ if app_stop_event is None:
203
+ app_stop_event = threading.Event()
204
+
205
+ # Start frame and audio publishing threads
206
+ frame_thread = threading.Thread(
207
+ target=_publish_frames,
208
+ args=(robot, ws_video, app_stop_event, config.VIDEO_FPS),
209
+ daemon=True,
210
+ )
211
+ audio_thread = threading.Thread(
212
+ target=_publish_audio, args=(robot, ws_audio, app_stop_event), daemon=True
213
+ )
214
+
215
+ frame_thread.start()
216
+ audio_thread.start()
217
+
218
+ logger.info("Frame and audio publishers started")
219
+
220
+ try:
221
+ # Start REST API if enabled
222
+ if config.ENABLE_REST_API:
223
+ logger.info(f"Starting REST API on {config.REST_API_HOST}:{config.REST_API_PORT}")
224
+ app = create_fastapi_app(monitor, config)
225
+
226
+ import uvicorn
227
+
228
+ uvicorn.run(app, host=config.REST_API_HOST, port=config.REST_API_PORT)
229
+ else:
230
+ logger.info("REST API disabled, waiting for stop signal...")
231
+ app_stop_event.wait()
232
+
233
+ except KeyboardInterrupt:
234
+ logger.info("Keyboard interrupt received, shutting down...")
235
+
236
+ finally:
237
+ # Cleanup
238
+ logger.info("Cleaning up...")
239
+ app_stop_event.set()
240
+
241
+ # Stop WebSocket clients
242
+ ws_controller.stop()
243
+ ws_video.close()
244
+ ws_audio.close()
245
+
246
+ # Give threads time to finish
247
+ time.sleep(1)
248
+
249
+ # Close robot media
250
+ try:
251
+ robot.media.close()
252
+ except Exception as e:
253
+ logger.debug(f"Error closing media: {e}")
254
+
255
+ # Disconnect robot
256
+ try:
257
+ robot.client.disconnect()
258
+ except Exception as e:
259
+ logger.debug(f"Error disconnecting robot: {e}")
260
+
261
+ logger.info("Shutdown complete")
262
+
263
+
264
+ def main() -> None:
265
+ """CLI entry point."""
266
+ args = parse_args()
267
+ run(args)
268
+
269
+
270
+ class ReachyMiniRemoteControlApp(ReachyMiniApp): # type: ignore[misc]
271
+ """Reachy Mini Apps entry point for the remote control app."""
272
+
273
+ custom_app_url = "http://0.0.0.0:7860/"
274
+ dont_start_webserver = False
275
+
276
+ def run(self, reachy_mini: ReachyMini, stop_event: threading.Event) -> None:
277
+ """Run the remote control app from the dashboard.
278
+
279
+ Args:
280
+ reachy_mini: ReachyMini instance provided by dashboard.
281
+ stop_event: Stop event for graceful shutdown.
282
+
283
+ """
284
+ # Create minimal args for run()
285
+ args = argparse.Namespace(
286
+ profile=None,
287
+ websocket_uri=None,
288
+ robot_name=None,
289
+ log_level=None,
290
+ no_rest_api=False,
291
+ )
292
+
293
+ run(args, robot=reachy_mini, app_stop_event=stop_event)
294
+
295
+
296
+ if __name__ == "__main__":
297
+ main()
src/reachy_mini_remote_control_app/monitoring/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ """Monitoring and metrics tracking."""
src/reachy_mini_remote_control_app/monitoring/connection_monitor.py ADDED
@@ -0,0 +1,200 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Connection monitoring with thread-safe metrics tracking."""
2
+
3
+ import logging
4
+ import threading
5
+ import time
6
+ from collections import deque
7
+ from typing import Deque, Optional
8
+
9
+ from reachy_mini_remote_control_app.monitoring.metrics import (
10
+ AudioMetrics,
11
+ ControlMetrics,
12
+ SystemStatus,
13
+ VideoMetrics,
14
+ )
15
+
16
+
17
+ logger = logging.getLogger("reachy_mini_remote_control_app.monitor")
18
+
19
+
20
+ class ConnectionMonitor:
21
+ """Thread-safe connection metrics tracking."""
22
+
23
+ def __init__(self, latency_window_size: int = 100):
24
+ """Initialize the connection monitor.
25
+
26
+ Args:
27
+ latency_window_size: Number of latency samples to keep for statistics.
28
+
29
+ """
30
+ self._lock = threading.Lock()
31
+
32
+ # Metrics
33
+ self.control_metrics = ControlMetrics()
34
+ self.video_metrics = VideoMetrics()
35
+ self.audio_metrics = AudioMetrics()
36
+ self.robot_connected = False
37
+
38
+ # Rolling windows for latency tracking
39
+ self._video_latencies: Deque[float] = deque(maxlen=latency_window_size)
40
+ self._audio_latencies: Deque[float] = deque(maxlen=latency_window_size)
41
+
42
+ # FPS tracking for video
43
+ self._frame_timestamps: Deque[float] = deque(maxlen=100)
44
+
45
+ def update_control_metrics(
46
+ self,
47
+ connected: Optional[bool] = None,
48
+ command_type: Optional[str] = None,
49
+ ) -> None:
50
+ """Update control metrics.
51
+
52
+ Args:
53
+ connected: Connection status.
54
+ command_type: Type of command received.
55
+
56
+ """
57
+ with self._lock:
58
+ if connected is not None:
59
+ self.control_metrics.connected = connected
60
+
61
+ if command_type:
62
+ self.control_metrics.commands_received += 1
63
+ self.control_metrics.last_command_type = command_type
64
+
65
+ self.control_metrics.last_activity = time.time()
66
+
67
+ def update_video_metrics(
68
+ self,
69
+ connected: Optional[bool] = None,
70
+ frame_sent: bool = False,
71
+ frame_dropped: bool = False,
72
+ latency_ms: Optional[float] = None,
73
+ ) -> None:
74
+ """Update video metrics.
75
+
76
+ Args:
77
+ connected: Connection status.
78
+ frame_sent: Whether a frame was sent.
79
+ frame_dropped: Whether a frame was dropped.
80
+ latency_ms: Latency in milliseconds.
81
+
82
+ """
83
+ with self._lock:
84
+ if connected is not None:
85
+ self.video_metrics.connected = connected
86
+
87
+ if frame_sent:
88
+ self.video_metrics.frames_sent += 1
89
+ now = time.time()
90
+ self._frame_timestamps.append(now)
91
+
92
+ # Calculate FPS from last 100 frames
93
+ if len(self._frame_timestamps) > 1:
94
+ time_span = self._frame_timestamps[-1] - self._frame_timestamps[0]
95
+ if time_span > 0:
96
+ self.video_metrics.fps = (len(self._frame_timestamps) - 1) / time_span
97
+
98
+ if frame_dropped:
99
+ self.video_metrics.frames_dropped += 1
100
+
101
+ if latency_ms is not None:
102
+ self._video_latencies.append(latency_ms)
103
+ self.video_metrics.latency_ms = latency_ms
104
+
105
+ # Update statistics
106
+ if self._video_latencies:
107
+ self.video_metrics.latency_avg_ms = sum(self._video_latencies) / len(
108
+ self._video_latencies
109
+ )
110
+ self.video_metrics.latency_min_ms = min(self._video_latencies)
111
+ self.video_metrics.latency_max_ms = max(self._video_latencies)
112
+
113
+ self.video_metrics.last_activity = time.time()
114
+
115
+ def update_audio_metrics(
116
+ self,
117
+ connected: Optional[bool] = None,
118
+ chunk_sent: bool = False,
119
+ chunk_received: bool = False,
120
+ latency_ms: Optional[float] = None,
121
+ ) -> None:
122
+ """Update audio metrics.
123
+
124
+ Args:
125
+ connected: Connection status.
126
+ chunk_sent: Whether a chunk was sent.
127
+ chunk_received: Whether a chunk was received.
128
+ latency_ms: Latency in milliseconds.
129
+
130
+ """
131
+ with self._lock:
132
+ if connected is not None:
133
+ self.audio_metrics.connected = connected
134
+
135
+ if chunk_sent:
136
+ self.audio_metrics.chunks_sent += 1
137
+
138
+ if chunk_received:
139
+ self.audio_metrics.chunks_received += 1
140
+
141
+ if latency_ms is not None:
142
+ self._audio_latencies.append(latency_ms)
143
+
144
+ # Update statistics
145
+ if self._audio_latencies:
146
+ self.audio_metrics.latency_avg_ms = sum(self._audio_latencies) / len(
147
+ self._audio_latencies
148
+ )
149
+ self.audio_metrics.latency_min_ms = min(self._audio_latencies)
150
+ self.audio_metrics.latency_max_ms = max(self._audio_latencies)
151
+
152
+ self.audio_metrics.last_activity = time.time()
153
+
154
+ def set_robot_connected(self, connected: bool) -> None:
155
+ """Set robot connection status.
156
+
157
+ Args:
158
+ connected: Whether the robot is connected.
159
+
160
+ """
161
+ with self._lock:
162
+ self.robot_connected = connected
163
+
164
+ def get_status(self) -> SystemStatus:
165
+ """Get current system status.
166
+
167
+ Returns:
168
+ SystemStatus with current metrics.
169
+
170
+ """
171
+ with self._lock:
172
+ return SystemStatus(
173
+ robot_connected=self.robot_connected,
174
+ control=ControlMetrics(
175
+ connected=self.control_metrics.connected,
176
+ commands_received=self.control_metrics.commands_received,
177
+ last_command_type=self.control_metrics.last_command_type,
178
+ last_activity=self.control_metrics.last_activity,
179
+ ),
180
+ video=VideoMetrics(
181
+ connected=self.video_metrics.connected,
182
+ frames_sent=self.video_metrics.frames_sent,
183
+ frames_dropped=self.video_metrics.frames_dropped,
184
+ fps=self.video_metrics.fps,
185
+ latency_ms=self.video_metrics.latency_ms,
186
+ latency_avg_ms=self.video_metrics.latency_avg_ms,
187
+ latency_min_ms=self.video_metrics.latency_min_ms,
188
+ latency_max_ms=self.video_metrics.latency_max_ms,
189
+ last_activity=self.video_metrics.last_activity,
190
+ ),
191
+ audio=AudioMetrics(
192
+ connected=self.audio_metrics.connected,
193
+ chunks_sent=self.audio_metrics.chunks_sent,
194
+ chunks_received=self.audio_metrics.chunks_received,
195
+ latency_avg_ms=self.audio_metrics.latency_avg_ms,
196
+ latency_min_ms=self.audio_metrics.latency_min_ms,
197
+ latency_max_ms=self.audio_metrics.latency_max_ms,
198
+ last_activity=self.audio_metrics.last_activity,
199
+ ),
200
+ )
src/reachy_mini_remote_control_app/monitoring/metrics.py ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Metrics dataclasses for connection monitoring."""
2
+
3
+ from dataclasses import dataclass, field
4
+ from typing import Dict, Any, Optional
5
+
6
+
7
+ @dataclass
8
+ class VideoMetrics:
9
+ """Metrics for video streaming."""
10
+
11
+ connected: bool = False
12
+ frames_sent: int = 0
13
+ frames_dropped: int = 0
14
+ fps: float = 0.0
15
+ latency_ms: Optional[float] = None
16
+ latency_avg_ms: float = 0.0
17
+ latency_min_ms: float = float("inf")
18
+ latency_max_ms: float = 0.0
19
+ last_activity: float = 0.0
20
+
21
+ def to_dict(self) -> Dict[str, Any]:
22
+ """Convert to dictionary for JSON serialization."""
23
+ return {
24
+ "connected": self.connected,
25
+ "frames_sent": self.frames_sent,
26
+ "frames_dropped": self.frames_dropped,
27
+ "fps": round(self.fps, 2),
28
+ "latency_ms": round(self.latency_ms, 2) if self.latency_ms is not None else None,
29
+ "latency_avg_ms": round(self.latency_avg_ms, 2),
30
+ "latency_min_ms": round(self.latency_min_ms, 2)
31
+ if self.latency_min_ms != float("inf")
32
+ else None,
33
+ "latency_max_ms": round(self.latency_max_ms, 2),
34
+ "last_activity": self.last_activity,
35
+ }
36
+
37
+
38
+ @dataclass
39
+ class AudioMetrics:
40
+ """Metrics for audio streaming."""
41
+
42
+ connected: bool = False
43
+ chunks_sent: int = 0
44
+ chunks_received: int = 0
45
+ latency_avg_ms: float = 0.0
46
+ latency_min_ms: float = float("inf")
47
+ latency_max_ms: float = 0.0
48
+ last_activity: float = 0.0
49
+
50
+ def to_dict(self) -> Dict[str, Any]:
51
+ """Convert to dictionary for JSON serialization."""
52
+ return {
53
+ "connected": self.connected,
54
+ "chunks_sent": self.chunks_sent,
55
+ "chunks_received": self.chunks_received,
56
+ "latency_avg_ms": round(self.latency_avg_ms, 2),
57
+ "latency_min_ms": round(self.latency_min_ms, 2)
58
+ if self.latency_min_ms != float("inf")
59
+ else None,
60
+ "latency_max_ms": round(self.latency_max_ms, 2),
61
+ "last_activity": self.last_activity,
62
+ }
63
+
64
+
65
+ @dataclass
66
+ class ControlMetrics:
67
+ """Metrics for robot control commands."""
68
+
69
+ connected: bool = False
70
+ commands_received: int = 0
71
+ last_command_type: Optional[str] = None
72
+ last_activity: float = 0.0
73
+
74
+ def to_dict(self) -> Dict[str, Any]:
75
+ """Convert to dictionary for JSON serialization."""
76
+ return {
77
+ "connected": self.connected,
78
+ "commands_received": self.commands_received,
79
+ "last_command_type": self.last_command_type,
80
+ "last_activity": self.last_activity,
81
+ }
82
+
83
+
84
+ @dataclass
85
+ class SystemStatus:
86
+ """Overall system status."""
87
+
88
+ robot_connected: bool = False
89
+ control: ControlMetrics = field(default_factory=ControlMetrics)
90
+ video: VideoMetrics = field(default_factory=VideoMetrics)
91
+ audio: AudioMetrics = field(default_factory=AudioMetrics)
92
+
93
+ def to_dict(self) -> Dict[str, Any]:
94
+ """Convert to dictionary for JSON serialization."""
95
+ return {
96
+ "robot_connected": self.robot_connected,
97
+ "control": self.control.to_dict(),
98
+ "video": self.video.to_dict(),
99
+ "audio": self.audio.to_dict(),
100
+ }
src/reachy_mini_remote_control_app/profiles/default/config.txt ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Remote server URI to connect to
2
+ # Example: ws://YOUR-SPACE.hf.space (Hugging Face Space)
3
+ # For local testing: ws://localhost:8000
4
+ WEBSOCKET_URI=ws://localhost:8000
5
+
6
+ # Video settings
7
+ VIDEO_JPEG_QUALITY=80
8
+ VIDEO_WITH_TIMESTAMP=false
9
+ VIDEO_FPS=25
10
+
11
+ # Audio settings
12
+ AUDIO_WITH_TIMESTAMP=false
13
+ AUDIO_BATCH_SIZE=4096
14
+
15
+ # Monitoring
16
+ ENABLE_METRICS_LOGGING=true
17
+ METRICS_LOG_INTERVAL_SEC=5
src/reachy_mini_remote_control_app/profiles/example_remote/config.txt ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Example profile for remote Hugging Face Space deployment
2
+ # Replace YOUR-SPACE with your actual Hugging Face Space name
3
+
4
+ # Remote server URI (Hugging Face Space)
5
+ WEBSOCKET_URI=wss://YOUR-SPACE.hf.space
6
+
7
+ # Video settings
8
+ VIDEO_JPEG_QUALITY=75
9
+ VIDEO_WITH_TIMESTAMP=true
10
+ VIDEO_FPS=25
11
+
12
+ # Audio settings
13
+ AUDIO_WITH_TIMESTAMP=true
14
+ AUDIO_BATCH_SIZE=4096
15
+
16
+ # Monitoring
17
+ ENABLE_METRICS_LOGGING=true
18
+ METRICS_LOG_INTERVAL_SEC=10
src/reachy_mini_remote_control_app/websocket_clients/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ """WebSocket client implementations."""
src/reachy_mini_remote_control_app/websocket_clients/audio_stream.py ADDED
@@ -0,0 +1,259 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """WebSocket client for bidirectional audio streaming."""
2
+
3
+ import asyncio
4
+ import logging
5
+ import threading
6
+ import time
7
+ from queue import Empty, Queue
8
+ from typing import Optional, Union
9
+
10
+ import numpy as np
11
+ import numpy.typing as npt
12
+ from websockets.asyncio.client import ClientConnection, connect
13
+
14
+ from reachy_mini_remote_control_app.monitoring.connection_monitor import ConnectionMonitor
15
+
16
+
17
+ logger = logging.getLogger("reachy_mini_remote_control_app.audio_stream")
18
+
19
+
20
+ class AsyncWebSocketAudioStreamer:
21
+ """WebSocket client for bidirectional audio streaming with remote server."""
22
+
23
+ # Target ~2048 samples per packet (approx 128ms)
24
+ # 2048 samples * 2 bytes (int16) = 4096 bytes
25
+ BATCH_SIZE_BYTES = 4096
26
+ # Don't hold audio longer than 200ms even if buffer isn't full
27
+ BATCH_TIMEOUT = 0.2
28
+
29
+ def __init__(
30
+ self,
31
+ ws_uri: str,
32
+ keep_alive_interval: float = 2.0,
33
+ monitor: Optional[ConnectionMonitor] = None,
34
+ ) -> None:
35
+ """Initialize the WebSocket audio streamer.
36
+
37
+ Args:
38
+ ws_uri: WebSocket URI to connect to (e.g., ws://server:8000/audio_stream).
39
+ keep_alive_interval: Interval in seconds to send keep-alive pings when no audio.
40
+ monitor: Optional ConnectionMonitor for metrics tracking.
41
+
42
+ """
43
+ self.ws_uri = ws_uri
44
+ self.keep_alive_interval = keep_alive_interval
45
+ self.monitor = monitor
46
+ self.send_queue: "Queue[bytes]" = Queue()
47
+ self.recv_queue: "Queue[bytes]" = Queue()
48
+ self.loop = asyncio.new_event_loop()
49
+ self.thread = threading.Thread(target=self._run_loop, daemon=True)
50
+ self.connected = threading.Event()
51
+ self.stop_flag = False
52
+ self.thread.start()
53
+
54
+ def _run_loop(self) -> None:
55
+ """Run the WebSocket streamer loop in a background thread."""
56
+ asyncio.set_event_loop(self.loop)
57
+ self.loop.run_until_complete(self._run())
58
+
59
+ async def _run(self) -> None:
60
+ """Run the main reconnect loop."""
61
+ while not self.stop_flag:
62
+ try:
63
+ async with connect(self.ws_uri) as ws:
64
+ logger.info("[Audio Stream] Connected to remote server")
65
+ self.connected.set()
66
+
67
+ # Update connection status
68
+ if self.monitor:
69
+ self.monitor.update_audio_metrics(connected=True)
70
+
71
+ # Run send and receive tasks concurrently
72
+ send_task = asyncio.create_task(self._send_loop(ws))
73
+ recv_task = asyncio.create_task(self._recv_loop(ws))
74
+
75
+ done, pending = await asyncio.wait(
76
+ {send_task, recv_task},
77
+ return_when=asyncio.FIRST_EXCEPTION,
78
+ )
79
+
80
+ # Cancel the other task if one fails or finishes
81
+ for task in pending:
82
+ task.cancel()
83
+ try:
84
+ await task
85
+ except Exception:
86
+ pass
87
+
88
+ except Exception as e:
89
+ logger.info(f"[Audio Stream] Connection failed: {e}")
90
+
91
+ # Update connection status
92
+ if self.monitor:
93
+ self.monitor.update_audio_metrics(connected=False)
94
+
95
+ await asyncio.sleep(1.0)
96
+
97
+ self.connected.clear()
98
+
99
+ async def _send_loop(self, ws: ClientConnection) -> None:
100
+ """Send outgoing audio chunks and keep-alive pings.
101
+
102
+ Args:
103
+ ws: WebSocket connection.
104
+
105
+ """
106
+ last_activity = time.time()
107
+ batch_buffer = bytearray()
108
+ batch_start_time = time.time()
109
+
110
+ while not self.stop_flag:
111
+ try:
112
+ # Try to pull data from the queue
113
+ chunk = self.send_queue.get(timeout=0.01)
114
+
115
+ # If this is the first chunk in the buffer, reset timer
116
+ if len(batch_buffer) == 0:
117
+ batch_start_time = time.time()
118
+
119
+ batch_buffer.extend(chunk)
120
+
121
+ except Empty:
122
+ pass
123
+ except Exception as e:
124
+ logger.info(f"[Audio Stream] Queue error: {e}")
125
+ break
126
+
127
+ # Check if we should send the batch
128
+ now = time.time()
129
+
130
+ # Condition A: Buffer is full enough (size based)
131
+ is_full = len(batch_buffer) >= self.BATCH_SIZE_BYTES
132
+
133
+ # Condition B: Buffer has data but it's getting too old (time based)
134
+ is_timed_out = (len(batch_buffer) > 0) and ((now - batch_start_time) > self.BATCH_TIMEOUT)
135
+
136
+ if is_full or is_timed_out:
137
+ try:
138
+ # Send the aggregated buffer
139
+ await ws.send(batch_buffer) # type: ignore
140
+
141
+ # Update metrics
142
+ if self.monitor:
143
+ self.monitor.update_audio_metrics(chunk_sent=True)
144
+
145
+ # Reset
146
+ batch_buffer = bytearray()
147
+ last_activity = now
148
+
149
+ except Exception as e:
150
+ logger.info(f"[Audio Stream] Send error: {e}")
151
+ break
152
+
153
+ # Keep-alive ping (only if completely idle)
154
+ if len(batch_buffer) == 0 and (now - last_activity) > self.keep_alive_interval:
155
+ try:
156
+ await ws.send("ping")
157
+ last_activity = now
158
+ logger.debug("[Audio Stream] Sent keep-alive ping")
159
+ except Exception as e:
160
+ logger.info(f"[Audio Stream] Ping failed: {e}")
161
+ break
162
+
163
+ # Tiny sleep to yield control if we are just spinning
164
+ if len(batch_buffer) == 0:
165
+ await asyncio.sleep(0.001)
166
+
167
+ async def _recv_loop(self, ws: ClientConnection) -> None:
168
+ """Receive incoming audio chunks.
169
+
170
+ Args:
171
+ ws: WebSocket connection.
172
+
173
+ """
174
+ while not self.stop_flag:
175
+ try:
176
+ msg = await ws.recv()
177
+ except Exception as e:
178
+ logger.info(f"[Audio Stream] Receive error: {e}")
179
+ break
180
+
181
+ if isinstance(msg, bytes):
182
+ try:
183
+ self.recv_queue.put_nowait(msg)
184
+
185
+ # Update metrics
186
+ if self.monitor:
187
+ self.monitor.update_audio_metrics(chunk_received=True)
188
+
189
+ except Exception as e:
190
+ logger.debug(f"[Audio Stream] Failed to enqueue received audio: {e}")
191
+ else:
192
+ logger.debug(f"[Audio Stream] Received non-binary message: {msg}")
193
+
194
+ def send_audio_chunk(
195
+ self,
196
+ audio: Union[bytes, npt.NDArray[np.int16], npt.NDArray[np.float32]],
197
+ ) -> None:
198
+ """Queue an audio chunk to be sent.
199
+
200
+ Args:
201
+ audio: Either raw bytes or a numpy array of int16 or float32.
202
+ Float32 arrays are assumed to be in [-1, 1] and will be converted to int16 PCM.
203
+
204
+ """
205
+ if self.stop_flag:
206
+ return
207
+
208
+ if isinstance(audio, bytes):
209
+ data = audio
210
+ else:
211
+ # Convert only if needed
212
+ arr = np.asarray(audio)
213
+ # Handle stereo if accidentally passed (take channel 0)
214
+ if arr.ndim > 1:
215
+ arr = arr[:, 0]
216
+
217
+ if arr.dtype == np.float32 or arr.dtype == np.float64:
218
+ # Scale if any value is above 1 or below -1
219
+ max_abs = np.max(np.abs(arr))
220
+ if max_abs > 1.0:
221
+ arr = arr / max_abs
222
+ # Convert float audio [-1,1] to int16 PCM
223
+ arr = np.clip(arr, -1.0, 1.0)
224
+ arr = (arr * 32767.0).astype(np.int16) # type: ignore
225
+ elif arr.dtype != np.int16:
226
+ arr = arr.astype(np.int16)
227
+
228
+ data = arr.tobytes()
229
+
230
+ self.send_queue.put(data)
231
+
232
+ def get_audio_chunk(self, timeout: Optional[float] = 0.01) -> Optional[npt.NDArray[np.float32]]:
233
+ """Retrieve a received audio chunk, if any.
234
+
235
+ Args:
236
+ timeout: Timeout in seconds for waiting for a chunk.
237
+
238
+ Returns:
239
+ Audio chunk as float32 array in [-1, 1] range, or None if no chunk available.
240
+
241
+ """
242
+ try:
243
+ if timeout == 0:
244
+ audio_bytes = self.recv_queue.get_nowait()
245
+ else:
246
+ audio_bytes = self.recv_queue.get(timeout=timeout)
247
+
248
+ # bytes -> int16 -> float32 in [-1, 1]
249
+ int16_arr = np.frombuffer(audio_bytes, dtype=np.int16)
250
+ float_arr = int16_arr.astype(np.float32) / 32767.0
251
+ return float_arr
252
+ except Empty:
253
+ return None
254
+
255
+ def close(self) -> None:
256
+ """Close the WebSocket audio streamer."""
257
+ self.stop_flag = True
258
+ if self.loop.is_running():
259
+ self.loop.call_soon_threadsafe(self.loop.stop)
src/reachy_mini_remote_control_app/websocket_clients/robot_control.py ADDED
@@ -0,0 +1,136 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """WebSocket client for receiving robot control commands."""
2
+
3
+ import asyncio
4
+ import json
5
+ import logging
6
+ import threading
7
+ from typing import Any, Dict, Optional
8
+
9
+ import numpy as np
10
+ from reachy_mini import ReachyMini
11
+ from websockets.asyncio.client import ClientConnection, connect
12
+
13
+ from reachy_mini_remote_control_app.monitoring.connection_monitor import ConnectionMonitor
14
+
15
+
16
+ logger = logging.getLogger("reachy_mini_remote_control_app.robot_control")
17
+
18
+
19
+ class AsyncWebSocketController:
20
+ """WebSocket client for receiving robot control commands from remote server."""
21
+
22
+ def __init__(
23
+ self,
24
+ ws_uri: str,
25
+ robot: ReachyMini,
26
+ monitor: Optional[ConnectionMonitor] = None,
27
+ ) -> None:
28
+ """Initialize the WebSocket controller.
29
+
30
+ Args:
31
+ ws_uri: WebSocket URI to connect to (e.g., ws://server:8000/robot).
32
+ robot: ReachyMini instance for robot control.
33
+ monitor: Optional ConnectionMonitor for metrics tracking.
34
+
35
+ """
36
+ self.ws_uri = ws_uri
37
+ self.robot = robot
38
+ self.monitor = monitor
39
+ self.loop = asyncio.new_event_loop()
40
+ self.thread = threading.Thread(target=self._run_loop, daemon=True)
41
+ self.stop_flag = False
42
+ self.thread.start()
43
+
44
+ def _run_loop(self) -> None:
45
+ """Run the WebSocket controller loop in a background thread."""
46
+ asyncio.set_event_loop(self.loop)
47
+ self.loop.run_until_complete(self._run())
48
+
49
+ async def on_command(self, cmd: Dict[str, Any]) -> None:
50
+ """Handle a command from the WebSocket.
51
+
52
+ Args:
53
+ cmd: Command dictionary from remote server.
54
+
55
+ """
56
+ typ = cmd.get("type")
57
+
58
+ if typ == "movement":
59
+ logger.debug("[Robot Control] Movement command received")
60
+ mov = cmd.get("movement", {})
61
+ logger.debug("[Robot Control] Movement command: %s", mov)
62
+
63
+ # Parse head matrix
64
+ head = mov.get("head")
65
+ if head is not None:
66
+ head_arr = np.array(head, dtype=float).reshape(4, 4)
67
+ else:
68
+ head_arr = None
69
+
70
+ # Parse antennas
71
+ antennas = mov.get("antennas")
72
+ if antennas is not None:
73
+ antennas_arr = np.array(antennas, dtype=float)
74
+ else:
75
+ antennas_arr = None
76
+
77
+ try:
78
+ # Execute movement command
79
+ await self.robot.goto(
80
+ head=head_arr,
81
+ antennas=antennas_arr,
82
+ duration=mov.get("duration", 1.0),
83
+ body_yaw=mov.get("body_yaw", 0.0),
84
+ )
85
+
86
+ # Update metrics
87
+ if self.monitor:
88
+ self.monitor.update_control_metrics(command_type="movement")
89
+
90
+ except Exception as e:
91
+ logger.error("[Robot Control] Error in goto: %s", e)
92
+
93
+ elif typ == "ping":
94
+ logger.debug("[Robot Control] Ping received")
95
+ return
96
+ else:
97
+ logger.debug("[Robot Control] Unknown command type: %s", typ)
98
+
99
+ async def _run(self) -> None:
100
+ """Run the WebSocket controller loop with automatic reconnection."""
101
+ while not self.stop_flag:
102
+ try:
103
+ ws: ClientConnection
104
+ async with connect(self.ws_uri, ping_interval=5, ping_timeout=10) as ws:
105
+ logger.info("[Robot Control] Connected to remote server")
106
+
107
+ # Update connection status
108
+ if self.monitor:
109
+ self.monitor.update_control_metrics(connected=True)
110
+
111
+ # Receive and process commands
112
+ async for msg in ws:
113
+ try:
114
+ data = json.loads(msg)
115
+ except Exception as e:
116
+ logger.debug("[Robot Control] Bad JSON: %s raw: %s", e, msg)
117
+ continue
118
+
119
+ # Process command
120
+ await self.on_command(data)
121
+
122
+ except Exception as e:
123
+ logger.info("[Robot Control] Connection failed: %s", e)
124
+
125
+ # Update connection status
126
+ if self.monitor:
127
+ self.monitor.update_control_metrics(connected=False)
128
+
129
+ # Backoff before reconnect
130
+ await asyncio.sleep(1)
131
+
132
+ def stop(self) -> None:
133
+ """Stop the WebSocket controller."""
134
+ self.stop_flag = True
135
+ if self.loop.is_running():
136
+ self.loop.call_soon_threadsafe(lambda: None)
src/reachy_mini_remote_control_app/websocket_clients/video_stream.py ADDED
@@ -0,0 +1,188 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """WebSocket client for sending video frames to remote server."""
2
+
3
+ import asyncio
4
+ import base64
5
+ import json
6
+ import logging
7
+ import threading
8
+ import time
9
+ from queue import Empty, Full, Queue
10
+ from typing import Optional
11
+
12
+ import cv2
13
+ import numpy as np
14
+ import numpy.typing as npt
15
+ from websockets.asyncio.client import ClientConnection, connect
16
+ from websockets.exceptions import ConnectionClosed
17
+
18
+ from reachy_mini_remote_control_app.monitoring.connection_monitor import ConnectionMonitor
19
+
20
+
21
+ logger = logging.getLogger("reachy_mini_remote_control_app.video_stream")
22
+
23
+
24
+ class AsyncWebSocketFrameSender:
25
+ """WebSocket client for sending video frames to remote server."""
26
+
27
+ def __init__(
28
+ self,
29
+ ws_uri: str,
30
+ jpeg_quality: int = 80,
31
+ with_timestamp: bool = False,
32
+ monitor: Optional[ConnectionMonitor] = None,
33
+ ) -> None:
34
+ """Initialize the WebSocket frame sender.
35
+
36
+ Args:
37
+ ws_uri: WebSocket URI to connect to (e.g., ws://server:8000/video_stream).
38
+ jpeg_quality: JPEG encoding quality (0-100).
39
+ with_timestamp: Whether to send timestamp with frames.
40
+ monitor: Optional ConnectionMonitor for metrics tracking.
41
+
42
+ """
43
+ self.ws_uri = ws_uri
44
+ self.jpeg_quality = jpeg_quality
45
+ self.with_timestamp = with_timestamp
46
+ self.monitor = monitor
47
+ self.queue: "Queue[bytes]" = Queue(maxsize=2)
48
+ self.loop = asyncio.new_event_loop()
49
+ self.thread = threading.Thread(target=self._run_loop, daemon=True)
50
+ self.connected = threading.Event()
51
+ self.stop_flag = False
52
+ self._last_frame: Optional[npt.NDArray[np.uint8]] = None
53
+ self.thread.start()
54
+
55
+ def _run_loop(self) -> None:
56
+ """Run the WebSocket loop in a background thread."""
57
+ asyncio.set_event_loop(self.loop)
58
+ self.loop.run_until_complete(self._run())
59
+
60
+ def _clear_queue(self) -> None:
61
+ """Empty the queue so we don't send old video on reconnect."""
62
+ while not self.queue.empty():
63
+ try:
64
+ self.queue.get_nowait()
65
+ except Empty:
66
+ break
67
+
68
+ async def _run(self) -> None:
69
+ """Run the WebSocket frame sender loop with automatic reconnection."""
70
+ while not self.stop_flag:
71
+ try:
72
+ ws: ClientConnection
73
+ async with connect(
74
+ self.ws_uri,
75
+ ping_interval=5,
76
+ ping_timeout=10,
77
+ close_timeout=1,
78
+ ) as ws:
79
+ logger.info("[Video Stream] Connected to remote server")
80
+ self.connected.set()
81
+ self._clear_queue()
82
+
83
+ # Update connection status
84
+ if self.monitor:
85
+ self.monitor.update_video_metrics(connected=True)
86
+
87
+ while not self.stop_flag:
88
+ try:
89
+ frame_data = self.queue.get_nowait()
90
+
91
+ # Send frame
92
+ await ws.send(frame_data)
93
+
94
+ # Update metrics
95
+ if self.monitor:
96
+ self.monitor.update_video_metrics(frame_sent=True)
97
+
98
+ except Empty:
99
+ # Queue is empty, yield to event loop
100
+ await asyncio.sleep(0.05)
101
+ continue
102
+
103
+ except Exception as e:
104
+ logger.error(f"[Video Stream] Send error: {e}")
105
+ break
106
+
107
+ except ConnectionClosed as e:
108
+ logger.error(f"[Video Stream] Connection lost ({type(e).__name__}). Retrying...")
109
+ self.connected.clear()
110
+ self._last_frame = None
111
+
112
+ # Update connection status
113
+ if self.monitor:
114
+ self.monitor.update_video_metrics(connected=False)
115
+
116
+ await asyncio.sleep(0.5)
117
+
118
+ except Exception as e:
119
+ logger.error(f"[Video Stream] Unexpected error: {e}")
120
+
121
+ # Update connection status
122
+ if self.monitor:
123
+ self.monitor.update_video_metrics(connected=False)
124
+
125
+ await asyncio.sleep(1)
126
+
127
+ self.connected.clear()
128
+ self._last_frame = None
129
+
130
+ def send_frame(self, frame: npt.NDArray[np.uint8]) -> None:
131
+ """Send a frame to the WebSocket (non-blocking).
132
+
133
+ Args:
134
+ frame: Video frame as numpy array (BGR format).
135
+
136
+ """
137
+ if self.stop_flag:
138
+ return
139
+
140
+ # Frame deduplication
141
+ if self._last_frame is not None:
142
+ if np.array_equal(frame, self._last_frame):
143
+ return
144
+ self._last_frame = frame.copy()
145
+
146
+ # Encode to JPEG
147
+ ok, jpeg_bytes = cv2.imencode(
148
+ ".jpg",
149
+ frame,
150
+ [int(cv2.IMWRITE_JPEG_QUALITY), self.jpeg_quality],
151
+ )
152
+ if not ok:
153
+ return
154
+
155
+ # Prepare data for sending
156
+ if self.with_timestamp:
157
+ # Send as JSON with timestamp
158
+ data = json.dumps(
159
+ {
160
+ "timestamp": time.time(),
161
+ "frame": base64.b64encode(jpeg_bytes.tobytes()).decode(),
162
+ }
163
+ ).encode()
164
+ else:
165
+ # Send as raw bytes
166
+ data = jpeg_bytes.tobytes()
167
+
168
+ # Add to queue (drop oldest if full)
169
+ try:
170
+ self.queue.put_nowait(data)
171
+ except Full:
172
+ # Queue is full, network is slower than camera
173
+ # Drop the oldest frame to make room for the newest
174
+ try:
175
+ self.queue.get_nowait() # Pop old frame
176
+ self.queue.put_nowait(data) # Push new frame
177
+
178
+ # Track dropped frame
179
+ if self.monitor:
180
+ self.monitor.update_video_metrics(frame_dropped=True)
181
+
182
+ except Empty:
183
+ logger.error("[Video Stream] Queue is full and empty, this should not happen.")
184
+
185
+ def close(self) -> None:
186
+ """Close the WebSocket frame sender."""
187
+ self.stop_flag = True
188
+ self.loop.call_soon_threadsafe(self.loop.stop)