Butterfly Desktop Environment — Progress

Overview

A remote desktop environment with a Rust (Actix) backend and Angular 21 frontend. The system mimics a traditional Windows-like desktop in the browser, receiving display/audio from VM agents with minimal lag.

Architecture

┌─────────────┐     WebSocket      ┌──────────────────┐     WebSocket      ┌─────────────┐
│  Angular 21 │◄──────────────────►│  Rust Actix Server│◄──────────────────►│ VM Agent exe │
│  (Browser)  │   display/audio    │  (REST + WS)      │   display/audio    │  (Rust)      │
│             │   HUD commands     │                    │   HUD commands     │             │
└─────────────┘                    └──────────────────┘                    └─────────────┘

Checklist

Phase 1: Rust Backend ✅ (compiles)

server/Cargo.toml — Dependencies: actix-web 4, actix-ws 0.4, actix-cors, dashmap, parking_lot, serde, uuid, chrono
server/src/main.rs — Actix HTTP server with CORS, compression, static file serving, SPA fallback
server/src/config.rs — Env-based config (BUTTERFLY_HOST, BUTTERFLY_PORT, etc.)
server/src/models.rs — Session, AgentConnection, WsMessage enum (serde-tagged), ApiResponse, HealthInfo
server/src/state.rs — AppState with DashMap sessions/agents, FrameBuffer ring buffer for late-joiners
server/src/api/ — REST endpoints: GET/POST/DELETE /api/sessions, GET /api/health, POST /api/sessions/{id}/hud
server/src/ws/ — WebSocket handler: agent/viewer connect, display/audio frame relay, HUD forwarding, heartbeat timeout
server/src/stream/ — StreamStats tracker (frame count, byte relay, uptime)
server/static/index.html — Placeholder loading page
Compiles with cargo check (warnings only for future-use types)

Phase 2: Angular 21 Frontend 🔲 (next)

Project scaffold with Angular CLI
Windows-like desktop shell (taskbar, start menu, window manager)
Remote display component (WebSocket frame rendering)
Audio playback component
HUD overlay (mouse/keyboard forwarding)
Built-in apps: File Explorer, Terminal, Settings, Text Editor, Browser

Phase 3: VM Agent Executable 🔲

Rust desktop agent that captures display and audio
Streams display frames (JPEG/PNG) and audio (Opus/PCM) via WebSocket
Receives HUD commands (mouse, keyboard, resize)
Low-latency design for LAN usage

Phase 4: Integration & Polish 🔲

End-to-end testing (agent → server → browser)
Session management UI
Multi-session support
Authentication (JWT)
Performance optimization

Recent Commits

bae17ae server: fix pong bytes reference — backend compiles
2e55d5c server: add .gitignore, fix compile errors
0c35db1 server: main.rs — Actix entry point
b2f4f7f server: fix Cargo.toml dependency versions
94a992d server: api/sessions.rs — CRUD endpoints
a0b834e server: stream/ — StreamStats tracker
74591a4 server: ws/handler.rs — WebSocket handler
40bf516 server: api/health.rs
843c7bb server: api/mod.rs
e00fbf4 server: ws/mod.rs
ff476f1 server: state.rs — AppState
fdde57c server: config.rs
5c8f848 server: models.rs
bf8e9f7 server: Cargo.toml

3.5 KiB Raw Blame History