HydraIssues

HydraEye: camera-based presence detection for kiosks
open feature Project: hydraeye Reporter: 9 Apr 2026 21:35

Description

## Overview

Camera-based presence detection service for kiosk Mac Minis. Auto-launches experiences when someone approaches, returns to attract screen when they leave.

## Architecture

Qt app (HydraExperienceNet) is the hardware broker for camera and microphone, avoiding macOS TCC permission issues for CLI tools.

- Qt app captures webcam frames (320x240, 0.5 FPS), serves on localhost:9742
- hydraeye is a hydranode role, polls Qt for frames, runs frame differencing detection, serves presence on localhost:9741
- hydraheadflatscreen polls hydraeye for auto-launch decisions
- Config comes from HydraCluster (EyeConfig on Node)

## Qt app as hardware broker

The Qt app also handles mic relay for native Heads:
- Captures mic audio, encodes Opus, sends RTP to Body:47995
- Same protocol as browser WebRTC path, hydravoice on Body needs no changes
- One TCC permission grant covers both camera and microphone

## Detection algorithm

- Frame differencing against reference frame (empty room)
- 15% pixel threshold + 3-frame debounce for presence
- 5% threshold + configurable idle timeout for absence
- CPU: <0.5% of one core (76,800 pixels at 320x240)

## HydraCluster config

EyeConfig on Node:
```yaml
eye_config:
threshold: 0.15
idle_timeout_seconds: 30
debounce_frames: 3
```

## Prerequisites

- Issue #92: hydraheadflatscreen queries Experience Library (deployed upstream)
- Qt app mic relay to Body via Opus RTP
- These give native Head feature parity with browser streaming before adding presence detection

## Scope

- New hydraeye Go binary (hydranode role)
- Qt app camera + mic broker endpoints on localhost:9742
- HydraCluster EyeConfig on Node store
- hydraheadflatscreen integration (poll hydraeye for auto-launch)