HydraIssues

Symptom

Session history on hydracluster shows 11/15 recent sessions ending with watchdog_timeout. The hydracluster session watchdog fires when body OR head heartbeat is silent >60s. In all affected sessions the head heartbeat stops 27–60 seconds after stream start — the body continues heartbeating normally.

Data

From /api/v1/sessions/history on 2026-05-22:

• node-b96-9 (chunky-turnip-23 + ipad-head-3): duration 83s, head last HB only 17s after start
• node-b96-8 (chunky-turnip-23 + ipad-head-3): duration 93s, head last HB 27s after start
• node-11d-14 (fluffy-dumpling-87 + ipad-head-4): duration 91s, head last HB ~37s after start
• Several sessions show head_last_heartbeat_at near zero (head never registered)

Cause (unknown)

The iPad app stops sending PUT /api/v1/heads/{id} heartbeats to hydracluster shortly after a stream starts. Either:

1. The app loses its hydracluster connection (network change when Moonlight connects?)
2. The app crashes or backgrounds itself
3. The heartbeat loop is suspended when the stream UI is active

Impact

The hydracluster session watchdog closes the session after 60s of head silence and sends TerminateStream: true to the body, killing the running experience. This is the dominant cause of premature stream termination (73% of sessions).

Investigation

Check hydraheadipad logs on ipad-head-3 and ipad-head-4 during a stream to see if heartbeat calls are being made. Confirm whether the heartbeat loop is paused when the Moonlight stream UI takes focus.