HydraIssues

hydravoice pipeline crash loop fleet-wide — pipeline-debug.log always empty
open bug Project: hydravoice Reporter: cederik 22 May 2026 06:15

Description

## Status (2026-05-22)

Fleet-wide: all 4 online body machines crash-looping. hydravoice's ffmpeg+hydravoice-player pipeline restarts every 2 seconds. Audio relay from iPad to UE experience is completely broken.

**Affected bodies:** chunky-turnip-23 (node-b961f1c8), cosmic-pretzel-98 (node-4c2be4b0), boom-pickle-38 (node-74f9fbf2), fluffy-dumpling-87 (node-11da9ea3)

**Current version:** v0.4.8 (just deployed)

## Symptom

session-debug.log shows:
```
[2026-05-22T08:05:38+02:00] session=1 pid=6380 cmd=cmd /c C:\ProgramData\hydravoice\pipeline.bat
[2026-05-22T08:05:40+02:00] pid=6380 exited (isProcessAlive=false)
```
Repeating every ~2 seconds. pipeline-debug.log always 0 bytes.

## Root cause chain (investigated)

1. **v0.4.6 regression (b359e09):** Added `2>>pipeline-debug.log` inline inside `cmd /c "..."`. cmd.exe exits immediately due to quoting/redirect interaction — never runs ffmpeg or hydravoice-player.
2. **v0.4.7 fix:** Switched to batch file (`pipeline.bat`). Fixed quoting. But pipeline-debug.log still empty — SYSTEM-created log file has BUILTIN\Users=ReadAndExecute only (no Write), so interactive session's cmd.exe can't open it for 2>>.
3. **v0.4.8 fix:** `os.Remove(errLog)` before launch so interactive user creates fresh file as CREATOR OWNER. Still empty — either os.Remove is blocked by Windows file locking, or CREATOR OWNER ACE doesn't propagate to new files in this directory.

## Next step

Change errLog path to `C:\Users\Public\hydravoice-debug.log` (guaranteed world-writable on all Windows). No deletion or ACL manipulation needed. This will finally let us see what ffmpeg and hydravoice-player output, revealing the actual crash cause.

## Original trigger

Session was started on node-b961f1c8 with mic enabled (issue #360). iPad sends RTP (790 packets received by hydravoice), but audio never reaches UE. Pipeline crash loop was pre-existing before the iPad mic session.