HydraIssues

Portrait streaming: orientation contract implementation + remaining robustness improvements
open improvement Project: hydrabody Reporter: 13 May 2026 15:59

Description

Portrait streaming — orientation contract implementation and remaining robustness improvements

## What was done

This tracks the full portrait streaming orientation work for `cosmic-pretzel-98` (and all multi-orientation bodies). The root cause was a visible artifact: a landscape kiosk overlay showing behind a portrait Unreal Engine app when a portrait stream started.

Root cause breakdown:
1. `kioskoverlay` did not respond to `WM_DISPLAYCHANGE` — overlay stayed landscape when VDD switched to portrait
2. `handleStreamStarted` did not receive orientation from the prep-cmd; it had to resolve it from cache with no explicit contract
3. `handleStreamEnded` was not implemented — the Undo hook silently 404'd, leaving VDD at the last stream's resolution
4. Hardcoded resolution strings scattered across multiple files — no single source of truth
5. No explicit pause between writing `dd_manual_resolution` and Sunshine beginning capture

## Changes shipped

**`cmd/kioskoverlay/main.go`**
- Added `WM_DISPLAYCHANGE` handler: on resolution change, reads new screen dimensions via `GetSystemMetrics` and resizes the overlay window to fill the screen.

**`pkg/provider/experiences.go`**
- Added `Resolution string` field to `liveExperience` and `experienceInstallInfo` for explicit WxH override (e.g. `1668x2224` for iPad kiosk).
- Added `streamResolution(info experienceInstallInfo) (vddRes, ueArgs string)` as the single source of truth for all three layers of the orientation contract.

**`pkg/provider/sunshine_provider.go`**
- Added `desiredApp` struct: `{cmd string, orientation string}`.
- Added `buildOrientationHooks` — replaces `sunshine.CallbackHooks()` from go-sunshine. Bakes orientation into the JSON body at app registration time so `handleStreamStarted` receives it explicitly.
- Changed `SyncApps` to accept `map[string]desiredApp`.
- Added post-sync read-back verification: after syncing all apps, lists them from Sunshine and logs WARNING if any desired app is missing or has no prep-cmd.

**`pkg/provider/sunshine.go`**
- Moved experience state load before the head loop so head apps (cmd = StreamAppID = experience name) can resolve orientation by StreamAppID.
- Head-based apps carry orientation from experience state cache.
- Experience-based apps use `streamResolution` for UE launch args.

**`pkg/provider/httpserver.go`**
- Added `Orientation` field to `streamEventRequest`.
- `handleStreamStarted`: uses orientation from prep-cmd request body; falls back to experience cache; logs mismatch; calls `streamResolution` for VDD resolution; sleeps 500 ms after `setSunshineConfig`; returns `{"orientation": "..."}` for head verification.
- Added `handleStreamEnded`: resets `dd_manual_resolution` to `1920x1080`. The Undo hook was silently 404'ing before this.

**`docs/testbooks/portrait-mode.md`**
- Step 4 updated: curl payload includes `orientation` field.
- Added Step 6b: verify landscape stream after portrait stream.

**`docs/testbooks/kiosk-mode.md`**
- Added Step 5: portrait resolution change / overlay resize test.

## Orientation contract (three layers)

| Layer | Controls | Owner |
|-------|----------|-------|
| 1 — Stream encoding | Resolution Moonlight client requests from Sunshine | Head (hydraheadflatscreen / hydraneckwebrtc) |
| 2 — Display resolution | `dd_manual_resolution` in sunshine.conf, applied on stream-started | `httpserver.go:handleStreamStarted` |
| 3 — App window | UE launch args registered in Sunshine app cmd | `sunshine.go:tickSunshineApps` |

The prep-cmd fires before Sunshine begins capture, so layer 2 is set before the first video frame. The 500 ms sleep inside the handler allows the VDD to stabilize. Layer 3 is set at app registration time and matches layer 2 because both derive from the same experience state cache.

## Remaining robustness improvements (tracked here)

### 1. Mutex on `setSunshineConfig`

`setSunshineConfig` in `pkg/provider/streamprovider.go` does read/merge/write on `sunshine.conf` with no lock. In practice Sunshine is single-session so this race cannot happen, but the fix is 3 lines. Files: `pkg/provider/provider.go`, `pkg/provider/streamprovider.go`.

### 2. Custom resolutions in `vdd_settings.xml`

`writeVDDSettings` in `vdd_windows.go` has five hardcoded resolutions and is called only on first VDD install. Any experience with a custom `Resolution` field (e.g. `1668x2224` for iPad kiosk) silently fails — VDD does not support that resolution, stream renders at wrong aspect ratio with no error.

Fix: add `ensureVDDResolutions()` called from `tickVirtualDisplay` each tick when VDD is already installed. Reads experience state, extracts custom Resolution values, hashes the list, rewrites `vdd_settings.xml` if changed, logs a one-time warning that a VDD reinstall/reboot is needed for the change to take effect.

## What was ruled out

- **Polling instead of 500ms sleep**: VDD switch happens after the handler returns (Sunshine reads `dd_manual_resolution` after prep-cmd completes). Polling inside the handler cannot detect the switch.
- **Post-switch VDD resolution verification from within `handleStreamStarted`**: Same timing problem.
- **Suppressing duplicate stream events**: No log watcher fires `OnStreamStarted`. `detectStreamStatus` in `stream_windows.go` is a log-reader for idle/streaming state only, bypassed entirely when `SunshineStreamProvider` is active.