HydraIssues

Goal

Enable microphone input for enable_microphone: true experiences (e.g. mercator-talks) on hydraheadipad, matching the voice path already working on hydraheadflatscreen.

Current state

The infrastructure is largely in place but the relay is never activated:

• Sources/HydraHeadiPad/Services/MicrophoneRelay.swift — AVAudioEngine mic capture → Opus → RTP/UDP to bodyHost:47995. Correct structure, correct protocol (PT=111, 12-byte RTP header, 48 kHz mono, 960 samples/frame matching hydraheadflatscreen).
• Sources/HydraHeadiPad/Bridging/HydraOpus.h/.c — C wrapper around opus_encode_float, singleton encoder.
• Sources/HydraHeadiPad/Models/HeadConfig.swift:63 — enableMicrophone: Bool? decoded from catalog JSON enable_microphone field.
• Sources/HydraHeadiPad/Views/ExperienceGridView.swift:86 — mic icon shown when enableMicrophone == true.
• Sources/HydraHeadiPad/Info.plist:32 — NSMicrophoneUsageDescription already present.

MicrophoneRelay is never instantiated or called anywhere. AppState does not reference it.

What needs to be done

1. Wire MicrophoneRelay into AppState

In AppState.swift:

• Add a private let micRelay = MicrophoneRelay() property.
• In startStream(assignment:): after the StreamingView is presented, check assignment.experience.enableMicrophone == true and call micRelay.start(bodyHost: host).
• In the stream stop/error handlers (or wherever state returns to .selfService or .error): call micRelay.stop().

The relay is non-fatal — if it fails (permission denied, AVAudioEngine error) the stream continues without mic input.

2. Fix AVAudioSession category

MicrophoneRelay.startAsync (line 53) sets:

try AVAudioSession.sharedInstance().setCategory(.playAndRecord, options: .defaultToSpeaker)

Missing .mixWithOthers. Without it, switching to .playAndRecord will disrupt SDL's audio output for the Moonlight stream (SDL sets .playback via CoreAudio when SDL_OpenAudioDevice runs). The category must be:

.setCategory(.playAndRecord, options: [.defaultToSpeaker, .mixWithOthers])

3. Tune Opus encoder (HydraOpus.c)

Currently uses OPUS_APPLICATION_AUDIO at 96 kbps. hydraheadflatscreen uses OPUS_APPLICATION_VOIP at 64 kbps — VOIP mode is optimised for speech (lower algorithmic delay, better DTX, comfort noise). Change HydraOpus.c to match:

_encoder = opus_encoder_create(48000, 1, OPUS_APPLICATION_VOIP, &error);
opus_encoder_ctl(_encoder, OPUS_SET_BITRATE(64000));

4. Verify via testbook step 9

Testbook step 9 (Mercator Talks) covers mic permission prompt and voice relay. Run it end-to-end after the AppState wiring is in place.

Reference

• hydraheadflatscreen mic relay: pkg/client/micrelay.go, micrelay_darwin.go — reference for RTP format, Opus params, relay lifecycle.
• hydravoice on body: receives UDP RTP at port 47995, decodes Opus, writes to VB-Cable CABLE Input. Unreal reads from CABLE Output.
• enable_microphone flag must be set on the experience in hydraexperiencelibrary.