HydraIssues

Feature Request #397 (revised): Network-quality telemetry for native Heads — jitter, loss, and wireless correlation
Summary
Add network-quality measurement to the HydraHead Flatscreen client and the venue
gateway, reporting jitter (p50/p95/p99/max), packet loss, reordering, and RTT
for the active path to the Body — correlated with wireless-layer data from the
AP and gateway CPU load. Report to HydraNeck as per-session diagnostics.
Today this data exists only implicitly for the WebRTC path (RTT drives quality-tier
selection). Native Moonlight Heads — over WireGuard mesh or direct LAN — have no
equivalent visibility.
Motivation
VR/XR streaming quality is governed by jitter and tail latency, not average RTT
or bandwidth. A steady 20 ms path beats a jumpy 12 ms one: jitter causes missed-frame
hitches and forces a larger jitter buffer, raising motion-to-photon latency.
Three things make this measurable-or-nothing rather than nice-to-have:
We are jitter-bound, not throughput-bound. At ~200 Mbps/head we use a small
fraction of a 6 GHz channel's capacity. Capacity metrics tell us nothing; timing
consistency is the whole story.
We deploy 6 GHz-only, no MLO, by design. Having removed cross-band
redundancy, the wireless hop is both the dominant jitter source and the one with
no fallback. Tunnel-side numbers alone can't explain a spike — we need the AP's view.
VR is self-hostile RF. Users in headsets are moving bodies between antenna and
AP, constantly, in every venue — not just contested events. 6 GHz is absorbed
harder by bodies than lower bands. Body-blocking is a permanent, load-bearing
jitter source we currently cannot see.
Operations cannot today answer "why does streaming feel bad at site X?" with data.
Hop count and an idle ping don't capture it.
Proposed behaviour
1. Path measurement (Head ↔ Body)
UDP probe computing inter-arrival jitter (RFC 3550 running estimate), plus
loss and out-of-order counts.
Probe must approximate the real stream's packet rate/size, and run on the path
the stream actually uses:
Bodyless site → inside the WireGuard tunnel.
Local Body → over the LAN (streams never enter the tunnel; measuring the
tunnel would measure the wrong path).
Remote head into a Body-equipped venue → through the mesh.
Record which path was used; discoverBody() already selects LAN-first with
WireGuard fallback, so tag the session accordingly.
Two contexts: a short pre-flight probe after path selection in discoverBody(),
and an optional low-rate in-session probe so hitches can be correlated in real time.
2. Wireless-layer correlation (new — the most important addition)
Pull L2 telemetry from the AP and join to the path data by client MAC:
Associated band (must confirm 6 GHz — a head that drifted to 5 GHz is a fault),
RSSI/SNR, PHY rate, retransmissions, channel and width.
Collection differs by tier — all three must be supported:
MikroTik venue gateway → RouterOS API (already used elsewhere in the stack).
Omada / Ruckus APs → controller API (client table, per-client signal/rate).
Home unit (OpenWrt) → SSH over the mesh: iwinfo <iface> assoclist,
/tmp/dhcp.leases, wg show. Depth is consumer-grade (associations, RSSI,
rates — not per-packet airtime); acceptable at 1–2 heads.
This is what turns "p99 spiked" into "head 3 dropped to a weak 6 GHz rate when the
user turned their back to the AP."
3. Gateway CPU load
Log gateway CPU alongside jitter. WireGuard on MikroTik is CPU-bound with no offload;
near its ceiling (~800 Mbps tunneled ≈ 4 heads) it introduces processing jitter
indistinguishable from network jitter unless CPU is visible. Applies equally to the
x86 (Atlas-class) gateway tier.
4. Separate device-side jitter from network jitter
Timestamp as close to the socket as possible on the Head; use a monotonic clock
(System.nanoTime). Flag Pico-side scheduling and thermal variance distinctly —
critical if any deployment terminates WireGuard on the headset, where sustained
crypto + decode can throttle. A throttle must not be misread as a path problem.
5. Transport tagging
Tag each session with the venue's upstream type — fibre / cable / Starlink /
cellular — and whether the site is bodyless or Body-equipped. Satellite and
cellular have materially different jitter profiles, and that only matters for
bodyless sites; a Body-equipped node on Starlink carries only mesh/management
traffic. Without this tag the fleet data is not comparable.
6. Reporting
Per-session diagnostics record to HydraNeck, tagged with district, venue, Body,
path (LAN / WireGuard), tier (home / small venue / venue / event), head type, AP
model, channel, and transport. Filterable and comparable across sites.
Report percentiles, never averages alone. For VR the tail is what hurts: an
average that looks fine can hide periodic p99 spikes that drop frames.
7. Hub-side heartbeat
The hub should detect a node's tunnel going silent. Remote telemetry is reachable
over the mesh, so a tunnel drop blinds us exactly when we most want data — the
hub-side signal covers that gap.
Correctness notes
Use one-way inter-arrival jitter: built from deltas on the receiver's monotonic
clock, so a constant clock offset cancels and no NTP/PTP sync is needed.
Do not report absolute one-way latency without real time sync — consumer NTP
error would make it meaningless. RTT (single-clock echo) is fine.
Acceptance criteria
Jitter (p50/p95/p99/max), loss %, reordering, and RTT recorded per head, per session.
Measurement runs on the actual streaming path (LAN or tunnel) at a realistic
packet profile.
Wireless data (band, RSSI, rate, retries) joined by MAC and visible alongside.
Gateway CPU captured for the same window.
Results filterable in HydraNeck by district / venue / Body / path / tier / transport.
In-session probe overhead < 1%.
Thresholds (venue qualification gate)
A site passes when, under a real full-head session:
Metric
Target
Jitter p99
< ~5 ms
Network RTT
< ~20 ms
Dropped frames
≈ 0
Motion-to-photon
< 50 ms (≈40 ms typical when tuned)
Gateway CPU
off the ceiling (not >80% sustained)
Head band
6 GHz for every head
Specifically watch the failure modes 6 GHz-only is most exposed to: body-blocking
and range-edge. If p99 climbs or CPU pegs, the remedy is lower bitrate, fewer
heads, better AP placement, or a gateway step-up — decided by the data, not estimate.
Non-goals
Not full APM; GPU/render metrics belong to HydraNode/HydraCluster.
Not synchronised absolute one-way latency in v1.
MLO A/B qualification (HBS mode, WPA3-only prerequisite, STR-vs-eMLSR
determination, band binding, MLO on/off comparison) — depends on this measurement
capability but is a separate validation workflow; file separately. Lower priority
given the 6 GHz-only decision, but it is the instrument that would let us reverse
that decision with data if a contested event's 6 GHz degrades.
Bodyless / on-prem-gateway guard-type gap — separate issue.