HydraIssues

Label or repair the dead venue ethernet port that took pi-node-003 offline for three days
open unclassified Project: hydraneck Reporter: 31 Aug 2026 14:47

Description

On 2026-08-28 pi-node-003 was moved and never came back on the network. It was assumed dead for three days. On 2026-08-31 it turned out the Pi and its NVMe were fine: the ethernet port or cable it was plugged into is dead. Moving the node to a known-good port brought it online within seconds.

The fault follows the port, not the machine. pi-node-004 was briefly plugged into the same port during diagnosis and showed the identical symptom (boots normally, never reaches the network, no ARP presence anywhere on the 192.168.68.0/22 LAN).

Ask: physically identify and label that port as dead, or repair or replace the cable, patch panel or switch port behind it, so no future node is plugged into it. The cost of not doing this was a three-day outage of 8 services plus a temporary Hetzner box.

Diagnostic note for next time: the venue LAN is a /22, not a /24. A sweep of only 192.168.68.x covers a quarter of the range. Sweep from a node already on the LAN via hydracluster exec and read ip neigh, matching Raspberry Pi MAC OUIs 2c:cf:67, d8:3a:dd, dc:a6:32, e4:5f:01, b8:27:eb. There is no router-side DHCP lease visibility at this venue today: the AD6 Omada is decommissioned (#535) and glinet-sintniklaas is unreachable over the mesh.