HydraIssues

Goal

Get hydraperforcewatcher v0.2.0 running on the visitflanders Perforce agent, so newly-submitted Perforce changelists publish to hydramirror (not the dead /var/www/releases path) and bodies can actually download them. The v0.2.0 binary is already published on releases.experiencenet.com (see #95); this issue is the deploy-side work that #95 did not cover.

Observed state (as of 2026-04-23)

• releases.experiencenet.com/hydraperforcewatcher/production/latest.json -> {"version":"0.2.0"} (CI run 24730910436 succeeded 2026-04-21).
• visitflanders agent is still running agent_version: v0.1.3 per hydraperforce state.yaml. That version predates even v0.1.4, which shipped 2026-03-24. So the auto-updater has been inactive on this agent for at least a month.
• Consequence: CL 92 "Add build v0.95" was processed on 2026-04-22 by the old code path. MercatorV0.95.zip sits at /var/www/releases/builds/soulmade/92/Builds/MercatorV0.95.zip on the release server but was never uploaded to hydramirror. mercator-talks has development_build: 7 in hydraexperiencelibrary with a broken build_url: https://releases.experiencenet.com/builds/soulmade/92/. If promoted to production today, bodies will 404 the same way they did for CL 91.
• I cannot reach the visitflanders machine from the Claude environment. It is not in any of the three configured hcloud contexts (hydraexperiencenet, nimsforest, cederik) and the environment SSH key is not authorized on the machine.

Scope

1. Diagnose why the auto-updater is stuck on the visitflanders agent. Candidate causes:
   • Service not restarting after an update (updater swaps the binary on disk but depends on a service restart to pick it up).
   • Stale version pin in a config field.
   • Network/TLS issue when the agent reaches releases.experiencenet.com (would show in journalctl).
   • WDAC or code-signing rejecting the new binary on a Windows host (we think this is a Linux box but confirm).
2. Migrate the agent config at ~/.hydraperforcewatcher/config.yaml to the v0.2.0 schema. Delete the release_server: block, add:

   mirror:
     url: "https://mirror-a.experiencenet.com"
     token: "<HYDRARELEASE_MIRROR_TOKEN from /etc/hydrarelease.env on 46.225.120.7>"
   

3. Restart the service and confirm it starts clean: systemctl status hydraperforcewatcher && journalctl -u hydraperforcewatcher -n 50.
4. Confirm v0.2.0 is running: next POST /api/v1/agents/visitflanders/state should report agent_version: v0.2.0 (visible in hydraperforce state.yaml on 138.199.195.170).
5. Patch CL 92 manually using the same procedure as the mercator-talks V0.9 fix on 2026-04-21:
   • On releases server: curl -T /var/www/releases/builds/soulmade/92/Builds/MercatorV0.95.zip -H "Authorization: Bearer $HYDRARELEASE_MIRROR_TOKEN" --upload-file https://mirror-a.experiencenet.com/api/v1/files/builds/soulmade/92/MercatorV0.95.zip
   • POST /api/v1/builds/notify on hydraexperiencelibrary with the mirror URL, so a new build entry (build #8) is created with a valid URL.
   • Update exe_path on the experience if the zip folder name differs from MercatorV0.9 (likely now MercatorV0.95 or similar).
   • Promote dev to staging to production when the developer is ready.
6. Run docs/testbooks/mirror-upload.md end-to-end on a scratch submission to confirm the v0.2.0 path works.

Unblocking remote access (one-time, for future work)

Either:

• Add a visitflanders hcloud context to ~/.config/hcloud/cli.toml on the Claude box if the machine is on a separate Hetzner project. Then HCLOUD_CONTEXT=visitflanders hcloud server ssh hydraperforcewatcher (or whatever the server is named) would work.
• Add /home/claude-user/.ssh/id_ed25519.pub to /root/.ssh/authorized_keys on the visitflanders machine. Then plain ssh root@perforce.visitflanders.experiencenet.com works.

Without one of these, this class of deploy work keeps requiring a human hand.

Acceptance

• hydraperforce state.yaml shows agent_version: v0.2.0 for visitflanders.
• A fresh test changelist flows end-to-end per docs/testbooks/mirror-upload.md.
• mercator-talks is live on V0.95 (build #8 via the manual patch from step 5) with a mirror URL and bodies can install it.

Related

• #95: code fix that this issue deploys.
• #97: district-local hydramirror fleet (separate scope).