HydraIssues

Summary

Give creators a git-push-to-deploy path for services (scales), mirroring the Perforce-push path that already exists for experiences (Unreal builds). A creator signs in to HydraMancer, requests a git repo for a scale, and pushes to it. The platform builds the image, publishes it to the scale registry, launches or updates the scale on a hydraskin node, and gives it a domain with HTTPS. The creator never handles a registry credential.

This is the service-side sibling of the Perforce work (#484, hydraperforceprovision, hydraperforcewatcher, hydraperforcescale).

Why

HydraMancer's /deploy quickstart lists five clean steps to ship a service, but step 2 (push the image to scaleregistry.experiencenet.com) silently assumes the creator already holds the SCALE_REGISTRY_TOKEN push credential. They do not, and it cannot be self-served, so the "publish fast" promise breaks at exactly that point.

Concretely: while deploying the rogue game as a scale (dogfooding /deploy), everything worked (multi-arch Dockerfile, image built and verified, hydraskin node ready, DNS ready) except publishing the image, which stopped dead at the push token. That is the gap this feature closes: the platform's builder holds the token, the creator only does git push.

The symmetry

What already exists (about 80 percent)

• Front door: HydraMancer /deploy and /experience, iamnim login, the provision proxy pattern (/api/v1/provision/perforce).
• Provision blueprint: hydraperforceprovision (mints access from a verified iamnim org membership).
• Watcher blueprint: hydraperforcewatcher (detects a push, fires a build-notify webhook, ExperienceLibrary auto-stages).
• Runtime: hydrascaleregistry (registry), hydraskin (launch, expose, update a scale), hydrascalerouter (dynamic domain routing from user.hydra.domain/port/health_path labels).
• Build recipe template: the rogue repo Dockerfile and deploy-image workflow are a working reference for what a pushed repo carries.

What is missing (about 20 percent)

Two new services, each a near-copy of a Perforce-side one:

1. Git provisioner (git analog of hydraperforceprovision). HydraMancer /deploy gains a "request a repo" button. The service creates a repo scoped to the creator's iamnim org, registers a push webhook, and records the mapping repo to scale-name to org.

2. Scale build-watcher (git analog of hydraperforcewatcher). On push it:

   • builds the repo's Dockerfile multi-arch (amd64 + arm64),
   • pushes to scaleregistry.experiencenet.com/<name>:<sha> using the token it holds,
   • launches the scale, or hydraskin updates it if it already exists, on a hydraskin node,
   • sets the user.hydra.domain and user.hydra.port labels and ensures the <name>.experiencenet.com DNS record,
   • carries the scale through the same draft to staging to live lifecycle the experiences use.

Build capacity: builders as scales

Run the container builder (BuildKit) as a scale, not as fixed infrastructure. There is already a hydraskin node on hcloud, so:

• Launch one or more BuildKit build scales on hydraskin. Use arm64 or amd64 as convenient; the hcloud hydraskin node can host an amd64 builder, and an arm64 builder can run on a Pi node or an arm hcloud node. A cross-arch pair removes the need for slow QEMU emulation.
• The build-watcher dispatches each push to a builder scale, which builds, pushes, and reports back.
• This dogfoods the platform: the thing that publishes scales is itself a scale.

Design decisions to settle

• Git host: self-hosted (Gitea) gives per-org provisioning that matches the Perforce depot model; GitHub with per-repo deploy keys is the lower-effort alternative.
• Zero-config vs Dockerfile: v1 requires a Dockerfile in the repo (rogue's is the reference). Later, buildpacks or nixpacks so a bare Go repo deploys with no Dockerfile.
• Secret handling: the push token lives only on the builder, never with the creator. This is the whole point.
• Naming and domains: auto-assign <repo>.experiencenet.com, label the scale, create the DNS record.
• Lifecycle: give /deploy the same draft to staging to live promote/rollback the experiences already have.

Out of scope for v1

• Per-org quotas and resource limits (noted, deferred).
• Buildpack / zero-Dockerfile support (Dockerfile required in v1).

First dogfood

rogue is the ideal first case. It is a real containerized service, its Dockerfile and deploy workflow are already written and verified, and it hit this exact gap. When the pipeline exists, publishing rogue should be a single git push.

Related

• #484 automate the creator access hand-off (Perforce side)
• #406 HydraSkin container host role
• repos: hydramancer, hydraperforceprovision, hydraperforcewatcher, hydraperforcescale, hydraskin, hydrascaleregistry, hydrascalerouter

Design decision: git hosting model (scale as the repo)

The original write-up assumed a central Gitea. This section revises that.

Decision

Adopt the scale-as-repo, push-to-deploy model (the Dokku and Heroku pattern) rather than running a central forge (Gitea or Forgejo). You push a project and it deploys. The forge is not the backbone.

Why it fits Hydra

• hydrascalerouter already makes the domain follow the service. If the push endpoint is served through the same edge, you push to https://<project>.experiencenet.com and reach it wherever the service runs, exactly as the web domain does.
• It reuses iamnim for auth and the router for reachability, so there is no central git server to run, back up, or hold an admin credential for.
• The mental model (push your project, it deploys) is the one creators already know from Heroku, which suits the "publish fast" goal.

Is that idiomatic

Yes. Self-hosted git spans bare git over SSH (minimalist), Gitea and Forgejo (lightweight forge with a UI), and GitLab (heavy full platform). All are legitimate. For push-to-deploy specifically, the Dokku and Heroku and Coolify style (the receiving endpoint is the deploy trigger) is the idiomatic choice, and it is leaner than running a forge.

What we accept by not running a forge

• Push auth is git-http gated by an iamnim token (or an ssh cert), not forge accounts. This is the one piece we build rather than get for free.
• No browsable code UI, no pull requests, no issue tracker. If creators later need a code home, add Forgejo as an optional scale; do not make it the backbone.

Deploying one source to many scales

Strict "repo equals one scale" breaks the moment you want the same source on many scales (a game on many venue kiosks, a service fanned across districts, blue and green). Resolve it by decoupling the source from the targets:

• The durable artifact is the OCI image in scaleregistry, not the git repo. A git push means "build this version". Deployment is a separate mapping of that one image onto a target set.
• A target set can be: one scale (the default); a load-balanced group behind one domain (the router already health-checks and load-balances at the edge); or a fan-out to many domains or districts (one per venue).
• So the push endpoint is per project, not literally one scale. Provisioning creates the project and returns its push remote; the project's .hydrabuild.yaml declares the target or targets. One push builds once and rolls the image to every target in the set. The registry is the fan-out point; git is only the version trigger.
• This keeps the simple push-to-deploy ergonomics for the common 1:1 case and supports 1:many with no second mechanism.

The nuance the 1:many case forces: because a project can target many scales, its git-receive endpoint should be a thin per-project receiver, not a bare repo bound to any single target scale. It receives the push, hands the SHA to the builder, and then deploys to the declared target set. That is still the scale-as-repo ergonomics, just with the receiver scoped to the project rather than to one running scale.

Impact on the code already built

• hydragitprovision (cederikdotcom/hydragitprovision): rescope from Gitea-admin-driver to "create a per-project push remote plus receive hook, gated by iamnim". It should hold no forge admin token.
• hydragitwatcher (cederikdotcom/hydragitwatcher, currently NEEDS_WORK): the deploy step must accept a target set (one scale, a group, or a fan-out), launch or update each from the single built image, and set labels and DNS per target.
• hydramancer /deploy (PR #1): "request a repo" returns a per-project push remote; add an optional "targets" concept later.

Open questions

• Push auth: iamnim token over git-http versus ssh certs. Prefer git-http to match the mesh and edge model.
• Where the per-project receiver physically runs so it survives node moves: a small central git-receive scale that dispatches to the builder is likely simpler than a bare repo pinned to one node, and it is what makes 1:many clean.
• Confirm hydrascalerouter can map one domain to N scales for the load-balanced group case, and how a group is declared.

Recommendation

Build the scale-as-repo push-to-deploy model with a thin per-project git-receive endpoint and the registry as the fan-out point. Keep a forge (Forgejo as a scale) only as an optional later add for creators who want a browsable code home.

Milestone: rogue deployed as the first pipeline test (2026-08-16)

rogue is live at https://rogue.experiencenet.com, running as a scale on pi-node-004 (arm64) with a valid Let's Encrypt cert, /data-persisted state, and a working live game over wss. This was done by executing the pipeline STAGES by hand (build multi-arch -> push to scaleregistry -> incus launch + labels + /data disk -> explicit A record -> ACME), which proves the whole path end to end against the real registry, hydraskin node, router and DNS. Full procedure and gotchas in rogue/docs/runbooks/deploy-hydra.md.

What this validated for the automated pipeline: the registry credential works (user hydra), the node pulls and launches from the registry, label-driven routing works, and ACME issues once DNS exists. Remaining to automate (this issue): the git-receive endpoint + hydragitwatcher doing these stages on a git push, in the scale-as-repo model above. hydragitwatcher is currently NEEDS_WORK and still assumes the Gitea model.