Description
## Context
The use case is that people host microservices on their Hydra, some of which need to be **publicly reachable**. The mechanics of the path are proven — district hub -> venue tunnel -> Pi LAN address -> Incus proxy device -> scale, measured at HTTP 200 in 73 ms. Automating ingress is tracked separately.
This issue is for the questions that automation does **not** answer, and which should be decided before anything customer-facing is switched on.
## 1. Whose domains, and who may claim one?
If tenants bring their own domain they CNAME to the district server and autocert issues a certificate on first request. Two consequences:
- **Anyone who points DNS at us can trigger a certificate request.** A dynamic `HostPolicy` must check *registration*, not merely accept whatever arrives, or the box becomes an open ACME client and will hit Let's Encrypt rate limits.
- **Nothing currently stops one tenant claiming another's domain.** Incus projects already isolate the containers; the domain namespace has no equivalent owner check. First-come-first-served across tenants is not acceptable once there is more than one.
Decide: are domains only ever `*.experiencenet.com` subdomains we control, or do we accept customer domains? The second needs a verification step (DNS TXT or similar) before a route is honoured.
## 2. Public traffic transits the district hub
The Pi is LAN-private behind NAT, so the mesh is the only inbound route and the hub is always on the path. That means:
- Customer traffic consumes **venue uplink** and **district hub bandwidth**. A popular service on a Pi at a venue is paid for twice.
- It places `brussels-district-v2` on the public path for customer workloads. That is the box whose backup is still unproven (#422), whose snapshots fail roughly 1 in 5 silently (#424), and which also runs hydraneckwebrtc and the WireGuard hub for 24 peers.
- It is a **DDoS surface**: public traffic to a customer service lands on the district box, and the mesh depends on that box staying up.
Decide whether public workloads should terminate somewhere other than the district hub, or whether the hub is hardened and monitored first.
## 3. One backend per domain, no health check, no failover
`hydrareverseproxy` builds one `httputil.NewSingleHostReverseProxy` per domain. There is no pool, no health check and no failover: a scale that dies is a 502 until someone notices.
That is tolerable for internal tools and weak for anything customer-facing. It also interacts with the state problem — most Hydra services are single-writer YAML stores, so "just run two" would split writes and diverge. Capacity and availability need state solved first, or need services that are genuinely stateless.
## 4. Resource isolation is capped but not guaranteed
Scales inherit 512 MiB memory and 5 GiB disk caps, kernel-enforced (verified: a 700 MB allocation in a 512 MiB scale is OOM-killed inside its own cgroup, node unaffected). But **CPU is deliberately unbounded** — `limits.cpu` in Incus is cpuset pinning rather than a share, and pinning would cripple Go services by forcing `GOMAXPROCS=1`.
So one tenant's busy scale can starve a neighbour's. Acceptable between internal services on a node we control; a different proposition when tenants are paying.
Also note disk caps only work because `btrfs quota enable` is run at provision time — without it a `size:` is accepted silently and enforces nothing.
## 5. Registry access for tenant images
If tenants deploy their own images they need to push somewhere. Today `scaleregistry.experiencenet.com` has a single shared credential distributed to nodes, with no per-tenant namespacing and no per-repository authorisation. Related maturity work is #427.
## 6. Blast radius of the alternative
For completeness, the rejected option: making scales full WireGuard mesh members would give every scale reachability from all 24 peers — iPads, Windows air units, venue routers. Public hosting does not need internal reachability, and this would widen exposure considerably for no benefit to this use case. Recorded so the decision is not revisited by accident.
## Suggested position
Start with `*.experiencenet.com` subdomains only, allocated by us, on services we operate. That removes questions 1 and 5 entirely and makes the ingress automation useful immediately. Revisit customer domains and tenant image pushes only once ownership and verification exist.