HydraIssues

Registry maturity: no request logging, no garbage collection, no backup, no storage alerting
open improvement Priority: medium Project: hydrascaleregistry Reporter: 2 Aug 2026 04:38

Description

## Context

The registry now holds the deploy artifacts for six services and is a **hard dependency for launching any scale**. It has outgrown the operational surface it shipped with. State verified 2026-08-02.

**Architecture** (worth writing down — it was not obvious):

```
:443 hydrascaleregistry.service auth proxy, TLS termination, autocert
└─ 127.0.0.1:5000 docker-registry.service Distribution, config at /etc/docker/registry/config.yml
```

Holding: `hydranps`, `hydravenues`, `hydraorganization`, `hydrapipeline`, `hydraexperiencelibrary`, `hydradistrict` — 68 MB, 6% of a 38 GB disk.

## 1. No per-request logging (do this first)

`hydrascaleregistry` logs only startup lines and TLS handshake errors. There is **no request logging at all**.

This is already blocking a real investigation: pulls through incusd fail `unauthorized` while the identical skopeo command succeeds by hand (hydraskin #426), and the registry logs were silent throughout — which proves nothing, and I initially over-read as "the request never arrived".

Log method, path, source IP, and **auth outcome**. The distinction that matters: a request arriving with *no* `Authorization` header (client stripping it) versus arriving with credentials that were *rejected*. Those point at completely different fixes, and today we cannot tell them apart.

## 2. No garbage collection

`storage.delete.enabled: true` is set, so manifests *can* be deleted — but **nothing ever runs `registry garbage-collect`**, so deleting a manifest never frees its blobs. Storage grows monotonically with every tag pushed.

`maintenance.uploadpurging` is enabled (168h), which handles abandoned uploads but not unreferenced blobs.

At 68 MB this is not urgent. It compounds silently, and the precedent is unpleasant: mirror-a reached 98% disk with a `max_storage` guard configured above the physical disk size, so it could never fire (#414).

Note GC in Distribution needs the registry read-only or stopped to be safe against concurrent pushes — worth scheduling deliberately rather than as a naive cron.

## 3. No backup

`hydrabackup` is not installed on this host, and it would not help if it were: it collects `*.yaml` service state and deliberately skips secrets, so registry *blobs* are outside its model entirely (see hydrabackup #425 for the boundary).

So the images are currently single-copy. They are rebuildable from source via CI, which softens this — but "rebuild six services from tags" is a bad recovery path during an incident, and CI has already proven able to produce phantom builds.

Decide explicitly: either back up `/var/lib/registry`, or write down that rebuild-from-CI *is* the recovery plan and confirm every image tag is reproducible.

## 4. No storage alerting

Nothing watches disk on this host. Same class of problem as mirror-a. Cheap to add, and the failure mode without it is a registry that silently stops accepting pushes.

## 5. Mutable tags

`hydranps` currently carries `v0.2.3`, `v0.2.2`, `latest`. Nothing prevents `v0.2.3` being overwritten with different content, so a version tag is not a stable identifier. For a deploy artifact that is a meaningful weakness — a scale relaunched from `v0.2.3` months later may not get what was tested.

Options: enforce immutability for `v*` tags in the auth proxy (reject `PUT` to an existing version tag), or pin by digest at launch time. The proxy already gates writes, so it is the natural place.

## 6. Single instance, now on the critical path

One host, no HA. Since no scale can be launched without it, a registry outage blocks all scale deployment and recovery. Acceptable today, but it should be a deliberate decision rather than an accident.

Related and already decided: the registry **must not run as a scale** — it cannot be the thing you need in order to recover the thing you need.

The planned edge mirroring (Distribution's `proxy.remoteurl` pull-through at district/venue) would also reduce this exposure, since an edge cache can serve a pull while the hub is down.

## Suggested order

1. Request logging — cheap, and unblocks hydraskin #426
2. Storage alerting — cheap, prevents a silent-fill incident
3. Garbage collection — scheduled with the read-only caveat understood
4. Backup decision — either implement, or document rebuild-from-CI as the plan
5. Tag immutability
6. Edge mirroring / HA posture