provisioning-catalog/components/MIGRATION_CONCERNS.md

# ServiceConcerns Migration — Component Reference

This document describes the mechanical pattern for migrating each component
under `provisioning/extensions/components/<X>/nickel/` to declare a
`ServiceConcerns` umbrella (ADR-008, ADR-009).

## Status

Components migrated as of this writing — patrón aplicado y validado:

**TLS endpoints / databases:**
- `docker_mailserver`  — `tls_endpoint_with_acme` style; backup `'pending` for mail-stack BackupGroup
- `zot`                — `tls_endpoint_with_acme` style; backup `'pending` for registry-stack BackupGroup
- `postgresql`         — `database`; backup `'pending` for workspace-level BackupPolicy with dump_strategy

**Storage backends / cert infrastructure:**
- `longhorn`           — `infra_storage_managed`; backup `'disabled` (engine state via SystemBackupDef.longhorn_engine)
- `cert_manager`       — `infra_storage_managed`; backup `'disabled` (cluster_resources via SystemBackupDef.tls_certmanager)

**Stateless runtimes / OS / kernel layer:**
- `containerd`         — `stateless`; container runtime
- `crio`               — `stateless`; container runtime
- `runc`               — `stateless`; OCI runtime
- `crun`               — `stateless`; OCI runtime
- `youki`              — `stateless`; OCI runtime
- `os`                 — `stateless`; base OS configuration
- `kubernetes`         — `stateless`; kubeadm-managed CP/kubelet (state in etcd via SystemBackupDef.etcd, PKI via SystemBackupDef.k8s_certs)
- `k0s`                — `stateless`; data_dir via SystemBackupDef.host_configs when k0s is the active distribution
- `k8s-nodejoin`       — `stateless`; one-shot join operation
- `vol_prepare`        — `stateless`; block-device prep
- `longhorn_node_prep` — `stateless`; node-level Longhorn prep

**Infrastructure glue (controllers / CSI / network):**
- `cilium`                  — `infrastructure_glue`; CNI controller
- `hccm`                    — `infrastructure_glue`; Hetzner cloud controller
- `hetzner_csi`             — `infrastructure_glue`; CSI driver
- `democratic_csi`          — `infrastructure_glue`; NFS dynamic provisioner
- `external_nfs`            — `infrastructure_glue`; NFS server
- `local_path_provisioner`  — `infrastructure_glue`; local-path provisioner
- `fip`                     — `infrastructure_glue`; floating IP attachment
- `etcd`                    — `infrastructure_glue`; data captured by SystemBackupDef.etcd
- `private_gateway`         — `infrastructure_glue`; Cilium gateway
- `buildkit_runner`         — `infrastructure_glue`; ephemeral by design (ADR-039)

**DNS providers:**
- `coredns`                 — `dns_provider`; zone files via SystemBackupDef.external_coredns
- `external_dns`            — `dns_provider`; controller, records live in upstream provider
- `resolv`                  — `stateless`; /etc/resolv.conf manager

**TLS endpoints with ACME:**
- `forgejo`                 — `tls_endpoint_with_acme`; backup `'pending` (Git+DB+actions)
- `woodpecker`              — `tls_endpoint_with_acme`; backup `'pending` (DB+agent state)
- `stalwart`                — `tls_endpoint_with_acme`; backup `'pending` (mail data+DB)
- `nats`                    — internal cluster service; backup `'pending` (JetStream state)
- `odoo`                    — bespoke; backup `'pending` for odoo-stack BackupGroup

**Databases:**
- `mariadb`                 — `database`; backup `'pending` (dump_strategy at workspace level)
- `surrealdb`               — `database`; backup `'pending` (dump_strategy at workspace level)

**VPN gateway:**
- `wireguard`               — stateless; peer keys + config in git/SOPS

**Total: 37 of 38 components migrated.** Only `backup_manager` is excluded —
it is the subsystem itself and does not consume its own ServiceConcerns schema.

**No components pending migration with `extensions/` directory.**

Components without `extensions/components/<name>/` directory (workspace-only):
- `prometheus.ncl`, `grafana.ncl`, `loki.ncl`, `vector.ncl` — declare `concerns`
  directly in the workspace `.ncl` file
- `ops_controller.ncl`, `audit_mirror.ncl`, `radicle_seed.ncl`,
  `coredns_vpn.ncl`, `coredns_vpn_wrk1.ncl` — workspace-declared, mostly
  reusing `ext.make_coredns` (which carries inherited `concerns` from coredns).

## Reusable presets

`provisioning/schemas/lib/concerns_presets.ncl` exports 7 archetype presets that
defaults.ncl can import directly to avoid duplicating the same 9-line block
across components. Components that match an archetype exactly should use:

```nickel
let _presets = (import "schemas/lib/concerns_presets.ncl").presets in

{
  <component> | default = {
    # ... existing fields ...
    concerns | default = _presets.<archetype>,
  },
}
```

**Components using presets directly (8):** `containerd`, `crio`, `runc`, `crun`,
`youki`, `os` (`stateless`); `cilium`, `hccm` (`infrastructure_glue`).

**Components using preset + override** (`_presets.<name> & { … | force = … }`)
**(22):** `k8s-nodejoin`, `vol_prepare`, `resolv`, `wireguard`,
`longhorn_node_prep`, `kubernetes`, `k0s` (stateless variants);
`hetzner_csi`, `democratic_csi`, `external_nfs`, `local_path_provisioner`,
`fip`, `private_gateway`, `buildkit_runner`, `etcd` (infrastructure_glue);
`longhorn`, `cert_manager` (infra_storage_managed); `coredns`, `external_dns`
(dns_provider); `postgresql`, `mariadb`, `surrealdb` (database).

**Components using parametric preset** (`_presets.tls_endpoint_with_acme {…}`)
**(6):** `zot`, `docker_mailserver`, `forgejo`, `woodpecker`, `stalwart`, `odoo`.

**Of these, 4 also declare `manifest_hooks` in their `defaults.ncl`** (standard
gateway-based TLS lifecycle via `catalog/lib/tls-hook-methods.sh`):
`forgejo`, `wordpress_site`, `odoo`, `rustelo_website`.

`zot`, `docker_mailserver`, `stalwart` use custom TLS lifecycle (own lib methods)
and do NOT declare `manifest_hooks` — the preset does not carry hooks.

**Components with full inline blocks (1):** `nats` (internal cluster service,
distinct from public TLS endpoints).

**Total: 36 of 37 migrated components use presets** (97%).

## Mechanical pattern

For each `provisioning/extensions/components/<X>/nickel/`:

### Step 1 — `contracts.ncl`

Add the import at the top of the file (above the outer `{`):

```nickel
let _concerns_lib = import "schemas/lib/concerns.ncl" in
```

Inside the component schema record (before the trailing `..` if it has one,
or as the last field otherwise) add:

```nickel
    # ServiceConcerns umbrella (ADR-008). See defaults.ncl for the populated value.
    concerns | _concerns_lib.ServiceConcerns | optional,
```

If the contract is a closed record (no `..`), also add `..` so legacy
`mode`/`operations`/`live_check` fields are accepted by the contract.

### Step 2 — `defaults.ncl`

Add a `concerns | default = { ... }` block before the closing `}` of the
default record. Choose the variant matching the component's archetype.

#### Stateless preset (runtime/OS)

```nickel
    concerns | default = {
      tls   = { kind = 'disabled, reason = "no TLS termination at this layer" },
      dns   = { kind = 'disabled, reason = "no DNS records owned by this component" },
      certs = { kind = 'disabled, reason = "no ACME issuer required" },
      backup = {
        kind   = 'disabled,
        reason = "stateless: configuration in git, no runtime data to capture",
      },
      observability = { kind = 'pending, reason = "ObservabilityImpl iteration deferred", backlog_ref = "OBS-001" },
      security      = { kind = 'pending, reason = "SecurityImpl iteration deferred", backlog_ref = "SEC-001" },
    },
```

#### Infrastructure glue (controllers/operators)

```nickel
    concerns | default = {
      tls   = { kind = 'disabled, reason = "controller-level RBAC, not TLS endpoint" },
      dns   = { kind = 'disabled, reason = "no DNS records owned by this component" },
      certs = { kind = 'disabled, reason = "no ACME issuer required" },
      backup = {
        kind   = 'disabled,
        reason = "state in K8s API captured by SystemBackupDef.cluster_resources",
      },
      observability = { kind = 'pending, reason = "ObservabilityImpl iteration deferred", backlog_ref = "OBS-001" },
      security      = { kind = 'pending, reason = "SecurityImpl iteration deferred", backlog_ref = "SEC-001" },
    },
```

#### TLS endpoint with ACME (public service)

```nickel
    concerns | default = {
      tls = {
        kind     = 'enabled,
        tls_impl = {
          secret_name = "<COMPONENT>-tls",
          issuer_ref  = "letsencrypt-prod",
          hostnames   = [],
        },
      },
      dns = {
        kind     = 'enabled,
        dns_impl = {
          internal = [],
          zone     = "",
        },
      },
      certs = {
        kind       = 'enabled,
        certs_impl = {
          acme_server = "https://acme-v02.api.letsencrypt.org/directory",
          email       = "",
          secret_ref  = "",
          provider    = 'cloudflare,
        },
      },
      backup = {
        kind        = 'pending,
        reason      = "BackupPolicy declared at workspace level",
        backlog_ref = "BACKUP-<COMPONENT>-001",
      },
      observability = { kind = 'pending, reason = "ObservabilityImpl iteration deferred", backlog_ref = "OBS-001" },
      security      = { kind = 'pending, reason = "SecurityImpl iteration deferred", backlog_ref = "SEC-001" },
    },
```

#### Database

```nickel
    concerns | default = {
      tls    = { kind = 'disabled, reason = "internal cluster service, ingress-level TLS handled separately" },
      dns    = { kind = 'disabled, reason = "no public DNS records" },
      certs  = { kind = 'disabled, reason = "no ACME issuer required" },
      backup = {
        kind        = 'pending,
        reason      = "BackupPolicy with database scope + dump_strategy declared at workspace level",
        backlog_ref = "BACKUP-DB-<COMPONENT>-001",
      },
      observability = { kind = 'pending, reason = "ObservabilityImpl iteration deferred", backlog_ref = "OBS-001" },
      security      = { kind = 'pending, reason = "SecurityImpl iteration deferred", backlog_ref = "SEC-001" },
    },
```

### Step 3 — Verify

```bash
cat > /tmp/check_<X>.ncl <<EOF
let mod = import "extensions/components/<X>/nickel/defaults.ncl" in
mod.<x>.concerns
EOF
nickel export --import-path provisioning /tmp/check_<X>.ncl
```

Expected output: a JSON object with `tls`, `dns`, `certs`, `backup`,
`observability`, `security` keys, each with `kind` and matching payload.

## Workspace-level overrides

After per-component defaults are populated, the workspace declarations in
`infra/<workspace>/components/<x>.ncl` can override individual concerns. The
most common override is to bring `backup` from `'pending` to `'enabled` with
a real `BackupPolicy` referencing `infra/<workspace>/lib/backup_policies.ncl`
presets.

Example — workspace activates backup for a component with full BackupPolicy:

```nickel
let bp_lib = import "../../lib/backup_policies.ncl" in
let ext    = import "extensions/components/<X>/nickel/main.ncl" in

{
  <x> = ext.make_<x> {
    # ... existing fields ...
    concerns = ext.defaults.<x>.concerns & {
      backup = {
        kind        = 'enabled,
        backup_impl = {
          provider     = { name = "restic" },
          destinations = [bp_lib.destinations.hetzner_primary, bp_lib.destinations.b2_replica],
          encryption   = bp_lib.encryption.component_key "<x>",
          schedule     = { kind = 'cron, cron_expr = "0 3 * * *" },
          retention    = bp_lib.retention.gold,
          scopes       = [
            { kind = 'service_full, name = "main", paths = ["/var/lib/<x>"] },
          ],
          tag_strategy = bp_lib.tag_strategy.component "<x>",
        },
      },
    },
  },
}
```

## Op-level manifest hooks (TLS gateway lifecycle)

Some cluster components require lifecycle steps that are structurally identical
across instances — Cloudflare secret, ClusterIssuer, Certificate, ReferenceGrant,
Gateway patch. Rather than repeating them in every `manifest_plan.ncl`, these
components declare `manifest_hooks` inside their `concerns` record in `defaults.ncl`.

The Nu bundle builder (`comp-build-cluster-bundle`) merges hooks from two sources:

```
effective_pre  = concerns.manifest_hooks.op.pre  ++ manifest_plan.hooks.op.pre
effective_post = manifest_plan.hooks.op.post ++ concerns.manifest_hooks.op.post
```

Generated `run-<op>.sh` executes: `effective_pre ++ plan_steps ++ effective_post`.

### Which components use manifest_hooks

Only components that deploy via the **standard gateway-based TLS lifecycle**
(cert-manager ClusterIssuer + Gateway API HTTPRoute + Cloudflare DNS-01):

- `forgejo`, `wordpress_site`, `odoo`, `rustelo_website`

Components with custom TLS lifecycle (`docker_mailserver`, `stalwart`, `zot`)
use the preset for concerns compliance only — no manifest_hooks.

### Required env vars for hook-based TLS components

Each component's `env-<name>.j2` must export these vars so that
`catalog/lib/tls-hook-methods.sh` can execute the hooks at deploy time:

```bash
TLS_HOOK_CF_SECRET_NAME=<cf-secret-name>       # e.g. dns-jesusperez-pro
TLS_HOOK_GATEWAY_NAME=<gateway-name>           # e.g. libre-wuji
TLS_HOOK_GATEWAY_NS=<gateway-namespace>        # e.g. kube-system
TLS_HOOK_LISTENER_NAME=https-<service-name>    # e.g. https-wordpress-diegodelgado
TLS_HOOK_DOMAIN=<primary-domain>              # e.g. diegodelgado.es
TLS_HOOK_TLS_SECRET=<tls-secret-name>         # e.g. wordpress-diegodelgado-tls
TLS_HOOK_NAMESPACE=<service-namespace>         # e.g. diegodelgado
TLS_HOOK_CLUSTER_ISSUER=<issuer-name>          # e.g. letsencrypt-prod-diegodelgado
```

These are populated from Tera variables that `vars.nu` must expose (no component
prefix): `cf_secret_name`, `gateway_ip`, `tls_secret`, `dns_zone`.

### Adding a new gateway-TLS component

1. Declare `concerns | default = (_presets.tls_endpoint_with_acme {...}) & { manifest_hooks = {...} }` in `defaults.ncl` (copy from forgejo/defaults.ncl).
2. Keep `manifest_plan.ncl` clean — no TLS steps (create-cf-secret, clusterissuer, certificate, referencegrant, patch-gateway-https, remove-gateway-https).
3. Export `TLS_HOOK_*` vars in `env-<name>.j2`.
4. Ensure `vars.nu` outputs `cf_secret_name`, `gateway_ip`, `tls_secret` (without component prefix).
5. No changes to lib.sh — TLS methods live in `catalog/lib/tls-hook-methods.sh`.

## Reference

- ADR-008: Service Concerns Umbrella
- ADR-009: Progressive Concerns Coverage
- ADR-010: Backup Architecture
- ADR-011: Multi-Destination Custody
- Schema: `provisioning/schemas/lib/concerns.ncl`
- Presets: `provisioning/schemas/lib/concerns_presets.ncl`
- Hook methods: `catalog/lib/tls-hook-methods.sh`
- Workspace defaults: `workspaces/libre-wuji/infra/lib/concerns_defaults.ncl`