Skip to content

ADR-0089: VALIDATOR.Art Phase B bridges + TBI Phase I-late hardening

  • Status: Accepted (2026-05-24) — Phase B foundation wave (PB-1..PB-4 + PVC-1 + TBI-LATE-1/2) shipped in PRs #1058 / #1059 / #1060 / #1061 / this PR
  • Date: 2026-05-24
  • Driver: closes the "Phase B deferred" items from ADR-0086 §"Deferred" + the "Phase I-late" items from ADR-0087 §"Deferred"
  • Related: ADR-0086 (VALIDATOR.Art Phase A) · ADR-0087 (TBI Image Phase I scaffold) · ADR-0088 (ML cortex Phase E) · PRs #1058 (PB-1 CA bridge) / #1059 (PB-2 k3s join) / #1060 (PB-3 + PB-4 WireGuard + GitOps) / #1061 (PVC-1 + TBI-LATE-1/2) / this PR (ADR + memo)

Context

ADR-0086 shipped VALIDATOR.Art with a JoinBundle that carried placeholder bytes for the four real-world materials a node needs to join the cluster: mTLS cert, k3s join token, WireGuard peers, ArgoCD module manifests. The deferred work in §"Phase B" was to wire each of those to a real source. ADR-0087 similarly deferred FIPS hardening + QEMU smoke + real PVC volume-listing.

The pattern across all six items is the same: adapter interface + Disabled/Fake/real impls + opt-in switchover. The validator never fails enrollment because a bridge is unavailable — it falls back to the V-1 placeholder silently and logs.

Decision

Ship six adapter-pattern bridges + two TBI Phase I-late hardening artifacts in a single wave (5 PRs):

Sub-PR Scope PR
PB-1 pkg/validator-art/internal/caart/ — Issuer interface + Disabled/Fake/CertManagerIssuer (stub) #1058
PB-2 internal/k3sjoin/ — Source interface + Disabled/Fake/FileTokenSource #1059
PB-3 internal/wgmesh/ — Source interface + Disabled/Fake/FilePeerSource #1060 (consolidated)
PB-4 internal/gitops/ — Writer interface + Disabled/Fake/PVCWriter (ArgoCD App-of-Apps YAML per node) #1060 (consolidated)
PVC-1 pkg/tbi-builder-sidecar/ Go module + sidecar Deployment + Dashboard proxy #1061
TBI-LATE-1 build/tbi/mkosi.fips.conf overlay + --fips flag on dispatcher #1061
TBI-LATE-2 .github/workflows/tbi-qemu-smoke.yml self-hosted runner #1061
ADR + memo This PR (this)

Locked decisions (Phase B)

# Decision Rationale
1 Adapter pattern with Disabled/Fake/real triple Same pattern as ML-3 mlclient (ADR-0088 §"Heuristic fallback always available"). Operator opt-in flips between modes per Config field; never required to wire every bridge to ship
2 Silent fallback on bridge error /enroll MUST NEVER fail because a bridge times out / errors / returns nonsense. Logs the error + keeps V-1 placeholder bytes in the JoinBundle. Same hard rule as ML-3 cortex switchover
3 CertManagerIssuer stays a stub in PB-1 The interface contract is what locks Phase B; the real k8s client-go + cert-manager v1 typed clientset wiring lands when an operator's cluster has both deployed. Stub returns "not implemented" so callers know to use FakeIssuer until then
4 FilePeerSource reads JSON, not ConfigMap The tbi-agent heartbeat can append to a shared mount via JSON merge-patch; ConfigMap controller is a Phase B-late upgrade. JSON is human-debuggable + works without K8s client-go
5 PVCWriter writes ArgoCD Application YAML, not raw manifests ArgoCD's App-of-Apps controller is the operator-facing reconciliation layer; writing one Application per module lets the operator see the deployment status in the ArgoCD UI
6 tbi-builder-sidecar is a SEPARATE Deployment (not a sidecar container in the CronJob pod) The CronJob pod runs only during the weekly build (~15 min); the artifact-listing API must stay up 24/7 between builds. Separate Deployment keeps the read path always-available
7 Dashboard /api/tbi/library keeps the placeholder fallback When the sidecar is unreachable, the Dashboard renders demo data + a clear banner; operator can still navigate the UI to figure out the sidecar problem. Same pattern as ML cortex (always degraded-but-functional)
8 FIPS overlay opt-in via --fips flag Most operators don't need FIPS. The overlay adds Ubuntu Pro tool dependency + a different kernel; default builds stay lean. --fips errors without COSIGN_KEY because FIPS releases MUST be signed
9 QEMU smoke runs on operator self-hosted runner GitHub-hosted runners don't grant /dev/kvm; self-hosted runner with tbi-smoke label is the contract. Workflow skips when no such runner is online — never blocks the gate

Bridge wire format (locked)

// Server.CA      caart.Issuer    — cfg.CAArtMode = "disabled" | "fake" | "cert-manager"
// Server.K3S     k3sjoin.Source  — cfg.K3SJoinMode = "disabled" | "fake" | "file"
// Server.WG      wgmesh.Source   — cfg.WGMeshMode = "disabled" | "fake" | "file"
// Server.GitOps  gitops.Writer   — cfg.GitOpsMode = "disabled" | "fake" | "pvc"

// Defaults: SimulatedMode → "fake"; real-mode → "disabled"

Each Config field has matching *Path / *BaseDir fields where applicable. Empty defaults to the canonical paths (/etc/validator-art/wg-peers.json, /var/lib/validator-art/gitops, etc.).

Consequences

Positive

  • Closes 5 of the 6 placeholders called out in ADR-0086 §"Phase B" (CA / k3s / WireGuard / GitOps / volume-listing)
  • Closes 2 of the 3 deferrals from ADR-0087 §"Deferred" (FIPS overlay + QEMU smoke; bare-metal customer-site test stays operator-driven)
  • Operator-facing surfaces (Dashboard /admin/validator-art + /admin/tbi-library) now degrade gracefully when bridges go offline — no broken pages
  • Adapter pattern means Phase B-real (cert-manager client-go wiring) is a same-file swap, not a re-architect

Negative / deferred

  • ~~CertManagerIssuer real impl~~ — Shipped Phase B-late (pkg/validator-art/internal/caart/cert_manager.go): generates ECDSA P-256 key, builds CSR with CN+SANs, POSTs CertificateRequest CR, polls status.conditions[Ready] until True (or Denied/Failed → error), returns CertBundle with in-process key. Stdlib-only (no client-go). Operator wires via CAArtMode="cert-manager" + CertManager{Namespace,ClusterIssuer,IssuanceTimeout} fields.
  • ~~ConfigMap-based WireGuard peer source~~ — Shipped Phase B-late (pkg/validator-art/internal/wgmesh/k8s_configmap.go): HTTP GET on /api/v1/namespaces/<ns>/configmaps/<name>, decodes data[Key] (default peers.json) as []Peer. Re-read per List call; bootstrap-safe (404 → empty slice). Operator wires via WGMeshMode="configmap" + WGMeshConfigMap{Namespace,Name,Key} fields.
  • ~~Git remote writer for GitOps~~ — Shipped Phase B-late (pkg/validator-art/internal/gitops/git_remote.go): clones shallow, writes one ArgoCD Application YAML per module, commits + pushes via os/exec of the git CLI. Idempotent on re-runs. Operator wires via GitOpsMode="git-remote" + GitOpsRemote{URL,Branch,BasePath,...} fields. Auth via SSH-agent (operator-side) OR token-file → GIT_ASKPASS helper.
  • Bare-metal first-boot customer smoke — operator-driven workflow until the QEMU smoke matures into a multi-vendor matrix (Phase J)

Alternatives considered

  • Single combined "Phase B bridge" interface — rejected; each adapter has different semantics (cert issuance is async + has timeouts; k3s token is a sync read; WireGuard list is multi-row; GitOps is write-side). Four interfaces is the right granularity
  • K8s admission webhook for role-decided manifests — rejected; would require validator-art to live in-line on every pod-create. ArgoCD App-of-Apps via PVCWriter is the right control plane
  • gRPC for the tbi-builder-sidecar — rejected; HTTP/JSON matches the rest of the project + Dashboard already speaks it
  • Sidecar container in the tbi-builder CronJob pod — rejected; CronJob runs ~15 min weekly, sidecar must serve 24/7

Compliance

Per CLAUDE.md: "Dashboard is the ONLY operator interface." All adapter modes are operator-toggled via Config fields surfaced through the K8s manifest's args — no kubectl edit required.

Per ZTP-prem posture: tbi-builder-sidecar runs readOnlyRootFilesystem: true, runAsNonRoot: true, runAsUser: 2098, drop: [ALL]. mounts tbi-library PVC read-only. The FIPS overlay's fips=1 audit=1 kernel cmdline enforces FIPS-validated crypto from boot.

References

  • ADR-0086 (VALIDATOR.Art Phase A — §"Deferred" called these out)
  • ADR-0087 (TBI Image Phase I scaffold — §"Deferred")
  • ADR-0088 (ML cortex Phase E — adapter-pattern precedent)
  • PRs #1058 (PB-1) · #1059 (PB-2) · #1060 (PB-3 + PB-4) · #1061 (PVC-1 + TBI-LATE-1/2) · this PR
  • discuss_module_validator_tbi_2026_05_10 strategic memo