Skip to content

ADR-0087: TBI image — TLSStress Bench Image multi-format release (Phase I scaffold)

  • Status: Accepted (2026-05-24) — Phase I scaffold (TBI-1..TBI-4) shipped to main in PRs #1050 / #1052 / #1054 / this PR. Production-grade FIPS hardening + bare-metal QEMU smoke test land in TBI Phase I-late
  • Date: 2026-05-24
  • Driver: discuss_module_validator_tbi_2026_05_10 §TBI contents + §"Image build pipeline" — closes the operator-experience gap "how do I get my new UCS server into the cluster?" with a bootable image instead of kubectl + Ansible + lots of YAML per node
  • Related: PRs #1050 (TBI-1), #1052 (TBI-2), #1054 (TBI-3), this PR (TBI-4); sibling ADR-0086 (VALIDATOR.Art receiver, slot .97); upcoming ADR-0088 (ML cortex sidecar, Phase E)

Context

discuss_module_validator_tbi_2026_05_10 named the artifact TBI — TLSStress Bench Image ("não sei o nome para correspondente disso para K8" — the operator's verbatim question; we answered with our own multi-format image). The image is the customer-facing operator handoff: hand them a USB / qcow2 / OCI bundle, they boot it, and the in-image agent enrolls with VALIDATOR.Art (/enroll slot .97) — node joins the cluster, MÓDULOs come up, Dashboard shows them. No kubectl, no helm, no ansible.

Patterns inspired by: Talos Linux (immutable K8s OS), Kairos (immutable Linux for k3s/k8s), Bottlerocket (AWS), Cluster API (declarative cluster lifecycle), MAAS (Canonical's Metal-as-a-Service). None of them ship with a ZTP enrollment cortex. That's our differentiator — TBI is paired with VALIDATOR.Art's role-assignment + ML cortex (ADR-0086 + ADR-0088), not just a bootable OS.

Decision

Ship TBI as a multi-format Phase I scaffold with the build pipeline + first-boot agent + K8s integration + operator-facing Dashboard library. Phase I-late hardens for FIPS / air-gap; Phase J adds intent-based provisioning.

Materialized in 4 sub-PRs:

Sub-PR PR Scope
TBI-1 #1050 pkg/tbi-agent/ Go scaffold — fingerprint collector (lshw / dmidecode / ipmitool best-effort, simulated-mode fallback for CI), enroll HTTP client (POST /enroll), JoinBundle writer (k3s/agent.env + CA bundle + WireGuard wg0.peers), heartbeat loop (60s default). 20 unit tests across 4 packages
TBI-2 #1052 build/tbi/ mkosi pipeline — Ubuntu 24.04 LTS noble base + minimal packages + cloud-init drop-in + systemd unit tbi-agent.service (ProtectSystem strict, runAsNonRoot) + gVisor install hook + k3s install hook (pre-installed but disabled) + build-tbi.sh multi-format dispatcher (iso/img/qcow2/oci/all) + optional Sigstore cosign signing when COSIGN_KEY env set
TBI-3 #1054 k8s/oobi/98-tbi-builder-cronjob.yaml slot .98 weekly rebuild (Sun 04:00 UTC, Forbid concurrency, 50 GiB RWX tbi-library PVC, optional cosign secret) + Dashboard /admin/tbi-library page with artifact table + "Rebuild now" button (MVP placeholder API + clear banner; Phase B wires real PVC volume-listing)
TBI-4 This PR ADR-0087 + docs/modules/tbi-image.{md,pt-BR.md,es.md} + project_tbi_image_phase_i_2026_05_24.md recovery memo + MEMORY.md index entry

Locked decisions (Phase I)

# Decision Rationale
1 Ubuntu 24.04 LTS minimal (not Talos / Bottlerocket / Kairos) LTS through 2034 (matches the bench's 10-year lifecycle expectation); CIS hardening + FIPS-ready images available; package ecosystem (lldpd, snmpd, ipmitool, smartmontools) is mature. Talos / Bottlerocket would force us to fork their immutable model the moment operators need a debug shell at first boot
2 mkosi (not Packer / debootstrap / livecd-rootfs) Single declarative mkosi.conf produces multi-format output (iso/img/qcow2/oci); systemd's official tool so it tracks Ubuntu noble's systemd version; no Ruby/HCL toolchain
3 HTTP/JSON for enroll, not gRPC The validator-art receiver (ADR-0086 §"Why HTTP/JSON not gRPC") already speaks HTTP/JSON; matching the protocol simplifies operator debugging (curl works at first boot before the cluster comes up)
4 k3s pre-installed but DISABLED at boot Cuts first-boot time (no DNS/internet fetch); the agent enables k3s.service only after the JoinBundle lands. The systemd ConditionFileNotEmpty=/etc/k3s/agent.env guard prevents k3s from racing the agent
5 gVisor + containerd, NOT containerd-only DoYour.Art + KALI.Art pods declare runtimeClassName: runsc for sandboxing; pre-installing means no per-node setup when those MÓDULOs land
6 Weekly rebuild on Sunday 04:00 UTC Low-traffic window; matches the bench's typical operator on-call rotation cadence. Forbid concurrency policy because builds take ~15 min and share the PVC
7 Optional cosign signing (Sigstore) The Sigstore mount is a Secret with optional: true so first-boot operators without a signing key still get builds. Production deploys mount the key; the resulting .sig sibling file is verified at distribution time
8 PVC-based library, NOT GHCR for artifacts TBI artifacts are 1-2 GB each; GHCR has per-layer size limits + would charge per-pull bandwidth. The on-prem PVC is the right model for an operator who's likely shipping the image on USB or via internal HTTP mirror
9 Heuristic-replay role assignment in TBI/ML cortex TBI doesn't make role decisions — those live in VALIDATOR.Art. TBI just collects the fingerprint + posts it; the validator returns the JoinBundle with the role
10 No bare-metal QEMU smoke test in CI for Phase I scaffold mkosi needs root + loop devices; CI runners don't grant either. Phase I-late wires a self-hosted QEMU runner

Phase B+ hooks already wired

The scaffold leaves clear seams for Phase B-J without forcing a re-architect:

  • ~~CA.Art integration (Phase B)~~ ✅ Closed by ADR-0089 (VALIDATOR.Art Phase B-late caart.CertManagerIssuer). The agent transparently writes real cert-manager-minted PEM to /etc/tlsstress-art/node-{cert,key}.pem with mode 0600. End-to-end coverage in pkg/tbi-agent/internal/integration_test.go
  • ~~k3s real server join (Phase B)~~ ✅ Closed by ADR-0089 (VALIDATOR.Art Phase B k3sjoin.FileTokenSource reads /var/lib/rancher/k3s/server/node-token). Token + ServerURL flow to K3S_TOKEN + K3S_URL in /etc/k3s/agent.env
  • ~~WireGuard mesh peer pinning (Phase B)~~ ✅ Closed by ADR-0089 (VALIDATOR.Art Phase B-late wgmesh.ConfigMapPeerSource). Real peer set flows through JoinBundle.WireGuardPeers to /etc/wireguard/wg0.peers
  • ML cortex switchover (Phase E): ADR-0088 lands the operator-side governance for --ml-cortex-url opt-in (the validator-art Phase E client + sidecar are ML-3 / ML-4 — already merged)
  • Active discovery (NMAP/SNMP) (Phase F): TBI's lldpd + snmpd daemons + dmidecode/ipmitool already on disk

Patent posture

TBI Phase I does not raise a new patent claim by itself. The defensible lever sits on the VALIDATOR.Art ML cortex side (Patent Family E — ADR-0088, claim #18). TBI is "image with embedded ZTP agent that posts to a central validator" — the novelty is the validator's decision algorithm, not the image format.

Total patent claims unchanged: 17 (TREX.Art's #17 most recent).

Consequences

Positive

  • Closes the customer-handoff gap: USB → boot → enrolled, no kubectl
  • Multi-format release lets the bench drop into USB / cloud / hypervisor / OCI without a per-format build matrix
  • Weekly rebuild + Sigstore signing means the published image is always patched + provenance-tagged
  • Phase B-J slot in at clearly-marked seams (no architectural rework)

Negative / deferred

  • Production-grade FIPS hardening (Phase I-late) — current scaffold uses CIS hardening only; FIPS validation lands when the first customer asks
  • Bare-metal QEMU smoke in CI (Phase I-late) — self-hosted runner with root + loop devices required
  • ~~Real PVC volume-listing in Dashboard (Phase B)~~ ✅ Closed — pkg/tbi-builder-sidecar/ (PR #1061) walks the tbi-library PVC and serves /list-artifacts; Dashboard /api/tbi/library proxies through. Placeholder fallback only when the sidecar pod is unreachable
  • Air-gap shipping logistics (Phase J) — USB physical distribution, customer signing-key handoff
  • Boot-on-bare-metal smoke — manual operator workflow until Phase I-late

Alternatives considered

  • Talos Linux — rejected; immutable model prevents operator debug at first boot, forks of mainstream packages would be required for hardware probes
  • Kairos — same forking concern as Talos; smaller ecosystem
  • Packer — rejected; HCL toolchain dependency + manual format split (one Packer template per output format)
  • debootstrap — rejected; manual systemd unit wiring + no multi-format dispatcher
  • Per-node Ansible playbook (no image at all) — rejected; defeats the "USB handoff" customer experience that drove the memo

Compliance

Per CLAUDE.md: "Dashboard is the ONLY operator interface." TBI-3 ships /admin/tbi-library with artifact table + "Rebuild now" button — operator never touches kubectl apply against tbi-builder CronJob.

Per ZTP-prem posture: TBI agent runs with ProtectSystem=strict, NoNewPrivileges=true, restricted ReadWritePaths. The CronJob runs privileged: true (mkosi needs root + loop devices) but the resulting artifact runs unprivileged at first boot.

Per ADR-0044 (OOBI Intrusion Detection): TBI agent talks to validator-art over OOBI fabric only (cluster-internal DNS by default).

References

  • discuss_module_validator_tbi_2026_05_10 — strategic memo (TBI contents, multi-format, 6-phase enrollment flow, 9-phase roadmap)
  • ADR-0086 — VALIDATOR.Art receiver
  • ADR-0088 — ML cortex sidecar (paired Phase E wave)
  • PRs #1050 (TBI-1), #1052 (TBI-2), #1054 (TBI-3), this PR (TBI-4)
  • Upstream patterns: Talos / Kairos / Bottlerocket / mkosi / Cluster API
  • Patent claim posture: no new claim in Phase I; Patent Family E reserved for ML cortex (ADR-0088)