Skip to content

MÓDULO ML cortex — VALIDATOR.Art Phase E sidecar

Python sidecar that learns to assign node roles from operator-validated examples.

Function

The ML cortex sidecar runs alongside validator-art in the same K8s pod (slot .97 Deployment). When the operator opts in (--ml-cortex-url=http://127.0.0.1:8087), the validator's /enroll flow consults the cortex first via 127.0.0.1 HTTP/JSON; the cortex returns a RoleAssignment derived from a trained ONNX model OR falls back to a line-for-line port of the Go heuristic when no model is loaded.

Per ADR-0088. Paired with VALIDATOR.Art ADR-0086 (receiver) and TBI Image ADR-0087 (image side).

Identity

Element Value
Plane MGMT (sidecar to VALIDATOR.Art)
Internal code validator-ml-cortex
Package pkg/validator-ml-cortex/ Python module
K8s namespace oobi-fabric (same as validator-art)
OOBI slot shares .97 (sidecar in validator-art Deployment)
Container port 8087 (cluster-internal, reached via 127.0.0.1 from validator-art)
ADR ADR-0088
Patent Family E reserve, claim #18

HTTP endpoints (port 8087, in-pod loopback)

Method Path Purpose
GET /healthz liveness probe
GET /metrics Prometheus text exposition
POST /predict-role HardwareFingerprint + TopologyHints + ClusterView → RoleAssignment

Architecture

                                 Same K8s pod (slot .97)
       ┌────────────────────────────────────────────────────────────┐
       │  validator-art (Go)                                        │
       │  ├─ /enroll receives POST from TBI agent                   │
       │  ├─ when --ml-cortex-url set:                              │
       │  │    ↓ HTTP POST 127.0.0.1:8087/predict-role (2s timeout) │
       │  ├─ on success: use ML RoleAssignment                      │
       │  └─ on error/timeout: roleassign.Compute heuristic         │
       │                                                            │
       │  ┌──────────────────────────────────────────────────────┐  │
       │  │ ml-cortex (Python FastAPI sidecar)                   │  │
       │  │ ├─ ONNX Runtime loads role-predictor.onnx (if file   │  │
       │  │ │  present at ML_CORTEX_MODEL_PATH)                  │  │
       │  │ ├─ /predict-role serves ML prediction OR             │  │
       │  │ │  heuristic.py fallback when model not loaded       │  │
       │  │ └─ RoleAssignment.source = "onnx" | "heuristic"      │  │
       │  └──────────────────────────────────────────────────────┘  │
       └────────────────────────────────────────────────────────────┘

Training pipeline (operator-side, not K8s)

# 1. Generate the heuristic-replay seed (deterministic per --seed)
python -m validator_ml_cortex.training.generate_replay \
  --output datasets/seed-heuristic-replay.jsonl \
  --count 200 --seed 42

# 2. Train a DecisionTree + export to ONNX
pip install "validator-ml-cortex[training]"
python -m validator_ml_cortex.training.train \
  --dataset datasets/seed-heuristic-replay.jsonl \
  --output models/role-predictor.onnx \
  --max-depth 12

# 3. Verify the model matches the heuristic ≥ 99% on the seed set
pip install "validator-ml-cortex[inference]"
python -m validator_ml_cortex.training.verify \
  --model models/role-predictor.onnx \
  --dataset datasets/seed-heuristic-replay.jsonl \
  --threshold 0.99

Why DecisionTree for Phase E baseline

  1. Interpretable — operator can walk the tree path that produced a prediction
  2. Compact — ONNX graph stays < 100 KB; ships embedded in the sidecar image
  3. Mirrors the heuristic structure — the Go heuristic IS nested if-else; DecisionTree is the right model class to LEARN that shape (a deep net would overfit the 200-example seed)
  4. scikit-learn maturity — DecisionTreeClassifier is stable, well-understood, no GPU required

Phase F upgrades to GradientBoosting once operator-validated examples accumulate beyond the heuristic-replay baseline.

Feature vector (12 positional features)

Pinned in validator_ml_cortex.training.featurize.featurize. Reordering or widening this list is a model-retraining event, guarded by test_label_classes_stable + test_featurize_vector_length.

Idx Feature Notes
0 cpu_count int
1 memory_gb int
2 dpdk_capable 0/1
3 has_gpu 0/1
4 existing_nodes int (cluster occupancy)
5 hint_role_idx 0=none, 1=trex-agent, …, 7=infra
6 hint_deployment_idx 0=none, 1=single, …, 4=multi
7 hint_plane_idx 0=none, 1=data, 2=control, 3=mgmt
8 role_counts[agents] int
9 role_counts[ngfw-dut] int
10 role_counts[trex-agent] int
11 role_counts[ml-cortex] int

Label classes (pinned, 7 canonical)

agents / infra / k6 / ml-cortex / ngfw-dut / playwright / trex-agent

Operator switchover semantics

  • Default (heuristic-only): --ml-cortex-url flag commented out in k8s/oobi/97-validator-art.yaml. validator-art never consults the sidecar; the sidecar boots anyway (cheap, ~256 MiB) but only serves its own /predict-role (operator can curl-test independently)
  • Opt-in (ML-served when available): operator un-comments the flag. validator-art consults sidecar with 2-second timeout; on error/timeout/unreachable, validator falls through to its own heuristic.Compute silently
  • Hard requirement: enrollment NEVER fails because ML is unavailable. The heuristic is always the safety net.

Tier policy

ML cortex is foundational infrastructure paired with VALIDATOR.Art — all tiers can use it. The opt-in flag is operator-controlled, not tier-gated.

Telemetry

Metric Description
validator_ml_cortex_simulated_mode 1 when no ONNX model loaded (heuristic fallback), 0 in ML-served mode
validator_ml_cortex_predictions_total{source} Counter labelled source="onnx" or source="heuristic"

Hardware footprint

Component Image RAM baseline CPU baseline
ml-cortex sidecar (FastAPI + Pydantic, no onnxruntime) ~150 MB 256 MiB 100m CPU
(with onnxruntime via [inference] extras) ~400 MB 512 MiB 100m CPU
(training stack via [training] extras — laptop / CI only) ~1 GB 1 GiB 1-2 vCPU

Sidecar runs on top of python:3.12-slim with non-root UID 2087.

Patent posture

Patent Family E — Enrollment Cortex — claim #18 reserved (provisional draft at docs/patents/family-e-enrollment-cortex-provisional-draft.md). Headline claim covers the combination of: hardware-fingerprint classification + heuristic-replay-bootstrapped ONNX + silent fallback + operator-validated retraining loop + auto-deploy with threshold gate + source attribution + opt-in operation.

Total patent claims after this wave: 18 (was 17 — TREX.Art #17 the prior most-recent).

Deferred (post-Phase-E-scaffold)

  • Phase F real trained model in production (needs ≥ 100 operator-validated examples first)
  • Phase F Dashboard "Approve / Override" workflow on /admin/validator-art capturing operator-validated training examples
  • Phase F K8s CronJob for weekly retraining + auto-deploy with verify.py threshold gate
  • Phase G the other 7 ML cortex functions from the memo (discovery method ranking, cascade topology optimization, failure prediction, auto-recovery, anomaly detection, drift prediction, scan pattern adaptation)
  • Phase F-late Federated learning + differential privacy (Flower + Opacus per discuss_validator_tech_stack)
  • Strategic memo: discuss_module_validator_tbi_2026_05_10 (§"ML cortex embedded (8 functions)")
  • Tech stack: discuss_validator_tech_stack_2026_05_10 (§Layer 4)
  • ADR: ADR-0088
  • Paired MÓDULO: VALIDATOR.Art (ADR-0086)
  • Paired wave: TBI Image (ADR-0087)
  • Patent: docs/patents/family-e-enrollment-cortex-provisional-draft.md