Skip to content

ADR 0039 — SyslogReflector.Art — Transparent Syslog Mirror

  • Status: Accepted (2026-05-13) — materialization in PR introducing pkg/syslog-reflector/ + K8s sidecar in platform/observability/promtail-syslog.yaml
  • Date: 2026-05-13
  • Deciders: TLSStress.Art project
  • Targets: v4.x
  • Builds on: SYSLOG.Art (existing — MÓDULO SYSLOG.Art = promtail-syslog), ADR 0019 (OOBI fabric), ADR 0007 (Public-Internet Realism)

Context

The TLSStress.Art bench already runs MÓDULO SYSLOG.Art — a promtail-syslog deployment that listens for RFC 3164 + RFC 5424 syslog over UDP/TCP 1514 (exposed as NodePort 514) and pushes the events into Loki for correlation, dashboard surfacing, and DUT-effectiveness analysis (ADR 0034 / ADR 0035).

But there's a meaningful operator scenario this doesn't cover:

Many DUTs allow only a single syslog destination. This is true of: - Older Cisco IOS/NX-OS (single logging host <ip> line, no second slot) - Some Fortinet appliances at lower license tiers - DUTs sized for the customer's site that don't have CPU/RAM headroom to dual-stream syslog (a non-trivial CPU cost at high event rates) - DUTs under change-control freeze where adding a second logging entry requires a change-management ticket the operator can't get within the test window

If we ask the operator "please point your DUT syslog at us instead of your production SIEM," they refuse — losing prod SIEM visibility is a non-starter. If we ask "please dual-stream," we hit one of the above walls.

Decision

Introduce SyslogReflector.Art as an enhancement to the existing MÓDULO SYSLOG.Art — not a new MÓDULO. The implementation is a new sidecar container (pkg/syslog-reflector/) running in the same Pod as promtail-syslog, transparently mirroring inbound syslog to both the local Promtail ingester AND the operator's original / intended downstream syslog server.

Architecture

       Customer DUT
           │  syslog UDP/TCP/TLS (MGMT LAN)
           ▼
 ┌─────────────────────────────────────────────────────────┐
 │  Pod: promtail-syslog (MÓDULO SYSLOG.Art)               │
 │                                                          │
 │  ┌───────────────────────┐                              │
 │  │ syslog-reflector      │  ← NEW (this ADR)            │
 │  │ (sidecar container)   │                              │
 │  │ listens :514 UDP/TCP  │                              │
 │  │         :6514 TLS     │                              │
 │  └────┬──────────────────┘                              │
 │       │                                                 │
 │       ├──► localhost:1514 → Promtail → Loki             │
 │       │                                                 │
 │       └──► <operator's original SIEM> (the mirror)      │
 │                                                          │
 │  ┌───────────────────────┐                              │
 │  │ promtail-syslog       │  ← EXISTING                  │
 │  │ listens :1514         │                              │
 │  │ pushes to Loki        │                              │
 │  └───────────────────────┘                              │
 └─────────────────────────────────────────────────────────┘

Why a sidecar (not a separate Deployment)

  • Same MÓDULO: per ADR 0019, MÓDULOs are addressed by slot, not by Pod count. Adding a sidecar keeps SYSLOG.Art at one slot (.X — promtail-syslog's existing canonical IP).
  • Shared network namespace: reflector → Promtail is localhost:1514, a single syscall round-trip, no Service routing, no NetworkPolicy.
  • Single lifecycle: rolling-restart of the SYSLOG.Art Pod restarts both. Operators see one "thing" to monitor.
  • Reuses existing NetworkPolicy + Service: the reflector binds the same exposed ports the Pod already serves.

Why Go (not rsyslog/syslog-ng pre-fronted)

  • rsyslog/syslog-ng are mature but unfit for our deployment envelope: they ship as system packages, not first-class containers in our air-gap inventory; their config languages are bespoke and hard to validate in CI; loop-detection is operator's responsibility.
  • A small Go binary (≤500 LOC core + tests, zero external deps) gives us: explicit loop detection at config validation time, structured metrics matching the rest of the fleet, an audit hook compatible with ZTP-prem (ADR 0029), and DSCP CS7 marking via pkg/oobi/client/ for future v2 (out of MVP scope per ADR 0038's "operator's SIEM may not expect CS7" note).

Public API surface

import (
    "github.com/nollagluiz/AI_forSE/pkg/syslog-reflector"
)

r, err := reflector.New(reflector.Config{
    UDPListenAddr:  ":514",
    TCPListenAddr:  ":514",
    TLSListenAddr:  ":6514",
    TLSCertFile:    "/etc/tls/tls.crt",
    TLSKeyFile:     "/etc/tls/tls.key",
    LocalIngestUDP: "127.0.0.1:1514",   // Promtail in the same Pod
    ReflectionRules: []reflector.ReflectionRule{
        {Name: "prod-siem-cisco-ftd",
         SourceCIDR: "10.10.0.0/16",
         Target:     "tcp://siem.corp:6514"},
        {Name: "audit-paloalto",
         SourceCIDR: "10.20.0.0/16",
         Target:     "tls://audit.corp:6514"},
    },
})
if err != nil { log.Fatal(err) }
r.Run(ctx)

Reflection rule semantics

  • Source filtering is optional per-rule. Rules with SourceCIDR only fire for messages whose source IP matches. Rules without are unconditional (fire for every message).
  • Multiple rules can match the same message — each match produces one outbound mirror copy. This is intentional (one operator may want the same syslog mirrored to both SIEM and audit archive).
  • Targets: udp://, tcp://, or tls:// URLs. Plain UDP is best-effort (operator accepts the same loss profile their DUT already has). TCP/TLS auto-redial on failure with exponential backoff.

Loop prevention

Hard fail at config validation (New() returns error) if any reflection target's host:port matches any of our own listeners, including 127.0.0.1 and localhost heuristics. The operator gets a clear error message at startup — never a silent loop in production.

Non-blocking guarantees

The ingestion path (DUT → reflector → Promtail) MUST NEVER be blocked by a slow/dead reflection target: - Each reflection target has its own bounded channel (default 10000 messages). - Channel full → drop oldest, increment per-rule drop counter. - Ingestion to local Promtail is via UDP (best-effort, lossy by design). - The reflector itself does not buffer for the ingestion path.

Consequences

Pros

  • Unblocks a meaningful segment of DUT deployments — single- destination syslog, change-controlled customers, underpowered DUTs.
  • Operator UX is unchanged from their perspective: their SIEM keeps receiving syslog just like before, we just sit in the middle as a reflector.
  • Zero-config for the SIEM: the original syslog server doesn't need any awareness of our presence (we forward via plain syslog).
  • Composes with existing SYSLOG.Art correlation pipeline (ADR 0034 3-way fusion, ADR 0035 6-pillar effectiveness — they consume Loki, unchanged).

Cons / risks

  • Source IP of mirrored traffic is the reflector, not the original DUT. Mitigation: the syslog body itself contains the original hostname in its RFC 3164/5424 header — most SIEMs key off that, not the L3 source. We do NOT spoof the source IP (would break uRPF on most underlays).
  • One more failure surface in the operator-facing chain. If our reflector dies, both ingestion AND mirroring stop. Mitigation: same PDB as promtail-syslog (≥1 replica), aggressive container restart policy, monitored via Prometheus/Loki alert.
  • Latency added (~sub-ms to ms) to syslog delivery. Acceptable for syslog — already not a real-time protocol.
  • Privileged port 514: same NodePort indirection (514 → 1514 inside pod, mapped externally) extended to also route 514 → 1514 reflector receiver. The Pod itself never needs CAP_NET_BIND_SERVICE.

Compatibility

  • Backwards compatible: deploying the reflector sidecar is opt-in via a Deployment patch. Operators not using SyslogReflector run promtail-syslog exactly as before.
  • TLS: the reflector can terminate TLS from DUTs that emit syslog over TLS (RFC 5425). Promtail does too, but only Promtail-specific config; the reflector adds it as a uniform listener regardless of Promtail's TLS settings.

Implementation breakdown

# Deliverable Scope
1 pkg/syslog-reflector/ Go module Listeners + forwarders + loop detection + tests
2 pkg/syslog-reflector/cmd/.../main.go Binary with config-file + flags + /metrics + /healthz
3 pkg/syslog-reflector/Dockerfile Alpine multi-stage, non-root UID 2300
4 platform/observability/promtail-syslog.yaml patch Sidecar container + ConfigMap with reflection rules
5 modules.yaml entry Doc-4 conformance
6 ADR 0039 (this doc) Architecture authoritative
7 CHANGELOG entry Releasability

Dashboard surface ("SyslogReflector.Art" feature)

Separate follow-up PR. The dashboard adds a feature page named "SyslogReflector.Art" with: - Per-DUT reflection rule CRUD (writes to the reflector's ConfigMap) - Live message rate / drop count / per-target health - Test-reflect button (send a synthetic syslog event, verify both Promtail ingestion AND target receipt)

The underlying MÓDULO remains SYSLOG.Art. "SyslogReflector.Art" is the Dashboard feature surface name only.

References

  • Code: pkg/syslog-reflector/
  • K8s: platform/observability/promtail-syslog.yaml (sidecar patch)
  • ADR 0019 — OOBI slot allocation (SYSLOG.Art canonical slot)
  • ADR 0029 — Sealed audit log (BypassEvent-style audit hook for reflections)
  • ADR 0034 — 3-way fusion correlator (consumes the unmodified Loki stream)
  • ADR 0038 — DSCP marking (CS7 marking deferred to v2)
  • RFC 3164, RFC 5424 — syslog message formats
  • RFC 5425 — syslog over TLS
  • RFC 6587 — syslog over TCP (framing)