Skip to content

Inspection Profile — operator methodology

Read in your language: English · Português · Español

Scope status (post-Scope-Freeze 2026-05-10) — Inspection Profile is now Test Kind #3 in the canonical 7-Test-Kind set. Combinatorial Test Plan schema (10 components × 5 presets + custom 2^10) lives in dashboard/src/lib/test-plan/schema.ts. See ADR 0010 for the original design lock + the Test Plan combinatorial primer for atomic 2PC across DUT + Personas + Agents.

TL;DR

Every NGFW under test exposes ~13 inspection dials (TLS decrypt, IPS, AMP, URL filtering, etc.). Vendor datasheets cherry-pick the lightest combo; comparing two vendor reports requires a translation step that nobody does. The TLSStress.Art bench fixes both problems by bundling the dials into 5 named profiles that map cleanly onto RFC 9411 §6.

minimal   → L3/L4 only — the throughput ceiling
balanced  → TLS decrypt + IPS at default — the SMB enterprise baseline
paranoid  → + AMP + App Control + URL filter + file inspection
compliance → paranoid minus App Control + max-verbosity logging
sandbox   → paranoid + on-box file detonation (slowest)

Each test plan optionally declares one profile. The test-run engine applies it to the NGFW before agents launch (scripts/ip-apply-<vendor>.sh) and removes it at cleanup. Annex K of the report attests the dial state that was actually applied — including drift detection if the vendor silently changed something.

For the full design rationale see ADR 0010.

Why 5 named profiles instead of free dials?

Two failure modes if profiles don't exist:

  1. Vendor comparison is intractable. "FTD with Balanced policy" and "Palo Alto with Strict Security Profile" enable different dial sets. A customer reading two vendor reports cannot tell which numbers are apples-to-apples.
  2. Capacity planning hides the cost. Vendor datasheet numbers come from the lightest setting; production deployments add dials and discover the cost only after deployment.

The 5-profile bundling produces a vendor-agnostic vocabulary that maps onto the same dials RFC 9411 §6 enumerates. A customer can cross-reference a TLSStress.Art Annex K against a NetSecOPEN-certified vendor report (e.g. the Cisco FPR-3105 / UNH IOL October 2024 report) without translation.

The dial matrix

Dial minimal balanced paranoid compliance sandbox
L3/L4 ACL ✓ ✓ ✓ ✓ ✓
TLS Inspection (TLS 1.3) ✓ ✓ ✓ ✓
IDS/IPS default strict strict strict
Anti-Malware (signature) ✓ ✓ ✓
Anti-Malware (detonation) ✓
App Control ✓ ✓
URL Filtering ✓ ✓ ✓
File Inspection ✓ ✓ ✓
Logging verbosity std std std max max

Why DNS Security is not in any profile: the agents do not yet emit realistic DNS query patterns; turning DNS Security on would measure the vendor's response to synthetic-low-volume queries, which is misleading. DNS Security joins the matrix in v4.7+ when the agent fleet exposes a DNS query rate parameter.

RFC 9411 §6 cross-reference

RFC 9411 §6 feature           → Profiles that include it
─────────────────────────────  ────────────────────────────────────────
TLS Inspection                 balanced, paranoid, compliance, sandbox
IDS/IPS                        balanced (default), paranoid+ (strict)
Anti-Malware                   paranoid, compliance, sandbox
App Control                    paranoid, sandbox
URL Filtering                  paranoid, compliance, sandbox
File Inspection                paranoid, compliance, sandbox
Logging                        all (verbosity differs)
DNS Security                   none (deferred to v4.7+)
WAF / DLP / CASB               none (out of scope)
Sandboxing                     sandbox (only)
TLS 1.3 Decryption             folded into TLS Inspection — bench is
                               TLS 1.3-only since v3.6

Vendor mapping — Cisco FTD (v4.5 MVP)

The operator does not configure FTD by hand. scripts/ip-apply-ftd.sh --profile <name> does the equivalent of these FMC REST API calls:

Profile FTD Access Policy IPS Policy File Policy URL Category AMP
minimal Allow-Trust none none none off
balanced L4-and-Inspect Balanced (default) none none off
paranoid L4-Inspect-Block Security (strict) Block-Malware Block-Threats inline
compliance L4-Inspect-Audit Security (strict) Block-Malware Block-Threats inline
sandbox L4-Inspect-Block Security (strict) Block-Malware-AMP Block-Threats Capacity Mode

The apply script resolves preset names to FMC IDs at runtime — a vendor software upgrade that renames a preset fails the apply with "preset 'Balanced' not found, list of currently-shipped presets is X, Y, Z" rather than silently picking the wrong policy.

Palo Alto Networks lands in v4.6 with an equivalent table.

Annex K — per-run attestation

The run report's Annex K ("Inspection Profile Attestation") records:

  • The profile name the operator selected
  • The expected dial state (the matrix row above)
  • The observed dial state, read from the device via vendor API
  • Any drift between expected and observed (gates the run in production)
  • The RFC 9411 §6 feature subset this run satisfies
  • A NetSecOPEN cross-reference pointer — e.g. "this run is comparable to the FPR-3105 HTTPS Throughput benchmark from the Oct 2024 NetSecOPEN report"
report.inspection_profile:
  applied: true
  profile_name: "paranoid"
  vendor: "cisco-ftd"
  rfc9411_features: ["TLS Inspection", "IDS/IPS", "Anti-Malware",
                     "App Control", "URL Filtering", "File Inspection"]
  expected_dials:
    tls_inspection: true
    ids_ips: "strict"
    anti_malware_signature: true
    anti_malware_detonation: false
    app_control: true
    url_filtering: true
    file_inspection: true
    logging: "standard"
  observed_dials:                       # read from FMC at apply + verify
    tls_inspection: true
    ids_ips: "strict"
    anti_malware_signature: true
    anti_malware_detonation: false
    app_control: true
    url_filtering: true
    file_inspection: true
    logging: "standard"
  drift: []                             # empty = run proceeds
  netsecopen_cross_ref:
    classification: "RFC 9411 §6 subset (paranoid)"
    nearest_published_test: "FPR-3105 HTTPS Throughput (Oct 2024 NetSecOPEN report)"
  attestation_sha256: "sha256:…"

Operator workflow

Selecting a profile

In a test plan YAML:

- identifier: CAP-FIND-KNEE-30M-PARANOID
  display_name: "Capacity — paranoid inspection (30 min)"
  # … standard plan fields …
  environment:
    inspection_profile:
      name: paranoid
      vendor: cisco-ftd

The plan's ngfw_state_required field is implicitly satisfied by the profile (each profile knows whether decrypt is on / off). If the plan declares both, they must agree — the loader fails with a clear message otherwise.

Apply / verify cycle

The dashboard's preflight runs:

  1. ip-apply-ftd.sh --profile paranoid — issues FMC REST calls, waits for the policy push to converge (FTD typically 30–90s).
  2. ip-verify-ftd.sh --profile paranoid — reads back the running policy state and compares against the matrix. Drift → run blocked.

Manual verification (rare; preflight does this automatically):

scripts/ip-verify-ftd.sh --profile paranoid
# Expected:
#   Profile 'paranoid' (cisco-ftd): PASS
#   - tls_inspection: true (expected true)
#   - ids_ips: strict (expected strict)
#   - anti_malware_signature: true (expected true)
#   - … (every dial verified)
#   - drift: none

After the run

ip-rollback-ftd.sh restores the FTD baseline (the policy state captured before the apply). This is idempotent and safe to call when no profile is in place — useful in cleanup paths where the test runner crashed mid-run.

Combining with Branch Office Simulation

Inspection Profile (this work) and Branch Office Simulation (ADR 0008) are explicitly orthogonal — the bench can apply both, neither, or either. The combined plan is the realistic enterprise scenario:

environment:
  bandwidth:                # Branch Office (ADR 0008)
    down_mbps: 100
    up_mbps: 30
  inspection_profile:       # this work (ADR 0010)
    name: balanced
    vendor: cisco-ftd

Combined plans land in v4.6 alongside the Palo Alto vendor mapping.

When to bypass (emergency only)

If you must run a test before profile verification passes (vendor API flapping, customer demo in 5 minutes), set IP_GATING=observational in the operator UI before launching the plan. The test will run and Annex K will mark the profile as APPLIED BUT NOT VERIFIED with a red flag. Do not use this in front of customers.

References

  • ADR 0010 — methodology
  • RFC 9411 — Benchmarking Methodology for Network Security Device Performance (March 2023)
  • Cisco FPR-3105 NetSecOPEN report (UNH IOL, October 2024) — the third-party reference the matrix cross-references
  • scripts/ip-apply-ftd.sh / ip-verify-ftd.sh / ip-rollback-ftd.sh — Cisco FTD operator scripts (v4.5 PR-B)
  • dashboard/templates/annex-k-inspection-profile.md.tmpl — report annex (v4.5 PR-D)
  • dashboard/src/lib/preflight/inspection-profile-checks.ts — preflight check (v4.5 PR-C)
  • ADR 0008 — Branch Office Simulation (orthogonal companion)