Performance
The published check() and propagation-path benchmark numbers.
Performance
check() sits on every guarded tool call, so it carries a latency budget:
under 1 ms of overhead per call in the common case. The benchmark and its
result are published here rather than only claimed.
Benchmark
bench.py is a plain script at the repository root, not a
pytest-discovered test and not a CI gate:
uv run python bench.pyIt measures three things: check() overhead for a small tainted argument
and for a 25 KB tainted argument, both against a two-rule sink policy, and
Tainted.splitlines() on a 5000-line tainted string.
Methodology: each check() benchmark warms up for 50 calls and then
measures 2000; splitlines() warms up for 10 and measures 200. Every
check() call passes an explicit run_id and reports through an
InMemoryReporter, so the numbers cover the decision path plus an in-memory
append, and exclude any reporter that performs I/O (see
the note on JsonlReporter).
Results
Measured on the reference development machine (CPython 3.12, macOS):
| Benchmark | Mean | Median | p95 |
|---|---|---|---|
check(), small arg (25 bytes) | 0.13 ms | 0.13 ms | 0.15 ms |
check(), large arg (25 KB) | 0.13 ms | 0.13 ms | 0.15 ms |
Tainted.splitlines(), 5000-line string | 2.2 ms | 1.8 ms | 2.0 ms |
check() overhead is essentially flat between a 25-byte and a 25 KB
argument, comfortably inside the 1 ms budget. Argument size does not drive
the cost; label inspection over a small frozen structure and one CEL
evaluation do.
Before the propagation-path fast path
Before the fixes below landed, splitlines() on the same 5000-line string
took 78 ms, about 35x slower, because every one of the 5000 returned
parts triggered a full label merge (_merge_labels), minting a fresh
uuid4 and Label per part at roughly 15 microseconds each. That also meant a Decision for a
sink call downstream of that split carried 5000 contributing_labels
instead of 1, which a JsonlReporter then serialized in full per event.
Four changes closed the gap:
- Single-label fast path. A single-parent derivation (a slice, a case
change, one part of a split) reuses the parent
Labelobject outright, including itsvalue_id, since there is nothing to merge. A freshvalue_idis now minted only at ingress, at a genuine multi-label merge, or at anendorse()hop. Policy.sources_tableprecomputed once, inPolicy.__init__, instead of rebuilt on every access, which previously happened once percheck()call.build_contextbuilt only when a sink has rules to evaluate. A tool with no matching sink now skips CEL-context construction entirely.- Single-pass trust resolution.
check()previously resolved every contributing label's trust up to four separate times, once each for the CELtaintlist,max_trust,trifecta, anduntrusted_sources. It now resolves each label once and derives all four from that.
The same 5000-line splitlines() now measures about 2.2 ms, and
contributing_labels for a split-then-sink flow collapses from 5000 labels
to 1. The de-duplication that produces that collapse is described in
Taint propagation internals.
What this does not cover
- The laundering audit (
audit=True) and any future flow-tracking instrument. Both are off by default, and running either takes that run outside the per-call budget. See Auditing. - Reporter I/O. The budget covers computing and emitting the decision,
not what a reporter does with it.
JsonlReporterfsyncs per record, and a custom reporter that ships to a collector inline adds its own latency to every guarded call. See Reporters. - Deeply nested arguments. Cost scales with the number of leaves
actually walked, bounded by
RECURSION_DEPTH. A call whose arguments are large nested containers pays more label-collection cost than the flat arguments measured here.