Your Claim Filter Caught 4 Assertions — the Real Block Was a Missing Directory

A claim filter is a pipeline gate that inspects model-generated assertions for unsourced or fabricated specifics and returns a non-zero exit code to halt execution — but its coverage ends at the boundary of its own logic, leaving structural failures such as missing directories outside its detection surface.

The two runs that expose the gap

Two live stdin probes of the same pipeline produced strikingly different outcomes. In the first run, the filter identified four assertions and dropped all of them — exit code rc=0, execution continued. In the second run, the process stopped with rc=2. The logged reason: no orchestrator directory existed in the project tree. A find command printed the absence. The filter had nothing to do with that stop.

The lesson is structural, not logical. The filter worked exactly as designed in both runs. The problem is that a passing filter and a blocked pipeline are not contradictory — they operate on different failure surfaces.

Why complexity compounds before your filter sees it

The orchestrator directory finding maps directly onto a broader pattern in multi-agent systems. According to analysis published by Gravitee via VentureBeat, enterprise AI deployments rarely involve a single agent. They deploy fleets, each calling APIs, calling other agents, and reaching into systems never designed for machine decision-making. Adding a second agent adds one connection; adding a tenth potentially adds dozens, because any agent may call any other, and each call can trigger downstream calls.

The Gravitee analysis frames the core risk precisely: complexity compounds with the number of paths between agents, not with agent headcount. A support ticket that once touched one system may now pass through four agents before a human reviews it, and every handoff is a decision point nobody explicitly approved.

The missing directory is the single-agent version of that same problem. The filter inspects claims. The harness checks for the directory. These are two separate paths — and the failure lived in the one the filter never touches.

What the spend column adds that assertions cannot

The second probe registered a cost of USD 0.002004 and prompted a structural change: a claim_challenge column was added to the fleet spend tracker under fleet_spend by_caller. That column did not exist before the run that failed structurally.

This matters because assertion filtering and cost attribution answer different questions. Filtering asks: is this output fabricated? Cost attribution asks: which caller produced this charge, and under what condition? A run that exits at rc=2 before completing still consumes budget. Without the column, that spend was unattributed — invisible to the accountability layer that should govern it.

The Gravitee analysis identifies exactly this blind spot at enterprise scale: ask a security team which agent triggered which downstream action three hops ago, and the answer is silence. The spend column is the single-path, low-cost version of drawing that graph.

The selftest numbers tell a consistent story

Across the measured runs, selftest counts moved in one direction: upward, and without regression. One measured session moved from 28 to 37 of 37 passing. A separate session on the harness moved from 3 of 3 to 6 of 6 after the stderr contract was corrected — the prior implementation routed stderr to stdout only, which caused the unsourced_claim_hook (whose contract requires stderr output and a return of 2) to be flagged as a contract violation rather than a normal stop. Fault injection in that session confirmed detection: setting the output channel incorrectly produced 4 of 6 passing, with 2 failures detected, then restored.

A third measured session on the budget module ran 26 tests initially, reached 35 of 35, then had two guards deliberately removed — a floor guard and a required-reason guard. Three failures were detected. After restoration, 35 of 35 passed again. The circuit breaker module ran its own selftest suite and returned all pass with no damage.

The pattern across all sessions: fault injection always finds failures, and restoration always recovers. That is the only evidence worth trusting in a gate system — not that faults never occur, but that the detection surface catches them when they are deliberately introduced.

The counter-intuitive principle: more assertions caught means less structural coverage

The instinct after seeing four assertions dropped cleanly is to extend the assertion filter. Write more rules. Cover more patterns. This is the wrong response.

The Gravitee analysis makes the enterprise version of this mistake explicit: treating agent governance like a checklist — approve the agent, log the agent, move on. A checklist checks a single point in time. The orchestrator directory was absent at runtime, not at approval time.

The principle is: assertion coverage and structural coverage are orthogonal. A filter that catches every fabricated claim still has zero visibility into whether the directory it depends on exists, whether the caller that invoked it has budget, or whether the stderr channel it writes to is actually wired to the harness. These are not filter problems. They are harness and infrastructure problems, and they require their own detection layer.

What this means for teams building claim gates

Three decisions follow from this analysis.

First, separate the exit-code contract from the logic contract. The filter’s job is to evaluate assertions. The harness’s job is to enforce exit codes and route stderr. Conflating them — as the prior stdout-only implementation did — means a correctly functioning filter can silently fail to stop execution because the channel is wrong, not the logic.

Second, attribute spend to the condition that triggered it, not just to the caller. A run that exits at rc=2 on a missing directory is structurally different from a run that exits at rc=2 on a caught assertion. The claim_challenge column addition reflects this: the condition gets its own accounting line.

Third, fault injection is not optional. The selftest suites across these sessions only became meaningful when they included deliberate breakage. A suite that never sees a fault is not a safety net — it is a false assurance. The floor guard removal and the stderr misdirection tests both confirmed failures that a passing suite alone would never surface.

The enterprise framing from Gravitee is worth keeping close: the failure mode that should concern operators is not the agent they can see. It is the windy, complicated path nobody has drawn yet. At the single-pipeline level, the missing orchestrator directory was that path. At fleet scale, it multiplies with every agent added.

Looking for more on ai & digital income? Visit SAVYX

Frequently Asked Questions

What does rc=2 mean in a claim filter pipeline?

rc=2 is the exit code the pipeline returns when execution is blocked — either because the filter caught an assertion violation or because a structural condition failed, such as a missing directory. The two causes are different failure surfaces and require separate handling.

Why did four caught assertions still result in rc=0?

In the first probe run, the filter identified and dropped four assertions as designed, and the run completed without a hard stop — hence rc=0. A clean filter pass does not guarantee the pipeline is structurally sound; it only means the assertion logic found no reason to halt.

What is the claim_challenge column in fleet spend tracking?

It is an accounting line added to the fleet spend tracker after a run exited at rc=2 on a structural failure still consumed budget. The column attributes cost specifically to claim-challenge conditions so that incomplete runs are visible in spend reporting, not hidden under the caller’s general total.

How does fault injection improve selftest reliability?

A selftest suite that never encounters a deliberate fault cannot confirm its own detection surface. By removing guards — such as a floor guard or a required-reason guard — and verifying that the suite reports failures, the team confirms the tests catch real breaks, not just happy-path passes.

Is agent complexity really more dangerous than individual agent behavior?

According to Gravitee’s analysis published via VentureBeat, the answer is yes for enterprise deployments. A single agent’s behavior is auditable; the compounding paths between agents in a fleet are not. Each additional agent multiplies potential call paths, creating governance blind spots no single approval checklist covers.

Disclosure: As an affiliate, this post may earn a commission from qualifying purchases at no extra cost to you.