Silent Stderr Drops Don’t Throw Errors — They Just Vanish (Until 6/6)

A CI stop hook contract is the agreed behavior between a hook script and the test harness — specifying which exit code, which output stream, and which content proves a check passed or failed.

The contract that looked correct but wasn’t

Stop hooks in a CI harness carry a precise contract: rc=2 signals a block, and the reason must travel on stderr so the harness can forward it to the model. When that channel is wrong, the harness reports “No stderr output” — a literal harness message — and the run continues as if nothing happened.

The problem is that “No stderr output” looks like clean output. A stdout-only check reads that phrase, finds no anomaly, and marks the hook green. The contract break goes undetected because the detector was watching the wrong stream.

That is exactly what our measured run on 2026-08-27 found. The prior axis — stdout only — was treating unsourced_claim_hook as a contract violator when it behaved correctly: writing its reason to stderr and returning exit code 2. The hook was right. The checker was wrong. And the suite was reporting 3/3 while half the signal was invisible.

What the numbers showed when stderr was connected

Once the stderr channel was wired correctly, the selftests moved from 3/3 to 6/6. That jump is not a performance gain — it is the recovery of three checks that existed on paper but were producing no observable signal in the harness output.

The broader hook registry confirmed the scope: 13 registered stop hooks, each with its own measured pass counts — done_proof at 25/25, session_ledger at 101/101, unsourced_claim at 23/23, done_proof_hook at 13/13, change_report_hook at 21/21, unsourced_claim_hook at 16/16, session_ledger_stop at 20/20, tool_zero_mark at 9/9, and unrun_edit at 8/8. A live change_report_hook run confirmed 72 bytes of stderr now appear on screen — previously that output existed and was silently dropped.

The non-obvious point: a hook suite at 13/13 passing is not evidence of correctness if the stream carrying failure signals was never being read.

Fault injection: the only honest test of a contract detector

Passing selftests prove the happy path. They do not prove the detector fires when something actually breaks. The standard for a contract-break detector is that it must catch a contract break.

On 2026-08-27, fault injection — substituting a wrong output value — produced 4/6, with exactly 2 failures detected and then reverted. The harness caught what it was supposed to catch, and the revert confirmed the detection was not a false positive.

A later measured run on 2026-09-01 extended the pattern. A fault injection using a restored old command triggered a detection at wrapup_checklist.py:351 — the exact line — after which revert brought hook_cmd_rows() to 0. The shared_watchers.discover function returned True across all three dimension checks (S 73, O 98, U 92). The rule_vs_hook_conflict selftest moved from 5 to 13/13, wrapup_checklist reached 13/13, and asset_accounting_missing reached 23/23.

The decision rule this produces: a hook suite is only verified when fault injection recovers a measurable failure count. A suite that stays at N/N through fault injection is not hardened — it is unobservable.

The structural principle: safety doesn’t compose by addition

Adding more hooks to a CI pipeline does not make it safer if each hook’s output channel is unverified. The harness message “No stderr output” is not an error signal — it is absence of evidence, and absence of evidence is the cover story every silent drop hides behind.

The pattern seen here applies beyond hook testing. Any quality gate that reads only one output stream while the system contracts specify two is measuring a projection of the system, not the system itself. The gate passes because it cannot see the failure, not because the failure did not happen.

For teams running contract-break detectors: the first question is not “how many hooks are registered” but “what happens when you inject a fault into one of them.” If the answer is “the count stays the same,” the detector is watching the wrong stream.

What this means for AI-adjacent quality systems

Google’s August 2026 additions to Google Ads and Analytics — AI-powered performance summaries, text-prompt report generation, and cross-business benchmarking — are agentic additions that surface data trends automatically. The value of those summaries depends entirely on the integrity of the measurement pipeline beneath them. A benchmark comparison that silently drops the signal that something is wrong is not a smarter tool; it is a more confident one.

The principle transfers: agentic systems that report on their own health need fault injection, not just selftests. A summary that says “no issues detected” is only trustworthy if the detection path has been tested against known issues. “No stderr output” is always worth asking about.

Looking for more on ai & digital income? Visit SAVYX

Frequently Asked Questions

What does ‘No stderr output’ mean in a CI harness?

It is a literal harness message indicating the harness received nothing on the stderr channel — not confirmation that the hook ran cleanly. If the hook contract requires stderr for failure reasons, this message is a warning sign, not a clean bill of health.

Why did the stop hook suite jump from 3/3 to 6/6?

The prior stdout-only axis could not read the stderr channel where hook failure reasons were written. Connecting stderr correctly made three previously invisible checks observable, recovering signal that existed in the hooks but never reached the harness.

What is fault injection and why does it matter for hook testing?

Fault injection means deliberately breaking a known contract — substituting a wrong value or restoring a bad command — to confirm the detector fires. A suite that passes through fault injection without producing any failures is not hardened; it is unobservable. Our measured runs produced exactly the expected failure counts before revert.

How do you know hook_cmd_rows() at zero is a good result?

A value of zero means no registered hook commands are leaking or unresolved after a verified revert. The 2026-09-01 measurement confirmed this only after fault injection detected the exact fault at wrapup_checklist.py:351 and the revert was applied — so the zero is a confirmed clean state, not an uninspected one.

Does this apply to AI tool pipelines, not just CI hooks?

Yes. Any agentic system that reports on its own health — including AI-powered analytics summaries — needs the same principle: the detection path itself must be tested against known failures, not just run until no errors appear. A tool that cannot see its own failure mode will report confidence, not accuracy.

Disclosure: As an affiliate, this post may earn a commission from qualifying purchases at no extra cost to you.