10/10 Tests Passed, But 7 Queue Items Still Sat Unprocessed Live

A self-test pass rate is a measure of how many scripted execution paths completed without error — not a guarantee that every real-world queue item reaches its destination and is acknowledged by a live consumer.

The number that looked like a win

Every engineer recognizes the relief: the test suite goes green. In SAVYX’s measured case, that meant 10/10 passes on the bidirectional self-test run on 2026-07-30 — and separately, a 41/41 bidirectional pass on a prior run. The scheduler, SAVYX_Health, fired on command each time. LastTaskResult returned 0. The feed record landed at the expected timestamp.

The scoreboard said healthy. The queue disagreed.

Seven open items sat unprocessed in the live feed. One closed item was correctly excluded from the count — the 8-item raw total became 7 genuinely unresolved. The unattended signal, jk_wait_reprobe, was written to the feed and confirmed arriving at the reader. The reader did not resolve the items.

That is not a test failure. It is a coverage failure — and the two are different problems with different fixes.

Why the gap exists: tests confirm paths, not outcomes

The inference-training asymmetry Apple’s Machine Learning Research team exploited in their August 2026 DLR-Lock paper offers a useful analogy here. Their point is that a model can perform identically on inference while being deliberately crippled for training — the forward pass looks fine; the backward pass is the trap. The asymmetry is invisible to any check that only measures forward behavior.

Queue processing has the same structure. The scheduler executes — that is the forward pass. The downstream consumer acknowledges, resolves, and clears the item — that is the backward pass. A test that only measures LastTaskResult=0 and confirms a feed record exists is checking the forward pass exclusively.

Seven items unresolved at 20:05:12 on 2026-07-30, despite a scheduler that provably ran at 20:04:20 with return code 0, is the live proof that the backward pass was not in scope.

The three conditions that hide this class of bug

Condition one: the trigger fires but the consumer is stateful. A zero return code confirms the task launched and exited cleanly. It says nothing about whether the process at the other end of the feed was in a state to consume. A consumer that is paused, rate-limited, or waiting on a dependency will silently let items accumulate.

Condition two: arrival confirmation is not acknowledgment. The measured data shows the feed record arrived — the savyx_feed unconfirmed-items count updated. Arrival and acknowledgment are different events. A delivery receipt confirms the envelope was accepted by the mailbox; it does not confirm anyone read it or acted on it. Seven items can arrive and seven items can sit.

Condition three: the test environment does not replicate live consumer state. Self-tests run clean: the consumer is fresh, the queue is empty, the clock is cooperative. Live systems accumulate state. The 2026-07-30 run that produced a feed record timestamped 06:30:18 — logged internally as a “dead output” event — illustrates this: the scheduler fired, the record landed, and the output was still classified as dead on arrival by the reader. The scheduler did not know.

What a 10/10 score actually certifies

It certifies that the designed test cases all completed their designed paths. That is genuinely useful — it eliminates an entire class of wiring and configuration errors. But it does not certify:

• That the consumer processed items in the queue before new ones arrived
• That items processed in test mode behave identically under live consumer state
• That the acknowledgment signal propagated back through the pipeline

This is not a criticism of the test design; it is a statement about what self-tests structurally can and cannot see. The same limitation applies at much larger scale: the UK’s Isambard-AI supercomputer, which went live at the University of Bristol with 21 exaflops of AI performance across 5,448 NVIDIA GH200 Grace Hopper Superchips, required collaborative engineering teams for physical installation and validation — even systems with known-good specifications require live confirmation that the system behaves as integrated. A benchmark score and a live-system health check are separate instruments.

The principle: measure the queue, not the trigger

The correct health metric for a queue-based system is queue depth over time, not task return codes.

A trigger that fires cleanly and leaves items unresolved is not healthier than a trigger that fails noisily — it is harder to diagnose, because it produces false confidence. The 7-item residual is more useful signal than the 10/10 pass, precisely because it is honest about the system’s actual state.

Google Research’s SymptomAI project, published July 2026, makes a structurally identical observation about AI diagnostic agents: existing evaluations performed well on curated, synthetic vignettes but faced uncertainty about real-world performance because everyday patients present with incomplete information and natural conversation complexity. The curated test passed; the live population revealed the gap. The research team identified this as a “key gap” that led to a national-scale in-situ study — measuring the system against real conditions, not modeled ones.

The pattern is consistent across domains: a test suite models the world as designed; live systems are the world as it is.

The decision framework

The fix is not to distrust the self-test — it is to scope it correctly and add the missing backward-pass measurement. For the SAVYX pipeline specifically: the queue depth at the end of each scheduler run is the health number. Seven items remaining is a threshold breach. Zero remaining is a pass.

The 10/10 score is true. The 7 unresolved items are also true. Systems thinking requires holding both.

Looking for more on ai & digital income? Visit SAVYX

Frequently Asked Questions

Can a system pass all self-tests and still have unprocessed queue items?

Yes — and the SAVYX measured data shows exactly this. A self-test confirms execution paths completed; it does not confirm the downstream consumer acknowledged and resolved each item. Both conditions can be true simultaneously.

What is the right health metric for a queue-based system?

Queue depth over time, not trigger return codes. A task that fires with LastTaskResult=0 and leaves items unresolved provides false confidence. The residual item count after each scheduler run is the honest health number.

What is the difference between a delivery receipt and an acknowledgment?

A delivery receipt confirms the feed record arrived and was accepted. An acknowledgment confirms the consumer read, processed, and cleared the item. Queue systems can show delivery confirmed while acknowledgment is absent — that gap is where items accumulate.

Why do self-tests miss live-state failures?

Test environments run with clean consumer state, empty queues, and cooperative timing. Live systems accumulate state between runs. A consumer that is paused, rate-limited, or waiting on a dependency will silently let items sit — a condition a fresh-start self-test cannot replicate.

How should teams interpret a perfect self-test score alongside unresolved queue items?

Treat them as measurements of different things, not contradictions. The self-test certifies that designed paths execute correctly. The queue depth certifies that the live end-to-end pipeline is clearing work. Both instruments are needed; neither substitutes for the other.

Disclosure: As an affiliate, this post may earn a commission from qualifying purchases at no extra cost to you.