Problem
A scheduling agent can say “booked” when no appointment was saved. The person relying on it is left with a promise the system never kept.
The design
I specified a synthetic lab with the agent, backend state and independent observer kept separate. I defined the requirements and acceptance checks and directed implementation. The observer compares the agent’s claim with what the backend saved; planted contradictions test whether the observer catches a mismatch.
What I would gate a release on
A success verdict needs matching saved state. A missing or mismatched appointment should fail the check. Ambiguous outcomes need review rather than a confident success label.
Checking the checker
The observer tests start with a passing evidence bundle and change what it contains. One test adds a booking claim for an unbooked slot and expects an unsupported-claim failure. Another removes the saved booking and expects the state comparison to fail. A missing database snapshot must return an inconclusive result rather than pass.
Read the pinned observer tests and the stored deterministic report. These are source and recorded results inspected for this case study, not a fresh run or a live-model benchmark.
What this establishes
The lab demonstrates a repeatable way to test false-success detection and the checker’s failure modes. It uses fictional data and a deterministic default agent. These checks establish evaluation-harness behavior; they do not establish live-model accuracy, real patient outcomes, clinical readiness or production readiness.