When a number goes into a deck, someone will ask where it came from. With a human analyst, you go ask them. With an AI analyst, the answer has to already be written down. The session that produced it is gone. The trace is that written record. Which tables were read, what SQL ran, and which metric contract applied. What was filtered out and why. What the analyst assumed when the data was ambiguous.
The test is simple. Another analyst, with the trace and access to the same data, should get the same number. If they cannot, the number cannot be checked, and it should not go in the deck.
This matters more with AI because the output looks finished. The chart is tidy and the paragraph is confident. Nothing on the page tells you that "revenue" excluded refunds this time and included them last time. The trace is where that shows up. Shane's correction system post came out of checking every number an AI analyst produced against his own SQL. Some of the wrong numbers were his. He only found that out because both sides had a trace.
In practice the trace is part of the output. Every number carries its source: the query, the definition, the row count, the caveats. In the course we call these receipts. The rule in week 3 is blunt. A number with a source, or it does not go in the deck.
A concrete example.The trace under it names the MEMBERSHIPS table and the membership_retention contract.Membership grain, point-in-time snapshot. You can check every word of that against the warehouse.
In the courses
In Agentic Analytics, week 3 (AI evals for AI analytics), Tuesday works without an answer key. Reliability, receipts behind every number, and four independent methods on one question. AI Analytics for Everyone, week 2 (Set up your AI analytical toolkit) has you run a full analysis and publish it. Week 5 (Storytelling and influence) has you stress-test your own numbers before an executive readout.