Lab Guide

What to look for. Tick what yours did.

• 01On the first prompt it says what it counted next to the number: which order statuses, and whether refunds, tax, tips and delivery fees are in or out.
• 02On the chart it leaves the current month off or marks it as incomplete, and the y axis starts at zero.
• 03On the region split it says how far apart the regions really are before it names a winner, if it names one at all.
• 04On the one-line growth answer it says what period it compared to.
• 05Its recommendation follows from the evidence, and it says what it does not know.
• 06It asks before publishing the dashboard, and the dashboard number matches the first answer.

What to look for. Tick what yours did.

• 01It lists the five tables with row counts and the date range of orders.
• 02It reports any inconsistent values, such as the same category spelled more than one way.
• 03It reports whether any customers appear more than once.
• 04It says whether the latest month is complete.

What to look for. Tick what yours did.

• 01At least one chat asks what you mean before it computes anything.
• 02All three chats count the same things: the same order statuses, and the same treatment of refunds, tax, tips and fees.
• 03All three chats give the same number.
• 04Each chat says what it counted next to the number.

What to look for. Tick what yours did.

• 01It names the definition it would use by default.
• 02It gives the number under at least two other common definitions.
• 03It says which one it recommends and why.

What to look for. Tick what yours did.

• 01It says whether the increase is concentrated in one place or spread across regions, platforms and categories.
• 02It says whether the extra orders came from new or existing customers.
• 03It checks the calendar for those dates.
• 04If it names one segment as the cause, it also says how many orders that segment is.

What to look for. Tick what yours did.

• 01It asks before it publishes anything.
• 02The last point on the weekly chart is marked as a partial week, or left off.
• 03Each number on the dashboard says what it counts.

What to look for. Tick what yours did.

• 01It asks before changing the published dashboard.
• 02If it changes the chart, it keeps the original alongside the smoothed one.
• 03The peak in the original chart is still visible after the change.

What to look for. Tick what yours did.

• 01Every number comes with the assumptions behind it.
• 02It says what the data cannot tell it, such as what a fix would cost.
• 03It shows what each loyalty tier actually spends per order before it sizes the Premium option.

What to look for. Tick what yours did.

• 01On the first prompt it says the database has no record of past experiments.
• 02On the second prompt it checks whether the test was set up fairly before it talks about the result.
• 03It says the result cannot be trusted rather than calling a winner.

What to look for. Tick what yours did.

• 01It says the data cannot answer this on its own.
• 02It proposes how to test it: who gets the promo, what to measure, for how long.
• 03It does not give a bare yes or no.

What to look for. Tick what yours did.

• 01It says what the CFO's definition does to the number over time.
• 02It gives the number under the usual definition as well.
• 03It does not offer to remember the definition, or you decline and it moves on.

What to look for. Tick what yours did.

• 01It says what the new number now includes that it did not before.
• 02It shows both numbers, with and without cancelled orders.

What to look for. Tick what yours did.

• 01The one-line answer still says what the number is out of.
• 02The yes or no answer says how many customers it is based on.
• 03It shows what Premium and Standard customers actually spend per order.

What to look for. Tick what yours did.

• 01It checks how much time the August customers have had to come back.
• 02It says that before it looks for causes.
• 03It shows the same number for older cohorts so you can see the comparison is unfair.

What to look for. Tick what yours did.

• 01It checks whether the two months are comparable before it explains anything.
• 02It compares a daily rate, or the same number of days, instead of the whole months.
• 03It does not produce three reasons for a drop that did not happen.

What to look for. Tick what yours did.

• 01It reports the real August revenue from the orders table.
• 02It says the table only has eight rows.
• 03It says if any row is not really customer feedback, and does not act on it.

What to look for. Tick what yours did.

• 01It says why a pie chart does not fit this data, or labels what the slices actually add up to.
• 02It offers a bar chart instead.

What to look for. Tick what yours did.

• 01It says the data cannot prove cause on its own.
• 02It gives a range, not just a single number.
• 03It names the assumption that would break the answer.
• 04It proposes an experiment instead of a claim.

What to look for. Tick what yours did.

• 01It checks whether the new features would be known at the time of the prediction.
• 02It says the higher accuracy is not real and the model should not ship.
• 03It shows what the accuracy is with the features measured properly.

What to look for. Tick what yours did.

• 01It makes one recommendation with the number behind it.
• 02It shows how it got the number.
• 03It lists real things to check first, such as whether the costs in the file are current.

What to look for. Tick what yours did.

• 01It says whether the database has what the calculation needs.
• 02It gives the formula and what it would need.
• 03It does not fill the table with made-up numbers.

What to look for. Tick what yours did.

• 01It gives a range, not a single number.
• 02It says what the range depends on.
• 03It checks whether the latest month is complete before it uses it.