A product manager with eight years in fintech and SaaS posted on r/analytics in March that they had set up Mixpanel, Amplitude and Google Analytics many times, and the pattern was always the same. "Month 2 -> 47 dashboards, 12 saved reports, nobody looks at any of them." By month six someone is asking in Slack for the numbers because nobody trusts the dashboards. Their fix was a smarter tool that tells the team what to do.
One reply went the other way: "The problem would more or less be solved if PMs and marketers had better analytics skills." We think the reply is closer. A tool that tells you what to do still needs someone who knows what decision is on the table and can tell whether the answer is right.
Before AI, a PM who wanted their own numbers had to learn SQL and find the right tables, or wait for an analyst. The query was the slow part. Now the slow part is deciding what counts and whether the answer is right. A PM is often well placed to do that, because they know the goal and the decision.
Where it goes wrong for a PM
A vague question gets a precise number. On September 18, 2026 we asked four fresh ChatGPT chats, memory off, how many active customers a company had, on the same file. They answered 99, 99, 2,268 and 4,633. The chats used three definitions of active between them, and none asked which one we meant. A PM who asks once and pastes the number into a launch review has picked a definition without knowing it. Why AI gives different answers covers the causes.
Our worked PM answer on how to answer "How do you use AI?" in a data interview has the fix. Ask the AI to state its definition and show its SQL before you read the number. Check any number you plan to repeat against a dashboard the team already trusts. If the two disagree, take both to the analyst and work out which definition you mean.
The second trap is picking the target after seeing the data. Write down what a good launch looks like, and what result would make you roll it back, before anyone pulls a number. If the AI helps you choose the target afterward, the target will fit the numbers.
The third is reading a pattern as a cause. In our ChatGPT tests the agent was careful when the question asked for one: told to find what causes churn, it built a model, refused the causal claim, and proposed an experiment instead. The risk sits with the reader. Ask which users retain better, hear "the ones who used the new feature", and it is easy to take that as proof the feature works, when they may have been your most engaged users to begin with.
What to leave to the data team
Company metrics. If "active customer" means something on the board deck, a PM should not redefine it in a side analysis. Hai Guan described in a workshop Q&A how his team certifies a metric: everyone who uses it gets in a room, they name one owner, they agree one definition, and if one team needs a different version it gets a different name. That is data team work, and it is the work that makes a PM's own analysis safe, because the AI then has a definition to read. We call that written definition a metric contract.
Anything that claims a cause when money depends on it, like whether the new onboarding flow reduced churn, needs an experiment, or a careful analysis by someone trained to do one without it. And anything that becomes the official number should go through whoever owns it.
How to start
Take a question you sent to the data team last month. Before you open any tool, write four lines: the goal it serves, the decision it feeds, the metric that would move that decision, and what you expect to see. Then ask an AI analyst, ask it to state its definition first, and ask it twice in fresh sessions. If the two answers agree and match a number the team already trusts, you have your answer. If they disagree, you have a sharper question for your analyst than the one you started with.
When the answer is in, the next job is getting someone to act on it. The three-slide readout and the follow-up are on getting people to act on your analysis.