I Quit Three Jobs Over Fake Data Work. Now I Ask Two Questions First.
Metrics that measure nothing and change nothing burn the reason senior ICs chose the work. The two questions that stop it, and the three kinds of work that fail them.
I quit my last three jobs over fake data work. The management chain burned my team’s time pulling metrics that measured nothing and changed nothing. We told them, they nodded, then did it again.
When this starts to annoy me, my wife usually asks something like, “Can’t you just clock in, clock out, and collect the paycheck?”
Why senior ICs can’t just clock in
I’ve talked to a lot of senior ICs about how they feel about this. Earlier in our careers, we can. Almost every problem feels new. But after you do something long enough, the work gets easier and the fake stuff gets harder to ignore.
Jobs take a lot of our waking hours. We like getting paid. But a paycheck is not a life or a purpose. There has to be something in the work itself.
When you choose the senior IC path, you often give up money you could make in management. You stay deep in the work while leading without formal authority. Nobody is forced to follow you the way they follow a manager. You have to become someone they want to follow on their own.
So why do it? Because we love the work. We want difficult problems that matter. We want to make things that would not have happened without us. When a company replaces that with metrics that mean nothing and analysis nobody will use, it takes away the reason many of us chose the path.
The two questions
Before I agree to another analysis or pull another number, I ask two questions.
Does it measure what actually matters? By “actually matters” I mean the number must tell us something about the user or the business in the real world, not just that it moved.
Will anyone actually use it to make a decision? By “decision” I mean the answer changes something. We launch, stop, prioritize, investigate, or decide not to act.
Three kinds of work that fail them
Orphaned analysis measures something that matters, but nobody will act on it. “It’s interesting” is not enough. Do not pull a number nobody intends to use.
Data theater measures nothing that matters and decides nothing either. Someone wants a number for a meeting so they can look data-driven to their boss. Do not create their prop.
Misleading numbers are the ones someone will decide from, but they don’t measure what matters. These are dangerous. They lead companies into very bad decisions that appear to be backed by data. The data is not broken. The metric is.
If it measures what matters and someone will use it for a real decision, only then does it earn our work. Decide what different results would cause you to do before you pull it. Measure it honestly. Then come back and see what happened.
If you are the leader pushing this work
You are not just wasting company resources. You are replacing the work your people chose the IC path to do with work they know does not matter. Do that long enough and they will find somewhere else to work on problems that do. Like me.
So yes, I could clock in, clock out, and collect the check. I just don’t want to spend that much of my life doing work I know does not matter.
Do you?
10+ years in product data science, causal inference and AI evaluation at Stripe, Nextdoor and Ontra.