Translating Vanity Metrics into Actionable KPIs

A number that cannot tell you what to change next is not a metric - it is a mood board.

Who This Is For

You are writing your first product review deck, or you have been writing them for two years and something still feels off. You ship, the numbers go up, leadership nods, and you walk out of the all-hands with no idea whether anything actually got better. This article is for that feeling.

You do not need to know statistics. You need to know how a number earns its place in a decision.

The Room Where the Number Gets Celebrated

Imagine the quarterly all-hands. The slide reads: 14 million total app installs. Up 23% quarter over quarter.

The room applauds. The PM who owns growth smiles. Someone in the back row starts a slide for their own deck that references this number.

No one asks: of those 14 million, how many opened the app more than once? How many completed the action the app exists for? How many are still active 30 days in?

Those questions do not get asked because the number on the slide already did its job. It made the room feel like things are working. That is a vanity metric in its natural habitat - not lying, exactly, but not informing anything either.

What a Vanity Metric Actually Is

A vanity metric answers the question: how much?

An actionable KPI answers: how efficiently, toward what outcome, and compared to what baseline?

The distinction is not about the metric itself - it is about what the metric connects to. Total orders is not inherently useless. Total orders as a north star, without any qualifier for what a "good" order means, is.

The most common vanity metrics in product reviews:

• Total registered users (not active, not retained)
• Total page views (not sessions with intent, not conversion events)
• Feature adoption rate (measured as clicks, not as task completion)
• App store rating average (not segmented by user cohort or experience type)
• Revenue growth percentage (not tied to margin, payback period, or churn rate)

Each of these answers "how much." None of them, by themselves, tell you what to build next.

The Five-Area Comparison

This table covers the five areas where vanity metrics most reliably mislead product teams. For each, the actionable equivalent answers a question that produces a decision.

The pattern in the right column is consistent: every actionable KPI adds a qualifier. It asks "of what population, doing what thing, over what time window." That qualifier is where the judgment lives.

What Swiggy Learned From Growing the Wrong Number

In its early expansion into smaller Indian cities, Swiggy watched total orders climb. The growth narrative was clean - new cities, new users, new volumes. The metric looked like it was validating the playbook.

What the total orders number did not surface: a rising share of those orders ended in a support ticket, a refund request, or a low rating. Delivery times in smaller cities were longer. Restaurant quality varied more. Cold food and late arrivals were inflating the order count while hollowing out the experience.

The pattern that emerged - common to food delivery platforms expanding into new geographies - is a shift from raw order volume toward a quality-adjusted view: orders that did not generate a complaint, refund, or negative rating. The number looks smaller. The signal is sharper.

That shift changes what the growth team optimizes for. Instead of asking "how do we get more orders in this city," the question becomes "how do we get more orders that do not generate a complaint." Those are very different product problems, and they produce very different roadmaps.

The vanity metric was not wrong - orders were going up. But it was a silent agreement that volume was the goal. A quality-adjusted metric breaks that agreement and forces a harder conversation about what growth actually means in that context.

The Metric Definition Meeting

Most metric problems do not start in the data. They start in the meeting where the metric gets defined - or, more often, where it does not get defined at all.

Here is what that meeting usually looks like: a PM proposes a goal for the quarter, someone in leadership suggests a number that sounds ambitious, the number gets written into the objective, and the team ships toward it.

Here is what that meeting rarely includes: a discussion of what the metric does not capture, what gaming it would look like, and what decision the number is actually supposed to inform.

The metric definition meeting is where the real negotiation happens. It is not a technical conversation about instrumentation. It is a political conversation about what your team agrees to be judged on.

Three questions that should be mandatory in that meeting:

1. What behavior does hitting this number require from users? If the answer is vague ("they use the product more"), the metric is almost certainly a vanity metric. If the answer is specific ("they complete a booking without abandoning the payment step"), you have something to build toward.

2. What would a team do that was gaming this metric? If a team can hit the number without improving the underlying experience - by sending more notifications, inflating clicks, or lowering the bar for what counts as an event - then the metric is measuring output, not outcome.

3. Who loses when this number goes down? A metric that does not connect to a user losing something real is a metric that only serves the deck. Retention drops because users leave. Support tickets rise because users are frustrated. Quality orders fall because the product is failing in the field. Every actionable KPI has a user on the other end of it.

The Judgment Turn

Here is the uncomfortable part.

The problem is not that teams track vanity metrics. It is that leadership rewards them.

PMs are not naive. They know the difference between a total install count and a day-30 retention curve. They have read the articles. They know which number tells the real story.

But they also know which number gets the nod in the all-hands. They know which slide produces applause and which one produces hard follow-up questions. And in most organizations, the incentive system is built around the first kind of slide, not the second.

Every vanity metric you report without context is a silent agreement with your leadership that optics matter more than diagnosis. You are not lying. You are participating in a shared fiction that keeps the meeting comfortable and the strategy unexamined.

Converting a vanity metric into an actionable KPI is not a data exercise. It is an act of institutional courage. It means walking into the metric definition meeting and saying: this number can be gamed, here is what gaming it looks like, and here is the qualifier we need to add to make it honest.

That conversation is uncomfortable. Leadership sometimes pushes back - a qualified metric is harder to hit, harder to celebrate, harder to put on a slide. That pushback is not a sign that you are wrong. It is a sign that the metric is doing its job.

The PM who reports a quality-adjusted order metric instead of raw total orders is not being pessimistic. They are refusing the false comfort of a number that sounds good but cannot tell you anything.

Key Takeaways

1. A vanity metric answers "how much." An actionable KPI answers "how efficiently, toward what outcome, for which users."
2. The qualifier - the population, the action, the time window - is where all the judgment lives. Removing it produces a number that cannot inform a decision.
3. Swiggy's expansion into smaller cities illustrates the canonical pattern: raw order volume looked healthy while signals underneath it (refunds, support tickets, low ratings) revealed the opposite. A quality-adjusted metric changed what the team optimized for and what problems ended up on the roadmap.
4. The metric definition meeting is a negotiation, not a technical exercise. The questions that matter: what behavior does hitting this number require, how would a team game it, and who loses when it goes down.
5. Leadership rewarding vanity metrics is a systemic problem, not a data literacy problem. The fix is not a better dashboard - it is a PM willing to name what the number does not capture.

Related Articles

• AARRR Is Not the Metric, It's the Diagnostic
• Designing the Vanity Metric Test for Executive Reporting
• Why Your Roadmap Is Never Actually Data-Driven

The cost of a vanity metric is not that it misleads you once. It is that it trains your organization to stop asking what the number actually means.