No tooling decision is going to fix a schema problem.
The Three Levels of Product Analytics Maturity
A diagnostic map of where your team's analytics practice actually sits - from basic event logging to behavioral cohorts to predictive modeling - and what it costs to move up. Most companies claim Level 2 but operate at Level 1 with better dashboards.
A diagnostic map of where your team's analytics practice actually sits - from basic event logging to behavioral cohorts to predictive modeling - and what it costs to move up.
The scenario that reveals everything
Your retention number drops 4 points month over month. You open your analytics dashboard. It shows you the drop. It does not show you which users churned, which actions they did or did not take in the 14 days before they left, or whether this cohort was acquired from a different channel than last month's retained users.
You file a data request. It comes back in five business days. By then, you have shipped the next sprint.
That gap - between what you can see and what you can actually answer - is the entire story of analytics maturity. The dashboards are not the problem. The question is what happens when you ask a question that was not already on the dashboard.
flowchart LR
L1["Level 1\nDescriptive\nWhat happened"] --> L2["Level 2\nDiagnostic\nWho did it"]
L2 --> L3["Level 3\nPredictive\nWho will churn"]
L1 --> T1["Tools\nGA4 · Segment · basic dashboards"]
L1 --> C1["Team\nShared analyst"]
L2 --> T2["Tools\nAmplitude · Mixpanel · data warehouse"]
L2 --> C2["Team\nDedicated data engineer"]
L3 --> T3["Tools\nFeature store · ML pipeline · scoring API"]
L3 --> C3["Team\nData science team"]Level 1: What happened
Level 1 analytics tells you events occurred. Page views, sessions, button clicks, sign-ups. Aggregate numbers over time. You can see that something went up or down. You cannot see who made it go up or down, or why.
Most startups spend their first two years here. This is not a failure. Volume is low, qualitative signal from user interviews is rich, and the cost of building a deeper data infrastructure exceeds the signal it would generate. Level 1 is appropriate when you have fewer than 10,000 monthly active users and your biggest decisions still hinge on whether the product works at all.
The danger of Level 1 is not that it gives you no data. It is that it gives you enough data to feel informed while making decisions that the data cannot actually support. A dashboard showing 40,000 page views tells you nothing about whether the people who viewed those pages ever came back.
What Level 1 infrastructure looks like:
- A tag management system dropping events into Google Analytics or a basic Segment connection
- Dashboards built on aggregated counts and rates
- Retention defined as a single aggregate number, not a cohort curve
- Data requests routed through a single analyst or the engineering team
Level 2: Who did it and when
Level 2 analytics gives you behavioral identity over time. You can answer questions like: of the users who signed up in February and completed onboarding, what percentage returned on day 7, day 14, and day 30? You can segment that cohort by acquisition channel, device type, or feature adoption. You can see the sequence of actions that precede churn versus the sequence that precedes expansion.
This is not just richer data. It is a structurally different question. Level 1 asks what happened to all users. Level 2 asks what happened to this specific population of users over a defined window.
Moving from Level 1 to Level 2 requires three things: an event taxonomy with consistent user identity across sessions, a data warehouse with queryable event history, and a person whose full-time job is maintaining both. That last item is the one teams skip. They buy Amplitude or Mixpanel, instrument 200 events across the product, and discover six months later that half the events fire inconsistently, user identity breaks at login, and the cohort analysis they need cannot be run because the schema was designed by three different engineers with three different naming conventions.
The tool did not fail them. They never had a data engineer.
Level 3: Who will churn before they know
Level 3 analytics is predictive. You are no longer asking what happened or who did it. You are computing a probability score for each user: how likely is this person to churn, expand, or convert in the next 30 days, based on behavioral signals in your event history?
Level 3 requires Level 2 as a prerequisite. You cannot train a churn model on inconsistent event data. The model will learn your schema bugs, not your user behavior.
At Level 3, the product team stops waiting for retention to drop and starts acting on the early signals that precede retention drops - reduced session frequency, abandonment of a core workflow, silence after a triggering event. You can build intervention triggers: if a user's engagement score drops below a threshold, fire an in-app message or route them to a success manager.
Very few companies under 500 employees need Level 3. The cost of building and maintaining predictive infrastructure - the data science team, the feature store, the model retraining pipeline - is almost never justified unless churn has a known dollar value per user and the volume of users makes individual intervention financially viable.
The maturity comparison table
| Dimension | Level 1 | Level 2 | Level 3 |
|---|---|---|---|
| Core question answered | What happened? | Who did it and when? | Who will do it next? |
| Minimum team size | 1 analyst shared across teams | 1 dedicated data engineer + 1 analyst | Data engineering team + at least 1 data scientist |
| Infrastructure required | Analytics tag + basic dashboard | Event warehouse + identity resolution + cohort tooling | Feature store + model training pipeline + prediction serving |
| PM autonomy on new questions | Low - almost all new questions need a data request | High - PMs can self-serve cohort cuts | Varies - model outputs are self-serve, model building is not |
| Decision speed | Days to weeks per new question | Hours to days | Real-time for scored outputs, weeks for new model development |
| False confidence risk | High - aggregate metrics hide behavioral variation | Medium - cohort analysis reveals variation but not causation | Low for prediction quality, high for model interpretability |
| Biggest failure mode | Mistaking page view trends for product health | Schema debt preventing reliable cohort cuts | Trusting model scores without understanding what drives them |
The schema debt lesson: tooling is not the bottleneck
Fast-growing consumer marketplaces face a version of this problem at scale. A team needs cohort-level answers about buyer retention and seller activation. The tools are already in place. The data warehouse exists.
What does not exist is a clean event taxonomy. Events have been instrumented by different squads over years with inconsistent naming, inconsistent user identity, and no shared schema standard. A core transaction event fires differently across the Android app, the iOS app, and the web checkout flow. Running a reliable cross-platform cohort on buyer behavior is not just difficult - it is structurally impossible without resolving those inconsistencies first.
The decision is to rebuild the event taxonomy from scratch. Not to adopt a new analytics tool. Not to layer a behavioral analytics platform on top of the existing instrumentation. The existing schema is the bottleneck, and no tooling decision is going to fix a schema problem.
That rebuild takes months and requires dedicated engineering time that cannot ship features during that window. It is the kind of investment that is difficult to justify in a roadmap conversation because the output is not a visible product change - it is the capability to answer questions that were previously unanswerable.
The lesson is not that this is a hard choice. The lesson is that most teams in the same position choose to buy a new tool instead, because it is faster to budget and easier to explain to leadership. And then they wonder why the new tool is not delivering the cohort insights they expected.
The judgment turn
Here is the uncomfortable position: most companies that describe themselves as Level 2 are Level 1 with better dashboards.
The diagnostic is simple. Ask a PM on that team to answer a new retention question - one that was not already on an existing dashboard - without filing a data request. If they cannot do it in under two hours using tools they already have access to, the team is operating at Level 1. The dashboard is a Level 1 artifact. The question-answering infrastructure is the thing that defines the level.
This matters because teams routinely make Level 2 decisions - about which segments to focus on, which onboarding flows to prioritize, which features drive retention - using Level 1 data. They are not asking the wrong questions. They are answering the questions with aggregate numbers that cannot support the specificity the decision requires. The retention drop is real. Whether it is driven by a specific cohort, a specific workflow failure, or a specific acquisition channel - that question requires Level 2 infrastructure to answer with any confidence.
Moving from Level 1 to Level 2 is not a tooling decision. It is a hiring decision. Someone has to own the event taxonomy. Someone has to enforce schema standards across engineering teams. Someone has to maintain user identity through authentication changes, platform migrations, and product redesigns. That is a full-time role. It is a data engineering role. Giving it to an existing analyst as a side project is how you end up with a warehouse full of events that cannot answer the questions the business actually needs.
The question to bring to your next planning conversation is not "which analytics tool should we adopt." It is "do we have a person whose job it is to maintain our event schema?" If the answer is no, you are at Level 1, and the honest response to any Level 2 ambition is that you need to hire before you invest in tooling.
Key takeaways
- Level 1 tells you what happened in aggregate. Level 2 tells you what specific cohorts of users did over time. Level 3 predicts what users will do before they do it.
- The tell for Level 1 masquerading as Level 2: a PM cannot answer a new retention question without filing a data request.
- Moving from Level 1 to Level 2 requires a dedicated data engineer who owns the event taxonomy - not a new analytics tool.
- Schema debt is a structural problem. No tooling decision resolves it. The data foundation has to be correct before Level 2 is possible.
- Level 3 is only worth the investment when churn has a known dollar value per user and the user volume makes intervention financially viable. Most teams should not be there yet.
Related articles
- What Retention Actually Measures (and What It Hides)
- The Data Request Is a Product Failure
Related Articles
A product team can answer questions about daily active users and page views instantly, but a PM needs to file a data request to understand why retention dropped last month. Which level is this team operating at?
Make the call in Reps and see how your reasoning holds up.
Make the call