Escaping the Local Maxima in Product Optimization

What it means: A local maxima is the point at which iterative optimization has extracted most of the value available from the current design surface - and further experiments produce diminishing returns without changing what is fundamentally being tested.

The Setup Everyone Misreads

You have a 30% experiment win rate. That is healthy - most teams land between 20 and 30. Your velocity is high. Stakeholders are happy. The testing culture you built is working.

And your north star metric has not moved in six months.

Most teams read this as a sequencing problem. Run more tests. Improve hypothesis quality. Go deeper on segmentation. The win rate will compound into north star movement eventually.

It will not. Not because the experiments are wrong, but because the surface you are testing cannot deliver the outcome you are optimizing for. Every win is a local improvement inside an architecture that has already priced in its ceiling.

What a Local Maxima Actually Looks Like

It does not arrive as a failed experiment. It arrives as a pattern across winning ones.

Signal one: high win rate, flat north star. Individual experiments win on their primary metric - click rate, form completion, page engagement. The north star does not move. The wins are real. They are also contained. Each one improves a step without changing what the step connects to.

Signal two: shrinking effect sizes. Your first round of checkout experiments moved conversion by 1.8 percentage points. The next round moved it 0.9. The round after that, 0.4. Effect sizes do not naturally shrink because hypotheses get worse. They shrink because the remaining extractable value in a surface gets smaller as you approach its ceiling.

Signal three: wins that do not hold at Day 30. A new onboarding flow wins in the first-week cohort. At Day 30 retention, there is no difference between control and variant. This means the experiment changed behavior at the surface level without touching the underlying driver of retention. The win was real. It was also not the thing that needed to move.

The Comparison Table You Need Before Any Redesign Conversation

Signal	What it means	What it does not mean
High win rate, flat north star	Architecture is the constraint	Experiments are poorly designed
Shrinking effect sizes per quarter	Approaching the ceiling of the surface	Sample sizes are insufficient
Day 30 wins that evaporate	Surface behavior changed, underlying driver did not	The experiment was invalid
Low win rate, flat north star	Hypothesis quality is the problem	You are at a local maxima
High win rate, north star moving	Room still exists - keep testing	Time to redesign
Competitor gains with different interaction model	Fundamental UX assumption may be wrong	You need more testing time
User research surfaces unaddressed jobs	Opportunity gap, not ceiling	Local maxima
Qualitative frustration on tested surfaces	Architecture friction, redesign signal	Edge-case user feedback

Read the table as a diagnostic, not a checklist. One signal does not make the case. The pattern across three or more signals does.

The Booking.com Pattern: Testing Culture Without Architecture Change

Booking.com's experimentation program is among the most documented and sophisticated in consumer internet - widely covered in industry research, conference talks, and HBR case studies. What makes their history instructive for local maxima is a pattern that product teams there have discussed publicly: running high volumes of experiments on a surface while a structural gap persists at a different level.

The mobile-to-desktop conversion gap that consumer travel products face is a well-documented industry pattern. When a product's information architecture is built for one interaction context - larger screens, higher cognitive bandwidth, more deliberate navigation - and then inherited by a mobile experience without fundamental rethinking, iterative testing tends to optimize within the inherited structure rather than challenge it. The trust signals, the information hierarchy, the comparison affordances: each can be individually improved while the overall flow remains mismatched to how mobile users actually navigate.

This is what local maxima looks like in a mature testing program. The program is functioning. The results are honest. The wins are real on the surfaces being tested. And the structural constraint sits one level above where the experiments are running.

The Booking.com story is worth naming not because the internal diagnostics are public - they are not - but because their public investment in experimentation infrastructure makes the pattern legible: teams that run more tests than almost anyone else can still find themselves optimizing a surface whose architecture is the actual constraint.

The Judgment Turn

Here is the position that most retrospectives will not put in writing.

A product manager who has run 50 experiments per quarter for two years and has never proposed a redesign has not built a testing culture. They have built a button-color optimization program with a sophisticated measurement wrapper around it.

Testing culture is not velocity. It is the ability to use evidence to change your mind about what to test next - including the decision that what needs to change is too large to test incrementally. The teams that treat "we should redesign this" as a failure of the testing program have inverted the logic. Redesign is the conclusion the testing program earns when it has extracted what iteration can extract.

The uncomfortable cost is this: the decision to redesign is not a product decision. It is a career decision disguised as one. Proposing a redesign means telling the people who funded your last 18 months of work that the wins were real but insufficient. It means accepting a 6-to-12 month period where your experiment velocity drops, your win rate goes to zero while the new surface stabilizes, and every quarter-end review will ask why the metrics look worse than they did before you started.

Most PMs do not make that call. They run one more experiment instead.

How to Tell Leadership the Last 18 Months Were Real but Insufficient

Do not say the wins were wasted. They were not. They mapped the ceiling.

The correct frame is this: the experiment program did exactly what it was supposed to do. It extracted the available value from the current architecture. The evidence that the architecture is the constraint is the same evidence that makes the redesign defensible - you have already ruled out every hypothesis that did not require rearchitecting.

Present the pattern, not the pivot. Show the declining effect size curve. Show the north star flatness against the individual metric wins. Show where Day 30 data diverges from Day 7 data. Let the data make the argument that the current surface has been optimized to its ceiling. Then position the redesign not as abandoning the work but as graduating from it.

The ask is not "let us start over." The ask is "the wins we have earned tell us what the new architecture needs to do. We know more now than we did 18 months ago. Let us use that."

Name the cost clearly. A redesign will create a period of regression on metrics that the current architecture has moved. That is not a risk to minimize - it is a tradeoff to own. Leadership respects the PM who names the cost before being asked about it more than the one who discovers it mid-quarter.

What Stays, What Goes

Not everything redesigns. The signals in your experiment history tell you what was working at the component level and what needs to be rethought at the flow level.

Wins that held at Day 30 are signal about component-level behavior that the new architecture should preserve. Wins that evaporated at Day 30 are signal about what the current surface trained users to do that did not serve them. The redesign inherits the former and discards the latter.

This is the part that most redesign proposals skip. They treat the experiment history as the thing being replaced rather than the research that informs what replaces it. That is why redesign proposals often feel like sunk-cost denial to the people who funded the testing. Frame it the other way: the experiments were the most honest user research you could run at scale. The redesign is the product of what they found.

Key Takeaways

A high experiment win rate alongside a flat north star is a local maxima signal, not a program health indicator.
Shrinking effect sizes per quarter are the clearest quantitative evidence that a surface is approaching its optimization ceiling.
Day 30 data that diverges from Day 7 data on a winning experiment tells you the surface behavior changed but the underlying driver did not.
The decision to redesign is a career decision disguised as a product decision - own the cost before leadership asks about it.
Experiment wins are not made invalid by a redesign proposal; they are the evidence base that makes the redesign defensible.

A local maxima arrives not as a failed experiment but a pattern across winning ones.

Escaping the Local Maxima in Product Optimization