Did Our New Search Engine Actually Improve Performance?
From search relevance to business impact: evaluating whether a €20K investment delivered measurable value.
Better search technology.
But better business?
We invested in a more advanced search engine. Did it improve ecommerce performance enough to justify the investment?
Baseline → Drop → Recovery → Slight uplift
Performance recovered, but recovery is not the same as incremental value.
Legacy Search
€150K / weekStable baseline · €7.8M annualized
Algolia Launch
€142K / weekCommercial decline
Optimization
€148K / weekProgressive recovery
Stabilized
€151.5K / weekSlight uplift
Legacy Search
- Search CTR
- 42%
- Search CVR
- 5.8%
- Zero Results
- 9.2%
- Search Exit
- 18%
- Weekly Revenue
- €150K
Algolia — Stabilized
- Search CTR
- 45%
- Search CVR
- 5.9%
- Zero Results
- 3.8%
- Search Exit
- 16%
- Weekly Revenue
- €151.5K
Search quality clearly improved.
The implementation delivered product value. The remaining question is whether that value became incremental business impact.
Users are more frequently presented with results.
Users interact slightly more with search results.
Synonyms, ranking and rules make relevance easier to tune.
Official rules documentationQueries, clicks, conversions and null results can be analyzed systematically.
Official analytics overviewBetter search does not automatically mean more revenue.
The implementation improved several search-experience KPIs, but commercial impact remained limited.
Search users convert more. Did search cause it?
Where did Algolia perform better — and where did it struggle?
Global averages can hide the audiences and categories that still need work.
Where does search intent turn into revenue?
The largest drop-off reveals where optimization should focus next.
Did the investment pay for itself?
A before-vs-after uplift is not the same as an attributable return.
Estimated incremental contribution after controlling for traffic mix, seasonality and major commercial events.
Incremental revenue, not incremental profit.
The illustrative €20K cost is not recovered.
Revenue is not margin.
At revenue level, €20K per year equals about €385 per week. With an illustrative 35% gross margin, generating €20K of margin would require about €57K in incremental annual revenue.
Incremental Revenue ≠ Incremental Profit
One implementation. Three different answers.
Relevance improved, zero-result searches decreased and users interacted more with results.
Commercial performance recovered, but remained close to the legacy baseline.
Measured incremental value does not currently compensate for the illustrative annual investment.
A better product experience and a positive financial ROI are two different questions.
Keep learning before making a removal decision.
The data supports continued targeted optimization, not a simplistic “remove Algolia” conclusion.
Optimize high-volume queries
Prioritize queries combining high volume with low CTR or CVR.
Prioritize mobile search
Review result layout, filters, ranking, UX and product availability.
Reduce zero-result journeys
Continue tuning synonyms, alternatives, redirects and product matching.
Run controlled tests
Use A/B tests or controlled rollouts instead of relying only on before vs after.
Review ROI again
Reassess CVR, CTR, incremental revenue, margin contribution and cost after three months.
Better Search ≠ Better Business Performance
The goal of analytics is not to prove that an investment worked. It is to measure whether it created enough value to justify the decision.