Algolia on Harlan D. Harris

11 Algolia A/B Testing Gotchas, Tips, and Lessons!!

harlan@harris.name (Harlan Harris) — Mon, 09 Feb 2026 00:00:00 +0000

I've supported A/B testing of Algolia search systems at three companies now, and have dived deep into A/B testing generally as well as specifically for search. The Algolia documentation on search A/B testing is technically adequate for getting started, and the dashboard has improved, but there are still many ways that you can go wrong when A/B testing Algolia search results. In the style of a 2014-era Buzzfeed listicle, here are 11 Algolia A/B Testing Gotchas, Tips, and Lessons!! All of the horrible illustrations are generated by AI, the rest are from earlier posts.

Post Query Refinement Suggestions in Search UX, and an Algolia Demo App

harlan@harris.name (Harlan Harris) — Fri, 23 Jan 2026 00:00:00 +0000

What are Post-Query Refinement Suggestions?

One of my favorite search UX patterns is post-query refinement suggestions — buttons that appear between the search box and results, adjusting the query in various ways. See, for instance, these suggestions on Etsy, which recommend that I filter by shipping speed, seller type, cost range, style, etc.

Filter Suggestions on Etsy

PQRS is a mouthful of an abbreviation, but most of what I've seen as alternatives either describes the UI element ("refinement pills") rather than the function, or is excessively vague ("guided search"). So PQRSs it is, sorry.

Vibe-Coding the Missing Algolia Comparison Dashboard

harlan@harris.name (Harlan Harris) — Tue, 13 Jan 2026 00:00:00 +0000

Algolia is a tremendously powerful search platform, especially for e-commerce. But like most SaaS tools, the management dashboard doesn't do everything you need it to do. In particular, it's hard to compare search result rankings across variations in indexes -- a super common pattern when you're testing and iterating on index configurations before A/B testing, implementing algorithm and UI proof-of-concepts, or refining search relevance.

At several places I've worked, I've built (or spec'ed out) comparison apps that allow you to run the same query in parallel and easily see and compare the results, for internal testing. Most recently, I vibe-coded an app that has worked very well. I can't share the code itself, so I've written a spec for a generic, equivalent app that is free for anybody to use and customize.