How E-Commerce Search Starts Closing the Gap to General Web Search

Once, when the internet wasn’t all that professional, search engines already were so good that they could give thousands of results in the blink of an eye for many searches. I remember reading articles entitled How to get the search hits you want, and 10,000 documents match your query. Now what? So while some searches yielded nothing, others had too many hits, or rather too little ordering of them.

Today, this is no longer so, although searching can be a challenge to the internet newcomer, and must be learned like any other skill. It is only natural that Google have millions of hits for boats or berries, or even boats and berries, but we all know that a good internaut is simply one who knows how to do precise searches and who can navigate the results list. And frankly − when did you last turn the Google search results page? If you’re not looking for the ultimate alchemy formula, chances are you hardly ever need to go past page 1. Search engines are that good.

E-commerce searches, however, seem to lag behind. An astonishing number of site internal searches on e-commerce web sites still return «no hits» for common queries, often despite coming from a product page showing what could have been a relevant hit to that precise search.

Situations like these are still too common in online retail. This screenshot is from 31st May, 2021.

Astonishing this is, because the stakes are high: in 2019, e-commerce in the UK had an 11% growth, surpassed £105 billon, and reached around 22% of the total retail. And this was before Covid! By 2024, e-commerce is expected to account for one third of the total retail. Other countries have other numbers, but the general direction is the same for all.

This infographic is from emarketer.com, as are the numbers in the paragraph above.

Now imagine you go to your local store on your way home from work to get a light bulb, but there are none in the shop, and the nearest place to get one, is 30 minutes away. Do you make a detour or do you post­pone the light bulb and go home for dinner? This out-of-stock problem is completely different in an e-commerce context. On the internet, you would simply switch to another lightbulb.com site and order from there without even thinking about the fact that you are leaving one shop and entering another. So merely accepting that your shop’s search yield zero hits for certain queries is like keeping staff that refuses to help customers in a shop and tell them to browse the shelves by themselves to see if a given item is in stock. Or even hide the goods behind the counter without telling.

There is attention to this problem in e-commerce, hence the title of this article. Web shops know they must help their customers on every push to the search button. Search results and search itself is becoming more and more frequent subjects in website optimisation, and the practical knowledge about handling and presenting searches in a good way, that the major internet search engines have had for years, is finally making it also to smaller, site-specific e-commerse site searches.

Here’s a something that we experienced in my own company, Sannsyn, where we work with personalisation and search optimization. In January, February and March this year, we worked with a wholesale electric components and tools company to improve their website search. It had a regular Solr installation in the back-end, but populated by different suppliers, so not too coherent internally. And since we then couldn’t streamline the data much, we adjusted some of the search parameters, and, most importantly, inspected the actual use of the search in order to, among other things, hunt down and get rid of the «no hits» searches.

So how did we go about it? Well, surveillance of the search gave us something to work on. Here are two small examples from areas that caused products to be absent from the search results:

• There were too few hits for «downlights». It turned out that while some suppliers had the word «downlights» in their product description, others did not. They didn’t even have other words for it. Solution: add «downlights» to a stopword list specific to the description field. We kept the word searchable in a category field, so that the search now embraces all products in the relevant category, regardless of this word’s appearence (or lack thereof) in the description field.
• Luminairies were also underrepresented. Maybe because of the extensive use of compound words in Norwegian (so things like «ceiling luminairies» are written as one, single word). The Solr solution is to define certain words as targets for compound word searching. We couldn’t use that, however, because searching for «luminairies» would then also bring up «luminaire rails» (which are rails, not luminairies). Or with an English example: if one defines «ball» as a compound word in order to include «basketball», one would also get «eyeballs» and «ballerinas». So then what? We defined synonyms. A lot of synonyms.

Missing, faulty or too short product descriptions (the downlight problem) gave us some work to define and create new categories in the system, and the synonyms to fix the compound word problem induced quite a bit of manual typing. (Luckily, we have tools to automate some of this, and I might write about that later.) But the important takeaway from these cases, is that examining the real life use of your e-commerce search is not only worthwhile, but an important step towards making your search setup and web presence behave more like the attentive shop staff from the old days.

Our efforts at the electric components and tools company had pretty convincing results. From January to March, the average shopping basket gained around 50% in total value, and by the 1st of June, it was more than twice the size of the January average! And this didn’t cost us five engineers for a complete year; we only had one man on the task for three months. So benefiting from Google’s old insights doesn’t have to cost a fortune, but certainly will pay off!