Introducing TellusR for Solr search enhancements
Part I − Implementing features that Solr lacks
According to a report from Forrester Research, 43 % of all visits in webshops start by using the search. I have previously discussed how webshops are improving their search functions, but there still is a gap between the performance of site internal searches in e-commerce and that of search engines that are not part of a webshop.
In Sannsyn, we work a lot with this. We have customers in the public sector and in web commerce, retailers and wholesale businesses, and they all need help with their search. Most of our assignments concern Apache Solr, which is one of the most popular search engines in the world. Solr is extremely fast and highly customizable. In fact, its very high number of customizable features and its enormous amount of toggleable buttons and levers are in themselves part of the reason people need help.
Solr has a visual dashboard where users can inspect various parts of the search engine: consult key figures, try out searches, read error descriptions etc. But all of this is for the IT technicians, who check to see that the installation is doing all right, that it is not using too much of the server resources, etc. The webshop manager, on the other hand, would rather know what people search for this month, which searches result in the highest share of convertions, how many searches simply fail, etc. But this is not visualized through the Solr web interface.
TellusR to the rescue
So we set out to create a dashboard that could make Solr useful for both techies and webshop managers: show graphs for conversions per search, alert about searches that yield zero hits, give information about what search terms are trending etc. We also wanted to enable non-tech staff to add synonyms to a Solr search or to elevate certain search terms or use some of the other features of Solr that are otherwise only available to those who know the Solr vocabulary and master the art of XML editing.
The result is TellusR. We call it a Solr Refiner.
The TellusR Solr Refiner consists of three parts. There is a graphical user interface (TellusR/Central); a component doing natural language processing (TellusR/NLP); and finally a Solr integrator, which brings information back and forth between your Solr installation and the two other components.
I shall go more into detail about the different features of TellusR in several articles. If you’d like to see and try it right away, TellusR is freely available on the TellusR.com website. In the meanwhile, here are a few of the technical highlights:
- Most of the code is Kotlin, which we found to be the easiest way to write safe and efficient code that would interoperate easily with the Solr kernel
- Although normal Solr plugins are simple jar files inserted in a given folder, TellusR is almost an ecosystem of its own, so we use docker to be able to supply all the functions without interfering with other setups on your server
- We improve the Solr search itself using word vectors built with neural networks
- We improve the Solr autocompleter in your search box using n-grams to analyze input words, which gives a better fault-tolerance
- We embed an A/B-tester within TellusR, so you can try out different configurations for your search and know what works by looking at the numbers instead of just trusting your gut feeling
- We surveil query time in searches, so you’ll see any slowness right away
- We register all query activity, so you can find out if you’re making the big zero-hit mistake (and if so, remedy it)
The TellusR Solr Refiner has become a rich collection of improvements and enhancements, so please try it out and tell us what you think!
