AppDirect Blog

News & Updates

The Path to Building a Recommendation Engine for Tech Support

By Alexis / Jul 25, 2014

Recommendation engines have become so popular we now use them every day, from book shopping to picking a movie to online dating. However, the science is still in its infancy and a lot of fine-tuning still needs to be done.

“Customers who viewed this item also viewed…” This now famous recommendation is one of the innovative features that has made Amazon a pioneer of the online marketplace. Today, peer-based recommendations are everywhere. Not only does it help many online services serve up content according to your specific needs, but it has been at the heart of our online lives since the inception of social media. After all, the second most visited website in America is entirely about what your friends like.

At AppHelp, we use peer-based recommendation to help tech support agents find the most relevant solutions on the web.

When a customer’s issue is complex and/or involves multiple vendors, they often have to peek outside their company’s knowledge base and search the open web. To do so, the only tools at their disposal are general-purpose search engines, which are not very efficient for that particular use.

In our research on agents’ web searching behaviour, we found that they try multiple keyword combinations before finding the most relevant document, and that the information they end up using has little connection with their initial search keywords. Typically, they have to conduct a series of search queries by trial and error, making the support call longer and less satisfactory than expected on both ends of the line.


Search engines have limitations in this context because they focus on the content of documents. On the other hand, recommendation engines focus on peer behaviour.

Say you want to play Minecraft on Windows, but the game won’t launch due to an obscure Java configuration issue. If you use a search engine, it will take some time before you make the connection. While there are many solutions for that Java issue on the web, very few mention its occurrence in Minecraft.

With a recommendation engine based on the online behaviour of tech support agents, you’ll find the right document much faster. The trick is to use the right algorithm on a sufficient amount of data.


First, we extract the agents’ query sessions on Google. Only the work-related sessions are extracted— see my other post about how we do this. Then we position them in a multi-dimensional space in the form of clusters containing all the queries that have been entered during the session until the agent found the document they needed (let’s call it Document X).

Each cluster looks something like [query, query, query, query, (…), URL of Document X]. So when someone enters any of these queries, the recommendation engine will quickly associate them with Document X, bypassing all the trial and error that would be necessary with a search engine.

Now, while this approach is very promising, it still poses some challenges. The biggest for us is data scarcity. 


As with all user-generated content on the Internet, you can only get so many people to add data to the engine. For example, there are fewer Netflix users rating the movies than watching them. Same thing with our Reveal solution; we can’t force agents into manually pointing out the best solution for every imaginable tech issue.

To compensate for scarcity, some guessing has to be done. A good recommendation engine should be able to spot the implicit connections between people, the issues they investigate, and the solutions they find. And sensitivity is key because the more subtle these connections, the more false positives are generated.

Going back to our multi-dimensional space, we have to set the right distance between clusters. Which is close enough to set off a recommendation, and which is too far and will generate useless results?

Engagement metrics can answer this. The more people click on a recommendation (i.e. URL of Document X in a cluster), the stronger it becomes. If another recommendation doesn’t get clicks, it may be irrelevant. So we integrate these metrics into the engine to make it more efficient—or rather, less faulty.

Recommendation has become increasingly popular in online services because it can produce very impressive results. However, as our own challenges demonstrate, the science is still far from perfect. I guess it will be a while before digital devices become smart enough to give us everything we want on the first guess. Then again, maybe that would be a bit scary!