Search Skills and Approaches for SQL Server (and other platforms)

One of the things that I suspect many companies are working through is the issue of information.  As in, too much of a good thing makes it harder to get the good stuff out of the good thing.

If there’s any trend that’s apparent on content sites like the (traditionally magazine) article sites and even internal databases that have information that we want to summarize and sift through, it’s that it’s very difficult to surface relevant information for a given search.  SSWUG suffers from this (I admit it – it’s something we work on regularly though!) where there is just so much information that it can make search results only marginally useful.

This is also quite apparent though on applications and utilities on internal applications and environments as well.  If you search for something, what types of things make the results better for your request?

This is a very difficult question.  It seems so easy – just search for the terms provided and show the results, perhaps sorted by some frequency or proximity scoring.  Full-text search is good about this.  You can get to some results fairly quickly that seem like they could be OK.

However, if you sit back and watch the users of the application – how many times are they having to refine their search by adding terms, or choosing “most recent” vs. “most relevant?”  Those types of options mean we’re not getting the job done, or not understanding the goal of the search at the very least.

Typically, you can add a good bit to the usefulness of the search results by considering different things –

  • Date – if you can stack the odds to take into account the age of the information, that may be helpful.1
  • There may be inferred external influences for the data – perhaps city or state or other elements that need to be considered in the sorting of results
  • Perhaps you only need show results that “belong” to the user?
  • Context – the more context you can gather, perhaps from the application state or form the user is on at the moment, you may be able to better infer the types of things that would be most helpful

I’ve been so frustrated lately myself in doing different types of queries through Google, applications, etc.  I don’t know the answer necessarily, particularly in the case of a general search engine like Google, but in applications, it seems like we could do so much better.  First by watching.  See what people THINK they should be looking for and what they enter to find that element.  Rather than educating them about better ways to enter it or whatever, I think we need to apply the “Seek first to understand, then to be understood” (Stephen Covey) logic to this.

It seems that if we understand, modify the search systems, show better results, we’ll have more productive users, make better use of the information we have on file, and be better about information use going forward, an issue that certainly isn’t going to get any easier.


1 You do have to be careful with this as date grouping or sorting could be related to what is being sought as well.  And, in some cases, if you apply a sort that considers date, you can exclude or sort out items that would have been helpful.  Too many times applications can consider older information as less valuable, where in reality, it’s pertinent and helpful.  Nothing is easy. 🙂