Supersonic fast

What is that we expect from a search engine? First: it has to be effective and return exactly what we want. Second: it has to deliver the first thing in no time. The optimal balance between those two is the key to success and that’s how Google have won the internet search engines popularity contest. And look where it’s now...

But we’re not here to talk about Google; we’re here to talk about Retrospective and its search engine. And there is what to talk about as it completely clears the search engine checklist without breaking a sweat. Let’s put it to the test and see how it performs against the famous Unix grep.

First try

Let’s start from searching the entire folder (over 400 Megs of data) and see what happens. Grep goes first. "Time zgrep ERROR. It takes grep precisely 1 min and 2.668 secs to process all 307 files looking for the exact match.

Now it’s Retrospective’s turn, so I quickly create a new profile and manually import remote log files. I wait a few seconds for the autoconfig feature to process log definitions and I’m good to go. I change the search profile in the search tab and type in ‘ERROR’ in the text search parameter field, leaving the case sensitive option enabled and hit the [Start Search] button. Can you guess how long did it take Retrospective to process that? 45 seconds? 1 minute? 2 minutes? No, no and not even close. Retrospective returned search results under 20 seconds. Yes, you got it right. Less than TWENTY seconds. How’s that possible? While grep takes its time and processes files one after another, the Retrospective executes the same query simultaneously on a few files which makes it so much faster.

Second try

Now let’s try something different. I’ll try and search that 2 Gigs fatty for entries from the 12th of May 2007 containing the phrase “ERROR”.

"Time zgrep “2007 May 12” 2GB_1.log | grep ERROR". It took grep 32.115s to process the file without returning any results. So now I create yet another user profile in Retrospective and add the very same 2 Gigs log file. After the autoconfig processes the log definition I switch to the search tab and select newly created profile. I set the time parameter to search for entries between 12.05.2007 00:00:00 and 12.05.2007 23:59:59 and I hit the [Start Search] button again. The search is complete literally within a blink of an eye. How? The answer is time frame optimization. Retrospective doesn’t even bother processing that log file as it knows there is no point in doing that since the log file contains entries from the year 2011 only. Isn’t that smart?

Summary

To recap what we’ve learned here...

They say you can’t have it all. Well with the Retrospective you actually can - you’ve seen it here and you can try it yourself.