03. Mar. 2015 by Marek
After several months of intensive work a new Retrospective version is ready to take log analysis to the next level. Version 3.4.0 is targeted at fulfilling all the requirements appearing in the complex enterprise environments. It also has many improvements both on the UI side, ensuring more convenient interaction and seamless UX as well as on the backend side, where significant changes of “stuff under the hood” make your searching faster whilst bothering your CPU less. Let’s quickly review the tastiest features of 3.4.0:
Sudo and su support – finally you will not have to bother your administrator about insufficient rights of your SSH user preventing you from getting to the log files
Jump servers support – when accessing your demilitarized zone (DMZ) all the necessary SSH tunneling will be automatically performed for you
Nice level adjustment – by increasing the *NIX niceness you will be able to make your interaction with remote severs less noticeable for applications already running there
Maximum connections restriction – another feature allowing you to control the Retrospective influence on the remote host by adjusting the max. number of simultaneous connections
Smarter behavior of Dynamic Tail and Autofind – you no longer have to worry about different formats of your logs during monitoring. After detecting some new formats, Retrospective will automatically recognize them and simply continue processing without any need for your intervention
Don’t you find these features tasty? Now let’s dig deeper into this 3.4.0 cake and also discover some of the cherries served on top.
In typical enterprise setups, administrators are highly concerned with the security. Therefore, often we can encounter mechanisms that prevent us from quickly accessing the desired log files after another bug detected on production. Most of the time, when your business is aided with the power of applications servers (APs), administrators ensure that APs have as few rights as possible and that no other user can access APs processes and related files. As a result, we are forced to use such tools as sudo or su to access e.g. jboss log files. Imagine you need insight into five different logs on five different machines and each time you are forced to sudo to the jboss user! How ineffective and tiring. Retrospective 3.4.0 allows you to provide a host configuration with all the details needed for the automation of sudo-ing and su-ing. Thanks to that you will quickly forget that you ever needed sudo and su in your daily work.
Another typical mechanism introduced by administrators to maintain high security is a demilitarized zone (DMZ), sometimes referred to as a perimeter network. Such a network is a perimeter that is located between the internet and the rest of enterprise network resources. DMZ has the most constrained security policy that allows enterprise employees to do very little. As a consequence, when you are outside your office, in order to access your log files located beyond DMZ, you are initially forced to SSH to some internet-facing server of DMZ, and only then connect to your final host. Again, it’s fine when you rarely have to do it. However if, in a critical situation, you need quick access to many log files/hosts, then doing the so-called host jumping manually can be quite irritating. Retrospective 3.4.0 relieves you of this by providing jump server options in the host configuration. Finally you will no longer notice the difference between accessing servers locally in your enterprise and accessing them from outside through DMZ.
In an enterprise with a high production workload we always want to make sure that nothing disturbs the “prod” machines in a way that influences applications used by the customers. This makes diagnosis of problems that “just happened” on the production quite challenging. Retrospective 3.4.0 comes to the rescue by allowing you to specify the execution priority used for searching and monitoring files on “prod” machines. The priority is translated to the process niceness with the use of the good old *NIX ‘nice’ tool. Assigning a low priority ensures that Retrospective will always wait until your critical applications finish delivering value to your customers and only then the searching/monitoring will continue.
The health of “prod” machines can also be aided by another newly delivered option that enables you to limit the maximum number of connections with the remote host. Retrospective is a smart beast that gradually increases the number of connections if the interactions with the host go smoothly. This can lead to creating up to 15 connections, each searching/monitoring a single file. When you feel that adjustment of execution priority does not guarantee a sufficient hard limit of resources consumed by Retrospective, you can always limit the max. connections. Then you will be able to do the log analysis on “prod” machines without the slightest worry of influencing the critical applications.
Retrospective 3.3.0 provided you with the cool feature of Dynamic Tail that dynamically includes in the monitoring new files appearing in the monitored directory. Sometimes you can encounter a situation in which the format of log files changes. For example the log entries dates start to include milliseconds. Dynamic Tail of 3.4.0 is able to detect such a situation and perform our innovative Autofind procedure that identifies the new date format and splits the file to log entries in the proper way. The Autofind itself was also improved in the scope 3.4.0. Now the file encoding is guessed more accurately thanks to taking into account the local present on the remote host. As a result of these improvements, you no longer need to worry about slight log format changes in monitored files and you can safely assume that guessed files encodings are simply correct.
Is that everything the 3.4.0 can offer you? Not at all! When it comes to the improvements of the “under the hood” stuff, we have optimized the hottest spots of the search pipeline. This will influence your CPU less and make your searching faster. We’ve also added some sophistication to the connection pooling ensuring that idle connections are closed when no longer needed. Thanks to that, Retrospective gets more stealth and your administrator stops worrying about processes appearing too often in the process table. Other improvements, visible to you in the UI cover:
Smarter host detection that greatly extends the amount of information displayed after testing the host in the Host Manager
Throughout support for windows shortcuts to both files and directories
Possibility to easily convert ad-hoc profiles (created typically by drag and drop of files to Retrospective) to permanent profiles appearing in the Profile Manager
You are also now able to quickly zoom/jump to individual sections of the result by using the summary bar chart visible in the search/monitor tabs. The support for accessing Windows network shares is another thing added in 3.4.0 that is worth mentioning. To aid a typical enterprise situation in which your home directory is present on a remote share and you are concerned with the access performance (for example you are connecting through VPN), 3.4.0 allows you to configure a custom Retrospective home located outside of the network share.
Clearly, 3.4.0 has a lot to offer when compared to its 3.3.0 buddy. We are confident that the new features make Retrospective completely ready for challenges encountered in modern enterprises. If the lack of these features previously prevented you from considering Retrospective as a powerful log analysis tool, we encourage you to give it a try now and see for yourself how easy and fun it can be working with your log files even in the most complex enterprise setups.