Vendor Accent

Tackling rootkits

Captain Raghu Raman takes a comprehensive look at rootkits and ways and means to deal with them

The word rootkit has its origins in the early days of hacking when an attacker managed to install a set of tools (toolkit) deep into the victim’s domain as close to the root as possible. Those were the days when most network defences were architected around watching the traffic coming in from the DMZ through the firewall, and the firewall rules were configured to break suspicious connects.

While an attacker could manage to sweep one odd machine behind the network, pinging any substantial range of IPs would set off the alarms in the intrusion detection systems. Also, some of the operations (such as enumeration) would be tedious from outside the DMZ simply because of bandwidth issues. So here is what the attacker did—he managed to compromise one machine within the network and installed a set of tools to perform these tasks from within the network.

A typical bag of tools in a rootkit would consist of a scanner (for enumeration), a privilege escalator (getadmin, pipeupadmin being old favourites), shell-spawner (NetCat) and at times a password cracker as well. More elaborate assortments would include a piping shell, though experienced hackers would do much of the diversion and piping using NetCat. In recent times, rootkits have included software to reconfigure the MAC address of the machine which is used in networks where the security policies are configured around the MAC address of the machine.

Another form of rootkits is recompiled commands which purport to do some task (usually the primary task) and also do a secondary activity. For instance, the code for creating an admin level user could be ‘wrapped’ around the ‘dir’ command (with echo turned off, of course) to create an exe or com file whose instance performs the directory listing (its normal function), creates an admin level user, and says open a NetCat shell on listening mode. For a touch of finesse, the command sequence would probably end with a ping to the attacker’s machine, letting him know that this ‘trojan’ was now active.

Purists often write and compile the entire code from scratch, but equally deadly effects could be achieved by simply hooking together various stubs using software such as Silkrope.

Ironically, the concept of rootkits hasn’t been limited to attackers. In a bid to implement copy protection, vendors have resorted to installing code that introduced vulnerabilities into systems. Sony’s CD copy protection raked up a controversy on the issue last year. Their system not only installed a rootkit-like unit into the users machine, it also made it impossible to be uninstalled—by the novice user anyway. (Refer to Mark Russinovich’s blog on sysinternals to get more information on this.) A rootkit can be used to hide anything such as a worm, virus or trojan, though that is not what a purist hacker would try to do.

Now that we understand what a rootkit is and what it can do, let us take a look at how to discover it and fix the problem.

While there is third-party software which can ‘discover’ and fix rootkits, elegant ones utilise simple tight code and are hard to discover using such tools. Hackers have been known to leverage existing SDKs as rootkits.

Here are the strategies which work in most cases.

• Shut it down

  Shut down the compromised computer and scan its storage media with suitable software. The rootkit detection software will analyse the files on the disc and determine whether a rootkit has been installed. The software will use signature detection to do so. (A signature is a sequence of bytes that makes for a fingerprint of the rootkit.) Running detection software on a live compromised computer has significant limitations since rootkits can hide from such scanners. They can also forcibly terminate such scanners.

• Heuristics

  Another approach involves analysing behavioural patterns of rootkits in memory via heuristics. Again, rootkits can hide from such detectors. Any approach that is using a compromised system to detect is vulnerable since it uses the lower level API calls of the system to search for rootkits.

• Cross-view-based detection

  These techniques assume that the operating system has been subverted, but this method leverages the fact that there is usually more than one way to ask the same question. In a cross-view-based detection method, the software used for the purpose calls the common APIs to enumerate key elements within the computer system such as the list of files, processes or registry keys. However, to be successful, the detection software must also have an algorithm to generate a similar data set that does not rely upon the common APIs. Any difference in these two data sets reveals something hidden because it did not exist in the data set generated using the common APIs.

• Integrity-based detection

  This approach compares the current state of the computer with a known uncompromised state of a computer. It highlights the changes between the two states.

• Traffic analysis

  Finally, a lot of rootkits send data from the compromised computer. This data can be captured by various network-monitoring devices such as routers and firewalls. An analysis of such traffic can tell whether a computer has been compromised.

Approaches 4 and 5 will not pinpoint the precise cause of the compromise, but can serve as a good indicator of something being wrong.

There are an increasing number of tools to detect rootkits. They utilise a variety of approaches including the ones discussed above. Some of the most promising tools are Sysinternal’s Rootkit Revealer, Microsoft’s Strider, and F-Secure’s Blacklight and Copilot. Copilot is a unique tool for rootkit detection since it comes in the form of a hardware component. This provides a high degree of assurance since the tool minimises the use of the subverted computer.

To conclude, many of the above-mentioned tools show promise even though some of them are in the proof-of- concept stage. A hybrid approach will be more effective since the different tools will balance out their strengths and weaknesses. At the moment, rootkit authors have the upper hand since rootkits do not need to be concerned with system stability. The detectors, especially the commercial ones, have this constraint, so avoid some useful but dangerous approaches.

The author is CEO Mahindra Special Services Group.
He can be reached at raghu@mahindrassg.com