A research group in the computer sciences faculty at North Carolina State University has written a prototype to prevent rootkits from manipulating kernel object hooks to do their damage.

The four researchers into the rootkit protector created and implemented a special virtualized system that defends against persistent rootkits that tamper with kernel execution. The system assembles specific function calls and messages, mirrors them in a shadow copy of the kernel hooks in a central location and protects them from hardware write access. To test their product, called HookSafe, the team let loose a few real rootkits and also measured the system load on the host system. The result showed highly effective protection with a mere 6% system slowdown.

The team found successful defense against, for example, the Adore-ng and Phalanx rootkits. Xuxian Jiang, one of the four team members, told Linux Pro Magazine that the HookSafe source code will probably be made public sometime in the future. Evidence from other projects of team members listed on the university's publication page makes this credible.

The team still recognizes two weaknesses of their system. The first is that it's based on so-called hook profiles that emerge from the dynamic analysis of the kernel hooks, a process they acknowledge as being far from perfect in that hook access points could be missed, resulting in protection gaps. A possible solution is either an improved dynamic analysis or a complementary process using static analysis. The second weakness is that HookSafe can only register attacks that it recognizes, so admins need to define the protective hooks ahead of time. The team intends to fix this by working together with projects such as HookFinder or its own HookMap predecessor product.

Peng Ning, Zhi Wang and Xuxian Jian of the state university have been working with Weidong Cui of Microsoft Research on the solution since September 2008 and want eventually to take HookSafe in the direction of operating system kernel protection. Their 10-page research paper developed out of the annual Security, Audit and Control (SIGSAC) working group conference of the Association for Computing Machinery (ACM) held in Chicago. The association has a worldwide membership of 100,000 from industry, research and the free software realm that grants the annual "computing Nobel Prize" known as the A. M. Turing Award.

Gallery (7 images)

A research group in the computer sciences faculty at North Carolina State University has written a prototype to prevent rootkits from manipulating kernel object hooks to do their damage.