Code Analysis: Open Source Keeps on Improving

Thus far, Coverity has investigated over 55 million lines of code from more than 250 Open Source projects for the “Scan Report on Open Source Software 2008”. The source code was analyzed over a period of two years using the Coverity Prevent analysis software. The project is funded by the Department for Home Security. Projects investigated include the free Apache Web server, the Linux kernel, the Firefox browser and script languages such as PHP and Ruby, or the Samba project.

The man behind the Scan project at Coverity is software developer David Maxwell, who writes code for the free NetBSD operating system himself. Maxwell will be revealing the results in a Webinar on Wednesday. Talking on how the tool works, Maxwell said that the source code was run through a static analysis tool to identify specific types of software defects, with the results then being investigated by developers. Coverity's investigation comes to the conclusion that the quality and security of Open Source software continues to improve. In the analysis period, more than 8500 individual source code defects were removed; this is a decrease of 16 percent. The statistics for the first year revealed more than 6000 bugfixes in some 50 Open Source projects. The analysis also allows conclusions to be drawn on the frequency of common errors: the most frequency error message was “Null pointer reference”. In contrast to previous assumptions, the researchers were unable to demonstrate a link between error density and function length in static code analysis. Coverity also discovered that the size of the code base and the number of defects are so closely related that predictions with a 72 percent hit rate were possible.

The analysis software was developed under the leadership of Stanford University and is sold as a commercial product by Coverity: the company also performed the current survey. The Software Scan is part of an “Open Source Hardening Project” launched by the Department of the Interior and is available to free projects from the Scan website. Most of the code investigated was writtenin C, but the statistics also include C++. The software tool gained the ability to investigate Java code in November 2007. Besides Coverity, security provider Symantec and Stanford University received sponsorship in the scope of the project, which has a budget of US$ 1.25 m. The project started in March 2006 and is scheduled for a period of three years.