Paw Prints: Writings of the maddog

The other day I read an article on the web that said that "Firefox, Adobe top buggiest-software list" and which stated that Firefox had reported the most "vulnerabilities" of all the "application programs".

The article admitted that the #2 and #3 "application programs" ("Adobe" and "Microsoft") reported were "closed source" and that "open source" programs tend to show all the blemishes, not just the ones reported by their customers, and reflected back through visible reports by the companies. To be even fairer, I would point out that comparing "Firefox" with all of the applications that Adobe has and all of the applications Microsoft has is a bit like apples and oranges....but that is not the main concept I will try to get across in this blog entry.

What is more important is Mean Time To Repair (MTTR) in any bug, and particularly a "vulnerability".

Having been an engineer and product manager for Digital's Unix product many years ago I can vouch for the fact that not every "vulnerability" (or even a simple bug) is reported to the customer. While we kept a log of all the problems reported to us by customers, there were the additional "problems" (and even "vulnerabilities") found by the engineers themselves that were never reported to the customers because the problems were the "one in ten trillion" types of problems that customers "would probably never see" and which the engineers would fix "sometime in the future" if the need arose.

Some of these bugs were scaling issues which the engineers were aware of, but would require an order of magnitude greater system size then we thought our customers were capable.

In addition, we released products with known bugs because our customers were waiting for bug fixes that would go out in the new release of the software.

As the current article admitted, all programs have bugs, and larger programs (and particularly ones with lots of plug-ins) may have lots of "vulnerabilities".

What would be more interesting to me would be a study of "mean time to repair" (MTTR) for the bugs reported. How fast were the fixes to the bugs presented to the customer so they could have relief on the issue?

Often bugs are rated in priority of criticality. Engineers work on the most critical bugs first, then devote their time to less critical bugs. Sometimes fixing bugs are deferred since the engineer knows that a certain section of code (containing the bug) would be re-written anyway, correcting the bug (but sometimes introducing new bugs).

Sometimes bugs are never fixed, since the code has been "retired" and is now "unsupported" (whether or not the customers are still using that code). In the case of support being terminated, with closed source code the MTTR goes to "infinity", throwing off any reasonable study of "mean time", but reflecting the helplessness of the customer to get the fix they need.

For-profit companies working with closed source code have limited resources....even ones like Microsoft. They have to balance their resources with the jobs they need to get done, such as putting out new functionality and making a profit for their shareholders. Large companies can have small divisions, and even small teams of engineers working on a particular product. Just working for a large company does not mean you have the resources needed to maintain a program.

Whether a particular bug is marked as low priority, or whether it has met with one of these other fates makes no difference. it is still a bug that, until fixed, may be keeping you from doing your job.

Some people argue that Free Software has "unlimited" resources. Every product or project is limited in resources in one way or another. The number of people who can work on Free Software, and particularly one piece of software is limited by the people with the skill, time and inclination to contribute. But what Free Software does have is the ability of the end user to escalate their own bug fix in "criticality", by seeking out their own resources to fix the problem if the developers do not have the time or inclination to fix it.

Free Software also allows the end user to have insight into the bug fixing process, to see the discussion revolving around the fixing of their bug (often hidden in closed source software), and to help escalate the likelihood of the bug fix by providing additional information about the bug, or even a suggested patch or solution to the developers.

I remember one particular incident where there was a vulnerability that affected every Unix and Linux system. It took Digital two weeks to generate, test and distribute the patch to our customers (we did have a work-around that we communicated to them, but even that took time to communicate). It took the Free Software community four hours (after they understood the problem) to generate a source code patch and put it up on the Internet.

Finally, even though Digital had dropped support for their Ultrix operating system by the time of this vulnerability, there were still people using Ultrix. This particular exploit was so horrific that Digital did generate a patch for the Ultrix system too, but it took over three months to generate the patch and deliver it to the grateful customers. So which group of customers got the best "service"?

Almost ten years ago a magazine had a survey done about "Best Customer Support", mostly based on the concept of "best MTTR" for software bug fixes. Two years in a row their readers gave the best marks to the Free Software community.

Carpe Diem!

Comments