In statistical computations, intuition can be very misleading
A Bad Penny Always Shows Up
To simulate what would happen with a bad (deformed, bent) penny that shows tails more often than heads, you could add more sides to the coin in line 5 of Listing 1:
my @sides = qw( H H H T T T T );
From seven tosses, the coin would then come up heads three times and tails four times; the script would then correspondingly compute (with random deviation) the p-value from:
$ Rounds: 1000 Tails: 565 p-value: 0.0351
The p-value is approximately 0.04 percent (i.e., well below the specified 5 percent threshold for the significance value). This seriously threatens the null hypothesis that the coin should land on both sides with the same likelihood.
Careful with Your Diagnosis
Experiments that test new medications or treatment procedures for their efficiency define the null hypothesis as "The medication has no effect," set the significance value to around 5 percent, and then alert if the p-value drops below this in the experiment – that is, if there are suddenly good reasons for assuming that the null hypothesis is incorrect. In this case, the miracle cure tested really has shown some positive treatment effects with a high degree of probability.
According to Alex Reinhart's recently published book on statistical blunders [3], however, it is common practice for studies to interpret the significance value incorrectly retroactively, thus giving patients false hope or needlessly causing patients to panic. These base rate fallacies [5] occur in the context of conditional probabilities and are caused by the fact that a certain result already has a certain probability a priori that needs to be considered in the computation.
The following experiment from Reinhart's book shows what for many people is an amazing deviation between popular opinion and precise science: A mammography returns the correct diagnosis for patients with breast cancer with a 90 percent probability. However, the test comes up with a diagnosis of breast cancer for approximately 7 percent of healthy patients, so that – in the case of positive findings – further diagnostic procedures are necessary for clarification. The question is now: Is this test suitable for effectively screening the population? If the mammography detects breast cancer, how great is the probability that a randomly selected woman really needs treatment?
Most people will think about this for a while, and then subtract the 7 percent false positive rate from 100 percent in their heads and end up with a result of around 93 percent. However, this assumption is totally wrong. What is the correct result? Maybe 70 percent? Or even 50 percent? The amazing truth is that a mammography performed on a randomly selected woman correctly diagnoses breast cancer in only around 9 percent of the cases.
Amazing Statistics
If your mind experiment led you to believe that the test accuracy was higher than it actually is, you probably fell into the typical base rate fallacy trap and forgot to consider in your calculations that, on average, only 0.8 percent of women in a given population have breast cancer.
Of 1,000 women, 992 thus do not have breast cancer and in 7 percent of these cases, mammography will return the wrong diagnosis; that is, 70 of the women tested will be given incorrect results. Of the eight women with breast cancer, the test diagnoses the medical condition of seven of these women correctly, which means, in total, of the 77 breast cancer findings after mammography, only seven are correct (i.e., approximately 9 percent). Given this low accuracy rate, it is inadvisable to perform across-the-board tests; instead, only certain high-risk groups should be tested where an inefficient test is still far better than no test at all.
« Previous 1 2 3 Next »
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Gnome 47.1 Released with a Few Fixes
The latest release of the Gnome desktop is all about fixing a few nagging issues and not about bringing new features into the mix.
-
System76 Unveils an Ampere-Powered Thelio Desktop
If you're looking for a new desktop system for developing autonomous driving and software-defined vehicle solutions. System76 has you covered.
-
VirtualBox 7.1.4 Includes Initial Support for Linux kernel 6.12
The latest version of VirtualBox has arrived and it not only adds initial support for kernel 6.12 but another feature that will make using the virtual machine tool much easier.
-
New Slimbook EVO with Raw AMD Ryzen Power
If you're looking for serious power in a 14" ultrabook that is powered by Linux, Slimbook has just the thing for you.
-
The Gnome Foundation Struggling to Stay Afloat
The foundation behind the Gnome desktop environment is having to go through some serious belt-tightening due to continued financial problems.
-
Thousands of Linux Servers Infected with Stealth Malware Since 2021
Perfctl is capable of remaining undetected, which makes it dangerous and hard to mitigate.
-
Halcyon Creates Anti-Ransomware Protection for Linux
As more Linux systems are targeted by ransomware, Halcyon is stepping up its protection.
-
Valve and Arch Linux Announce Collaboration
Valve and Arch have come together for two projects that will have a serious impact on the Linux distribution.
-
Hacker Successfully Runs Linux on a CPU from the Early ‘70s
From the office of "Look what I can do," Dmitry Grinberg was able to get Linux running on a processor that was created in 1971.
-
OSI and LPI Form Strategic Alliance
With a goal of strengthening Linux and open source communities, this new alliance aims to nurture the growth of more highly skilled professionals.