Management and decision making 102: Bayes' theorem

Imagine yourself as the leader of a health agency and that you have to decide whether to go ahead with a large universal screening program e.g. screening for cancer. You have been told that the test used in the screening is 99% correct and that about 0.5% of the population has the disease.

Many people at this point simply conclude that the screening is a good thing: We will pick up a lot of cases at an early stage, they will get treatment and may even survive because the cancer was discovered early. You, however, are not most people. You try to think about this carefully.

First of all you might wonder about the cost of the screening and how many cases it discovers compared to the no universal screening case. Those with high risk migh go to the doctor anyway and so on, so the question is how many extra cases you will find and be able to cure. This is all true, but you also start to wonder about potential harmful effects.What about those who will get the message "you have cancer" because of mistakes in the test. Well, the test seems good, 99% certain, som maybe this is nothing to worry about?

Unfortunately, this is wrong. We can show this using an example as well as more formally using Bayes' Rule.

To illustrate the problem, imagine 1 000 000 people doing the test. Out of these, 5 000 have the disease and 995 000 do not have the disease. Given an accuracy of 99%, this implies that:
- Among the 5000 with the disease, 4950 will get the message "you have the disease" and 50 will get "you do not have the disease"
- Among the 995 000 who do not have the disease, 985 050 will be told "you do not have the disease" and 9950 will be told "you have the disease"

Altogether this means that 14 900 have been told that the have the disease (9950 + 4950). Out of these only 4950 really have the disease ie. 33.2% In other words, given the information in this example, when a person in the screening program is told "you have the disease" the probability of actually having the disease is only 33.2%

Why? The key problem is the fact that although the test itself seems accurate, the probability of not having the disease is very high. For every case the probability of making a mistake is small, but since there are so many cases, and since most of them are people who do not have the disease, even a small probability of making a mistake will result in many false alarms.

Formally we can use Bayes rule to calculate the probability of actually having a disease given a positive test. We want to know the probability of having the disease (A) given a positive test result (B): P(A|B). From basic probability theory we know:

P(AB) =  P(A) P(B|A)

That is, the probability that both A and B will happen can be found by multiplying the probability that A will happen and the probability that B will happen given that A has happened.

Similarly we know

P(BA) = P(B) P(A|B)

Since P(BA) = P(AB), we also have

P(B) P(A|B)= P(A) P(B|A)

Rearranged:

P(A|B)= P(A) P(B|A) / P(B)

This is called Bayes rule and it can be used to calculate the correct probability. Recall
P(A) : Probability that a person the disease = 0.005
P(-A) : Probability that a person does not have the disease = 0.995
P(B):  Probability that a person tests positive
P(B|A): Probability that a person tests positive given that she has the disease = 0.99
P(B|-A): Probability that a person tests positive given that she does not have the disease = 0.01


To work out the probability that a random person tests positive, remember that there are two ways of testing positive: Either you have the disease and test positive, or you do not have the disease and test positive.
Probabability of having the disease and testing positive: P(A) P(B|A) = 0.005 * 0.99
Probabability of not having the disease and testing positive: P(-A) P(B|-A) = 0.995 * 0.01
We then find

P(B) = P(A) P(B|A) + P(-A) P(B|-A)
P(B) = 0.005 * 0.99 + 0.995 * 0.01 = 0.0149

We then have all the information neede to use Bayes rule to calculate the probability of having the disease given a positive test:

P(A|B)= P(A) P(B|A) / P(B)

P(A|B) = 0.005 * 0.99 / 0.149 = 0.32

In short, knowing a three hundred year old mathematical rule, will enable you to make a better decision. You need not even know its precise formulation - all you have to remember is that the importance of considering not only the accuracy of the test (high), but also the base rate prevalence of the disease (low) and that the end result will be in between.