From venture capitalist Tom Evslin’s Fractals of Change blog, a reminder of two things: First, the map is not the territory and the model is not the reality; and second, we need to be more certain about some decisions than about others.
Concerning the first point, that the map is not the territory:
“If a test to detect a disease whose prevalence is one in a thousand has a false positive rate of 5%, what is the chance that a person found to have a positive result actually has the disease, assuming you know nothing about the person’s symptoms or signs?”
First to answer correctly was Matt Crawford:
“Assume that the test is performed on everyone regardless of symptoms of the disease. Then out of every thousand people who receive the test, one has the disease and 999 do not.
“Further, assume that the test has no false negatives: anyone who actually has the disease gets a positive result.
“Then 1 out of every thousand tests are true positives. The remaining 999 should be negative results, but the 5% false positive rate means that 49.95 (so round to 50) of these people will receive false positive results.
“Then out of our 1000 tests, 51 return positive results. But only one of these is a true positive, so the chance that a positive test identified someone who actually has the disease is 1/51 or about 2%.”
(Emphasis and extra paragraphing added.)
Second, the more important a decision, the more certain we want to be about it:
The point is that you must weight the costs of being right and the costs of being wrong both for the positive and the negative case.
Back to medicine, suppose your doctor is one of the benighted 81%. He or she tests you using the test in the first question and you come up positive. Let’s suppose that the disease is always fatal if not treated and there’s a treatment available but it has a 25% chance of killing you itself.
If the doctor believes that there’s a 95% chance you have the disease, the dangerous treatment is clearly justified; but, since the true likelihood is less than 2%, the treatment is more dangerous than your untreated prognosis.
Always a good idea to get a second opinion AND check your doctor’s math.
(Emphasis and extra paragraphing added.)
Leave a Reply