Comments on: We Are Accused of Over-cheerfulness

By: elizabeth brown

elizabeth brown — Sat, 16 May 2009 23:29:36 +0000

re: wainer’s statistics. the error here is easy to spot- the .1% who are innocent is .1% of the total number of people convicted of crimes, not .1% of the total population. if .1% of the total population was convicted of crimes that they did not commit, that would bear no relation to the data given in the beginning of the problem: wainer stated that the number of criminals (or let’s say crimes, for simplicity) is 10,000 per year. if the justice system punishes the right person for the crime 99.9% of the time, that leaves the other .1% when an innocent person is blamed. the population of the country is totally irrelevant in this problem; what we want to know is the proportion of convictions that are innocent. you can only conflate the two statistics if there are the same number of crimes investigated and charged each year as there are residents in the US, which is obviously not the case. To put it another way, saying that .1% of convicts are innocent is not the same as saying that there is a .1% chance of being wrongly convicted. why? because his jump from statistics on those convicted to statistics on the general population leave out the largest group of people- those who have neither committed a crime NOR been convicted. For everyone who has been convicted, innocent or guilty, there was first a crime commited that was then investigated and charged. There is a crime for every convict, even if they are not correctly matched. there is not a crime for every US resident.

By: Margaret Cibes

Margaret Cibes — Wed, 29 Apr 2009 14:51:14 +0000

I’m not qualified to critique the assumptions behind the math, which are in most cases appropriate, but I suggest that there’s a typo in the line containing “186,000 / (186,000 + 3.35 million).” As someone pointed out here earlier, the 4% is slightly low. The denominator should represent the sum of correct (186,000) and incorrect (?) cancer diagnoses. As it stands, the 3.35m figure represents 10% of all 33.5m mammograms; however, it should be “slightly” smaller because it should represent 10% of mammograms for only people who do not actually have cancer (approx. 33.3m = 33.5m – 0.22m = 33.5m – 186,000/0.85). This doesn’t change the final probability (4%) enough to matter much. In any case, I applaud Wainer’s spotlight on Bayes theorem!

By: Al Warner

Al Warner — Tue, 28 Apr 2009 17:11:25 +0000

Loathe as I am to take on any professor on home turf, I do think there is something awry with this analysis and it has to do with how the argument subtly shifts from investigating a crime to assessing a population. Prof. Weiner stipulates a justice system that is 99.9% accurate – but applied to what? That is, he applies the math to an entire population (as in the government seeking out and sequestering criminals) versus the application to specific cases (as in the government responding to a specific crime and seeking the single person responsible). The question is not how many people are there in the US – but how many crimes are committed? If we assume that the 10,000 criminals commit 10,000 crimes (not the 309.990 implied in the letter) and it is these events that are investigated with 99.9% efficiency, we should get a rather different result. We’d still have imprisoned the 9,990 true bad guys – but only 10 innocents. (Well, “only” in the sense that 10 is not as bad as 300,000).

By: Anonymous User

Anonymous User — Tue, 28 Apr 2009 14:12:57 +0000

The problem with Professor Wainer’s example is, as stated, the assumptions- not the assumption that everyone is equal under the law (and should therefore be considered in the denominator) but the assumption that there are only 10K true criminals. According to recent statistics, there are 1.6million prisoners under state or federal authority and the US adult (18+) population is 217.8million. Assuming that the number of prisoners under supervision represents the number of crimes for which someone is responsible (not necessarily guilty), then the accuracy rate is over 88% and the innocents are less than 12%. The mathematical reality is that this represents almost 200k real people.I am more disturbed by the assumption Professor Wainer makes based on the medical example: “Women less than 50…should not have mammograms”. Decreasing the denominator in this example increases the accuracy of the mammograms, but does not help the women whose cancer then goes undetected. The important thing to remember in using statistics is the impact of the results- in this case, it is far better for an individual to get the test and have it be a false positive (96% of the time), than to not be tested and find the disease too late for it to be treated.

By: Anonymous User

Anonymous User — Mon, 27 Apr 2009 23:21:49 +0000

Editor, Professor Wainer is entitled to hypothesize an ‘accuracy rate’ of 99.9% for criminal justice. He is also allowed to hypothesize a group of 10,000 structured in a most unusual way. If he defines accuracy rate as the percent of convicted who are truly guilty, he can then determine the number of correct and incorrect convictions among his 10,000 under his scenario. What he is not entititled to do is apply his hypothetical error rate of .001 to his ‘innocent’ population of 300 million. The reasons are complex but we can use an example – as does Wainer.Hypothesize an accuracy rate of 100%,- surely a trivial difference from 99.9%.Now his ratio becomes 10,000/ 10,000 + .00 ( 300 million) which = 1.00 and the probability of an innocent having being convicted is zero, not his 97%. This is so regardless of the number of true criminals. It is also as legitimate a conclusion as Wainer’s and illustrates the dangers of such statistical speculation.

By: Anonymous User

Anonymous User — Sun, 26 Apr 2009 20:58:53 +0000

“I see this comment posed twice now: “Two sentences from the same paragraph… Huh?” This is not a discrepancy. One is the probability that the test is positive, given that you have cancer. The other is the probability that you have cancer, given that the test is positive.”That is crystal clear. Thank you for the clarification. (Your example just made it more confusing.)Perhaps this is the reason the article in question is under so much scrutiny: the english is sloppy and ambiguous.

By: Tamas Oravecz

Tamas Oravecz — Sun, 26 Apr 2009 20:10:17 +0000

I agree with those pointing out that the denominator should be 10,000 or less. Prof Wainer’s assumption is that the justice system is working like a quaranteen, lumping up people when an infection detected. Knowing the efficiency of the justice system to solve crimes makes it unlikely that more people are put on trial than the number of actual criminals, even if each comits multiple crimes. If 10,000 is used as the denominator (and leaving the other assumptions correct), then the probability of an innocent person being convicted is closer to 0.0001%.The numbers in the equotion of the breast cancer example are correct, except the solution: the result is actually 5.3% rather than 4%. Another way to look at the usefulness of mammograms is the following (after some additional calculation): If the mammogram is positive, the individual’s risk for having breast cancer increased ~ 8-fold compared to the general population who are taking mammogram (from 0.65% to 5.3%); if it’s negative, then the risk decreased ~ 7-fold (to 0.099%).

By: Anonymous User

Anonymous User — Sat, 25 Apr 2009 18:08:43 +0000

I see this comment posed twice now: “Two sentences from the same paragraph… Huh?” This is not a discrepancy. One is the probability that the test is positive, given that you have cancer. The other is the probability that you have cancer, given that the test is positive. Suppose for example that 101 people take the test — 1 has cancer and the other 100 don’t. And suppose that the test has a 100% chance of being positive if the patient has cancer, and a 10% chance of being positive if they don’t. Then you can expect 11 people to test positive (1 cancer case and 10 non-cancer cases), so that the probability of having cancer, given that the test is positive, is 1/11, or about 9%.

By: Anonymous User

Anonymous User — Sat, 25 Apr 2009 03:01:55 +0000

Two sentences from the same paragraph: “Mammograms identify breast cancers correctly 85 percent of the time.” and “Thus if you have a mammogram and it results in a positive (you have cancer) result, the probability that you have cancer is:…4%.” Huh?

By: Anonymous User

Anonymous User — Fri, 24 Apr 2009 23:23:04 +0000

It seems like there are a number of ways to diagnose the problem; here is my understanding of it.Let’s say that 10,000 “true” criminals means 10,000 crimes. AND, let’s say that there are 10,000 arrests, trials, and convictions (assume that there are no mistrials and every crime is “solved” in the sense that someone is sent up for trial). If the system works properly 99.9% of the time, then 9,990 true criminals have been convicted and 10 innocent people have been convicted. In other words, the probability that you are guilty if you are convicted is 99.9%, which is what we should expect if the system works properly 99.9% of the time. On the assumption that someone is convicted for every crime, the percentage of prisoners who are innocent is 0.1%.Wainer’s numbers would work if there were 10,000 real criminals, but they were all very busy, committing among them a total of 309,990 crimes–and if the police and courts arrested and convicted someone different for each crime, putting 300,000 innocent people in jail and catching 9,990 of the true criminals. But then it wouldn’t be true at all that “the probability of an innocent person being convicted is but 0.1 percent,” as Wainer specifies in his original letter.

By: Anonymous User

Anonymous User — Fri, 24 Apr 2009 14:52:04 +0000

There’s nothing wrong with Bayes’s theorem, but if you apply it with unreasonable inputs, you get unreasonable answers. Prof Wainer’s mistake is in trying to think of the justice system in the same way as medical screening. Imagine visiting your attorney every February… “OK, let’s schedule you for your annual grand larceny trial; oh, and it’s been 5 years since you were tried for murder, so let’s schedule you for that as well.” Note in Wainer’s calculations that he is speculating that 300,000 innocent people are CONVICTED each year. If we suppose, say, that if a case goes to trial, there is a 10% chance of being convicted if innocent, then Wainer is speculating that 3,000,000 innocent people are tried for criminal cases each year. That is not a reasonable supposition. So I agree with the editors — he did the math right, but the assumptions are way out of whack.

By: Anonymous User

Anonymous User — Thu, 23 Apr 2009 23:22:03 +0000

Two sentences from the same paragraph:”Mammograms identify breast cancers correctly 85 percent of the time.”and”Thus if you have a mammogram and it results in a positive (you have cancer) result, the probability that you have cancer is:…4%.”Huh?

By: Anonymous User

Anonymous User — Thu, 23 Apr 2009 22:49:48 +0000

Prof. Wainer is attempting to apply Bayes rule to the criminal justice system. Bayes rule leads to many surprising and counter intuitive results, and an understanding of it (and statistics in general) is critical to all kinds of fields.That said, I’m a bit suspicious about applying Bayes rule to the criminal justice system. In particular, the police won’t arrest someone unless a crime has been committed, and this needs to be worked into the model. Also, the assumptions are a bit arbitrary. The medical testing example is much better.

By: Anonymous User

Anonymous User — Thu, 23 Apr 2009 21:10:51 +0000

The error in Prof Wainer’s calculation stems from his assertion that “… the probability of an innocent person being convicted is but 0.1 percent.” Not true. The 0.1 percent reflects the probability of a guilty person “not” being convicted. We have no way of going from these numbers to the probability of an innocent persen being convicted.

By: David Ehnebuske

David Ehnebuske — Thu, 23 Apr 2009 20:33:37 +0000

It seems to me that the reason Professor Wainer’s mathematical model of the justice system produces such a surprising — and, hopefully, incorrect — result is that it doesn’t capture the essence of what’s being modeled. Good arithmetic won’t help a model that doesn’t capture the system.In particular, I can see no justification for calculating the number of wrongly convicted people by multiplying the number of innocent people in the US by the rate at which the system fails to convict guilty persons (0.1%, in the example). The rate at which the system fails to convict a guilty party tells us nothing about the rate at which it convicts innocents.Even so, Professor Wainer is likely correct in his larger point that many more people are wrongly convicted than is generally assumed. Because the number of innocent people is so large, convicting them at even tiny rates will result in substantial numbers.