How to work with this page

This page is the workshop for Lesson 3 of the Logic unit. You work through it alone, in about 45 minutes, with your Lesson 3 handout beside you. It has one job: to make sure that after this lesson you never again hear “the chance of this evidence if she were innocent” as “the chance that she is innocent”.

Each part is built the same way:

1. Explanation — short, with the lesson’s own numbers.
2. Exercise — you type or choose, then press Check. Nobody sees your mistakes. The page counts your first try; that number is for you.

The page has three stages. Stage 1 puts the lesson’s case on trial and asks you to take a side before you have the tools. Stage 2 gives you the tools: the one-line rule, counting in whole people, the prosecutor’s fallacy, the biases with names. Stage 3 sends you back to your verdict with the tools in hand. Your sheet tells you when each stage is due.

Recall — Lesson 2 from memory

Before anything new: eight fields on Lesson 2, without your notes. Guess first: how many of the eight will you get? …… of 8.

Stage 1 — The case first

In November 1999 the solicitor Sally Clark was convicted of murdering her two infant sons, who had died in 1996 and 1998. The Crown’s expert witness, the paediatrician Sir Roy Meadow, told the jury that the probability of two cot deaths in a single affluent non-smoking family was 1 in 73 million — a figure he got by squaring the single-cot-death rate. The figure was wrong twice over. It assumed the two deaths were independent, which they are not: cot deaths cluster within families. And what the jury heard as “the chance Mrs Clark is innocent” was in fact “the chance of two cot deaths given she is innocent” — a different quantity, which the jury would have had to weigh against the chance of two infant murders in one family, also vanishingly small. The Royal Statistical Society said so publicly in 2002; the conviction was quashed in 2003; Sally Clark died in 2007. The Dutch nurse Lucia de Berk was convicted in 2003 on similar statistical grounds and acquitted in 2010.

The lesson puts two remedies on the table for the next time an expert offers a jury a single figure: the training route (mandatory statistical instruction for judges, juries and expert witnesses) and the exclusion route (single-figure statistical claims kept out unless a court-appointed expert presents the full calculation, in posterior form).

Stage 2 · Part A — The rule, in whole people

A1. One line

Most reasoning in life is not deductive. The question is not “does this follow?” but “does this evidence make this hypothesis more likely — and by how much?” The rule is one line:

posterior is proportional to prior × likelihood.

Three words to keep apart:

The likelihood is not the posterior. \(P(\text{evidence} \mid \text{hypothesis})\) is the probability of the evidence given the hypothesis; \(P(\text{hypothesis} \mid \text{evidence})\) is the probability of the hypothesis given the evidence. Confusing the two is the single most common error in the lesson, in court, and in the clinic.

A2. Count in whole people

Do not multiply probabilities; count people. Imagine 1,000 people walking through the screening centre.

A 99%-accurate test, a positive result, and the rational answer is nine percent. The base rate (1 in 1,000) outweighs the test’s accuracy. Two ways this goes wrong: answering “99%” (the likelihood read as the posterior), and dividing by 1,000 instead of by the 11 positives (the wrong denominator).

Part B — The prosecutor’s fallacy

B1. Two sentences that sound alike

The prosecutor’s fallacy consists in asserting \(P(\text{evidence} \mid \text{innocent})\) — how likely this evidence would be if the defendant were innocent — and inviting the jury to hear it as \(P(\text{innocent} \mid \text{evidence})\) — how likely innocence is, given the evidence. The two can differ by orders of magnitude. It is not a miscalculation but a substitution: the likelihood is put where the posterior belongs, and the prior — how probable innocence was before the figure was heard — silently disappears.

The same substitution appears with DNA. “The chance of a random match is 1 in 10 million” is a likelihood. Searched across a database of 30 million people, that likelihood produces, on average, three matches by chance alone — so the figure does not, by itself, identify a guilty party.

Part C — Biases with names, and errors to spot

C1. Three names

Representativeness: judging how likely something is by how well it matches a stereotype. Asked whether “Tom W” — quiet, neat, orderly — studies computer science or humanities, both lay subjects and trained psychologists pick computer science, even when told the humanities cohort is many times larger. Base-rate neglect is the error that results: the prior probability of the category — how common it is — is left out. Kahneman’s chapter title on the cab problem is exact: Causes Trump Statistics. We reach for the causal story (the witness saw a blue cab, so the cab was blue) and discard the statistical fact (most cabs in the city are green). The prosecutor’s fallacy is the third name: a conditional probability, inverted.

Stage 3 — Back to the case

Your verdict, with the tools

Take out what you wrote at Stage 1. Now rewrite it in three sentences:

1. Your rule — and its nearest neighbour on the table: the training route or the exclusion route.
2. Your reason — and say which step of the calculation your rule protects: the prior (so the jury hears what “vanishingly small” is measured against), the likelihood (so the independence error is caught before the figure is spoken), or the swap into the posterior (so the jury cannot hear the one as the other).
3. The price your rule pays. Training: thirty years of professional commentary have not stopped the fallacy recurring — say what your training does differently. Exclusion: the cases it loses — the DNA match, the epidemiological causation case, the defendant whose acquittal depends on the base rate being heard — say which you keep and how.

Then check your own paragraph:

• Is your rule stated on the case — what it does to Meadow’s “1 in 73 million” specifically — and not as a preference for “better statistics” in the abstract?
• Did you name the neighbour, and the exact point where your rule stops being theirs?
• Did you keep the likelihood and the posterior apart in your own sentences? “The jury heard the chance she was innocent” is the exact swap this lesson exists to end.

Only now: compare with the three model paragraphs — two good ones, one bad one.

Done?

You can now:

• say the one-line rule and name its three words: prior, likelihood, posterior;
• compute a posterior by counting in whole people, and divide by the positives, not by everyone;
• say why a 99%-accurate test and a positive result can mean 9%;
• state the prosecutor’s fallacy as a swap of two named probabilities, and hear it in a courtroom sentence;
• turn a random-match probability and a database size into an expected number of chance matches;
• tell representativeness, base-rate neglect and the prosecutor’s fallacy apart;
• say where the prior comes from — and where, in a real case, that becomes the argument.

Go back to your sheet.