How reliable is forensic evidence?

AnalysisTwo opposite errors circulate about forensic evidence, and both are damaging. One treats a laboratory result as settling a case. The other treats the existence of any limitation as showing that the discipline is worthless. A reader who holds either will misread every forensic finding they encounter.

Start with the framing the reviewers themselves use. The final NIST scientific foundation review of DNA mixture interpretation opens: "All scientific methods have limits. To use a method appropriately, one must understand those limits, which are inevitably tied to the risk one is willing to accept either as an individual or as a society."

The same report supplies the sharpest available illustration of why a single reliability figure is impossible. For high-quality single-source samples, it records, it has been demonstrated that different laboratories will arrive at the same result regardless of the specific instruments, tests and software used. For mixtures, it records the opposite: multiple interlaboratory studies conducted by different groups over the past two decades have demonstrated that different laboratories can produce a wide range of results when interpreting the same DNA mixtures.

The review also explains why the difficult case became common rather than exceptional. In the 1990s an evidence sample needed to contain thousands of cells, such as from a visible blood or semen stain. Analysts can now extract a profile from the few skin cells someone might leave when handling an object — which extended the technique’s usefulness and, at the same time, made mixtures ordinary.

Sensitivity brings its own interpretive problem. People constantly shed small amounts of DNA into the environment, and by touching objects can transfer small amounts — including someone else’s DNA — from one surface to another. Where more than one contributor is detected, it can be difficult to distinguish one person’s DNA from another, to estimate how many people contributed, to determine whether the DNA is relevant to the crime or is contamination, and to determine whether a trace amount from a particular person is present at all.

Reviews of this kind are a programme rather than an event. NIST states that its scientific foundation reviews identify the scientific foundations underpinning forensic methods, evaluate the empirical evidence for their reliability, explore capabilities and limitations, and identify knowledge gaps; that the method is to gather literature and publicly available information, convene NIST scientists and outside experts, publish a draft for public comment, and finalise after considering comments; and that the programme answers a call in the 2009 National Academy of Sciences report for studies establishing the scientific bases demonstrating the validity of forensic methods. Congress appropriated funds for the reviews from 2018.

• Common belief: DNA evidence is the gold standard, so it is reliable in every case.

  In practice: The final NIST review records that different laboratories will arrive at the same result on high-quality single-source samples, and that interlaboratory studies over two decades have shown they can produce a wide range of results interpreting the same mixtures. Which kind of sample it is changes the answer.

• Common belief: Forensic science has been shown to be unreliable.

  In practice: That is not what the reviews found; it overstates them in the opposite direction. They describe capabilities as well as limitations, and NIST states that forensic science can exclude innocent people from investigations and help exonerate them in cases of wrongful conviction.

• Common belief: More sensitive methods make forensic conclusions safer.

  In practice: Increased sensitivity extended what DNA analysis can be used for and also made mixtures ordinary, because people shed DNA constantly and can transfer someone else’s from surface to surface. A better detector can produce a harder interpretive problem.

• Common belief: A published review settles what a discipline is worth.

  In practice: Some reviews are final and some are drafts under comment. The DNA mixture review is final; the bitemark analysis review is still a draft, and nothing on this platform is drawn from that draft.

• Common belief: Stating limits means the evidence should be ignored.

  In practice: The reviewers put it the other way: all scientific methods have limits, and using a method appropriately requires understanding them. Stating a limit is a condition of relying on a result, not a reason to discard it.

What varies is not confidence but the thing being asked about.

• The same method on a high-quality single-source sample: reproducible between laboratories.
• The same method on a mixture: a wide range of results between laboratories, shown repeatedly over two decades.
• Across disciplines: the 2009 National Academy of Sciences review found substantial variation in scientific foundation, standardisation and quality assurance — a finding about the United States as of 2009.

AnalysisLimits are what make a forensic conclusion examinable. A result presented with its uncertainty can be tested against the evidence for that uncertainty; a result presented as certainty offers a court nothing to weigh and a defence nothing to challenge.

That is why the honesty of reporting has mattered more than technical advance. The NIST review says as much: these challenges need to be carefully considered throughout the forensic science process and clearly communicated when describing forensic results, and failure to do so can lead to misunderstandings about the strength and relevance of the evidence.

• The specific findings here come from one final scientific-foundation review, of one discipline, published by a United States federal body. They are not evidence about other disciplines and not evidence about practice in other countries.
• This page describes what published reviews found. It does not assess any laboratory, any provider, or any case.

Related: what DNA analysis establishes, what forensic science cannot decide, and what courts do.