The role of prior probability in forensic assessments

Abstract

As the importance of forensic science in the legal system has grown, debate has arisen about the way forensic scientists should characterize their findings in order to communicate most effectively with legal fact-finders. This article will focus on one aspect of that debate: the framing of conclusions involving elements of probability. In particular, we will examine the contentious issue of whether forensic scientists, when asked to provide evidence that will be used to evaluate various competing propositions about physical evidence, should consider the prior probabilities that those propositions are true. Disputes about this issue have arisen in a number of contexts and recent examples suggest that opinions still diverge (e.g., Budowle et al., 2011; Biedermann et al., 2012). In this comment, we will argue that a reasoned approach to this issue depends on the role that forensic scientists are expected to play in the legal system. To illustrate the underlying issues, let us begin with a generic example. A forensic scientist is asked to perform DNA profiling analyses of blood found at a crime scene and to compare the result to the DNA profile of a defendant who is charged with the crime. The defendant's guilt or innocence will be determined by a jury. The jurors' decision will depend in part on their assessment of two propositions of interest—H1: that the defendant was the source of the blood; and H2: that someone else was the source of the blood. What should the forensic scientist tell the jurors about the results of the DNA analysis? The jurors might want the expert to tell them definitively which hypothesis is true, or to give them particular values for the so-called source probabilities—saying, for example, that there is a 0.998 probability the defendant is the source of the blood and only a probability of 0.002 that someone else was the source. But there is no way for the forensic scientist to reach such conclusions based on the forensic findings alone. To assess source probabilities, the forensic scientist must also consider other evidence in the case. Suppose, for example, that the expert found that the defendant and the blood from the crime scene share a set of genetic markers found in one person in 1 million in the relevant population. Without considering other evidence in the case, the expert might make statements about the conditional probability of finding these results under the two hypotheses of interest. For example, the expert might conclude that the shared genetic markers were virtually certain to be found under H1 (defendant was the source), but had only 1 chance in 1 million of being found under H2 (someone else was the source). Based on this assessment the expert might also provide to the jury a so-called likelihood ratio—saying, for example, that the DNA profiling results are 1 million times more probable if the defendant rather than some other person was the source of the blood. But a likelihood ratio is not the same thing as a source probability. The likelihood ratio reflects the relative probability of the findings under the relevant propositions, not the probability that the propositions are true. The only coherent way to draw conclusions about source probabilities on the basis of forensic evidence is to apply Bayes' rule, which requires that one begins with an assignment of prior probabilities to the propositions of interest (e.g., Robertson and Vignaux, 1995; Finkelstein and Fairley, 1970). Bayes' rule specifies how one ought to combine prior probabilities with the results of a DNA profiling analysis in order to find the so-called posterior probabilities that the defendant is the source of the blood. But the Bayesian approach will only work if the expert can begin with a prior probability. This brings us to the crux of the debate: whether forensic scientists should even try to specify prior probabilities and, if so, how. It is occasionally suggested that forensic scientists should assume equal prior probabilities. This is sometimes described as a position of neutrality and is often justified with references to vague accessory “principles,” such as the “Principle of Indifference” or the “Principle of Maximum Entropy,” borrowed from other disciplines and contexts (Biedermann et al., 2007). A prominent illustration can be found in paternity cases. When DNA analysts are asked to assist in the assessment of whether a particular man is the father of a child, they usually analyze the profiles of the mother, child, and the accused man, and assign conditional probabilities that the genetic characteristics found in the child (Ec) would be observed under two relevant hypotheses specifying that the accused is the father (H1) and that some other man (from a particular reference population) is the father (H2) conditioned on the alleged parents' DNA profiles (Em and Eam, for the mother and the accused man, respectively). In some cases, the analysts limit themselves to reporting the ratio of these conditional probabilities—i.e., Pr(Ec|Em,Eam,H1)/Pr(Ec|Em,H2)—which is a likelihood ratio (although it is also referred to as the paternity index). But quite often, analysts go farther. They assume that the prior odds of H1 and H2 are equal and then, in accordance with Bayes' rule, they multiply the prior odds by the likelihood ratio (paternity index) to determine the posterior odds of paternity. Recall that odds are defined as a ratio between two probabilities; in this particular scenario, it is the ratio between Pr(H1) and Pr(H2). The posterior odds are typically restated as a probability. For example, if the DNA evidence supports paternity with a likelihood ratio of 1 million some analysts would report a probability of 0.999999 that the accused is the father. While this approach is commonly used in civil paternity cases, courts in the United States have generally not allowed analysts to characterize...