Making Determinations About the Accuracy of Eyewitnesses’ Identifications of Criminal Suspects Based on the Level of Confidence Expressed by Eyewitnesses.

Laura Smalarz, Arizona State University
Yueran Yang, University of Nevada – Reno
Gary L. Wells, Iowa State University

Objectives: We assessed recent policy recommendations to collect eyewitnesses’ confidence statements in witnesses’ own words as opposed to numerically. We conducted an experiment to test whether eyewitnesses’ free-report verbal confidence statements are as diagnostic of eyewitness accuracy as their numeric confidence statements and whether the diagnostic utility of eyewitnesses’ verbal and numeric confidence statements varies across witnessing conditions. Hypotheses: We hypothesized that eyewitnesses’ verbal and numeric confidence statements are both significantly associated with identification accuracy among choosers and that their diagnostic utility holds across varying witnessing conditions. Method: In the first phase of the experiment, eyewitnesses (N = 4,795 MTurkers; 48.8% female; 50.8% male; .3% other; age M = 36.9) viewed a videotaped mock- crime and made an identification decision from a culprit-present or culprit-absent lineup. We manipulated witnessing conditions at encoding and retrieval to obtain varied levels of memory performance. In the second phase of the experiment, evaluators (N = 456 MTurkers; 35.5% female; 62.7% male .4% other; age M = 36.5) translated witnesses’ verbal confidence statements to a numeric estimate and we used calibration and confidence-accuracy characteristic analyses to compare the diagnosticity of witnesses’ verbal and numeric confidence statements across the two levels of memory performance. Results: Witnesses’ verbal and numeric confidence statements were significantly and nondifferentially diagnostic of eyewitness accuracy for both choosers and nonchoosers, and their diagnostic utility held across variations in witnessing conditions. Conclusions: These findings suggest the applied utility of collecting either verbal or numeric confidence statements from eyewitnesses immediately following an identification decision.

eyewitness identification, eyewitness confidence, confidence-accuracy calibration, confidence- accuracy characteristic analysis

“The results of the current experiment suggest that eyewitnesses’ verbal and numeric confidence statements provide diagnostic information about eyewitness accuracy for both choosers and for nonchoosers, and that the diagnostic utility of eyewitnesses’ verbal and numeric confidence statements holds across variations in witnessing conditions. These findings contribute to a growing literature showing that eyewitnesses’ verbal and numeric confidence statements are similarly informative regarding eyewitness accuracy. Theoretically, these findings suggest that any potential interpretive difficulties associated with verbal communications of uncertainty may be counteracted by the capacity of verbal statements to more accurately tap witnesses’ underlying feelings of certainty. This is consistent with the idea that people are largely “pre-Bernoullian” in their reasoning about uncertainty and are naturally inclined to provide uncertainty estimates verbally rather than numerically” (p. 147-148).

“We were surprised by the results of our pilot experiments showing that confidence was similarly diagnostic of accuracy regardless of whether it was presented to evaluators in witnesses’ own words and voice, as a verbatim written representation, or in a coarse summary for- mat. After all, witnesses’ audiotaped statements contained substantially more information than the transcribed and summarized statements. These results suggest that any additional cues that were available to evaluators in the audio recordings were not particularly informative regarding eyewitness accuracy. Our analyses of witnesses’ statement characteristics, presented in the online supplemental materials, corroborate this interpretation. Nevertheless, future research might examine the diagnostic utility of videotaped eyewitness confidence statements, as video recording eyewitness identification procedures is recommended by eyewitness researchers. Although our findings suggest that even a summary statement of witnesses’ confidence can be useful for discerning eyewitness accuracy, it is essential that the summary statement represent a faithful re- cord of the witness’s confidence and not a subjective interpretation biased by a nonblind lineup administrator’s knowledge of who the suspect is. Indeed, knowledge of which lineup member is the suspect affects interpretations of ambiguous eyewitness statements during the identification procedure and interpretations of witnesses’ verbal confidence statements following an identification” (p. 148).

“More than 100 years after Hugo Munsterburg’s wholesale dismissal of eyewitness confidence, we know not only that witnesses’ numeric confidence statements are diagnostic of identification accuracy but that so too are their verbal confidence statements, even when provided in witnesses’ own words. Moreover, eyewitnesses’ numeric and verbal confidence statements are diagnostic of accuracy among choosers as well as nonchoosers, regardless of whether evaluators are presented with the confidence statement in the witnesses’ own words or in a written transcription or summary, and across at least some variations in witnessing conditions. These findings suggest that recent efforts to reform police procedures for collecting eye- witness identification evidence (e.g., United States Department of Justice, 2017) have not erred in recommending that witnesses pro- vide confidence statements in their own words. Collecting eyewitness confidence statements under pristine, carefully-controlled conditions—regardless of whether the statements are reported verbally or numerically—remains a key component of legal advances in the treatment of eyewitness identification evidence” (p. 150).

“Our findings indicate that eyewitnesses’ expressions of verbal and numeric confidence provide diagnostic information to evaluators about an eyewitness’s likely accuracy. It is critical, however, that eyewitnesses’ confidence statements are collected in a pristine manner. In the current research, witnesses received proper prelineup instructions warning that the actual culprit might not be present, the lineup was not biased against the suspect, and confidence was collected immediately by a double-blind administrator (pilot experiments) or by a computer program (main experiment). It is only if best-practices are used to collect eyewitness evidence that eyewitnesses’ expressions of confidence can be relied on to provide diagnostic information about accuracy” (p. 149).

“Eyewitness researchers have recently begun to place a premium on suspect identifications made with high confidence. It has been argued that high-confidence suspect identifications are highly likely to be accurate and that the accuracy of high-confidence suspect identifications holds across variations in memory strength. Criticisms to these claims not- withstanding, high-confidence suspect identification accuracy was indeed high in the current research, with approximately 95% of suspect identifications being accurate among witnesses who expressed at least 80% numeric confidence or whose translated verbal confidence statements were associated with at least 80% confidence. These findings suggest that that members of the legal system can trust suspect identifications made by highly confident eyewitnesses, regardless of whether the confidence is reported verbally or numerically. But there is an important caveat to the legal system’s treatment of eyewitness’s verbal confidence statements: The legal system must somehow determine which verbal confidence statements constitute ‘high confidence.’ This may prove challenging. An inspection of witnesses’ high-confidence verbal statements in the current research indicated that witnesses who reported numeric confidence of at least 80% made verbal confidence statements ranging from absolutely, completely, or extremely confident to very, pretty, or mostly confident, and even down to fairly or somewhat confident. Hence, it may be difficult for the legal system to determine a precise cutoff for what constitutes a high level of verbal confidence. Moreover, research suggests that contextual information, such as eyewitnesses’ justifications for their identification decisions, information about whether the eyewitness picked the suspect or a filler, and information about the speed of the eyewitness’s identification decision can influence interpretations of eyewitnesses’ verbal confidence statements, further complicating legal determinations of what constitutes a high-verbal-confidence identification” (p. 149).

“Across all of our experiments, evaluators’ translated estimates of witnesses’ verbal confidence statements tended to systematically underestimate witnesses’ own numeric confidence. This was evident from a direct comparison of evaluators’ translated estimates to those same witnesses’ numeric confidence statements (in the pilot experiments) as well in the between-subjects sample in the main experiment. We can conceive of a few possible explanations for this finding” (p. 148).

“One possibility is that providing a confidence statement verbally as opposed to numerically reduces witnesses’ confidence… [A]nalyses from the witnessing phase of both sets of experiments indicated that, if anything, witnesses reported greater numeric confidence after making a verbal confidence statement than before making a verbal confidence statement. Hence, providing a confidence statement verbally as opposed to numerically does not appear to reduce witnesses’ confidence” (p. 148).

“A second possibility is that evaluators downgraded their numeric estimates of witnesses’ confidence because witnesses’ verbal confidence statements contained indicators of uncertainty (e.g., linguistic hesitations; delay to respond). ... Even among evaluators who assessed the summary version of witnesses’ verbal confidence statements—in which these contextual elements were absent—underestimation of witnesses’ numeric confidence based on witnesses’ verbal statements still occurred” (p. 148).

“A third possibility is that evaluators’ numeric estimates of witnesses’ verbal confidence statements were moderated because evaluators were not subject to the same motivational biases that may have affected witnesses’ own confidence reports. Research has demonstrated that decision-makers sometimes exhibit greater overconfidence than do evaluators of those decisions. However, the tendency for eyewitnesses’ verbal confidence statements to be perceived as less confident than their numeric statements was also observed when both confidence statements were interpreted by evaluators: In Pilot Experiment 1b, evaluators tended to be less likely to believe witnesses based on their verbal confidence statements than based on their numeric confidence statements, though this effect was not statistically significant” (p. 148).

“A final possibility is that there is something inherently less compelling about verbal confidence statements compared with numeric confidence statements…Regardless of the particular mechanism responsible for evaluators’ tendency to underestimate witnesses’ numeric confidence based on witnesses’ verbal confidence reports, however, this underestimation did not differentially affect evaluations of accurate and inaccurate eyewitnesses and thus did not compromise the diagnostic utility of the verbal confidence statements” (p. 148).

As always, please join the discussion below if you have thoughts or comments to add!