Both numeric (e.g., “I’m 95% conﬁdent”) and verbal (e.g., “I’m absolutely sure”) conﬁdence statements, collected using best practices, provide diagnostic information about eyewitness accuracy. This is the bottom line of a recently published article in Law and Human Behavior. Below is a summary of the research and findings as well as a translation of this research into practice.
Featured Article | Law and Human Behavior | 2021, Vol. 45, No. 2, 138-151
Laura Smalarz, Arizona State University
Yueran Yang, University of Nevada – Reno
Gary L. Wells, Iowa State University
Objectives: We assessed recent policy recommendations to collect eyewitnesses’ conﬁdence statements in witnesses’ own words as opposed to numerically. We conducted an experiment to test whether eyewitnesses’ free-report verbal conﬁdence statements are as diagnostic of eyewitness accuracy as their numeric conﬁdence statements and whether the diagnostic utility of eyewitnesses’ verbal and numeric conﬁdence statements varies across witnessing conditions. Hypotheses: We hypothesized that eyewitnesses’ verbal and numeric conﬁdence statements are both signiﬁcantly associated with identiﬁcation accuracy among choosers and that their diagnostic utility holds across varying witnessing conditions. Method: In the ﬁrst phase of the experiment, eyewitnesses (N = 4,795 MTurkers; 48.8% female; 50.8% male; .3% other; age M = 36.9) viewed a videotaped mock- crime and made an identiﬁcation decision from a culprit-present or culprit-absent lineup. We manipulated witnessing conditions at encoding and retrieval to obtain varied levels of memory performance. In the second phase of the experiment, evaluators (N = 456 MTurkers; 35.5% female; 62.7% male .4% other; age M = 36.5) translated witnesses’ verbal conﬁdence statements to a numeric estimate and we used calibration and conﬁdence-accuracy characteristic analyses to compare the diagnosticity of witnesses’ verbal and numeric conﬁdence statements across the two levels of memory performance. Results: Witnesses’ verbal and numeric conﬁdence statements were signiﬁcantly and nondifferentially diagnostic of eyewitness accuracy for both choosers and nonchoosers, and their diagnostic utility held across variations in witnessing conditions. Conclusions: These ﬁndings suggest the applied utility of collecting either verbal or numeric conﬁdence statements from eyewitnesses immediately following an identiﬁcation decision.
eyewitness identification, eyewitness confidence, confidence-accuracy calibration, confidence- accuracy characteristic analysis
Summary of the Research
“The results of the current experiment suggest that eyewitnesses’ verbal and numeric conﬁdence statements provide diagnostic information about eyewitness accuracy for both choosers and for nonchoosers, and that the diagnostic utility of eyewitnesses’ verbal and numeric conﬁdence statements holds across variations in witnessing conditions. These ﬁndings contribute to a growing literature showing that eyewitnesses’ verbal and numeric conﬁdence statements are similarly informative regarding eyewitness accuracy. Theoretically, these ﬁndings suggest that any potential interpretive difﬁculties associated with verbal communications of uncertainty may be counteracted by the capacity of verbal statements to more accurately tap witnesses’ underlying feelings of certainty. This is consistent with the idea that people are largely “pre-Bernoullian” in their reasoning about uncertainty and are naturally inclined to provide uncertainty estimates verbally rather than numerically” (p. 147-148).
“We were surprised by the results of our pilot experiments showing that conﬁdence was similarly diagnostic of accuracy regardless of whether it was presented to evaluators in witnesses’ own words and voice, as a verbatim written representation, or in a coarse summary for- mat. After all, witnesses’ audiotaped statements contained substantially more information than the transcribed and summarized statements. These results suggest that any additional cues that were available to evaluators in the audio recordings were not particularly informative regarding eyewitness accuracy. Our analyses of witnesses’ statement characteristics, presented in the online supplemental materials, corroborate this interpretation. Nevertheless, future research might examine the diagnostic utility of videotaped eyewitness conﬁdence statements, as video recording eyewitness identiﬁcation procedures is recommended by eyewitness researchers. Although our ﬁndings suggest that even a summary statement of witnesses’ conﬁdence can be useful for discerning eyewitness accuracy, it is essential that the summary statement represent a faithful re- cord of the witness’s conﬁdence and not a subjective interpretation biased by a nonblind lineup administrator’s knowledge of who the suspect is. Indeed, knowledge of which lineup member is the suspect affects interpretations of ambiguous eyewitness statements during the identiﬁcation procedure and interpretations of witnesses’ verbal conﬁdence statements following an identiﬁcation” (p. 148).
“More than 100 years after Hugo Munsterburg’s wholesale dismissal of eyewitness conﬁdence, we know not only that witnesses’ numeric conﬁdence statements are diagnostic of identiﬁcation accuracy but that so too are their verbal conﬁdence statements, even when provided in witnesses’ own words. Moreover, eyewitnesses’ numeric and verbal conﬁdence statements are diagnostic of accuracy among choosers as well as nonchoosers, regardless of whether evaluators are presented with the conﬁdence statement in the witnesses’ own words or in a written transcription or summary, and across at least some variations in witnessing conditions. These ﬁndings suggest that recent efforts to reform police procedures for collecting eye- witness identiﬁcation evidence (e.g., United States Department of Justice, 2017) have not erred in recommending that witnesses pro- vide conﬁdence statements in their own words. Collecting eyewitness conﬁdence statements under pristine, carefully-controlled conditions—regardless of whether the statements are reported verbally or numerically—remains a key component of legal advances in the treatment of eyewitness identiﬁcation evidence” (p. 150).
Translating Research into Practice
“Our ﬁndings indicate that eyewitnesses’ expressions of verbal and numeric conﬁdence provide diagnostic information to evaluators about an eyewitness’s likely accuracy. It is critical, however, that eyewitnesses’ conﬁdence statements are collected in a pristine manner. In the current research, witnesses received proper prelineup instructions warning that the actual culprit might not be present, the lineup was not biased against the suspect, and conﬁdence was collected immediately by a double-blind administrator (pilot experiments) or by a computer program (main experiment). It is only if best-practices are used to collect eyewitness evidence that eyewitnesses’ expressions of conﬁdence can be relied on to provide diagnostic information about accuracy” (p. 149).
“Eyewitness researchers have recently begun to place a premium on suspect identiﬁcations made with high conﬁdence. It has been argued that high-conﬁdence suspect identiﬁcations are highly likely to be accurate and that the accuracy of high-conﬁdence suspect identiﬁcations holds across variations in memory strength. Criticisms to these claims not- withstanding, high-conﬁdence suspect identiﬁcation accuracy was indeed high in the current research, with approximately 95% of suspect identiﬁcations being accurate among witnesses who expressed at least 80% numeric conﬁdence or whose translated verbal conﬁdence statements were associated with at least 80% conﬁdence. These ﬁndings suggest that that members of the legal system can trust suspect identiﬁcations made by highly conﬁdent eyewitnesses, regardless of whether the conﬁdence is reported verbally or numerically. But there is an important caveat to the legal system’s treatment of eyewitness’s verbal conﬁdence statements: The legal system must somehow determine which verbal conﬁdence statements constitute ‘high conﬁdence.’ This may prove challenging. An inspection of witnesses’ high-conﬁdence verbal statements in the current research indicated that witnesses who reported numeric conﬁdence of at least 80% made verbal conﬁdence statements ranging from absolutely, completely, or extremely conﬁdent to very, pretty, or mostly conﬁdent, and even down to fairly or somewhat conﬁdent. Hence, it may be difﬁcult for the legal system to determine a precise cutoff for what constitutes a high level of verbal conﬁdence. Moreover, research suggests that contextual information, such as eyewitnesses’ justiﬁcations for their identiﬁcation decisions, information about whether the eyewitness picked the suspect or a ﬁller, and information about the speed of the eyewitness’s identiﬁcation decision can inﬂuence interpretations of eyewitnesses’ verbal conﬁdence statements, further complicating legal determinations of what constitutes a high-verbal-conﬁdence identiﬁcation” (p. 149).
Other Interesting Tidbits for Researchers and Clinicians
“Across all of our experiments, evaluators’ translated estimates of witnesses’ verbal conﬁdence statements tended to systematically underestimate witnesses’ own numeric conﬁdence. This was evident from a direct comparison of evaluators’ translated estimates to those same witnesses’ numeric conﬁdence statements (in the pilot experiments) as well in the between-subjects sample in the main experiment. We can conceive of a few possible explanations for this ﬁnding” (p. 148).
“One possibility is that providing a conﬁdence statement verbally as opposed to numerically reduces witnesses’ conﬁdence… [A]nalyses from the witnessing phase of both sets of experiments indicated that, if anything, witnesses reported greater numeric conﬁdence after making a verbal conﬁdence statement than before making a verbal conﬁdence statement. Hence, providing a conﬁdence statement verbally as opposed to numerically does not appear to reduce witnesses’ conﬁdence” (p. 148).
“A second possibility is that evaluators downgraded their numeric estimates of witnesses’ conﬁdence because witnesses’ verbal conﬁdence statements contained indicators of uncertainty (e.g., linguistic hesitations; delay to respond). ... Even among evaluators who assessed the summary version of witnesses’ verbal conﬁdence statements—in which these contextual elements were absent—underestimation of witnesses’ numeric conﬁdence based on witnesses’ verbal statements still occurred” (p. 148).
“A third possibility is that evaluators’ numeric estimates of witnesses’ verbal conﬁdence statements were moderated because evaluators were not subject to the same motivational biases that may have affected witnesses’ own conﬁdence reports. Research has demonstrated that decision-makers sometimes exhibit greater overconﬁdence than do evaluators of those decisions. However, the tendency for eyewitnesses’ verbal conﬁdence statements to be perceived as less conﬁdent than their numeric statements was also observed when both conﬁdence statements were interpreted by evaluators: In Pilot Experiment 1b, evaluators tended to be less likely to believe witnesses based on their verbal conﬁdence statements than based on their numeric conﬁdence statements, though this effect was not statistically signiﬁcant” (p. 148).
“A ﬁnal possibility is that there is something inherently less compelling about verbal conﬁdence statements compared with numeric conﬁdence statements…Regardless of the particular mechanism responsible for evaluators’ tendency to underestimate witnesses’ numeric conﬁdence based on witnesses’ verbal conﬁdence reports, however, this underestimation did not differentially affect evaluations of accurate and inaccurate eyewitnesses and thus did not compromise the diagnostic utility of the verbal conﬁdence statements” (p. 148).
Join the Discussion
As always, please join the discussion below if you have thoughts or comments to add!