What Works and What Doesn’t – Bias Awareness and Correction Strategies Among Forensic Evaluators

Tess M.S. Neal, Arizona State University
Stanley L. Brodsky, The University of Alabama

A qualitative study with 20 board-certified forensic psychologists was followed up by a mail survey of 351 forensic psychologists in this mixed-methods investigation of examiner bias awareness and strategies used to debias forensic judgments. Rich qualitative data emerged about awareness of bias, specific biasing situations that recur in forensic evaluations, and potential debiasing strategies. The continuum of bias awareness in forensic evaluators mapped cogently onto the “stages of change” model. Evaluators perceived themselves as less vulnerable to bias than their colleagues, consistent with the phenomenon called the “bias blind spot.” Recurring situations that posed challenges for forensic clinicians included disliking or feeling sympathy for the defendant, disgust or anger toward the offense, limited cultural competency, preexisting values, colleagues’ influences, and protecting referral streams. Twenty-five debiasing strategies emerged in the qualitative study, all but 1 of which rated as highly useful in the quantitative survey. Some of those strategies are consistent with empirical evidence about their effectiveness, but others have been shown to be ineffective. We identified which strategies do not help (such as introspection), focused on promising strategies with empirical support, discussed additional promising strategies not mentioned by participants, and described new strategies generated by these participants that have not yet been subjected to empirical examination. Finally, debiasing strategies were considered with respect to future directions for research and forensic practice.

bias, forensic, expert judgment, decision making, mixed methods, qualitative

“A historical controversy has existed in the legal and psychological literature regarding whether objectivity on part of the expert is possible. Social psychological literature attests to the difficulty people may have in divorcing their decisions from cognitive and emotional biases, and experts are not immune from these biases. Some researchers suggest forensic mental health evaluators underestimate the prevalence and severity of such influences on their work, but the literature does not address forensic clinicians’ personal experiences with bias or how they try to correct for them” (p. 58). “To address these gaps in the literature, the present study was designed to explore forensic clinicians’ experiences with and perceptions of biases, and to investigate the strategies they use to try to mitigate perceived biases. A few suggestions exist for how forensic clinicians might consider the impact of bias, such as actively generating alternative conclusions, identifying and using relevant base rates, minimizing the role of memory, and identifying and weighing the most valid sources of data. However, much remains to be learned about strategies used to manage bias” (p. 59).
“We asked what influences might bias clinicians in the forensic context, and what strategies forensic clinicians would report using to reduce bias. Qualitative methods were used in the first study, as little previous literature has asked these kinds of questions. We followed up with a complementary large-scale survey of forensic mental health professionals to generate more representative answers to our questions, asking participants to rate the perceived usefulness of various bias-correction strategies that emerged from the qualitative analysis” (p. 60). The first, qualitative study concluded that awareness of bias falls on a continuum. “Some of the clinicians immediately dismissed the possibility of bias in their work. For these psychologists, the objectivity mandate in forensic work may be so salient and accessible that it generates defensiveness when thinking or talking about any possibility of bias. For others, the topic may be less threatening, and they were able to reflect on the possibility of bias in their work, with some clinicians able to identify specific areas of potential bias in their work” (p. 63). Another interesting finding was that, “Participants had no trouble identifying bias in their colleagues, but fewer reported ever having any concern about their own potential biases… These results are consistent with research showing that people perceive themselves as less vulnerable to bias than others. Pronin [and colleagues] found that this ‘bias blind spot’ persisted even when people were explicitly taught how various specific biases could have affected their assessments” (p. 69).
The second, quantitative study used a survey to assess the use of debiasing strategies of 351 forensic psychologists. The items on this survey included the 25 debiasing strategies generated during study 1 including: critically examining conclusions, basing conclusions on sound data, taking careful notes during evaluation, reading professional literature, consulting with colleagues, continuous introspection about personal biases, emotional disengagement, and many other methods for counteracting bias. Most of these strategies were endorsed as being useful by the participants in the second study. While some of these strategies have been identified by relevant literature as useful techniques (i.e., professional training, taking time to think before writing a report, consulting colleagues, using structured evaluation methods), other strategies suggested by forensic mental health professionals, such as introspection, have been deemed ineffective by research. “Consistent with what the social psychological literature about the bias blind spot would suggest, all of the participants in the qualitative study emphasized introspection as their primary strategy for identifying potential biases. Introspection was similarly one of the highest rated strategies in the follow-up quantitative study with a much larger and more representative sample of forensic clinicians. Unfortunately, introspection is not a realistic strategy for debiasing success. Classic psychological science shows that people have little or no direct introspective access to higher-order cognitive processes. In fact, Pronin and Kugler described introspection as a cognitive illusion that actually functions as a source of the bias blind spot. They showed that people rely on overt behavior to assess bias in other people but that they look inward for biased motives when assessing for bias in themselves.… Forensic clinicians may believe that they can identify and then work on their biases after identifying them via introspection, however, the common cognitive “bias blind spot” is likely to prevent the success of this endeavor, as described previously. After engaging in the ineffective strategy of introspection, forensic clinicians may develop a false confidence that they are bias-free, a confidence they may convey to the courts. Pronin and Kugler did report one encouraging finding for forensic mental health professionals. The one situation in which their participants ceased denying their relative susceptibility to bias was when they were educated about the fallibility of introspection” (p. 72).

“…participants described the importance of training about “objectivity” in the abstract, whereas research indicates educational curricula need to focus concretely on how humans make decisions and what can go awry (and why) to educate clinicians effectively about bias and correction strategies. Graduate school coursework, internship and postdoctoral didactics, and continuing education workshops should explicitly focus on the psychology of decision making to train clinicians how bias might affect their work and what they can do about it” (p. 72).
“One of the strategies that emerged in our qualitative study was examining patterns of personal decision making (e.g., agreement with referral party preferences), which was also rated as useful in the larger survey. This strategy represents a behavioral marker forensic clinicians could use to examine their potential biases rather than introspection. These findings suggest that educational curricula in forensic psychology might focus directly on the reasons why introspection is not a useful bias recognition or mitigation strategy and stress attention to behavioral markers instead of introspection to examine one’s own biases” (p. 72-73).

“In addition to ideas for future directions as discussed above, other future directions include investigating individual differences in biases and bias awareness. For example, how do individual differences in personality traits (e.g., openness to experience) and cognitive styles (e.g., rational vs. experiential modes of thinking) relate to biases and bias awareness? Is the “size” or “strength” of an individual forensic clinician’s bias blind spot systematically related to level of confidence, perhaps with larger or stronger blind spots associated with overconfidence? Neal and Brodsky showed that forensic clinicians with higher occupational socialization scores were more likely to believe in their ability to be objective in forensic work. Four questions follow: How does the occupational socialization process affect the perceived need for relying on bias-correction strategies? How does the strength of belief in one’s objectivity relate to the perceived need for implementing strategies to reduce bias (for oneself and others)? Might clinicians with more pride in their professional identity be paradoxically more biased because of desire for and confidence in objectivity? Does confidence in one’s ability to be objective paradoxically increase bias by preventing the use of strategies to mitigate bias?” (p. 73).
“If a single overarching research need calls out from these two studies, it is to investigate systematically the degree to which the promising strategies actually reduce bias. Once that becomes known, then the task is to mobilize ways in which such effective strategies become part of routine practice. The field is ripe for controlled and responsible study of issues of bias and generating empirically supported methods for improving clinical judgment and decision-making. On that foundation can we develop pedagogical and workplace interventions for implementing accountability and reducing bias in assessments” (p. 73).

As always, please join the discussion below if you have thoughts or comments to add!