Forensic Risk Assessments Must Account for Cultural Considerations

Stephanie M. Shepherd Swinburne University of Technology
Roberto Lewis-Fernandez Columbia University and New York State Psychiatric Institute, New York, New York

A Canadian Federal court recently impugned the administering of 5 risk assessment instruments with Canadian Aboriginal prisoners. The ramifications of the ruling for the field are notable given the universal employment of risk instruments with Indigenous offenders and patients. Effectively, forensic clinicians and researchers can no longer overlook the role of culture in risk assessment—a robust academic dialogue on this subject matter is consequently warranted. This article explores how culture can shape the entire risk assessment process; from instrument construction and validation, to risk marker sensitivity, symptom articulation, and client-clinician interaction. Future directions for cross-cultural assessment are discussed.

violence risk assessment, Ewert v. Canada, cross-cultural mental health, transcultural psychiatry, forensic psychology

“On September 18, 2015 a Canadian court (Ewert v. Canada, 2015) strongly cautioned the continued use of five risk instruments (Hare Psychopathy Checklist Revised [PCL-R], Violence Risk Appraisal Guide [V-RAG], Sex Offender Risk Appraisal Guide [SORAG], Static 99, Violence Risk Scale—Sex Offender [VRSSO]) by the Correctional Service Canada (CSC) with Aboriginal inmates. The instruments evaluate risk for future violence (VRAG), risk for sexual violence/offending (SORAG, Static 99, VRS-SO), and the presence of psychopathic traits (PCL-R). The court concluded that the research base supporting the psychometric properties of the five above instruments with Canadian Aboriginal prisoners was currently inadequate. Additionally, actuarial tests were accepted by the court as susceptible to cultural biases and were therefore viewed as unreliable predictors of reoffense for Aboriginal inmates. By using such instruments, the CSC was found to be in breach of statutory obligations necessitating cultural responsiveness” (p. 427).

“The decision by the Canadian Federal Court has potentially wide-reaching ramifications for forensic risk assessment. The judicial rationale for this finding could, in theory, extend to other jurisdictions (i.e., United States, Australia, New Zealand, and United Kingdom) where actuarial instruments are used with Indigenous offenders. The Ewert v. Canada decision underscored an enduring concern in forensic assessment—that risk assessment instruments may not be cross-culturally suitable in their current state. While the court decision will undoubtedly galvanize academic inquiry into cross-cultural risk assessment, a developing research base has outlined apprehensions over the use of existing risk instruments with minority populations” (p. 427).

“Since their inception into clinical practice, risk instruments have predominantly assumed one of two templates: an actuarial design which encompasses a finite suite of predetermined risk items drawn from a construction sample of offenders or patients; or a Structured Professional Judgment (SPJ) model that enables clinical discretion to be anchored from a concert of risk factors identified in the risk literature. While the actuarial model minimizes clinical discretion in favor of statistical forecasting, the SPJ model provides a scientifically substantiated platform complemented by a rater’s contextual oversight. Both brands of risk assessment have more or less demonstrated equivalent predictive capacities for future offending” (p. 428).

“The risk instrument validation literature is extensive. The number of adult and adolescent risk instrument validation publications would exceed 1,000. However, validation samples predominantly comprise White participants. Of samples that include diverse populations, few compare within-sample validation estimates across participant cultural backgrounds, particularly for violent outcomes. The disproportionate focus on White offenders and patients contrasts with actual correctional populations where some minority groups (i.e., Indigenous; African American, Latino/as) are heavily overrepresented” (p. 428).

“Findings from the comparatively smaller number of investigations analyzing within-sample cross-cultural generalizability of risk instruments are inconclusive” (p. 428). Several research studies have found that, “Indigenous offenders regularly receive higher risk scores across several major adult and youth instruments [Static 99; LS/CMI; LSI-R; SAVRY; VRS-SO; YLS/CMI] compared with White offenders” and that risk assessment tools, “demonstrate reduced predictive validity for this group compared with White offenders” (p. 428). “Much less scientific attention has been afforded to African American and Latino/a offending populations in this space. The structural inequities and subsequent criminal justice contact experienced by both groups is well documented and a continual subject of forensic inquiry, yet this has not translated to comparable participant representation in risk validation investigations” (p. 428).

“This literature warrants several observations. First, evidence for the equivalent cross-cultural predictive validity of risk assessment instruments is unclear. The restricted and varied nature of the available findings precludes an assertion that instruments commensurately extend to minority groups. Moreover, such an assertion is perhaps premature given that minority groups do not receive the levels of representation in validation studies typically afforded to White populations. Second, while some studies have indicated that specific instruments demonstrate predictive accuracy for particular minority samples, this accuracy is recurrently poorer compared with White populations and necessitates interrogation. Third, the information derived from risk instruments often influences client liberties and treatment decisions—if instruments are consistently less accurate for particular cultural groups, then individuals from such groups may be legally and/or medically disadvantaged by their use. As such, cross-cultural biases that might unintendedly permeate the risk assessment process require deliberation” (p. 429).

“A client’s culture plays a pivotal role in the manifestation of risk. Culture determines behavioral norms and expectations, acceptable responses to threat, emotional presentation, modes of communication, goals, and motivations as well as explanations or remedies for illness, dysfunction, and delinquency. Moreover, culture helps to define what is a disease or harmful behavior. Risk assessment instruments are often crafted within the precepts of one cultural group, ignoring the broad experiential variability across diverse populations. Prediction estimates are often less accurate for ethnic minority groups, yet the notion of “close enough is good enough” enables the field to maintain current standards of risk assessment practice. This is unfortunate for some overrepresented, disadvantaged ethnic minority groups with histories of injustice and mistreatment by both criminal justice and mental health systems. Risk assessment instruments need to be reflective of the population they are measuring given the influence assessment information has on decision making impacting both public safety, offender treatment, and civil liberties. For these reasons all scientific efforts must be exhausted to ensure instruments are culturally fair, relevant, and nondiscriminatory. This article has outlined suggestions to facilitate such a process. The Ewert v. Canada decision should not be viewed as unreasonable by forensic clinicians and researchers, but rather as an opportunity to review and perhaps modify measures that may be unjustly impacting a portion of their clientele” (p. 434).

“Several actions can be undertaken to ensure that risk assessment is a fair and balanced exercise for all cultural groups concerned. The ultimate questions we must first examine are if, how and where culture impacts risk assessment. Put simply, is risk assessment compromised by cultural differences in (a) the underlying properties of instruments, (b) instrument item content, and (c) evaluator bias, or a combination of the three? These inquiries must be negotiated before contemplations of cultural renorming transpire” (p. 433).

“First, greater methodological sophistication is required to investigate factorial invariance in risk instruments. Rarely have risk instruments been subject to such testing. Without analysis of measurement equivalence, we cannot be sure if differences in mean scores across cultural groups are entirely attributed to differences between groups on the latent construct. To date, this connection has been largely assumed minus the necessary testing” (p. 433).

“Second, item content may need to be modified to mirror culturally specific explanations and manifestations of behavioral phenomena. This approach requires substantial input from multicultural health professionals and community members to ensure that item descriptions are in line with their experiential reality. This may involve the broadening of certain categories to encompass typical minority practices and understandings (i.e., family/kinship systems, cultural specific bereavement coping strategies), and in some cases demarcating ambiguous terminology (i.e., perceiving\hostile cues)… It is important that minority communities and stakeholders are engaged in the instrument development and validation processes. The specialized knowledge gained from cultural inclusion provides this exercise with a level of authenticity and credibility that existing tools have failed to secure” (p. 433).

“Third, AUC estimates must be complemented with additional predictive validity procedures. The comparison of regression slopes allows for the detection of significant differences in the outcome variable (offending) by equivalent increases in risk by cultural group. Positive and Negative predictive values also enable researchers to compare the prospective accuracy of high-risk and low-risk participants across culture” (p. 433).

“Our final suggestion pertains to a practitioner’s capacity to work effectively cross-culturally. Clinicians must be regularly educated in cultural competency or safety to ensure that risk assessment interviews gather relevant and meaningful information without unintentionally alienating, offending, or demeaning the client. A clinician’s cross-cultural aptitude can shape the risk assessment experience (i.e., patient candor or rapport, symptom interpretation, and risk precision) and should not be undervalued. Efforts to familiarize oneself with the practices, perspectives, values and belief systems of multicultural clientele can improve cultural awareness. An understanding of the historical injustices, structural inequalities and ongoing discrimination faced by minority clients, and the impact these incidents have on contemporary agency and wellbeing allows for a multilayered, contextualized assessment. A self-awareness of one’s own biases, privilege and assessor/patient power imbalance can also help minimize harmful heuristics and strengthen therapeutic relationships. The extent to which assessor biases play a role in cross-cultural risk assessment is an empirical question and warrants immediate academic scrutiny. This is particularly crucial given the capacity for assessor bias already identified in forensic assessment” (p. p. 434).

“These concerns affect both actuarial and SPJ modes of risk assessment in different ways. The rigidity of actuarial instruments (the norms of which are often sample-specific) could impede the identification of cultural idiosyncrasies. For SPJ instruments, the clinical liberty afforded to risk evaluation enables raters to potentially incorporate culture-specific information. However, without adequate training on culture-specific behavioral norms, biases (stereotypical or otherwise) could easily pervade the process. SPJ instruments also draw items from a body of empirical literature based on decades of research, though much of it framed by Western methodologies, belief systems, and behavioral expectations. Arguably, both forms of risk assessment may contain structural biases in their constitution. Furthermore, the discretionary element of SPJ instruments similarly enhances or prejudices an assessment. Either way, it is important to acknowledge that an ethnocentric “starting point” for instrument construction” (p. 429).

“One difficulty with this approach is the potential essentialization of culture. There is often greater heterogeneity within a culture than between cultures. However, ignoring culture entirely in risk assessment (a colorblind approach) could mean a deferral to dominant culture contexts and a diminishing of cultural experiences. In light of these concerns, a culturally themed instrument should still take a “person-centered” approach and avoid social categorization. If an individual identifies with a particular cultural group, we must examine his or her own interpretations of behavior, deviance and illness, rather than associating the person’s cultural identity as a proxy for a suite of traits and characteristics. It may be useful here to consider the inclusion of cultural interview guidelines as an addendum to risk instruments to bridge cultural distance. For example, the DSM–5 Cultural Formulation Interview is a supplementary set of questions that help clinicians understand a patient’s cultural experience of illness as part of a diagnostic assessment” (p. 433).

As always, please join the discussion below if you have thoughts or comments to add!