YLS/CMI cross-cultural validity uncertain for low-risk Indigenous youth

Stephane M. Shepard, Forensic Behavioral Science, University of Technology, Melbourne, Australia
Jay P. Singh, Global Institute of Forensic Research, Reston, Virginia; Faculty of Health Sciences, Molde University College, Molde, Norway, Department of Psychology, Universitat Konstanz, Konstanz, Germany
Rachael Fullam, Centre for Forensic Behavioural Science; Victorian Institute of Forensic Mental Health (Forensicare), Melbourne, Australia

Few studies have explored the cross-cultural utility of youth violence risk instruments. Moreover both the discrimination and calibration performance indicators of such instruments are rarely investigated. This study aimed to address both gaps in the literature by exploring the predictive validity of the YLS/CMI instrument for an Australian multi-ethnic cohort of young offenders in custody. The YLS/CMI total score was able to discriminate between reoffenders and non-reoffenders for the overall sample; however, a breakdown across ethnicity showed only strong effects for Australian English Speaking Background youth. Despite an inability to distinguish re-offenders from non-reoffenders for both Culturally and Linguistically Diverse and Indigenous youth, the instrument was able to accurately predict recidivism for high-risk youth for all ethnic groups. Findings highlight the importance of adopting both calibration and discrimination indicators when assessing predictive validity.

YLS/CMI, re-offending, violence risk assessment, forensic psychology, risk assessment

“The risk-need-responsivity (RNR) model is the dominant offender assessment and rehabilitation model in correctional systems around the globe. A key to this model is identifying higher risk individuals such that additional resources can be allocated to them. Criminogenic needs which are dynamic factors linked to offending behaviours are targeted for treatment so that an individual’s level of risk is reduced. The success of the RNR model spawned the development of a number of risk instruments which help practitioners design informed treatment strategies after identifying a client’s needs and level of risk. Only one of these instruments was developed for use with young offenders, namely the Youth Level of Service/Case Management Inventory” (p. 193).

“The YLS/CMI is a general risk/needs instrument for young offenders aged 12–17 years. It comprises static and dynamic risk factors encompassing previous offending behaviours, environmental influences, substance use and antisocial attitudes and behaviours. The YLS/CMI total score has demonstrated moderate associations with young offender recidivism across a variety of international correctional settings. A meta-analysis of 22 YLS/CMI related studies found that the instrument predicted general recidivism and to a lesser extent violent recidivism… Despite these results, a dearth of research has examined instrument’s cross-cultural generalizability… Risk assessment instruments must be able to demonstrate utility across cultural groups to avoid misclassification and subsequent correctional mismanagement. This is of critical importance for Indigenous Australians who have historical systematic discrimination, particularly within Australian government institutions including health and justice… Failure to validate risk instruments for minority groups could have deleterious repercussions given contemporary research pointing to the growing use and influence of the YLS/CMI during legal decision-making at various levels. Regional validity is of further value given that YLS/CMI validation studies conducted outside of North America have been shown to obtain lower predictive accuracy estimates compared to North American studies” (pp.193-195).

“This study aims to address these significant gaps in the literature by exploring the comparative predictive and discriminative ability of the YLS/CMI for three ethnic subgroups in a representative young Australian custodial population. Although this is, to our knowledge, the first Australian study to ascertain the cross-cultural validity of the YLS/CMI in custodial settings, we anticipate potential differences in item and total scores across ethnicity…The final sample comprised 207 participants remanded or serving a custodial order in two Youth Justice Centres in the Australian state of Victoria. The study capture rate over 12 months was exceptionally high ensuring a representative sample” (p.195). The sample was comprised of English Speaking Background (ESB; white and/or Caucasian participants of European descent.), Culturally and linguistically diverse (CALD; minority groups from non- English-speaking backgrounds) and Indigenous (IND; Aboriginal Australian or Torres Strait Islander heritages) youth. Participants were assessed using a semi-structured interview to complete the YLS/CMI and six month follow-up for recidivism.

“As anticipated, group differences across instrument total and item scores were observed. The Indigenous group obtained the highest aggregate mean total score. This score was significantly higher than the aggregate mean total score for the CALD group… Regarding specific risk items, the Indigenous group received significantly higher scores on two domains (i.e., substance abuse, leisure/recreation) compared to the CALD group” (pp.199-200).

“Using the suggested YLS/CMI high-risk cut-off point, CALD offenders who scored above the threshold were almost three times more likely to reoffend than those who scored below the cut-off. In comparison, ESB and IND youth who scored above the threshold were approximately two times more likely to have reoffended. High-risk youth from all three ethnic subgroups were more likely to violently reoffend than to not reoffend. These findings suggest that the discriminative utility of the YLS/CMI total score is clearly greater for clients deemed as a higher risk, regardless of ethnicity. Low to moderate scores on the YLS/CMI total are unable to distinguish reoffenders from non-reoffenders for CALD youth in particular” (pp. 200-201).

“High proportions of high-risk participants across ethnic groups re-offended. Fewer participants who were adjudged to be high risk violently reoffended, although the instrument still correctly predicted this outcome for the majority of high-risk offenders. These findings indicate that the YLS/CMI demonstrates strong predictive accuracy for general recidivism for high-risk youth regardless of ethnic background… Conversely, the instrument demonstrated low predictive ability for participants who scored below the threshold. The majority of participants with scores below the threshold regardless of ethnicity still reoffended” (p. 201).

“The YLS/CMI instrument was found to be able to identify with reasonable accuracy who will reoffend across different ethnic groups. However, in the study, this facility appears to transpire for ESB youth and only for CALD and Indigenous youth deemed as a high risk by the instrument. The instrument exhibits a noticeable weakness in identifying Indigenous and CALD recidivists from non-recidivists who are a lower risk for future offending. In clearer terms, the instrument has difficulties discriminating between high- and low-risk ethnic minority young offenders” (p. 201). “We cannot rely on the AUC [area under the curve; a statistical analysis commonly used to measure discriminant validity] estimate alone when ascertaining the predictive utility of a risk instrument. This study clearly demonstrates the ostensible predictive accuracy of the YLS/CMI for CALD and Indigenous youth despite those same groups receiving low AUC indices. As such, it is necessary to adopt both discrimination and calibration analyses in future investigations or else important clinical information is potentially lost. This study indicates that assessors can be confident that high-risk young offenders regardless of ethnicity, who are judged as a high risk on the YLS/CMI, are a strong likelihood of reoffending. At the same time, assessors need to be cognizant that the ability of the YLS/CMI to predict recidivism for lower-risk CALD and Indigenous offenders is uncertain” (p. 201).

“Clinicians need to be acutely aware of the prejudicial impact of risk instruments if they are misused, particularly given the supposed inaccuracy of the moderate/low-risk classification as a predictor of minority desistance. Mirroring earlier research, the Indigenous group in this study collectively received a comparatively higher mean instrument total score despite the instrument demonstrating lower discriminative value. A heavy focus on risk level (which for the YLS/CMI is associated with a cumulative total score) may need to be obviated when assessing clients from specific cultural backgrounds. Total scores may be artificially inflated because of unalterable static factors associated with minority disadvantage. Moreover, the unique influence of both static and dynamic factors on recidivism for some minority groups is without the requisite supporting empirical data. In addition, assessing CALD and Indigenous youth while the knowledge base on cultural specific protective factors is limited, presents challenges for balanced risk assessment with these groups. While the risk categorization of a client is undeniably an important part of the RNR model, replicative work needs to be conducted before we are confident that the instrument satisfactorily adjudicates cross-cultural risk. It may be that risk thresholds have distinct cut-off points on the instrument for different ethnic groups” (p. 202).

“It is important to remember that the focus of risk assessment, especially with under-researched minority groups, should be on specifically targeting problematic individual risk factors for risk management purposes. Reviewing the criterion coding for risk items to ensure cultural applicability is recommended. Notably, subsequent treatment schemes should be facilitated in ways that incorporate cultural learning styles. Therefore it is incumbent on forensic professionals to first, be aware of cultural differences and then translate such awareness into culturally service delivery. As the cross-cultural risk instrument validation and culturally specific risk factor literature are both still in their infancy, it is important that clinicians embrace and attain cultural proficiency in the interim” (p.202).

“The higher instrument scores are a likely reflection of the well-documented disadvantaged and marginalized status of many Indigenous Australians who are exposed to deleterious circumstances including family disruption, unemployment, community violence, substance abuse, and stress… These factors undoubtedly impact the likelihood of engagement in criminal activity and contribute to Indigenous overrepresentation in the justice system. As such, the tool appears to accurately account for this greater exposure to risk items for Indigenous youth. However, it is important to note that widespread socioeconomic disadvantage may engender higher scores on unchangeable historical items for many Aboriginal forensic clients potentially inflating risk scores” (p. 199-200).

“Interestingly, participants who were not released from custody or did not permit access to their criminal records obtained significantly lower YLS/CMI total scores compared to the consenting cohort who were released into the community. While this finding may have accounted for the unusually high recidivism rate of the released group, it is difficult to speculate why young offenders who are potentially serving longer sentences or refuse access to their offending history are more likely to present as a lower risk for re-offense” (p.200).

As always, please join the discussion below if you have thoughts or comments to add!