YLS/CMI and SAVRY helpful for Juvenile Probation Officers’ case planning decisions

Laura S. Guy, Psychiatry, University of Massachusetts Medical School, Worcester, Massachusetts, USA
Rebecca J. Nelson, University of Massachusetts Medical School, Worcester, Massachusetts, USA
Samantha L. Fusco-Morin, Psychology, Fordham University, Astoria, New York, USA
Gina M. Vincent, Psychiatry, University of Massachusetts Medical School, Worcester, Massachusetts, USA

Juvenile probation officers (JPOs; n = 71) in the United States were interviewed three and ten months after the SAVRY or YLS/CMI was implemented in their office. Quantitative and qualitative analyses were used to explore their experiences using the instruments and adherence to practice guidelines. JPOs typically perceived the instruments as being ‘somewhat’ or ‘very’ helpful for guiding their case planning decisions. A frequently cited barrier to using both instruments in practice related to the increased length of time it took to complete reports; yet, at the same time, some JPOs also acknowledged that use of the measures forced them to gather important information about the youth’s background and current situation that proved useful. Most JPOs (77%, n = 33 of 43) using the SAVRY expressed preference for a risk assessment model that emphasized use of appropriate professional discretion rather than a score-based approach. “Buy-in” for the instruments and the reported difficulties varied across sites. The present findings may inform recommendations specifically for delivering training on the SAVRY and YLS/CMI and, more broadly, strategies to promote their effective implementation in juvenile justice settings.

juvenile probation, risk assessment, implementation, SAVRY, YLS/CMI

Juvenile probation officers’ (JPOs) “decision-making regarding the appropriate programming for youth can lead to significant reductions in re-offense rates particularly with respect to various intervention programs (e.g., individual counseling, behavioral programs, interpersonal skills) and appropriate matching to services. Despite the availability of instruments in juvenile justice to structure decisions about issues highly relevant to disposition, such as risk for future offending, juvenile justice professionals tend to rely on unstructured professional judgment. Meta-analytic data demonstrate this approach to be substantially less accurate than structured approaches. Therefore, it is noteworthy that, despite the pervasiveness of the use of unstructured professional judgment for assessing risk and formulating intervention and case management plans, jurisdictions that follow the recommendations in the Juvenile Justice and Delinquency Prevention Act (JJDPA, 2002) now have in place some form of structured instrument for assessing risk” (p. 227).

“This study investigates JPOs’ expectations of and experiences using the YLS/CMI and SAVRY for case planning with probationers. It extends earlier work examining the implementation of these instruments in the Risk/Needs Assessment in Juvenile Probation: Implementation Study (RNAJP).” Three study aims were examined, including “JPOs’ broad perceptions about the usefulness and difficulties of the instruments; JPOs’ experiences using the instruments with respect to (1) rating specific items and (2) making overall ratings about risk level; and among SAVRY users only, the degree to which JPOs reported rating the items and making a summary risk rating (SRR) in a manner consistent with the SPJ model’s concepts of manifestation, relevance, and linearity” (pp. 229-230).

Participants were 71 JPOs sampled from six probation offices that participated in the RNAJP study; three offices implemented the YLS/CMI, and three offices implemented the SAVRY. “JPOs participated in two-day training workshops conducted by a co-author of the instrument, typically completed three post-workshop practice cases and received feedback, and participated in booster training six months following the initial workshop. Eight trained researchers interviewed JPOs three times about their case management practices and experiences supervising probationers: prior to implementation of and training on the risk assessment instrument, three months after implementation, and 10 months after implementation. Only data from the two post-implementation interviews are reported here, and subsequently are referred to as the first and second interviews, respectively” (pp. 230).

Aim 1. Perceptions About the Usefulness and Difficulties of the Instruments

“Overall, the majority of users of both instruments perceived them to be ‘somewhat’ or ‘very’ helpful for making recommendations about disposition, services, and level of supervision across both follow-up periods. The most frequently identified theme was use of the [YLS/CMI] to “back up” their opinions about risk level and recommendations regarding services and level of supervision, which they believed they would have reached using only their professional experience” (pp. 229).

“JPOs using the SAVRY provided more comments about usefulness than YLS/CMI users, [primarily] SAVRY’s emphasis on professional judgment or lack of reliance on a total score, enhanced data collection that led to more useful recommendations, and increased knowledge about risk factors. Several JPOs indicated their knowledge about empirically based risk factors increased after completing the SAVRY training workshop and becoming familiar with the manual” (pp. 232)

Aim 2. Experiences Rating Specific Items and Making Overall Risk Ratings

Thirty-six percent of JPOs found specific items [on the YLS/CMI] difficult to rate. “Items in the Attitudes/Orientation domain were identified most frequently, followed by the Substance Abuse and Leisure/Recreation domains.” Fifty-seven percent of JPOs found one or more [SAVRY] items difficult to rate, most frequently among the “Social/Contextual scale: Peer Rejection, Stress and Poor Coping, Lack of Personal/ Social Support, and Community Disorganization. Over 90% of JPOs using the SAVRY reported at the first interview that making the SRR was ‘very easy’ (52%) or ‘somewhat easy’ (39%). By the second interview, the majority (23 of 44, 52%) reported the SRR was ‘very easy’ to generate. Among JPOs who reported some difficulty, 23% indicated they had insufficient information, 21% cited lack of clarity in the manual’s item description, and 11% reported that the source difficulty varied depending on the particular item” (pp. 234-235).

Aim 3. Among SAVRY Users, To What Degree Did JPOs Report Using the Instrument in a Manner Consistent With Principles of the SPJ Model?

“Responses indicated that just over 90% of JPOs’ understood that a single risk factor could be present for two youth for different reasons. Most JPOs (89%) replied in a manner that suggested that they understood the concept of individual relevance well. However, only a minority of respondents indicated (correctly) that youth with the same risk factors present would not necessarily have the same level of overall risk” (p .235).

“JPOs perceived both the YLS/CMI and SAVRY as being ‘somewhat’ or ‘very’ helpful for guiding case planning decisions. Most SAVRY users demonstrated good understanding of key components of the SPJ model. Both measures generally were experienced as being easy to use, including the aspect of the SAVRY requiring the most discretion – developing the overall summary judgment about the youth’s risk level. The majority of SAVRY users expressed preference for risk assessment procedures that called for professional discretion rather than a score-based approach. Finally, [these] results suggest that many anticipated barriers to putting a risk assessment instrument into routine practice can be overcome when sound implementation procedures are followed, allowed frontline users and agencies more generally to benefit from research based developments in the science of risk assessment” (p. 239).

“High-quality implementation strategies are crucial in probation settings for risk assessment practices to effectuate change in decision-making. Implementation is difficult in organizations in which staff members are cynical about the changes. Thus, quality training with and buy-in from JPOs appear to be necessary pre-requisites to successful implementation of risk assessment systems” (pp. 237).

“Results of the present study offer guidance for areas of curriculum that should be emphasized during training on risk assessment with youth populations.” Some of the difficulties in rating items were related to insufficient information and unclear rating instructions in the manual. “Clarification [on the YLS/CMI] regarding how to rate dichotomous items when users feel the case information is more nuanced appears warranted during training workshops. Regarding the SAVRY, trainers should devote ample time for instruction on the Social/Contextual risk factors. With respect to training about some of the standard SPJ assessment practices, our results indicate the concept of manifestation is relatively easy for JPOs to understand. [These] findings suggest that SAVRY workshops could be improved by including more explicit instruction regarding the need to consider the idiographic relevance of each risk factor for the particular youth being evaluated, as well as guidance about how to apply information about items’ relevance when making the SRR.”

“Other minor themes observed related to positive aspects of the SAVRY included the promotion of objectivity and transparency in the risk assessment process, having a research based procedure “back up” their professional opinion and recommendations, the utility of the SAVRY for tracking changes in risk over time, ease of communication between professionals trained in the same instrument (e.g., speaking the “same language”), and increased ability to “pinpoint” the most critical criminogenic needs to be targeted for treatment” (pp. 233).

“Despite the additional guidance of the YLS/CMI for identifying need areas (in the form of scales with explicit labels), there appeared to be little difference in JPOs’ perceived helpfulness of these instruments for making recommendations about disposition, service referrals, or supervision level, albeit this was not the main focus of the study. Developing a case plan with a high likelihood of reducing a youth’s risk requires JPOs to identify and then triage the youth’s most important dynamic risk factors (also known as criminogenic needs) to be targeted for intervention. SAVRY users mentioned a benefit of the instrument was its ability to identify dynamic risk and protective factors; therefore, the instrument ostensibly should lead to quality case planning. How well disposition recommendations and case plans align with youths’ criminogenic needs and whether the alignment differs between the YLS/CMI and the SAVRY are areas in need of further study” (p. 239).

As always, please join the discussion below if you have thoughts or comments to add!