Site Differences Impact Clinician Applications and Predictive Accuracy of the J-SOAP-II
The J-SOAP-II should be used as part of a battery of risk assessment tools rather than in a strict actuarial manner for assessing future risk. This is the bottom line of a recently published article in International Journal of Forensic Mental Health. Below is a summary of the research and findings as well as a translation of this research into practice.
Featured Article | International Journal of Forensic Mental Health | 2015, Vol. 14, 56-65
Predictive Validity of the J-SOAP-II: Does Accuracy Differ Across Settings?
Ricardo Martinez, Fordham University
Barry Rosenfeld, Fordham University
Keith Cruise, Fordham University
Jacqueline Martin, Hoboken University Medical Center
Court and mental health workers are frequently asked to determine which juvenile sex offenders (JSOs) are most likely to reoffend. One instrument commonly used to guide decision making with JSOs is the Juvenile Sex Offender Assessment Protocol-II (J-SOAPII). However, research utilizing this instrument has often generated contradictory results, perhaps related to the types of samples studied. The current study sought to compare the predictive accuracy of the J-SOAP-II across two samples of JSOs (a medium-security correctional setting versus an unlocked residential sex offender treatment program). Although the overall predictive accuracy for identifying post-release arrests for sexual offenses (i.e., sexual recidivism) was modest (AUC = .64) and not statistically significant, differences emerged with regard to the accuracy of some individual scales and subscales. Similarly, while no significant differences in predictive accuracy were observed between the two study sites, a number of interesting findings were observed. These findings highlight the need to consider risk assessment measures in light of the setting in which they are used in order to maximize predictive accuracy and optimize treatment and dispositional decision making.
sex offenders, juveniles, risk assessment, J-SOAP-II, recidivism
Summary of the Research
There have been mixed findings regarding the predictive validity of the Juvenile Sex Offender Assessment Protocol (J-SOAP-II). “This research is no doubt hindered by the small sample sizes used in most studies, and the low rates of sexual reoffending that are typically found. Nevertheless, it appears that the scale’s predictive utility is somewhat stronger with JSOs in less restrictive settings, whereas its use with institutionalized offenders being considered for release into the community may be questionable. However, none of the published studies to date have directly compared samples drawn from different types of correctional/treatment settings” (p.57).
“The current study sought to address this gap in the existing literature by comparing the predictive accuracy of the J-SOAP-II across two samples of JSOs, a medium-security correctional setting versus an unlocked residential sex offender treatment program. In addition, unlike much of the existing literature that has examined risk assessments conducted at the time of intake, this study examined risk of reoffense at the time of release from the facility” (p.57).
Overall, results were moderate but insignificant when examining predictive accuracy using J-SOAP-II total scores. “Stronger, but still moderate predictive accuracy was observed for the Dynamic summary scale and its components (Scales III and IV)…However, neither the Static summary scale nor its components (Scales I and II) significantly predicted sexual reoffending in the total sample…Predictive accuracy was generally weaker for non-sexual recidivism though these small effects were often significant. In fact, only Scale II (Impulsive/Antisocial Behavior) generated a medium effect size” (p.62)
“Despite some small differences in AUC estimates across the two study sites for the individual J-SOAP-II subscales, there were also no significant differences in predictive accuracy. Similarly, no site differences were observed with regard to general reoffending, as only Scale II was significantly associated with general reoffending and this association was roughly comparable for both study sites” (p.61).
Translating Research into Practice
Clinicians may be most successful using the J-SOAP-II as a tool to generally and systematically review relevant risk factors for treatment and intervention purposes. The modest findings of the predictive accuracy of the tool suggest that the J-SOAP-II may not be appropriate as a pure actuarial measure in terms of anchoring risk estimates on total, summary, or subscale scores. “Thus, the scale may have ample utility for its recommended uses, despite the limited utility in predicting reoffense among institutionalized JSOs when used as a set of “scores”” (p.64).
“Interestingly, although the J-SOAP-II manual suggests that Scale IV (Community Stability/Adjustment) be omitted for youth who are incarcerated or housed in a secure treatment facility at the time of the evaluation, ratings based on treatment records demonstrated some utility for Scale IV. Reliability for the scale (internal consistency and interrater) was adequate and internal consistency was stronger when total scores included the Scale IV items. In addition, JSOAP-II total scores that included Scale IV were somewhat stronger (though not significantly so) than those without Scale IV, and the Dynamic summary scale (which combines Scales III and IV) was superior (though again, not significantly) to Scale III alone. Thus, while the J-SOAP-II authors recommend omission of this scale for detained youth, these findings suggest that the scale may have utility even for these individuals” (p. 63).
Other Interesting Tidbits for Researchers and Clinicians
“Although far from definitive, these findings may reflect the greater intensity of sex offender treatment at PRTC, as this setting focused much more intensively on sex offender programming than did JTSB. The significantly lower scores on JSOAP-II Scale III (Intervention), reflecting fewer intervention-based risk factors, supports this hypothesis but is clearly insufficient to “test” the differences in treatment programming across the two sites” (p.63).
Join the Discussion
As always, please join the discussion below if you have thoughts or comments to add!
Authored by Andrea Patrick
Andrea Patrick is completing her M.A. in Forensic Psychology at John Jay College of Criminal Justice. In the future, she hopes to be directly working with forensic populations providing risk assessments, clinical evaluations as well as conducting research within the field.