Site Differences Impact Clinician Applications and Predictive Accuracy of the J-SOAP-II

Ricardo Martinez, Fordham University
Barry Rosenfeld, Fordham University
Keith Cruise, Fordham University
Jacqueline Martin, Hoboken University Medical Center

Court and mental health workers are frequently asked to determine which juvenile sex offenders (JSOs) are most likely to reoffend. One instrument commonly used to guide decision making with JSOs is the Juvenile Sex Offender Assessment Protocol-II (J-SOAPII). However, research utilizing this instrument has often generated contradictory results, perhaps related to the types of samples studied. The current study sought to compare the predictive accuracy of the J-SOAP-II across two samples of JSOs (a medium-security correctional setting versus an unlocked residential sex offender treatment program). Although the overall predictive accuracy for identifying post-release arrests for sexual offenses (i.e., sexual recidivism) was modest (AUC = .64) and not statistically significant, differences emerged with regard to the accuracy of some individual scales and subscales. Similarly, while no significant differences in predictive accuracy were observed between the two study sites, a number of interesting findings were observed. These findings highlight the need to consider risk assessment measures in light of the setting in which they are used in order to maximize predictive accuracy and optimize treatment and dispositional decision making.

sex offenders, juveniles, risk assessment, J-SOAP-II, recidivism

There have been mixed findings regarding the predictive validity of the Juvenile Sex Offender Assessment Protocol (J-SOAP-II). “This research is no doubt hindered by the small sample sizes used in most studies, and the low rates of sexual reoffending that are typically found. Nevertheless, it appears that the scale’s predictive utility is somewhat stronger with JSOs in less restrictive settings, whereas its use with institutionalized offenders being considered for release into the community may be questionable. However, none of the published studies to date have directly compared samples drawn from different types of correctional/treatment settings” (p.57).

“The current study sought to address this gap in the existing literature by comparing the predictive accuracy of the J-SOAP-II across two samples of JSOs, a medium-security correctional setting versus an unlocked residential sex offender treatment program. In addition, unlike much of the existing literature that has examined risk assessments conducted at the time of intake, this study examined risk of reoffense at the time of release from the facility” (p.57).

Overall, results were moderate but insignificant when examining predictive accuracy using J-SOAP-II total scores. “Stronger, but still moderate predictive accuracy was observed for the Dynamic summary scale and its components (Scales III and IV)…However, neither the Static summary scale nor its components (Scales I and II) significantly predicted sexual reoffending in the total sample…Predictive accuracy was generally weaker for non-sexual recidivism though these small effects were often significant. In fact, only Scale II (Impulsive/Antisocial Behavior) generated a medium effect size” (p.62)

“Despite some small differences in AUC estimates across the two study sites for the individual J-SOAP-II subscales, there were also no significant differences in predictive accuracy. Similarly, no site differences were observed with regard to general reoffending, as only Scale II was significantly associated with general reoffending and this association was roughly comparable for both study sites” (p.61).

Clinicians may be most successful using the J-SOAP-II as a tool to generally and systematically review relevant risk factors for treatment and intervention purposes. The modest findings of the predictive accuracy of the tool suggest that the J-SOAP-II may not be appropriate as a pure actuarial measure in terms of anchoring risk estimates on total, summary, or subscale scores. “Thus, the scale may have ample utility for its recommended uses, despite the limited utility in predicting reoffense among institutionalized JSOs when used as a set of “scores”” (p.64).

“Interestingly, although the J-SOAP-II manual suggests that Scale IV (Community Stability/Adjustment) be omitted for youth who are incarcerated or housed in a secure treatment facility at the time of the evaluation, ratings based on treatment records demonstrated some utility for Scale IV. Reliability for the scale (internal consistency and interrater) was adequate and internal consistency was stronger when total scores included the Scale IV items. In addition, JSOAP-II total scores that included Scale IV were somewhat stronger (though not significantly so) than those without Scale IV, and the Dynamic summary scale (which combines Scales III and IV) was superior (though again, not significantly) to Scale III alone. Thus, while the J-SOAP-II authors recommend omission of this scale for detained youth, these findings suggest that the scale may have utility even for these individuals” (p. 63).

“Although far from definitive, these findings may reflect the greater intensity of sex offender treatment at PRTC, as this setting focused much more intensively on sex offender programming than did JTSB. The significantly lower scores on JSOAP-II Scale III (Intervention), reflecting fewer intervention-based risk factors, supports this hypothesis but is clearly insufficient to “test” the differences in treatment programming across the two sites” (p.63).

As always, please join the discussion below if you have thoughts or comments to add!