When Assessing Sexual Violence, Guided Clinical Decision Making Can Outperform Algorithms

Lee M. Vargen, Simon Fraser University
Karla J. Jackson, Simon Fraser University
Stephen D. Hart, Simon Fraser University and University of Bergen

Objective: This study empirically evaluated risk judgments made using the Risk for Sexual Violence Protocol (RSVP; Hart et al., 2003), a widely used set of structured professional guidelines for assessing and managing sexual violence risk. Hypothesis: Based on evaluations of other structured professional judgment guidelines, we hypothesized that judgments made using the RSVP would demonstrate good interrater reliability, concurrent validity, and predictive validity. Method: Based on file review, research assistants made ratings using the RSVP and two commonly used actuarial tools for sexual violence risk assessment in a sample of 100 adult male sexual offenders who successfully completed a community- based sex offender treatment program. Recidivism information was obtained from official records 10 years after treatment completion. Results: With respect to interrater reliability, judgments of the presence and relevance of individual risk factors ranged from moderate to almost perfect, and those for composite scores reflecting the sum of these ratings were almost perfect. Interrater reliability for integrative summary risk ratings was substantial to almost perfect. Regarding concurrent validity, the findings indicated that judgments made using the RSVP had moderate to large and statistically significant correlations with scores on the actuarial tools. Finally, with respect to predictive validity, RSVP presence total scores and summary risk ratings were predictive of new sexual violence over the follow-up, and the magnitude of their predictive validity effect sizes was similar to that of scores on the actuarial tools. Conclusions: The findings supported the potential utility of the RSVP in practice.

Risk for Sexual Violence Protocol, risk assessment, interrater reliability, concurrent validity, predictive validity

“Since the late 1990s, there has been a proliferation of violence risk assessment measures oriented around sexual violence, and with good reason. Sexual violence remains a pernicious problem, and victims of this violence span all age groups, ethnicities, and socioeconomic statuses. Although conceptually broad, sexual violence can be usefully and succinctly defined as the ‘actual, attempted, or threatened sexual contact with another person that is non-consensual.’ The prevalence of sexual violence has long been difficult to establish, in part due to the reluctance of victims to disclose their victimization, however it is clear that sexual violence is a disturbingly prevalent crime” (p. 37-38).

“In response to the demonstrable need to address and manage the risk posed by perpetrators of sexual violence, there has been a considerable amount of interest in and work on the assessment of risk for sexual violence, which has resulted in the publication of numerous tools to aid practitioners in this task. These measures can generally be categorized as reflecting two approaches to violence risk assessment: the actuarial approach and the structured professional judgment (SPJ) approach. Actuarial approaches can be simply described as those that do not incorporate an evaluator’s clinical judgment or discretion but instead structure all components of the assessment process, including the identification and measurement of risk factors, the combination and weighting of risk factors, and the final judgment of an individual’s risk level, all according to fixed and explicit rules. Actuarial estimates typically take the form of an absolute recidivism rate, a risk ratio, or a relative probability of an individual’s risk for future violence, derived from normative group data. The most popular actuarial sexual violence risk measures include the Sex Offender Risk Appraisal Guide or SORAG” (p. 38).

“In contrast, the SPJ approach involves both the structured identification and measurement of evidence-based risk factors, as well as the clinical judgment of an evaluator in making determinations about the risk for future violence. As Logan (2016) notes, the SPJ approach to risk assessment and management is an evidence-based one derived from practices in clinical medicine and involves not only an assessment of an individual’s propensity for future violence, but also the formulation of management strategies to reduce that individual’s risk for violence. One popular SPJ tool is the Sexual Violence Risk-20 or SVR-20, which was recently updated and published as the SVR-20 Version 2 or SVR-20 V2” (p. 38).

“Research has generally supported the efficacy of both actuarial and SPJ measures of sexual violence risk, and both approaches have been found to have good to excellent interrater reliability within the context of controlled research studies, with actuarial measures evidencing slightly higher interrater reliability on average. In addition, both approaches tend to have fair to moderate predictive validity. However, some have criticized the actuarial approach for not providing an adequate framework for comprehensive clinical formulation and for omitting dynamic risk factors that may change over time. Given that the ultimate goal of risk assessment is the prevention of future violence (i.e., risk management), this is seen by some as a significant limitation of an actuarial approach relative to an SPJ approach, wherein the development of management strategies is a foundational part of the risk assessment process” (p. 38).

“The SVR-20 was the only SPJ tool for assessing risk of sexual violence in adults prior to the publication of the Risk for Sexual Violence Protocol, or RSVP. The RSVP is intended to aid in the comprehensive, management-oriented assessment of risk for sexual violence. The RSVP includes 22 basic risk factors for sexual violence across five broad domains: History of Sexual Violence, Psychological Adjustment, Mental Disorder, Social Adjustment, and Manageability. Although derived from and therefore similar to the SVR-20 in terms of form and content, the RSVP was also intended to facilitate a more involved and robust management-oriented assessment of risk, including steps to facilitate scenario planning and the development of management plans, and was the first SPJ tool to explicitly incorporate steps for these plans. For this reason, the RSVP risk factors have a greater emphasis on psychological functioning (e.g., problems with coping, deficits in insight), and its administration procedure incorporates steps to determine the relevance of individual risk factors; identify plausible scenarios of possible future sexual violence; and develop scenario-based management plans. Like other SPJ measures, the RSVP was developed over several years and involved a review of the literature on risk factors for sexual violence, a review of existing guidelines and assessment procedures, the construction of the RSVP guidelines, and acquiring consumer feedback and conducting field tests. Compared to the SVR-20, the RSVP is recommended for use in situations where a more in-depth, management-oriented assessment is indicated and especially when the assessor has a greater degree of expertise in mental disorder and sexual violence. As with the SVR-20, the RSVP can be used as a stand-alone instrument and does not need to be used in conjunction with another sexual violence risk assessment instrument” (p. 38-39).

“In this study, we sought to evaluate the interrater reliability of judgments made using the RSVP, their concurrent validity with respect to the SORAG and Static-99R, and their predictive validity with respect to new convictions, charges, or police contacts for sexual violence in a sample of convicted sexual offenders who completed a community treatment program as part of the conditions of probation” (p. 47).

“With respect to interrater reliability, the findings indicated that judgments of the presence and relevance of individual risk factors ranged from moderate to almost perfect, and the interrater reliability composite scores reflecting the sum of individual presence and relevance ratings were almost perfect. Interrater reliability for integrative summary risk ratings was substantial to almost perfect. With respect to concurrent validity, the findings indicated that judgments made using the RSVP had moderate to large and statistically significant correlations with total scores and score categories on two commonly used actuarial measures of risk for sexual violence, the SORAG and Static-99R. Finally, with respect to predictive validity, RSVP presence total scores and summary risk ratings were predictive of new sexual violence over a 10-year follow-up, and the magnitude of their predictive validity effect sizes was similar to that of total scores and score categories on the SORAG and Static-99R. Overall, then, the findings were supportive of the potential utility of the RSVP in practice” (p. 47).

“The predictive validity findings are consistent with meta- analytic evidence reviewed by Guy, Douglas, and Hart (2015), which indicated that risk judgments made using SPJ guidelines appear to have accuracy that is equivalent to that of risk judgments made using actuarial tools” (p. 47).

“In sum, our findings support three conclusions regarding the psychometric properties of the RSVP. First, the risk judgments made with the RSVP are reliable across raters. Second, risk judgments made with the RSVP correspond with those made with two widely used actuarial sexual violence risk assessments. Third, the RSVP is able to predict sexual recidivism with an accuracy similar to that of the aforementioned actuarial measures” (p. 49).

“The present study considered both RSVP total scores and summary risk ratings with regards to predictive accuracy, which is consistent with previous research in this area. The results of the current study suggest that RSVP total scores and case prioritization had similar predictive accuracy. These findings may lead someone to assume that RSVP total scores can be relied on in terms of implying a certain level of risk. Although it is true that, on average, the number of risk factors present and total scores are correlated with various facets of risk, this general pattern may not hold in individual cases; an offender may have few risk factors yet still pose a risk for sexual recidivism that is high in likelihood, severity, or imminence. This is why the RSVP does not use cutoff scores to characterize overall judgments of risk” (p. 48).

“It has been recommended that both actuarial and structured professional judgment instruments be utilized as part of sexual violence risk assessments. In terms of determining the appropriate type of instrument(s) to be used in a violence risk assessment, it is important to take the referral question(s) into consideration. For example, a referral may simply request an assessment to determine the likelihood that an offender will sexually reoffend. In this case, the results of this study suggest that it may be redundant and unnecessary to include both the RSVP and an actuarial risk assessment instrument in a sexual violence risk assessment given that both types of instruments performed equally well overall and showed good concurrent validity. In other words, the predictive accuracy and risk ratings were generally consistent between the RSVP and the actuarial instruments. But different risk assessment approaches may not always lead to consistent judgments of risk for the same individual. When multiple risk assessment instruments are used, it is important for evaluators to explain the reasons underlying their judgments of risk. In the case where the referral for the sexual violence risk assessment is requesting an estimate of risk to reoffend and also treatment and management recommendations, then it may be appropriate to use both actuarial and structured professional judgment instruments. However, given that structured professional judgment instruments, such as the RSVP, are able to address both risk and treatment/management strategies, it appears that using both an actuarial and structured professional judgment approach is unnecessary” (p. 48-49).

“As discussed earlier, multiple violence risk instruments are often integrated for the purpose of violence risk assessments. Yet it is still relatively ambiguous how final risk judgments are made when there are discrepancies across the instruments in terms of risk for sexual recidivism. Research has begun to emerge in this area, although more work is needed. Such research could help to inform whether it is appropriate to use multiple instruments in an assessment and, if so, how final risk judgments should be made when there are discrepancies among the instruments. Resolving these issues would advance violence risk assessment in part by adding to the transparency of the process” (p. 49).

As always, please join the discussion below if you have thoughts or comments to add!