Non-Violent, Repeat Offenders Impact the Accuracy of Risk Assessments

Jeremy W. Coid, Violence Prevention Research Unit, Wolfson Institute of Preventive Medicine, Queen Mary University of London, Royal London Hospital, London, United Kindgom
Min Yang, School of Community Health Sciences, University of Nottingham, Innovation Park, Nottingham, United Kingdom
Simone Ullrich and Tianqiang Zhang, Violence Prevention Research Unit, Wolfson Institute of Preventive Medicine, Queen Mary University of London, Royal London Hospital, London, United Kingdom
Stephen Sizmur, Picker Institute Europe, King’s Mead House, Oxford, United Kingdom
David P. Farrington, Institute of Criminology, University of Cambridge, Cambridge, United Kingdom
Mark Freestone, Violence Prevention Research Unit, Wolfson Institute of Preventive Medicine, Queen Mary University of London, Royal London Hospital, London, United Kingdom; Forensic Personality Disorder Service, East London NHS Foundation Trust, London, United Kingdom
Robert D. Rogers, School of Psychology, University of Bangor, Gwynedd, United Kingdom

Accuracy of risk assessment instruments in predicting violence may appear poor if substantial numbers of study participants subsequently reoffend non-violently instead of violently as predicted. This study examined effects of changing the violent outcome on predictive accuracy of five instruments (OGRS, RM2000(V), VRAG, PCL-R, HCR20) for 1,353 male and 304 female released prisoners in England and Wales. Adding self-reported violence to criminal convictions resulted in a moderate increase in violent outcome among women, but was small among men. After removing offenders who subsequently reoffended non-violently, significant improvement in accuracy was found on all instruments for men, but not women. We concluded that risk assessment instruments for violence may be more accurate than previously described, but improvement can only be achieved with certain samples. Instruments relying heavily on previous criminal history for predictive power can demonstrate improved accuracy, but only after removing non-violent offenders from samples with extensive previous offending.

risk assessment, predictive accuracy, violence, criminal careers, prisoners

The overall aim of this study was to observe the effects of changing the outcome variable on the predictive accuracy of five risk and psychopathology assessment instruments: Psychopathy Checklist Revised (PCL-R); Violence Risk Appraisal Guide (VRAG); Historical-Clinical Risk Management-20 (HCR-20); Risk Matrix 2000 Violence (RM2000(V)); and the Offender Group Reconviction Scale II (OGRS). Three associated research questions were considered: to observe whether predictive accuracy differed between qualitatively different outcome measures of violence, including self- reported and officially recorded violence; whether by improving the criterion variable to detect a greater proportion of the violent outcome by combining both measures resulted in improved accuracy; and whether the removal of individuals who had been convicted, but for non-violent offending rather than actual violence, further improved accuracy.

Improving the Criterion for Violent Outcomes

To increase the prevalence of the criterion variable of violence, self-reports of violence were combined with recorded criminal convictions. By adding self-reports, the base rate increased by 15% in men and 22% in female prisoners. When all prisoners who reoffended non-violently were removed from the combined outcome (self report + criminal convictions), there was only a slight decrease in self-reported violence. The authors explain this outcome as a result of criminals engaging in many different types of crime and a criminal specialization in violence is rare.

Improving the Predictive Accuracy of the Instruments

Removal of non-violent offenders resulted in significantly higher AUC values. Base rates of AUC (area under the curve) values are insensitive to base rates for violence thus the addition of self-reported violence did not create a change in AUC values.

For both self-reported violence and recorded criminal convictions, all instruments demonstrated improvement in predictive accuracy when non-violent offenders were removed. This result was only applicable for men and not female prisoners. Only the PCL-R, VRAG and HCR20 demonstrated significant improvement while the OGRS and RM2000-V each demonstrated increases in AUC values, but the difference was not significant. Overall, actuarial instruments showed no greater accuracy for self-reported violence while clinical assessments showed a more noticeable difference in AUC values for the ‘all participants’ and ‘violent offender only’ groups.

There may be fundamental differences in the effect of removing non-violent reoffenders for female risk assessments. This removal effect produced small, statistically insignificant differences in predictive validity. “Actuarial measures, including the OGRS, VRAG, and RM2000-V, performed less accurately than the PCL-R and the HCR20 for each outcome. For the OGRS, confidence intervals indicate significantly worse performance than the PCL-R and HCR-20 for outcomes of violent convictions and self-reported violence before removal of nonviolent offenders. The PCL-R performed significantly better than the OGRS after removal of nonviolent offenders for violent convictions, self-reported violence, and the combined outcome. The HCR20 performed significantly better than the OGRS after removal of nonviolent offenders for the combined outcome.” However, the main difference is accounted for by differences in men and women prisoners’ criminal careers and the degree to which the five instruments rely on these factors for their predictive accuracy. The gender differences in accuracy may be explained by the need to modify existing clinical risk assessment tools to capture female-specific risk factors.

Though outcome specification significantly improved the accuracy of the risk assessment tools, a significant amount of variability for in the prediction of violent reoffending is unexplained. This may be explained by prisoners’ tendency to reoffend more non-violently than violently throughout their criminal careers. A greater degree of accuracy could be improved by combining all sources of information regarding the prisoner.

Instrument properties examined in this study along with sample effects may offer an interesting perspective: “For example, OGRS and RM2000-V contrast with the HCR-20 and PCL-R in that they are actuarial instruments developed using experimental samples whereas the HCR-20 was developed to guide clinical risk assessment and the PCL-R as a diagnostic measure of psychopathic personality. All five instruments lack outcome specificity, with each able to predict acquisitive offending as well as violence.”

It has been argued that these measures may be limited to measuring a general construct of criminal risk rather than specific tendencies to violence as originally intended, although it must be pointed out that the OGRS was developed specifically to measure the risk of general reoffending. It is possible therefore that lack of outcome specificity is more a characteristic of the three actuarial instruments (RM2000v, OGRS and VRAG), which show somewhat larger increases in predictive ability for violent convictions compared to the PCL-R and HCR-20 when non-violent reoffenders are excluded.

As always, please join the discussion below if you have thoughts or comments to add!