Exploring Tools to Evaluate Violence Risk in Forensic Psychiatric Contexts

Featured Article

Violence Risk Assessment Instruments in Forensic Psychiatric Populations: A Systematic Review and Meta-Analysis

Maya G T Ogonah, BSc
Aida Seyedsalehi, MPhil
Daniel Whiting, DPhil
Prof Seena Fazel, MD 

Summary Background Although structured tools have been widely used to predict violence risk in specialist mental health settings, there is uncertainty about the extent and quality of evidence of their predictive performance. We aimed to systematically review the predictive performance of tools used to assess violence risk in forensic mental health, where they are routinely administered. Methods In our systematic review and meta-analysis, we followed PRISMA guidelines and searched four databases (PsycINFO, Embase, Medline, and Global Health) from database inception to Nov 1, 2022, to identify studies examining the predictive performance of risk assessment tools in people discharged from forensic (secure) mental health hospitals. Systematic and narrative reviews were excluded from the review. Performance measures and descriptive statistics were extracted from published reports. A quality assessment was performed for each study using the Prediction Model Risk of Bias Assessment Tool. Meta-analysis was conducted on the performance of instruments that were independently externally validated with a sample size greater than 100. The study was registered with PROSPERO, CRD42022304716. Findings We conducted a systematic review of 50 eligible publications, assessing the predictive performance of 36 tools, providing data for 10 460 participants (88% men, 12% women; median age [ from 47 studies] was 35 years, IQR 33–38) from 12 different countries. Post-discharge interpersonal violence and crime was most often measured by new criminal offences or recidivism (47 [94%] of 50 studies); only three studies used informant or self-report data on physical aggression or violent behaviour. Overall, the predictive performance of risk assessment tools was mixed. Most studies reported one discrimination metric, the area under the receiver operating characteristic curve (AUC); other key performance measures such as calibration, sensitivity, and specificity were not presented. Most studies had a high risk of bias (49 [98%] of 50), partly due to poor analytical approaches. A meta-analysis was conducted for violent recidivism on 29 independent external validations from 19 studies with at least 100 patients. Pooled AUCs for predicting violent outcomes ranged from 0·72 (0·65–0·79; I²=0%) for H10, to 0·69 for the Historical Clinical Risk Management-20 version 2 (95% CI 0·65–0·72; I²=0%) and Violence Risk Appraisal Guide (0·63–0·75; I²=0%), to 0·64 for the Static-99 (0·53–0·73; I²=45%). Interpretation Current violence risk assessment tools in forensic mental health have mixed evidence of predictive performance. Forensic mental health services should review their use of current risk assessment tools and consider implementing those with higher-quality evidence in support.

This research study addresses a critical issue within forensic psychiatric care – the risk of violence posed by individuals discharged from such care. There is a concern regarding public safety, which is causing significant costs to the broader economy, public health, and social care systems. The issue is that individuals who have been released from forensic psychiatric care tend to reoffend violently, with reconviction rates being around 12% within two years of discharge.

Assessing and managing the risk of post-discharge violence is a crucial aspect of forensic psychiatric care. Various risk assessment instruments have been developed globally to aid in this process. However, the content highlights a significant gap in the evidence base regarding the predictive performance of these instruments. Despite the widespread use of over 400 risk assessment tools, there needs to be more definitive syntheses of evidence for their effectiveness.

This study seeks to fill this gap by conducting a comprehensive meta-analysis of 36 risk assessment instruments used in forensic mental health settings. The analysis involves a substantial number of participants (10,460) and reveals concerning findings. Most notably, there is a high risk of bias in existing studies, with limited reporting on key aspects such as discrimination and calibration.

The meta-analysis suggests that current practices in using risk assessment tools may need a thorough review. It emphasizes the importance of validation studies for these tools, mainly when applied in new settings and among different populations. This recommendation holds significance for implementing risk assessment instruments, especially in low- and middle-income countries that lack research in this area.

One noteworthy aspect of the study is its attention to the gender dimension. The existing research tends to focus mainly on male participants, but women constitute a notable percentage (5-18%) of forensic psychiatric patients in Europe. The study underscores the importance of evaluating whether commonly used risk assessment instruments are applicable and effective for women in forensic psychiatric populations. The limited evidence suggests that female-only samples may have different predictive performance outcomes than predominantly male ones.

In summary, the content highlights the critical need for a more rigorous and evidence-based approach to the use of risk assessment instruments in forensic psychiatric care, particularly in the context of post-discharge violence risk. The study's findings and recommendations have implications for clinical decision-making, discharge planning, and resource allocation in forensic mental health services.

The findings from this systematic review and meta-analysis, examining the predictive performance of risk assessment instruments for individuals discharged from forensic psychiatric hospitals, have significant clinical implications for forensic mental health professionals.

Mixed Predictive Performance: The findings indicate mixed predictive performance of the assessed risk assessment instruments, with wide variation in study quality, outcome reporting, and tool effectiveness.

High Risk of Bias: Almost all studies (98%) were assessed as having a high risk of bias. Forensic mental health professionals need to be cautious in relying solely on these tools for decision-making.

Limited Reporting Measures: A substantial number of studies (82%) reported only the Area Under the Curve (AUC) for discrimination, overlooking other crucial measures such as calibration, sensitivity, and specificity. This poses challenges in interpreting the absolute risk estimations and making informed clinical decisions.

Small Sample Sizes: A significant portion of studies (38%) had small sample sizes, leading to uncertain risk estimates. Strategies like using multisite or nationwide registry data could address this limitation.

Performance Variation Between Tools: The meta-analysis highlighted that some tools, such as H10, HCR-20 version 2, VRAG, and Static-99, performed similarly in predicting violent recidivism. However, the varied performance across studies emphasizes the need to consider each tool's context and population carefully.

Gender Disparities: The studies predominantly focused on male-dominated samples (88%), raising questions about the generalizability of tools to female patients. Gender-specific risk factors are crucial for accurate risk assessment in diverse populations.

Clinical Decision-Making: Forensic mental health professionals should use risk assessment tools complementary to clinical decision-making, not as the sole determinant. Decisions about length of stay and discharge planning should involve validations showing high sensitivity and specificity.

Tool Development Considerations: New risk assessment tools should include multivariable models, account for the most decisive risk factors (e.g., age, sex, previous antisocial behavior), and undergo internal validation. Tools' feasibility, acceptability, and effectiveness in different settings should be thoroughly assessed.

Future Research Directions: Future research should focus on adequately powered studies reporting multiple predictive performance measures. Inclusion of women, samples from low-income and middle-income countries, and exploration of novel risk factors could enhance the understanding and effectiveness of risk assessment tools in diverse settings.

Improving Risk Management: Higher quality risk assessment tools and a comprehensive understanding of their limitations could contribute to more effective risk management in forensic mental health settings.

Comparison of Instruments: Instruments developed for specific outcomes may be used for different predictions, affecting their accuracy. When comparing their performance, overlapping content between instruments and redundant items should be considered.

Efficiency in Risk Assessments: Short forms or tool subscales can be more efficient in clinical services if they are equally predictive. The H10, a subscale of HCR-20, performed similarly to the full scale, suggesting potential resource efficiency.

Focus on Sensitivity and Specificity: In forensic mental health settings, choosing between sensitivity and specificity depends on public safety (sensitivity) and protecting patient rights (specificity). Research is needed to determine optimal levels for these risk assessment instruments.

Continuous Risk Management: Continuous use of violence risk assessment instruments for ongoing risk management makes it challenging to distinguish false-positive errors. Reporting treatment and risk management procedures is vital for accurately appraising risk prediction potential.

Need for Multivariable Models: New tools should include multivariable models and the most substantial risk factors (age, sex, previous antisocial behavior) for accuracy in new settings. Considerations include development on forensic mental health populations, feasibility, acceptability, and internal validation.

Future Research Directions: Methodological improvements, including adequately powered studies reporting multiple predictive performance metrics, are needed. Research should include diverse populations (including women) and investigate the incremental value of novel risk factors for improved performance.

Global Perspective: Current research is predominantly from high-income countries. Future studies should seek the inclusion of low-income and middle-income countries to enhance the global applicability of findings. Exploring the impact of novel risk factors, such as previous neurological damage, is also recommended for performance improvement.

Blog Posts

An Introduction to Violence Risk Assessments

Juvenile General Violence Risk Assessments

Violence Risk Assessment Certificate into Career

 

Training

LIVE: Use of Violence Risk Assessment to Guide Treatment Planning in Forensic Settings

Violence Risk Assessment Certificate | CONCEPT Professional Training