Predicted Compliance With a Web-Based Intervention

JMIR Mental Health | 2025, Vol. 12, p. 1-19

The Effect of Predicted Compliance With a Web-Based Intervention for Anxiety and Depression Among Latin American University Students: Randomized Controlled Trial

Corina Benjet, PhD; Center for Global Mental Health Research, National Institute of Psychiatry Ramón de la Fuente Muñiz, Mexico City, Mexico

Nur Hani Zainal, PhD; Department of Health Care Policy, Harvard Medical School, Boston, MA, United States; Department of Psychology, Kent Ridge Campus, National University of Singapore, Kent Ridge, Singapore

Yesica Albor, PhD; Center for Global Mental Health Research, National Institute of Psychiatry Ramón de la Fuente Muñiz, Mexico City, Mexico

Libia Alvis-Barranco, PhD; Department of Psychology, Universidad Popular de Cesar, Valledupar, Colombia

Nayib Carrasco Tapia, PhD; Department of Psychology, Universidad Cooperativa de Colombia, Medellin, Colombia

Carlos C Contreras-Ibáñez, PhD; Department of Psychology, Universidad Cooperativa de Colombia, Medellin, Colombia

Jacqueline Cortés-Morelos, MD; Department of Psychiatry and Mental Health, Universidad Nacional Autónoma de México, Mexico City, Mexico

Lorena Cudris-Torres, PhD, Department of Psychology, Fundación Universitaria del Area Andina, Valledupar, Colombia

Francisco R de la Peña, MD; Unit of Research Promotion, National Institute of Psychiatry Ramón de la Fuente Muñiz, Mexico City, Mexico

Noé González, MA; Center for Global Mental Health Research, National Institute of Psychiatry Ramón de la Fuente Muñiz, Mexico City, Mexico

Raúl A. Gutierrez-Garcia, PhD; Center for Global Mental Health Research, National Institute of Psychiatry Ramón de la Fuente Muñiz, Mexico City, Mexico

Eunice Vargas-Contreras, PhD; Facultad de Ciencias Administrativas y Sociales, Universidad Autónoma de Baja California, Ensenada, Mexico

Maria Elena Medina-Mora1, PhD; Center for Global Mental Health Research, National Institute of Psychiatry Ramón de la Fuente Muñiz, Mexico City, Mexico; Seminar of Studies on Globality, National Autonomous University of Mexico (UNAM), Mexico City, Mexico; Faculty of Psychology, National Autonomous University of Mexico (UNAM), Mexico City, Mexico

Pamela Patiño, PhD; Center for Global Mental Health Research, National Institute of Psychiatry Ramón de la Fuente Muñiz, Mexico City, Mexico

Sarah M Gildea, BS; Department of Health Care Policy, Harvard Medical School, Boston, MA, United States

Chris J Kennedy, PhD; Department of Psychiatry, Massachusetts General Hospital, Boston, MA, United States

Alex Luedtke, PhD; Department of Statistics, University of Washington, Seattle, WA, United States; Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, WA, United States

Nancy A Sampson, BA; Department of Health Care Policy, Harvard Medical School, Boston, MA, United States

Maria V Petukhova, PhD; Department of Health Care Policy, Harvard Medical School, Boston, MA, United States

Jose R Zubizarreta, PhD; Department of Health Care Policy, Harvard Medical School, Boston, MA, United States

Pim Cuijpers, PhD; Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

Alan E Kazdin, PhD; Department of Psychology, Yale University, New Haven, CT, United States

Ronald C Kessler, PhD; Department of Health Care Policy, Harvard Medical School, Boston, MA, United States

Background: Web-based cognitive behavioral therapy (wb-CBT) is a scalable way to reach distressed university students. Guided wb-CBT is typically superior to self-guided wb-CBT over short follow-up periods, but evidence is less clear over longer periods. 

Objective: This study aimed to compare short-term (3 months) and longer-term (12 months) aggregate effects of guided and self-guided wb-CBT versus treatment as usual (TAU) in a randomized controlled trial of Colombian and Mexican university students and carry out an initially unplanned secondary analysis of the role of differential predicted compliance in explaining these differences. 

Methods: The 1319 participants, recruited either through email and social media outreach invitations or from waiting lists of campus mental health clinics, were undergraduates (1038/1319, 78.7% female) with clinically significant baseline anxiety (Generalized Anxiety Disorder–7 score≥10) or depression (Patient Health Questionnaire–9 score≥10). The intervention arms comprised guided wb-CBT with weekly asynchronous written human feedback, self-guided wb-CBT with the same content as the guided modality, and TAU as provided at each university. The prespecified primary outcome was joint remission (Generalized Anxiety Disorder–7 score=0-4 and Patient Health Questionnaire–9 score=0-4). The secondary outcome was joint symptom reduction (mean scores on the Patient Health Questionnaire Anxiety and Depression Scale) at 3 and 12 months after randomization.

Results: As reported previously, 3-month outcomes were significantly better with guided wb-CBT than self-guided wb-CBT (P=.02) or TAU (P=.02). However, subsequent follow-up showed that 12-month joint remission (adjusted risk differences=6.0-6.5, SE 0.4-0.5, and P<.001 to P=.007; adjusted mean differences=2.70-2.69, SE 0.7-0.8, and P<.001 to P=.001) was significantly better with self-guided wb-CBT than with the other interventions. Participants randomly assigned to the guided wb-CBT arm spent twice as many minutes logged on as those in the self-guided wb-CBT arm in the first 12 weeks (mean 12.5, SD 36.9 vs 5.9, SD 27.7; χ2 1=107.1, P<.001), whereas participants in the self-guided wb-CBT arm spent twice as many minutes logged on as those in the guided wb-CBT arm in weeks 13 to 52 (mean 0.4, SD 7.5 vs 0.2, SD 4.4; χ2 1=10.5, P=.001). Subgroup analysis showed that this longer-term superiority of self-guided wb-CBT was confined to the 40% (528/1319) of participants with high predicted self-guided wb-CBT compliance beyond 3 months based on a counterfactual nested cross-validated machine learning model. The 12-month outcome differences were nonsignificant across arms among other participants (all P>.05). 

Conclusions: The results have important practical implications for precision intervention targeting to maximize longer-term wb-CBT benefits. Future research needs to investigate strategies to increase sustained guided wb-CBT use once guidance ends.

Anxiety; depression; web-based cognitive behavioral therapy; compliance; randomized controlled trial

“Web-based cognitive behavioral therapy (wb-CBT) has been suggested as a promising strategy for increasing the scalability, reach, and affordability of mental health services for clinically significant anxiety and depression, especially in populations with internet access and literacy, such as university students, and for populations with limited availability of in-person mental health services, such as those in Latin America. wb-CBT has generally been found to be as effective as face-to-face cognitive behavioral therapy (CBT). However, uptake and engagement have been important challenges” (p. 2).

“This report presents the first results of a controlled trial in LMICs to compare the effects of guided and self-guided wb-CBT 12 months after randomization. We focused on a sample of Colombian and Mexican university students with anxiety or depression. We also expanded our earlier preplanned investigation of heterogeneity in comparative intervention effects with a secondary analysis focused on determining whether a subset of participants can be identified at baseline (ie, before randomization) who would have equal or better longer-term outcomes with self-guided compared to guided wb-CBT and the extent to which differential long-term intervention compliance might account for such heterogeneity” (p. 3).

“Participants were 1319 undergraduate students (n=1038, 78.7% female; median age 21, IQR 19-22 years) from 7 universities in Colombia and Mexico who were recruited either through email and social media outreach invitations or from waiting lists of campus mental health clinics in the universities that had such clinics. Inclusion criteria were completing the baseline assessment and reporting clinically significant anxiety (Generalized Anxiety Disorder–7 [GAD-7] scores of ≥10) [19] or depression (Patient Health Questionnaire–9 [PHQ-9] scores of ≥10) [20] and consenting to be randomly assigned to guided wb-CBT, self-guided wb-CBT, or TAU… The wb-CBT program (both guided and self-guided modalities) was a culturally adapted version of SilverCloud Health by Amwell’s Space from Depression and Anxiety program, a transdiagnostic wb-CBT program… Participants assigned to the guided program receive weekly asynchronous written messages during the first 8 weeks after randomization through the platform from trained Bachelor of Arts–level coaches with a degree in psychology intended to generate personalized experiences and offer feedback [24]. The intervention has 7 primary modules that focus on cognitive restructuring, behavioral activation, and relaxation techniques and several other ancillary modules (eg, sleep and anger)... TAU consisted of referral to the clinic in the 3 universities that had student mental health clinics and referral to informal counseling services that faculty provide in the other universities to place students with anxiety or depression with community treatment providers” (p. 3).

“We focused on university students with anxiety or depression in 2 upper-middle–income Latin American countries, Colombia and Mexico. There are 2 main findings that are noteworthy. First, 12-month outcomes were significantly better for participants randomly assigned to self-guided wb-CBT than for those randomly assigned to either guided wb-CBT or TAU, whereas 12-month outcomes were not better for those randomly assigned to guided wb-CBT than for those randomly assigned to TAU, even though 3-month outcomes were best for those randomly assigned to guided wb-CBT. Second, we found that these significant differences were confined to the 40% (528/1319) of participants with the highest predicted compliance with self-guided wb-CBT over weeks 13 to 52 based on the covariates assessed before randomization. This specification suggests that the superiority of self-guided wb-CBT at 12 months is due to the higher continued use of self-guided wb-CBT than of guided wb-CBT after the guidance ends” (p. 14).

“There are several clinical implications to these findings. First, the results argue that the dominant view of self-guided wb-CBT as inferior to guided wb-CBT is unwarranted if one considers 12-month outcomes. Ideally, we would want to promote long-term remission rather than the shorter-term remission that has been the focus of most previous wb-CBT research. Second, the importance of sustained use of the self-guided wb-CBT platform suggests that learning CBT skills is not enough in itself but that ongoing practice and review of materials is needed, presumably to help consolidate acquired CBT skills in various life contexts. Participants randomly assigned to guided wb-CBT were trained to be extrinsically, instead of intrinsically, motivated to engage with the intervention by virtue of the external reinforcement they received from their guide, leading to a greater reduction in use of the intervention once guidance ended. Importantly, this was true even for the subset of these participants whose baseline profiles suggested that they would have complied intrinsically if they had been randomly assigned to self-guided wb-CBT. The individuals with this baseline profile who were randomly assigned to self-guided wb-CBT, in comparison, had significantly better longer-term outcomes than if they had been randomly assigned to guided wb-CBT because self-guidance allowed these individuals to consolidate their intrinsic motivation to support longer-term use of the intervention. One implication of this finding is that guided wb-CBT programs need to consider how best to foster intrinsic motivation, possibly through strategies such as intermittent guidance, the use of longer-term booster sessions, tapering guidance over a longer time, or offering guidance on demand or just-in-time adaptive guidance. Although some limited research on such possibilities exists, our results suggest that this area warrants further study. For individuals assigned to the self-guided modality, compliance may have been more intrinsically motivated from day 1 as these participants never had the positive reinforcement of a human guide, increasing the probability that participants with high predicted compliance continued to use the intervention over the full 12-month access period. Interestingly, not being employed and having high comorbidity (especially SAD) were among the strongest predictors of longer-term self-guided wb-CBT compliance. Why not being employed would be a predictor is unclear considering that this was a university student sample, but it might be due to having more time available and being less overextended. Participants with SAD might have complied more with the self-guided wb-CBT because they found it less threatening than interacting with a guide. Finally, whether the focus of treatment planning should be more on short-term or longer-term outcomes is unclear. It may be more critical to reduce symptomatology as quickly as possible (and, thus, prioritize short-term outcomes), but we also want intervention effects to persist because of the recurrent nature of anxiety and depression. It is unclear whether an approach exists to do both given that guidance appears to increase short-term compliance but reduce longer-term compliance” (p. 14-15). 

“Our study had 4 noteworthy limitations. First, this study was carried out during the COVID-19 pandemic. It is unclear whether this influenced the results. Second, overall intervention compliance was low. This is consistent with many other web-based intervention studies, may be due not only to characteristics of the users but also to characteristics of the programs, and might have been exacerbated by the pandemic. Because of this low compliance and context, our results involving interactions between intervention assignment and predicted compliance might not be generalizable beyond the specific web-based interventions considered here. Third, even though we had a large set of baseline covariates, these variables were chosen as predictors of treatment response rather than of intervention compliance. Future research designed to study the effects of compliance should include a broader set of baseline covariates that include known predictors of compliance. Fourth, TAU was heterogeneous across universities and was mainly administered via videoconferencing because this trial was conducted during the COVID-19 pandemic lockdown. Taken together, these limitations suggest that caution should be taken in assuming that the results are generalizable beyond the specific time and setting in which this trial was carried out” (p. 15).