Protocol for a hybrid type 2 cluster randomized trial of trauma-focused cognitive behavioral therapy and a pragmatic individual-level implementation strategy

Aim 1 will evaluate, relative to control, the effects of TF-CBT conditions on TF-CBT’s identified mechanisms of change (trauma-related cognitions, emotion regulation, behavioral avoidance), differences in child mental health outcomes, and intervention costs and cost-effectiveness.

Aim 2 will evaluate the main effects of BASIS, relative to control, on its theoretical implementation mechanisms, implementation outcomes, as well as costs and cost-effectiveness. We will also evaluate “hypothesis-defying residuals” (i.e., clinicians whose implementation behaviors are unaccounted for by the theoretical model) to further refine our BASIS theory of change.

Clinician participants will include 120 SMH providers (40 per wave). SMH providers will consist of school psychologists, social workers, mental health counselors, etc. recruited from elementary, middle, and high schools from economically and ethnically diverse districts. One clinician per school will be recruited with the assistance of district administrators. The research team will contact eligible participants to describe the purpose of the study, research procedures, and incentives. Clinicians will be included if they (a) serve in a professional role to provide school-based mental health services, (b) hold a graduate degree or equivalent certification or experience, (c) have not previously received formal training in TF-CBT, and (d) are not actively receiving support to implement another related intervention.

Youth participants will include 480 students (160 per wave), recruited by TF-CBT and TAU therapists. Youth will meet TF-CBT eligibility criteria, including (a) be enrolled in grades 3–12 and (b) have traumatic event exposure (e.g., exposure to violence) and (c) significant post-traumatic stress symptoms. SMH clinicians will recruit students by screening through their standard referral pathways (e.g., teacher- or self-referral; screening) for PTSD symptoms. Once students are identified, clinicians will ask caregivers whether they are open to being contacted by the research team. For caregivers who agree, contact information will be relayed to the team, who will follow-up by phone to describe the research project, estimated time to participate, participant compensation, and obtain consent (see Additional File 4 for all consent forms). For caregivers who consent, research staff will explain the study to students, answer questions, and obtain assent for participation. Four to six students will be recruited by each clinician. Based on previous research [54, 55], it is expected that students will be ~ 50% female and representative of the schools in which they are enrolled.

TF-CBT is a 12–16 session intervention for children aged 3 to 18 years with trauma-exposure and related mental health sequelae. TF-CBT includes individual sessions for the youth, individual sessions for parents, and conjoint sessions that include both the youth and parent. TF-CBT has established training and consultation protocols, as well as a psychometrically strong objective fidelity instrument (see Table 1) [56]. All clinicians assigned to one of the TF-CBT conditions will participate in “gold standard” virtual TF-CBT training including completing of an online, self-paced 8–10 h didactic training, a virtual live 3-day training (3–4 h per day) with a certified TF-CBT trainer, and 6 months of post-training consultation. Consultation groups (6–8 providers/each) will be formed within condition to avoid contamination.

The enhanced TAU condition is intended to ensure safety and support SMH providers who have variable levels of experience treating trauma directly and reduce unwanted variance in TAU. The research team developed a brief protocol to provide guidelines for psychoeducation and post assessment connection and support as scaffolding for usual care in schools. To maintain internal validity, this companion protocol does not include in depth attention to the elements of TF-CBT (e.g., exposure) hypothesized to be responsible for its effects. TAU scaffolding will be provided via an online presentation with a live question and answer section at the conclusion.

Second, BASIS includes social influence techniques to alter perceptions of subjective norms. Evidence-based social influence strategies consist of two broad categories: (1) social proofing messages that use data or testimonials to describe the behavior or attitudes of others and (2) strategies to induce cognitive dissonance. Social proofs are most influential from individuals with whom they closely identify [57], especially when testimonials speak to the usefulness of the specific behavior [58, 59]. Strategies to induce cognitive dissonance operate on the premise that individuals strive for consistency between their attitudes and actions [60]. Thus, desired behaviors can be increased by evoking commitments that are active (vs. passive), public (vs. private), and voluntary (vs. coerced) [61, 62]. These strategies are integrated throughout BASIS. For instance, normative data and testimonials are used to normalize clinician experience of barriers to EBT implementation (e.g., lack of time, low administrative support), express commitment to problem solve barriers, and debunk common myths about EBTs.

Third, motivational interviewing (MI) is used to enhance self-efficacy. MI is a nondirective, patient-centered approach with strong evidence for building engagement and commitment for behavior change [63‐68]. The BASIS facilitator utilizes group MI techniques by adopting an empathic, nondirective, and person-centered style to elicit self-motivational statements and encourage “change talk” (i.e., statements about making behavioral changes). Participants engage in a values affirmation activity [69, 70] that has been shown to decrease defensiveness toward change and enhance motivation [71]. To further enhance self-efficacy, participants also anticipate barriers that may arise in implementation and collaborate to generate solutions to those barriers and engage in decisional balance activities to reflect on the pros and cons of changing or not changing.

To address the intention-behavior gap, BASIS includes volitional planning interventions derived from HAPA to increase the likelihood that clinicians will maintain self-efficacy and act upon their intentions by enacting implementation behaviors. Specifically, action planning and problem-solving planning have been shown to facilitate health behaviors such as breast cancer self-examinations, medication adherence, exercise, and healthy eating [72‐77]. Action planning supports translation of intentions into actions through detailed planning of how to perform behaviors in specific contexts. Problem-solving generates solutions in response to both situational and internal (e.g., cognitive) barriers to facilitate follow through with the action plan. In combination, action planning and problem-solving planning increase the likelihood that implementation intentions translate into behavior change [39]. Action planning and problem-solving planning occur immediately post-EBT training.

The BASIS pre-training session targets attitudes, subjective norms, and self-efficacy via the motivational components listed above. The pre-training opens with the facilitator engaging SMH providers in an activity to clarify their professional values (MI component). Strategic education components are not delivered didactically, but rather to facilitate interaction among participants. Open-ended questions are used to elicit change talk. Testimonials are interspersed throughout (social influence). At the end of the pre-training session, providers collaborate with each other to develop an individualized menu of potential solutions to common implementation barriers that they can select from when encountering challenges to adopting and delivering new practices with fidelity. Last, they set value-congruent goals related to participation and engagement in the upcoming EBT training.

The BASIS post-training session includes volitional strategies shown in prior research to maintain implementation intentions and facilitate actual enactment of behavior change. Specifically, clinicians are supported to develop action plans and problem-solving plans. Clinicians are provided with an action planning template to detail precisely what TF-CBT components, how, with whom, where/when, and the environmental cues and resources needed to initiate delivery of TF-CBT with fidelity. The problem-solving plan involves clinicians anticipating situational and internal barriers and generating solutions to overcome those barriers to develop personalized if-then plans that can be used when confronted with specific barriers.

The BASIS online booster is delivered at approximately 15 days post-training, a time point when clinicians’ implementation intentions and behaviors may first weaken [78]. The aim of the BASIS booster is to provide adaptive content to clinicians to either increase intentions to implement or maintain self-efficacy to implement the EBT, depending on whether or not clinicians have initiated TF-CBT implementation. For instance, clinicians who have not yet initiated implementation will (1) be provided with a distilled version of the pre-training BASIS content (improving attitudes, subjective norms, and self-efficacy) and (2) be supported to revise their action and problem-solving plans with specific attention to components of these plans that were not aligned with their implementation intentions or the constraints of their service settings.

A BASIS fidelity tool [78] will be used to rate recordings of BASIS delivery by trained research assistants. Each recording will be coded independently by two different raters and disagreements resolved through consensus dialog to capture facilitator adherence and participant responsiveness via engagement [79, 80].

Clinician surveys will be administered via a secure web-based system at 12 time points for BASIS/AC and 9 time points for TAU, beginning in the fall of each year. Clinicians will self-report their demographic characteristics, BASIS mechanisms (attitudes, subjective norms, self-efficacy, maintenance self-efficacy), implementation intentions, organizational moderators (implementation climate, leadership), and TF-CBT sessions delivered. For BASIS/AC, surveys will be administered at baseline through post-training (T1 prior to the self-paced course, T2 after the self-paced course, T3 after BASIS/AC pre-training session, T4 after the TF-CBT virtual training, and T5 at end of the BASIS/AC post-training session), 2 months from baseline and after the BASIS/AC booster (T6), winter (T7), spring (T8), and end of school year (T9), as well as three sustainment time points during the subsequent year (T10 in fall or year 2, T11 in winter, and T12 in spring). For TAU, we will administer the T1, T5, and T6–T12 surveys.

To assess TF-CBT fidelity, clinicians will record all sessions for participating TF-CBT and TAU students. Three sessions (one from each phase of TF-CBT) will be randomly selected and coded for fidelity using an established, systematic TF-CBT coding protocol (Table 2).

Adoption and sustainment data will be collected via the TF-CBT “Toolkit,” an online TF-CBT tracking system used by clinicians to determine client eligibility, log client sessions, and facilitate consultation.

To fully address aim 2 and identify factors unaccounted for by our BASIS theory of change (Fig. 1), unexplained residuals from aim 2 mediation analyses will be explored qualitatively via semi-structured phone interviews at the end of the active implementation phase. Residuals are defined as clinicians whose implementation behavior is insufficiently accounted for by our mediation model (e.g., clinicians with favorable implementation outcomes, but who demonstrate low levels of BASIS mechanisms). Clinicians from aim 2 will be identified at the end of their first year of participation based on the results of aim 2 quantitative modeling. Participants will include those whose predicted probability score is greater than 1.0 standard deviations of the mean from their predicted behavior, balanced between implementers and non-implementers and TF-CBT and AC conditions (approximately 15–19 clinicians total). The mixed methods design will be sequential in structure; the functions are sampling and expansion; and the process is connecting [98‐100]. We will develop a systematic, comprehensive semi-structured interview guide that draws from the EPIS framework [17] to examine multilevel (i.e., intervention, individual, inner setting, outer setting) [101] determinants that explain what processes facilitated or hindered EBT implementation and sustainment.

Activity-based costing will estimate the respective, incremental costs of enhanced TAU, TF-CBT, and BASIS. For TF-CBT/TAU costs, we will estimate direct costs of TF-CBT and enhanced TAU, such as training, consultation and delivery of services, and indirect costs, such as lost opportunities for alternative activities. For BASIS costs, we will directly measure the resource use and incremental costs associated with BASIS and TF-CBT training as compared to status quo TF-CBT training (i.e., TF-CBT implementation-as-usual) [102]. We will identify activities related to training and associated labor and non-labor inputs. Inputs can include time, supplies, travel, overhead, and costs associated with TF-CBT and BASIS training meetings, including pre-work, scheduling, etc.

We will combine incremental costs collected for BASIS, TF-CBT, and TAU with estimates of effectiveness on implementation and clinical (see “Cost and cost-effectiveness analyses” section).

We will test the direct main effects of TF-CBT on the hypothesized intervention mechanisms (trauma-related cognitions, emotion regulation, behavioral avoidance) and child mental health outcomes (e.g., symptoms of post-traumatic stress), via longitudinal mixed effects models (time within client within clinician) for each mechanism, using effect coding for each of the 3 conditions (BASIS, AC, enhanced TAU) in an intention-to-treat approach, and focusing our comparisons on the main effect of TAU vs. TF-CBT conditions. To increase statistical power to detect an effect of TF-CBT, we will include as covariates any variables at the client, provider, or school level that are associated with the mechanism variable, as indicated during preliminary bivariate analyses. Path analysis will then be used to test the mediated effect of mechanisms. Mixed effect models will estimate standardized coefficients for each path separately, as well as together, and any significant reduction in the condition to outcome coefficient when including the other paths will be suggestive of partial or full mediation, using the bias-corrected bootstrap method to test significance [103].

For aim 2, clinicians assigned to the enhanced TAU condition will be excluded from analyses. We will test for the impact of BASIS on proximal implementation mechanisms of change via a series of piecewise longitudinal mixed effects models (time within clinician) to examine between-condition differences on the rates of change across all twelve time points pieced into three epochs (T1–T5, T6–T9, T10–T12) for each of the five primary BASIS mechanisms of change. As compared to AC, we hypothesize steeper gains for BASIS from T1 to T5, smaller rates of decline for BASIS from T6 to T9, and higher levels of sustainment for T10 to T12. The impact of BASIS on fidelity will be tested using mixed effects models, with fidelity measurement occurrence nested within clinician. Adoption and sustainment will be tested using logistic regressions using condition as a predictor (adoption: yes/no used TF-CBT in first year; sustainment: yes/no used TF-CBT in second year). We will stratify sustainment analyses (T10–T12) as follows: (1) the entire population of participants, regardless of year 1 implementation, in order to capture any sustained impact of BASIS, (2) the subgroup of clinicians who implemented in year 1, to examine the sustainment of TF-CBT implementation. Mediation will be tested as in aim 1.

Based on aim 2 quantitative models, we will identify clinicians who have a difference between predicted and actual implementation behavior of ≥ 1 SD. Transcribed data will be coded using an integrated directed content analysis [104] approach as certain codes will be conceptualized during the interview guide development (deductive approach) and other codes will be developed through a close reading of an initial subset of transcripts (inductive approach) [105]. These themes will provide a way of identifying and understanding the most salient factors that impact implementation and extend beyond the existing BASIS mechanisms and theory of change [106, 107]. Directed coding will be driven by the EPIS framework [17]. After a stable set of codes is developed, a consensus process will be used in which all reviewers independently code and compare their coding to arrive at consensus judgments through open dialog [79, 80, 108].

In all main effects analyses, the Benjamini-Hochberg procedure adjusts for familywise error within outcome families (e.g., “attitudes”). Missing data will be addressed in modeling using restricted maximum likelihood estimation.

For both aims, cost data collection and analyses of TF-CBT and BASIS will estimate incremental total and unit costs for TAU, BASIS+TF-CBT, and TF-CBT during both the implementation and sustainment periods [109]. We will estimate the average weighted cost metrics across the sample of study sites. For each condition, we will estimate the average incremental costs for (1) total economic costs and (2) cost per clinician trained (stratified by TAU training, BASIS training, and TF-CBT training). The cost-effectiveness analysis will compare the incremental net costs with the benefits defined as changes in clinical outcomes, such as PTS, depression, and anxiety, across the control (TAU) and the two arms TF-CBT and BASIS+TF-CBT.

Our conservative power analyses indicate that we are powered to detect minimum detectable effect sizes (MDES) of Cohen’s d > .35 for direct effects on student mechanisms and outcomes, assuming an intraclass correlation of .05, paired two-group (e.g., BASIS vs. enhanced TAU) tests with 35 schools/clinicians in each group (to account for attrition/missing data) and 4 clients served by each provider. Assuming some variance in the number of clients per clinician ranging from 1 to 7, a Monte Carlo simulation estimated that the maximum detrimental impact on power will be less than 10% [110]. Using bias-corrected bootstrapping, we will have sufficient power to detect mediation when each of the two mediation paths is of d > .2 (small effects) [111].

Conservatively assuming 35 clinicians in each group and 12 time points, we will have sufficient power to detect a Cohen’s d of .68 for clinician implementation outcomes, which is lower than most significant effects during our pilot study [78]. We will have power to detect mediation effects when the relationship of BASIS to implementation mechanisms is d > .59 (as expected) and mechanisms to implementation outcomes has at least small/moderate effects (d > .31) [111].