Addressing COVID-19 vaccine hesitancy in rural community pharmacies: a protocol for a stepped wedge randomized clinical trial

Compared to the standard implementation approach, test whether adding virtual facilitation increases (a) the fidelity with which pharmacists implement the vaccine hesitancy counseling intervention and (b) the number of vaccine hesitant patients who receive the vaccine. Our primary outcome is pharmacist fidelity to the vaccine hesitancy counseling intervention, and our secondary outcome is intervention effectiveness. We hypothesize that virtual facilitation improves fidelity to the intervention and intervention effectiveness when compared to standard implementation.

Conduct a cost assessment and explore the potential sustainability of the implementation approaches. We will conduct time-driven, activity-based cost-effectiveness analysis and budget impact analyses to estimate the relative value and feasibility of standard implementation and virtual facilitation approaches. Additionally, a payer advisory committee (PAC) will review these data and advise on how to make virtual facilitation sustainable through reimbursement models.

All data will be collected from rural pharmacies that participate in the Rural Reseach Alliance of Community Pharmacies (RURAL-CP), which was established by this trial’s multiple principal investigators Carpenter and Curran in 2020 [52]. RURAL-CP is the first practice-based research network for rural community pharmacies and is registered with the Agency for Healthcare Research and Quality. The mission of RURAL-CP is to reduce rural health disparities by supporting high-quality implementation research with community pharmacies. At the time of the proposal’s submission, RURAL-CP consisted of 111 rural community pharmacies from five southeastern states (Alabama, Arkansas, Mississippi, North Carolina, and South Carolina) and included a mix of independent, regional chain, and small grocery store pharmacies, ranging from isolated non-metro rural areas to larger, micropolitan rural areas [53]. RURAL-CP has since expanded to seven states and 127 pharmacies (recruitment of new pharmacies ongoing).

We will recruit 30 RURAL-CP pharmacies. In order to target rural communities with high levels of vaccine hesitancy, 15 pharmacies will be recruited from counties that have African American populations ≥ 25.1%, which is the mean percentage of African Americans in RURAL-CP counties in 2020 according to the U.S. Census, and 15 pharmacies will be recruited from counties that had at least 64.9% vote for a Republican president in 2020, which is the mean percentage of individuals who voted for a Republican in RURAL-CP counties. We have selected this recruitment strategy because both African American and Republican/conservative populations have consistently had very high levels of vaccine hesitancy [7, 54, 55]. Pharmacies will be invited to participate initially via email and before randomization. RURAL-CP-affiliated state leads will do follow-up calls if emails generate no response. Trial MPIs (Curran and Carpenter) will obtain informed consent over the phone with pharmacy leads. If a pharmacy declines participation or “drops out” after initially enrolling, we will replace it with a randomly selected RURAL-CP pharmacy from the same or similarly qualifying county.

The ASORT intervention will ask pharmacists to identify and engage in COVID-19 vaccine hesitancy counseling with one to two vaccine-hesitant individuals weekly. As noted earlier, we have refined an evidence-based vaccine hesitancy counseling intervention with extensive feedback from rural pharmacists. These refinements resulted in a 5-step counseling process (ASORT; see Table 1) as well as an online resource that provides example verbiage for over 25 vaccine concerns that have been expressed in rural communities. The vaccine hesitancy verbiage is updated periodically with input from our rural pharmacist (four RURAL-CP pharmacists) and rural patient (four rural patients; two African American, two with Republican party affiliation) advisory boards.

The standard approach will train and prepare pharmacists to implement ASORT and provide discrete implementation support tools to support intervention fidelity. Specifically, a trial website will include numerous tools, including example vaccine hesitancy verbiage, sample workflows, marketing materials, and patient pamphlets. The standard approach also includes an online training module developed by the trial team that incorporates similar instructional design principles that have been used previously to develop pharmacist communication-focused modules [56, 57]. Finally, just prior to the start time of each block of pharmacies, participants will attend a live webinar with continuing education (CE) credit that includes interactive training on the intervention, updated vaccination recommendations, vaccine storage and delivery, and documentation.

The virtual facilitation approach will provide expert guidance from trained facilitators regarding intervention content and implementation processes. The facilitators will perform the following evidence-supported functions [58‐62]: engaging stakeholders; building relationships; identifying and training a local facilitator/champion; monitoring progress; providing feedback on progress; identifying implementation barriers; problem-solving; re-training and coaching; and reinforcing change (see Fig. 1). Facilitators will attend a 16-h virtually-delivered training in implementation facilitation provided by the Implementation Facilitation Learning Hub, a training center supported by the U.S. Department of Veterans Affairs (VA) Quality Enhancement Research Initiative (QUERI). The training teaches principles and techniques contained in Dollar et al.’s manual Using Implementation Facilitation to Improve Healthcare (Version 3 [61];), which was developed by the VA Behavioral Health QUERI, a research center devoted to supporting the implementation of behavioral health interventions with the VA. The training is highly interactive [63], involving significant practice and role play. Training topics include knowledge, skills, core competencies of facilitators; facilitation roles and activities (e.g., assessing the site, engaging stakeholders, problem identification and resolution); phases of implementation; delivering facilitation virtually; and evaluating facilitation. After the training, Dr. Curran (trial co-principal investigator) will provide ongoing coaching and supervision to the trial facilitators.

Both quantitative and qualitative data will be collected. We will use Qualtrics (Provo, UT) to collect aggregated, de-identified data on effectiveness (the secondary outcome) and adoption. Research staff will complete assessments of fidelity (the primary outcome) and costs (time and activity logs subsequently converted to costs). At the end of the standard and virtual facilitation periods, pharmacy staff from each pharmacy will complete surveys to assess the other implementation outcomes listed in Table 2. We also will select one high and one low-performing pharmacy from each block of our stepped-wedge design and conduct interviews with at least four pharmacy staff (pharmacists and pharmacy technicians) per pharmacy, immediately after the virtual facilitation period has ended, to provide context for the quantitative outcomes.

Fidelity (the primary outcome) to the vaccine hesitancy counseling intervention will be assessed at the pharmacy level. Our fidelity measure is based on a theoretical framework of fidelity measurement [64] as well as a validated fidelity checklist [65]. The fidelity measure focuses on the competence of the pharmacist (7 items) in their delivery of the vaccine hesitancy counseling intervention.

The competence items focus on the skillfulness of intervention delivery: expressed empathy; used a non-confrontational manner; spoke confidently without using jargon; emphasized patient autonomy; reflected back patient’s statements accurately; used a respectful demeanor, and used evidence-based responses when responding to patient vaccine concerns. Each competence item will be assessed on a scale from 0 to 2, with 0 = skill not demonstrated, 1 = skill needs development, and 2 = skill demonstrated with competence. Competence scale scores will range from 0 to 14, with higher scores reflecting greater competency in the delivery of ASORT. Fidelity will be measured for each pharmacist approximately twice per month under the standard implementation approach and approximately twice per month under the virtual facilitation approach. In pharmacies with more than one pharmacist, fidelity ratings will be averaged to achieve a pharmacy-level measure.

Trained staff will rate fidelity after reaching 80% inter-rater reliability during training [66]. Staff who are blinded to the pharmacist’s group assignment will use a fidelity observation guide to rate pharmacists’ vaccine hesitancy counseling during recorded counseling sessions. Pharmacists will be instructed to seek permission from patients to audio record all interventions until two are recorded and submitted each month. Patients will provide verbal consent before the session is recorded. Previous research has demonstrated that recording health professionals’ communication does not alter their communication behavior [67, 68]. A HIPAA-compliant phone application will be used to allow the pharmacists to record the interventions on their own phones, securely send the recordings to a trial team member, and then delete the recordings once they have been rated. During virtual facilitation, ratings will be shared with the facilitators who provide feedback and coaching to pharmacists towards improving fidelity.

Effectiveness (the secondary outcome) will be assessed on a monthly basis. Pharmacists will be instructed to deliver vaccine hesitancy counseling to one to two vaccine-hesitant individuals each week (towards a target of at least 5 per month). Using Qualtrics, pharmacists will document the following on a daily basis:

Acceptability, appropriateness, and feasibility of the vaccine hesitancy intervention and implementation approaches (standard, facilitation) will be measured using validated surveys [69]. Each measure includes 4 items (e.g., “intervention X seems doable” [feasibility]) measured on a 5-point response scale that ranges from “completely disagree” (coded as 1) to “completely agree” (coded as 5). For each pharmacy, we will average scores across the five pharmacy staff members who complete the surveys.

In this stepped-wedge trial, pharmacies will be randomized in blocks at the time they begin the initial intervention condition (standard implementation). We will create 6 blocks of pharmacies, with 5 pharmacies randomized to each block by the trial statistician (Ounpraseuth). Figure 2 depicts the “stairstep” [76] pharmacy assignment and randomization plan. We will employ two-stage randomization—first to a block and then to a start time. To ensure balance, the block randomization will be stratified by two measures of pharmacy size—the number of patients and number of pharmacists (which are reported on the organizational structure survey). Each block will participate in either one or two 8-week standard implementation periods followed by one or two 8-week virtual facilitation periods. For blocks 1–4, we will continue to collect data during one 8-week follow-up period to evaluate the potential impact of the virtual facilitation approach on sustained intervention fidelity and effectiveness once virtual facilitation has been stopped.

The design described here is a modification of our original design. We had to revise the design to follow the expected future distribution of COVID-19 vaccines in the USA, which will follow the Northern Hemisphere’s “flu season”—approximately August-February each year. We expect that the first COVID-19 vaccination season in the fall of 2023 will be delayed due to vaccine availability and will begin in October and run through March 2024. In 2024, we expect that availability issues will not occur and the vaccine will be available during the standard flu season from August-February. The original design built on “year-round” availability of vaccine instead of seasonal or intermittent availability, with 3 blocks of 10 pharmacies receiving 4 months of standard implementation, 4 months of virtual facilitation, and at least one 4-month period of follow-up.

This unavoidable change in design comes with potential positive and negative impacts. A potential negative effect is the need to reduce the timeframe of each implementation observation period from 4 to 2 months. This potential negative impact is counteracted by the fact that the ASORT intervention is focused and brief, and that impacts of the standard and virtual facilitation approaches can be observed within two months’ time. Further, the adjustment to the flu season calendar allows us to explore the relative impacts of receiving the standard implementation and virtual facilitation approaches for 2 or 4 months, which has helpful implications, especially for the cost-effectiveness analysis. The design change negatively impacts plans to assess implementation sustainment in follow-up periods. However, we are still able to assess for evidence of immediate sustainment across two blocks of pharmacies (1 and 4), while assessing for evidence of lagged sustainment across two blocks of pharmacies (2 and 3). Immediate sustainment will be measured in the 8-week period immediately following receipt of virtual facilitation, possible only for pharmacies in blocks 1 and 4. Lagged sustainment will be measured within the initial 8 weeks of vaccine availability in the next “season,” possible only for blocks 2 and 3 within the timeline and budget constraints. The proposed assessment of sustainment was already a secondary outcome, but the design change allows only a preliminary look at sustainability potential of the two implementation approaches.

We will use the principles of intention-to-treat, using multiple imputations if needed for missingness, for all statistical analyses related to primary and secondary endpoints. Descriptive statistics will be computed for all quantitative implementation outcomes in Table 2. The primary outcome of fidelity is captured at the pharmacy level. For our primary analysis of the fidelity outcome, we will use linear mixed-effects models (LMM) to compare the level of fidelity between the two implementation approaches (standard approach vs. virtual facilitation). We will report point estimates for the group mean difference along with a 95% confidence interval. The model-building approach will follow four analyses steps: (1) an unadjusted before/after of the effect of the virtual facilitation approach (ignoring period/time effect); (2) the time period (i.e., steps/blocks) to examine if any potential intervention effect relates only to the intervention or also to an independent effect of calendar time; (3) an adjustment for potential pharmacy-level confounders, such as size and learning climate; and (4) the interaction between period and intervention effect.

For the secondary outcome of effectiveness, we will use generalized linear mixed models (GLMM) to investigate whether pharmacies are more effective at addressing vaccine hesitancy during virtual facilitation when compared to standard implementation. The effectiveness outcome will be binary (vaccine-hesitant patient accepts vaccine after counseling = 1; vaccine-hesitant patient does not accept vaccine after counseling = 0). We will report the odds ratio estimate of accepting the vaccine after counseling for the virtual facilitation approach (facilitation versus standard) along with a 95% confidence interval. The model-building approach for our secondary effectiveness outcome will follow four analysis steps: (1) an unadjusted before/after of the effect of the virtual facilitation approach (ignoring period/time effect); (2) the inclusion of time period (i.e., steps) to examine if any potential intervention effect relates only to the intervention or also to an independent effect of calendar time; (3) an adjustment for patient’s age, race, and gender and potential pharmacy-level confounders, such as pharmacy size; (4) the possible interaction between time period and intervention. The impact of virtual facilitation on effectiveness could potentially change over time if vaccine acceptance rates increase with time and as pharmacists gain experience with counseling vaccine-hesitant patients. We aim to explore this question through the inclusion of an interaction between period/time and intervention effect in Model 4. Additionally, the intraclass correlation coefficients (ICC) will be estimated and reported, so this information will be available for power analyses for future investigations using similar designs and outcomes. In order to reduce potential bias, during the virtual facilitation period, pharmacists will be instructed to approach patients that they did not approach during the standard facilitation period, which is feasible given the large number of unvaccinated individuals in the communities in which RURAL-CP pharmacies are located.

For the primary outcome of fidelity, captured at the pharmacy level, we wish to compare the fidelity scores under standard implementation versus fidelity scores under virtual facilitation. We expect each pharmacy to have approximately 4 fidelity assessments per time period (i.e., a period of 8 weeks). A sample of 30 pharmacies in an incomplete stepped-wedge cluster-randomized design with six periods (five steps), and an average of 10 fidelity assessments per pharmacy yields a total sample size of 320 assessments, which achieves over 90% power to detect a difference between means of 0.53 with a standard deviation of 1 (i.e., moderate effect size). The test statistic is based on a two-sided Wald Z-test with ICC = 0.6 and alpha = 0.05. Given that we will have repeated fidelity measures from the same pharmacists over time, we have specified a conservative ICC (which in a stepped wedge design does not impact power calculations significantly).

For the secondary outcome of effectiveness, we expect each pharmacy to identify approximately 10 vaccine-hesitant patients during each time period (i.e., a period of 8 weeks). A sample of 30 pharmacies in an incomplete stepped-wedge cluster-randomized design with six periods (five steps), and an average of 27 patients per pharmacy yields a total sample size of 800 patients, which achieves over 90% power to detect a difference between effectiveness proportions of 10%. The proportion of vaccine-hesitant patients accepting the vaccine during the virtual facilitation approach is assumed to be 15% compared to 5% under the standard implementation period. These estimates are based on a review of reported changes in vaccine acceptance for evidence-based vaccine hesitancy interventions [77]. The test statistic is based on a two-sided Wald Z-test with ICC = 0.05 and alpha = 0.05. We note that the actual sample size should exceed 800 patients since we expect pharmacies will continue to identify vaccine-hesitant individuals during the follow-up periods.

We will select one high and one low-performing pharmacy in each block and conduct interviews after their virtual facilitation period has ended. Semi-structured interviews will be conducted [78] with at least 4 pharmacy staff members at each pharmacy, including pharmacy owners/managers, pharmacists, and technicians, to triangulate with and expand upon the quantitative implementation outcomes. Interview guides will be built around “grand tour” questions for each implementation outcome (Table 2), e.g., “How would you describe the implementation of vaccine hesitancy counseling?,” “What has gone well or not well?,” and “How feasible was it to receive facilitation as delivered?” Additionally, we will include questions to explore: (1) any observed associations between pharmacy size, patient demographics (race, age, gender), and organizational structure and context measures with fidelity and effectiveness; (2) sustainability potential; and (3) costs at the pharmacy level. Themes on sustainability and costs will be summarized and reported to the Payer Advisory Committee (PAC) during the sustainability exploration in Aim 2.

We will use rigorous procedures [79] for the analysis of qualitative interview data. Interviews will be transcribed by a professional transcriptionist, who will remove identifying information, and be imported into MAXQDA, a qualitative data analysis program. The trial team will review several transcripts and meet to discuss the overarching themes related to the CFIR framework (e.g., “not enough time” coded as an “inner context barrier”). These themes will then be incorporated into a codebook with definitions and example quotes to enable structured coding [80, 81]. Using rigorous analysis techniques [82], two researchers will use the codebook to independently code each interview and meet to resolve discrepancies. Inter-coder reliability will be calculated. Additionally, we will add specific attributes (e.g., number of scripts filled per day, pharmacy level of rurality) to each transcript, allowing us to examine whether fidelity and effectiveness vary by attribute. Because these interview guides and the quantitative data are both mapped to CFIR constructs, we will be able to conduct concurrent triangulation (comparing results from both data sources on the same questions) as well as elaboration analyses (using the qualitative data to provide depth of understanding to the quantitative findings) [83, 84].

Cost-effectiveness analysis and budget impact analysis provide estimates of value and affordability, respectively. When used to examine the relative value and cost of implementation strategy bundles, these approaches inform decision-makers about the optimal approach to implementation and also whether that approach is financially feasible in a given setting. The calculation of the incremental cost-effectiveness ratio (ICER) comparing the relative value of implementation approaches for the vaccine hesitancy counseling intervention will be accomplished in three steps (Fig. 3).

For step 1, see the fidelity and effectiveness analysis described in the “Aim 1 data analysis” section. For step 2, implementation strategy costs for each site will be calculated using the data sources listed in Fig. 3. We have tested and found REDCap facilitator activity logs feasible in a prior study [61]. In step 3, the numerator will be the incremental difference in total implementation costs incurred at sites while receiving virtual facilitation compared to the standard implementation approach. The denominator will include the difference in the changes in fidelity and effectiveness while receiving virtual facilitation compared to the period receiving the standard implementation approach. Typical standard error estimation methods are not appropriate for cost-effectiveness ratios because the possibility of having a zero or near zero denominator is not negligible and cost and effectiveness estimates are rarely independent. Therefore, we will use a nonparametric bootstrap with a replacement method with 1000 replications to generate an empirical joint distribution of incremental implementation costs and fidelity and effectiveness change scores and acceptability curves representing the probability of falling below a range of cost-effectiveness thresholds identified by the PAC for an 8-point increase in fidelity and 10% increase in effectiveness. We will also conduct a budget impact analysis to provide an estimate of the cost for rural community pharmacies to implement the intervention. The budget impact analysis estimates the cost to implement and provide the intervention during the (a) standard implementation period and (b) virtual facilitation period for each site. Intervention costs will be estimated by multiplying the average time to complete the intervention by the number of times hesitancy counseling was provided. This time will be converted to costs using location-based salary and fringe estimates for the pharmacist. These estimates will be combined with implementation costs to estimate the budget impact of the intervention from the perspective of a pharmacy under each implementation condition

In order to scale up new, effective interventions in the healthcare system, applied research efforts must engage key stakeholders, including payers, providers, and patients. The PAC will review data and prepare a business case and sustainability plan for implementing the ASORT intervention using the standard implementation and virtual facilitation approaches. Current ‘pain points’ and gaps that lead to reimbursement models that support the scalability and sustainability of the intervention and implementation approaches will be identified. At present, while pharmacists can receive a low administration fee for delivering vaccines, there is not yet a model of reimbursement for counseling. One such reimbursement pathway to explore is the North Carolina Department of Health and Human Services Collaborative Care (CoCM) codes [85], which are behavioral health codes for providers to share with other care providers. The PAC will comprise representatives from several health insurance payers, including Blue Cross and Blue Shield, and a provider organization to facilitate ‘team-based’ reimbursement strategies. The PAC will meet to reflect on the cost analysis results and develop feasible plans for reimbursement and sustainability of the implementation approaches.