Implementation fidelity of the Systems for Person-Centered Elder Care (SPEC): a process evaluation study

The quality of care for older residents in nursing homes (NHs) is closely related to their quality of life because the care is provided where they live [1‐3]. If older adults residing in NHs do not receive quality services, they may experience poor health and functionality, more emergency room visits, unnecessary acute hospital admission, and premature death; these outcomes add to the burdens of older adults, families, and society as a whole [4, 5]. While innovative interventions to improve nursing home quality are valuable and much needed, such interventions for vulnerable older residents are often complex and difficult to implement [6].

Moreover, evaluating the effectiveness of innovative public health interventions is also challenging. Problems targeted by such interventions are associated with multiple causes targeting multiple levels, so the public health interventions include multiple components that can interact with one another, affecting both the respective and overall outcomes [7, 8]. Therefore, when evaluating a complex intervention study, it is important to understand the complex nature of interventions and to combine the outcome evaluation and process evaluation [7]. Effectively combining them is accomplished by focusing on the background elements that impact how an intervention’s goals are achieved, as these elements may also be applied when implementing the same intervention in other settings [7, 9]. Careful evaluation of contextual factors related to the intervention process helps identify the important functions of an intervention, even though complex interventions may seem uncontrollable due to various related phenomena [10‐12].

Carroll et al. [13] proposed a conceptual framework for implementation fidelity (CFIF), which focuses on various aspects of adherence and moderating factors influencing intervention adherence. Several key components of CFIF are unique and valuable for assessing implementation fidelity: facilitation strategies such as intervention manuals, guidelines, training, and feedback are used to optimize and standardize how an intervention is conducted [13]. The quality of delivery is an assessment of whether the intervention delivery process is appropriate for achieving the original intentions of the intervention [13]. Quality of delivery is a significant potential moderating factor for the relationship between an intervention and its implementation fidelity [13]. The responsiveness of participants relates both to the persons who receive the intervention and to the persons who deliver the intervention [13]. The intervention complexity is also an important component of implementation fidelity, and it contributes to both the real nature of the intervention and the description of the intervention [13]. The more complex the intervention, the harder it is to obtain a higher level of implementation fidelity; however, if an intervention is described in great detail, it may enhance implementation fidelity [13, 14]. Additionally, CFIF assumes that an intervention delivered by a person with enthusiasm will have higher implementation fidelity [13]. Hasson [10] presented a modified CFIF by adding context and recruitment to Carroll et al.’s model, described above [13, 15]. Context refers to encompassing social systems that include structures and cultures of organizations and historical and concurrent events [15]. When implementing a complex intervention, organizational culture and the role of upper management are very important [16]. Both the older residents and the staff in NHs can experience optimal dignity, well-being, and health through a safe and caring organizational culture [17]. Participant recruitment includes reasons for nonparticipation, the consistency of recruitment procedures, and judgments by participants regarding the outcomes and relevance of interventions [18]. Hasson’s modified CFIF suggested using a systematic evaluation of all moderating factors, as the interactions between moderating factors are known to affect the implementation of an intervention [15, 18]. Hasson’s modified CFIF has been empirically examined as a useful conceptual framework to systematically evaluate implementation fidelity and possible moderating factors affecting implementation fidelity [15, 19, 20]. It is valuable to measure and analyze implementation fidelity by providing guidance in organizing the data collection of adherence and associated moderators in complex interventions [15].

The Systems for Person-Centered Elder Care (SPEC) is a technology-enhanced, multidisciplinary, integrated care model for older adults with frailty [9]. The intervention study hypothesized that the implementation of the SPEC model will improve the quality of care in NHs with limited healthcare provision, and the improvement will promote the health/functional outcomes and quality of life of old NH residents with multimorbidity [9]. This complex intervention was implemented in ten nursing homes that had been certified and reimbursed by public long-term care insurance (LTCI) in South Korea over 21 months between 2015 and 2016 (Fig. 1). As a complex intervention model based on Wagner’s Chronic Care Model, the SPEC [9] consists of five components: comprehensive geriatric assessment (CGA) based on the Korean version of the interRAI long-term care facility [21], individualized need-based care planning (CP), interdisciplinary case conferences (ICCs), care coordination (CC), and information and communications technology (ICT) tools (Fig. 2).

Each participating NH had an on-site SPEC coordinator team, typically composed of a nurse and a social worker, who performed CGA, developed a care plan from the assessment of each participating resident, ran optional ICCs, and communicated with the SPEC consultant. The SPEC consultant trained and empowered the NH staff, particularly the on-site SPEC coordinator team, to improve the care quality of the participating NHs. The SPEC consultant facilitated and evaluated the entire implementation process of the SPEC intervention for each participating NH; offered training sessions, care coordination, and on- and off-line consultation; and monitored the progress of the intervention through a cloud-based ICT system on a regular basis. The effectiveness of the intervention was evaluated through stepped-wedge cluster-randomized trials, and the intervention (the SPEC program) significantly improved quality of care as measured by a composite quality indicator (QI; mean difference −0.025 [CI −0.037 to −0.014, p < .0001]) [22].

Kim et al.’s [22] study focused on evaluating only the effectiveness of the SPEC intervention; the process evaluation described in this article was conducted in parallel with the effectiveness study. We designed this study to evaluate implementation fidelity and to identify moderating factors that could influence the implementation fidelity of complex interventions. The conceptual framework of the current study (Fig. 3) was developed by combining Hasson’s [15] modified CFIF with Grant et al.’s [23] framework for designing process evaluations for cluster-randomized trials of complex interventions [15, 21]. The latter includes domains (e.g., recruitment of clusters, recruitment, and reach of individuals) that can help to investigate the external validity of an effectiveness study for the SPEC intervention.

This study hypothesized that fidelity is influenced by six moderating factors in the combined/modified CFIF: participant responsiveness to SPEC interventions, the complexity of interventions, facilitation strategies, the quality of delivery, context, and participant recruitment (Fig. 3). We measured the adherence of implementation fidelity by rating the degree of executed and delivered intervention components of the SPEC model based on four subdomains (contents, frequency, duration, and coverage) of the modified CFIF. We also analyzed the moderating factors by using a deductive approach based on the six moderating factors of the combined CFIF. This study aims to evaluate the level of implementation fidelity and explore the mechanism and moderating factors to optimize implementation fidelity for SPEC interventions for frail older adults in Korean nursing homes.

This process evaluation study aimed to investigate how the SPEC intervention, an ICT-enhanced, multidisciplinary, integrated care management model [9], was effective (positive outcomes) through examining the intervention’s implementation process. We found that the five components of the SPEC model were designed well for the purpose (quality improvement) according to the interviews with key participants (Table 4). We also examined how well the components were implemented as planned (Table 2). The strong adherence of participating homes—the implementation fidelity—was attributed to the moderating factors hypothesized in our conceptual model for this study. The importance of approaching implementation issues from a theoretical perspective and assessing fidelity within the process evaluation of a health program has been emphasized [19]. We were able to systematically evaluate adherence and moderating factors of implementation fidelity alongside the modified/combined CFIF. It was valuable to assess and analyze the roles of the six domains of the modified/combined CFIF. This is in line with previous studies emphasizing that the six moderating factors in the framework influenced fidelity in a complex and interrelated manner [18‐20]. Tailored facilitating strategies, assurance of the quality of delivery, and recruitment strategies were facilitators. While intervention complexity was a barrier to adherence, the context and participant response both positively and negatively influenced implementation fidelity.

The main results of the SPEC study and the interviews with interdisciplinary team members (Table 4) supported the effectiveness of the five key components of the SPEC model. The health statuses of participating older residents were relatively severe and complicated; thus, those participants needed comprehensive and proactive interventions [27]. Therefore, the SPEC intervention required extra time and effort.

There were several facilitating factors for the successful implementation of the SPEC model. First is tailored delivery of evidence-based comprehensive interventions over process evaluation work guided by the theoretical framework. Pre-specifying and publishing study protocols improves the compliance of randomized trials of complex interventions by comparing intended interventions with implemented interventions [23, 28]. Previous studies suggested that consolidated framework and multi-faceted facilitating strategies are important for successfully implementing complex interventions [29, 30]. The SPEC research team developed theory-based process evaluations and followed the process evaluation protocols as much as possible (Table 2). According to a previous process evaluation study for dementia-specific case conferences in NHs, an optimizing process structure is very important for increasing the effectiveness of randomized controlled trials [31].

The second facilitating factor is the role of the SPEC consultant and facilitating strategies. The SPEC research team attempted to recruit proactive NH heads. We held presentations for key care teams because the expectations of the health care team for changes in the care process or improvement in the quality of care promote the performance of interventions [29, 30]. The SPEC coordinator provided well-prepared training and manuals to NH coordinators. In addition, the SPEC consultant monitored implementation fidelity through the SPEC ICT system regularly and gave immediate feedback to increase implementation fidelity. This finding correlates with previous studies suggesting that some actions of a facilitator, such as catalytic action in multidisciplinary teams and encouraging advice, influenced the effectiveness of the intervention positively [10]. Most on-site SPEC coordinators agreed that the tailored consulting was very helpful. A prior study’s finding that the support of a facilitator was the most attractive aspect of the nurse-led cognitive program regarding falls in older adults with frailty was consistent with the results of the current study [32].

The third facilitating factor is the cloud-based online ICT system, which supports resident care and generates institutional data. The ICT system was a prototype, computerized version of a CGA-based care management tool. Given the low staffing, the ICT system made it easier for the on-site coordinating team to implement the SPEC interventions (Table 3). The ICT system promoted communication between the care teams and the research team because they had access anytime and anywhere to on-site SPEC coordinators, who commented that the ICT system facilitated communication among multidisciplinary teams. The system improved the ease of storing and tracking resident data, generated useful tailored reports on the NHs, and provided resources for care providers/managers. Our findings were consistent with a previous study’s results indicating that e-consultation promotes data-driven improvements [33]. In this study, our research team was able to monitor the implementation fidelity in real time through the ICT system and manage the timeline for planned interventions for each participating NH. These findings resonate with a systematic review paper that e-consultation facilitates timely specialty advice [34].

In addition, several lessons to expand the SPEC model were learned. Firstly, sufficient institutional support is necessary to implement a new technology-enhanced integrated care model such as SPEC. Our results also highlight that high workloads and time pressure were the most widely represented barriers in applying new interventions in NHs [31]. According to the on-site SPEC coordinators, the SPEC model was useful for planning care and for improving the quality of care; however, participating NHs did not want to increase their usage of the SPEC model due to their heavy workloads and insufficient compensation (Tables 3 and 4). Therefore, sufficient workforce staffing and compensation through institutional and public LTCI policies for supporting institutions should be implemented to accommodate the adoption of a new system. Secondly, tailored and continuing education and training for NH staff are necessary for applying an ICT system in NHs. Via the user opinion questionnaire and interviews of key respondents, the participating staff members pointed out that they had difficulty using the SPEC ICT system and that the SPEC ICT system did not reduce the time required for effective care planning (Tables 3 and 4). According to a previous study with NH residents [35], heavy pressure to complete the interRAI-LTCF evaluation and insufficient computer equipment to perform it were hindrance factors [21]. Third, the addition of more specific and practical contents for participating NHs residents’ care is essential to promoting the performance of the interventions. In this study, some of the on-site SPEC coordinators asked for more practical and realistic care planning lists that apply to their residents (Table 4).

The SPEC study is considered to have been effective in achieving some expected results by conducting planned interventions, but it also has several limitations. First, the intervention and follow-up periods were short. The intensive intervention with the support of the research team for 3 months and the follow-up interval after the staff did not have the support of the research team for another 3 months was the minimum requirements for performance evaluation. If the intervention were to take place for a longer period, more NH residents would be able to participate in the optional ICC, and various effects might be observed. Second, although qualitative and quantitative assessments were used to understand the moderating factors for implementation fidelity, there may be missing subjects or areas. Moreover, we did not conduct a process evaluation of the responsiveness of residents and family members. Residents were likely to be unable to participate as they often had poor cognitive function, insufficient understanding of interventions, and difficulty with communication. The family members did not visit often and were not interested in being contacted for investigation. For this reason, this study mainly relied on the opinions of the service providers and did not investigate the resident experience and family/caregiver satisfaction. Third, the results of this study may not be objective because process evaluation was done by the SPEC research team. Fourth, there are limited studies of process evaluation on ICT-based quality improvement interventions that have been applied to clinical trials, making it difficult to compare this study with previous studies.

The process evaluation study of moderating factors that affect the implementation fidelity of SPEC indicates that the effectiveness of the SPEC model may depend on whether or not the evidence-based interventions are strictly implemented. This study helps address the importance of process evaluation in promoting the implementation fidelity of interventions. It also provides evidence for developing theory-based process evaluations for adopting and diffusing a technology-enhanced care model for NH residents.

Therefore, this study offers several suggestions for future research. The development and application of research tools are required to reflect the responsiveness of participating residents with limited functional status and their family members, which will be helpful for comprehensive process evaluation. We also suggest a repeated study of SPEC on a large scale, under different settings, and in other countries. In addition, developing a more user-friendly ICT system can facilitate large-scale implementations of the SPEC program.