Contributions to the literature
-
This study provides an overview of previously published frameworks and models used to study de-implementation of interventions and policies from a wide array of disciplines, including healthcare, public health, and public policy.
-
The frameworks and models identified can be applied to future studies of de-implementation of ineffective, contradicted, mixed, or untested health care practices or public health policy.
-
This study highlights multiple gaps in de-implementation research and suggests actions to advance future work in the field.
Background
Methods
Search strategy
Inclusion and exclusion criteria
Screening procedures
Data extraction
Quality assessment
Results
Yield
Study characteristics
Author, year | Setting | Sample characteristics | Topic/content area | De-implementation intervention | Primary action | Secondary action | Evidence for de-imp. | Cost | Stakeholder | Method | Study design | Measures used | Primary outcomes |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Cuttler et al., 2005 [39] | Clinical (academic, specialty) | Physicians (n = 222) | Pediatric specialty medicine | Physician decision to terminate growth hormone therapy in pediatric patients | Reduce, remove | NA | Mixed | Y | Y | Mixed methods | Cross-sectional | Interview; survey | Physician recommendation for case Scenarios |
Goodwin, 2013 [40] | Healthcare (broad) | Healthcare manager, staff, and clinicians (n = 13) | Reducing low-value care and costs in healthcare | Program budgeting and Marginal analysis | Reduce | NA | NR | Y | Y | Mixed methods | Case study | Interviews; archival data | Satisfaction and compliance with pbma process; cost |
Grimshaw et al., 2020 [41] | Healthcare (broad) | Hospitals (n = 137) | Preoperative tests; imaging for lower back pain | Identify low-value care; identify local priorities; identify barriers and potential interventions; evaluate choosing wisely implementation; spread effective choosing wisely programs | Reduce | NA | Ineffective | Y | Y | Mixed methods | Case study | Hospital administrative data | % reduction in low-value care |
Gupta et al., 2019 [42] | Hospital | Patient records (n = 4,781) | Neutropenic diet for immunocompromised patients | Multi-step implementation strategy bundle targeting clinician and system-level change (e.g., training, EHR updates) | Remove | NA | Ineffective | Y | Y | Mixed methods | Case study | Content analysis of neutropenic diet prescribing; EHR review | Absolute reduction in prescribing |
Harris et al., 2017a [29] | Health service network | Healthcare experts, steering committee members, workshop attendees (n = 28) | Disinvestment of clinically or cost ineffective health services (broadly) | Development of deimplementation framework | Remove | NA | NR | Y | Y | Mixed methods | Case study | Literature review; interview; survey; workshop | Development of deimplementation framework |
Harris et al., 2017b [43] | Health service network | Healthcare experts (n = 15), healthcare staff (n = 65), senior administrators (n = 18) | Disinvestment of clinically or cost ineffective health services (broadly) | Development of deimplementation framework | Remove | NA | NR | Y | Y | Mixed methods | Case study | Literature review; interview; survey; workshop | Describe methods for disinvestment |
Harris et al., 2017c [44] | Health service network | Healthcare experts (n = 15), healthcare staff (n = 65), senior administrators (n = 18) | Disinvestment of clinically or cost ineffective health services (broadly) | Development of deimplementation framework | Remove | NA | NR | Y | Y | Mixed methods | Case study | Literature review; interview; survey; workshop | Development of deimplementation framework |
Harris et al., 2018 [45] | Health service network | NA | Disinvestment of clinically or cost ineffective health services (broadly) | Development of deimplementation framework | Remove | NA | NR | Y | Y | Mixed methods | Case study | Literature review; interview; survey; workshop | Development of deimplementation framework |
McKay et al., 2017 [46] | Non-profit community based organization | Organization staff (n = 5); clients (n = 396) | HIV prevention | Counseling intervention to identify and reduce risk behaviors. De-adoption involved transition of resource to replacement intervention. | Remove | Replace | Ineffective | Y | N | Mixed methods | Archival secondary data analysis, interview | Data abstraction of agency archival and client records, interviews (with program staff) | Intervention deadoption process and consequences |
Padek et al., 2018 [47] | State health departments | Program staff and leaders (n not available in protocol) | Cancer prevention and control programs in public health departments | NR | Reduce | NA | Ineffective | Y | Y | Mixed methods | Study protocol (quantitative crosssectional, qualitative case study, abm simulation) | Survey, case studies (interview), abm | Mis-implementation of cancer prevention and control programs |
Skolarus et al., 2018 [48] | Clinical (VA) | Patients and physicians (n not available in protocol) | Prostate cancer (androgen deprivation therapy) | Organization policy and behavior change (assess preferences and barriers; discrete choice experiment; formulary restriction; strategy targeting patient/provider decision making) | Reduce | Restrict | Ineffective, contradicted | N | Y | Mixed methods | Study protocol (crosssectional) | Interviews, surveys | Acceptability, feasibility, scalability |
Tangpong et al., 2015 [49] | Organization/firm (multiple industries) | Organizations (n = 96) | Business/management | Organizational behavior, layoffs, divestments, geographic exits | Reduce | Remove, restrict | NR | Y | N | Quantitative | Longitudinal | Survey | Likelihood of turnaround success, changes in form operating conditions, internal firm performance, external capital market support |
Voorn et al., 2018 [50] | Hospital | Hospitals (n = 21) | Patient blood management in transfusion medicine (surgery) | Information provision, goal specification, clinician feedback, benchmark with comparison to best practice hospitals (behavior change) | Reduce | NA | Ineffective | Y | Y | Quantitative | Cluster randomized control trial | Survey | Use of low- value care (esa + blood salvage) |
Author, year | Setting | Topic/content area | De-implementation intervention | Primary action | Secondary action | Evidence for de-imp. | Cost | Stakeholder | Method | Study design |
---|---|---|---|---|---|---|---|---|---|---|
Amankwah-Amoah, 2017 [51] | Industries (broad) | Technological lifecycle | Describe the lifecycle of a product, ultimately leading to discontinued use of a technology | Remove | Replace | NR | Y | N | Non-empirical | Commentary |
Bain et al., 2008 [52] | Clinical | Medication discontinuation | Recognize indication for discontinuing medication; identify medication for discontinuation; discontinue; monitor patient | Remove | NA | Ineffective | Y | Y | Non-empirical | Case example; commentary |
Bauer, 2014 [53] | Legislative bodies (broad) | Environmental (air pollution, water protection, wildlife protection); social (child benefits, pension, unemployment) policy change | Public policy dismantling (including strategies) | Remove | NA | NR | Y | N | Non-empirical | Literature summary; commentary |
Davidson et al., 2017 [54] | Clinical | Behavior change | Broad cycle of identifying low-value practice, plan and execute de-implementation, evaluate consequences, plan for new practice implementation | Remove | Replace | NR | Y | Y | Non-empirical | Commentary |
Helfrich et al., 2018 [37] | Clinical | De-implementation of low-value or ineffective clinical practices (broadly) | Unlearning an active process and substitution of an alternative practice | Reduce; replace | NA | Ineffective | N | Y | Non-empirical | Commentary |
Herald et al., 2009 [55] | Systems (broad, primary example military) | Technological lifecycle | Discuss 6 components of obsolescence management: technology road mapping, system costing, system obsolescence life cycle forecasting, technology trade study analysis and product selection, technology/product surveillance and health assessment, technology transition | Replace | NA | NR | Y | N | Non-empirical | Commentary |
Hyun-Ju et al., 2016 [56] | Healthcare system (broad) | Healthcare technology reassessment | Strategies to reassess and manage obsolete medical technologies | Reduce; replace | Remove | Mixed | Y | Y | Review | Systematic review |
Kirkpatrick et al., 1999 [26] | Public policy | Public policy termination | Consider political environment, policy characteristics, and system constraints | Reduce, remove | NA | NR | Y | Y | Non-empirical | Conceptual |
Niven et al., 2015 [57] | Clinical | Low-value clinical practices | Selected and tailored deadoption intervention | Remove | Reduce | Ineffective, contradicted | Y | Y | Review | Scoping review |
Norton et al., 2019 [58] | Clinical | Cancer care delivery | De-implementation strategies (at patient, provider, setting, societal levels) | Reduce, remove replace, restrict | NA | Ineffective, contradicted, mixed, untested | N | N | Non-empirical | Commentary |
Prasad, 2014 [23] | Healthcare system (broad) | Contradicted, untested, novel healthcare practices | Factor prioritization for testing unproven practices | Remove | NA | Untested | Y | N | Non-empirical | Commentary |
Scott et al., 2013 [59] | Clinical | Polypharmacy in older populations | Behavior change (begin using discontinuation guide) assess for deprescribing | Remove | NA | Ineffective, contradicted | Y | Y | Non-empirical | Commentary |
Soril et al., 2020 [60] | Healthcare system (broad) | Healthcare technology | Policy process (selection, decision, execution, reassessment) | Reduce | Remove | Ineffective, contradicted | Y | Y | Non-empirical | Commentary (literature summary, environmental scan, workshop) |
Model characteristics
Author | Model name | De-implementation terms used | Relationship between constructs | Socio-ecological framework (SEF) |
---|---|---|---|---|
Determinants models | ||||
Amankwah-Amoah, 2017 [51] | Framework of technology obsolescence | Abandon; obsolescence; Terminate | Linear | Organization; system |
Cuttler et al., 2005 [39] | Framework for physician decisions to discontinue ongoing medications | Discontinuation; termination | Linear | Intrapersonal; interpersonal |
Gupta et al., 2019 [40] | Cost (culture, oversight, systems change, training) framework | Abandon; de-adoption; disinvestment | No relationship indicated | Intrapersonal; Interpersonal; organization; system |
Taxonomy for evaluation and explication of disinvestment project | De-adopt; de-implement; defund; discontinue; disinvestment; obsolescence | No relationship indicated | Intrapersonal; interpersonal; organization; system | |
Kirkpatrick et al., 1999 [26] | Process model for termination of public goods | Dismantle; retrenchment; termination | Linear | System; policy |
Norton et al., 2019 [58] | Continuum of factors influencing de-implementation process | De-implementation; discontinue | No relationship indicated | Intrapersonal; interpersonal; organization; community; system |
Padek et al., 2018 [47] | Conceptual framework for mis-implementation | De-adoption; deimplementation; discontinuation; termination | Nested | Intrapersonal; interpersonal; organization; community; system; policy |
Skolarus et al., 2018 [48] | Conceptual model for deimplementation of low value prostate cancer care | De-implementation; discontinue | Linear | Intrapersonal; interpersonal; organization; system |
Evaluation frameworks | ||||
Framework for evaluation and explication of disinvestment projects | De-adopt; de-implement; defund; discontinue; disinvestment; obsolescence | Linear | Intrapersonal; organization; system | |
Goodwin, 2013 [40] | Pbma (program budgeting and marginal analysis) evaluation framework | Discontinue; disinvestment | No relationship indicated | Organization |
Prasad, 2014 [23] | Potential considerations in prioritizing the testing of unproven medical practice | Abandonment; deimplementation | No relationship indicated | System |
Process models | ||||
Bain et al., 2008 [52] | Medication use process framework | Discontinuation | Linear; cyclical/feedback | Intrapersonal; interpersonal; system |
Davidson et al., 2017 [54] | Virtuous cycle of deimplementation | Abandon; deimplementation | Cyclical/feedback | Intrapersonal; organization |
Grimshaw et al., 2020 [41] | Choosing wisely deimplementation framework | De-implementation | Linear | Intrapersonal; interpersonal; organization, system |
Helfrich et al., 2018 [37] | Model for de-implementation strategies | De-implementation; discontinue; obsolete | Linear | Intrapersonal |
Herald et al., 2019 [55] | Obsolescence management framework | Obsolescence; reassess | Linear | System |
Hyun-Ju et al., 2016 [56] | Health technology reassessment process in Korea | Discontinue; disinvestment; obsolescence; reassessment | Linear | Organization; system |
McKay et al., 2017 [46] | Implementation framework with EBI de-adoption as a distinct stage | Abandonment; de-adoption; de-implementation; discontinuation; disinvestment | Linear | Interpersonal; organization |
Niven et al., 2015 [57] | Synthesis model for the process of de-adoption | De-adoption; discontinue | Cyclical, feedback | Interpersonal; organization |
Scott et al., 2013 [59] | Tool for identifying and discontinuing potentially inappropriate drugs | Discontinuation | Linear | Interpersonal |
Soril et al., 2020 [60] | Health technology reassessment model | De-adoption; decreased use; disinvestment; obsolescence; reassess | Linear | Interpersonal; organization; system |
Voorn et al., 2018 [50] | Grol 2005 implementation model | Abandonment; de-adoption; decrease use; deimplementation; disinvestment | Linear, feedback | Intrapersonal; interpersonal |
Theoretical frameworks | ||||
Bauer, 2014 [53] | Analytical framework for the explanation of policy dismantling | Abandonment; deregulation; dismantle; retrenchment; termination | Linear | Intrapersonal; system; policy |
Conceptual framework of potential settings and methods to integrate disinvestment into health service systems and processes | De-adopt; de-implement; defund; discontinue; disinvestment; obsolescence | Linear | Organization; system | |
Framework for an organization-wide approach to disinvestment in the local healthcare setting | De-adopt; de-implement; defund; discontinue; disinvestment; obsolescence | Nested; linear | Interpersonal; intrapersonal; organization | |
Tangpong et al., 2015 [49] | Path-dependent pattern of retrenchment and corporate turnaround | Retrenchment | Linear, feedback | Organization |