Validating real-time three-dimensional echocardiography against cardiac magnetic resonance, for the determination of ventricular mass, volume and ejection fraction: a meta-analysis

Cardiovascular diseases are the leading cause of death worldwide and as such, sensitive diagnostic methods are vital in early diagnosis and, therefore, prevention [1]. The evaluation of ventricular mass, volume and ejection fraction are important parameters in diagnosis [2]. For the last few decades, two-dimensional echocardiography (2DE) has been the routinely used method, with the ability to provide information on each of these parameters [3]. However, two-dimensional echocardiography is limited with regards to the need for geometrical assumptions, foreshortened views and suboptimal endocardial border detection [4]. Two-dimensional echocardiography is operator-dependent, relying on the visual interpretation of moving images, and prone to inter-observer and intra-observer variability and poor test–retest reliability. Moreover, to calculate a volume from 2DE, geometric modelling of chamber shape must be performed and consequently, LVEF estimation from 2DE is subject to bias and error in the presence of pathology. This produces less accurate and less consistent geometric modelling [5].

It has been suggested that 3D echocardiography does not show variability in geometric modelling and has higher inter-observer and intra-observer reliability [6]. RT3DE shows great promise in being included in routine CVD diagnosis in the future, not only because it provides more reliable volume quantification, but also has the ability to crop and visualise specific structures in greater detail. Currently, CMR is considered as the gold standard for three-dimensional imaging that produces the most reliable and accurate imaging of the heart. However, it is both costly and time-consuming in acquisition and processing of images [7].

Previous studies have shown high concordance between RT3DE and CMR in the assessment of ventricular mass volume and ejection fraction [8‐12]. This can be attributed to the ability to simultaneously view an image in multiple planes in RT3DE. Chosen long-axis planes can then be analysed using either a biplane or multiplane method for ventricular volumetric analysis and determination of mass. A simpler approach is the biplane method which takes 2-chamber and 4-chamber long-axis views of the image, traces the epicardial and endocardial surfaces of these two planes and uses these to calculate mass and volume. The more complex multiplane approach traces the surfaces of the epicardial and endocardial surfaces of the heart in multiple long-axis planes. This is followed by a correction of the tracings in short-axis views [13].

However, despite the significant advances made in three-dimensional echocardiography, there are still some areas which current studies aim to address, before it is integrated into routine clinical use. The spatial and temporal resolution still do not match that of 2DE, the analysis can be time consuming and the breath-hold time to acquire these images is very long as multiple beats are needed. These beats are stitched together to form a full image which is manually done, potentially causing artefacts in the image. A stitch artefact appears as a fault line in an image, compromising interpretation [14]. Advances are now being made allowing full-volume acquisition to be conducted using only a single beat, instead of multiple to avoid stitch artefacts and shorten breath-hold time [15].

Additionally, to analyse 3DE using either the fully automated or semi-automated algorithms, very high image quality is required. This requires highly trained professionals for image acquisition. Image quality is suggested to be affected not only by the expertise of the professional, but also patient factors. Therefore, current studies aim to programme more sensitive artificial intelligence to analyse images of different quality. They further aim to fully automate the analysis process to speed up the process of RT3DE, to make it a more practical too to be utilised in a clinical setting [7].

This study aims to fill in the research gap by synthesising data from numerous studies which compare RT3DE to the current gold standard, CMR, so a more reliable conclusion can be made surrounding the efficacy of RT3DE, a cheaper and faster tool for CVD diagnosis [7].

A protocol containing the method and study design is published in Prospero (registration number: CRD42021262783). Studies published from 2000 to present are filtered due to its recent development in RT3DE method, and a broad search is conducted on the databases PubMed, Embase and Scopus with terms (3D echocardiography OR RT3DE OR real-time 3D echocardiography) AND (cardiac magnetic resonance OR CMR). Total articles are noted and then abstract and title screening is conducted to ensure studies fit within inclusion and exclusion criteria based on PICO approach. P: Live human adults (18+ years) of all ethnicities and both genders are the target population of the study, excluding children. I: The studies must mention terms RT3DE. Imaging conducted at rest is taken, as results during stress echocardiography can be significantly different. Post-mortem analyses and computer simulation which can produce significantly different results are excluded. C: CMR to be included. O: outcomes including at least one of the following primary outcomes: LVESV, LVEDV, LVEF, LVM, RVESV, RVEDV and RVEF.

All records are collected onto an Endnote library and then full-text articles for each included study are found. After screening the full-text articles, studies are eliminated if data are not present on the primary outcomes of the meta-analysis and those with duplicated or overlapping data. The Endnote library is then compressed and sent to a peer, along with the search terms, search databases and eligibility criteria, to mitigate reviewer bias. The final screening is conducted by a third reviewer and a final decision is made on the articles to be used for data extraction.

Data extraction is conducted on a Microsoft Excel document with each primary outcome on a different sheet. Data are extracted for primary outcomes (LVESV, LVEDV, LVEF, LVM, RVESV, RVEDV and RVEF) and subgroups. Subgroups of continuous variables from studies are converted to categorical variables to prepare for subgroup analysis. The categories used in this study include age (1 = 50 and less, 2 = more than 50 years old), disease condition (1 = disease, 2 = disease and healthy, 3 = healthy), quality of study (1 = high and 2 = medium), publication year (1 = 2010 and earlier, 2 = after 2010) and RT3DE analysis planes (1 = biplane, 2 = multiplane).

Study characteristics including study location, gender ratio, average age, study design, imaging method and brand, analysis method, statistics, disease conditions and outcome measures are all collected. The data collected for analysis include RT3DE and CMR mean values, standard deviations and sample size for calculation of effect size and mean difference. Additionally, regression values and Bland–Altman test values are also collected. Where mean and standard deviation are not reported, median values are taken to equal the mean and standard deviation is taken to be range/4. All data are checked by two independent reviewers to reduce human error and bias.

The quality assessment check is conducted using the The Grading of Recommendations Assessment, Development and Evaluation (GRADE) tool where only high- or medium-quality studies are used [16]. Two authors independently reviewed studies using this approach to ensure study quality is reliably noted.

A random model using effect size measures including standardised mean difference and effect size (using Cohen’s d method) between RT3DE and CMR will be used for all continuous variables including primary outcomes in ventricular volumes, mass, ejection fraction. Mean difference is used to indicate whether there is any significant difference between RT3DE and CMR. Effect size is used to indicate the size of the difference between CMR and RT3DE. Additionally, concordance results and Bland–Altman analysis results are pooled using mean and recorded. Many studies report standard deviation instead of upper and low limits of agreement (LOA), which is required for this study. In this case, the standard deviation is converted to LOAs using the formulas [17]:

Heterogeneity analysis using I² will be used to identify the level of variability across studies. An I² value of more than 50% indicates a high level of heterogeneity. Subgroup analysis was conducted to identify sources of heterogeneity if I² is more than 50%.

All data and selected studies were checked by two researchers to ensure no errors in data collection were made which can lead to erroneous conclusions. Inclusion and exclusion criteria are predetermined and applied uniformly to all studies to ensure objective selection of studies. Statistical analyses were conducted to consider the possibility of publication bias. As this is a meta-analysis, any human ethical considerations were not required for this study.

The search process can be seen on Fig. 1 and included studies and study characteristics on Table 1. A total of 2073 potential studies from the databases Pubmed, Embase and Scopus were identified. After the removal of 28 duplicates, 40 articles remained. Full text screening was conducted on the remaining articles and 12 were excluded due to irrelevance or data insufficiency. Overall, 28 articles are used for quantitative synthesis in this meta-analysis.

The majority of mean effect estimates of individual studies for LVESV, LVEDV, LVEF, RVESV, RVEDV and RVEF, as reflected by the overall effect estimate, lie to the left of the central line, favouring RT3DE over CMR. Although for LVM, the majority of mean effect estimates, as reflected by the overall effect estimate, lie to the right of the central line, favouring CMR over RT3DE.

The 28 studies included were published between the years 2004 and 2017 and were all case–control studies. The total 1215 (800 males, 415 females) patients received RT3DE followed by CMR across all studies and all participants received both RT3DE (case) and CMR (control) imaging. Exceptions include Caiani et al. [21] where two patients received only CMR as they had dilated cardiomyopathy preventing their heart from being able to fit into the pyramidal scan volume for RT3DE and Zhang et al. [35] where two patients did not have adequate image quality for RT3DE analysis. Participants were recruited from 17 different countries including Argentina [18], Sweden [19], Brazil [20], USA [21‐27], South Korea [28], Netherlands [13, 29, 30], UK [31, 32], Australia [33‐35], Germany [26, 27, 36], Switzerland [37], China [35, 38], France [39, 40], Japan [41, 42], Taiwan [43], Italy [44], Austria [26, 27] and Hong Kong [35].

All studies include had at least one of the required outcome variables including LVESV, LVEDV, LVEF, LVM, RVESV, RVEDV and RVEF. It should be mentioned here that insufficient studies were available on the right ventricular mass (RVM) outcome variable and, therefore, this variable was not included in the meta-analysis. Information on the RT3DE and CMR imaging technologies, analysis method (automatic or semi-automatic), beat number used for RT3DE imaging, disease conditions of patients, age group, male to female ratio and RT3DE analysis plane (biplane or multiplane) were collected as these varied among the 28 studies, summarised in Table 1. In addition key results from the GRADE quality assessment as well as statistical analysis tests used in each study are also summarised in Table 1 (Fig. 2).

The studies used in this meta-analysis were either moderate or high quality based on the GRADE assessment conducted. Only moderate or high-quality studies were used in an attempt to prevent bias. The results from the Egger regression test for publication bias displayed in STable 5 indicated no significant bias for LVESV, LVEDV, LVEF, LVM, RVEDV and RVEF (p > 0.05). However, the Egger regression test indicates statistically significant bias for RVESV (p < 0.05). These results can be observed in the funnel plots in Fig. 3.

According to Table 2, the pooled mean differences for were − 5.064 (95% CI − 10.132, 0.004, p > 0.05), 4.654 (95% CI − 4.947, 14.255, p > 0.05), − 0.783 (95% CI − 5.630, 4.065, p > 0.05, − 0.200 (95% CI − 1.215, 0.815, p > 0.05) for LVEF, LVM, RVESV and RVEF, respectively. This indicates no significant difference between RT3DE and CMR for these variables, meaning that results of RT3DE are similar to CMR.

Subgroup analyses were conducted for variables with significant heterogeneity, including LVESV, LVEDV, LVEF and RVESV. The subgroup analyses which were conducted indicate some differences which may have contributed to differences in results for each variable as well as significant heterogeneity. For LVESV, there is a significant difference between RT3DE and CMR for those aged over 50 years [MD = − 13.896 (95% CI − 20.480, − 7.311, p < 0.001)], but no significant differences for those under 50 [− 21.627 (− 48.932, 5.678)]. Furthermore, studies including only participants with cardiovascular disease indicated a significant mean difference [MD = − 19.286 (95% CI − 33.345, − 5.227), p < 0.01] as well as those including both healthy and diseases participants [MD = − 13.657 (95% CI − 23.492, − 3.823), p < 0.01]. Additionally, there is a significant difference between RT3DE and CMR for high-quality studies [MD = − 14.784 (95% CI − 21.377, − 8.192), p < 0.001], compared to moderate-quality studies indicating no significant difference. Further, studies published in 2010 and earlier also indicated a significant difference between the two diagnostic methods [MD = − 12.301 (95% CI − 20.995, − 3.608), p < 0.01] compared to those published after 2010 [MD = − 21.590 (95% CI − 38.495, − 4.685), p < 0.05]. Finally, studies using the biplane method indicated a significant difference between RT3DE and CMR [MD = − 14.335 (95% CI − 21.132, − 7.537, p < 0.001)] compared to the multiplane method indicating no significant difference [MD = − 7.918 (− 21.416, 5.580), p > 0.05)].

For the LVEDV variables, there is a significant difference between RT3DE and CMR for those aged over 50 years [MD = 13.896 (95% CI − 20.480, − 7.311, p < 0.001)], but no significant differences for those under 50 [MD = − 52.758 (− 110.466, 4.950), p > 0.05]. Furthermore, studies including only participants with cardiovascular disease indicated a significant mean difference [MD = − 19.286 (95% CI − 33.345, − 5.227), p < 0.01] as well as those including both healthy and diseases participants [MD = − 13.657 (95% CI − 23.492, − 3.823), p < 0.01]. Additionally, there is a significant difference between RT3DE and CMR for high-quality studies [MD = − 14.784 (95% CI − 21.377, − 8.192), p < 0.001], compared to moderate-quality studies indicating no significant difference. Further, studies published in 2010 and earlier also indicated a significant difference between the two diagnostic methods [MD = − 12.301 (95% CI − 20.995, − 3.608), p < 0.01] as well as those published after 2010 [MD = − 37.027 (95% CI − 66.971, − 7.082), p < 0.05]. However the mean difference in studies published before 2010 is more significant. Finally, studies using the biplane method indicated a significant difference between RT3DE and CMR [MD = − 14.335 (95% CI − 21.132, − 7.537, p < 0.001)] compared to the multiplane method indicating no significant difference [MD = − 12.958 (− 27.566, 1.649), p > 0.05].

For the LVEF variable, for those aged over 50 years, there is a significant mean difference between RT3DE and CMR of − 7.403 (95% CI − 13.852, − 0.954, p < 0.05) where RT3DE reports lower left ventricular end-systolic volume compared to those aged 50 years old and under indicating no significant difference [MD = 0.981 (− 7.051, 9.013), p > 0.05]. Furthermore, there is no significant mean difference for neither the disease [MD = − 3.002 (− 6.769, 0.764), p > 0.05], nor the disease and healthy subgroup [MD = − 8.522 (− 21.784, 4.739), p > 0.05]. There is significantly high heterogeneity for mean difference in both studies with diseased participants (I² = 91.963, p < 0.001) and studies with both diseased and healthy participants (I² = 77.387, p < 0.001). Additionally, for studies of high quality, RT3DE reports significantly lower LVEDV compared to CMR [MD = − 4.180 (95% CI − 6.882, − 1.478), p < 0.01], compared to moderate-quality studies showing no significant difference [− 7.193 (− 26.802, 12.417), p > 0.05]. Further, the pooled mean difference for studies published in 2010 and earlier [MD = − 4.159 (95% CI − 6.807, − 1.511), p < 0.01], compared to studies published after 2010 indicating no significant difference [MD = − 7.107 (− 21.622, 7.408), p > 0.05]. Finally, there is no significant mean difference for neither the biplane [MD = − 9.998 (− 21.211, 1.214), p > 0.05] nor the multiplane [MD = 0.944 (− 2.008, 3.896), p > 0.05) methods.

For the RVESV variable, there is no significant difference between RT3DE and CMR for mean difference for any subgroups.

Pooled correlation and Bland–Altman analysis results:

According to STable 6, there is high correlation between RT3DE and CMR for all variables, and each individual correlation coefficient which was pooled was statistically significant. The statistical analysis method for concordance differed among studies, as specified in Table 1. All variables except for left ventricular end-systolic volume and left ventricular end-diastolic volume have low bias. Although, the limits of agreement for all variables are very wide.

This is the first meta-analysis study which has validated RT3DE against gold standard method, CMR approach including a large number of recently published case–control clinical studies. The meta-analysis has comprehensively assessed the value of RT3DE in clinical application. However, there are a number of limitations in the study. First, the studies used in the meta-analysis themselves had low sample size due to the nature of clinical study. This may explain the significant heterogeneity observed in both variable and subgroup analysis. It is difficult to perform studies in diagnostic methods with large sample sizes and this may have contributed to the lower power and larger margin of error.

Second, there was low agreement between RT3DE and CMR in LV volume assessment, no final judgment can be made about the comparison between RT3DE and CMR in LV volume measurement. A further study encompassing a comparison between RT3DE, CMR and 2D ECHO is needed to confirm the results in our study. Additionally, with improvements to the methodology to increase the agreement between RT3DE and CMR, future studies and meta-analyses are then required to assess similarity.

Furthermore, many studies used a combination of healthy and diseased participants, but did not separate these results. This could have created further variation in results. There were further variations such as difference in equipment, analysis method and participant factors such as ethnicity which subgroup analyses could not be conducted on due to low study number and some studies not reporting on these parameters. Additionally, the number of studies included in the subgroup analyses were also small and therefore may have lacked power to stratify for any methodological differences between the selected studies. Therefore, in future, we aim to reconduct a meta-analysis once more studies have been published in the field, to produce a meta-analyses with a higher power. In addition, we hope to potentially find more studies which include a larger sample size.

This meta-analysis included a very detailed analysis in terms of difference between RT3DE and CMR. Further steps have been taken in subgroup analyses which previous studies have not conducted. Through the collation of data from a range of different countries and RT3DE methods, the generalisability of study findings are high.

Overall, this meta-analysis indicated promising results for the use of RT3DE, with no significant difference to CMR (for LVEF, RVESV and RVEF). Although in some cases, RT3DE appears to underestimate volumes when compared to CMR (LVESV, LVEDV and RVEDV).

Currently, the most commonly used analysis method is semi-automatic where the borders are manually traced by the observer. Although, improvements are still being made to fully automate the process, to reduce processing time. In addition, currently analysis is conducted via a biplane a multiplane method. This meta-analysis indicates that there is no difference between RT3DE and CMR for the multiplane method, but a significant difference for the biplane method for LVESV and LVEDV. This indicates that the multiplane method is potentially a superior method. This is supported by previous research suggesting the benefit of the multiplane method in more in-depth analysis of heart structures, the results surrounding this is inconclusive and further research and development is required here.

Further advancements are required to compensate for biological changes including age, sex and BMI. This is of particular importance considering that the demographics most in need of these imaging methods are patients over the age of 50 with heart pathologies. This meta-analysis indicates lower concordance between RT3DE and CMR for older individuals. There is also greater underestimation by RT3DE compared to CMR in individuals with diseased heart, as indicated in this study. Previous research has suggested that RT3DE had provide more detail into heart structures when compared with the routinely used 2DE method. However, the above developments are being made to improve temporal and spatial resolution, which is lower in RT3DE compared to 2DE.

With technological advancement, RT3DE can be integrated into routine clinical practice. Further development should improve efficiency, workflow, image quality, speed, accuracy and simplicity of the RT3DE method. This will make RT3DE more accessible, and likely to be chosen over the current, more expensive and time-consuming method of CMR.