A comparison of artificial intelligence-based algorithms for the identification of patients with depressed right ventricular function from 2-dimentional echocardiography parameters and clinical features

Introduction

The cardiology community has historically neglected the right ventricle’s (RV) role in the development and progression of heart failure (HF). However, with the recognition of RV function as an important prognostic factor in several diseases such as HF and pulmonary hypertension (PH) along with the need of accurate RV measurements for ventricular assist devices (VADs), assessment of the RV has become a cornerstone in cardiac evaluation (1-6). Unfortunately, the irregular shape of the RV and lack of normative data challenged the cardiologists when trying to integrate the assessment of the RV as part of the standard Echo evaluation and in clinical decision-making (7).

Currently, assessing RV function by 2-dimensional echocardiography (2D-ECHO) represents the first choice for clinicians (7). However, despite years of experience in measuring RV function, cardiologists still find this task difficult and frequently the RV function is misclassified. In one study assessing concordance levels between 2D-ECHO and CMR imaging there was fair agreement between the two modalities with the prevalence-adjusted bias-adjusted Kappa being 0.43 (range, 0.36–0.45), and poor inter-echocardiographer agreement with a Kappa of 0.12 (7). Simply, we cannot extrapolate data and long-term experience gained from conventional left ventricular (LV) evaluation to the RV. This could be attributed to the complex three-dimensional RV shape, poor definition of RV endocardial surface (LV is much thicker than RV), limited direct visualization due to the retrosternal position of RV, as well as uneven complex contraction-relaxation forces along the RV segment (8-11).

RV EF calculation from 2D-ECHO requires several geometric assumptions, and this has proven to be an inaccurate way for EF estimation. Another way is to derive RV systolic function from other measurements such as the RV fractional area-change (FAC_RV) and lateral TA peak systolic excursion (TAPSE), which provide similar information to RVEF (8,12,13). Other quantitative values also include RV strain, peak S wave velocity of the lateral tricuspid annulus by tissue Doppler imaging (RV S’) (9). The abnormality thresholds for S’, TAPSE, RV Strain, and FAC_RV found in the literature are <9.5 cm/s, <17 mm, <−25%, and <48.5%, respectively (10,11,14,15). With TAPSE being the most sensitive (93%) and specific (100%) of all 4.

While 3D-ECHO is supposed to give a better RVEF estimation in comparison to CMR-derived RV EF and negate the geometrical assumptions made with 2D-ECHO RVEF estimation, it has its own limitations such as being technically challenging, not being widely available yet, and the difficulty of tracing RV border (16,17). When the RV is dilated, such as in corrected Tetralogy of Fallot patients, identifying specific landmarks such as the true RV apex and the pulmonary valve becomes very difficult (18).

Machine learning develops a prediction model using data, algorithms and computing power. The use of logistic regression models to predict complex biological relationships may be limited in some contexts, partly because many biological relationships are non-linear. In the realm of ML and imaging, the use of artificial intelligence approaches (19,20), has enabled automated detection of LV and left atrial endocardial boundaries throughout the cardiac cycle from 3DE data sets, allowing accurate measurements of LV and left atrial volumes and EF (21-25).

The aim of our study is to develop models that can predict low RV function by integrating several 2D-ECHO derived parameters in reference to cardiac magnetic resonance, and then compare these models to single-parameter assessment of RV function. We present the following article in accordance with the TRIPOD checklist. Available at http://dx.doi.org/10.21037/cdt-20-471.

Methods

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Institutional Review Board of the American University of Beirut (Registration Number: IRB00003225). Patient consent was waived due to the retrospective nature of the research. Patient characteristics were extracted by one investigator (AA) while ECHO parameters were extracted and validated by 2 different investigators (ZI and LM). CMR parameters were extracted by an experienced operator (HI), blinded to the clinical and ECHO parameters.

Selection of subjects

We retrospectively identified consecutive patients older than 18 years of age who underwent a 2D-ECHO and a CMR within 3 months of each other at our institution (American University of Beirut Medical Center, Beirut, Lebanon) from 2012 to 2013. Indications for the standard Two-Dimensional Echocardiography (2D-ECHO) were various and no exclusion criteria were applied. From the ECHO, LV EF, TAPSE, S’ and FAC_RV were extracted from four standard views [Apical Four Chamber (A4C), Parasternal Long axis (PLAX), Parasternal Short axis: Aortic Valve level (PSAX), and RV inflow view]. These views are done as per the standard 2D-ECHO protocol in our institution. We then used the CMR determined RV function as our reference standard. In our population, the CMR was mostly ordered to assess the LV (function, scar burden and/or viability) (25%), to rule out myocarditis (13%), or to rule out Arrhythmogenic Right Ventricular Disease (ARVD) (13%). Other indications included cardiac sarcoidosis evaluation, ruling out hypertrophic cardiomyopathy and others. As per recent data from the UK Biobank population cohort, we classified CMR RVEF values below 45% as abnormal (26). Furthermore, patient related characterizes (age, sex, cardiovascular risk factors) at the time of the studies were also extracted from their records.

Statistical analyses

All statistical analyses and data curation were performed using the R programming language. The data was cleaned by excluding patients with missing values (especially outcome values). Cohort-specific categorical and continuous data as well as corresponding inferential and descriptive statistics are shown in Table 1. Continuous variables distributed normally were expressed as the mean ± standard deviation, categorical variables were expressed as frequency (percentage). Enrolled patients were divided into two groups—those with RV Dysfunction (RVEF <45%) and those without RV dysfunction (RVEF >45%). For between-group comparisons, unpaired t-test was used for normally distributed continuous variables, Mann-Whitney U test for non-normally distributed variables, and χ² test (and Fisher’s exact test) for categorical variables. For all tests, a P value <0.05 was considered statistically significant. Accuracy refers to [true pos/pos + true neg/neg]/2.

Table 1 Data used for predictive modelling
Full table

Machine learning pipeline

After splitting the data into training and validation sets, we randomly replicate samples from the minority class (abnormal RV) in the training set to deal with data imbalance (over sampling technique) in the subsequent cross-validation step. To test the ability of 2D-ECHO and laboratory records to predict RV dysfunction, we implemented four machine learning methods with varying complexity using the CARET package in R: a generalized linear model (GLM) (logistic regression), a support vector machine (SVM) using a radial basis function (RBF), a Random Forest (RF) which employs an ensemble of decision trees to construct a predictive model and Elastic net regularization, a regression method that combines the penalties of lasso and ridge regressions (Rpart).

For all ML models, seed values were fixed, tuning and hyper-parameters were set to default as implemented in CARET. We randomly partitioned the data into the ratio of 60:20:20 for training, validation and testing respectively. Then, we opted for the repeated train/validation split or leave group out cross-validation strategy and the ratio of 3:1 to split the training set into train/validation splits; this step is repeated 100 times and is coupled, within the cross-validation, with oversampling for class imbalance in CARET (since out data is skewed towards normal RV).

Each of the ML methods has a built-in feature selection scheme and ranks variables by importance. The top four most important variables were retained with regard to each of the algorithms. In addition, the Boruta algorithm implemented in R was also independently used for feature selection (27). It is built around the random forest classification algorithm and tries to capture all the significant features in a dataset. Subsequently, several ML models are built with either the whole set of variables or selected features from Boruta. A decision tree was built using the best algorithm with the version with the lowest out-of-bag (OOB) error.

Finally, the model with the optimal receiver operating characteristic (ROC) curve was selected to further investigate its corresponding performance on the test set (20% of the population). Furthermore, and to confirm that RF outperformed other methods in terms of AUROC, we used a pairwise t-test with Bonferroni correction to compare the mean of AUROC values, across the same resampling results, from the leave group out cross-validation folds extracted from all ML methods. To compare resampling results across ML methods, we matched the same indices in the R object trainObject$control$index.

Results

One-hundred ninety-six [196] patients were eligible to be included in our study. Twenty-five were excluded because of missing outcomes data and 16 with missing ECHO data. One-hundred fifty-five [155] patients were included in our analysis. Average age was 43±17.1 years and 52/155 (33.5%) were females. Baseline characteristics are summarized in Table 2, patients with abnormal RV function had higher diabetes (33.3% vs. 12.7%; P=0.02) and lower LV EF (34.2 vs. 52.7; P<0.0001). According to CMR, 21 patients were identified to have RV dysfunction, with an RVEF of 34.7%±6.4%, as opposed to 54.7%±6.7% in the normal RV population (P<0.0001).

Table 2 Baseline characteristics
Full table

All four ML algorithms optimized the ROC and generated best models, from either all variables (Table 1) or selected features from Boruta algorithm (Table 3), which were evaluated on the test set (20% of the population). Representative confusion matrices for all variables and selected top variables are shown in Tables 4,5, respectively. Clearly, the Random Forest (RF) model (OOB error of 3.24%, 500 trees, and a mtry of 2) yielded the highest accuracy and the best balance between sensitivity and specificity, especially when using Boruta features: age, FAC_RV, hypertension, TAPSE and S’. Its AUC reached 0.82 when trained with these features, higher than other ML models and general linear regression analysis (AUC =0.71). When comparing ROC curves to confirm that RF outperformed all other methods (Table 6), there was no significant difference in area under ROC between RF and ElasticNet when the same cross-validation folds were compared (P=1). However, ElasticNet seems to overfit; thus, did not generalize well on the test dataset. However, and as predicted, the accuracy drops when we generated the MCC for our models. However, when comparing MCCs; RF (MCC_RF =0.52) was significantly a better choice for classification in our dataset when compared to other algorithms (MCC_SVM =0.3) and general linear regression (MCC_GLM =0.31) (P=0.002).

Table 3 Top five important variables from built-in feature selection with different models
Full table

Table 4 Confusion matrices for the test set
Full table

Table 5 Confusion matrices for the test set using important attributes (age, S, TAPSE, FAC, HTN), from the Boruta Feature selection algorithm, were used to train the ML models
Full table

Table 6 Comparison of area under ROC curves among ML methods using the same folds from the leave group out cross-validation
Full table

Interestingly, the five important variables, flagged with Boruta, were also concordantly selected as the top five most important features only from RF and SVM built-in feature selection functions. All models have an improved performance when trained with the top variables from Boruta except in the ElasticNet model, where it was unchanged.

Also, from the top five important variables extracted from the built-in feature selection functions (Table 3), FAC_RV and hypertension were featured by the different models as top predictors of RV dysfunction. TAPSE and S’ were emphasized by two methods; SVM and random forest but not GLM and Elastic Net. While age was not correlated with the outcome (spearman rho =0.056, P=0.48), it featured in all models except Elastic Net. Diabetes type 2 status showed up only in the regression-based models (GLM and Elastic Net), while smoking exposure was chosen in all models but SVM.

The tree (Figure 1) from the random forest model which classifies best the training data (mtry: 3, OOB error: 4.63%, number of trees: 500). The tree has the maximum number of nodes (25 terminal nodes corresponding to predictions, either normal or abnormal). Finally, an online calculator was constructed using both RF and Elastic-Net Models and is using this link: https://nelhachem.shinyapps.io/ShinyApp_ML/.

Figure 1 Decision Tree built based on the random forest machine learning algorithm. FAC, fractional area change; TAPSE, tricuspid annular plane systolic excursion; S, S-wave velocity.

Discussion

In this paper, we describe a new ML-based algorithm based on a random forest plot analysis that provides reasonable accuracy in detecting low RV function using clinical characteristics and 2D-ECHO derived parameters. The top predictors of RV dysfunction vary between ML-models used, but age, FAC_RV, and HTN were recurring selections. TAPSE and S’ appeared in the two most accurate models.

Accurate evaluation of RV function is crucial as a diagnostic and prognostic factor of several diseases. This has led to appreciation of its complex 3D shape and underscored the limit of 2D-ECHO images in assessing this function. However, 3D-echocardiography requires expertise and is not yet widely available. In some cases of multiple ECHO-derived parameter RV assessment, contradictory interpretation of the measurements might occur, often leading to more confusion. Hence using clinical features in addition to echo data would be able to guide physician effort to detect impaired RV function. Therefore, the use of parameters other than FAC_RV, TAPSE, and S’ should be investigated.

Since TAPSE and S’ are inherently similar and usually correlate (28), the use of another widely studied RV 2D-ECHO derived parameter, such as RV strain, might provide an additional benefit in detection of low RV function when added to the production of the accuracy of our models. In fact, RV strain was shown to have a good correlation with RV function in multiple diseases such as COPD, PH and certain types of HF (15,28-31). Furthermore, our group recently showed that using a speckle-based strain analysis could help differentiate between types of cardiomyopathy in an artificial convoluted network, a commonly used ML method (32). Finally, the use of automatic-border detection (ABD) for strain analysis could be used to eliminate intra-operator variability and to lessen the need for heavy post-processing of images as shown in two recent studies which used ABD and ML-based algorithm to asses RV function from 3D-ECHO (33,34). Another study used ABD for volumetric analysis of the RV, an important factor when assessing RV function (35). Recently, the use of ABD in the estimation of RVEF with 3D-ECHO was shown to be superior to manually drawn border, reducing inter- and intra-observer variability of measurements (36-40).

Another easily accessible and cheap method that could be used in detecting low ventricular function is an ECG. In a study published in nature, investigators were able to produce an AI that would detect low LV function (<35%) with an AUC of 0.93 (41). Replicating the model with a large dataset to create an AI to detect RV function or to incorporate ECG with other parameters to help detect low RV function should be studied.

Alternatively, perhaps there would be value in building predictive models that prioritize either specificity or sensitivity. In our study, models were designed to maximize overall predictive accuracy (F1 score), but possibly in this context a useful model might be one prioritizing high sensitivity for identifying low RV function so we can identify patients in need for CMR.

Finally, future investigations will include ABD along with automatic detection of TAPSE + S’ from captured ECHO clips, spontaneously inserting them into the model and hence getting an instant RV assessment of RV function. Several studies have previously applied ABD technology to extract RV parameters. This cutting-edge technology is still limited by availability of the advanced imaging machines.

Limitations

A major limitation of our study is our small population. Using a larger population would make the algorithm more reliable/replicable and would allow the use of additional data features, such as ECG waveforms. In addition, our population represents a negatively balanced dataset, where dysfunctional RV represented only 14% of our total population. Negatively balanced datasets are a notorious limitation for machine learning algorithm. This is the reason we calculated the Matthews correlation coefficient, which is regarded as a balanced measure in that can be used when classes are of different sizes, i.e., unbalanced datasets. Another limitation is that our study is a single center study, with a majority of subjects from a Middle Eastern ethnic background. Thus, more studies should be conducted to validate our model in other populations.

Another limitation is that the study was retrospective, and thus some limitations in extracting parameters from the ECHO windows, since the studies were not done to asses RV function solely. This has led to limitations in computing RV strain, which could have been used in our algorithm as another feature, as discussed above. The studies included in this analysis were mostly done between 2012 and 2013, before the acquisition of newer higher resolution echocardiography machines. The new machine would have made it easier to acquire images of the RV despite its retrosternal, angle-dependent, respiratory cycle-dependent, position.

Furthermore, the use of RV strain and automated 3D-ECHO-derived RVEF measurements from ECHO images could have provided an added value to our model. However, these measurements pose a technical difficulty outside specialized tertiary medical centers and are not widely used yet, therefore our model uses common 2D-ECHO measurements. Finally, the use of raw ECHO DICOM images in a convolutional neural network-based algorithm might have had an incremental value to the diagnosis of RV dysfunction; however, convolutional neural networks require a significantly larger data set than what is available for training, validating, and testing the models.

Conclusions

Non-invasive assessment of RV function is a technically challenging task and clinicians usually rely on visual assessment of RV size and contractility. In this study, we trained and validated an ML-based algorithm that could detect low RV function from clinical and 2D-ECHO parameters. The algorithm performed better than common statistical methods.

Acknowledgments

We would like thank Dr. Hussein Taleb for his contribution to the proposal.

Funding: Medical Practice Plan at American University of Beirut (https://www.aub.edu.lb/fm/fao/Pages/MPP.aspx).

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at http://dx.doi.org/10.21037/cdt-20-471

Data Sharing Statement: Available at http://dx.doi.org/10.21037/cdt-20-471

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/cdt-20-471). HI serves as an unpaid editorial board member of Cardiovascular Diagnosis and Therapy from Jul 2019 to Jun 2021. The other author has no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Institutional Review Board of the American University of Beirut (Registration Number: IRB00003225). Patient consent was waived due to the retrospective nature of the research.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Burgess MI, Mogulkoc N, Bright-Thomas RJ, et al. Comparison of echocardiographic markers of right ventricular function in determining prognosis in chronic pulmonary disease. J Am Soc Echocardiogr 2002;15:633-9. [Crossref] [PubMed]
2. Ghio S, Gavazzi A, Campana C, et al. Independent and additive prognostic value of right ventricular systolic function and pulmonary artery pressure in patients with chronic heart failure. J Am Coll Cardiol 2001;37:183-8. [Crossref] [PubMed]
3. Mehta SR, Eikelboom JW, Natarajan MK, et al. Impact of right ventricular involvement on mortality and morbidity in patients with inferior myocardial infarction11Drs. Mehta and Eikelboom were recipients of Heart and Stroke Foundation of Canada Research Fellowship Awards. Professor Yusuf is the recipient of a Medical Research Council of Canada Senior Scientist Award and holds a Heart and Stroke Foundation of Ontario Research Chair. J Am Coll Cardiol 2001;37:37-43. [Crossref] [PubMed]
4. Pueschner A, Chattranukulchai P, Heitner JF, et al. The Prevalence, Correlates, and Impact on Cardiac Mortality of Right Ventricular Dysfunction in Nonischemic Cardiomyopathy. JACC Cardiovasc Imaging 2017;10:1225-36. [Crossref] [PubMed]
5. Zornoff LAM, Skali H, Pfeffer MA, et al. Right ventricular dysfunction and risk of heart failure and mortality after myocardial infarction. J Am Coll Cardiol 2002;39:1450-5. [Crossref] [PubMed]
6. Addetia K, Maffessanti F, Yamat M, et al. Three-dimensional echocardiography-based analysis of right ventricular shape in pulmonary arterial hypertension. Eur Heart J Cardiovasc Imaging 2016;17:564-75. [Crossref] [PubMed]
7. Dandel M, Hetzer R. Evaluation of the right ventricle by echocardiography: particularities and major challenges. Expert Rev Cardiovasc Ther 2018;16:259-75. [Crossref] [PubMed]
8. Sade LE, Gulmez O, Ozyer U, et al. Tissue Doppler study of the right ventricle with a multisegmental approach: comparison with cardiac Magn Reson Imaging. J Am Soc Echocardiogr 2009;22:361-8. [Crossref] [PubMed]
9. Lang RM, Badano LP, Mor-Avi V, et al. Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. J Am Soc Echocardiogr 2015;28:1-39.e14. [Crossref] [PubMed]
10. Rajesh GN, Raju D, Nandan D, et al. Echocardiographic assessment of right ventricular function in inferior wall myocardial infarction and angiographic correlation to proximal right coronary artery stenosis. Indian Heart J 2013;65:522-8. [Crossref] [PubMed]
11. Saxena N, Rajagopalan N, Edelman K, et al. Tricuspid annular systolic velocity: a useful measurement in determining right ventricular systolic function regardless of pulmonary artery pressures. Echocardiography 2006;23:750-5. [Crossref] [PubMed]
12. Anavekar NS, Gerson D, Skali H, et al. Two-dimensional assessment of right ventricular function: an echocardiographic-MRI correlative study. Echocardiography 2007;24:452-6. [Crossref] [PubMed]
13. Alonso P, Andres A, Miro V, et al. Diagnostic power of echocardiographic speckle tracking of the tricuspid annular motion to assess right ventricular dysfunction. Int J Cardiol 2014;172:e218-9. [Crossref] [PubMed]
14. Simsek E, Nalbantgil S, Ceylan N, et al. Assessment of right ventricular systolic function in heart transplant patients: Correlation between echocardiography and cardiac Magn Reson Imaging. Investigation of the accuracy and reliability of echocardiography. Echocardiography 2017;34:1432-8. [Crossref] [PubMed]
15. Vitarelli A, Conde Y, Cimino E, et al. Assessment of right ventricular function by strain rate imaging in chronic obstructive pulmonary disease. Eur Respir J 2006;27:268-75. [Crossref] [PubMed]
16. Rudski LG, Lai WW, Afilalo J, et al. Guidelines for the echocardiographic assessment of the right heart in adults: a report from the American Society of Echocardiography endorsed by the European Association of Echocardiography, a registered branch of the European Society of Cardiology, and the Canadian Society of Echocardiography. J Am Soc Echocardiogr 2010;23:685-713. [Crossref] [PubMed]
17. Kurtz C. The practice of clinical echocardiography: Right ventricular anatomy, function, and echocardiographic evaluation. 4th edition. Philadelphia, PA: Elsevier Saunders, 2012.
18. Dragulescu A, Grosse-Wortmann L, Fackoury C, et al. Echocardiographic assessment of right ventricular volumes: a comparison of different techniques in children after surgical repair of tetralogy of Fallot. Eur Heart J Cardiovasc Imaging 2012;13:596-604. [Crossref] [PubMed]
19. van Ginneken B. Fifty years of computer analysis in chest imaging: rule-based, machine learning, deep learning. Radiol Phys Technol 2017;10:23-32. [Crossref] [PubMed]
20. Wang S, Summers RM. Machine learning and radiology. Med Image Anal 2012;16:933-51. [Crossref] [PubMed]
21. Medvedofsky D, Mor-Avi V, Amzulescu M, et al. Three-dimensional echocardiographic quantification of the left-heart chambers using an automated adaptive analytics algorithm: multicentre validation study. Eur Heart J Cardiovasc Imaging 2018;19:47-58. [Crossref] [PubMed]
22. Medvedofsky D, Mor-Avi V, Byku I, et al. Three-Dimensional Echocardiographic Automated Quantification of Left Heart Chamber Volumes Using an Adaptive Analytics Algorithm: Feasibility and Impact of Image Quality in Nonselected Patients. J Am Soc Echocardiogr 2017;30:879-85. [Crossref] [PubMed]
23. Tsang W, Salgo IS, Medvedofsky D, et al. Transthoracic 3D Echocardiographic Left Heart Chamber Quantification Using an Automated Adaptive Analytics Algorithm. JACC Cardiovasc Imaging 2016;9:769-82. [Crossref] [PubMed]
24. Narang A, Mor-Avi V, Prado A, et al. Machine learning based automated dynamic quantification of left heart chamber volumes. Eur Heart J Cardiovasc Imaging 2019;20:541-9. [Crossref] [PubMed]
25. Maffessanti F, Lang RM, Niel J, et al. Three-dimensional analysis of regional left ventricular endocardial curvature from cardiac magnetic resonance images. Magn Reson Imaging 2011;29:516-24. [Crossref] [PubMed]
26. Petersen SE, Aung N, Sanghvi MM, et al. Reference ranges for cardiac structure and function using cardiovascular magnetic resonance (CMR) in Caucasians from the UK Biobank population cohort. J Cardiovasc Magn Reson 2017;19:18. [Crossref] [PubMed]
27. Kursa MB, Rudnicki WR. Feature Selection with the Boruta Package. J Stat Softw 2010;36:13. [Crossref]
28. Chia EM, Hsieh CH, Boyd A, et al. Effects of age and gender on right ventricular systolic and diastolic function using two-dimensional speckle-tracking strain. J Am Soc Echocardiogr 2014;27:1079-86.e1. [Crossref] [PubMed]
29. Barakat AF, Sperry BW, Starling RC, et al. Prognostic Utility of Right Ventricular Free Wall Strain in Low Risk Patients After Orthotopic Heart Transplantation. Am J Cardiol 2017;119:1890-6. [Crossref] [PubMed]
30. Hamada-Harimura Y, Seo Y, Ishizu T, et al. Incremental Prognostic Value of Right Ventricular Strain in Patients With Acute Decompensated Heart Failure. Circ Cardiovasc Imaging 2018;11:e007249. [Crossref] [PubMed]
31. Hulshof HG, Eijsvogels TMH, Kleinnibbelink G, et al. Prognostic value of right ventricular longitudinal strain in patients with pulmonary hypertension: a systematic review and meta-analysis. Eur Heart J Cardiovasc Imaging 2019;20:475-84. [Crossref] [PubMed]
32. Walsh JL, AlJaroudi WA, Lamaa N, et al. A speckle-tracking strain-based artificial neural network model to differentiate cardiomyopathy type. Scand Cardiovasc J 2020;54:92-9. [Crossref] [PubMed]
33. Genovese D, Rashedi N, Weinert L, et al. Machine Learning-Based Three-Dimensional Echocardiographic Quantification of Right Ventricular Size and Function: Validation Against Cardiac Magnetic Resonance. J Am Soc Echocardiogr 2019;32:969-77. [Crossref] [PubMed]
34. Nillesen MM, van Dijk AP, Duijnhouwer AL, et al. Automated Assessment of Right Ventricular Volumes and Function Using Three-Dimensional Transesophageal Echocardiography. Ultrasound Med Biol 2016;42:596-606. [Crossref] [PubMed]
35. Muraru D, Spadotto V, Cecchetto A, et al. New speckle-tracking algorithm for right ventricular volume analysis from three-dimensional echocardiographic data sets: validation with cardiac magnetic resonance and comparison with the previous analysis tool. Eur Heart J Cardiovasc Imaging 2016;17:1279-89. [Crossref] [PubMed]
36. Hamilton-Craig CR, Stedman K, Maxwell R, et al. Accuracy of quantitative echocardiographic measures of right ventricular function as compared to cardiovascular magnetic resonance. Int J Cardiol Heart Vasc 2016;12:38-44. [Crossref] [PubMed]
37. Knight DS, Grasso AE, Quail MA, et al. Accuracy and reproducibility of right ventricular quantification in patients with pressure and volume overload using single-beat three-dimensional echocardiography. J Am Soc Echocardiogr 2015;28:363-74. [Crossref] [PubMed]
38. Medvedofsky D, Mor-Avi V, Kruse E, et al. Quantification of Right Ventricular Size and Function from Contrast-Enhanced Three-Dimensional Echocardiographic Images. J Am Soc Echocardiogr 2017;30:1193-202. [Crossref] [PubMed]
39. Park JB, Lee SP, Lee JH, et al. Quantification of Right Ventricular Volume and Function Using Single-Beat Three-Dimensional Echocardiography: A Validation Study with Cardiac Magnetic Resonance. J Am Soc Echocardiogr 2016;29:392-401. [Crossref] [PubMed]
40. Lopez-Candales A. Applicability of automated functional imaging for assessing right ventricular function. Echocardiography 2013;30:919-28. [Crossref] [PubMed]
41. Attia ZI, Kapa S, Lopez-Jimenez F, et al. Screening for cardiac contractile dysfunction using an artificial intelligence-enabled electrocardiogram. Nat Med 2019;25:70-4. [Crossref] [PubMed]