Skip to main content
The Ultrasound Journal
Download PDF
  • Original Article
  • Open access
  • Published:

Measuring the accuracy of cardiac output using POCUS: the introduction of artificial intelligence into routine care

The Ultrasound Journal volume 14, Article number: 47 (2022) Cite this article

Abstract

Background

Shock management requires quick and reliable means to monitor the hemodynamic effects of fluid resuscitation. Point-of-care ultrasound (POCUS) is a relatively quick and non-invasive imaging technique capable of capturing cardiac output (CO) variations in acute settings. However, POCUS is plagued by variable operator skill and interpretation. Artificial intelligence may assist healthcare professionals obtain more objective and precise measurements during ultrasound imaging, thus increasing usability among users with varying experience. In this feasibility study, we compared the performance of novice POCUS users in measuring CO with manual techniques to a novel automation-assisted technique that provides real-time feedback to correct image acquisition for optimal aortic outflow velocity measurement.

Methods

28 junior critical care trainees with limited experience in POCUS performed manual and automation-assisted CO measurements on a single healthy volunteer. CO measurements were obtained using left ventricular outflow tract (LVOT) velocity time integral (VTI) and LVOT diameter. Measurements obtained by study subjects were compared to those taken by board-certified echocardiographers. Comparative analyses were performed using Spearman’s rank correlation and Bland–Altman matched-pairs analysis.

Results

Adequate image acquisition was 100% feasible. The correlation between manual and automated VTI values was not significant (p = 0.11) and means from both groups underestimated the mean values obtained by board-certified echocardiographers. Automated measurements of VTI in the trainee cohort were found to have more reproducibility, narrower measurement range (6.2 vs. 10.3 cm), and reduced standard deviation (1.98 vs. 2.33 cm) compared to manual measurements. The coefficient of variation across raters was 11.5%, 13.6% and 15.4% for board-certified echocardiographers, automated, and manual VTI tracing, respectively.

Conclusions

Our study demonstrates that novel automation-assisted VTI is feasible and can decrease variability while increasing precision in CO measurement. These results support the use of artificial intelligence-augmented image acquisition in routine critical care ultrasound and may have a role for evaluating the response of CO to hemodynamic interventions. Further investigations into artificial intelligence-assisted ultrasound systems in clinical settings are warranted.

Background

While multiple methods for measuring cardiac output (CO) exist [1], methods like point-of-care ultrasound (POCUS) and transthoracic echocardiography are increasingly popular given their non-invasive nature, cost-effectiveness, and availability in most healthcare settings [2,3,4]. Specifically, POCUS allows for repeated assessment of CO during therapeutic maneuvers in shock management [5]. Ultrasonographic assessment of CO is calculated from velocity time integral (VTI) via the left ventricular outflow tract (LVOT). LVOT VTI is often preferred for measurement because it is easily measurable, has good inter/intra-observer rater reliability, and correlates strongly with PAC (pulmonary artery catheter) thermodilution [6]. As such, using VTI for CO measurements is a reasonable surrogate for invasive estimates for CO as determined by ESICM (European Society of Intensive Care Medicine) consensus guidelines [7].

In practice, however, VTI has been criticized for poor accuracy and reproducibility [6, 8]. However, automation and artificial intelligence (AI) may mitigate some of the VTI’s shortcomings in POCUS. AI now allows for real-time user feedback during image acquisition, which may reduce human error and narrow discrepancies in training. To evaluate accuracy, reproducibility, and feasibility of AI-assisted ultrasound, we conducted a study testing the ability of novice ultrasound (US) trainees to measure CO using automation-assisted VTI (VTIauto).

Methods

Participants

This prospective study was conducted at a regional POCUS training conference with critical care trainees from five academic centers in Southern California [9]. We recruited first-year fellows in critical care fellowships with limited experience in echocardiography. All VTI studies were conducted following an 8-h didactic session and a preliminary hands-on training session on obtaining basic echocardiographic windows. All study participants, including both trainees and study subject, provided informed consent. The Institutional Review Board at University of California Los Angeles has approved this study (UCLA IRB# 20-000690).

Measurements

Multiple consecutive US assessments of CO were performed on a single healthy volunteer placed in the lateral decubitus position. Over a period of 3 h, 28 first-year critical care trainees performed manual and automated LVOT VTI measurements. For both manual and automated measurements, LVOT diameter (DLVOT) was calculated in the parasternal long axis (PLAX) position. Aortic outflow velocities were obtained by placing the pulsed-wave Doppler tracer in the LVOT immediately proximal to the aortic valve cusps in the apical 5-chamber (AP5c) view. Only the largest of five consecutive beats were selected to account for respiratory variation. While no direct assistance was given to trainees, image acceptability was verified by two senior fellows in critical care medicine with 3 years of formal training in basic and advanced ultrasonography.

Manual VTI measurements by trainees (VTImnl-train) were calculated by tracing the spectral envelope of the single-best beat with the largest, most clearly visible Doppler profile (determined with the assistance of senior fellows). CO was then calculated using the modified Bernoulli’s equation (CO = VTI × (DLVOT/2)2 × π × HR). VTIauto was calculated using the ultrasound manufacturer’s propriety software (Venue Go, GE Healthcare, Waukesha, WI). This software included real-time feedback to correct image acquisition for optimal aortic outflow velocity measurement, along with fully automated spectral envelop tracing. The measurements of 5 sequential beats were then used as part of the automated algorithm calculation for VTIauto and automated CO (COauto). The averages of three separate, manual VTI measurements obtained by two board-certified echocardiographers (VTImnl-exp) were used as reference for VTI and CO. These measurements were performed at the beginning and end of the study on the same volunteer to capture any physiologic changes in VTI that may have occurred over the course of the study (approximately 3 h) (Fig. 1).

Fig. 1
figure 1

Comparison between manually obtained measures by trainees (Mnl-train), automated measurements (Auto), and manual measurements of certified ultrasonographers (Mnl-exp). A Mean velocity time integral (VTI), B range of obtained VTI measurements, C coefficient of variance of VTI measures, D mean cardiac output (CO), E range of obtained CO measurements, F coefficient of variance of CO measure

Full size image

Statistical analysis

Feasibility of image capture was defined as the percent of trainees able to acquire adequate PLAX and AP5c views for DLVOT and LVOT-VTI measurements. We evaluated the inter-rater reproducibility of both manual and automated VTI tracing and assessed the range of measurements, the standard deviation, and coefficient of variation between these two measurements. We also assessed the agreement between automated and manual measurements using Bland–Altman matched-pairs analysis. To determine accuracy, we used DLVOT and VTI obtained by advanced echocardiographers as reference measurements for cardiac measurements obtained by novice US users.

Results

Baseline characteristics of novice echocardiographers

Mean self-reported experience in POCUS was 2.6 ± 2 years. 12 of 28 participants stated they had received prior ultrasound training; however, none had previous training in VTI or obtaining CO estimates through US (Table 1). In terms of feasibility, all 28 participants (100%) were able to obtain ultrasonographic views adequate for capturing required metrics of VTI and CO after hands-on training.

Table 1 Operators’ characteristics and study measurements
Full size table

Precision and reproducibility of VTI measurements

Measurement results are presented in Table 1. The average VTIauto obtained by trainees was 14.8 ± 2.0 cm, while the average VTImnl-train was 15.4 ± 2.3 cm. The correlation between VTIauto and VTImnl-train was not significant (ρ = 0.30, p = 0.11) (Fig. 2a). Using Bland–Altman matched-pairs analysis, we determined that the mean paired difference between VTImnl-train and VTIauto was not significantly different from zero (mean difference = 0.56, p = 0.27), indicating that there was no systematic bias between the two methods (Fig. 2b). The 95% limits of agreement were between − 4.6 and + 5.6. VTIauto was more reproducible across raters than VTImnl-train, with a narrower measurement range (6.2 cm vs. 10.3 cm), and smaller standard deviation (1.98 cm vs. 2.33 cm). The coefficient of variation across raters (i.e., the standard deviation relative to the mean) was smaller for VTIauto compared to VTImnl-train (13.6% vs. 15.4%). The coefficient of variation was lowest (11.5%) for our control group of advanced echocardiographers.

Fig. 2
figure 2

Comparative analysis of trainee-obtained manual measurements (VTImnl-train and COmnl-train) and automated measurements (VTIauto and COauto). A Correlation between VTImnl-train and VTIauto, B Bland–Altman agreement analysis between VTImnl-train and VTIauto, C correlation between COmnl-train and COauto, D Bland–Altman agreement analysis between COmnl-train and COauto

Full size image

Accuracy of VTI measurements

Both mean VTIauto (14.8 ± 2.0 cm) and mean VTImnl-train (15.4 ± 2.4 cm) underestimated the mean VTI obtained by the two advanced echocardiographers (16.7 ± 1.4 cm). Considering VTImnl-exp as the gold standard, VTIauto represented an 11.4% underestimate and VTImnl-train a 7.8% underestimate of true VTI. All DLVOT measurements were done by trainees as no automatic measurement was possible. This measurement demonstrated a wide range of values (1.9–2.9 cm) with ± 20.8% variability. By contrast, DLVOT by both expert echocardiographers was only 2.2 cm.

Precision and accuracy of CO measurement

While COauto and COmnl-train correlated well (ρ = 0.76 p < 0.001), Fig. 2c, both mean COauto (4.5 ± 1.2 L/min) and mean COmnl-train (4.6 ± 1.0 L/min) underestimated mean COmnl-exp (5.7 ± 2.5 L/min). Bland–Altman matched-pairs analysis showed no significant difference between the paired means (mean difference = 0.1, p = 0.45), indicating no systematic bias between the two methods (Fig. 2d). Additionally, 95% of the paired differences were expected to fall between − 1.54 and + 1.86. Interestingly, the measurement range across raters was wider with COauto (4.2 vs. 3.5 L/min) while inter-rater coefficient of variation was also higher with automatic measurement (26.7% vs. 21.7%).

Sensitivity analysis for VTI inter-rater reproducibility

To evaluate the effects of physiologic variability on VTI precision and reproducibility, we analyzed the physiologic alterations of VTI during the 3 h of our study by splitting participants into three groups that would take measurements at each hour and compared their VTI means to each other. We then evaluated reproducibility within each time interval for comparison to the previous time interval’s estimates. The VTI means across the three time intervals were 14.3 cm, 15.5 cm, and 16.3 cm for manual measurements and 14.0 cm, 15.0 cm, and 15.5 cm for automated measurements. These differences in means were not statistically significant. The inter-rater standard deviations (SDs) within the three time intervals were 2.49, 2.40 and 2.06 cm for manual measurements and were 1.96, 2.14, and 1.86 cm for automated measurements. The corresponding coefficient of variation (CV) estimates were 17.4%, 15.5%, and 12.7% for manual measurements and were 14.0%, 14.3%, and 12.0% for automated measurements (Fig. 3). The pooled SD combining the three time intervals was 2.31 cm with a CV (pooled SD divided by overall mean) of 15.0% for manual measurements. Meanwhile, pooled SD was 1.98 cm with a CV of 13.4% for automated measurements across all three time intervals. These estimates were close to previous estimates that ignored potential time effects.

Fig. 3
figure 3

Physiologic changes in VTI during testing represented as mean measurements of both automatic and manual methods in three consecutive time intervals

Full size image

Discussion

In this study, we evaluated the ability of novice critical care trainees to quantify CO using both manual and automated VTI measurements. We then compared their measurements to those attained by our control group of expert echocardiographers. We found that the feasibility of using AI-augmented image acquisition was excellent and, as defined by the coefficient of variation, the precision of VTI capture was higher with automatic VTI tracing when compared to manual VTI tracing.

Our relatively small, prospective study addresses several important issues related to the use of POCUS in cardiac output assessment. First, our study demonstrates that the ability to obtain relatively adequate measurements required for CO estimation does not require absolute expertise but a short, structured course with opportunities for hands-on practice. All 28 trainees were able to obtain adequate PLAX and AP5c views, resulting in 100% feasibility for image acquisition despite minimal US training. However, our reported feasibility for VTI is significantly higher than those previously published [10,11,12] even when accounting for training level [10]. This may be because all examinations were obtained on a single healthy volunteer who was hemodynamically stable and adequately positioned by experienced ultrasonographers, thus facilitating image acquisition. However, it must be noted that medical professionals with minimal training can obtain adequate parasternal and apical views 88% of the time when assessing critically ill patients [13]. These data further solidify the need for including POCUS education in medical training [14, 15].

Our study also considered accuracy and precision separately when measuring cardiac output and VTI because their distinction is clinically relevant. For example, accuracy of cardiac output has diagnostic implications in determining etiologies for specific hemodynamic states such as cardiogenic or distributive shock. By contrast, precision of cardiac measurements carries therapeutic implications. As an example, knowing with statistical confidence that SV or CO changed in response to an intervention, independent of SV or CO accuracy, is crucial for determining treatment effectiveness. In our study, trainees had larger measurement variations for DLVOT (1.9 to 2.9 cm) when compared to expert echocardiographers. Interestingly, both VTImnl-train and VTIauto underestimated true VTI as determined by expert echocardiographers. As a result, both manual and automatic cardiac output measurements had poor accuracy compared to the standardized control. Measurement variation likely has multiple sources, including human factors [16, 17], ultrasound factors [18], and intrinsic fluctuation secondary to patient physiology (e.g., respiratory variation). However, ultrasound and patient factors were less likely to contribute to measurement variability in our study since trainees and expert echocardiographers used the same machines and obtained measurements from only one patient. Therefore, differences in the coefficient of variation are likely to be operator dependent. Other differences noted in our study include the number of beats sampled before and after an intervention to detect a 10% SV change with statistical confidence. Expert echocardiographers, automatic VTI tracing, or manual VTI tracing by trainees were 10, 27, and 34, respectively [19]. These beat numbers are greater than the recommended 3 in clinical practice [20].

Hemodynamic assessment in our study was based on velocity time integral, which has been criticized for its difficulty of attainment in critical care settings. For example, the reported feasibility of VTI in the intensive care unit (ICU) ranges from 37 to 90% [10, 11, 21], likely due to inter-rater variability [10]. Other downsides to using VTI to measure CO include its dependence on insonation angles, which presents their own set of challenges [22,23,24]. Nevertheless, VTI is commonly used in tracking changes in SV and CO in critically ill patients [22, 25, 26] and has also demonstrated the ability to predict outcomes in select populations [27]. Together with other Doppler-based measures [28], VTI has become an increasingly reliable metric to assess fluid responsiveness in shock [29, 30]. Indeed, our results showcase VTI’s reproducibility between trainees and expert operators. However, it should be noted that VTIauto was more reproducible across raters than VTImnl-train, demonstrating narrower measurement range, smaller standard deviation, and lower coefficient of variation. Taken together, these results show that automation could further solidify VTI as a reliable metric for cardiac output measurements and reduce operator-dependent variability in clinical practice.

AI-assisted programs not only augment user experience, but also redefine the capabilities of US in a critical care setting [31]. As an example, automated VTI measuring systems highlight the importance of real-time feedback by aiding fatigued users, who may be more prone to making errors, perform routine echocardiographic tasks such as spectral envelope tracing, chamber volume estimations, and tracing endomyocardial boundaries. Equally important, AI can help determine view quality and assist directly with US image acquisition by guiding users towards optimal viewing angles. In fact, Zhang et al. trained an algorithm through a convolutional neural network model to accurately identify 23 viewpoints and segment cardiac chambers across 5 different views. The model was able to perform these tasks correctly 96% of the time. Even more impressive was its ability to flag views with partially obscured cardiac chambers [32]. As demonstrated in our study, our US device was able to identify foreshortened, or otherwise inadequate, 5-chamber views with poor aortic outflow jets and guide the user to an acceptable view through a series of coded Doppler box colors. This live feedback can provide users with accurate measurements for clinical decision-making in addition to training users to improve their US skills through a positive feedback loop. Therefore, we believe that educating operators along with the development of AI will not only train operators to use ultrasound, but also train ultrasound itself to tolerate different operators.

There are numerous studies demonstrating the utility of AI in estimating left ventricular hypertrophy, spectral wave tracing, right and left ventricular ejection fraction, three-dimensional chamber volume analysis, and regional wall motion [33,34,35,36,37,38,39,40,41]. However, LVOT VTI automation remains understudied. To our knowledge, there is only one report assessing automated VTI (VTIauto) accuracy, which used an animal model for hemorrhagic shock [42]. In this study, however, feasibility was low (60%) and the correlation coefficient between VTIauto and PAC thermodilution was moderate at best (r = 0.66). As a result, our study will add to the somewhat sparse body of knowledge surrounding automated VTI and hopefully pave a path for larger, structured prospective studies that can further evaluate AI approaches in POCUS for hemodynamic assessments.

There are several limitations to our study. First, manual and automated measurements of CO are highly dependent upon DLVOT, so any inaccuracies in DLVOT are amplified by the modified Bernoulli equation. This was a large source of inaccuracy in CO measurement by the trainee group and could be improved by incorporating DLVOT measurement into the automation model. All exams were also done on one standardized volunteer to standardize comparison between participants and experts. While we minimized movement and stress on our study subject, stroke volume, and therefore VTI, can change during the study period, warranting the need for more frequent control measurements by expert echocardiographers. Additionally, our choice to only study novice sonographers inherently decrease endpoint measurement accuracy when compared to experienced echocardiographers. However, we chose this population for the study because there is no standardized US training in the ICU, and we sought to shed light on this subgroup of clinicians. In addition, novice US users are likely to benefit the most from AI assistance since they lack knowledge on US standard operating procedures. We also did not assess how long it took trainees to obtain appropriate LVOT VTI images as it was beyond the scope of our study. However, previous studies with participants of similar proficiency with US found the median acquisition time to be about 2 min [18, 19]. Further studies are needed to assess whether automated processes can help hasten image acquisition time. Finally, our study is small, with only 28 participants assessing VTI on one study patient who was healthy and hemodynamically stable. Therefore, future studies are needed to assess whether AI-assisted POCUS is a viable option in a clinical setting with patients with varying pathologies or experiencing active hemodynamic instability. Nevertheless, the variability in VTI measurement that we encountered between novice and expert sonographers remained significant despite a small sample size. Further evaluation of AI-augmented VTI to improve accessibility, reliability, and accuracy will be needed to fully appreciate the effects of AI on POCUS in critical care.

Conclusion

The role of Doppler-based evaluation of cardiac output has not been firmly established in daily practice with POCUS, primarily due to difficulty with acquiring adequate images, concern about the ability to precisely measure VTI, and the complexities associated with calculating stroke volume. Automation may offer new, efficient ways to assess CO by addressing each of these aforementioned challenges and allow for more reliable and simpler repetitive hemodynamic assessment that can be performed by an operator with limited ultrasonographic experience. In our study, automatic VTI tracing improved the coefficient of variation in novice critical care trainees compared to manual tracings. These results support the use of AI-augmented image acquisition and may have implications for evaluating the hemodynamic interventions using VTI and SV.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

AI:

Artificial intelligence

AP5c:

Apical 5-chamber

CO:

Cardiac output

CV:

Coefficient of variation

DLVOT :

Left ventricular outflow tract diameter

ESICM:

European Society of Intensive Care Medicine

HR:

Heart rate

ICU:

Intensive care unit

LVOT:

Left ventricular outflow tract

PAC:

Pulmonary artery catheter

PLAX:

Parasternal long axis

POCUS:

Point-of-care ultrasound

SD:

Standard deviation

SV:

Stroke volume

UCLA:

University of California, Los Angeles

US:

Ultrasound

VTI:

Velocity time integral

References

  1. Mehta Y, Arora D (2014) Newer methods of cardiac output monitoring. World J Cardiol 6(9):1022–1029

    Article  Google Scholar 

  2. Millington SJ, Arntfield RT (2013) Advanced point-of-care cardiac ultrasound examination: Doppler applications, valvular assessment, and advanced right heart examination. Glob Heart 8(4):305–312

    Article  Google Scholar 

  3. Huntsman LL, Stewart DK, Barnes SR et al (1983) Noninvasive Doppler determination of cardiac output in man. Clin Valid Circul 67(3):593–602

    CAS  Google Scholar 

  4. Zhang Y, Wang Y, Shi J et al (2019) Cardiac output measurements via echocardiography versus thermodilution: a systematic review and meta-analysis. PLoS ONE 14(10):e0222105

    Article  CAS  Google Scholar 

  5. Rhodes A, Evans LE, Alhazzani W et al (2017) Surviving sepsis campaign: international guidelines for management of sepsis and septic shock: 2016. Crit Care Med 45(3):486–552

    Article  Google Scholar 

  6. Maeder MT, Karapanagiotidis S, Dewar EM et al (2015) Accuracy of echocardiographic cardiac index assessment in subjects with preserved left ventricular ejection fraction. Echocardiography 32(11):1628–1638

    Article  Google Scholar 

  7. Cecconi M, De Backer D, Antonelli M et al (2014) Consensus on circulatory shock and hemodynamic monitoring. Task force of the European Society of Intensive Care Medicine. Intens Care Med 40(12):1795–1815

    Article  Google Scholar 

  8. Blancas R, Martínez-González Ó, Ballesteros D et al (2019) Lack of correlation between left ventricular outflow tract velocity time integral and stroke volume index in mechanically ventilated patients. Med Intens (Engl Ed) 43(2):73–78

    Article  CAS  Google Scholar 

  9. Anklesaria ZCLR, Chang C, Kory P, Williams J, Barjaktarevic I (2017) Critical care ultrasound training standardization: southern California experience American thoracic society international conference abstracts. Am J Respir Crit Care Med 195(A3293):2017

    Google Scholar 

  10. Betcher J, Majkrzak A, Cranford J et al (2018) Feasibility study of advanced focused cardiac measurements within the emergency department. Crit Ultrasound J 10(1):10

    Article  Google Scholar 

  11. McGregor D, Sharma S, Gupta S et al (2019) Emergency department non-invasive cardiac output study (EDNICO): a feasibility and repeatability study. Scand J Trauma Resuscit Emerg Med 27(1):30

    Article  CAS  Google Scholar 

  12. Bergamaschi V, Vignazia GL, Messina A et al (2019) Transthoracic echocardiographic assessment of cardiac output in mechanically ventilated critically ill patients by intensive care unit physicians. Braz J Anesthesiol (Engl Ed) 69(1):20–26

    Article  Google Scholar 

  13. Vignon P, Dugard A, Abraham J et al (2007) Focused training for goal-oriented hand-held echocardiography performed by noncardiologist residents in the intensive care unit. Intens Care Med 33(10):1795–1799

    Article  Google Scholar 

  14. LoPresti CM, Jensen TP, Dversdal RK et al (2019) Point-of-care ultrasound for internal medicine residency training: a position statement from the alliance of academic internal medicine. Am J Med 132(11):1356–1360

    Article  Google Scholar 

  15. Stowell JR, Kessler R, Lewiss RE et al (2018) Critical care ultrasound: a national survey across specialties. J Clin Ultrasound 46(3):167–177

    Article  Google Scholar 

  16. Lui EY, Steinman AH, Cobbold RS et al (2005) Human factors as a source of error in peak Doppler velocity measurement. J Vasc Surg 42(5):972

    Article  Google Scholar 

  17. Radparvar JR, Lim G, Chiem AT (2019) Effect of insonation angle on peak systolic velocity variation. Am J Emerg Med 1:1

    Google Scholar 

  18. Hoskins PR (2011) Estimation of blood velocity, volumetric flow and wall shear rate using Doppler ultrasound. Ultrasound 19(3):120–129

    Article  Google Scholar 

  19. Van Belle G (2011) Statistical rules of thumb. Wiley, London

    Google Scholar 

  20. Jozwiak M, Mercado P, Teboul J-L et al (2019) What is the lowest change in cardiac output that transthoracic echocardiography can detect? Crit Care 23(1):116

    Article  Google Scholar 

  21. Am Dinh V, Ko HS, Rao R et al (2012) Measuring cardiac index with a focused cardiac ultrasound examination in the ED. Am J Emerg Med 30(9):1845–1851

    Article  Google Scholar 

  22. Young LE, Blissitt KJ, Bartram DH et al (1996) Measurement of cardiac output by transoesophageal Doppler echocardiography in anaesthetized horses: comparison with thermodilution. Br J Anaesth 77(6):773–780

    Article  CAS  Google Scholar 

  23. Schober P, Loer SA, Schwarte LA (2009) Transesophageal Doppler devices: a technical review. J Clin Monit Comput 23(6):391–401

    Article  Google Scholar 

  24. Marik PE (2016) Fluid responsiveness and the six guiding principles of fluid resuscitation. Crit Care Med 44(10):1920–1922

    Article  Google Scholar 

  25. Tan C, Rubenson D, Srivastava A et al (2017) Left ventricular outflow tract velocity time integral outperforms ejection fraction and Doppler-derived cardiac output for predicting outcomes in a select advanced heart failure cohort. Cardiovasc Ultrasound 15(1):18

    Article  Google Scholar 

  26. Blanco P, Aguiar FM, Blaivas M (2015) Rapid ultrasound in shock (RUSH) velocity-time integral: a proposal to expand the RUSH protocol. J Ultrasound Med 34(9):1691–1700

    Article  Google Scholar 

  27. Blanco P (2020) Rationale for using the velocity-time integral and the minute distance for assessing the stroke volume and cardiac output in point-of-care settings. Ultrasound J 12(1):21

    Article  Google Scholar 

  28. Barjaktarevic I, Toppen WE, Hu S et al (2018) Ultrasound assessment of the change in carotid corrected flow time in fluid responsiveness in undifferentiated shock. Crit Care Med 46(11):e1040–e1046

    Article  Google Scholar 

  29. Monnet X, Marik PE, Teboul JL (2016) Prediction of fluid responsiveness: an update. Ann Intensive Care 6(1):111

    Article  Google Scholar 

  30. Muller L, Toumi M, Bousquet PJ et al (2011) An increase in aortic blood flow after an infusion of 100 mL colloid over 1 minute can predict fluid responsiveness: the mini-fluid challenge study. Anesthesiology 115(3):541–547

    Article  CAS  Google Scholar 

  31. Barjaktarevic I, Kenny JS, Berlin D et al (2020) The evolution of ultrasound in critical care: from procedural guidance to hemodynamic monitor. J Ultrasound Med 1:1

    Google Scholar 

  32. Zhang J, Gajjala S, Agrawal P et al (2018) Fully automated echocardiogram interpretation in clinical practice: feasibility and diagnostic accuracy. Circulation 138(16):1623–1635

    Article  Google Scholar 

  33. Kwon J-M, Jeon K-H, Kim HM et al (2020) Comparing the performance of artificial intelligence and conventional diagnosis criteria for detecting left ventricular hypertrophy using electrocardiography. EP Europace 22(3):412–419

    Article  Google Scholar 

  34. Cobey FC (2017) Intelligent algorithms in perioperative echocardiography: a new era. J Am Soc Echocardiogr 30(10):A26–A27

    Article  Google Scholar 

  35. Tsay D, Patterson C (2018) From machine learning to artificial intelligence applications in cardiac care: real-world examples in improving imaging and patient access. Circulation 138(22):2569–2575

    Article  Google Scholar 

  36. Thavendiranathan P, Liu S, Verhaert D et al (2012) Feasibility, accuracy, and reproducibility of real-time full-volume 3D transthoracic echocardiography to measure LV volumes and systolic function: a fully automated endocardial contouring algorithm in sinus rhythm and atrial fibrillation. JACC Cardiovasc Imaging 5(3):239–251

    Article  Google Scholar 

  37. Tamborini G, Piazzese C, Lang RM et al (2017) Feasibility and accuracy of automated software for transthoracic three-dimensional left ventricular volume and function analysis: comparisons with two-dimensional echocardiography, three-dimensional transthoracic manual method, and cardiac magnetic resonance imaging. J Am Soc Echocardiogr 30(11):1049–1058

    Article  Google Scholar 

  38. Abdi AH, Luong C, Tsang T et al (2017) Automatic quality assessment of echocardiograms using convolutional neural networks: feasibility on the apical four-chamber view. IEEE Trans Med Imaging 36(6):1221–1230

    Article  Google Scholar 

  39. Jafari MH, Girgis H, Van Woudenberg N et al (2019) Automatic biplane left ventricular ejection fraction estimation with mobile point-of-care ultrasound using multi-task learning and adversarial training. Int J Comput Assist Radiol Surg 14(6):1027–1037

    Article  Google Scholar 

  40. Medvedofsky D, Addetia K, Hamilton J et al (2015) Semi-automated echocardiographic quantification of right ventricular size and function. Int J Cardiovasc Imaging 31(6):1149–1157

    Article  Google Scholar 

  41. Kusunose K, Abe T, Haga A et al (2020) A deep learning approach for assessment of regional wall motion abnormality from echocardiographic images. JACC Cardiovasc Imaging 13(2):374–381

    Article  Google Scholar 

  42. Bobbia X, Muller L, Claret P-G et al (2019) A new echocardiographic tool for cardiac output evaluation: an experimental study. Shock 52(4):449–455

    Article  CAS  Google Scholar 

Download references

Acknowledgements

The authors would like to thank GE Healthcare for allowing us to borrow and use the Venue Go ultrasound machine and for the support in the organization of the Southern California Regional Critical Care Ultrasound training program. We also thank all trainees of this program who consented to participate in this study.

Funding

No external funding related to the preparation of this manuscript. This study was supported by NIH Grant UL1TR001881.

Author information

Authors and Affiliations

  1. Division of Interventional Pulmonology, Beth Israel Medical Center and Massachusetts General Hospital, Boston, MA, USA

    Faisal Shaikh

  2. Health Sciences North Research Institute, Sudbury, ON, Canada

    Jon-Emile Kenny

  3. Flosonics Medical, Sudbury, ON, Canada

    Jon-Emile Kenny

  4. Division of Pulmonary and Critical Care, Washington DC VA Medical Center, Washington, DC, USA

    Omar Awan

  5. Department of Medicine Statistics, David Geffen School of Medicine at University of California, Los Angeles, CA, USA

    Daniela Markovic

  6. Division of Cardiothoracic Surgery, Cedars Sinai Medical Center, Los Angeles, CA, USA

    Oren Friedman & Sidharth Singh

  7. Division of Pulmonary and Critical Care, David Geffen School of Medicine at University of California Los Angeles, Los Angeles, CA, USA

    Tao He, Nida Qadir & Igor Barjaktarevic

  8. Division of Pulmonary and Critical Care Medicine, David Geffen School of Medicine, University of California Los Angeles, 10833 Le Conte Avenue, CHS Building, 43118, Los Angeles, CA, 90095, USA

    Peter Yan

Authors
  1. Faisal Shaikh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jon-Emile Kenny
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Omar Awan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniela Markovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Oren Friedman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tao He
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sidharth Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Peter Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Nida Qadir
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Igor Barjaktarevic
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors were involved in the conceptualization of the study. FS, OA, OF, TH, SS, NQ, and IB acquired the data, developed the analytic plan, and write-up of the paper. DM provided statistical analysis. JK and PY edited the paper. All authors reviewed and approved the final version of the manuscript for submission.

Corresponding author

Correspondence to Peter Yan.

Ethics declarations

Ethics approval and consent to participate

All study participants including the trainees and study subject provided informed consent. This study was reviewed and approved by the UCLA Institutional Review Board (20-000690).

Consent for publication

Trainees and study subject enrolled in this study consented to having study outcomes published.

Competing interests

Authors report no competing interests related to this study. IB has received grants and consulting fees for GE Healthcare. JEK is working with Flosonics, a start-up developing a commercial version of the Doppler patch. Other authors report no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shaikh, F., Kenny, JE., Awan, O. et al. Measuring the accuracy of cardiac output using POCUS: the introduction of artificial intelligence into routine care. Ultrasound J 14, 47 (2022). https://doi.org/10.1186/s13089-022-00301-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13089-022-00301-6

Keywords