- Original article
- Open Access
Basic critical care echocardiography training of intensivists allows reproducible and reliable measurements of cardiac output
The Ultrasound Journal volume 11, Article number: 5 (2019)
Although pulmonary artery catheters (PACs) have been the reference standard for calculating cardiac output, echocardiographic estimation of cardiac output (CO) by cardiologists has shown high accuracy compared to PAC measurements. A few studies have assessed the accuracy of echocardiographic estimation of CO in critically ill patients by intensivists with basic training. The aim of this study was to evaluate the accuracy of CO measurements by intensivists with basic training using pulsed-wave Doppler ultrasound vs. PACs in critically ill patients.
Critically ill patients who required hemodynamic monitoring with a PAC were eligible for the study. Three different intensivists with basic critical care echocardiography training obtained three measurements of CO on each patient. The maximum of three separate left-ventricular outflow tract diameter measurements and the mean of three LVOT velocity time integral measurements were used. The inter-observer reliability and correlation of CO measured by PACs vs. critical care echocardiography were assessed.
A total of 20 patients were included. Data were analyzed comparing the measurements of CO by PAC vs. echocardiography. The inter-observer reliability for measuring CO by echocardiography was good based on a coefficient of intraclass correlation of 0.6 (95% CI 0.48–0.86, p < 0.001). Bias and limits of agreement between the two techniques were acceptable (0.64 ± 1.18 L/min, 95% limits of agreement of − 1.73 to 3.01 L/min). In patients with CO < 6.5 L/min, the agreement between CO measured by PAC vs. echocardiography improved (0.13 ± 0.89 L/min; 95% limits of agreement of − 1.64 to 2.22 L/min). The mean percentage of error between the two methods was 17%.
Critical care echocardiography performed at the bedside by intensivists with basic critical care echocardiography training is an accurate and reproducible technique to measure cardiac output in critically ill patients.
Cardiac output (CO) is the reference standard measurement for assessing target organ perfusion and oxygen delivery in shock. Assessing CO in critically ill patients allows physicians to determine hemodynamic status, identify the most appropriate therapeutic strategy, and monitor the effects of therapy.
Insertion of a pulmonary artery catheter (PAC) has been historically required to calculate CO by thermodilution . However, routine use of PACs in patients with shock is no longer recommended, except in those patients presenting refractory shock, cardiogenic shock, or right-ventricular dysfunction . In recent years, there has been increasing interest to develop non-invasive or minimally invasive technologies to measure CO. Among them, critical care echocardiography (CCE) has emerged as a promising technique that is commonly available, less expensive, and non-invasive (transthoracic echocardiography), or minimally invasive (transesophageal echocardiography) [3, 4].
In stable patients, estimation of CO by CCE has been shown to be accurate when compared to the standard thermodilution technique using a PAC [5,6,7]. A few studies have compared the accuracy of these techniques in critically ill patients , likely due to limited ability to acquire high-quality images in critically ill patients . Despite this, technological advancements are making it easier to obtain high-quality images, and as recommendations on appropriate use of CCE in intensive-care units (ICUs) have emerged [10,11,12,13], CCE has become standard practice in many ICUs to evaluate cardiac function.
The primary objective of this study was to compare CO measured by intensivists with basic CCE skills using pulsed-wave Doppler (PWD) vs. PAC in critically ill patients. The secondary objective was to evaluate the inter-observer reliability of PWD-CO measured amongst intensivists with basic CCE skills, as well as identify factors associated with difficult acquisition of PWD-CO measurements with CCE.
We performed an observational study in a 30-bed medical ICU at Joan XXIII University Hospital in Spain. Approval was obtained from the Joan XXIII University hospital Ethics Committee (IRB # 88/2013), and the study was considered to present minimal risk to subjects. Informed consent was obtained from each subject or their next of kin.
Critically ill patients who required hemodynamic monitoring and were admitted to the ICU were eligible for enrollment from May 2013 to May 2015. Additional eligibility criteria included age > 18, monitoring with a PAC, and interpretable images acquired by CCE. Exclusion criteria included a medical history of congenital heart disease, severe tricuspid regurgitation, severe aortic regurgitation, aortic stenosis, pregnancy, and atrial fibrillation. CO measurements were acquired independent of the subject’s medical and nursing care, and investigators did not change medical management based on findings of this study.
Before study enrolment, three intensivists were trained to measure CO with a portable ultrasound machine by attending a CCE course that included 10 h of didactics and 4 h of hands-on instruction on acquisition of high-quality parasternal long-axis and apical 5-chamber views. Training also included 10 h of didactics and 6 h of hands-on instruction for advanced cardiac training to learn how to use cardiac software to measure left-ventricular outflow tract diameter (LVOTd) and the left-ventricular outflow tract velocity time integral (VTI).
Study protocol and data measurements
Subjects were enrolled during the first 24 h of being invasively monitored with a PAC. Decision to insert a PAC was at the discretion of the treating physician. The following demographic, clinical, and physiologic data were collected: age, sex, weight, height, heart rate (HR), central venous pressure (CVP), mean arterial blood pressure (MAP), Acute Physiology and Chronic Health Evaluation II score (APACHE II) , the Sequential Organ-Failure Assessment (SOFA) score , use of mechanical ventilation (MV), positive end-expiratory pressure (PEEP), use of renal replacement therapy, need for vasoactive drugs, and interpretability of the ultrasound images.
All echocardiographic measurements were done with an Esaote MyLab 30 GOLD cardiovascular ultrasound system (Esaote, Geneva, Italy) equipped with a 3.5 MHz phased-array transducer. Measurements were obtained independently by three blinded intensivists that included a set of hemodynamic parameters with LVOTd, VTI, and HR. All ultrasound images obtained by the three intensivists were stored in digital format and analyzed independently by two blinded investigators to assess the interpretability of the images using a standardized rating scale .
Once a subject was enrolled, the three intensivists performed sequential measurements of PWD-CO. The PAC-CO was obtained after each echocardiographic measurement. The PWD-CO was calculated using the maximum value of three LVOTd measurements and the average of three VTI values . The PWD-CO was calculated as follows:
The LVOTd was measured from a parasternal long-axis view (Fig. 1). The distance from the inner edge to inner edge of the LVOT was measured in a line parallel to the aortic annulus from the base of the right aortic valve coronary cusp to the base of the non-coronary cusp. The VTI was measured by obtaining an apical 5-chamber view and then placing a pulsed-wave Doppler cursor in the LVOT below the aortic valve annulus (Fig. 2). We measured the VTI, at the same time, in the respiratory cycle, ideally at the end of expiration.
The Doppler signal was traced using cardiac software to calculate the VTI, and an average of three measurements was used. The HR was calculated using the ultrasound cardiac software and not by physical examination or telemetry.
The PAC-CO was performed using a 7-French balloon-tipped standard four-lumen PAC model 131HF7 (Edwards Lifesciences Corp, Irvine, CA, USA) connected to a cardiac output monitor LCD medical display-model MOLVL 150-05 (General Electrics, Milwaukee, Wisconsin). PAC-CO measurements were obtained by injecting 10 mL of cold 0.9% saline throughout the respiratory cycle. The CO was measured three times and the results were averaged .
All the PWD-CO and PAC-CO measurements were obtained within a maximum of 1 h. The intensivists obtaining the thermodilution results (PAC-CO) were blinded to the PWD-CO measurements and vice versa.
First, a descriptive analysis was performed. Normal distribution of the study variables was confirmed using the Kolmogorov–Smirnov test. Discrete variables were expressed as counts and percentages, and continuous variables were expressed as means with standard deviations (SD) or as medians with interquartile ranges (25th–75th percentile). Differences between groups were assessed using a Chi-squared test or Fisher’s exact test, and Student’s t test or Mann–Whitney U test, as appropriate. A p < 0.05 was considered statistically significant.
The measurement of PAC-CO was considered to be the gold standard measurement for comparison. PWD-CO measurements were compared to the PAC-CO measurements for each individual time-point. Comparisons between these measurements were performed by the linear technique described by Bland and Altman . We defined a clinically acceptable level of agreement between the two techniques when the percentage of error was less than 30% as described by Critchley and Critchley . This cut-off is based on an assumption that a new device destined to monitor CO should have a similar level of precision as the gold standard technique, which in this case is the PAC-CO .
The mean differences between the two techniques (bias), the standard deviation (SD) and precision and percentage of error (PE), together with the 95% limits of agreement (LOA) were determined for both techniques. PE for agreement between the two techniques was calculated using the following equation:
The coefficient of variation (CV) and coefficient of error (CE) were also calculated for both techniques and between them.
The intra- and inter-observer variability was measured by the coefficient of intraclass correlation (CIC) and organized according to the Fleiss kappa scale (Fleiss index). A CIC greater > 0.6 was consider acceptable. Data were analyzed using the SPSS Statistics for windows version 15.0 (IBM corp. Armonk, NY, USA).
A total of 42 critically ill patients were assessed for enrolment in this study. Among them, 14 patients (33.3%) were excluded due to inability to acquire a high-quality image from the parasternal long-axis view to measure LVOTd or apical 5-chamber view to measure VTI. An additional eight patients (19%) were excluded due to atrial fibrillation (n = 5), aortic valve disease (n = 2), or technical difficulties in obtaining the PAC-CO measurement (n = 1).
Data were analyzed from 20 subjects [mean age 67 (± 14) years), 70% males]. Baseline characteristics of the study population are shown in Table 1. Briefly, the most common diagnosis for ICU admission was septic shock (45%). The majority of patients were receiving mechanical ventilation (90%) and vasopressor medications (80%).
Compared to included patients, the excluded patients had a faster heart rate and required higher norepinephrine doses. Variables associated with inability to acquire high-quality echocardiographic views were an abdominal wall dressing (p = 0.043) and high tidal volumes (p = 0.008) (Table 2).
PWD-CO was acquired successfully in 20 patients. To acquire the desired measurements with PWD-CO and PAC-CO, a mean of 54 (± 23) min elapsed to perform a complete examination, from setting up the ultrasound machine for the PWD-CO measurement to acquiring the PAC-CO measurement. For measurement of the PWD-CO alone, a mean of 12 (± 4) min elapsed. The mean LVOTd was 1.92 cm (± 0.13 cm) and the mean VTI was 20.85 cm (± 3.72 cm). The average PWD-CO was 5.22 L/min (± 1.17 L/min), which was less than the average PAC-CO of 6.26 L/min (± 1.96 L/min).
Pearson correlation index demonstrated a reasonable correlation between PWD-CO and PAC-CO measurements (r = 0.78, p < 0.0001) (Fig. 3). To compare CO by both techniques, a Bland–Altman analysis was performed and showed a bias of 1.03 L/min (± 1.27 L/min) with 95% limits of agreement ranging from − 1.50 to 3.56 L/min (Fig. 4). Less difference was seen between both techniques in patients with reduced cardiac output. In those patients with CO < 6.5 L/min, a bias of 0.46 L/min (± 0.88 L/min) with 95% limits of agreement of − 1.29 to 2.22 L/min was found.
The bias, precision, level of agreement, percentage of error, coefficient of variation, and coefficient of error are listed in Table 3. The mean PE between PWD-CO and PAC-CO was 17%. In one patient, the mean PE was higher than 30%. In this case, cardiac rate was normal with a high stroke volume and we could not explain the reason for this outlier.
Finally, we found an excellent intra-observer and a good inter-observer agreement between the LVOTd and VTI measurements using the Fleiss kappa scale. Detailed results are shown in Table 4.
In this study, we found an acceptable agreement of CO measured by CCE vs. PAC with thermodilution, and the inter- and intra-observer reliability was high. These findings suggest that CO can be accurately measured in critically ill patients by intensivists with the basic CCE training. However, it is important to recognize that high-quality transthoracic images to calculate CO could only be obtained in about half of eligible patients.
Although studies since the 1980s have shown that PWD measurements can accurately determine CO [4,5,6,7,8,9], a few studies have compared PWD-CO vs. PAC-CO in non-selected, critically ill patients. A recently systematic review of cardiac output measurements by echocardiography vs. thermodilution  concluded that the two techniques are not interchangeable. Twenty-four studies of critically ill and non-critically ill patients were included and both transesophageal and transthoracic echocardiography were used in these studies. None of the studies assessed inter- and intra-observer variability. Important limitations of the studies in this systematic review were small sample sizes, heterogeneity, and inadequate statistical analyses.
To our knowledge, one study that compared the use of PWD-CO vs. PAC-CO in critically ill patients found high accuracy and precision between the two techniques . Although the design of this study is comparable to our study, the PWC-CO measurements were obtained by intensivists with extensive experience in CCE.
Similar to previously published studies, our bias analysis showed a systematic underestimation of CO by PDW compared to thermodilution by PAC . This discrepancy was more notable in patients with high cardiac outputs (Fig. 4), probably related to the influence of high flow velocities and turbulent flow over the PWD signal, variability of the VTI angle , physiologic fluctuations in stroke volume, and size of the aortic valve orifice .
Strengths and limitations
Our study demonstrated that intensivists with basic CCE training can assess cardiac output in an unselected population of critically ill patients with an acceptable level of agreement between the PWD-CO and PAC-CO measurements. Although isolated CO values should be interpreted with caution, our findings indicate that PWD-CO measurements were accurate over a wide range of cardiac outputs, showing an even stronger correlation in patients with a cardiac output < 6.5 L/min, which can have important implications for the management of vasopressors and fluid therapy.
Additionally, our study is one of the few studies that assessed the inter- and intra-observer variability, and reported the challenges of acquiring high-quality transthoracic images by intensivists with basic CCE training. The intra-observer agreement was excellent and inter-observer agreement was good for ultrasound measurements of LVOT diameter, VTI, and CO. The coefficients of intraclass correlation were acceptable and similar to values described in the literature , suggesting that serial measurements, even if performed by different observers with basic training, can be sufficiently reproducible in clinical practice. We also found a significant association between abdominal wall dressings and poor-quality images.
Our study has several limitations. First, the total number of subjects from whom data was analyzed was small (n = 20). Approximately half of the patients were excluded due to difficulty in acquiring high-quality images. This limitation of our study is similar to the other studies [28, 29] where high-quality images were not acquired due to use of mechanical ventilation and high levels of PEEP. Furthermore, use of PACs for hemodynamic monitoring has been progressively decreasing in our intensive care unit given the availability of non-invasive methods to measure CO. Thus, use of a PAC was left to the discretion of the attending physician when another less invasive methods of monitoring CO could not be utilized.
Another limitation of our study is the time required to acquire the CO measurements, which averaged close to an hour for a complete examination [mean 54 (± 23) min]. Although this amount of time would be impractical in clinical practice, it is important to note that several measurements were obtained to follow our research study protocol. Most important, the mean time to acquire only the PWD-CO was 12 min, which is realistic to perform in clinical practice. The time and accuracy of these measurements could potentially be improved if acquired by experienced intensivists or cardiac sonographers.
Finally, limited experience of the intensivists in our study was likely an important factor that reduced the accuracy of the PWD-CO measurements. This limited experience is probably due in part to the fact that standards for CCE education currently vary by country, and there is no widely accepted consensus on the training of intensivists , despite the recommendations of professional societies to define competencies for basic and advanced training levels [10, 11].
Although an acceptable level of agreement was achieved between CO measured by CCE vs. PAC, the effect of individual or serial measurements of CO on clinical outcomes in critically ill patients is unknown. A recent study found a moderate level of agreement in the hemodynamic assessments performed using transpulmonary thermodilution (TPT) vs. CCE in ventilated patients with septic shock. However, there was no impact in mortality or lactate clearance . Future studies should explore the impact of assessing CO by CCE on mortality and other important clinical outcomes.
In conclusion, our findings demonstrate that intensivists with basic critical care echocardiography training can accurately and reliably measure CO in critically ill patients compared to gold standard measurements using a pulmonary artery catheter. However, an important limitation is the inability to obtain high-quality transthoracic images to calculate CO in approximately half of eligible patients.
pulmonary artery catheter
critical care echocardiography
pulsed-wave Doppler ultrasound
left-ventricular outflow tract diameter
left-ventricular outflow tract velocity time integral
central venous pressure
mean arterial blood pressure
- APACHE II:
Acute Physiology and Chronic Health Evaluation II score
Sequential Organ-Failure Assessment score
positive end-expiratory pressure
limits of agreement
coefficient of variation
coefficient of error
coefficient of intraclass correlation
Connors AF Jr, Speroff T, Dawson NV (1996) The effectiveness of right heart catheterization in the initial care of critically ill patients. JAMA 276(11):889–897
Cecconi M, De Backer D, Antonelli M, Beale R, Bakker J, Hofer C, Jaeschke R, Mebazaa A, Pinsky MR, Teboul JL, Vincent JL, Rhodes A (2014) Consensus on circulatory shock and hemodynamic monitoring. Task force of the European Society of Intensive Care Medicine. Intensive Care Med 40(12):1795–1815
Ihlen H, Amlie JP, Dale J (1984) Determination of cardiac output by Doppler echocardiography. Br Heart J 51(1):54–60
McLean AS, Needham A, Stewart D, Parkin R (1997) Estimation of cardiac output by noninvasive echocardiographic techniques in the critically ill subject. Anaesth Intensive Care 25(3):250–254
Axler O, Megarbane B, Lentschener C (2003) Comparison of cardiac output measured with echocardiographic volumes and aortic Doppler methods during mechanical ventilation. Intensive Care Med 29(1):208–217
Evangelista A, Garcia-Dorado D, Del Castillo H (1995) Cardiac index quantification by Doppler ultrasound in patients without left ventricular outflow tract abnormalities. J Am Coll Cardiol 25(3):710–716
Schuster AH, Nanda NC (1984) Doppler echocardiographic measurement of cardiac output: comparison with a non-golden standard. Am J Cardiol 53(1):257–259
Mayer SA, Sherman D, Fink ME (1995) Noninvasive monitoring of cardiac output by Doppler echocardiography in patients treated with volume expansion after subarachnoid hemorrhage. Crit Care Med 23(9):1470–1474
Vignon P, Mentec H, Terre S (1994) Diagnostic accuracy and therapeutic impact of transthoracic and transesophageal echocardiography in mechanically ventilated patients in the ICU. Chest 106(6):1829–1834
Mayo PH, Beaulieu Y, Doelken P (2009) American College of Chest Physicians/La Société de Réanimation de Langue Française statement on competence in critical care ultrasonography. Chest 135(4):1050–1060
Expert Round Table on Ultrasound in ICU (2011) International expert statement on standards for critical care ultrasonography. Intensive Care Med 37(7):1077–1083
Orde S, Slama M, Hilton A, Yastreboy K, McLean A (2017) Pearls and pitfalls in comprehensive critical care echocardiography. Crit Care 21(1):279
Price S, Via G, Sloth E, Guarracino F, Breitkreutz R, Catena E, Talmor D, World Interactive Network Focused On Critical UltraSound ECHO-ICU Group (2008) Echocardiography practice, training and accreditation in the intensive care: document for the World Interactive Network Focused on Critical Ultrasound (WINFOCUS). Cardiovasc Ultrasound 6:49
Knaus WA, Draper EA, Wagner DP, Zimmerman JE (1985) APACHE II: a severity of disease classification system. Crit Care Med 13(10):818–829
Vincent JL, de Mendonca A, Cantraine F (1998) Use of the SOFA score to assess the incidence of organ dysfunction/failure in intensive care units: results of a multicenter, prospective study. Crit Care Med 26(11):1793–1800
Dinh VA, Ko HS, Rao R, Bansal RC, Smith DD, Kim TE, Nguyen HB (2012) Measuring cardiac index with a focused cardiac ultrasound examination in the ED. Am J Emerg Med 30(9):1845–1851
Armstrong W, Ryan T (2000) Feigenbaum’s echocardiography, 7th edn. Lippincott Williams & Wilkins, Philadelphia
Jansen JRC, Schreuder JJ, Bogaard JM, Van Rooyen W, Versprille A (1981) Thermodilution technique for measurement of cardiac output during artificial ventilation. J Appl Physiol Respir Environ Exerc Physiol 51(3):584–591
Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1(8476):307–310
Critchley LA, Critchley JA (1999) A meta-analysis of studies using bias and precision statistics to compare cardiac output measurement techniques. J Clin Monit Comput 15(2):85–91
Taylor RW, Clavin JE, Matuschat GM (1997) Pulmonary artery catheter consensus conference: the first step. Crit Care Med 25(12):910–925
Wetterslev M, Møller-Sørensen H, Johansen RR, Perner A (2016) Systematic review of cardiac output measurements by echocardiography vs. thermodilution: the techniques are not interchangeable. Intensive Care Med 42(8):1223–1233
Mercado P, Maize J, Beyls C, Titeca-Beauport D, Joris M, Kontar L, Riviere A, Bonef O, Soupison T, Tribouilloy C, de Cagny B, Slama M (2017) Transthoracic echocardiography: an accurate and precise method for estimating cardiac output in the critically ill patient. Crit Care 21(1):136
Valtier B, Cholley BP, Belot JP (1998) Noninvasive monitoring of cardiac output in critically ill patients using transesophageal Doppler. Am J Respir Crit Care Med 158:77–83
Espersen K, Jensen EW, Rosenborg D (1995) Comparison of cardiac output measurement techniques: thermodilution, Doppler, CO2-rebreathing and the direct Fick method. Acta Anaesthesiol Scand 39(2):245–251
Fisher DC, Sahn DJ, Friedman MJ, Larson D, Valdes-Cruz LM, Horowitz S, Goldberg SJ, Allen HD (1983) The mitral valve orifice method for non invasive two-dimensional echo Doppler determinations of cardiac output. Circulation 67(4):872–877
Prieto L, Lamarca R, Casado A (1998) Assessment of the reliability of clinical findings: the intraclass correlation coefficient. Med Clín 110(4):142–145
Boussuges A, Blanc P, Molenat F (2002) Evaluation of left ventricular filling pressure by transthoracic Doppler echocardiography in the intensive care unit. Crit Care Med 30(2):362–367
Nagueh SF, Kopelen HA, Zoghbi WA (1995) Feasibility and accuracy of Doppler echocardiographic estimation of pulmonary artery occlusive pressure in the intensive care unit. Am J Cardiol 75(17):1256–1262
Labbé V, Ederhy S, Pasquet B, Miguel-Montanes R, Rafat C, Hajage D, Gaudry S, Dreyfuss D, Cohen A, Fartoukh M, Ricard JD (2016) Can we improve transthoracic echocardiography training in non-cardiologist residents? Experience of two training programs in the intensive care unit. Ann Intensive Care 6(1):44
Vignon P, Begot E, Mari A, Silva S, Chimot L, Delour P, Vargas F, Filloux B, Vandroux D, Jabot J, François B, Pichon N, Clavel M, Levy B, Slama M, Riupoulene B (2018) Hemodynamic assessment of patients with septic shock using transpulmonary thermodilution and critical care echocardiography: a comparative study. Chest 153(1):55–64
All authors contributed to study conception and design, data analysis, and drafting the manuscript. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Availability of data and materials
The authors added the database of the present study as Additional file 1.
Consent for publication
It was obtained a consent for participation and for publication.
Ethics approval and consent to participate
The study was approved by the Ethics and Clinical Research Committee. It was obtained an informed consent for all patients.
There was no funding to support this study.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.