This study took place at a freestanding tertiary care children’s hospital with approximately 90,000 emergency department (ED) visits per year and with a cardiology service that performs 21,000 echocardiograms a year. PEM fellows completed FOCUS training, consisting of a 1 h lecture on echocardiography, a month of basic POCUS training with regular supervised ultrasound scans, and quality assurance sessions with PEM POCUS-trained faculty. In addition, each PEM fellow competed a minimum of 10 FOCUS exams reviewed by PEM POCUS-trained faculty. The fellows were taught to determine function by qualitatively evaluating how closely the anterior leaflet of the mitral value approached the interventricular septum in the parasternal long axis view. They were taught to evaluate pericardial effusion in all four views. FOCUS exams included four standard cardiac views: the parasternal long, parasternal short, apical 4 chamber, and subxiphoid views. Specifically in the parasternal long axis view, they were taught to note any anechoic structures above the descending aorta, within the pericardium.
A convenience sample of patients’ ages 1 month to 21 years, with an indication for an echocardiogram as determined by a pediatric cardiologist, were enrolled between April 2014 and June 2015. The patients were enrolled in the pediatric ED, cardiology clinic, or the inpatient service. The indications for the echocardiograms included chest pain, syncope, altered mental status, hypoxia, and follow-up of known cardiac disease. Premature infants with post-conception age less than 36 weeks and patients who were less than 8 day post-sternotomy were excluded. Because the cardiology echocardiogram was the reference standard, we also excluded patients if they experienced significant therapeutic interventions or changes in clinical conditions between the FOCUS exam and the cardiology echocardiogram. This included progression of illness requiring transfer of the patient to a higher level of care, need for emergent surgery, initiation of ECMO, intubation, or the initiation of intravenous vasopressor agents.
Eight PEM fellows performed FOCUS, using a Sonosite M-Turbo® ultrasound system (Bothell, WA). Within 24 h of FOCUS, a cardiologist or echocardiography technician performed an echocardiogram with a Phillips IE 33 ultrasound system (Andover, MA). PEM fellows were blinded to the results of the cardiology echocardiogram and vice versa. The PEM fellows were blinded to the clinical status of the patient to the extent possible; the PEM fellow performing the FOCUS did not access the medical record or discuss the clinical case with the patient or with the provider caring for the patient. However, we could not blind the fact that a patient was in the resuscitation bay or the ICU, for example.
The PEM fellows performing FOCUS completed a case report form (CRF) immediately following the performance of the FOCUS. The CRF included the fellows’ interpretation of the presence or absence of significant pericardial effusion and the qualitative global systolic function as either normal or depressed. We used dichotomous qualitative variables for pericardial effusion (absent/trivial versus present) and cardiac function (normal versus depressed) rather than attempting to quantify amount of fluid or ejection fraction.
A board-certified pediatric cardiologist, blinded to the patient’s clinical presentation, chief complaint, cardiac history and PEM fellow interpretation of FOCUS, reviewed and interpreted the recorded clips of each FOCUS and cardiology echocardiogram. This cardiologist used a CRF with a similar format to the PEM fellows to subjectively note the presence or absence of significant effusion, the qualitative global function and if the quality of the study was adequate to perform these assessments. The blinded interpretation of the cardiology-performed echocardiogram was used as the reference standard for subsequent analysis.
In the second phase of the study, PEM fellow interpretation skills were assessed remotely in a classroom setting. Thirty matched studies were compiled: 15 FOCUS exams performed by PEM fellows and 15 cardiology-acquired echocardiograms on the same patients. We selected studies, so that 5 of these sets had pericardial effusions, 5 had diminished function, and 5 were normal. Eight PEM fellows were shown these studies in random order and were asked to comment on the presence or absence of pericardial effusion and global cardiac function. PEM fellows were not given any clinical information about the patients. Only the four standard views were presented. Only one of these eight PEM fellows had acquired images in the first phase of the study.
IBM SPSS version 24 (Armonk NY) and Microsoft Excel (Microsoft, Inc, Redmond, WA) were used to calculate test characteristics of PEM fellow FOCUS exams, with cardiology echocardiograms as the reference standard. Similar methodology was used to calculate test characteristics of PEM fellows’ interpretation of both FOCUS and cardiology studies in the remote setting.