Participants
Participants included 56 ultrasound-naive first, second, or third-year medical students who had not yet started their clinical rotations. Fourth-year students were excluded because of the possibility of encountering ultrasound within their clinical rotations. Students from three medical schools in the area were recruited via email. Participants were not required to participate and were not compensated for their time. Participants who had experience with the tested exams were excluded, as assessed by a pre-assessment questionnaire. Significant ultrasound experience was also a disqualifying factor, which was defined as over 20 h of lifetime ultrasound experience. The participants were subsequently randomized into traditional in-person (“Traditional”) or tele-ultrasound groups (“Tele-ultrasound”) based on the order of arrival to the simulation center.
Lumify Ultrasound system
The Lumify Ultrasound system is a portable application-based system that incorporates an ultrasound probe, specially designed to directly connect to a tablet or smart-phone. The S4-1 broadband sector array transducer was utilized for the FAST exam (bandwidth of 4–1 MHz, a scan depth of up to 24 cm, a footprint of 20.2 mm, and imaging features including 2D, color Doppler, and M-mode). For the LEDVT screen and ultrasound-guided vascular access, the L12-4 Linear array transducer (Bandwidth 12–4 MHz, a scan depth of up to 12 cm, a footprint of 34 mm, and imaging features including 2D, steerable color Doppler, M-mode) was utilized. Using the Philips Lumify app, the system allows for “video calls” to another smart device over wireless internet signal. This “call” allows for face-to-face interaction between the operator and educator, allowing the remote instructor to see the positioning of the transducer as well as the generated ultrasound images in real-time. The software additionally includes a pointer that can be utilized by the remote instructor to highlight relevant structures on the ultrasound image.
Examination techniques
The FAST exam is a bedside examination technique predominantly used in trauma settings to detect the presence of pathologic free fluid, notably allowing for rapid detection of hemoperitoneum in order to initiate and coordinate appropriate interventions. The exam comprised four windows: left upper quadrant, right upper quadrant, pelvic and pericardial windows [8].
The lower extremity deep vein thrombosis (LEDVT) evaluation is important diagnostic tool used for assessment of thrombosis of the common femoral, femoral and popliteal veins. Participants were taught the proximal lower extremity compression technique which begins at the common femoral vein and ends at the trifurcation of the popliteal vein. The comprehensive exam, encompassing continuous compressions every 2.5 cm, was utilized as it is a widely accepted methodology. This method has been previously demonstrated to be a safe, reliable and sensitive method of evaluating for LEDVT [9,10,11,12].
Ultrasound-guided vascular access facilitates visualization of the both vessel and needle which is particularly helpful for patients with difficult vasculature with conventional techniques. Ultrasound-guided vascular access was taught in both transverse and longitudinal views, both of which are commonly utilized in clinical practice [13]. A branched 4-vessel ultrasound training phantom (CAE Blue Phantom, Sarasota, FL) was utilized for this exercise.
Study design
This study was approved by the MetroHealth Medical Center’s Internal Review Board and allowed for verbal consent for participants. Participants who volunteered and met inclusion criteria were then given access to assigned online modules that educated them on basic ultrasound physics and instrumentation as well as each exam that was to be taught. These modules were created by the head of Ultrasound Education at Case Western Reserve University School of Medicine, Dr. Robert Jones. They were presented on emsono.com, an on-demand ultrasound learning and education website that is utilized by many EM residencies to teach ultrasound principles and methods. Modules walked students through important components of the exams with accompanying videos and questions to prepare the students in advance for the study. There was no difference between the modules assigned to the Traditional and Tele-ultrasound groups. Modules included practical scanning, the EFAST exam-hemoperitoneum, vascular access, and lower extremity DVT ultrasound. The total time to complete these modules was approximately 3 h.
Once completed, the participants were instructed to convene on certain dates at pre-specified locations (ex. simulation center at a nearby hospital) and were then randomly assigned to either Traditional or Tele-ultrasound Groups in an alternating fashion based on the order in which they arrived. Both groups received a pre-assessment questionnaire to fill out. This questionnaire was used to assess prior ultrasound experience and expectations of the study. A full breakdown of the questionnaire can be found in Additional file 1: Appendix S1.
The participants were taught the three exams as indicated above. Both groups utilized a 4 participants:1 teacher ratio. The teaching portion of the exam lasted approximately 2 h, with time being divided among the three skills. During this time, all participants were able to practice with the probe and be walked through each exam by the instructor. Students in both groups were given feedback on their hand and probe placement throughout the training period, as well as feedback on what structures needed to be visualized on exam. As these students were considered ultrasound naïve, importance was placed on visualizing and recognizing the structure, with less emphasis on the image quality and ability to operate the ultrasound machine. For a detailed guide on what structures and procedures were considered pertinent for each exam, refer to Additional file 1: Appendix S2.
In the Tele-ultrasound group, students were isolated from the instructor, who was in a separate room within the simulation center. The instructor was kept consistent for all tele-ultrasound participants, and was experienced with the Lumify system. In the Traditional group, students were paired with an experienced sonographer or physician, of which two were utilized for different groups depending on their respective availability. One of three student investigators was utilized as a standardized patient for each exam, with attempts to keep the student-models consistent between the training and exam sessions. The primary outcome of interest at was the ability to perform certain tasks for each exam, a complete breakdown of which can be found in Additional file 1: Appendix S2.
After the teaching portion of the session was complete, the assessment portion began. During this portion, participants rotated between three stations in which they were assessed on the skills they had previously learned. Participants were graded on a pass/fail basis on whether they were able to perform the specified skill. No teaching was done with the participants during the assessment period, and no assistance was given. Medical student investigators were trained by POCUS faculty on the tested exam prior to the session to ensure the student investigators were able to properly assess the participants. The medical student investigators then assessed the participants based on a specified checklist, with the investigators remaining consistent at each station between participants and sessions (i.e., one student investigator assessed all DVT exams performed by participants). This was done to ensure that standards were maintained. The assessment checklists (which can be found in Additional file 1: Appendix S2) mainly consisted of probe choice and important anatomical structures, which had to be visualized sufficiently in order to be considered a “pass”. As specified earlier, since participants with little to no ultrasound experience were recruited, the main goal of this study was to assess whether structures could be visualized and recognized. Participants were not assessed on their ability to calibrate the machine or optimize the image.
Statistical analysis
The purpose of this study was to show that the tele-ultrasound teaching method was as efficacious as traditional teaching, rather than proving it better or worse. For this reason, a non-inferiority study design was utilized. Outcomes were considered on a “meets” or “does not meet” basis on whether the student was able to perform each designated task for the exam as determined by the medical student investigators who were present. Using the cumulative binary data, means were calculated for each task of the exam. Using the binomial data, a Fisher’s exact test was performed to determine whether inferiority was present. Data were analyzed using STATA v 14.0 (College Station, TX).
Demographic data were collected to ensure adequate randomization between groups. Pre- and Post-assessment surveys were utilized in order to gauge the experience of participants, as well as their subjective opinion on the different systems. Within the pre-assessment, first overall ultrasound experience was assessed, as this was an important excluding factor. Demographic data were collected, and then each participant answered 3 questions relating to their confidence in ability to perform each of the three exams prior to the teaching component of the study. For the post-assessment, both groups had six questions that inquired as to how they felt about their learning experience. The Tele-ultrasound group had two additional questions specifically on the Lumify units and their opinion of remote learning. These questionnaires can be found in Additional file 1: Appendix S3 (Tele-ultrasound) and 4 (Traditional). All surveys utilized the Likert scale, with rankings from 1 (strongly disagree) to 5 (strongly agree). Significant difference between Traditional and Tele-ultrasound groups was calculated using a Mann–Whitney U test.