Comparison of four handheld point-of-care ultrasound devices by expert users
The Ultrasound Journal volume 14, Article number: 27 (2022)
Point-of-care ultrasound (POCUS) is rapidly becoming ubiquitous across healthcare specialties. This is due to several factors including its portability, immediacy of results to guide clinical decision-making, and lack of radiation exposure to patients. The recent growth of handheld ultrasound devices has improved access to ultrasound for many clinicians. Few studies have directly compared different handheld ultrasound devices among themselves or to cart-based ultrasound machines. We conducted a prospective observational study comparing four common handheld ultrasound devices for ease of use, image quality, and overall satisfaction. Twenty-four POCUS experts utilized four handheld devices (Butterfly iQ+™ by Butterfly Network Inc., Kosmos™ by EchoNous, Vscan Air™ by General Electric, and Lumify™ by Philips Healthcare) to obtain three ultrasound views on the same standardized patients using high- and low-frequency probes.
Data were collected from 24 POCUS experts using all 4 handheld devices. No single ultrasound device was superior in all categories. For overall ease of use, the Vscan Air™ was rated highest, followed by the Lumify™. For overall image quality, Lumify™ was rated highest, followed by Kosmos™. The Lumify™ device was rated highest for overall satisfaction, while the Vscan Air™ was rated as the most likely to be purchased personally and carried in one’s coat pocket. The top 5 characteristics of handheld ultrasound devices rated as being “very important” were image quality, ease of use, portability, total costs, and availability of different probes.
In a comparison of four common handheld ultrasound devices in the United States, no single handheld ultrasound device was perceived to have all desired characteristics. POCUS experts rated the Lumify™ highest for image quality and Vscan Air™ highest for ease of use. Overall satisfaction was highest with the Lumify™ device, while the most likely to be purchased as a pocket device was the Vscan Air™. Image quality was felt to be the most important characteristic in evaluating handheld ultrasound devices.
Point-of-care ultrasound (POCUS), or use of bedside ultrasound by a clinician to answer a specific diagnostic question or guide performance of an invasive procedure, is becoming more common across healthcare specialties. Historically, POCUS has been performed using portable cart-based ultrasound machines that offer a wide range of modalities and consistently generate high-quality images, but access to portable ultrasound machines has been a top barrier to POCUS use [1,2,3,4,5,6]. In recent years, ultraportable handheld ultrasound devices have emerged improving access to POCUS technology at a fraction of the cost, especially in resource-limited settings .
Handheld ultrasound devices have demonstrated similar accuracy compared to cart-based ultrasound machines for multiple applications including bedside procedures, such as thoracentesis and epidural analgesia, and diagnostic evaluation of left ventricular function, female reproductive organs, abdominal pathologies (ascites, hydronephrosis, abdominal aortic aneurysm), musculoskeletal system, and lungs/pleura [8,9,10,11,12,13,14,15,16]. Discrepant findings between handheld ultrasound devices and cart-based ultrasound machines were not clinically significant in these studies [17, 18]. In contrast, few studies have performed head-to-head comparisons of different handheld ultrasound devices. Though a simple comparison of available modes, settings, and probes of different handheld devices can be performed relatively easily , POCUS users are seeking comparative data on the performance of handheld ultrasound devices to guide purchasing decisions.
We compared four commonly available handheld ultrasound devices in the United States for ease of use, image quality, and overall satisfaction with a multidisciplinary group of POCUS experts as operators. Our secondary objective was to identify the characteristics that POCUS experts consider to be most important when comparing different handheld ultrasound devices for use in clinical practice.
Subjects and setting
We conducted a prospective observational study in December of 2021 during a 2-day in-person POCUS continuing medical education course. Twenty-four expert POCUS faculty specializing in emergency, critical care, hospital, pediatrics, and pulmonary medicine acquired three standard POCUS views using four commercially available handheld ultrasound devices on the same three standardized patients with body mass index < 24. The University of Texas Health San Antonio Investigational Review Board deemed this educational study to be non-regulated research.
Four handheld ultrasound devices with both low- and high-frequency transducer capabilities were compared: Butterfly iQ + ™ (Butterfly Network, Inc.) probe connected by a USB-C cable to an Apple iPhone™ (iPhone 11 Pro Max™); Kosmos™ (EchoNous, Inc.) probe connected by a cord to a proprietary tablet as one unit; Lumify™ (Philips Healthcare) connected by a USB-C cable to an Apple iPad™ (8th generation), and Vscan Air™ (GE Healthcare) connected wirelessly to a Samsung Galaxy S7™ tablet.
Each standardized patient was assigned to one of three POCUS views: Focused Assessment with Sonography in Trauma (FAST) right upper quadrant view, transverse view of the neck with the internal jugular vein and carotid artery, or parasternal long-axis view of the heart. Using the four handheld devices, all 24 POCUS experts independently acquired the same view on the same model. For the parasternal long-axis view, experts were instructed to use the low-frequency probe with a cardiac preset and focus on the mitral valve, aortic valve, and endocardial lining. Color flow Doppler was then applied over the mitral and aortic valves. For the right upper quadrant FAST view, experts were instructed to use the low-frequency probe with an abdominal preset and focus on the liver, kidney, and diaphragm. For the transverse view of the neck, experts were instructed to use the high-frequency probe with a venous or vascular preset and focus on the internal jugular vein and common carotid artery.
Data were collected on ease of use, image quality, and overall satisfaction (Additional file 1). For ease of use, experts rated the physical characteristics, software navigation, maneuverability of the probe and tablet for imaging, and overall satisfaction. For image quality, experts rated the detail resolution, contrast resolution, penetration, clutter, and overall satisfaction. The overall ranking assessed satisfaction and recommendation for purchase. Ratings were made using standardized statements on a Likert scale of 1 (“strongly disagree” or “very dissatisfied”) to 5 (“strongly agree” or “very satisfied”). Qualitative feedback was collected in each category using free text. Experts completed the data collection form after scanning each model (< 72 h). All data were captured electronically using REDCap™ (Vanderbilt University, Nashville, TN, USA).
Descriptive statistics were reported as frequencies with percentage for categorical variables and medians with interquartile range (IQR) for continuous data. Differences between medians were assessed by Kruskal–Wallis H test. Post hoc analysis was performed with Dunn’s test using a Bonferroni correction for multiple comparisons of groups. The rank analysis was performed via Friedman’s test followed by a post hoc Sign test for paired data. Spearman correlation coefficients were calculated to evaluate the correlation between experts’ prior experience with using each handheld device and ratings for ease-of-use, image quality, and overall satisfaction. A p value < 0.05 denoted statistical significance. All analyses were performed with R statistical software version 4.0.2.
Free text participant responses were analyzed using a qualitative deductive and inductive coding process aligned with a framework method approach . The framework for deductive codes included advantages and disadvantages of all four handheld devices. We allowed for new codes that inductively arose from the data. Two investigators (AMM and JD) began the analysis by immersing in the data and developing the initial coding framework while coding free text responses of two surveys together. They then independently applied the coding framework to a subset of transcripts, reconvening multiple times to identify differences in coding, resolve them through discussion and refine the coding framework based on that discussion.
Twenty-four POCUS experts specializing in internal medicine/hospital medicine, emergency medicine, critical care medicine, pulmonary medicine, and pediatrics participated in this study. Most experts (71%) had completed a dedicated POCUS certification or fellowship, and most had been practicing medicine and using POCUS for > 5 years (Table 1).
Ratings of the handheld ultrasound devices’ ease of use and image quality are displayed in Table 2. For ease of use, handheld devices were rated on physical characteristics, software navigability, maneuverability, and overall ease of use. Vscan Air™ was rated highest for physical characteristics and maneuverability, while Butterfly iQ + ™ was rated highest for software navigability. Overall ease of use was highest with the Vscan Air™.
For image quality, the handheld devices were rated on detail resolution, contrast resolution, penetration, clutter, and overall satisfaction with image quality. Lumify™ was rated highest for detail resolution, contrast resolution, and clutter, while Kosmos™ was rated highest for penetration. Overall image quality was highest for Lumify™. The mean ratings for ease of use and image quality are displayed in Fig. 1. Examples of the parasternal long-axis view captured in early systole using the four handheld ultrasound devices are shown in Fig. 2.
In the final section, experts were asked about their overall satisfaction with each handheld device. The Lumify™ and Vscan Air™ received the highest number of “satisfied” responses (Fig. 3A). Furthermore, experts ranked the four devices in order from 1 (“best”) to 4 (“worst”) and the Lumify™ and Vscan Air™ received the highest rankings (Fig. 3B). When experts were asked if they would purchase or recommend purchase of a specific handheld, a “yes” response was received most often for Lumify™ followed by Vscan Air™. However, when asked which device experts would “buy today to carry in their coat pocket,” the Vscan Air™ was most frequently selected (Fig. 3C).
Experts were asked about the most important characteristics of handheld devices (Additional file 3: Table S2). The top five most important characteristics were image quality, ease of use, portability, total costs, and availability of different probes. We evaluated for potential bias due to prior exposure to a device and found no association between experts’ experience levels and their ratings for a device’s ease-of-use or image quality (Additional file 2: Table S1). There also was no association between experience and overall satisfaction for three of the devices. However, a weak negative association was identified for the Vscan Air™, indicating that experts with some experience with the device were less satisfied with it than experts with no experience. None of the experts had extensive experience with the Vscan Air™.
From the qualitative data, two main themes emerged. First, there is not yet a perfect handheld device. Experts perceived all four devices as having notable advantages and disadvantages. Experts valued high image quality, the convenience of having only one probe, intuitive user interfaces, and advanced features, like Doppler ultrasound and artificial intelligence (AI). Ultimately, no single device was perceived as having all desired qualities or features. Second, adequate image quality was considered the most important aspect of a handheld device because it determined the user’s ability to make a clinical decision. Multiple experts commented that poor image quality limited clinical decisions and for this reason devices with poor image quality were considered unusable by some experts. One expert stated, “The poor image quality is a deal breaker,” and another said, “Image quality is poor and would result in me getting a larger machine to confidently make a clinical decision which is double the work in my mind.” Some experts also noted poor image quality may negatively affect the ability for inexperienced POCUS users to attain competency. One expert remarked, “This (device) is being marketed to physicians with minimal training, and if you take a minimally trained physician and give them a device that is very poor quality, then you have a recipe for disaster”.
We directly compared four common handheld ultrasound devices and found Lumify™ was rated highest for overall image quality, while Vscan Air™ was rated highest for overall ease of use. The overall ranking, satisfaction, and recommendation for purchase was highest with Lumify™ followed by Vscan Air™, and Vscan Air™ was most often selected for use as a personal pocket device. No single handheld ultrasound device was perceived to have all desired characteristics. We explored the characteristics of handheld ultrasound devices that are considered most important from the perspective of expert POCUS users, and image quality was felt to be the most important.
Handheld ultrasound devices offer unique advantages, including greater portability and ease of disinfection [7, 15, 21, 22], but few studies have directly compared different handheld ultrasound devices. One study compared three different handheld devices with the same operator performing gynecological ultrasound exams . In contrast, our study has important advantages and adds substantively to our understanding of handheld ultrasound devices. First, we had POCUS experts use the same four handheld ultrasound devices on the same standardized patients to control for potential patient, device, and operator variables that could confound results. Second, POCUS experts were asked to use a variety of presets using both high- and low-frequencies for common diagnostic and procedural views. Third, our study included 24 POCUS experts as operators from 5 different specialties. Thus, our data may be more generalizable across common POCUS applications and different specialties. Our study sought to quantitatively assess ease of use and image quality based on user experience and provide a global rating of overall satisfaction and recommendation for purchase. We believe our study better replicates the myriad of considerations that clinicians face when making a purchasing decision.
Ultimately, no single handheld ultrasound device was rated highest in all categories. A lack of consensus of a single handheld ultrasound device being superior to others was similar to the findings by Toscano et al. in which investigators sought one suitable handheld device to perform gynecologic ultrasound exams in a resource-poor setting and ultimately selected the Lumify™ based on its ease of use, battery-life, portability, cost, and ease with depth and gain adjustments . Even though POCUS experts rated Lumify™ highest for “overall satisfaction” in our study, more experts selected Vscan Air™ as the device they would buy “today as a personal device to carry in my coat pocket”.
Our study sought to identify specific characteristics of handheld ultrasound devices that expert POCUS users considered important. Quantitative and qualitative data demonstrated image quality to be the most important characteristic. Comments from experts reflected that image quality is the most important characteristic because poor quality images require a repeat evaluation with a cart-based ultrasound machine, negating the benefits of having a handheld. After image quality, the most important characteristics of handheld devices were ease of use, portability, total costs, and availability of different probe types. Perhaps once image quality is adequate for clinical decision-making, these secondary characteristics, such as ease of use and software options, become more of a deciding factor in selecting a handheld ultrasound device. Some of the characteristics, such as portability and number probes, are inherently related. A deeper understanding of the characteristics that POCUS users consider important, as well as the variables that affect purchasing decisions, warrant further investigation and may guide manufacturers in product development.
We recognize our study has limitations. First, POCUS experts could not be blinded to the different handheld devices and completed their scanning of the 3 standardized patients in the same room. Despite the large size of the room, we could not prevent experts from sharing their thoughts about the devices which could have biased their ratings. Further, our POCUS experts were not provided with dedicated training on the different handheld ultrasound devices, and lack of training may have limited the experts’ ability to navigate the software and optimize image quality. Third, bias from prior experience with some of the devices may have been a factor in the experts’ evaluations; however, we did not find an association between experts’ prior experience and the overall satisfaction ratings for three devices (Additional file 2: Table S1). Although there was a weak association detected between the experts’ experience level and overall satisfaction for the Vscan Air™, the association was negative, indicating that familiarity with the device did not inflate the ratings from experts. Fourth, experts rated handheld devices against one another, but we did not assess whether each handheld device met a minimal acceptable standard for ease of use and image quality. Additionally, we did not evaluate all potential diagnostic applications, including lungs, first trimester pregnancy, bladder, and lower extremity deep venous thrombosis, and we did not evaluate the handheld devices for procedural applications. Finally, POCUS users will need to consider several factors that were not explored in our study, including total costs of device over its lifespan; compatibility with preferred smart phone or tablet; probe characteristics (ergonomics, overheating, wired versus wireless connectivity); battery life; quality assurance and image archiving options; and desire for remote teleguidance or artificial intelligence software. Thus, handheld ultrasound purchasing decisions are complex, involving individual user preferences and device features, as well as external factors, such as institutional device restrictions and purchasing contracts, which were not addressed in our study.
For clinicians, the implications of our findings are threefold. First, though Vscan Air™ was rated highest for overall ease-of-use and Lumify™ for overall image quality, a perfect handheld device that combines all desired features does not currently exist. Second, when rating the importance of sixteen characteristics of handheld ultrasound devices, image quality was the only characteristic rated by all twenty-four experts as being “very important,” and new users should consider giving image quality priority when evaluating devices. Third, we focused our comparison of handheld ultrasound devices using 2-dimensional imaging alone. However, handheld ultrasound technology is advancing rapidly with new features, including artificial intelligence to guide image acquisition and interpretation, image sharing capabilities for remote teaching and real-time image interpretation by off-site experts, and robotics to facilitate probe placement for diagnostic imaging and invasive procedures [16, 23]. Future studies will be needed to compare the advantages and disadvantages of these advanced technologies in handheld ultrasound devices.
As more handheld ultrasound devices become available to clinicians, understanding the advantages and disadvantages of different devices is imperative. In our comparison of four common handheld ultrasound devices, we demonstrated that overall image quality was best with Lumify™, while ease of use was best with Vscan Air™. Though most POCUS experts recommended Lumify™ for purchase, Vscan Air™ was most often selected as a personal pocket device. Thus, no single handheld ultrasound device was perceived to be superior in all categories. Future comparative studies of handheld ultrasound devices shall consider advanced technologies, such as artificial intelligence and tele-ultrasound, and their impact on training and patient outcomes.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Focused assessment with sonography in trauma
LoPresti CM, Boyd JS, Schott C et al (2019) A national needs assessment of point-of-care ultrasound training for hospitalists. Mayo Clin Proc 94(9):1910–1912
Micks T, Sue K, Rogers P (2016) Barriers to point-of-care ultrasound use in rural emergency departments. CJEM 18(6):475–479
Boyd JS, LoPresti CM, Core M et al (2019) Current use and training needs of point-of-care ultrasound in emergency departments: a national survey of VA hospitals. Am J Emerg Med 37(9):1794–1797
Buchanan MS, Backlund B, Liao MM et al (2014) Use of ultrasound guidance for central venous catheter placement: survey from the American board of emergency medicine longitudinal study of emergency physicians. Acad Emerg Med 21(4):416–421
Russell FM, Zakeri B, Herbert A, Ferre RM, Leiser A, Wallach PM (2021) The state of point-of-care ultrasound training in undergraduate medical education: findings from a national survey. Acad Med. https://doi.org/10.1097/ACM.0000000000004512
Maw A, Ortiz-Lopez C, Morris M et al (2020) Hospitalist perspectives of available tests to monitor volume status in patients with heart failure: a qualitative study. Cureus 12(6):e8844
Thavanathan RS, Woo MY, Hall G (2020) The future is in your hands—handheld ultrasound in the emergency department. CJEM 22(6):742–744
Newhouse SM, Effing TW, Dougherty BD, D’Costa JA, Rose AR (2020) Is bigger really better? Comparison of ultraportable handheld ultrasound with standard point-of-care ultrasound for evaluating safe site identification and image quality prior to pleurocentesis. Respiration 99(4):325–332
Carvalho B, Seligman KM, Weiniger CF (2019) The comparative accuracy of a handheld and console ultrasound device for neuraxial depth and landmark assessment. Int J Obstet Anesth 39:68–73
Jenkins S, Alabed S, Swift A et al (2021) Diagnostic accuracy of handheld cardiac ultrasound device for assessment of left ventricular structure and function: systematic review and meta-analysis. Heart 107(22):1826–1834
Zardi EM, Franceschetti E, Giorgi C, Palumbo A, Franceschi F (2019) Accuracy and performance of a new handheld ultrasound machine with wireless system. Sci Rep 9(1):14599
Toscano M, Szlachetka K, Whaley N, Thornburg LL (2020) Evaluating sensitivity and specificity of handheld point-of-care ultrasound testing for gynecologic pathology: a pilot study for use in low resource settings. BMC Med Imaging 20(1):121
Corte G, Bayat S, Tascilar K et al (2021) Performance of a handheld ultrasound device to assess articular and periarticular pathologies in patients with inflammatory arthritis. Diagnostics 11(7):1139
Jung EM, Dinkel J, Verloh N et al (2021) Wireless point-of-care ultrasound: first experiences with a new generation handheld device. Clin Hemorheol Microcirc 79(3):463–474
Maheshwarappa HM, Mishra S, Kulkarni AV, Gunaseelan V, Kanchi M (2021) Use of Handheld ultrasound device with artificial intelligence for evaluation of cardiorespiratory system in COVID-19. Indian J Crit Care Med 25(5):524–527
Rykkje A, Carlsen JF, Nielsen MB (2019) Hand-held ultrasound devices compared with high-end ultrasound systems: a systematic review. Diagnostics. https://doi.org/10.3390/diagnostics9020061
Falkowski AL, Jacobson JA, Freehill MT, Kalia V (2020) Hand-held portable versus conventional cart-based ultrasound in musculoskeletal imaging. Orthop J Sports Med 8(2):2325967119901017
Fröhlich E, Beller K, Muller R et al (2020) Point of care ultrasound in geriatric patients: prospective evaluation of a portable handheld ultrasound device. Ultraschall Med 41(3):308–316
Malik AN, Rowland J, Haber BD et al (2021) The use of handheld ultrasound devices in emergency medicine. Curr Emerg Hosp Med Rep. https://doi.org/10.1007/s40138-021-00229-6
Gale NK, Heath G, Cameron E, Rashid S, Redwood S (2013) Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 13:117
Haji-Hassan M, Lenghel LM, Bolboacă SD (2021) Hand-held ultrasound of the lung: a systematic review. Diagnostics. https://doi.org/10.3390/diagnostics11081381
Dadon Z, Levi N, Orlev A et al (2021) The utility of hand-held cardiac and lung ultrasound in predicting outcomes of hospitalized patients with COVID-19. Can J Cardiol. https://doi.org/10.1016/j.cjca.2021.11.016
Baribeau Y, Sharkey A, Chaudhary O et al (2020) Handheld point-of-care ultrasound probes: the new generation of POCUS. J Cardiothorac Vasc Anesth 34(11):3139–3145
The contents of this publication do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.
Nilam J. Soni receives funding from the U.S. Department of Veterans Affairs, Quality Enhancement Research Initiative (QUERI) Partnered Evaluation Initiative Grant (HX002263-01A1) and National Center for Patient Safety.
Ethics approval and consent to participate
It was determined that the project does not require IRB approval because it is not regulated research as defined by DHHS regulations at 45 CFR 46 and FDA regulations at 21 CFR 56. The proposed activity is not funded by DHHS as research; AND is not a systematic investigation to test a hypothesis and permit conclusions to be drawn; AND is not designed to develop or contribute to generalizable knowledge; AND the purpose is not to investigate the safety or effectiveness of a drug, medical device or biologic.
Consent for publication
All authors and editors consent to the publication of this data.
Echonous, Philips, General Electric, and Butterfly Network, Inc. did not have any involvement in the development of this study, data review and analysis, writing of the manuscript, or any approval or decision making in the submission of this manuscript. None of the authors are or were in any contractual agreement with Echonous, Philips, General Electric, or Butterfly Network, Inc. regarding the study equipment (other than purchasing or loan agreements). None of the authors serve as speakers or consultants for any of the companies whose devices were included in this study. The authors did not receive any payment, support, or benefits from these companies for the present study or in the past. The Kosmos™ (Echonous), Lumify™ (Phillips), and Vscan Air™ (General Electric) ultrasound equipment were temporarily loaned from their respective companies at the request of one of our authors (NS). The Butterfly IQ+™ (Butterfly Network, Inc.) equipment was previously purchased by one of our authors (GJ) for personal use and temporarily loaned for this study. One of the authors (HS) has received loaned equipment with shipping from Butterfly Network, Inc. for an unrelated investigator-initiated study.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Comparison of handheld point-of-care ultrasound devices.
: Table S1. Individual expert’s experience with devices compared to ratings for overall satisfaction, image quality, and ease-of-use.
: Table S2. Importance of characteristics for a handheld ultrasound device.
About this article
Cite this article
Le, MP.T., Voigt, L., Nathanson, R. et al. Comparison of four handheld point-of-care ultrasound devices by expert users. Ultrasound J 14, 27 (2022). https://doi.org/10.1186/s13089-022-00274-6