- Open Access
Rules for the road: an evidence-based approach to understanding diagnostic test performance of point-of-care ultrasound for pediatric abdominal emergencies
Critical Ultrasound Journalvolume 1, pages101–103 (2010)
I like to think there are many similarities between ultrasound machines and automobiles. Just as cars come in all shapes, sizes, functions (sports-car, minivans, pickup trucks, etc.), and price ranges, so do ultrasound machines—from the newer handheld ones now entering the market, laptop-sized ultrasound machines (which has allowed point-of-care ultrasound to rapidly diffuse into many areas of medicine), and to console-type ultrasound machines used by radiology departments . Learning (and teaching someone) how to use an ultrasound machine is not so different from learning (and teaching someone) how to drive a car. Cars with automatic transmission are easier to learn to drive than those with manual transmission; driving in a large parking lot, or a rural area with no other cars on the road is easier to learn than driving in places like New York City or London. Similarly, some ultrasound applications are easier to learn than others: from fairly simple—looking at the bladder to assess urine volume prior to collection; a little more complex—a Focused Assessment with Sonography in Trauma (FAST) examination; to the very difficult—systematically finding and demonstrating a normal appendix.
We follow rules for the road in driving and when they are not followed we know on a daily basis in our emergency departments that catastrophes can occur; ultrasound in a clinician’s hands is no different—especially in those just learning to scan. Point-of-care ultrasound is a diagnostic test like any other and understanding the diagnostic test performance characteristics of ultrasound will help us stay safe on the “road to diagnosis” and here is where evidence-based medicine, or more specifically “evidence-based critical ultrasound ” can help us. Most discussion of errors in medicine have been related to preventing treatment-related harm, with little attention given to preventing diagnostic errors and misdiagnosis-related harm which are more common than harm to patients from medication-related errors . When we look at the sensitivities and specificities for ultrasound to diagnose various abdominal conditions afflicting children such as free fluid in the abdomen in pediatric trauma [4, 5], pyloric stenosis [6–8], intussusception [9, 10], and appendicitis [11, 12], in general, the specificities are higher (usually >90%) than the sensitivities (Table 1). How does it help us to know that a diagnostic test tends to have higher specificity than sensitivity?
In David Sackett’s evidence-based medicine book , there is a handy mnemonic: A high Specificity test, a Positive result effectively “rules IN” disease (SpPIN). For tests with a high Sensitivity, a Negative result effectively “rules OUT” disease (SnNOUT). In general, ultrasound with its higher specificities relative to sensitivities is more accurate for “ruling in” disease, than “ruling out” disease (with very few exceptions). Thus, ultrasound is more reliable in identifying pathology (e.g., free fluid on a FAST exam, a shadowing appendicolith, the classic target of intussuception, etc.) than in completely ruling out the presence of disease or abnormality (e.g., absence of free fluid on FAST exam, normal muscle wall thickness of a vomiting infant’s pylorus, etc.).
However, sensitivity and specificity are traditional measures of diagnostic test utility that can be difficult to interpret, particularly when the values are not very high or not very low. Furthermore, they do not directly address the probability of disease in an individual patient after a diagnostic test has been applied. Likelihood ratios (LRs) are a measure of the predictive power of a test and may be more intuitive to understand than sensitivity and specificity. They are defined as the likelihood that a given test result would be found in a patient with a target disorder, relative to the likelihood of the same test result occurring in a patient without the target disorder. LRs can be derived for any diagnostic test, whether it is a clinical sign or symptom, laboratory test, or diagnostic imaging modality like ultrasound (presented in Table 1). LRs can be applied to a pre-test probability of disease in order to generate a post-test probability of disease. The direction and magnitude of change in probability from pre-test to post-test are determined by a test’s performance characteristics, which are incorporated by the likelihood ratio. Diagnostic tests with LRs >10 or <0.1 significantly alter pre-test probabilities, LRs >5 or <0.2 have moderate effects on pre-test probabilities, while LRs closer to 1 have little effect. Looking at Table 1, likelihood ratios for positive ultrasound results tend to increase pre-test probabilities more than it decreases pre-test probabilities with negative ultrasound results. If a resulting post-test probability of disease is high enough to make the diagnosis certain, it facilitates the clinician’s decision to treat. However, if the post-test probability is not high enough to confirm the diagnosis, a clinician may pursue further diagnostic testing. Detailed discussions of likelihood ratios are available elsewhere [13, 14].
Thus, the prime rule for the road when using point-of-care ultrasound: do not rely on ultrasound alone to “rule out” disease. With respect to likelihood ratios for negative ultrasound results, the post-test probability will probably not be low enough to confidently exclude disease. This will keep you from sending home the intussusception, pyloric stenosis or appendicitis solely based on a “negative” ultrasound. The worst thing that will happen is a confirmatory test or someone will tell you that the child you admitted did not have “x”, “y” or “z”. If you have an inconclusive or “negative” ultrasound (e.g., you did not see free fluid on a FAST exam, target sign of intussusception, or a big fat >6 mm appendicitis that hurts the patient on sono-palpation, etc.), take a step back, reassess the clinical picture, and get that confirmatory study or continue to observe your pediatric patient. An unaddressed area of research is the effect of serial ultrasound exams on overall diagnostic accuracy (either by same operator or by a different one) as an advantage of point-of-care ultrasound is that it is easily repeatable with abnormal findings highly reproducible. Most likely, with an abnormal or positive ultrasound result—you have probably clinched the diagnosis with a sufficiently high post-test probability that allows you to move forward with further evaluation if necessary, treatment and then onto your next emergency pediatric patient.
It is recognized that we need to minimize the amount of ionizing radiation that children receive from imaging studies , and with ongoing research, we can be hopeful that point-of-care ultrasound may have a role in this. In the coming years, we will likely see research studies that will investigate not just the accuracy of clinician-performed point-of-care ultrasound to diagnose various pediatric abdominal emergencies and other pathologies, but also to identify where preventable diagnostic errors may arise and to develop strategies to avoid them—whether it involves standardized scanning techniques, improved ultrasound machine functionality, or operator training factors. Just as road-traffic safety is an important discipline that has led to use of air-bags, car seats for infants, and seat-belt laws, efforts to study diagnostic errors related to point-of-care ultrasound use can help us avoid misdiagnosis-related harm in our patients. Bottom line: in general, think of point-of-care ultrasound as a “rule-in” diagnostic test and not a tool to “rule-out” disease.
Ishrak O (2009) Ultrasound—a revolution in the making. Crit Ultrasound J 1(1):27–32
Elbarbary M (2009) Evidence-based critical ultrasound: a mission possible. Crit Ultrasound J 1(1):3–4
Newman-Toker DE, Pronovost PJ (2009) Diagnostic errors—the next frontier for patient safety. JAMA 301(10):1060–1062
Holmes JF, Brant WE, Bond WF et al (2001) ED ultrasonography in the evaluation of hypotensive and normotensive children with blunt abdominal trauma. J Pediatr Surg 36:968–973
Holmes JF, Gladman JF, Chang CH (2007) Performance of abdominal ultrasonography in pediatric blunt trauma patients: a meta-analysis. J Pediatr Surg 42(9):1588–1594
Godbole P, Sprigg A, Dickson JA, Lin PC (1996) Ultrasound compared with clinical examination in infantile hypertrophic pyloric stenosis. Arch Dis Child 75(4):335–337
Malcom GE 3rd, Raio CC, Del Rios M et al (2009) Feasibility of emergency physician diagnosis of hypertrophic pyloric stenosis using point-of-care ultrasound: a multi-center case series. J Emerg Med 37(3):283–286
McVay MR, Copeland DR, McMahon LE et al (2009) Surgeon-performed ultrasound for diagnosis of pyloric stenosis is accurate, reproducible and clinically valuable. J Pediatr Surg 44(1):169–171
Kairam N, Kaiafis C, Shih R (2009) Diagnosis of pediatric intussusceptions by an emergency physician-performed bedside ultrasound: a case report. Pediatr Emerg Care 25(3):177–180
Eshed I, Gorenstein A, Serour F et al (2004) Intussusception in children: can we rely on screening sonography performed by junior residents? Pediatr Radiol 34(2):134–137
Garcia Pena BM, Mandl KD, Kraus SJ et al (1999) Ultrasonography and limited computed tomography in the diagnosis and management of appendicitis in children. JAMA 282(11):1041–1046
Fox JC, Solley M, Anderson CL et al (2008) Prospective evaluation of emergency physician performed bedside ultrasound to detect acute appendicitis. Eur J Emerg Med 15(2):80–85
Sackett DL, Straus SE, Richardson WS et al (2000) Evidence-based medicine: how to practice and teach EBM, 2nd edn. Churchill Livingstone, UK, p 74
Guyatt G, Rennie D, Meade MO et al (2008) Users’ guides to the medical literature: a manual for evidence-based clinical practice, 2nd edn. McGraw-Hill Companies, Inc., USA
Hall EJ, Brenner DJ (2008) Cancer risks from diagnostic radiology. Br J Radiol 81(965):362–378
Conflict of interest