This was a prospective observational study conducted in two academic emergency departments in Ontario (combined visits of ~ 110,000/year). The Queen’s University Health Sciences Research Ethics Board approved this study, which accrued subjects over 30 months (April 2011–July 2013). All subjects provided written consent prior to enrollment.
Patients aged 16–65 years who had either CT or radUS ordered for suspected renal colic were screened for eligibility. The treating emergency physicians enrolled eligible patients 24 h a day, 7 days a week, assisted by dedicated research personnel during daytime to evening hours up to 7 days a week, which matched the availability of advanced imaging for this indication.
Patients were excluded for hemodynamic instability, fever, suspected urinary tract infection based on symptoms or urinalysis positive for leukocytes and nitrites, pregnancy, renal transplant, single functioning kidney, known abdominal aortic aneurysm or incarceration. Patients were ineligible if no formal imaging was ordered at the time of their emergency visit.
Attending or resident emergency physicians performed the PoCUS scans. To be eligible to perform study scans, physicians had to complete an accredited emergency department ultrasound course (such as those endorsed by the Canadian Point of Care Ultrasound Society) or the local, required, introductory point-of-care ultrasound course for emergency medicine residents focusing on focused assessment with sonography in trauma (FAST), aortic and obstetrical ultrasound skills. Scanning physicians had to attend a didactic lecture where the study was described and renal anatomy, ultrasound technique and several examples were reviewed. They attended a training session where physicians scanned live models with hydronephrosis. The newly trained physicians had to complete 25 observed renal ultrasounds with an expert physician (fellowship trained or additional training in renal ultrasound) prior to enrolling patients.
Other than undergoing PoCUS, subjects were treated according to usual practice. The decision and type of formal imaging was left entirely to the discretion of the treating physician, and potential subjects were approached by research staff only after formal imaging had been ordered. Scans were performed using a My Lab 5 (Esaote, Genoa Italy) ultrasound machine with a curvilinear 3.5-MHz probe. Scanning physicians performed a B-mode scan of the abdominal aorta and bilateral kidneys (short and long axis). The scanning physician recorded subject weight, height, if hydronephrosis was present/absent/indeterminate for each kidney, as well as the diameter of the abdominal aorta on a standardized form while blinded to any formal imaging. In cases where the formal imaging was performed prior to PoCUS, the scanning physician was directed not to look at images or the formal radiology report.
A radiologist (CS) blinded to the PoCUS reviewed all formal CT or radUS images to establish the degree of hydronephrosis, graded as absent, mild, moderate or severe [19]. The radiologist also documented the presence, position and size of any urinary calculus, signs suggestive of recent stone passage, as well as any alternative diagnosis that could account for the patient’s symptoms and the presence of abdominal aortic aneurysm (AAA) on a standardized form.
Research assistants performed 30-day telephone follow-up and asked scripted standardized questions regarding urgent consultations with urologists, any interventions or hospitalizations. A chart review was performed at 30 days (NR and SS). Details of urologic consultations, interventions, hospitalizations, sepsis and death within the regional healthcare system were extracted from the record, recorded on standardized forms, and entered into a RedCap database. Patients were considered lost to follow-up if they could not be contacted by telephone and had no record of emergency room visit, urology intervention or hospital admission in the medical record.
The primary outcome measure was the sensitivity and specificity of PoCUS for the presence or absence of hydronephrosis compared to formal imaging in patients with suspected renal colic.
The secondary outcome was the association between PoCUS findings and complications within 30 days, defined as the composite of any urologic intervention (e.g., lithotripsy, stent, or percutaneous nephrostomy), sepsis, hospital admission or death. Urgent complications were defined as any of the composite outcomes that occurred within 7 days of initial emergency department visit. Urgent complications were assessed separately to differentiate patients who presented with an acute worsening of their symptoms versus patients who had urology follow-up and were scheduled for a surgical procedure within 30 days. The effects of BMI, time interval to formal imaging and physician scanning experience on the accuracy of PoCUS were analyzed.
For the primary analysis, indeterminate PoCUS scans were deemed negative for hydronephrosis, but reclassified as positive in a separate sensitivity analysis. The degree of severity of hydronephrosis on formal imaging was dichotomized at different cut points to determine the diagnostic accuracy of PoCUS when hydronephrosis was considered positive only when graded “mild, moderate and severe”, “moderate and severe”, or “severe”. Patients who were lost to 30-day follow-up were excluded from the analysis for the secondary outcome of complications and no imputation was performed.
Statistical analysis was performed SAS software, version 9.4 (Cary, North Carolina). Baseline characteristics were summarized as means and standard deviations and medians and quartiles for continuous variables and proportions for binary and categorical variables. Sensitivity, specificity and likelihood ratios were calculated for detection of hydronephrosis with PoCUS using the formal radiology result as the reference standard. A Chi square test was used to compare risk of complications, and to assess for changes in accuracy for detection of hydronephrosis compared to formal imaging time between PoCUS and formal imaging, and BMI.
Results are reported in accordance with the STARD 2015 guidelines for studies of diagnostic accuracy [20].
We performed an a priori sample size calculation using estimates taken from Edmonds et al. [21], a retrospective chart review of patients undergoing formal ultrasound for the diagnosis of renal colic in emergency patients. These investigators reported that 0.6% of subjects with no abnormality on ultrasound received urologic intervention versus 6.2% of patients with a visualized stone and 6.8% of patients with ultrasounds suggestive of ureterolithiasis. Assuming a 7% versus 1% in complications/interventions between patients with and without hydronephrosis on PoCUS, two-tailed α of 5%, and a power of 80%, it was determined that 167 subjects would be required in each group. We estimated that this would provide 95% confidence bands of ± 10% around the point estimate of sensitivity and specificity for the primary outcome.