The reproducibility of measuring maximum abdominal aortic aneurysm diameter from ultrasound images
The Ultrasound Journal volume 13, Article number: 13 (2021)
Accurate repeat assessment of the diameter of an abdominal aortic aneurysm (AAA) is important. This study investigated the reproducibility of different methods of measuring AAA diameter from ultrasound images.
Fifty AAA patients were assessed by ultrasound. Maximum AAA diameter was measured independently by three trained observers on two separate occasions using a standardised protocol. Five diameters were measured from each scan, three in the anterior–posterior (AP) and two in the transverse (TV) plane, including inner-to-inner (ITI), outer-to-outer (OTO) and leading edge-to-leading edge (LETLE). Intra- and inter-observer reproducibility were reported as reproducibility coefficients. Statistical comparison of methods was performed using linear mixed effects models.
Intra-observer reproducibility coefficients (AP LETLE 2.2 mm; AP ITI 2.4 mm; AP OTO 2.6 mm) were smaller than inter-observer reproducibility coefficients (AP LETLE 4.6 mm: AP ITI 4.5; and AP OTO 4.8 mm). There was no statistically significant difference in intra-observer reproducibility of three types of measurements performed in the AP plane. Measurements obtained in the TV plane had statistically significant worse intra-observer reproducibility than those performed in the AP plane.
This study suggests that the comparison of maximum AAA diameter between repeat images is most reproducibly performed by a single trained observer measuring diameters in the AP plane.
Approximately 2% of men and 0.5% of women aged > 65 years develop an abdominal aortic aneurysm (AAA) [1, 2]. Maximum AAA diameter is the most established predictor of AAA growth and rupture and is used in clinical practice to guide decision-making [3,4,5]. Current guidelines recommend considering AAA repair when the maximum diameter is ≥ 55 mm in men and ≥ 50 mm in women [6, 7]. Management of smaller asymptomatic AAAs is by surveillance with repeat imaging performed at intervals to monitor maximum AAA diameter. Furthermore, there is growing interest in the identification of drug therapies to slow AAA growth [8, 9]. Trials testing potential drugs usually assess outcome by monitoring maximum AAA diameter growth over time . Reproducible methods to measure AAA diameter are therefore of both clinical and research importance.
In clinical practice and most previous clinical trials ultrasound imaging has been used to estimate maximum AAA diameter [6, 10]. Despite the importance of accurate determination of AAA diameter, measurement protocols are often incompletely reported and vary in both plane of acquisition and calliper placement . The United Kingdom Small Aneurysm Trial reported maximal anterior–posterior outer-to-outer diameter (OTO) from ultrasound [12, 13]. AAA screening programmes have used a variety of different methods to measure AAA diameter on ultrasound, including the inner-to-inner (ITI) or leading edge-to-leading-edge (LETLE) methods of calliper placement [14, 15]. Disparate methods of calliper placement has been reported to cause differences of up to 5 mm in maximal AAA diameter with implications for decision-making regarding surgical repair and surveillance intervals which could impact on patient care [16, 17]. A number of theoretical principles have been cited as justification for different calliper selection, but no standardised method has been ubiquitously adopted. Three recent studies have compared these three methods of calliper placement, but had inconsistent findings [16,17,18].
A key aspect in choosing a method of measurement is its repeatability or reproducibility. This is particularly important for AAA diameter as this is commonly remeasured at intervals during which only small changes occur. This study aimed to compare the inter- and intra-observer repeatability of five different methods of measuring AAA diameter using distinct measurement planes and calliper placement.
Study design and participant recruitment
Participants were recruited from The Department of Vascular and Endovascular Surgery at Townsville University Hospital . All participants provided written informed consent for inclusion and ethics approval was obtained from The Townsville Hospital and Health Service Human Research Ethics Committee (13/QPCH/16). A convenience sample size of 50 participants was used in concordance with previous similar studies [16, 17, 20]. Participants met the following inclusion criteria: (1) a small infra-renal AAA measuring 30–55 mm in maximal diameter and (2) an ultrasound performed by one of three experienced vascular sonographers within the Townsville University Hospital Vascular Laboratory using a standardised protocol. Patients with a history of previous abdominal aortic surgery were excluded. At recruitment an interview and physical examination was conducted to collect relevant medical history and clinical measurements. Ischaemic heart disease, hypertension and diabetes were defined as a previous diagnosis or treatment of these conditions by a qualified medical physician. Stroke was defined as a documented history of ischaemic or haemorrhagic stroke. The presence of aneurysms at other sites was determined through documented history or relevant medical imaging. Brachial blood pressure, waist circumference and body mass index were measured as previously described .
All ultrasound scans were performed using a Phillips iu22 machine (Phillips Medical Systems, United States) with a C5-1 MHz general purpose curvilinear abdominal transducer by one of three experienced sonographers between December 2013 and December 2015. Each sonographer was a formally trained and registered sonographer with specialised experience in vascular ultrasound. Each participant was fasted for 12 h prior to their ultrasound scan to minimise interference from bowel gas. All participants were scanned in the supine position. The abdominal aorta was examined in both the transverse and sagittal planes to identify infra-renal landmarks, vessel tortuosity and obliqueness. Static ultrasound images in the transverse plane acquired in systole were obtained at the point of maximal dilation of the infra-renal abdominal aorta perpendicular to the central vessel line. The images were then centrally stored in accordance with health service policy.
Three observers were trained to measure maximal AAA diameter on static ultrasound images using a predefined protocol. Observers were selected based on pre-existing knowledge of aortic anatomy, cardiovascular physiology and imaging. Observer one was a qualified vascular sonographer with extensive experience in acquiring and interpreting ultrasound imaging of the aorta. Observer two was a clinical medical student with previous experience measuring AAA growth on computerised tomographic angiography scans. Observer three was a research worker and exercise physiologist with extensive experience in interpreting static ultrasound images of the abdominal aorta. A set measurement protocol was developed in consultation between a vascular sonographer, vascular surgeon and researcher. Five measurements were performed to assess AAA diameter. Three in the anterior–posterior (OTO, ITI and LETLE) and two (OTO and ITI) in the transverse plane (Fig. 1). Observer training involved both theoretical discussion and practical demonstration of the measurement protocol. Observers then measured a separate series of ten static ultrasound scan images independently before a consensus discussion was conducted on calliper placement between observers. Static ultrasound images were imported as DICOM images to the OsiriX Lite 32-bit version (Pixmeo, Geneva, Switzerland) software for analysis. To avoid bias, only static images where the sonographer placed measurement callipers had been omitted were included in this study. Prior to each measurement, each observer measured a 10-mm marked interval on a 100-mm scale to ensure accurate calibration of callipers. Each observer independently measured identical static images from the 50 participants and were blinded to the other observers’ measurements. Each observer repeated measurements 1 week later blinded to their earlier results.
Data analyses were performed in SPSS Version 23.0 (IBM SPSS Inc., Chicago, IL, United States) and R (Foundation for Statistical Computing, Vienna, Austria) with assistance from an expert statistician (REJ). Demographic data were reported as count (%) for dichotomous data and median (inter-quartile range 25th to 75th centile) for continuous data.
Intra- and inter-observer reproducibility
The mean and standard deviation (SD) of the differences between measurement one and two were calculated for each observer. The SD of the differences were then multiplied by 1.96 to obtain the reproducibility coefficient, or 95% limits of agreement (LOA), for each individual observer. The MethComp package in R was used to combine data from all three observers to calculate the pooled reproducibility coefficient for each individual method. Each scan–observer combination was treated as an individual item. Linear mixed effects models were used to formally test for a significant difference in repeatability between each individual method by comparing squared mean differences from a reference method (anterior–posterior ITI). Observers and subjects were both treated as random effects and mean difference and calliper placement method as fixed effects. The overall mean difference between ITI, LETLE and OTO for each scan was also calculated.
The intra-observer reproducibility coefficients for each individual observer are shown in Table 2. There was notable variation in reproducibility coefficient between each individual method and each individual observer. Scans obtained in the anterior–posterior plane and with LETLE calliper placement had the lowest overall intra-observer reproducibility coefficient (± 2.2 mm), but this was not statistically significantly different from ITI and OTO calliper placement in the same plane. All three observers consistently measured images in the transverse plane with a poorer repeatability as highlighted by the poorer overall intra-observer reproducibility coefficient for both ITI (± 4.9 mm) and OTO (± 5.6 mm) calliper placement. Measurements in the transverse plane (ITI P = 0.001; OTO P < 0.001) were significantly less reproducible than those measured in the AP plane (Table 2).
The inter-observer reproducibility coefficient for each method is shown in Table 3. The inter-observer reproducibility coefficients were poorer than the corresponding intra-observer reproducibility coefficient for each method. The inter-observer reproducibility was poorest for images measured transversely (Table 3).
This study examined the influence of alternative methods of measurement of AAA diameter under conditions typically required in clinical trials. There was no statistically significant difference between alternative methods of measurement where calliper placement was in line with probe positioning (anterior–posterior for supine position). Measurements obtained perpendicular to the probe (transverse in the supine position) have been reported to be less repeatable due to poorer resolution of the lateral vessel walls . This study supports this finding with statistically significant worse intra-observer reproducibility in both ITI and OTO measurements obtained in the transverse direction compared to those measured anterior–posterior.
Sixteen previous studies reporting the reproducibility of abdominal aortic measurements with ultrasound were identified [14, 16,17,18, 20, 23,24,25,26,27,28,29,30,31,32,33]. There was marked variation in reproducibility coefficients for both intra-observer (range ± 0.9 mm to ± 4.0 mm) and inter-observer (range ± 1.7 mm to ± 12.6 mm) repeatability. Gurtelschmid et al. reported better inter-observer reproducibility coefficients in anterior–posterior LETLE (± 4.0 mm) and anterior–posterior ITI (± 4.6 mm) calliper placement when compared with anterior–posterior OTO (± 5.3 mm) calliper placement. Borgbjerg et al. reported similar findings with better inter-observer reproducibility coefficients with anterior–posterior LETLE (± 3.8 mm) and anterior–posterior ITI (± 3.9 mm) calliper placement compared with anterior–posterior OTO (± 5.2 mm) [16, 18]. These findings are in contrast to those of Chui et al. who reported no statistical difference in reproducibility coefficients between these three methods (anterior–posterior LETLE ± 3.5 mm; anterior–posterior ITI ± 4.8 mm; anterior–posterior OTO ± 3.4 mm) . The current study found no statistically significant differences between different methods of calliper placement when only measurements obtained in the same plane as the ultrasound probe are considered. The overall intra-observer reproducibility found in the current study are similar to those previously reported [16, 18].
The mean difference between AAA diameter measured by the ITI, LETLE and OTO methods were comparable to those previously reported and relate to vessel wall thickness [16, 17]. These differences highlight the importance of having clearly defined methods of calliper placement that are consistently used in both clinical practice and research. Multiple studies have looked at the influence of using alternative methods of calliper placement on the recruitment of patients into surveillance programmes. ITI measurements underestimate AAA size and lead to reduced sensitivity when used as a screening tool. A previous study analysed the influence of calliper placement on AAA prevalence in a cohort of 18,698 patients and found that it led to a significant difference in AAA diagnosis and subsequent recruitment into surveillance programmes (AAA prevalence ITI = 3.3%, LETLE = 4.0% and OTO = 5.9%) .
This study suggests that the ITI, OTO and LETE calliper placement methods can be equally well reproduced when placed in the same plane as the US probe, i.e. anterior–posterior. The measurement of transverse ITI or OTO diameter is not as reproducible. These findings suggest that measurement in the anterior–posterior plane should be used in clinical practice and clinical trials. Since the repeatability of measurements is much better within rather than between individuals it is also preferable for measurements to be performed by the same observer.
The current study used modern ultrasound technology and standardised methodology to directly compare the three leading methods of calliper placement. Of the 16 previous reproducibility studies identified seven [23, 27, 29,30,31,32,33] were published prior to 2000 and only three [16,17,18] reported the inter-observer reproducibility for all three methods of calliper placement and two [16, 17] the intra-observer reproducibility. This study examined the variation in measurements introduced by different methods of calliper placement on static AAA images obtained at a single time point. Measurement error introduced during the acquisition of scans was not assessed and therefore in clinical practice the reproducibility coefficients are likely larger.
In conclusion, AAA diameter measurements obtained perpendicular to the orientation of the ultrasound probe (anterior–posterior) can be performed more reproducibly than those performed in the transverse plane. Measurements performed by the same observer also have better repeatability than those performed by different observers. The findings suggest that measurements of AAA size should be performed in the anterior–posterior plane and compared between different time periods by the same observer, particularly for situations such as clinical trials where high precision is required to sensitively detect changes in AAA diameter over time.
Availability of data
All data generated or analysed during this study are included in this published article.
Abdominal aortic aneurysm
Anterior–posterior anatomical plane of the body
Transverse plane of the body
Inner-to-inner walls of the aorta
Outer-to-outer walls of the aorta
Leading edge-to-leading edge within the walls of the aortas
Limits of agreement
Grondal N, Sogaard R, Lindholt JS (2015) Baseline prevalence of abdominal aortic aneurysm, peripheral arterial disease and hypertension in men aged 65–74 years from a population screening study (VIVA trial). Br J Surg 102(8):902–906. https://doi.org/10.1002/bjs.9825
Li X, Zhao G, Zhang J, Duan Z, Xin S (2013) Prevalence and trends of the abdominal aortic aneurysms epidemic in general population-a meta-analysis. PLoS ONE. https://doi.org/10.1371/journal.pone.0081260
Lederle FA, Johnson GR, Wilson SE, Ballard DJ, Jordan WD Jr, Blebea J et al (2002) Rupture rate of large abdominal aortic aneurysms in patients refusing or unfit for elective repair. J Am Med Assoc 287(22):2968–2972. https://doi.org/10.1001/jama.287.22.2968
Brady AR, Thompson SG, Fowkes FGR, Greenhalgh RM, Powell JT (2004) Abdominal aortic aneurysm expansion: risk factors and time intervals for surveillance. Circulation 110(1):16–21. https://doi.org/10.1161/01.CIR.0000133279.07468.9F
Vardulaki KA, Prevost TC, Walker NM, Day NE, Wilmink ABM, Quick CRG et al (1998) Growth rates and risk of rupture of abdominal aortic aneurysms. BJS 85(12):1674–1680. https://doi.org/10.1046/j.1365-2168.1998.00946.x
Wanhainen A, Verzini F, Van Herzeele I, Allaire E, Bown M, Cohnert T et al (2019) European Society for Vascular Surgery (ESVS) 2019 clinical practice guidelines on the management of abdominal aorto-iliac artery aneurysms. Eur J Vasc Endovasc Surg 57(1):8–93. https://doi.org/10.1016/j.ejvs.2018.09.020 (Epub 2018/12/12)
Chaikof EL, Dalman RL, Eskandari MK, Jackson BM, Lee WA, Mansour MA et al (2018) The Society for Vascular Surgery practice guidelines on the care of patients with an abdominal aortic aneurysm. J Vasc Surg 67(1):2-77.e2. https://doi.org/10.1016/j.jvs.2017.10.044
Golledge J (2019) Abdominal aortic aneurysm: update on pathogenesis and medical treatments. Nat Rev Cardiol 16(4):225–242. https://doi.org/10.1038/s41569-018-0114-9
Golledge J, Norman PE, Murphy MP, Dalman RL (2017) Challenges and opportunities in limiting abdominal aortic aneurysm growth. J Vasc Surg 65(1):225–233. https://doi.org/10.1016/j.jvs.2016.08.003
Morris DR, Cunningham MA, Ahimastos AA, Kingwell BA, Pappas E, Bourke M et al (2015) TElmisartan in the management of abDominal aortic aneurYsm (TEDY): The study protocol for a randomized controlled trial. Trials. https://doi.org/10.1186/s13063-015-0793-z
Long A, Rouet L, Lindholt JS, Allaire E (2012) Measuring the maximum diameter of native abdominal aortic aneurysms: review and critical analysis. Eur J Vasc Endovasc Surg 43(5):515–524. https://doi.org/10.1016/j.ejvs.2012.01.018
Powell JT, Brown LC, Forbes JF, Fowkes FG, Greenhalgh RM, Ruckley CV et al (2007) Final 12-year follow-up of surgery versus surveillance in the UK Small Aneurysm Trial. The Br J Surg 94(6):702–708. https://doi.org/10.1002/bjs.5778 (Epub 2007/05/22)
Participants TU (1998) Mortality results for randomised controlled trial of early elective surgery or ultrasonographic surveillance for small abdominal aortic aneurysms. The UK Small Aneurysm Trial Participants. Lancet 352(9141):1649–1655 (Epub 1998/12/16)
Hartshorne TC, McCollum CN, Earnshaw JJ, Morris J, Nasim A (2011) Ultrasound measurement of aortic diameter in a national screening programme. Eur J Vasc Endovasc Surg 42(2):195–199. https://doi.org/10.1016/j.ejvs.2011.02.030 (Epub 2011/03/29)
Wanhainen A, Bjorck M (2011) The Swedish experience of screening for abdominal aortic aneurysm. J Vasc Surg. 53(4):1164–1165. https://doi.org/10.1016/j.jvs.2010.10.099 (Epub 2011/03/29)
Borgbjerg J, Bogsted M, Lindholt JS, Behr-Rasmussen C, Horlyck A, Frokjaer JB (2018) Superior reproducibility of the leading to leading edge and inner to inner edge methods in the ultrasound assessment of maximum abdominal aortic diameter. Eur J Vasc Endovasc Surg 55(2):206–213. https://doi.org/10.1016/j.ejvs.2017.11.019 (Epub 2017/12/27)
Chiu KWH, Ling L, Tripathi V, Ahmed M, Shrivastava V (2014) Ultrasound measurement for abdominal aortic aneurysm screening: a direct comparison of the three leading methods. Eur J Vasc Endovasc Surg 47(4):367–373. https://doi.org/10.1016/j.ejvs.2013.12.026
Gurtelschmid M, Bjorck M, Wanhainen A (2014) Comparison of three ultrasound methods of measuring the diameter of the abdominal aorta. Br J Surg 101(6):633–636. https://doi.org/10.1002/bjs.9463 (Epub 2014/04/12)
Pinchbeck JL, Moxon JV, Rowbotham SE, Bourke M, Lazzaroni S, Morton SK, et al. Randomized Placebo-Controlled Trial Assessing the Effect of 24-Week Fenofibrate Therapy on Circulating Markers of Abdominal Aortic Aneurysm: Outcomes From the FAME-2 Trial. J Am Heart Assoc. 2018;7(19):e009866. doi: https://doi.org/10.1161/JAHA.118.009866. Epub 2018/10/30
Thapar A, Cheal D, Hopkins T, Ward S, Shaloub J, Yusuf SW (2010) Internal or external wall diameter for abdominal aortic aneurysm screening? Ann R Coll Surg Engl 92(6):503–505. https://doi.org/10.1308/003588410X12699663903430
Fernando ME, Crowther RG, Cunningham M, Lazzarini PA, Sangla KS, Golledge J (2015) Lower limb biomechanical characteristics of patients with neuropathic diabetic foot ulcers: The diabetes foot ulcer study protocol. BMC Endocr Disord. https://doi.org/10.1186/s12902-015-0057-7
Beales L, Wolstenhulme S, Evans JA, West R, Scott DJ (2011) Reproducibility of ultrasound measurement of the abdominal aorta. Br J Surg. 98(11):1517–1525. https://doi.org/10.1002/bjs.7628 (Epub 2011/08/24)
Akkersdijk GJM, Puylaert JBCM, Coerkamp EG, De Vries AC (1994) Accuracy of ultrasonographic measurement of infrarenal abdominal aortic aneurysm. Br J Surg. 81(3):376. https://doi.org/10.1002/bjs.1800810317
Bonnafy T, Lacroix P, Desormais I, Labrunie A, Marin B, Leclerc A et al (2013) Reliability of the measurement of the abdominal aortic diameter by novice operators using a pocket-sized ultrasound system. Arch Cardiovasc Dis 106(12):644–650. https://doi.org/10.1016/j.acvd.2013.08.004
Bredahl K, Eldrup N, Meyer C, Eiberg JE, Sillesen H (2013) Reproducibility of ECG-gated ultrasound diameter assessment of small abdominal aortic aneurysms. Eur J Vasc Endovasc Surg 45(3):235–240. https://doi.org/10.1016/j.ejvs.2012.12.010 (Epub 2013/01/22)
Crilly MA, Mundie A, Bachoo P, Bruce P, Colvin FA, Geddes WA et al (2016) Clinical agreement between nurses in the ultrasound measurement of abdominal aortic diameter within a National Screening Programme. Ann Vasc Surg 33:194–201. https://doi.org/10.1016/j.avsg.2015.11.020
Ellis M, Powell JT, Greenhalgh RM (1991) Limitations of ultrasonography in surveillance of small abdominal aortic aneurysms. Br J Surg 78(5):614–616. https://doi.org/10.1002/bjs.1800780529
Grøndal N, Bramsen MB, Thomsen MD, Rasmussen CB, Lindholt JS (2012) The cardiac cycle is a major contributor to variability in size measurements of abdominal aortic aneurysms by ultrasound. Eur J Vasc Endovasc Surg 43(1):30–33. https://doi.org/10.1016/j.ejvs.2011.09.025
Jaakkola P, Hippelainen M, Farin P, Rytkonen H, Kainulainen S, Partanen K (1996) Interobserver variability in measuring the dimensions of the abdominal aorta: Comparison of ultrasound and computed tomography. Eur J Vasc Endovasc Surg 12(2):230–237. https://doi.org/10.1016/S1078-5884(96)80112-2
Lanne T, Sandgren T, Mangell P, Sonesson B, Hansen F (1997) Improved reliability of ultrasonic surveillance of abdominal aortic aneurysms. Eur J Vasc Endovasc Surg 13(2):149–153. https://doi.org/10.1016/S1078-5884(97)80011-1
Lindholt JS, Vammen S, Juul S, Henneberg EW, Fasting H (1999) The validity of ultrasonographic scanning as screening method for abdominal aortic aneurysm. Eur J Vasc Endovasc Surg 17(6):472–475. https://doi.org/10.1053/ejvs.1999.0835
Pleumeekers HJCM, Hoes AW, Mulder PGH, Van Der Does E, Hofman A, Laméris JS et al (1998) Differences in observer variability of ultrasound measurements of the proximal and distal abdominal aorta. J Med Screen 5(2):104–108. https://doi.org/10.1136/jms.5.2.104
Singh K, Bonaa KH, Solberg S, Sorlie DG, Bjork L (1998) Intra- and interobserver variability in ultrasound measurements of abdominal aortic diameter. The Tromso study. Eur J Vasc Endovasc Surg 15(6):497–504. https://doi.org/10.1016/S1078-5884(98)80109-3
This research was supported by grants from the NHMRC (1079369, 1098717, 1180736 and 1022752) and Queensland Government. JG holds a Practitioner Fellowship from the NHMRC (1117061) and a Senior Clinical Research Fellowship from the Queensland Government.
Ethics approval and consent to participate
All participants provided written informed consent for inclusion and ethics approval was obtained from The Townsville Hospital and Health Service Human Research Ethics Committee (13/QPCH/16).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Matthews, E.O., Pinchbeck, J., Elmore, K. et al. The reproducibility of measuring maximum abdominal aortic aneurysm diameter from ultrasound images. Ultrasound J 13, 13 (2021). https://doi.org/10.1186/s13089-021-00211-z
- Abdominal aortic aneurysm