Skip to main content

Unravelling the skillset of point-of-care ultrasound: a systematic review

Abstract

Background

The increasing number of physicians that are trained in point-of-care ultrasound (POCUS) warrants critical evaluation and improvement of current training methods. Performing POCUS is a complex task and it is unknown which (neuro)cognitive mechanisms are most important in competence development of this skill. This systematic review was conducted to identify determinants of POCUS competence development that can be used to optimize POCUS training.

Methods

PubMed, Web of Science, Cochrane Library, Emcare, PsycINFO and ERIC databases were searched for studies measuring ultrasound (US) skills and aptitude. The papers were divided into three categories: “Relevant knowledge”, “Psychomotor ability” and ‘Visuospatial ability’. The ‘Relevant knowledge’ category was further subdivided in ‘image interpretation’, ‘technical aspects’ and ‘general cognitive abilities’. Visuospatial ability was subdivided in visuospatial subcategories based on the Cattell-Horn-Carroll (CHC) Model of Intelligence v2.2, which includes visuospatial manipulation and visuospatial perception. Post-hoc, a meta-analysis was performed to calculate pooled correlations.

Results

26 papers were selected for inclusion in the review. 15 reported on relevant knowledge with a pooled coefficient of determination of 0.26. Four papers reported on psychomotor abilities, one reported a significant relationship with POCUS competence. 13 papers reported on visuospatial abilities, the pooled coefficient of determination was 0.16.

Conclusion

There was a lot of heterogeneity in methods to assess possible determinants of POCUS competence and POCUS competence acquisition. This makes it difficult to draw strong conclusions on which determinants should be part of a framework to improve POCUS education. However, we identified two determinants of POCUS competence development: relevant knowledge and visuospatial ability. The content of relevant knowledge could not be retrieved in more depth. For visuospatial ability we used the CHC model as theoretical framework to analyze this skill. We could not point out psychomotor ability as a determinant of POCUS competence.

Background

The number of physicians that are trained in point-of-care ultrasound (POCUS) is growing. POCUS is defined as the use of portable ultrasonography (US) at the patient’s bedside, which is performed and interpreted instantaneously [1, 2]. The utility of POCUS depends on the experience and skill of the operator and, therefore, proper training and assessment of competence are crucial [3]. As more and more US training and assessment methods are being developed, differences in both training and assessment methods become more apparent [4,5,6]. With the increasing number of physicians that are required to learn POCUS, there is a need for critical evaluation of the methods used to teach POCUS, as well as the methods used to assess competence and competence development. Current educational US literature mainly focuses on the overall effects of US courses, with the assessment of performance before and after training, often leading to positive results. The underlying mechanism on which this process of improvement is based, i.e. the set of (cognitive) factors that predict efficient POCUS competence development, is often approached as a black box and therefore remains largely unknown.

Performing and interpreting US examination comprises a unique and complex set of actions. The operator must be familiar with ultrasonography physics and must have sufficient knowledge of regional 3-dimensional (3D) anatomy and pathophysiology. That can be a challenge, since the US screen displays the image in 2D and the operator needs to construct and manipulate a mental 3D representation. The anatomy changes by applying pressure with the probe, and factors like breathing, bowel contents, excess of subcutaneous and visceral adipose tissue, anatomical variations and pathology must be considered to create an adequate image. US is unique compared to other imaging modalities in its operator dependency, requiring correct manipulation of the probe and various US parameters to achieve good image quality.

POCUS competence incorporates a unique combination of skills perceived to include, beside adequate knowledge, various (neuro)cognitive mechanisms, like visuospatial abilities, and psychomotor abilities [6,7,8]. From other complex medical skills, like laparoscopy and arthroscopy, we know that visuospatial and psychomotor ability are predictors of competence achievement in that field [6, 9, 10]. Visuospatial ability is the capability to generate, transform, and retain structured visual images. That is, to mentally manipulate two-dimensional and three-dimensional figures [11]. Dimensions of visuospatial ability are visuospatial perception and visuospatial manipulation. Visuospatial perception refers to the ability to appropriately perceive the physical location of an object in relation to one’s own body and to identify the physical relationship between different objects. Concretely, in US visuospatial perception describes the interpretation of size, shape, position and motion of organs [12]. Visuospatial manipulation is the ability to perceive complex patterns and mentally simulate how they might look when transformed (e.g. rotated, changed in size, partially obscured, and so forth). This is often tested with the mental rotation test (MRT) in which a more simple object-based transformation is performed [13]. Psychomotor ability means performing motor tasks with exactitude and dexterity, for example, using manual- and finger dexterity and hand–eye coordination while handling a probe [14]. There are various validated tests to assess different domains of visuospatial ability as well as psychomotor skills [15,16,17,18,19,20,21,22]. Relevant knowledge can be measured in several ways. Multiple-choice tests can be designed to measure trainee knowledge of ultrasound physics, while image interpretation of still images or short videos can be utilized to evaluate knowledge of anatomy or the recognition of pathology [6, 23]. Furthermore, it is known that a set of other general cognitive abilities are needed to successfully learn a new skill, including for example general reasoning [24, 25]. We hypothesize that this also applies to learning POCUS. The question arises whether there are relevant determinants, like knowledge, psychomotor ability, visuospatial ability and others, of POCUS competence development for POCUS practitioners, and if so, to what extent.

This systematic review summarizes current knowledge on determinants of POCUS competence and competence development in order to identify the framework of skills needed to develop and improve POCUS competence.

Methods

This protocol (ID 239322) is available for review at the PROSPERO website (https://www.crd.york.ac.uk/PROSPERO/). The review was conducted and reported according to PRISMA standards of quality [26, 27].

Information sources and search strategy

PubMed, Web of Science, Cochrane Library, Emcare, PsycINFO and ERIC databases were searched for studies measuring US skills and abilities on 5 March 2021. (Fig. 1) The entire search strategy can be found in Additional file 1: Appendix S1.

Fig. 1
figure 1

Flow diagram of study selection

Eligibility criteria and study selection

The following criteria were used to assess the eligibility of studies found by the search strategy: the study has to be an original, full peer-reviewed paper written in English, the study must be either an observational or interventional clinical trial, which includes an objective measurement of specific skills and a description or calculation of a relationship with US performance and the study subjects must be studying or working in the medical field. We excluded clinical trials with self-reported measurements of skills, conference papers, meeting abstracts, letters to the editors, reviews, meta-analyses, comments, and study protocols.

Two independent authors (TM and TV) screened all titles and abstracts in duplicate and excluded clearly irrelevant studies. The remaining articles underwent an independent, full-text screening by the same authors in duplicate. Conflicts during the selection process were resolved by a third reviewer (BH).

Data extraction

Data from eligible studies were extracted using an extraction sheet. Data items extracted are as follows: number of participants, description of participants, baseline measurements before training or intervention regarding either POCUS competence and/or competence determinants, post-intervention scores regarding POCUS competence and correlations between used interventions and POCUS competence.

Risk of bias in individual studies

To assess methodological quality in individual studies, the Medical Education Research Study Quality instrument (MERSQI) was used by the two authors independently [28]. This tool has been validated for medical education [29]. The tool has 18 points in 6 domains: study design, sampling, data type, validity, analysis, and outcomes. Furthermore, we assessed validity of possible determinants and outcome measures using the Messick framework [30].

Data analysis

Descriptive analysis was used to summarize the included studies and to describe the effects of various skills on US performance. The papers were divided into three categories based on the skills that were measured during the studies. These categories were: “Relevant knowledge”, “Psychomotor ability” and ‘Visuospatial ability”. The ‘Relevant knowledge’ category was further subdivided in ‘image interpretation’, ‘technical aspects’ and ‘general cognitive abilities’. Visuospatial ability was further focused on visuospatial manipulation and visuospatial perception. The visuospatial manipulation category entails all the tests which primarily measure mental manipulation of (limited) visual information. The visuospatial perception category entails all the tests which primarily measure perceptual accuracy.

As we did not encounter sufficient studies of consistent design and quality, a formal meta-analysis was not feasible. However, when studies reported an R2 statistic, this was taken into account when analyzing variance explained by the relevant determinants. If a study did not report an R2, linearity of data was assessed and, if applicable, R2 was calculated ad hoc for the purpose of this review. Furthermore, after examination of the papers a decision was made to calculate pooled correlations, to gain more insight in how certain tests and domains relate to the ability to learn and perform US. Using the metacor function from the meta package in R version 4.1.3 all correlations underwent Fisher’s Z transformation and were then pooled using the random-effects model [31]. These pooled correlations were then squared to obtain a pooled coefficient of determination. The determination coefficient reports how much variance of a dependent variable is explained by a determinant [32]. The random-effects model was applied because during examination of the papers, heterogeneity (using Cochran’s Q) of multiple kinds, e.g. differences in study samples and test instruments, was found. This makes the fixed-effects model inappropriate to calculate the pooled effect size. This was done for both the complete set of reports/studies, as well as separately for the studies in the knowledge and visuospatial domains. This could not be done for the psychomotor domain since no correlations were reported in those studies.

Results

Study selection

The applied search strategy yielded a total of 5535 potentially relevant papers. Removing duplicates and then screening both titles and abstracts resulted in the removal of 5324 papers. The remaining 211 papers were screened in full text, and a final 26 papers were selected for inclusion in the review. This process is highlighted in Fig. 1. There was a substantial agreement between both authors (TM and TV) during the study selection. Both during title and abstract screening (Cohen’s kappa 0.65) and during the full text selection (Cohen’s kappa 0.65). A large number of studies in the full text screening section appeared eligible at first, but at closer inspection lacked meaningful analysis regarding the relationship between measured variables and the ability to learn and/or assess US competence.

Study characteristics

Of the 26 studies, 15 reported on the relationship between relevant knowledge and US competence, three studies reported on the relationship between measured psychomotor ability and (gaining) US competence and a total of 13 papers reported on the relationship between visuospatial ability measurements and (gaining) US competence. The papers spanned various US domains; general US, sonography for trauma (FAST), musculoskeletal US, transthoracic echocardiography, US-guided central venous access, obstetric US, brachial plexus US, US-guided regional anaesthesia (UGRA), and ultrasonography for veterinary students. US competence was assessed on standardized patients, volunteers, bench models, simulators, or turkey breasts. Various competence measures were used to assess ultrasound skill level. OSCE scores of performing ultrasound on a standardized patients were mostly used (n = 11). Furthermore, time of completion of the ultrasound task and image interpretation of live images during an ultrasound examination were used. Few studies tried to identify determinants by looking at differences of those determinants between novice and expert ultra-sonographers. Validity evidence was not found for all measures used. Most assessment methods were validated in terms of content and relationships with other variables. See Additional file 1: Appendix S2 for all study characteristics.

Study appraisal

To assess risk of bias for each study the MERSQI was used. The lowest score attained on the MERSQI was a 10, while a 15.5 was the highest score. There was a median score of 12.5 across all included studies. MERSQI scores can be found in Additional file 1: Appendix S3.

Relevant knowledge

From the 15 papers reporting on the relationship between relevant knowledge and the ability to learn or perform US, eight reported a significant relationship between at least one of their measured variables and the ability to learn or perform US. [33,34,35,36,37,38,39,40] The studies describing these relationships covered various medical domains (Table 1). The significant associations were found in FAST, musculoskeletal US training for rheumatology fellows, transthoracic echocardiography, US-guided central venous access and general US education, as well as in US education in low to middle income countries [33,34,35,36,37,38,39,40]. Relevant knowledge was tested with various multiple-choice tests, mostly containing questions about US physics, knobology, image interpretation and basic anatomical knowledge. 11 studies tested relevant knowledge by means of image interpretation, 6 studies used questions about technical aspects of US (knobology, US physics). Furthermore, five papers looked at the relationship between general reasoning, memory and cue utilization (the application of cue-based associations retrieved from memory), and US performance, of which Berman et al. [34] looked at general reasoning scores using the Kit of Factor Referenced Cognitive tests. Two of the 15 papers reported a determination coefficient. Stolz et al. [41] reported this to describe the relationship between baseline US knowledge, consisting of basic US physics, system workflow, and anatomy, ability to recognize anomalies, appropriate US settings, and US competence. With an R2 of 0.028 they state that their written pre-test is not a good predictor of US interpretation ability. On the other hand, Schott [39] reported a much higher determination coefficient of 0.60 between their knowledge test and POCUS competence. For all other papers reporting a correlation statistic, primarily Pearson correlation and Spearman’s rho, R2 was calculated, see Table 2. The knowledge domain had a pooled correlation value of r = 0.51, p ≤ 0.0001. This correlation equates to a coefficient of determination of 0.26. This implies that roughly 26% of the ability to learn and or perform US in these papers is attributed to relevant knowledge. See Fig. 2. When the knowledge domain was assessed for heterogeneity, Cochran’s Q was 73.96, p ≤ 0.0001.

Table 1 Summary of included studies divided by cognitive domain with pooled correlation and determination coefficient
Table 2 Calculated and reported determination coefficients of the included studies
Fig. 2
figure 2

Forest plot of the pooled correlations of the included studies divided by cognitive domain. Rcorrelation coefficient. 95% CI95% Confidence Interval. R2determination coefficient. The pointed line represents the pooled correlation of the random effects model of all included studies, the grey box represents the weight of the studies. I2 fraction of variance due to heterogeneity. T2 the estimated standard deviation of underlying effects across the studies

Psychomotor ability

Four papers reported on psychomotor abilities in relation to US performance (Table 1). Psychomotor ability was measured by various tests, i.e. Projected Image Testing (Zig–Zag Test), Purdue Peg Board Test, Crawford Small Parts Dexterity Test, Sennes-Weinstein Monofilament Sensory Testing and the Dimensionless Squared Jerk. One study reported a significant relationship between the Dimensionless Squared Jerk, a validated motion metric that measures deliberate hand movements, and US expertise in obstetric US [58], while the other three papers did not find a significant relationship between psychomotor skills and US competence. [45, 48, 49] No correlation or determination coefficient was reported in the studies. Walker [49] found a regression coefficient of 0.00056 (p = 0.580) for the Grooved Pegboard test performed by the non-dominant hand, and of − 0.0013 (p = 0.329) when it was executed by the dominant hand, and time to complete an ultrasound guided cystocentesis task.

Visuospatial ability

A total of 13 papers reported on the relationship between visuospatial ability measurements and US competence (Table 1). Out of these 13 papers, 10 reported a significant relationship between at least one visuospatial ability measurement and US competence. Significant results were found in brachial plexus sonography, transthoracic echocardiography, UGRA, ultrasonography for veterinary students and general ultrasonography. To further narrow down which tests were able to provide good predictions for US competence and why, visuospatial subcategories based on the Cattell-Horn-Carroll (CHC) Model of Intelligence v2.2 were used [59]. This model makes a distinction between 11 different forms of visual processing: visualization, speeded rotation, closure speed, flexibility of closure, visual memory, spatial scanning, serial perceptual integration, length estimation, perceptual illusions, perceptual alterations, and imagery. For a description of these categories, see Table 3. The most frequently used ability test for visuospatial ability was the MRT (n = 8). In these 13 papers, 12 tests were done that fit into the visuospatial manipulation category. 8 out of these 12 tests were adaptations of the MRT. 4 specified that they used the Revised Vanderberg and Kruse Mental Rotation Test A. Therefore, mental rotation is reported here in its own category to see if this specific test warrants its prominent appearance in US research. A total of 13 tests belonging to the visuospatial perception category were used in these papers [12]. See Additional file 1: Appendix S4 for an overview of the different aptitude tests used, divided by main and subcategory. Six papers reported correlation coefficients and two papers a determination coefficient (Table 2). Clem et al. [51] reported that 0.36 of US competence can be predicted by visuospatial ability. The other determination coefficient is reported by another study of Clem et al. [46] They state that 0.23 of US competence can be predicted by spatial ability after two full semesters of instructions. The pooled correlation of the visuospatial domain had a value of r(8) = 0.39, p ≤ 0.0001. The coefficient of determination was 0.16. This implies that roughly 16% of the ability to learn and or perform US across these studies could be attributed to the measured visuospatial ability. When the visuospatial domain was assessed for heterogeneity, Cochran’s Q was 27.37, p = 0.011. The papers using tests in the visuospatial manipulation category had a pooled correlation of r(7) = 0.37, p = 0.0005 and a pooled coefficient of determination of 0.14. This implies that roughly 14% of the ability to learn and or perform US across these studies could be attributed to the measured visuospatial manipulation abilities. The papers using tests in the visuospatial perception category had a pooled correlation of r(3) = 0.33, p =  < 0.0001 and a pooled coefficient of determination of 0.11. This implies that roughly 11% of the ability to learn and or perform US across these studies could be attributed to the measured visuospatial perception abilities. See Fig. 2. To see if the MRT warrants its prominent position in US research, pooled correlations were also calculated separately for the MRT, compared to the other visuospatial manipulation tests used. All the MRTs combined had a pooled correlation of r(5) = 0.415, p ≤ 0.01 and a pooled coefficient of determination of 0.17.

Table 3 Cattell–Horn–Carroll (CHC) explanation

Discussion

In this systematic review and meta-analysis, we describe several (neuro)cognitive mechanisms that correlate with the development of POCUS competence. Combined data from various studies revealed relevant knowledge and visuospatial ability as determinants of the ability to acquire POCUS competence. Psychomotor skills have been described in only one study to significantly affect POCUS competence development.

To design effective competency-based skills training programs, it is imperative to determine which underlying mechanisms or skills relate to the acquisition of POCUS competence. In our dataset of 26 papers, only four described a determination coefficient to predict how much variance of US competence could be explained by their measured determinants. [39, 42, 46, 51] Therefore, we decided to use the published data to perform a post-hoc calculation of the determination coefficients of 17 additional studies and found a pooled coefficient of determination of 16%. This implies that 16% of the ability to learn and/or perform US, as measured in these studies, can be attributed to the variables that were reported. These variables could be used to predict learner performance and to finetune personalized and adaptive education in the future. This is important as a systematic review and meta-analysis by Fontaine et al. [62] describes that adaptive e-learning environments have improved learning outcomes on both knowledge and practical skills compared to traditional methods of education and training.

Relevant knowledge is a nonspecific term and the type of knowledge that is actually relevant for POCUS competence development cannot be easily distinguished. Despite that, the pooled coefficient of determination for the knowledge domain implies that roughly 26% of the ability to learn and/or perform US might be attributed to relevant forms of knowledge. In many studies, both anatomical knowledge and image interpretation are used as outcome measures, but not all studies describe significant relationships between these types of knowledge and POCUS competence development. The fact that not all studies found significant relations is probably due to the lack of standardized tests for assessing both knowledge and POCUS competence. As expected, many studies identified relationships between POCUS competence development and pre-existing knowledge about technical aspects of ultrasound. However, since all studies used multiple-choice tests to assess various aspects of knowledge, we cannot distinguish the contribution of pre-existing technical knowledge from the other types of knowledge. When looking at general cognitive abilities, e.g. among others the capacity to acquire knowledge and competence, results are equivocal. No correlations were found between POCUS competence development and the numerical reasoning test (testing fluid intelligence, abstract reasoning, and problem-solving) or Alice Heim Group Ability test (verbal, mathematical, and spatial reasoning). [44, 47] And although Berman et al. [34], using a paper-and-pencil test, describe a correlation between general reasoning and POCUS competence development, Shafqat et al., [47] could not find such a relationship using a validated score of a UGRA task. Apparently, both tests measured different aspects of cognitive ability, and therefore one can only draw conclusions about the relation between POCUS competence development and a specific test score rather than drawing conclusions about underlying cognitive mechanisms in general.

Various tests are available to measure aspects of visuospatial ability. When looking at pooled correlations between the visuospatial manipulation and visuospatial perception categories, the visuospatial manipulation category appears to be more correlated with the ability to learn and or perform US (coefficient of determination of 14% vs 11%). Although this is only a slight difference, the skill to mentally transform and rotate the image of e.g. an organ is possibly a more important determinant than the mere observational ability to perceive and visually understand spatial information such as shapes, positions, and motions. [12] While high MRT test scores often relate to high POCUS competence levels (see Table 1) others, like the snowy picture test do not. As visuospatial ability inherits various aspects of spatial cognition, like mental rotation and transformation [63, 64], the ability to mentally rotate objects may be more relevant for US performance than the ability to quickly identify a familiar visual object from incomplete visual stimuli. When focussing on the other aspects of the CHC model, studies reported correlations with closure speed and flexibility of closure [34, 47, 48, 53], but no correlations were reported for the other perception subcategories. For this reason, no meaningful analysis can be done on which perception subcategories are more relevant than others. In addition, it remains difficult to draw any conclusions about the precise cognitive skill(s) that is/are responsible for modifying POCUS competence development, as the aptitude tests usually cover more than one skill. The underlying framework of visuospatial ability can be used in various ways to improve US education. Chuan et al. [52] showed that if medical students with low visuospatial ability receive extra training in mental rotation, they can achieve the same UGRA performance scores as their fellow students with higher visuospatial abilities. Furthermore, Hewson et al. [55] specifically trained students’ mental rotation with a simple task and improved UGRA performance. Although UGRA is probably a more complex skill than non-interventional POCUS, visuospatial skills also contribute to non-interventional US performance. [46, 53, 54, 56].

Less insight was gained into the relationship with psychomotor ability. Within our dataset, only Dromey et al. [58] described a relation between Dimensionless Squared Jerk scores and POCUS competence. Dimensionless Squared Jerk is a measure of deliberate hand movements and is often used as a measure for psychomotor skills. [65] However, when measured while performing US it will also depend on US competence and cannot be used anymore as a unique measure for psychomotor skills. When it comes to the assessment of other skills, various tests do not clearly distinguish between e.g. visuospatial ability and psychomotor skills, like the Block Design Test and the Digit Symbol Substitution test [17, 18]. Therefore, the psychomotor ability could play a more prominent role than the current literature suggests.

Our findings suggest that it may be beneficial to adjust training based on student characteristics. In our experience, students that fail the POCUS exams are often advised to simply practice more. However, it is known that complex skills are easier to learn if broken down into component skills. [66] Thus, it is conceivable that by identifying a student’s weaker points beforehand and by training this specific shortcoming isolated from the whole complex POCUS skill, the learning curve may steepen. Not only cognitive load may be decreased in an isolated task, but it is also plausible that a specific skill can be taught better and faster in a task specifically designed for that purpose. [67, 68] This skill training does not necessarily have to be integrated into an ultrasound task, but could also be trained in an alternative way. [69].

Limitations

Limitations can be subdivided into limitations of the included studies and limitations of this systematic review and meta-analysis.

Considering the included studies, one of the major issues in interpreting their results and attempting to construct a framework based on their measurements, is the large amount of heterogeneity among the test instruments used to measure determinants of POCUS competence as well as measuring POCUS competence itself. Moreover, validity evidence was not equivalent for all tests, which added to the difficulty in interpreting the data. A second limitation in some of the studies include the use of cross-sectional design in assessing for the relationship between determinants and competence. Therefore, we cannot be sure if these determinants predict competence. A third limitation is the lack of POCUS-specific papers. For example, US combined with an intervention such as UGRA might provide different outcomes than specific POCUS-focused studies because of e.g. the added complexity of the anaesthesia tasks, especially in the psychomotor domain.

Considering the current study, while the papers found in this systematic review give new insight into the underlying mechanisms of gaining POCUS competence, these mechanisms are unlikely to be solely responsible for the way someone gains POCUS competence. Although we decided, based on an extensive literature search on learning ultrasound skills, to stratify the results into the categories mentioned earlier, our categories may be incomplete. Secondly, when calculating pooled correlations for relevant knowledge and visuospatial skills, many papers did not report correlations or selectively only reported significant correlations. Therefore, pooled correlations should be interpreted with caution. Finally, to construct a framework in which evidence-based variables are used to improve training, or assessment for US competence, a proper understanding of underlying factors is required. Thus, more standardized research needs to be done, with a clear definition of determinant variables, how to measure these, and methods of assessing US competence.

Conclusion

We identified two determinants of POCUS competence development: relevant knowledge and visuospatial ability. The content of relevant knowledge could not be retrieved in more depth. For visuospatial ability we used the CHC model as a theoretical framework to analyze this skill. We could not point out psychomotor ability as a determinant of POCUS competence. The heterogeneity of results makes it difficult to draw strong conclusions about what should and should not be part of a framework used to improve POCUS education and assessment.

Availability of data and materials

All data are available upon request.

Abbreviations

3D:

3-Dimensional

FAST:

Focused assessment with sonography in trauma

MERSQI:

Medical education research study quality instrument

MRT:

Mental rotation test

OSCE:

Objective structured clinical examination

POCUS:

Point-of-care ultrasound

UGRA:

Ultrasound guided regional anaesthesia

US:

Ultrasonography

References

  1. Moore CL, Copel JA (2011) Point-of-care ultrasonography. N Engl J Med 364(8):749–757

    Article  CAS  PubMed  Google Scholar 

  2. Tarique U, Tang B, Singh M, Kulasegaram KM, Ailon J (2018) Ultrasound curricula in undergraduate medical education: a scoping review. J Ultrasound Med 37(1):69–82

    Article  PubMed  Google Scholar 

  3. Bhagra A, Tierney DM, Sekiguchi H, Soni NJ (2016) Point-of-care ultrasonography for primary care physicians and general internists. Mayo Clin Proc 91(12):1811–1827

    Article  PubMed  Google Scholar 

  4. Ambasta A, Balan M, Mayette M, Goffi A, Mulvagh S, Buchanan B et al (2019) Education indicators for internal medicine point-of-care ultrasound: a consensus report from the Canadian Internal Medicine Ultrasound (CIMUS) group. J Gen Intern Med 34(10):2123–2129

    Article  PubMed  PubMed Central  Google Scholar 

  5. Davis JJ, Wessner CE, Potts J, Au AK, Pohl CA, Fields JM (2018) Ultrasonography in undergraduate medical education: a systematic review. J Ultrasound Med 37(11):2667–2679

    Article  PubMed  Google Scholar 

  6. Kumar A, Kugler J, Jensen T (2019) Evaluation of trainee competency with point-of-care ultrasonography (POCUS): a conceptual framework and review of existing assessments. J Gen Intern Med 34(6):1025–1031

    Article  PubMed  PubMed Central  Google Scholar 

  7. Nicholls D, Sweet L, Hyett J (2014) Psychomotor skills in medical ultrasound imaging: an analysis of the core skill set. J Ultrasound Med 33(8):1349–1352

    Article  PubMed  Google Scholar 

  8. Tolsgaard MG (2018) Assessment and learning of ultrasound skills in obstetrics and gynecology. Dan Med J 65(2):211

    Google Scholar 

  9. Mason EM, Deal MJ, Richey BP, Baker A, Zeini IM, Service BC et al (2021) Innate arthroscopic and laparoscopic surgical skills: a systematic review of predictive performance indicators within novice surgical trainees. J Surg Educ 78(1):178–200

    Article  PubMed  Google Scholar 

  10. Kramp KH, van Det MJ, Hoff C, Veeger NJ, ten Cate Hoedemaker HO, Pierie JP (2016) The predictive value of aptitude assessment in laparoscopic surgery: a meta-analysis. Med Educ 50(4):409–427

    Article  PubMed  Google Scholar 

  11. Carroll JB (1993) Human cognitive abilities: a survey of factor-analytic studies. Cambridge University Press, Cambridge

    Book  Google Scholar 

  12. Simmons A (2003) Spatial perception from a cartesian point of view. Philos Top 31(1/2):395–423

    Article  Google Scholar 

  13. Vandenberg SG, Kuse AR (1978) Mental rotations, a group test of three-dimensional spatial visualization. Percept Mot Skills 47(2):599–604

    Article  CAS  PubMed  Google Scholar 

  14. Dikmen SS, Heaton RK, Grant I, Temkin NR (1999) Test-retest reliability and practice effects of expanded Halstead-Reitan neuropsychological test battery. J Int Neuropsychol Soc 5(4):346–356

    Article  CAS  PubMed  Google Scholar 

  15. Petrusic WM, Varro L, Jamieson DG (1978) Mental rotation validation of two spatial ability tests. Psychol Res 40(2):139–148

    Article  Google Scholar 

  16. Peters M, Laeng B, Latham K, Jackson M, Zaiyouna R, Richardson C (1995) A redrawn Vandenberg and Kuse mental rotations test: different versions and factors that affect performance. Brain Cogn 28(1):39–58

    Article  CAS  PubMed  Google Scholar 

  17. Wechsler D (2008) Wechsler adult intelligence scale–fourth edition (WAIS–IV). American Psychological Association, San Antonio

    Google Scholar 

  18. Kohs SC (1920) The block-design tests. J Exp Psychol 3(5):357

    Article  Google Scholar 

  19. Elliott DB, Sanderson K, Conkey A (1990) The reliability of the Pelli-Robson contrast sensitivity chart. Ophthalmic Physiol Opt 10(1):21–24

    Article  CAS  PubMed  Google Scholar 

  20. Crawford JE, Crawford DM (1956) Crawford small parts dexterity test. Manual, Rev, vol 1956. American Psychological Association, Washington

    Google Scholar 

  21. Takács Á, Shilon Y, Janacsek K, Tiffin J (1968) The purdue pegboard: examiner manual. Science Research and Associates, Chicago

    Google Scholar 

  22. Verte S, Geurts HM, Roeyers H et al (2005) Executive functioning in children with autism and Tourette’s syndrome. Dev Psychopathol 17:415–445

    Article  PubMed  Google Scholar 

  23. Hempel D, Sinnathurai S, Haunhorst S, Seibel A, Michels G, Heringer F et al (2016) Influence of case-based e-learning on students’ performance in point-of-care ultrasound courses: a randomized trial. Eur J Emerg Med 23(4):298–304

    Article  PubMed  Google Scholar 

  24. Hunter JE, Hunter RF (1984) Validity and utility of alternative predictors of job performance. Psychol Bull 96(1):72

    Article  Google Scholar 

  25. Keehner M, Lippa Y, Montello DR, Tendick F, Hegarty M (2006) Learning a spatial skill for surgery: how the contributions of abilities change with practice. Appl Cogn Psychol 20(4):487–503

    Article  Google Scholar 

  26. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M et al (2015) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 350:g7647

    Article  PubMed  Google Scholar 

  27. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6(7):e1000097

    Article  PubMed  PubMed Central  Google Scholar 

  28. Cook DA, Reed DA (2015) Appraising the quality of medical education research methods: the medical education research study quality instrument and the Newcastle-Ottawa scale-education. Acad Med 90(8):1067–1076

    Article  PubMed  Google Scholar 

  29. Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM (2007) Association between funding and quality of published medical education research. JAMA 298(9):1002–1009

    Article  CAS  PubMed  Google Scholar 

  30. Messick S (1984) The psychology of educational measurement. ETS Research Report Series 1984(1):i–55

    Article  Google Scholar 

  31. Schulze R (2004) Meta-analysis: a comparison of approaches. Hogrefe & Huber, Gottingen

    Google Scholar 

  32. Nagelkerke NJD (1991) A note on a general definition of the coefficient of determination. Biometrika 78(3):691–692

    Article  Google Scholar 

  33. Bell G, Wachira B, Denning G (2016) A pilot training program for point-of-care ultrasound in Kenya. Afr J Emerg Med 6(3):132–137

    Article  PubMed  PubMed Central  Google Scholar 

  34. Berman MC (1986) Report of a study correlating the performance of sonographers with their cognitive abilities. J Diagn Med Sonogr 2(1):13–18

    Article  Google Scholar 

  35. Chung GK, Gyllenhammer RG, Baker EL, Savitsky E (2013) Effects of simulation-based practice on focused assessment with sonography for trauma (FAST) window identification, acquisition, and diagnosis. Mil Med 178(10 Suppl):87–97

    Article  PubMed  Google Scholar 

  36. Janjigian M, Dembitzer A, Srisarajivakul-Klein C, Hardower K, Cooke D, Zabar S et al (2021) Design and evaluation of the I-SCAN faculty POCUS program. BMC Med Educ 21(1):22

    Article  PubMed  PubMed Central  Google Scholar 

  37. Kissin EY, Niu J, Balint P, Bong D, Evangelisto A, Goyal J et al (2013) Musculoskeletal ultrasound training and competency assessment program for rheumatology fellows. J Ultrasound Med 32(10):1735–1743

    Article  PubMed  Google Scholar 

  38. Nielsen DG, Gotzsche O, Sonne O, Eika B (2012) The relationship between immediate relevant basic science knowledge and clinical knowledge: physiology knowledge and transthoracic echocardiography image interpretation. Adv Health Sci Educ Theory Pract 17(4):501–513

    Article  PubMed  Google Scholar 

  39. Schott CK, Kode KR, Mader MJ (2020) Teaching vs learning: impact of deliberate practice and formative feedback on developing point of care ultrasound skills. J Clin Ultrasound 48(8):437–442

    Article  PubMed  Google Scholar 

  40. Werner HC, Vieira RL, Rempell RG, Levy JA (2016) An educational intervention to improve ultrasound competency in ultrasound-guided central venous access. Pediatr Emerg Care 32(1):1–5

    Article  PubMed  Google Scholar 

  41. Stolz LA, Amini R, Situ-LaCasse E, Acuña J, Irving SC, Friedman L et al (2018) Multimodular ultrasound orientation: residents’ confidence and skill in performing point-of-care ultrasound. Cureus 10(11):e3597

    PubMed  PubMed Central  Google Scholar 

  42. Sisley AC, Johnson SB, Erickson W, Fortune JB (1999) Use of an objective structured clinical examination (OSCE) for the assessment of physician performance in the ultrasound evaluation of trauma. J Trauma 47(4):627–631

    Article  CAS  PubMed  Google Scholar 

  43. Woodworth GE, Chen EM, Horn JL, Aziz MF (2014) Efficacy of computer-based video and simulation in ultrasound-guided regional anesthesia training. J Clin Anesth 26(3):212–221

    Article  PubMed  Google Scholar 

  44. Carrigan AJ, Stoodley P, Fernandez F, Sunday MA, Wiggins MW (2020) Individual differences in echocardiography: visual object recognition ability predicts cue utilization. Appl Cogn Psychol 34(6):1369–1378

    Article  Google Scholar 

  45. Chapman E, Hagen S, Gallagher H (2016) Is there a relationship between ultrasound scanning ability (sonography) and visuospatial perception or psychomotor ability? Ultrasound 24(4):214–221

    Article  PubMed  PubMed Central  Google Scholar 

  46. Clem DW, Donaldson J, Curs B, Anderson S, Hdeib M (2013) Role of spatial ability as a probable ability determinant in skill acquisition for sonographic scanning. J Ultrasound Med 32(3):519–528

    Article  PubMed  Google Scholar 

  47. Shafqat A, Ferguson E, Thanawala V, Bedforth NM, Hardman JG, McCahon RA (2015) Visuospatial ability as a predictor of novice performance in ultrasound-guided regional anesthesia. Anesthesiology 123(5):1188–1197

    Article  CAS  PubMed  Google Scholar 

  48. Smith HM, Kopp SL, Johnson RL, Long TR, Cerhan JH, Hebl JR (2012) Looking into learning: visuospatial and psychomotor predictors of ultrasound-guided procedural performance. Reg Anesth Pain Med 37(4):441–447

    Article  PubMed  Google Scholar 

  49. Walker MA, Maccormick MRA, Kilkenny JJ, Phillips J, Singh A, Zur LA (2019) Visuospatial skills are better predictors than dexterity for basic ultrasonographic and fluoroscopic skills in veterinary students. Vet Radiol Ultrasound 60(1):81–92

    Article  PubMed  Google Scholar 

  50. Baker S, Willey B, Mitchell C (2011) The attempt to standardize technical and analytic competence in sonography education. J Diagn Med Sonogr 27(5):203–211

    Article  Google Scholar 

  51. Clem D, Anderson S, Donaldson J, Hdeib M (2010) An exploratory study of spatial ability and student achievement in sonography. J Diagn Med Sonogr 26(4):163–170

    Article  Google Scholar 

  52. Chuan A, Jeyaratnam B, Iohom G, Shorten G, Lee P, Miglani S et al (2021) Using psychometric ability to improve education in ultrasound-guided regional anaesthesia: a multicentre randomised controlled trial. Anaesthesia. https://doi.org/10.1111/anae.15353

    Article  PubMed  Google Scholar 

  53. Duce NA, Gillett L, Descallar J, Tran MT, Siu SC, Chuan A (2016) Visuospatial ability and novice brachial plexus sonography performance. Acta Anaesthesiol Scand 60(8):1161–1169

    Article  CAS  PubMed  Google Scholar 

  54. Frederiksen CA, Juhl-Olsen P, Nielsen DG, Eika B, Sloth E (2012) Limited intervention improves technical skill in focus assessed transthoracic echocardiography among novice examiners. BMC Med Educ 12:65

    Article  PubMed  PubMed Central  Google Scholar 

  55. Hewson DW, Knudsen R, Shanmuganathan S, Ferguson E, Hardman JG, Bedforth NM et al (2020) Effect of mental rotation skills training on ultrasound-guided regional anaesthesia task performance by novice operators: a rater-blinded, randomised, controlled study. Br J Anaesth 125(2):168–174

    Article  PubMed  Google Scholar 

  56. Miller GT, Scerbo MW, Zybak S, Byars DV, Goodmurphy CW, Lattanzio FA et al (2017) Learner improvement from a simulation-enhanced ultrasonography curriculum for first-year medical students. J Ultrasound Med 36(3):609–619

    Article  PubMed  Google Scholar 

  57. Tolsgaard M, Veluppillai C, Gueneuc A, Taksøe-Vester C, Hajal N, Levaillant JM et al (2019) When are trainees ready to perform transvaginal ultrasound? An observational study. Ultraschall Med 40(3):366–373

    Article  PubMed  Google Scholar 

  58. Dromey BP, Ahmed S, Vasconcelos F, Mazomenos E, Kunpalin Y, Ourselin S et al (2021) Dimensionless squared jerk: an objective differential to assess experienced and novice probe movement in obstetric ultrasound. Prenat Diagn 41(2):271–277

    Article  PubMed  Google Scholar 

  59. Schneider J, McGrew K (2012) The Cattell-Horn-Carroll (CHC) model of intelligence v2. 2: a visual tour and summary. Inst Appl Psychom 1:03–13

    Google Scholar 

  60. Booth KL, Reed MJ, Brady S, Cranfield KR, Kishen R, Letham K et al (2015) Training paramedics in focused echo in life support. Eur J Emerg Med 22(6):430–435

    Article  PubMed  Google Scholar 

  61. Cawthorn TR, Nickel C, O’Reilly M, Kafka H, Tam JW, Jackson LC et al (2014) Development and evaluation of methodologies for teaching focused cardiac ultrasound skills to medical students. J Am Soc Echocardiogr 27(3):302–309

    Article  PubMed  Google Scholar 

  62. Fontaine G, Cossette S, Maheu-Cadotte MA, Mailhot T, Deschênes MF, Mathieu-Dupuis G et al (2019) Efficacy of adaptive e-learning for health professionals and students: a systematic review and meta-analysis. BMJ Open 9(8):e025252

    Article  PubMed  PubMed Central  Google Scholar 

  63. Gordon HW (1986) The cognitive laterality battery: tests of specialized cognitive function. Int J Neurosci 29(3–4):223–244

    Article  CAS  PubMed  Google Scholar 

  64. Kozhevnikov M, Hegarty M (2001) A dissociation between object manipulation spatial ability and spatial orientation ability. Mem Cognit 29(5):745–756

    Article  CAS  PubMed  Google Scholar 

  65. Hogan N, Sternad D (2009) Sensitivity of smoothness measures to movement duration, amplitude, and arrests. J Mot Behav 41(6):529–534

    Article  PubMed  Google Scholar 

  66. van Merriënboer JJG (1997) Training complex cognitive skills: a four-component instructional design model for technical training. Educational Technology Publications, Englewood Cliffs

    Google Scholar 

  67. van Merrienboer JJ, Sweller J (2010) Cognitive load theory in health professional education: design principles and strategies. Med Educ 44(1):85–93

    Article  PubMed  Google Scholar 

  68. Kneebone R (2005) Evaluating clinical simulations for learning procedural skills: a theory-based approach. Acad Med 80(6):549–553

    Article  PubMed  Google Scholar 

  69. Olgers TJ, Bij de Weg AA, Ter Maaten JC (2021) Serious Games for Improving Technical Skills in Medicine: Scoping Review. JMIR Serious Games. 9(1):e24093

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We would like to thank Jan Schoones from the Leiden University Medical Center Library for developing the search strategies. Furthermore, we thank Ineke van der Ham, department of Health, Medical, and Neuropsychology, Leiden University for her insight in the neurocognitive mechanisms and Hein Putter, from the Department of Medical Statistics and Bioinformatics, Leiden University Medical Center for supporting the data-analysis.

Funding

There was no funding for this research.

Author information

Authors and Affiliations

Authors

Contributions

TM involved in all levels of the project including study conception, methodological design, data collection, data analysis and interpretation and manuscript preparation. TV, BH, and MP involved in methodological design, data collection, data interpretation and preparation of the manuscript. ED, BB and TO involved in interpretation of the results, preparation and revision of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Tessa A. Mulder.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

There are no financial or non-financial competing interests for any of the authors to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Appendix S1.

Search strategy. Appendix S2. Study characteristics. Appendix S3. MERSQI table. Appendix S4. Aptitude tests.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mulder, T.A., van de Velde, T., Dokter, E. et al. Unravelling the skillset of point-of-care ultrasound: a systematic review. Ultrasound J 15, 19 (2023). https://doi.org/10.1186/s13089-023-00319-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13089-023-00319-4

Keywords