Original Research

Observations during development of an internal medicine residency training program in cardiovascular limited ultrasound examination




Despite the future potential of using ultrasound stethoscopes to augment the bedside cardiac physical, few data exist on a general cardiovascular imaging protocol that can be taught to physicians on a perpetual basis as a curriculum in graduate medical education.


During the past decade, we developed and integrated a cardiovascular limited ultrasound training program within the confines of an internal medicine residency. The evidence‐based rationale for the exam, the teaching methods, and curriculum are delineated, and subsequent observations regarding program requirements, proficiency, and academic outcomes are explored. Analysis of variance and linear regression assessed for relationships between academic scores, chief resident selection, and gender to proficiency in ultrasound.


A brief, 5‐minute cardiovascular limited ultrasound exam (CLUE) was taught using both didactic and bedside methods, and practiced primarily within the cardiology consult, outpatient clinic, and intensive care rotations. Program costs were minimized by employing readily available institutional resources. After a 2‐year lead‐in training phase, the subsequent 4 years of senior resident performance (n = 41 residents) showed an 81% pass rate in CLUE competency. Resident ultrasound performance did not relate to academic scores (r = 0.05, P = 0.75), chief resident selection, nor gender. Observations regarding resident pitfalls in CLUE practice and increased participation in extracurricular research are described.


We report our initial experience in developing and implementing a training program for bedside cardiovascular ultrasound examination that employed evidence‐based techniques, set proficiency goals, and assessed resident performance. It may be feasible to teach future internist–hospitalists the technique of bedside ultrasound during residency. Journal of Hospital Medicine 2012;. © 2012 Society of Hospital Medicine

Copyright © 2012 Society of Hospital Medicine

Although the advent of small ultraportable bedside ultrasound devices have heralded the age of the ultrasonic stethoscope,15 realizing the widespread potential of ultrasound‐assisted physical examination68 requires the creation of an imaging protocol that can be successfully taught to all physicians within the confines of accredited medical education. Prior feasibility studies of teaching internal medical residents are characterized by heterogeneity in imaging protocols, nonrandomized enrollment of a small number of trainees, and training that is short‐lived,6, 914 making their results difficult to generalize. Few data exist on the effects of sustained incorporation of a comprehensive, structured program within a conventional 3‐year internal medicine residency.

Over the past 14 years, we have developed cardiovascular limited ultrasound examination (CLUE), with the specific purpose of detecting prevalent cardiovascular pathologies that: (1) have been shown to affect morbidity and mortality in an adult population, (2) are often missed by physical examination, and (3) have been detected by medical residents who have been taught a simplified ultrasound examination. In this report, we will detail our observations regarding CLUE and its training curriculum with assessment of proficiency, program requirements, and the overall academic effect once firmly integrated into an internal medicine residency program.


Setting and Participants

The ultrasound training program was created at Scripps Mercy Hospital San Diego Campus, a 500‐bed community hospital in San Diego, California, for integration into a 3‐year internal medicine residency program. It was accredited by the Accreditation Council for Graduate Medical Education (ACGME) and consisted of approximately 33 residents, and 23 full‐time and 82 part‐time faculty. Since 2005, all internal medicine residents have been participating in the ultrasound training program and their progress followed as a part of the ACGME Educational Innovation Project. Of the 41 consecutive graduating residents in whom performance data were collected, no resident had prior formal training in ultrasound.

Program Overview

Based upon initial studies of performing limited echo examination,1520 the following imaging protocols were combined to comprise CLUE, a brief, quick‐look two‐dimensional multi‐targeted ultrasound examination: (1) the extracranial carotid bulb for carotid atherosclerosis, (2) parasternal long‐axis view for left ventricular systolic dysfunction and left atrial enlargement, (3) apical lung views for interstitial edema, (4) basal lung views for pleural effusion, (5) a subcostal 4‐chamber view for isolated right ventricular enlargement or pericardial effusion, (6) the longitudinal view of the inferior vena cava for elevated central venous pressures, and (7) a mid‐abdominal longitudinal view for abdominal aortic aneurysm. Evidence‐basis for the exam targets and specifics of subjective diagnostic CLUE criteria (Table 1) have been published elsewhere.2130

CLUE Diagnostic Criteria and Commonly Observed Pitfalls
DiseaseDiagnostic CriteriaPitfalls
  • NOTE: The CLUE ultrasound targets are listed (left column) with the corresponding subjective diagnostic 2‐dimensional criteria (middle column) and corresponding pitfalls observed during the training program (right column). Abbreviations: AP, anterior‐posterior; CLUE, cardiovascular limited ultrasound exam; COPD, chronic obstructive pulmonary disease; FPs, false positives; IVC, inferior vena cava; LA, left atrium; LV, left ventricle; PLAX, parasternal long axis; RV, right ventricle; SN, sensitivity.

1. Carotid atheromaFocal thickened/calcified region of plaque22Reduced SN for isoechoic clot or dissection; not for use in acute neurologic syndromes
2. LV systolic dysfunctionMitral anterior leaflet tip does not approach septum (<1 cm) in diastole21, 23, 26Reduced SN for acute or apical wall motion abnormalities; FPs due to severe aortic regurgitation, mitral stenosis
3. Left atrial enlargementLA appears larger than aortic root (AP diameter) throughout the cardiac cycle21, 2426Reduced SN when LA asymmetrically enlarges (elongates); FPs due to far field artifact mistaken for posterior LA wall.
4. Lung comet‐tail artifactThree or more linear artifacts extending from pleura to the far field, moving with respiration26Reduced SN when probe not tilted to scan perpendicular to convex apical lung surface or imaging during inspiration only. Apical comets can be present in COPD with subclinical interstitial disease
5. Pleural effusionAnechoic region above the diaphragm and below lung27, 28Reduced SN for small effusions when probe not placed posterior enough. FPs of ascites or gastric fluid
6. Pericardial effusionAnechoic region seen deep to LV and above descending aorta in PLAX,15 or between the liver and RV in the subcostal view27FPs of an epicardial fat pad or right pleural effusion. A large effusion and dilated IVC are mandatory in the consideration of tamponade by the resident
7. RV enlargementSize (AP diameter) of the RV appears equal or greater than the LV29.Reduced SN due to lack of imaging during a deep inspiration or due to off‐axis imaging
8. IVC plethoraIVC AP diameter equals or exceeds the same‐level aortic diameter and fails to reduce size with respiration14, 26, 30Reduced SN when mistaking a hepatic vein for the IVC. FP when mistaking the descending aorta for a dilated IVC, particularly when IVC is collapsed.
9. Abdominal aortic aneurysmFocal dilation 1.5 the size of neighboring segment21Reduced SN due to bowel gas or mistaking a normal IVC for the aorta. FPs of cysts identified as aneurysmal disease

Two useful mnemonics were created to teach the imaging protocol. If using only the 3 MHz cardiac probe, residents were taught to work backward against the flow of blood, in regards to physiologic effects and the sequence of CLUE views. Starting in the left ventricle, systolic function was first evaluated, followed by left atrial enlargement, the presence of lung comets, then lung effusions, then right ventricular enlargement, the presence of pericardial effusion, then elevation of central venous pressures. If the high‐frequency 5 MHz linear probe was available for carotid imaging, then an additional mnemonic was remembered that atherosclerotic progression increased from top to bottom in CLUE, typified by the frequent detection of early disease in the carotid bulb, then occasional cardiac manifestations, followed by the infrequent late manifestation of an abdominal aortic aneurysm. In our practice, performance of the complete CLUE starting at the top (carotids), changing transducers to work backward in the thorax (cardiac, lung, and inferior vena cava), and finishing with the bottom (aorta) was often dependent upon equipment and linear probe availability at the point‐of‐care.

A formalized CLUE curriculum was implemented into the residency in 2006. Twelve monthly 1‐hour CLUE lectures were given per year. Most lectures were 3045 minutes in length, leaving 1530 minutes for imaging resident or patient volunteers. All forms of ultrasound devices available to the residents, including pocket‐sized, hand‐carried, cart‐based, and standard ultrasound machines, were used in this forum. To learn the fundamentals of imaging technique, the intern during the cardiology consultation month rotation was first expected to image 1030 patients in the echocardiography and vascular ultrasound labs under the tutelage of the sonographers. Once weekly, 1‐hour bedside teaching was given to junior and senior residents on the intensive care unit (ICU) and cardiology consult rotations, in a traditional case‐based format. Over the ICU month rotation, junior and senior residents could each image an additional 1030 patients, resulting in a minimum of 30 studies obtained on acutely ill patients during the ICU rotations of residency. During clinical care rotations over the 3‐year residency, all residents imaged a minimum of 30 patients (at least 10 proctored studies during their internship cardiology consultation month, 10 proctored during ICU junior year rotations, and 10 proctored during ICU senior year rotations), with some residents imaging over a hundred patients (Table 2). To assist their education in CLUE, multiple learning aides were made available, including instructional how‐to‐image videos, a 200‐page syllabus, self‐assessment tests, and an instructional web site. Overall, the independent study and performance of CLUE was encouraged, but without formal performance incentives, monitoring, or effect upon residency evaluations.

Summary of Resident Curriculum and Estimates of Hours Spent
  • NOTE: CLUE curriculum (lectures and bedside teaching, imaging opportunities, and extracurricular) as noted by postgraduate year as provided. Estimated hours typically observed by faculty summarized at bottom and account for excused absences due to mandatory resident hour limitations, vacations, and away rotations. Abbreviations: CHF, congestive heart failure; CLUE‐CEX, cardiovascular limited ultrasound exam‐clinical exercise; ED, emergency department; ICU, intensive care unit; PGY, postgraduate year.

PGY‐1 (intern)12 (1‐hr) conferences; Web site instruction; syllabus; 12 (1‐hr) bedside ICU roundsEcho lab imaging with 20 (10 proctored) studies on cardiology consults; outpatient cardiology clinicsResearch; imaging in ICU, CHF, and medical clinics; ED
PGY‐2 (junior)12 (1‐hr) conferences; Web site instruction; syllabus; 8 (1‐hr) bedside ICU rounds20 (10 proctored) during 2 ICU rotationsResearch; teaching others; imaging in CHF and medical clinics; ED; echo lab
PGY‐3 (senior)12 (1‐hr) conferences; Web site instruction; syllabus; 12 (1‐hr) bedside ICU rounds20 (10 proctored) during 2 ICU rotations, cardiology consults, echo labResearch; teaching others; imaging in CHF and medical clinics; ED; CLUE‐CEX
Time completed (estimate)50 hr60 cases (30 proctored)

At our institution, the medical director of the Echocardiography and Vascular ultrasound laboratory was a cardiologist (B.J.K.) who directed the CLUE training program. The Director provided the monthly lecture series to the entire residency and was responsible for weekly 1‐hour bedside ICU rounds. If given maintenance responsibilities of weekly bedside ICU rounds (1 hour/week), monthly lecture and preparation (5 hours/month), and availability to teach the cardiology intern (3 hours/month) and maintain the Web site (4 hours/month), the program required 4 hours/week of the Director's time. The program used 3 dedicated devices: the SonoSite 180 (SonoSite, Inc, Bothell, WA), the MicroMaxx (SonoSite, Inc) and, in 2010, a pocket‐sized cardiac ultrasound stethoscope, the Vscan (GE Healthcare, Wauwatosa, WI). No patient charges were submitted for performance or interpretation of any CLUE.

Assessment and Follow‐Up

A proficiency test was performed at the end of each resident's senior year. The test, cardiovascular limited ultrasound exam‐clinical exercise (CLUE‐CEX), involved imaging any available, consenting patient and assessing the resident's technical skills by image quality, knowledge of diagnostic criteria, and ability to discuss the clinical aspects of potential findings in a question‐and‐answer oral interview format, typically requiring 2030 minutes to perform. Each resident CLUE view was rated for: (1) image quality which accounted for 44% of total exam points, (2) specific knowledge related to each view which accounted for 28% of total exam points, and (3) diagnostic accuracy of the interpretation of each view which accounted for 28% of total exam points (see Figure 1). CLUE‐CEX scores were recorded as a percentage of total possible points, normalized to the difficulty of imaging the individual patient as determined by the Director's imaging. The test encompassed performance of all 7 views, demonstrated in 2 exams employing 2 transducers (cardiac and vascular) on the same patient (Figure 1). A passing threshold had been empirically derived at >80% of the total available points, a value that: (1) required performance in all 3 categories, (2) subjectively correlated to competency when assessed by the Director, and (3) had parity with other thresholds of clinical skill assessment by faculty and in graduate education. The Director had no knowledge of non‐CLUE resident evaluations, In‐training scores, or academic performance outside of CLUE. Residents were not remanded for CLUE‐CEX failure.

Figure 1

Cardiovascular limited ultrasound exam‐clinical exercise (CLUE‐CEX) form.

The graduating class of 2011 was the first class to initially enter into an entire residency program fully immersed in the CLUE curriculum, and was therefore specifically asked to report their impression of the CLUE program after graduation through a post‐residency questionnaire. A Likert‐type scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree) was used to assess the perceived validity of the following statements: (1) CLUE improved my own bedside cardiovascular evaluation; and (2) I would use CLUE if ultrasound were available in my future position. Each resident was then asked if too much, not enough, or an appropriate amount of time was spent to learn CLUE, and to choose the most effective form of CLUE teaching to which they were exposed: didactic lectures, bedside ICU teaching, Web site/syllabus, and one‐to‐one training with the Director or sonographer.

Statistical Analysis

The CLUE experience was divided into 3 phases: (1) pre‐CLUE era, the 4‐year period (classes graduating 20022005) prior to the institution of the formal CLUE curriculum; (2) the 2‐year CLUE phase‐in period (classes graduating 20062007), in which portions of the residency were undergoing the 3‐year curriculum; (3) the 4‐year CLUE‐CEX era (classes graduating 20082011) when all residency classes were trained in the standardized fashion and underwent CLUE‐CEX assessment. In‐training postgraduate year‐3 (PGY‐3) scores, the result of a nationwide standardized test developed by the American College of Physicians, were used as representative of senior resident academic knowledge. A percentile rank score is provided to compare residents to nationwide data. The group of residents who had been selected to be the following year's chief residents had their CLUE‐CEX scores analyzed as a subgroup.

Data are presented as mean standard deviation and analyzed in SPSS, version 12.0 (SPSS, Inc, Chicago, IL). Linear regression was used to investigate the relationship between In‐training percentile ranks and CLUE‐CEX scores. Analysis of variance was used to determine any effect of gender and chief resident selection on CLUE‐CEX, and to assess average resident In‐training percentile ranks during the pre‐CLUE and CLUE‐CEX periods. Subset analysis of individual CLUE‐CEX scores was performed in regards to image quality, diagnostic knowledge, and interpretative skills. A value of P < 0.05 was considered significant.


Observations During CLUE Program Development

CLUE‐CEX scores (20082011) included data from 41 residents; 51% were male. In the class of 2009, one second‐year male resident transferred to another program for nonacademic reasons, reducing its number to 9. We observed that the impact of the CLUE program depended in part upon resident‐to‐resident teaching and required a critical mass of residents to be trained during a phase‐in period before a maximal effect could be appreciated. We observed that didactic knowledge occurred before imaging skills and remained dominant by graduation, with mean percentile CLUE‐CEX scores for image quality, knowledge, and interpretative accuracy at 82% 5%, 91% 3%, and 91% 8%, respectively. Residents typically found apical lung imaging the easiest to perform (CLUE‐CEX score of 89% 19%), followed by carotid (84% 18%), inferior vena cava (IVC) imaging (84% 26%), screening for abdominal aortic aneurysm (AAA) (83% 2 4%), parasternal long‐axis (79% 30%), and subcostal cardiac 4‐chamber imaging (73% 33%). Each view had technical and diagnostic pitfalls that were noted during resident practice (see Table 1), resulting in changes in our teaching and case review in subsequent years.

Residency and CLUE Performance

In attempting to achieve a CLUE proficiency score of >80% on the CLUE‐CEX in their graduating year, 8/41 (19.5%) senior residents failed. In these 8 residents, imaging quality, knowledge, and interpretative accuracy were all depressed: 55% 19%, 79% 11%, and 75% 11%, respectively. Two of these 8 had been selected as future chief residents over the 4‐year period, positions typically awarded to 2 residents per graduating year. The performance of the residents is seen in Table 3. The CLUE program did not exert a negative effect upon the academic performance of the residency, as evidenced by the lack of a significant difference in the Pre‐CLUE, 2‐year CLUE, and CLUE‐CEX periods in regards to average resident In‐training percentile rank scores (67.5 20.1, 62.3 20.5, 69.4 16.9, respectively; P = 0.37).

Resident Performance
Time Era (Year of Graduation)nFail RateCLUE‐CEX (Mean SD)Resident IT Percentile Rank (Mean SD) (Range)
  • NOTE: Table shows mean standard deviation of CLUE‐CEX scores and resident In‐training percentile rank which represents the average of the residents' national percentile ranks of their In‐training PGY‐3 total scores during the corresponding time (Pre‐CLUE 4 years, CLUE phase‐in 2 years, CLUE‐CEX 4 years). Fail rate represents the % of residents who did not pass the CLUE‐CEX (80% correct criterion). Yearly data is listed for each of the CLUE‐CEX years, 20082011. Year denotes the year of graduation. Abbreviations: CLUE‐CEX, cardiovascular limited ultrasound exam‐clinical exercise; IT, In‐training; PGY, postgraduate year; SD, standard deviation.

Pre‐CLUE (20022005)39 67.5 20.1 (2099)
Phase‐in CLUE (20062007)19 62.3 20.5 (2097)
CLUE‐CEX (20082011)4119%87.4 11.969.4 16.9 (3499)
Year 20081136%84.3 13.974.7 17.9 (4599)
Year 2009911%89.1 7.073.0 16.6 (3493)
Year 20101030%84.2 16.957.1 12.7 (4287)
Year 2011110%92.1 5.772.4 15.9 (3499)

Figure 2 shows the relationship between CLUE‐CEX scores and In‐training PGY‐3 scores. There was no significant relationship between resident academic performance and CLUE capabilities (r = 0.05, P = 0.75). Similarly, chief resident performance (n = 14) was not significantly associated with CLUE‐CEX scores (r = 0.15, P = 0.37), nor was male gender (P = 0.07). Approximately one‐half (49%) of the residents in the 4‐year CLUE‐CEX era entered fellowships, unchanged from historic rates, with only 1 resident during this era entering into a cardiology fellowship.

Figure 2

Relationship between CLUE and academic performance. graph of CLUE‐CEX versus In‐training PGY‐3 percentile ranks. Trendline is shown (r = 0.051, P = 0.75) and demonstrates no significant correlation. Abbreviations: CLUE‐CEX, cardiovascular limited ultrasound exam‐clinical exercise; PGY‐3, postgraduate year‐3.

The Likert‐type questionnaire was returned by 11/11 graduating residents in 2011. Mean score of 4.3 0.6 (range: 35), with 6/11 responding agree, was given for the statement of whether CLUE improved the resident's own bedside exam. A score of 4.5 0.7 (range: 35), with 7/11 responding strongly agree, was given for whether the resident would use CLUE in the future if ultrasound were available. The majority (9/11) of residents felt that the time spent on CLUE was appropriate, with 2 residents responding not enough. Residents ranked one‐to‐one training with the Director(n = 6), followed by bedside ICU rounds (n = 5) as the preferred teaching methods to learn CLUE.


We report the experience of enrolling 6 consecutive classes, in an internal medicine residency, to test the feasibility of incorporating ongoing training in a specific, evidence‐based cardiovascular limited ultrasound examination within an already existing 3‐year curriculum. Using unbiased and complete enrollment, we found that residents who perform well on standardized academic testing or who are selected as chief residents do not necessarily perform more competently in CLUE, and that a significant overall initial resident failure rate can be anticipated. By questionnaire, residents felt confident in using the technique to improve their future bedside exams.

Burgeoning interest in the limited or focused application of ultrasound during bedside evaluation has already resulted in the incorporation of ultrasound training into emergency medicine residencies and critical care fellowships, with minimal standardization on curriculum, teaching methodology, or competency requirements. Given the multiple subspecialty applications for ultrasound, the potential exists of excessive diversity in bedside ultrasound practice, weakening the development of a single, simplified exam technique as a clinical tool for all physicians.31 Prior feasibility studies914 have evaluated the learning curve of internal medicine or primary care residents in performing various limited exams, but have not provided the rationale regarding the imaging protocols, the methods used for teaching, and the assessment of the program results over a sustained period of time. Furthermore, prior studies have not randomized subject trainees, likely resulting in the selected enrollment of highly motivated or skilled residents who want to perform a particular technique or have a bias to learn it. Our reported 19% unremanded failure rate on CLUE‐CEX will likely be more reflective of the general experience when initially integrating entire classes of internal medicine residents into a standard curriculum. The feasibility of introducing ultrasound at an earlier stage than residency may improve familiarity with the modality, and a 4‐year medical‐student curriculum has been recently described.32 Although introduction in medical school could allow for more adept and specific clinical training during residency, the optimal time for education in bedside ultrasound remains unclear.

Critical to the development of our program was the necessity to commit to teaching a single exam, the CLUE. We derived CLUE to quickly screen for important targets that had evidence‐basis to affect outcome, such as manifestations of subclinical atherosclerosis or chamber enlargement due to elevated filling pressures. Subsequent CLUE outcome studies have demonstrated diagnostic accuracy and prognostic value in its components,18, 26, 29, 30 and an effect upon medical decision‐making,21 even when performed by briefly trained novices.18, 21, 30 It is anticipated that this cardiovascular examination will later expand to a more advanced version or become a component of a full‐body ultrasound‐assisted physical. Therefore, evidence‐basis and brevity governed the development of a practical and teachable fundamental CLUE, and our skill assessment results are likely specific to CLUE itself.

This report contains primarily observations noted during the development of our program, written in retrospect with emphasis on real world feasibility. It was not a rigorous evaluation of specific ultrasound teaching methods. We found that training is feasible, at modest costs, when existing in‐hospital resources are utilized and include a part‐time faculty appointment and shared devices. Training the sonographers to perform CLUE as a part of the standard echocardiogram was a trivial task, but created the great benefit of being able to retrospectively review both the CLUE and formal echo in case review and teaching. Monthly CLUE lectures in the daily noon conference docket, and the use of the cardiology consultation and ICU rotations, allowed integration of the CLUE curriculum into preexisting venues and persistent practice opportunities within the residency. To prevent bias, we intentionally did not track, bring attention to, or incentivize resident performance in CLUE over any other topic; therefore, we can only approximate lecture and bedside teaching hours spent by each resident in light of detractions due to residency hour restrictions, vacations, and away rotations (Table 2). The CLUE‐CEX, although subject to the biases of any subjective resident skill assessment, was easily accomplished using a single form and faculty member, and was an efficient tool for program feedback and development.

In conclusion, we report the feasibility of sustained incorporation of an ultrasound training program in an internal medicine residency. We await studies regarding clinical outcome and validation of similar experiences in larger, multicenter programs.


The authors acknowledge the sonographers of the Scripps Mercy Cardiovascular Ultrasound Laboratory and Dudie Keane, for their dedication and assistance in the implementation of the CLUE program.

Disclosure: Nothing to report.

Note: The correction that was made, was the text in Fig. 1 and Fig. 2 were reversed. This article was published online on May 17, 2012. An error was subsequently identified. This notice is included in the online and print versions to indicate that both have been corrected on May 22, 2012.


Online-Only Materials

Microsoft Office document icon Supporting Information (1)69.5 KB
   Comments ()