The American Board of Internal Medicine (ABIM) changed its certification policy for bedside procedures over a decade ago.1 Acquiring manual competence in abdominal paracentesis, arterial catheter placement, arthrocentesis, central venous catheter placement, lumbar puncture, and thoracentesis is no longer an expectation of residency training. ABIM diplomates should “know” these procedures but not necessarily “do” them. Hospitalists, most of whom are themselves ABIM diplomates, are still, however, expected to do them as core competencies,2perhaps because hospitalists are often available off-hours, when roughly half of bedside procedures are performed.3
Hospitalists increasingly perform bedside procedures with ultrasound guidance.4 Yet training in ultrasound guidance is significantly varied as well,5 simply because point-of-care ultrasound (POCUS) has only recently become widespread.6 And though some skills are transferrable from landmark-guided to ultrasound -guided procedures, many are not.7-10 Furthermore, ultrasound guidance is often not explicitly delineated on the privileging forms used by hospitals,11 even where ultrasound guidance has become standard.12
Therefore, the Society of Hospital Medicine (SHM) Education Committee convened a group of experts and conducted a systematic literature review in order to provide recommendations for credentialing hospitalist physicians in ultrasound-guided bedside procedures. These recommendations do not include training recommendations, aside from recommendations about remedial training for hospitalists who do not pass certification. Training is a means to competence but does not guarantee it. We believe that training recommendations ought to be considered separately.
Working Group Formation
In January 2015, the SHM Board of Directors asked the SHM Education Committee to convene the POCUS Task Force. The purpose of the task force was to develop recommendations on ultrasound guidance for bedside procedures. The SHM Education Committee appointed 3 chairs of the task force: 1 senior member of the SHM Education Committee and 2 POCUS experts. The chairs assembled a task force of 31 members that included 5 working groups, a multispecialty peer review group, and a guideline methodologist (supplemental Appendix 1). Invitation was based on members’ past contributions to SHM POCUS-related activities, up-front commitment, and declared conflicts of interest. Working group members self-identified as “hospitalists,” whereas peer reviewers were nonhospitalists but nationally recognized POCUS physician-leaders specializing in emergency medicine, cardiology, critical care medicine, and anesthesiology. Task force membership was vetted by a chair of the SHM POCUS Task Force and the Director of Education before work began. This position statement was authored by the Credentialing Working Group together with the chairs of the other 4 working groups and a guideline methodologist.
Signed disclosure statements of all task force members were reviewed prior to inclusion on the task force (supplemental Appendix 2); no members received honoraria for participation. Industry representatives did not contribute to the development of the guidelines nor to any conference calls or meetings.
Literature Search Strategy
A literature search was conducted by a biomedical librarian. Records from 1979 to January of 2017 were searched in Medline, Embase, CINAHL, Cochrane, and Google Scholar (supplemental Appendix 3). Search limiters were English language and adults. Articles were manually screened to exclude nonhuman or endoscopic ultrasound applications. Final article selection was based on working group consensus.
The Credentialing Working Group drafted initial and ongoing certification pathways (Figure 1 and Figure 2). The other 4 working groups from the task force were surveyed about the elements and overall appropriateness of these draft pathways. This survey and its results have already been published.12 The Credentialing Working Group then revised the certification pathways by using these survey results and codified individual aspects of these pathways into recommendations.
Development of Position Statement
Based on the Grading of Recommendation Assessment Development and Evaluation methodology, all final article selections were initially rated as either low-quality (observational studies) or unclassifiable (expert opinion).16 These initial ratings were downgraded further because of indirectness, because none of the articles involved the intervention of interest (a credentialing pathway) in a population of interest (hospitalists) measuring the outcomes of interest (patient-level outcomes).17 Given the universal low-quality evidence ratings, we altered the task force strategy of developing guidelines, which the other 4 working groups are writing, and instead developed a position statement by using consensus gathering in 3 steps.
First, the Credentialing Working Group drafted an initial position statement composed of recommendations for credentialing pathways and other general aspects of credentialing. All final article selections were incorporated as references in a draft of the position statement and compiled in a full-text compendium. Second, feedback was provided by the other 4 task force working groups, the task force peer reviewers, and the SHM Education Committee. Feedback was incorporated by the authors of this statement who were the Credentialing Working Group, the chairs of the other 4 working groups, and a guideline methodologist. Third, final suggestions from all members of the SHM POCUS Task Force and SHM Education Committee were incorporated before final approval by the SHM Board of Directors in September 2017.
A total of 1438 references were identified in the original search. Manual selection led to 101 articles, which were incorporated into the following 4 domains with 16 recommendations.
General Credentialing Process
Basic Cognitive Competence Can Be Certified with Written or Oral Examinations
The ABIM defines cognitive competence as having 3 abilities: “(1) to explain indications, contraindications, patient preparation methods, sterile techniques, pain management, proper techniques for handling specimens and fluids obtained, and test results; (2) to recognize and manage complications; and, (3) to clearly explain to a patient all facets of the procedure necessary to obtain informed consent.”1 These abilities can be assessed with written or oral examinations that may be integrated into simulation- or patient-based assessments.18-21
Minimum Thresholds of Experience to Trigger the Timing of a Patient-Based Assessment Should Be Determined by Empirical Methods
Learning curves are highly variable22-25 and even plateaus may not herald basic competence.26 Expert opinions27 can be used to establish minimum thresholds of experience, but such opinions may paradoxically exceed the current thresholds of experts’ own hospitals.12 Thus, empirical methods, such as those based on cumulative sum analysis28-30 or local learning curves,31,32 are preferred. If such methods are not available, a recent survey of hospitalist experts may provide guidance.12 Regardless, once established, minimum thresholds are necessary but not sufficient to determine competency (see “Basic manual competence must be certified through patient-based assessments” section).
Hospitalists Should Formally Log All of Their Attempted Procedures, Ideally in an Electronic Medical Record
Simple self-reported numbers of procedures performed often misrepresent actual experience33,34 and do not include periprocedural complications.35,36 Thus, hospitalists should report their experience with logs of all attempted procedures, both successful and unsuccessful. Such logs must include information about supervising providers (if applicable) and patient outcomes, including periprocedural adverse events,37 but they must also remain compliant with the Health Insurance Portability and Accountability Act.
Health Information Technology Service Should Routinely Pull Collations of All Attempted Procedures from Comprehensive Electronic Medical Records
Active surveillance may reduce complications by identifying hospitalists who may benefit from further training.38 In order to facilitate active surveillance systems, documentation (such as a procedure note) should be both integrated into an electronic medical record and protocol driven,39 including procedure technique, ultrasound findings, and any safety events (both near misses and adverse events).
Basic Manual Competence Must Be Certified Through Patient-Based Assessments
Multiple interacting factors, including environment, patients, baseline skills, training, experience, and skills decay, affect manual competence. Certifications that are based solely on reaching minimum thresholds of experience, even when accurate, are not valid reflections of manual competence,15,40-43 and neither are those based on self-perception.44 Patient-based assessments are, thus, necessary to ensure manual competence.45-48
Certification Assessments of Manual Competence Should Combine 2 Types of Structured Instruments: Checklists and Overall Scores
Assessments based on direct observation are more reliable when formally structured.49,50 Though checklists used in observed structured clinical examinations capture many important manual skills,51-56 they do not completely reflect a hospitalist’s manual competence;57 situations may occur in which a hospitalist meets all the individual items on a checklist but cannot perform an entire procedure with basic competence. Therefore, checklists should be paired with overall scores.58-61 Both checklists and overall scores ought to be obtained from reliable and valid instruments.
Certification Assessments Should Include Feedback
Assessments without feedback are missed learning opportunities.62 Both simulation-63 and patient-based assessments should provide feedback in real time to reinforce effective behaviors and remedy faulty ones.
If Remedial Training is Needed, Simulator-Based Training Can Supplement but Not Replace Patient-Based Training
Supervised simulator-based training allows hospitalists to master basic components of a procedure64 (including orientation to equipment, sequence of operations, dexterity, ultrasound anatomy, and real-time guidance technique) while improving both cognitive and manual skills.42,43,65-71 In addition to their role in basic training (which is outside the scope of this position statement), simulators can be useful for remedial training. To be sufficient for hospitalists who do not pass their patient-based assessments, however, remedial training that begins with simulation must also include patient-based training and assessment.72-75
Initial Credentialing Process
A Minimum Threshold of Experience Should Be Reached before Patient-Based Assessments are Conducted (Figure 1)
Initial Certification Assessments Should Ideally Begin on Simulators
Simulators allow the assurance of safe manual skills, including proper needle insertion techniques and disposal of sharp objects.3,79 If simulators are not available, however, then patient-based training and assessments can still be performed under direct observation. Safe performance of ultrasound-guided procedures during patient-based assessments (without preceding simulator-based assessments) is sufficient to certify manual competence.
Certification to Perform Ultrasound-Guided Procedures Should Be Routinely Re-Evaluated During Ongoing Credentialing (Figure 2)
Observed Patient-Based Assessments Should Occur When a Periprocedural Safety Event Occurs that is Potentially Caused by “Provider Error”
Safety events include both near misses and adverse events. Information about both is ideally “flagged” and “pushed” to hospitalist group leaders by active surveillance and reporting systems. Once reviewed, if a safety event is considered to potentially have been caused by provider error (including knowledge- and skill-based errors),83 then the provider who performed the procedure should undergo an observed patient-based assessment.
Simulation-Based Practice Can Supplement Patient-Based Experience for Ongoing Credentialing
When hospitalists do not achieve a minimum threshold of patient-based experience since the antecedent certification, simulation-based training can supplement their patient-based experience.84 In these cases, however, an observed patient-based assessment must occur. Another consideration is whether or not the privilege should be relinquished because of an infrequent need.
Hospitalists Themselves Should Not Bear the Financial Costs of Developing and Maintaining Training and Certification Programs for Ultrasound-Guided Procedures
Equipment and personnel costs85,86 commonly impede ultrasound-guided procedure programs.4,87,88 Hospitalists whose job descriptions include the performance of ultrasound-guided procedures should not be expected to bear the costs of ultrasound machines, image archival software, equipment maintenance, and initial and ongoing training and certification.
Assessors Should Be Unbiased Expert Providers Who Have Demonstrated Mastery in Performance of the Procedure Being Assessed and Regularly Perform It in a Similar Practice Environment
Assessors should be expert providers who regularly perform the ultrasound-guided procedure in a similar practice environment.9,89-94 For example, providers who are not hospitalists but who are experts in an ultrasound-guided procedure and commonly perform it on the hospital wards would be acceptable assessors. However, a radiologist who only performs that procedure in a fully-staffed interventional radiology suite with fluoroscopy or computed tomography guidance would not be an acceptable assessor. More than 1 assessor may balance idiosyncratic assessments;95 but when assessments are well structured, additional assessors are generally not needed.18Candidate assessors should be vetted by the hospitalist group leader and the hospital privileging committee.
If Intramural Assessors Are Not Available, Extramural Assessors May Be Considered
Intramural assessors are generally preferred because of familiarity with the local practice environment, including the available procedure kits and typical patient characteristics. Nevertheless, extramural assessors27,77,85,96 may theoretically provide even more valid assessments than intramural ones because extramural assessors are neither influenced by relationships with local hospitalists nor biased by local hospitalists’ skills.97,98 Remote performance assessment through video recordings99 or live-video streaming is another option100 but is not sufficient unless a room camera is available to simultaneously view probe movement and the ultrasound screen.101 In addition, remote assessment does not allow the assessor to physically assume control of the procedure to either salvage it or perhaps, in some cases, prevent a complication.
There are no high-quality randomized trials in support of a single credentialing pathway over any other.94,102 The credentialing pathways at the center of this position statement are based on expert opinion. Our methods can be criticized straightaway, therefore, for reliance on the experience and expertise of our working group and task force. Any position statement written without high-quality supportive evidence would be appropriately subject to the same criticism. Without evidence in support of an overall pathway, we codified specific aspects of the pathways into 16 individual recommendations.
Patient-level outcomes do not back these recommendations. Consider, for example, our recommendation that certification assessments be made from structured instruments and not simply from an assessor’s gestalt. Here, the basis is not improved patient-level outcomes from a trial (such as reduced complications or increased procedural success) but improved psychometric performance from reliability studies. The body of evidence for our recommendations is similarly indirect, mostly because the outcomes studied are more proximate and, thus, less meaningful than patient-level outcomes, which are the outcomes of greatest interest but are woefully understudied for clinical competence.17,97,103
The need for high-quality evidence is most pronounced in distinguishing how recommendations should be modified for various settings. Wide variations in resources and patient-mix will make some recommendations impracticable, meaning that they could not be carried out with available resources. For example, our recommendation that credentialing decisions should ultimately rely on certifications made by assessors during patient-based assessments may not be practicable at small, rural hospitals. Such hospitals may not have access to local assessors, and they may not admit enough patients who need the types of ultrasound-guided procedures for which hospitalists seek certification (especially given the need to coordinate the schedules of patients, procedure-performing hospitalists, and assessors). Collaborative efforts between hospitals for regional certification may be a potential solution to consider. But if recommendations are truly impracticable, the task force recognizes they may need to be modified. Given the low quality of evidence supporting our recommendations, such modifications would be readily defendable, especially if they emerged from collaborative discussions between privileging committees, hospitalist directors, and local experts.
One way for hospitals to implement our recommendations may be to follow a recommendation proposed by the authors of the original hospitalist core competencies over a decade ago: “The presence of a procedural skill in the Core Competencies does not necessarily indicate that every hospitalist will perform or be proficient in that procedure.”104 In other words, bedside procedures may be delegated to some but not all hospitalists. Such “proceduralists” would have some proportion of their clinical responsibility dedicated to performing procedures. Delineation of this job description must be made locally because it balances 2 hospital-specific characteristics: patients’ needs for procedures against the availability of providers with basic competence to perform them, which includes hospitalists but also emergency medicine physicians, specialists, and interventional radiologists. A salutary benefit for hospitals is that hospitalists who are not proceduralists would not need to undergo certification in basic competence for the bedside procedures they will not be performing.
Regardless of whether some or all hospitalists at a particular hospital are expected to perform bedside procedures, technology may help to improve the practicability of our recommendations. For example, simulators may evolve to replace actual patient-level experience in achieving minimum thresholds. Certification assessments of manual skills may even someday occur entirely on simulators. Real-time high-definition video streaming enhanced with multiple cameras may allow for remote assessments. Until such advances mature, high-quality patient-level data should be sought through additional research to refine our current recommendations.
We hope that these recommendations will improve how basic competence in ultrasound-guided bedside procedures is assessed. Our ultimate goal is to improve how hospitalists perform these procedures. Patient safety is, therefore, considered paramount to cost. Nevertheless, the hospital administrative leaders and privileging committee members on our Task Force concluded that many hospitals have been seeking guidance on credentialing for bedside procedures, and the likely difficulties of implementing our recommendations (including cost) would not be prohibitive at most hospitals, especially given recognition that these recommendations can be tailored to each setting.
Collaborators from SHM POCUS Task Force are Saaid Abdel-Ghani, Michael Blaivas, Dan Brotman, Carolina Candotti, Jagriti Chadha, Joel Cho, Ria Dancel, Ricardo Franco, Richard Hoppmann, Susan Hunt, Venkat Kalidindi, Ketino Kobaidze, Josh Lenchus, Benji Mathews, Satyen Nichani, Vicki Noble, Martin Perez, Nitin Puri, Aliaksei Pustavoitau, Sophia Rodgers, Gerard Salame, Daniel Schnobrich, Kirk Spencer, Vivek Tayal, Jeff Bates, Anjali Bhagra, Kreegan Reierson, Robert Arntfield, Paul Mayo, Loretta Grikis.
Brian P. Lucas received funding from the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development and Dartmouth SYNERGY, National Institutes of Health, and National Center for Translational Science (UL1TR001086). Nilam Soni received funding from the Department of Veterans Affairs, Quality Enhancement Research Initiative (QUERI) Partnered Evaluation Initiative (HX002263-01A1). The contents of this publication do not represent the views of the United States Department of Veterans Affairs or the United States Government.