Background: Procedural training for skill maintenance is important for patient safety. There are several validated procedural skill evaluation methods for trainees, but these have not been extensively studied for physicians in practice. This study sought to determine whether a checklist and global rating scale previously validated in resident trainees could be validated for use in attending physicians, and evaluate the feasibility of checklist inclusion in ongoing professional practice evaluation (OPPE) procedural workshops
Methods: Study participants were practicing Emergency Medicine (EM) physicians at two academic medical centers. Forty-four eligible physicians consented to take part in the study. Each participant placed a triple lumen central catheters (CL) on a task trainer as part of an OPPE workshop. One in-person rater completed a 30-item checklist and provided a global rating of seven subjective domains of procedural skill. A second rater reviewed videos to provide global ratings only and was blinded to the checklist. Data were analyzed using mean and standard deviation(SD), frequency distribution, t-tests, and Pearson correlations.
Results: The majority of participants (59%) had placed five or less central lines in the past year. Average completion of the checklist were similar between Site 1 (26.04/30 (87%)) and Site 2 (27.24/30 (91%)) (t=-1.823, df=36.74, p=0.077). Average “overall” GRS for rater 1 was 4.39/5 (SD=0.8) and for rater 2 was 3.77/5 (SD=0.5). Total checklist score correlates with rater 1 (r=0.762; p < 0.01) but not with rater 2 (r=0.292; p=0.054) “overall” GRS. Sixty six percent of participants that “passed” (≥ 90% of items on the checklist) the assessment had a significantly higher rater 1 overall GRS score (4.76) than those that did not pass (3.67) (t=4.723, df=19.84, p < 0.001). Participants evaluated the simulation as a good way to refresh skills (94.3% ≥ 4/5 Likert) and reported being likely to attend future sessions (97.1% ≥ 4/5 Likert).
Conclusions: The central line procedural performance of EM attending physicians at these two academic medical centers is very good. The GRS, when assessed concurrently with a checklist, does discriminate between pass and fail on central line procedural skill. Overall the OPPE workshop was a feasible mechanism to allow skill practice and also assessment.