Background : Core competencies of physician practice have been classified into discrete milestones as part of the paradigm shift towards outcomes-based assessment. Assessing unique milestones can be challenging as integration of multiple milestones is required to execute clinical tasks, often defined as Entrustable Physician Activities (EPAs). EPAs are often more practical for assessment, which is especially true for the sub-competencies relating to Interpersonal and Communication Skills (ICS) and Professionalism (PROF).
Objective : Simulation based scenarios integrating simulated patient actors (SPs) are well suited to assess ICS and PROF. Our objective was to develop customized SP based scenarios requiring adeptness in ICS and PROF to serve as a longitudinal series of valid assessments that appropriately match EPAs to residents at various stages of training, while also providing the opportunity for targeted performance feedback.
Methods : Residents completed two EPA assessments matched to their level of training every 6 months. Milestones were mapped to critical actions using published best practices for each EPA as behavioral anchors. After the case, assessment forms were completed by the Faculty facilitator, SP, and resident indicating “yes” or “no” to whether the resident adequately demonstrated the milestone. This was followed by a 10-15 minute debriefing for standardized formative feedback.
Outcomes : Success at demonstrating individual milestones was evaluated by the faculty facilitator, SP, and resident after each case. We found that the consensus between raters for “yes” agreement averaged out to 80% with a standard deviation of 0.09. Content validity was supported in that EM Faculty with expertise in educational methods found the EPAs to be of appropriate breadth and depth to assess for the identified key competencies.