Oral Themed Presentation
The use of high fidelity simulation in high stakes summative OSCEs (Objective Structured Clinical Exams)
Tuesday, May 15
13:30 - 15:00
Location: Neptune 1&2
Context: : High fidelity simulation is used to some extent in almost all medical training programs at this time. Likewise, the use of OSCEs as an assessment tool is commonplace, both internally, as well as for licensing exams. One of the most common implementations of the OSCE model focuses on 5 to 8 minute long encounters using standardized patients. This study uses results from three years of summative OSCE examination data to look at pros and cons of using high fidelity mannequins in high stakes OSCEs.
Description: : In order to allow the inclusion of these simulation stations the administrations required that the high fidelity exam station must conform to the same restrains as the other OSCE stations: the station must be eight minutes, and it must be standardized (each student is exposed to the same scenario/questions/etc.). With this in mind, scenarios are designed to assess students on a straightforward task.
Station development included the following steps:
1) Objectives - Creation of 2-3 straightforward objectives
2) Student Instructions - It is important to keep station instructions brief, and with as limited interpretation to there meaning as possible.
3) Patient chart - The chart is also kept very brief. Name, age, sex, chief complaint, and vitals are included for every patient, other information is only included in the chart if it is deemed essential for completion of the OSCE.
4) Simulation outline - The standard medical school template for high fidelity simulations is used.
5) Assessment rubric - Students are assessed on a limited number of items, each directly tied to one of the station objectives.
Since this is a high stakes examination, each station is tested multiple times using practice volunteers and faculty. The same procedures used with training and rehearsing standardized patients are applied with the simulation techs used to run the station.
Observation/Evaluation: : Survey data showed mixed results. While the use of high fidelity mannequins allows for the assessment of certain skills which are more difficult to assess using standardized patients, there are many challenges to using this type of simulation in a high stakes testing environment. These challenges include technical limitations, added difficulty ensuring a standardized approach, and a possible increase in the number of different directions a student may steer the scenario. The results also point to an increase in the breadth and depth of the competencies that are able to be assessed during these OSCE exams.
Discussion: : Ultimately, the majority of the faculty for this program felt that the gains in the range and depth of items that could be assessed outweighed the difficulties of implementing such OSCE stations. However, a number of steps have been taken to increase the standardization of the testing. Additionally, the number of items assessed on each station has been decreased, which has improved the discriminators for these OSCE stations. The stations continue to be evaluated and adjusted as necessary with each iteration of the exam.