Background: In 2017, the Association of American Medical Colleges introduced the Standardized Video Interview (SVI) for applicants to emergency medicine (EM) residencies. In 2018, the AAMC also prepared free online practice exam modules to assist applicants in preparing for their SVI. Applicants were able to take two different practice tests an unlimited number of times through the AAMC website. These practice tests used retired SVI questions and the same testing interface as the actual SVI, although a resultant score was not provided. Our objective was to define the impact practice tests had on the final SVI score.
Methods: Data was collected from EM applicants in the 2018-19 cycle and merged among those participants who had representation across at least two of the following datasets – SVI participants, practice test participants, and respondents to a post-SVI survey that was administered immediately after the applicant finished the SVI. Correlations, ANOVAs, and multiple regression were calculated.
Results: 3568 individuals were included in the analyses. On average, individuals who completed at least one practice test (N=1058, M=19.19, SD=2.86) or even started a practice test (N=493, M=19.07, SD=2.87) were more likely to have higher scores on the SVI than those who were not matched to any practice test data (N=2017, M=18.57, SD=2.91, p < .05). However, the difference between groups was small (d ranged from .04 to .21). There was no significant difference in SVI score between people who took 1, 2, 3 or 4+ practice tests. SVI score correlated with self-reported ratings of alternative forms of practice for the SVI such as studying interview questions related to the assessed competencies, reading the AAMC’s preparation guide, and rehearsing responses to hypothetical questions (r = .14, p < .001). Self-reported number of types of alternate methods of preparation was a significant predictor of final SVI score beyond number of practice tests completed (adjusted R2 = .007), although very small in magnitude.
Conclusion: These results indicate that completion of multiple practice tests does not necessarily improve SVI score. SVI score seems more likely to be improved by overall knowledge of, and preparation for, SVI content and process using available materials. This information can help to guide advisors and applicants as they approach the SVI.