Traditional Poster Round
BACKGROUND: Assessing clinical performance allows educators to determine performance gaps, identify strengths and weaknesses, and to perform needs assessments for future educational interventions. However, clinical performance is a complex entity and is often viewed as having both a process and an outcome component. In clinical practice, outcomes such as rates of central line infection, or hand washing by clinicians are easily measured objectively. Process measurement relates to what a person or team does in a situation and is more challenging to measure. A classic checklist using simple dichotomous items such as done/not done may not be robust enough to evaluate clinical performance. For complex clinical performance tasks, extra layers of checklist assessment are required, such as more categories (done/done incorrectly/not done), weighted items based on importance, and a time component. A modified Delphi process can accommodate these advanced needs for a clinical assessment tool. Specific research questions for this project include 1.) Can the integrative checklist design process provide sound validity evidence for infant respiratory distress? 2.) How does this checklist development process compare qualitatively to other checklist and standard setting procedures? 3.) Does the checklist measure what we want it to measure in an easy to use manner?
METHODS: This is an embedded and sequential mixed methods study following a QUANT->QUAL->QUANT->QUAL sequence. First, an assessment tool will be developed via an integrative modified Delphi approach. This phase will be followed by an embedded QUAL stage meant to examine the assessment tool development process. Next, a QUANT implementation sequence will occur to collect further validity evidence including interrater reliability. Last, a second QUAL phase meant to examine assessment tool ease of use and response process validity. The checklist development process will use an integrated 5-step approach described by Schmutz et al., 2014.
ANALYSIS: Numerous forms of validity evidence will be collected including content validity, construct validity, and internal consistency. The integrative process supports content and construct validity. Interrater reliability as measured by Kappa, will provide data for internal consistency. After the checklist is developed, a survey will be used to evaluate the development process. After the small implementation trial, a second round of qualitative data will be collected via similar means investigating checklist ease of use and response process validity. Information obtained will be used to improve the checklist if needed.
RESULTS: Currently, the checklist development process has occurred via the modified Delphi method. The checklist went through 3 rounds of revisions before consensus was obtained. Checklist item weighing has also been completed. We are in the process of collecting the first stage of QUAL data. We hope to have completed data collection and analysis prior to IPSSW 2019
DISCUSSION: We have developed a clinical assessment tool for infant respiratory distress management by pediatric clerkship students using a modified Delphi approach. To date, the checklist development process has been completed and we are in the process of collecting and analyzing validity evidence and other QUAL measure evaluating ease of use of the checklist as well as ease of the development process itself.