Award Candidate Poster Presentation
Collecting real-time observational data about teamwork using an iPad tool during simulations and real cases
Tuesday, May 15
09:45 - 10:45
Location: Saturn 1&2
Background: : Questionnaires and interviews are often used to evaluate simulations but such retrospective methods may suffer from observers’ memory limitations and various biases. Ethnographic approaches can highlight details of a practice which would otherwise be difficult to capture. And the use of video research may be powerful but raises complex ethical and legal issues. Both video analysis and ethnography are notably time-consuming. Logging tools on tablet computers may open up interesting opportunities.
Research Question: : The aim of this study was to contribute to the understanding of non-technical aspects of teamwork during simulations as well as in real, acute care cases, by collecting data using live observation data with logging tools on tablet computers. A special focus has been communication and the type of talk that the teams engage in. Research questions: (1) What types of talk do the teams engage in? (2) How much time is spent on different kinds of talk? (3) What is the quality of orders and confirmations? (4) Do observers’ real time observations using a logging tool show agreement and correspond to retrospective questionnaire assessments?
Methodology: : Observational data were collected independently by two observers using the Obansys logging software on iPads during simulations and high priority codes at two major emergency departments in Sweden. A coding scheme focused on frequencies and durations of different kinds of talk as well the number of clear/unclear orders and confirmations. Retrospective assessments were done by the observers using the TEAM instrument (Cooper et al, 2010). Approval was received from the regional ethical review board.
Results: : Preliminary results show that the teams engaged mostly in talk relating to the patient, but the teams varied greatly regarding time spent on planning, evaluating, other talk and silence. The observations revealed improvable behaviors, e.g., clear confirmations of orders were only observed every second time. Retrospective assessments showed high scores on e.g. the item ‘The team acted with composure and control’ but mediocre scores regarding teamwork and communication. Inter-rater agreement was lower in the real-time observations than in the questionnaires.
Discussion/Conclusions: : The real-time observations generate detailed data about types and durations of talk which can be useful input for improving teamwork. And the questionnaires can identify areas requiring more training. While the real time ratings only involved noting behaviors that were directly observable, the questionnaires required generalizations about a team’s performance over time. The latter nevertheless had higher inter-rater agreement. This could be explained by a need for more training of the observers but also by the fact that the observers make different observations from different angles in the room. However, agreement in retrospective assessments may also reflect a systematic bias: observers tend to remember events and behaviors that stand out and base their assessments (generalizations) on these.
Klas Karlgren, PhD
Karolinska Institutet & the Södersjukhuset hospital, Stockholm Sweden, and the Western Norway University of Applied Sciences
Stockholm, Stockholms Lan, Sweden
Fredrik Larsson, MD
Department of Neonatology, Sachs' Children and Youth Hospital, Södersjukhuset
Catherine Allan, MD
Associate Program Director - Simulator Program
Boston Children's Hospital