Award Candidate Poster Presentation


Collecting real-time observational data about teamwork using an iPad tool during simulations and real cases

Tuesday, May 15
09:45 - 10:45
Location: Saturn 1&2

Background: : Questionnaires and interviews are often used to evaluate simulations but such retrospective methods may suffer from observers’ memory limitations and various biases. Ethnographic approaches can highlight details of a practice which would otherwise be difficult to capture. And the use of video research may be powerful but raises complex ethical and legal issues. Both video analysis and ethnography are notably time-consuming. Logging tools on tablet computers may open up interesting opportunities.

Research Question: : The aim of this study was to contribute to the understanding of non-technical aspects of teamwork during simulations as well as in real, acute care cases, by collecting data using live observation data with logging tools on tablet computers. A special focus has been communication and the type of talk that the teams engage in. Research questions: (1) What types of talk do the teams engage in? (2) How much time is spent on different kinds of talk? (3) What is the quality of orders and confirmations? (4) Do observers’ real time observations using a logging tool show agreement and correspond to retrospective questionnaire assessments?

Methodology: : Observational data were collected independently by two observers using the Obansys logging software on iPads during simulations and high priority codes at two major emergency departments in Sweden. A coding scheme focused on frequencies and durations of different kinds of talk as well the number of clear/unclear orders and confirmations. Retrospective assessments were done by the observers using the TEAM instrument (Cooper et al, 2010). Approval was received from the regional ethical review board.

Results: : Preliminary results show that the teams engaged mostly in talk relating to the patient, but the teams varied greatly regarding time spent on planning, evaluating, other talk and silence. The observations revealed improvable behaviors, e.g., clear confirmations of orders were only observed every second time. Retrospective assessments showed high scores on e.g. the item ‘The team acted with composure and control’ but mediocre scores regarding teamwork and communication. Inter-rater agreement was lower in the real-time observations than in the questionnaires.

Discussion/Conclusions: : The real-time observations generate detailed data about types and durations of talk which can be useful input for improving teamwork. And the questionnaires can identify areas requiring more training. While the real time ratings only involved noting behaviors that were directly observable, the questionnaires required generalizations about a team’s performance over time. The latter nevertheless had higher inter-rater agreement. This could be explained by a need for more training of the observers but also by the fact that the observers make different observations from different angles in the room. However, agreement in retrospective assessments may also reflect a systematic bias: observers tend to remember events and behaviors that stand out and base their assessments (generalizations) on these.

Klas Karlgren, PhD

Senior researcher
Karolinska Institutet & the Södersjukhuset hospital, Stockholm Sweden, and the Western Norway University of Applied Sciences
Stockholm, Stockholms Lan, Sweden

Klas Karlgren is a senior researcher at Karolinska Institutet (a medical university) and the Södersjukhuset hospital (the “South central hospital”) in Stockholm, Sweden. He is also an associate professor at the SimArena simulation center at the Western Norway University of Applied Sciences in Bergen, Norway. He holds a Ph.D. (2003) in human-machine interaction and his research lies in the areas of medical education and especially simulation and technology-enhanced learning.


Send Email for Klas Karlgren

Fredrik Larsson, MD

Department of Neonatology, Sachs' Children and Youth Hospital, Södersjukhuset

Fredrik Larsson
Pediatrician, Neonatal Fellow, Simulator instructor at CEPS
Sachsska Children- and Youth Hospital
Stockholm South General Hospital
Karolinska Institutet
I am a Swedish physician and simulator Instructior från Stockholm. My current focus and interest in simulation is the use of high fidility simulation with video assisted feedback in CRM training.


Send Email for Fredrik Larsson

Italo Masiello, Associate professor

Head of the Education and Innovation Center, Södersjukhuset, Stockholm
Karolinska Institutet & Södersjukhuset, Stockholm, Sweden


Send Email for Italo Masiello

Anders Dahlström, MD, Med.Lic.

Department of Neonatology, Sachs' Children and Youth Hospital, Södersjukhuset


Send Email for Anders Dahlström

Catherine Allan, MD

Associate Program Director - Simulator Program
Boston Children's Hospital
Boston, Massachusetts

Catherine Allan, MD is the Associate Program Director of the Simulator Program and Medical Director of the Cardiac Intensive Care Unit at Boston Children’s Hospital. Over the last 10 years she has helped to develop a robust, interdisciplinary program in ECMO Simulation at Boston Children’s Hospital. Dr. Allan has translated this model to other insitutions on both a National and International level to help these programs build new clinical ECMO Programs. She has published on the use of ECMO Simulation Skills Training and on clinical outcomes following ECMO support in the congenital heart disease population and has presented on ECMO simulation at multiple National and International Meetings.


Send Email for Catherine Allan


Collecting real-time observational data about teamwork using an iPad tool during simulations and real cases


Attendees who have favorited this

Please enter your access key

The asset you are trying to access is locked. Please enter your access key to unlock.

Send Email for Collecting real-time observational data about teamwork using an iPad tool during simulations and real cases