Machine learning to operationalize team cognition: A case study of patient handoffs

Abstract

Pediatric care transitions are important to patient safety, and improved methods to provide feedback about care transitions may lead to improvement. We conducted a secondary analysis of five simulations of the handoff of a pediatric trauma patient from the operating room (OR) to the pediatric intensive care unit (PICU) at a high-fidelity simulation center in the Midwestern United States using practicing clinicians as participants. By synthesizing literature with a qualitative content analysis of the simulation recordings, we identified communication content for these transitions: patient information, anesthesia information, surgical information, nursing information, equipment and technology transfer, patient transfer, professional environment, and simulation. We then used the five simulations to demonstrate use of natural language processing (NLP) models to classify communication content and team cognition behaviors, which could automate real-time evaluation of handoffs. We coded the communication content and behavior of each utterance of the simulations to test and train six support vector machine (SVM) models. Our most frequent communication content codes on average were anesthesia information (19.6 occurrences) followed by surgical information (17.6 occurrences). We found that a SVM model utilizing K-fold stratification with hyperparameter sensitivity analysis best classified communication content and behaviors, with accuracy scores of 74.19% and 54.84%, respectively. We believe our model could be improved through training with a larger dataset from real-life handoffs. Our communication content codes are organized according to what role is likely to provide that information, but handoff protocols must be explicitly team-based to allow team members to share information regardless of role.