Abstract
Purpose
Excessive stress experienced by the surgeon can have a negative effect on the surgeon’s technical skills. The goal of this study is to evaluate and validate a deep learning framework for real-time detection of stressed surgical movements using kinematic data.
Methods
30 medical students were recruited as the subjects to perform a modified peg transfer task and were randomized into two groups, a control group (n=15) and a stressed group (n=15) that completed the task under deteriorating, simulated stressful conditions. To classify stressed movements, we first developed an attention-based Long-Short-Term-Memory recurrent neural network (LSTM) trained to classify normal/stressed trials and obtain the contribution of each data frame to the stress level classification. Next, we extracted the important frames from each trial and used another LSTM network to implement the frame-wise classification of normal and stressed movements.
Results
The classification between normal and stressed trials using attention-based LSTM model reached an overall accuracy of 75.86% under Leave-One-User-Out (LOUO) cross-validation. The second LSTM classifier was able to distinguish between the typical normal and stressed movement with an accuracy of 74.96% with an 8-second observation under LOUO. Finally, the normal and stressed movements in stressed trials could be classified with the accuracy of 68.18% with a 16-second observation under LOUO.
Conclusion
In this study, we extracted the movements which are more likely to be affected by stress and validated the feasibility of using LSTM and kinematic data for frame-wise detection of stress level during laparoscopic training. The proposed classifier could be potentially be integrated with robot-assisted surgery platforms for stress management purposes
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anton NE, Montero PN, Howley LD, Brown C, Stefanidis D (2015) What stress coping strategies are surgeons relying upon during surgery? Am J Surg 210:846–851
Arora S, Sevdalis N, Nestel D, Tierney T, Woloshynowych M, Kneebone R (2009) Managing intraoperative stress: what do surgeons want from a crisis training program? Am J Surg 197(4):537–543. https://doi.org/10.1016/j.amjsurg.2008.02.009
Arora S, Sevdalis N, Aggarwal R, Sirimanna P, Darzi A, Kneebone R (2010) Stress impairs psychomotor performance in novice laparoscopic surgeons. Surg Endosc 24(10):2588–2593
Bahdanau D, Cho KH, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: 3rd international conference on learning representations, ICLR 2015 - Conference track proceedings, International conference on learning representations, ICLR, 1409.0473
Berguer R, Smith WD, Chung YH (2001) Performing laparoscopic surgery is significantly more stressful for the surgeon than open surgery. Surg Endosc 15(10):1204–1207
Böhm B, Rötting N, Schwenk W, Grebe S, Mansmann U (2001) A prospective randomized trial on heart rate variability of the surgical team during laparoscopic and conventional sigmoid resection. Arch Surg 136(3):305–310
Boucsein W (2012) Electrodermal activity. Springer, US
Chollet F (2015) Keras. https://github.com/fchollet/keras
Czyzewska E, Kiczka K, Czarnecki A, Pokinko P (1983) The surgeon’s mental load during decision making at various stages of operations. Eur J Appl Physiol 51(3):441–446
DiPietro R, Lea C, Malpani A, Ahmidi N, Vedula SS, Lee GI, Lee MR, Hager GD (2016) Recognizing surgical activities with recurrent neural networks. In: Ourselin S, Joskowicz L, Sabuncu MR, Unal G, Wells W (eds) Medical image computing and computer-assisted intervention - MICCAI 2016. Springer International Publishing, Cham, pp 551–558
Fard MJ, Ameri S, Darin Ellis R, Chinnam RB, Pandya AK, Klein MD (2018) Automated robot-assisted surgical skill evaluation: predictive analytics approach. Int J Med Robot Comput Assisted Surg. https://doi.org/10.1002/rcs.1850
Goodell KH, Cao CG, Schwaitzberg SD (2006) Effects of cognitive distraction on performance of laparoscopic surgical tasks. J Laparoendosc Adv Surg Tech 16(2):94–98. https://doi.org/10.1089/lap.2006.16.94
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Kannan S, Yengera G, Mutter D, Marescaux J, Padoy N (2020) Future-state predicting LSTM for early surgery type recognition. IEEE Trans Med Imag 39(3):556–566
Leonard G, Cao J, Scielzo S, Zheng Y, Tellez J, Zeh HJ, Fey AM (2020) The effect of stress and conscientiousness on simulated surgical performance in unbalanced groups: a Bayesian Hierarchical Model. J Am Coll Surg 231(4):S258. https://doi.org/10.1016/j.jamcollsurg.2020.07.397
Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. In: Proceedings of 2018 10th international conference on knowledge and systems engineering, KSE 2018 pp 25–30, http://arxiv.org/abs/1709.00893, 1709.00893
Martin JA, Regehr G, Reznich R, Macrae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84(2):273–278. https://doi.org/10.1046/j.1365-2168.1997.02502.x
Milenkoski M, Trivodaliev K, Kalajdziski S, Jovanov M, Stojkoska BR (2018) Real time human activity recognition on smartphones using LSTM networks. In: 2018 41st International convention on information and communication technology, electronics and microelectronics, MIPRO 2018 - Proceedings, Institute of Electrical and Electronics Engineers Inc., pp 1126–1131
Moorthy K, Munz Y, Dosis A, Bann S, Darzi A (2003) The effect of stress-inducing conditions on the performance of a laparoscopic task. Surg Endosc Other Interv Tech 17(9):1481–1484
Nammous MK, Saeed K (2019) Natural language processing: Speaker, language, and gender identification with LSTM. In: Advances in intelligent systems and computing, Springer Verlag, vol 883, pp 143–156, https://doi.org/10.1007/978-981-13-3702-4_9
Pandey PS (2017) Machine Learning and IoT for prediction and detection of stress. In: Proceedings of the 2017 17th International conference on computational science and its applications, ICCSA 2017, Institute of electrical and electronics engineers Inc., https://doi.org/10.1109/ICCSA.2017.8000018
Qin Y, Feyzabadi S, Allan M, Burdick JW, Azizian M (2020) daVinciNet: Joint prediction of motion and surgical state in robot-assisted surgery. arXiv http://arxiv.org/abs/2009.11937, 2009.11937
Ryan ED (1962) Effects of stress on motor performance and learning. Research quarterly. Am Assoc Health, Phys Educ Recreat 33(1):111–119
Sielberger C, Gorsuch R, Vagg P, Jacobs G (1983) Manual for the state-trait anxiety inventory (form y)
Tendulkar AP, Victorino GP, Chong TJ, Bullard MK, Liu TH, Harken AH (2005) Quantification of surgical resident stress oncall. J Am College Surg 201(4):560–564
Vedula SS, Malpani A, Ahmidi N, Khudanpur S, Hager G, Chen CCG (2016) Task-level vs. segment-level quantitative metrics for surgical skill assessment. J Surg Educ 73(3):482–489
Wang Z, Majewicz Fey A (2018) Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery. Int J Comput Assisted Radiol Surg. https://doi.org/10.1007/s11548-018-1860-1
Weenk M, Alken AP, Engelen LJ, Bredie SJ, van de Belt TH, van Goor H (2018) Stress measurement in surgeons and residents using a smart patch. Am J Surg 216(2):361–368
Zhang B, Xiong D, Su J (2020) Neural machine translation with deep attention. IEEE Trans Pattern Anal Mach Intell 42(1):154–163. https://doi.org/10.1109/TPAMI.2018.2876404
Zheng Y, Leonard G, Zeh H, Tellez J, Majewicz Fey A (2021) Identifying kinematic markers associated with intraoperative stress during surgical training tasks. In: IEEE International Symposium on Medical Robotics (ISMR), pp 1–7
Acknowledgements
The authors would like to thank the UTSW Training Resident Doctors As Innovators in Science (TARDIS) program, supporting co-Author Grey Leonard. We would like to thank the UTSW Simulation Center director and staff for their invaluable assistance with data collection. We also thank Dr. Chang Su at Weill Cornell Medical College, Cornell University, for his help with understanding deep learning architectures.
Funding
This work was partially supported by the National Science Foundation (NSF) grant number #1846726 and the National Institutes of Health (NIH) grant number #1R01EB030125-01. The UTSW Training Resident Doctors As Innovators in Science (TARDIS) program financially supported co-Author Grey Leonard.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of Interest
The authors declared that they have no conflict of interest.
Availability of Data and Material
Data used in this study could be made available by request.
CodeAvailability
The analysis code used in this study could be made available by request.
Ethics Approval
This experiment was conducted using ethical practices in accordance with the University of Texas at Dallas (#14-57) and UTSW IRB offices (STU #032015-053).
Consent to Participate
Written consent was obtained from the subjects to participate in this study, following IRB guidelines.
Consent to Publish
The consent form for this study included language indicating that this research would potentially lead to a scientific publication.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zheng, Y., Leonard, G., Zeh, H. et al. Frame-wise detection of surgeon stress levels during laparoscopic training using kinematic data. Int J CARS 17, 785–794 (2022). https://doi.org/10.1007/s11548-022-02568-5
Received:
Accepted:
Published:
Version of record:
Issue date:
DOI: https://doi.org/10.1007/s11548-022-02568-5