Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 16.06.20 in Vol 7, No 2 (2020): Apr-Jun

This paper is in the following e-collection/theme issue:

    Original Paper

    Comparison of the Effects of Automated and Manual Record Keeping on Anesthetists’ Monitoring Performance: Randomized Controlled Simulation Study

    1Department of Applied Psychology, Lingnan University, Hong Kong, China (Hong Kong)

    2Department of Anaesthesia and Intensive Care, Tuen Mun Hospital, Hong Kong, China (Hong Kong)

    Corresponding Author:

    Simon Y W Li, BSc, MSc, PhD

    Department of Applied Psychology

    Lingnan University

    WYL-306, WYL Building

    8 Castle Peak Road, Tuen Mun

    Hong Kong,

    China (Hong Kong)

    Phone: 852 26167129

    Email: simonli2@ln.edu.hk


    ABSTRACT

    Background: Anesthesia information management systems (AIMSs) automatically import real-time vital signs from physiological monitors to anesthetic records, replacing part of anesthetists’ traditional manual record keeping. However, only a handful of studies have examined the effects of AIMSs on anesthetists’ monitoring performance.

    Objective: This study aimed to compare the effects of AIMS use and manual record keeping on anesthetists’ monitoring performance, using a full-scale high-fidelity simulation.

    Methods: This simulation study was a randomized controlled trial with a parallel group design that compared the effects of two record-keeping methods (AIMS vs manual) on anesthetists’ monitoring performance. Twenty anesthetists at a tertiary hospital in Hong Kong were randomly assigned to either the AIMS or manual condition, and they participated in a 45-minute scenario in a high-fidelity simulation environment. Participants took over a case involving general anesthesia for below-knee amputation surgery and performed record keeping. The three primary outcomes were participants’ (1) vigilance detection accuracy (%), (2) situation awareness accuracy (%), and (3) subjective mental workload (0-100).

    Results: With regard to the primary outcomes, there was no significant difference in participants’ vigilance detection accuracy (AIMS, 56.7% vs manual, 56.7%; P=.50), and subjective mental workload was significantly lower in the AIMS condition than in the manual condition (AIMS, 34.2 vs manual, 46.7; P=.02). However, the result for situation awareness accuracy was inconclusive as the study did not have enough power to detect a difference between the two conditions.

    Conclusions: Our findings suggest that it is promising for AIMS use to become a mainstay of anesthesia record keeping. AIMSs are effective in reducing anesthetists’ workload and improving the quality of their anesthetic record keeping, without compromising vigilance.

    JMIR Hum Factors 2020;7(2):e16036

    doi:10.2196/16036

    KEYWORDS



    Introduction

    An anesthesia information management system (AIMS) is a computer-based system that automatically imports real-time vital signs from physiological monitors to replace traditional handwritten records [1] and is increasingly being adopted by hospitals [2]. Despite the increasing popularity of AIMSs, recent studies on AIMSs mainly addressed the completeness of anesthetic records [3,4] but not the other attributes that are central to anesthetists’ monitoring performance, such as situation awareness and mental workload. The purpose of this paper was to report a full-scale high-fidelity simulation that compared the effects of AIMS use and manual record keeping on anesthetists’ monitoring performance.

    Vigilance is the ability to maintain sustained attention over a long period of monitoring [5]. The most recent studies examining the effect of automated record keeping on vigilance were conducted 20 years ago [6,7]. Those studies focused on visual vigilance, which was operationalized as the time taken by participants to detect visual stimuli, including simulated abnormal values on a patient monitor [6] and flashing of an alarm light [7]. Anesthetists’ vigilance was not affected when record keeping was carried out by machines or assistants [6].

    Situation awareness refers to one’s mental representation of the status of a dynamically changing environment. Situation awareness is measured at the following three levels: perception (level 1), comprehension (level 2), and projection (level 3) [8]. Situation awareness is critical to the administration of anesthesia because anesthetists need to monitor and be aware of numerous patient physiological variables (perception), detect unstable conditions and intervene appropriately (comprehension), and anticipate the effects of the intervention (projection) [9]. Situation awareness affects and is affected by mental workload, which is characterized as a subjective experience of the level of attentional demands imposed by performing tasks [10]. Noel suggested that anesthetists might become less attentive to the details of anesthetic events and patients’ status when they do not have to scan patients’ vital signs and write them down, as required in manual charting [11].

    An AIMS would change the role of anesthetists from active processers of information to passive recipients [12,13]. As a result, anesthetists might be less attentive to the operating room (OR) surroundings and their patients’ status during monitoring. However, an AIMS is expected to reduce anesthetists’ subjective mental workload. Our three hypotheses specify that when compared with anesthetists who use manual record keeping, anesthetists who use AIMSs would have lower vigilance detection accuracy (H1), would have lower situation awareness accuracy (H2), and would experience lower subjective mental workload (H3).


    Methods

    Study Design and Approval

    A parallel group experimental design was employed in this study. Ethical approval was obtained from Tuen Mun Hospital (TMH) (NTWC/CREC/17065) and Lingnan University (EC-063/1617). Written informed consent was obtained from all participants in advance and their data were deidentified.

    Participants

    Participants were recruited from among the members of the Anaesthesia and Intensive Care Unit, TMH between September 2017 and March 2018. Participants were eligible if they were resident trainees or specialists. Based on the limited availability of anesthetists, we included 10 participants in each of the two conditions (ie, AIMS and manual), with a total of 20 participants. To achieve simple randomization of group assignment, one experimenter (MKT) placed 10 red (representing the AIMS condition) and 10 green (representing the manual condition) stickers into an opaque envelope and then randomly drew a sticker to generate the allocation sequence. As soon as participants enrolled in the study, they were assigned to a condition according to the allocation sequence.

    Simulation Design

    A full-scale high-fidelity simulation was carried out in an OR at TMH. A clinical scenario specific for this study was designed by three anesthetists (THC, CPC, and KML). The scenario was designed to simulate uneventful monitoring with few critical incidents at intervals [14]. The scenario was set during the intraoperative portion of an emergency amputation below the right knee with general endotracheal anesthesia. It lasted for 45 minutes and comprised the following three phases: (1) preincident, (2) incident, and (3) postincident. The pre- and postincident phases were relatively uneventful, but the incident phase included the following three clinically relevant events: tourniquet pain, tourniquet deflation, and bleeding. The patient vital signs and progression were designed by an anesthetist (THC) and verified by a consultant anesthetist (CPC). When participants entered the simulation, they were asked to take over a case from a senior anesthetist (THC), who was a confederate in the study.

    Apart from the participant, the simulation involved seven people, each with a specific role as follows: (1) senior anesthetist (THC); (2) runner nurse (a registered nurse colleague at TMH); (3) surgeon (CWL); (4) scrub nurse (KML); (5) patient simulator operator (CPC); and (6) two experimenters (MKT and SYWL). The confederates and the patient simulator operator were clinicians from TMH. The two experimenters were researchers from Lingnan University.

    Each simulation session was recorded by two digital video recorders; one captured a general view of the OR (Figure 1A) and the other was head-mounted (GoPro Hero 5; GoPro, San Mateo, California, USA) to capture the participant’s point of view (Figure 1B). A Fluke ProSim 8 Vital Signs Patient Monitor Simulator (Fluke Biomedical, Cleveland, Ohio, USA) was connected to a SimMan 3G (Laerdal Medical AS, Stavanger, Norway) patient simulator and a physiological monitor to display vital signs during the simulation.

    Figure 1. Video capture from the perspective of the operating room (A) and participant (B) while the participant was entering data into the anesthesia information management system during the simulation scenario.
    View this figure

    Before the simulation began, participants were given a briefing to introduce them to the purpose of the study. The participants were then informed about the role of each confederate and the function of the patient simulator. In a training session, participants were given instructions and demonstrations on how to respond to assessments of vigilance, situation awareness, and mental workload during the simulation. Participants in the manual condition were also trained on how to manually complete an anesthetic record, because resident anesthetists at the hospital use an AIMS in their usual work practice. The simulation began when the senior anesthetist completed the handover to the participant. The participants were debriefed when the simulation was completed.

    Design of Situation Awareness Queries

    The situation present assessment method (SPAM) [15] was used to measure participants’ situation awareness. At predetermined moments of the simulation, the experimenter MKT called the participants’ mobile phone to deliver situation awareness queries. The queries covered the three levels of situation awareness (perception, comprehension, and projection). For generating the situation awareness queries, we followed the process recommended by Endsley [16] to conduct a goal-directed task analysis (GDTA), which involved semistructured interviews, formulating a goal tree, and extracting and finally translating situation awareness requirements into scenario-specific queries. Details of the GDTA and situation awareness requirements are provided in Multimedia Appendix 1 and Multimedia Appendix 2, respectively. A total of nine situation awareness queries (Table 1) were generated with input from five anesthetists (CPC, KML, THC, an associate consultant, and a resident specialist).

    Table 1. The nine situation awareness queries used in the scenario with their locations of information and their target answers.
    View this table

    Primary Outcomes

    There were three primary outcomes as follows: (1) accuracy of detecting suction tubing sounds (ie, vigilance detection accuracy), which were sounds made from actual suction tubing controlled by the scrub nurse (KML); (2) accuracy of correctly answering scenario-specific situation awareness queries (ie, situation awareness accuracy); and (3) self-reported mental workload ratings on The National Aeronautics and Space Administration Task Load Index (NASA-TLX) [17]. Measurements of the primary outcomes were performed by the experimenters MKT and SWL at predetermined times during the 45-minute scenario. Figure 2 shows how the measures were distributed over the three phases of the scenario.

    Figure 2. Design of the predetermined vital signs used in the clinical scenario of the simulation and the timeline of vigilance (V), situation awareness (S), and mental workload (W) assessments. BP: blood pressure; HR: heart rate.
    View this figure

    Secondary Outcomes

    The secondary outcomes involved the distribution of the participants’ time across different task activities (ie, task time distribution), the quality of their anesthesia record (ie, anesthesia record completeness), and their attitude toward the AIMS. We assessed participants’ attitude toward the AIMS in terms of trust and acceptance, using a 45-item questionnaire (Multimedia Appendix 3) after the simulation was completed.

    Statistical Analysis

    Operationalization of the Primary Outcomes

    Vigilance was operationalized as detection accuracy for each participant. The score was calculated as the proportion (%) of the six tubing sounds that a participant detected. Situation awareness was operationalized as response accuracy, which was calculated as the proportion (%) of the nine situation awareness queries that the participant answered correctly. Each participant’s answers to the situation awareness queries were first evaluated against a predetermined marking scheme. When an answer did not match the target answer, an anesthetist (THC), who was blinded to the condition allocation, helped determine the accuracy of the answer according to expert judgement.

    We performed the subjective mental workload measurement at the end of each simulation phase, in which participants rated each NASA-TLX dimension on a scale from 0 (lowest) to 100 (highest). The NASA-TLX comprises six dimensions (mental demand, physical demand, temporal demand, effort, frustration, and performance). The mean overall TLX score for each participant was calculated across the three simulation phases.

    Operationalization of the Secondary Outcomes

    Participants’ task activities in the simulation were video recorded and were reviewed to extract data on the different task activities. Task time distribution for each individual task category was computed as a percentage of the time spent on that category over the total time for all four tasks, including (1) entering record data, (2) monitoring the patient (eg, looking at the patient record, physiological monitor, anesthetic gas machine, or simulated patient), (3) performing patient care activities (eg, administering medication into patient’s intravenous access), and (4) interacting with the surgical team (eg, talking to the surgeon, asking the runner nurse to order medication, etc). Data were not coded for tasks that did not fall into any of the four task categories (eg, tidying up equipment wires, walking around the OR, etc).

    Two raters assessed the participants’ anesthetic records for completeness using the 15-item checklist by Edwards et al [4], which was modified from the Australian and New Zealand College of Anesthetists’ recommendations on anesthetic records [18]. The two raters were an anesthetist (THC) and a consultant anesthetist (CPC), and they scored each checklist item with 1 (present), 0.5 (partially present), or 0 (absent) for the anesthetic records. The scoring was carried out by the raters independent from each other. The scores of individual checklist items were summed to produce a total score for each anesthetic record.

    The trust and acceptance questionnaire had the following two parts: “trust in the AIMS” (adapted from a scale on trust in automated systems [19]) and “acceptance of the AIMS” (adapted from a scale based on the technology acceptance model [20-22]). All items in the questionnaire were rated on a 5-point Likert scale, with 1 indicating strongly disagree, 2 indicating disagree, 3 indicating neutral, 4 indicating agree, and 5 indicating strongly agree. Separate mean scores for trust and acceptance were calculated for each participant.

    Prior to analysis, the Shapiro-Wilk test and Levene test were performed to assess the normality and homogeneity of variance, respectively, of the studentized residuals of the data. The independent sample t test was used to compare differences between the manual and AIMS conditions for normally distributed data. The Mann-Whitney U test was performed for non-normally distributed data.

    According to the directions of the hypotheses, one-tailed significance tests were performed for the primary outcomes, whereas two-tailed tests were performed for the secondary outcomes. Task time distributions of the four tasks were compared between the two conditions with Bonferroni correction to obtain a more stringent alpha level of .0125 (.05/4).


    Results

    Response Rate

    All 20 participants completed the trials without any dropout (Figure 3). Participants in the AIMS condition and those in the manual condition had comparable years of experience in anesthesia, with mean experience durations of 3.4 and 3.2 years, respectively.

    Figure 3. CONSORT disgram for the simulation study.
    View this figure

    Primary Outcomes

    There was no significant difference in vigilance accuracy between the AIMS (mean 56.7%, SD 32.6%) and manual conditions (mean 56.7%, SD 31.6%) (t18=0.00, P=.50, one-tailed); therefore, H1 was not supported. Although there was no significant difference in situation awareness accuracy between the AIMS (median 88.9%, range 66.7%-100%) and manual conditions (median 88.9%, range 77.8%-100%) (U=40.5, P=.48), we carried out a post-hoc power analysis using G*Power [23] on the basis of an emerging difference in trend between the two conditions. The achieved power (1 ̶ β) calculated was 0.13, which was below the lowest conventionally acceptable level of 0.8. This suggests that the study did not have enough power to detect a difference in situation awareness accuracy between the AIMS and manual conditions. Therefore, H2 was inconclusive. However, we found that participants reported a significantly lower overall TLX score in the AIMS condition (mean 34.2, SD 12.5) than in the manual condition (mean 46.7, SD 11.5) (t18=−2.34, P=.02, one-tailed). Therefore, H3 was supported.

    Secondary Outcomes

    Some video data were not coded (30% in the AIMS condition and 26% in the manual condition), as they either could not be classified or involved tasks that did not fall into our predefined task categories. Of the data that were coded according to the four task categories, only the proportion of time spent on record data entry differed significantly between the AIMS (mean 26.0%, SD 4.9%) and manual conditions (mean 33.7%, SD 6.9%) (t18=−2.87, P=.01, two-tailed). We also found that the level of completeness of anesthetic records was significantly higher in the AIMS condition (median 100%, range 93%-100%) than in the manual condition (median 75%, range 55%-87%) (U=0.000, P<.001, two-tailed). The two raters for record completeness had a high degree of reliability, with an average intraclass correlation coefficient of 0.893 and a 95% CI ranging from 0.68 to 0.96 (F19,19=11.59, P<.001). Finally, data from the trust and acceptance survey indicated that 45% (9/20) of respondents showed a positive attitude (agree or highly agree) of trust toward the AIMS and the remaining 55% (11/20) showed a neutral attitude. In terms of acceptance, 90% (18/20) of respondents showed a positive attitude (agree or highly agree) and 10% (2/20) showed a neutral attitude.


    Discussion

    Overall Findings

    Despite the increasing adoption rate of AIMSs in hospitals [2], their effect on the monitoring performance of anesthetists has not been thoroughly examined. This study compared the effects of AIMS use and manual record keeping in terms of anesthetists’ levels of vigilance, situation awareness, and subjective mental workload with a randomized controlled trial in a high-fidelity simulation setting. The primary outcomes indicated that while there was no relevant difference in participants’ vigilance between AIMS use and manual record keeping, subjective mental workload was much lower among participants using the AIMS than among those using the manual method. However, the effect on situation awareness accuracy was inconclusive because the study was under-powered to detect its difference between the two conditions.

    AIMS use might have two advantages over manual record keeping with respect to mental workload. First, the lower subjective mental workload with AIMS use might be a product of reduced physical movements. Informal inspection of our GoPro video data revealed that participants in the manual condition exhibited extensive head movements owing to the shifting of attention between the physiological monitor and the paper anesthesia chart. These movements may imply that more cognitive and perceptual activities (eg, remembering, looking, and searching for information) are involved in manual record keeping, and thereby, they result in higher subjective mental workload. Second, manual record keeping might have placed a high demand on participants’ prospective memory (remembering a future task) [24], because they needed to remind themselves to update vital signs on the paper chart regularly.

    The secondary outcomes indicated further benefits of AIMS use. First, participants who used the AIMS spent about 8 percentage points less of their total time on record data entry than those who used manual record keeping. This result confirms previous findings that electronic record keeping allows anesthesia residents to spend less time on record keeping as compared to that with manual record keeping [7]. Second, AIMS use produced more complete anesthetic records than those produced by manual record keeping. This finding is consistent with the result of a previous study that retrospectively assessed 400 anesthetic records created by AIMS or manual record keeping methods [4] and reported more complete AIMS records than manual records. It is likely that AIMS use spares anesthetists from charting patients’ vital signs and allows them to spend more time on including other required information in the anesthetic records. Third, the attitude survey of AIMS use indicated that participants had a positive attitude toward trusting and accepting AIMS use in their practice.

    Compared with previous studies on AIMS use that only examined visual vigilance [6,7], our study tested auditory vigilance. In this study, vigilance was operationalized as participants’ accuracy of detecting suction tubing sounds. This stimulus was chosen based on its clinical relevance, given that anesthetists often interpret it as a sign of patient blood loss during surgery. Although a direct comparison to visual vigilance might be impossible, our current results and those from previous studies suggest that AIMS use does not harmfully decrease anesthetists’ vigilance level [6,7]. However, irrespective of the type of record keeping, participants in this study demonstrated only a fair vigilance level in that they only detected, on average, 3.2 out of all 6 suction sounds (54%) in the vigilance assessments. We had not anticipated this result, but given the clinical importance of detecting suction sounds, this should be further investigated in future studies.

    Limitations

    This study had six limitations. First, our simulated scenario only represented anesthetic cases that involve an uneventful period followed by critical incidents. Therefore, our findings can only be applied to the context of anesthesia with critical incidents. In anesthesia, many cases occur without any critical events. When the anesthetic procedure is uneventful, the effect of AIMS use on anesthetists’ vigilance and situation awareness might be different because complacency might arise, and this warrants further investigation.

    Second, our participants were more accustomed to AIMS use than manual record keeping in their usual practice because junior anesthetists at TMH are trained on the AIMS but not on manual record keeping. Therefore, participants in our simulation had to be retrained on manual record keeping for comparison. While this retraining might seem artificial, it was the aim of TMH’s Department of Anaesthesia & ICU to investigate the tacit assumption of the effectiveness of AIMS use over manual record keeping. Retraining in the manual condition might have increased participants’ perceived mental workload, degraded their vigilance, and decreased their record keeping efficiency. This possible confounding factor could be addressed in future studies by sample screening or providing participants with prolonged training in manual record keeping.

    Third, the findings of our study cannot be generalized to all models or brands of AIMSs. Other models of AIMSs might have different functions or interfaces and might interact with anesthetists differently.

    Fourth, the participants, experimenters, and confederates were not blinded to the condition assigned to each participant owing to the nature of the manual and automated record keeping conditions.

    Fifth, although our results suggest that AIMS use reduced the time spent on record data entry, it is unclear whether the time reduction led to an increase in time spent on monitoring patients or performing patient care activities. This could be addressed in future studies by examining how anesthetists reallocate the time saved with AIMS use to other tasks.

    Sixth, we used a GoPro camera attached to each participant’s head in an attempt to capture visual data. However, the GoPro camera, at its best, could only provide us with the participant’s gaze direction. If accurate visual attention data are to be gathered, a mobile eye tracker should be used in future studies. Eye tracking data would allow for not only better inference of participants’ visual attention in general, but also identification of what activities they focus on when not interacting with the AIMS.

    Conclusions

    Despite the increasing popularity of AIMSs in hospitals, no previous studies have analyzed their effects on comprehensive monitoring performance. The findings of this study provide support for the adoption of AIMSs in the OR by demonstrating a number of benefits of AIMS use, including reducing anesthetists’ perceived mental workload, saving their time spent on data entry, and producing complete anesthetic records, without compromising vigilance. Moreover, the majority of our anesthetists expressed a positive attitude toward trusting and accepting AIMSs in the OR.

    The level of automation in health care is likely to increase as medical technology advances. It is important to know the effects that automation will have on patient care, as it could affect clinicians’ care quality and, ultimately, patients’ well-being and safety.

    Acknowledgments

    This study would not have been possible without support from the Department of Anaesthesia and Intensive Care and the Quality and Safety Division at Tuen Mun Hospital. We would like to sincerely thank Tuen Mun Hospital’s anesthetists who participated in the study and Francis Leung Wai Sing who generously made time to prepare and participate in the simulation. We would also like to express our gratitude to Professor Penelope Sanderson and Professor Robert Loeb for their encouragement and valuable comments that helped us improve the research. This research was supported by a postgraduate studentship from Lingnan University awarded to MKT.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    The goal-directed task analysis.

    PDF File (Adobe PDF File), 556 KB

    Multimedia Appendix 2

    Pooled situation awareness requirements for the scenario.

    PDF File (Adobe PDF File), 417 KB

    Multimedia Appendix 3

    Questionnaire for trust and acceptance of anesthesia clinical information systems.

    PDF File (Adobe PDF File), 784 KB

    Multimedia Appendix 4

    CONSORT-eHEALTH checklist (V 1.6.1).

    PDF File (Adobe PDF File), 2251 KB

    References

    1. Kadry B, Feaster W, Macario A, Ehrenfeld J. Anesthesia information management systems: past, present, and future of anesthesia records. Mt Sinai J Med 2012;79(1):154-165. [CrossRef] [Medline]
    2. Stol I, Ehrenfeld J, Epstein R. Technology Diffusion of Anesthesia Information Management Systems into Academic Anesthesia Departments in the United States. Anesthesia & Analgesia 2014;118(3):644-650 [FREE Full text] [CrossRef]
    3. Driscoll WD, Columbia MA, Peterfreund RA. An observational study of anesthesia record completeness using an anesthesia information management system. Anesth Analg 2007 Jun;104(6):1454-1461. [CrossRef] [Medline]
    4. Edwards K, Hagen S, Hannam J, Kruger C, Yu R, Merry A. A randomized comparison between records made with an anesthesia information management system and by hand, and evaluation of the Hawthorne effect. Can J Anaesth 2013 Oct;60(10):990-997. [CrossRef] [Medline]
    5. Davies DR, Parasuraman R. The Psychology Of Vigilance (Organizational and Occupational Psychology). London: Academic Press; 1982.
    6. Loeb R. Manual record keeping is not necessary for anesthesia vigilance. J Clin Monit 1995 Jan;11(1):9-13. [CrossRef] [Medline]
    7. Weinger M, Herndon O, Gaba D. The effect of electronic record keeping and transesophageal echocardiography on task distribution, workload, and vigilance during cardiac anesthesia. Anesthesiology 1997 Jul;87(1):144-55; discussion 29A. [CrossRef] [Medline]
    8. Endsley M. Toward a Theory of Situation Awareness in Dynamic Systems. Hum Factors 2016 Nov 23;37(1):32-64 [FREE Full text] [CrossRef]
    9. Drews F, Westenskow D. The right picture is worth a thousand numbers: data displays in anesthesia. Hum Factors 2006;48(1):59-71. [CrossRef] [Medline]
    10. Tsang PS, Vidulich MA. Mental workload and situation awareness. In: Salvendy G, editor. Handbook of Human Factors and Ergonomics. Hoboken, NJ: Wiley; 2012:243-273.
    11. Noel TA. Computerized anesthesia records may be dangerous. Anesthesiology 1986 Feb;64(2):300. [CrossRef] [Medline]
    12. Weinger MB, Smith NT. Vigilance, alarms, and integrated monitoring systems. In: Ehrenwerth J, Eisenkraft JB, editors. Anesthesia Equipment: Principles and Applications. St. Louis, MO: Mosby; 1993:350-384.
    13. Endsley M, Kiris E. The Out-of-the-Loop Performance Problem and Level of Control in Automation. Hum Factors 2016 Nov 23;37(2):381-394 [FREE Full text] [CrossRef]
    14. Slagle J, Weinger M. Effects of intraoperative reading on vigilance and workload during anesthesia care in an academic medical center. Anesthesiology 2009 Feb;110(2):275-283. [CrossRef] [Medline]
    15. Durso FT, Dattel AR, Banbury S, Tremblay S. SPAM: The real-time assessment of SA. In: Banbury S, Tremblay S, editors. A cognitive approach to situation awareness: Theory and application. Aldershot, England: Ashgate; 2004:137-154.
    16. Endsley MR. A Survey of Situation Awareness Requirements in Air-to-Air Combat Fighters. The International Journal of Aviation Psychology 1993 Apr;3(2):157-168. [CrossRef]
    17. Hart S, Staveland L. Development of NASA-TLX (Task Load Index): Results of empiricaltheoretical research. In: Hancock PA, Meshkati N, editors. Advances in psychology. Amsterdam, The Netherlands: North-Holland; 1988:139-183.
    18. Australian and New Zealand College of Anesthetists. 2006. The anesthesia record recommendations on the recording of an episode of anesthesia care   URL: http://www.anzca.edu.au/documents/ps06-2006-the-anaesthesia-record-recommendations-o [accessed 2018-03-30]
    19. Jian J, Bisantz AM, Drury CG. Foundations for an Empirically Determined Scale of Trust in Automated Systems. International Journal of Cognitive Ergonomics 2000 Mar;4(1):53-71. [CrossRef]
    20. Davis F. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 1989 Sep;13(3):319 [FREE Full text] [CrossRef]
    21. Venkatesh V, Davis F. A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Management Science 2000 Feb;46(2):186-204 [FREE Full text] [CrossRef]
    22. Gagnon MP, Orruño E, Asua J, Abdeljelil AB, Emparanza J. Using a modified technology acceptance model to evaluate healthcare professionals' adoption of a new telemonitoring system. Telemed J E Health 2012;18(1):54-59 [FREE Full text] [CrossRef] [Medline]
    23. Faul F, Erdfelder E, Lang A, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007 May;39(2):175-191. [CrossRef] [Medline]
    24. McDaniel M, Einstein G. Prospective memory: An overview and synthesis of an emerging field. Thousand Oaks, CA: SAGE Publications; 2007.


    Abbreviations

    AIMS: anesthesia information management system
    GDTA: goal-directed task analysis
    NASA-TLX: National Aeronautics and Space Administration’s Task Load Index
    OR: operating room
    SPAM: situation present assessment method
    TMH: Tuen Mun Hospital


    Edited by A Kushniruk; submitted 12.09.19; peer-reviewed by M Wright, E Borycki, M Ameen; comments to author 03.11.19; revised version received 27.02.20; accepted 28.03.20; published 16.06.20

    ©Man-Kei Tse, Simon Y W Li, Tsz Hin Chiu, Chung Wai Lau, Ka Man Lam, Chun Pong Benny Cheng. Originally published in JMIR Human Factors (http://humanfactors.jmir.org), 16.06.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on http://humanfactors.jmir.org, as well as this copyright and license information must be included.