This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on http://humanfactors.jmir.org, as well as this copyright and license information must be included.
Few intensive care unit (ICU) staffing studies have examined the collaboration structures of health care workers (HCWs). Knowledge about how HCWs are connected to the care of critically ill patients with COVID-19 is important for characterizing the relationships among team structures, care quality, and patient safety.
We aimed to discover differences in the teamwork structures of COVID-19 critical care by comparing HCW collaborations in the management of critically ill patients with and without COVID-19.
In this retrospective study, we used network analysis methods to analyze the electronic health records (EHRs) of 76 critically ill patients (with COVID-19: n=38; without COVID-19: n=38) who were admitted to a large academic medical center, and to learn about HCW collaboration. We used the EHRs of adult patients who were admitted to the COVID-19 ICU at the Vanderbilt University Medical Center (Nashville, Tennessee, United States) between March 17, 2020, and May 31, 2020. We matched each patient according to age, gender, and their length of stay. Patients without COVID-19 were admitted to the medical ICU between December 1, 2019, and February 29, 2020. We used two sociometrics—eigencentrality and betweenness—to quantify HCWs’ statuses in networks. Eigencentrality characterizes the degree to which an HCW is a core person in collaboration structures. Betweenness centrality refers to whether an HCW lies on the path of other HCWs who are not directly connected. This sociometric was used to characterize HCWs’ broad skill sets. We measured patient staffing intensity in terms of the number of HCWs who interacted with patients’ EHRs. We assessed the statistical differences in the core and betweenness statuses of HCWs and the patient staffing intensities of COVID-19 and non–COVID-19 critical care, by using Mann-Whitney U tests and reporting 95% CIs.
HCWs in COVID-19 critical care were more likely to frequently work with each other (eigencentrality: median 0.096) than those in non–COVID-19 critical care (eigencentrality: median 0.057;
Network analysis methodologies and data on EHR use provide a novel method for learning about differences in collaboration structures between COVID-19 and non–COVID-19 critical care. Health care organizations can use this information to learn about the novel changes that the COVID-19 pandemic has imposed on collaboration structures in urgent care.
The COVID-19–Associated Hospitalization Surveillance Network has reported that the overall cumulative COVID-19 hospitalization rate in the United States is 199.8 people per 100,000 people for the week that ended October 24, 2020 [
ICU staffing (eg, the assignment of patients to a set of health care workers [HCWs]) can impact care quality and patient safety [
Few studies have investigated the collaboration structures of COVID-19 critical care [
We conducted a secondary analysis on EHR use to learn about the collaborations among HCWs. We created networks by identifying connections among HCWs who conducted activities with the EHRs of the same patients on the same day.
We used two sociometrics—eigencentrality and betweenness centrality—to measure the core and betweenness status of an HCW in the collaboration network. Eigencentrality characterizes the degree to which an HCW is a core person in collaboration structures. Betweenness centrality refers to whether an HCW lies on the path of other HCWs who are not directly connected. An HCW who has a broad skill set and cares for a wide spectrum of patients could frequently be in a high-betweenness position.
We analyzed data on EHR system use in the Vanderbilt University Medical Center, which is a large academic medical center in Nashville, Tennessee that created its COVID-19 unit in the middle of March 2020. The high density of clinical ICU data and large volume of EHR activities for each ICU patient episode allow for the investigation of HCW collaboration in the management of critically ill patients before and during the COVID-19 pandemic.
We learned about the collaboration structures in COVID-19 critical care by comparing structures that were associated with the management of critically ill patients with and without COVID-19.
We screened for adult patients who were admitted to the COVID-19 ICU between March 17, 2020, and May 31, 2020. We matched each patient with COVID-19 with an adult patient without COVID-19 who was admitted to the medical ICU (MICU) between December 1, 2019, and February 29, 2020, via propensity score matching.
The propensity score was based on age, gender, and patients’ length of stay. The distribution of the COVID-19 and non–COVID-19 groups’ propensity scores is depicted in
Characteristics of the critically ill patients in this study.
Characteristics | Patients with COVID-19a | Patients without COVID-19b | ||
Patients, n | 38 | 38 | ||
|
||||
Age (years), median (IQR; SD) | 54 (47-66; 14) | 54 (49-64; 12) | ||
|
||||
Female | 15 (39) | 15 (39) | ||
Male | 23 (61) | 23 (61) | ||
|
||||
White | 22 (58) | 32 (84) | ||
African American | 6 (16) | 5 (13) | ||
Asian | 4 (11) | 0 (0) | ||
Other | 6 (16) | 1 (3) | ||
|
||||
Length of stay (days), median (IQR; SD) | 13.5 (6.50-18.75; 10) | 13.5 (7.50-19.00; 9) | ||
|
||||
Home | 29 (76) | 24 (63) | ||
Other | 9 (24) | 14 (37) | ||
|
||||
Hypertension | 19 (50) | 33 (87) | ||
Cardiovascular disease | 14 (37) | 23 (61) | ||
Renal disease | 19 (50) | 22 (58) | ||
Diabetes | 10 (26) | 16 (42) | ||
Chronic metabolic disease | 14 (37) | 18 (47) | ||
Chronic lung disease | 9 (24) | 17 (45) |
aPatients with COVID-19 were admitted to the intensive care unit between March 17, 2020, and May 31, 2020.
bPatients without COVID-19 were admitted to the medical intensive care unit between December 1, 2019, and February 29, 2020.
The study population had several notable aspects. First, we noticed that there was a disproportionate number of males. Second, while there were more self-reported White patients than patients of other races, the number of White patients in the COVID-19 group was substantially smaller than that in the non–COVID-19 group. Third, patients without COVID-19 had a high incidence of comorbidities; specifically, patients without COVID-19 exhibited the six comorbidities that are common in patients with COVID-19 (ie, those reported by the COVID-19–Associated Hospitalization Surveillance Network) [
The analysis consisted of two primary components. First, we used network analysis methods to learn about the HCW networks that were involved in the management of critically ill patients. Second, we statistically compared and contrasted the network structures in COVID-19 and non–COVID-19 settings.
We analyzed the actions that HCWs performed with patients’ EHRs to measure worker-worker connections. There are six types of HCW actions, including condition-related (eg, assigning a diagnosis), procedure-related (eg, intubation), medication-related (eg, prescriptions), note-related (eg, writing progress notes), order-related (eg, ordering laboratory tests), and measurement-related (eg, measuring respiratory rate) actions.
Research has shown that a 1-day window is enough to capture the meaningful, collaborative relationships among HCWs [
The nodes in the COVID-19 and non–COVID-19 networks were defined as follows:
To better interpret the networks, we used an HCW’s specialty (eg, respiratory care) and type (eg, respiratory therapist) to label each node. We combined these factors to define expertise (ie, “specialty: type”; eg, respiratory care: respiratory therapist). Expertise in the COVID-19 and non–COVID-19 networks were defined as follows:
In equations 1-4, Z and EXP were used to describe the composition of COVID-19 or non–COVID-19 networks.
In each network, we used two sociometrics—eigenvector centrality and betweenness centrality—to quantify an HCW’s core and betweenness status in the network, respectively. We used Gephi (ie, an open-source network analysis and visualization software package) [
Eigencentrality characterizes the degree to which an HCW is densely connected to other HCWs who are also densely connected with other HCWs. A high-eigencentrality HCW is likely to be a core person who actively works with other HCWs when performing actions on EHRs. An example HCW network with eigencentrality values is shown in
The betweenness centrality of an HCW refers to the number of shortest paths between two other HCWs that pass through the HCW in question. An HCW with a broad skill set who cares for a wide spectrum of patients could frequently be in a high-betweenness position. An example HCW network with betweenness centrality values is shown in
We investigated whether differences in the eigencentrality and betweenness of COVID-19 and non–COVID-19 critical care structures were statistically significant at the network and expertise levels. The network-level comparison was conducted by assessing the network as a whole (ie, COVID-19 vs non–COVID-19 networks), while an expertise-level comparison was conducted for each expertise (eg, internal medicine physicians). Since eigencentrality and betweenness are not Gaussian distributed, we conducted a Mann-Whitney U test with a significance level of α=.05. The tests for expertise included at least 8 HCWs and involved Bonferroni correction to account for multiple hypotheses.
We defined the set of inpatient stays in COVID-19 and non–COVID-19 settings as follows:
Since each inpatient stay (ie, si) can last for more than 1 day, we defined the jth day of a stay as si,j (ie, 1≤j≤li); li represents the last day of a patient’s hospital stay (ie, si). For si,j, we calculated the number of HCWs (ie, Nsi,j,expk) in each expertise category (ie, expk) who interacted with the EHRs of patient i on day j. For each inpatient stay (ie, si), we calculated the average number of HCWs in each expertise category (ie, expk) who interacted with the EHRs of the same patient on each day, as follows:
In equation 7, losi refers to the length of hospital stay (ie, the total number of hours between the start and end times of an inpatient stay divided by 24 hours). Since each inpatient stay may start and end at different times of the day,losi may be different from li. Daily patient staffing intensity was defined as an expertise-level value (ie,
To learn about the differences in the daily patient staffing intensities of COVID-19 and non–COVID-19 critical care, we conducted a set of tests. Specifically, for each investigated expertise (eg, internal medicine nurse practitioners), we tested whether critically ill patients with COVID-19 required a significantly higher daily patient staffing intensity than critically ill patients without COVID-19. We focused on the 20 expertise categories with the highest mean daily staffing intensity values in COVID-19 and non–COVID-19 critical care and used the Mann-Whitney U test, which had a Bonferroni-corrected significance level of .05.
We also assessed the differences in the overall patient staffing intensities of COVID-19 and non–COVID-19 critical care in terms of the number of HCWs who were involved in the management of a patient. Overall staffing intensity was defined as the number of HCWs who interacted with the EHRs of a patient with or without COVID-19. Our hypothesis was as follows: critically ill patients with COVID-19 require a significantly higher overall staffing intensity than critically ill patients without COVID-19. We tested this hypothesis by using the Mann-Whitney U test, which had a Bonferroni-corrected significance level of .05.
The number of HCWs, types of HCWs, specialties, and expertise categories in COVID-19 and non–COVID-19 critical care was 759 and 1331, 24 and 24, 92 and 128, and 133 and 207, respectively. These values indicated that patients without COVID-19 required more expertise, specialties, and HCWs. A possible reason for this is that critically ill patients without COVID-19 were admitted to the MICU for a wide range of major conditions. With regard to patient-level values, COVID-19 and non–COVID-19 critical care consisted of 79.5 and 88.2 HCWs, 9.8 and 10.6 types of HCWs, 23.0 and 27.1 departments, and 29.2 and 34.0 expertise categories, respectively. The patient-level values for COVID-19 and non–COVID-19 critical care were highly similar.
The expertise categories with the largest number of health care workers in the COVID-19 and non–COVID-19 settings. There are 11 expertise categories shown, which correspond to the union of the top 10 expertise categories in COVID-19 (ie, excluding nurses with myelosuppression expertise) and non–COVID-19 (ie, excluding COVID-19 unit nurses) critical care. Each expertise is reported in the following format: “specialty: health care worker type.” MICU: medical intensive care unit.
There were no registered nurses with myelosuppression expertise in the COVID-19 setting. Upon further analysis, we found 4 patients with COVID-19 and cancer, but none were in need of invasive intervention at the time of their care.
A depiction of the health care worker eigencentrality (A) and betweenness (B) in COVID-19 and non–COVID-19 networks. C and D show the subnetworks of internal medicine physicians and nurse practitioners in the COVID-19 and non–COVID-19 networks, respectively. In A and C, eigencentrality directly correlated with the size of the corresponding node. In B and D, betweenness centrality directly correlated with the size of the corresponding node. The legend in the figure shows the 10 expertise categories with the largest number of health care workers in the combined network (ie, both the COVID-19 and non–COVID-19 networks). MICU: medical intensive care unit.
After removing expertise categories that had less than 8 HCWs, we performed pairwise tests on the remaining 12 expertise categories. The results of these tests are provided in Table S1 in
(A) The subnetworks of resident physicians in the COVID-19 and non–COVID-19 networks. (B) The subnetworks of internal medicine physicians and nurse practitioners in the COVID-19 and non–COVID-19 networks. Eigencentrality directly correlated with the size of the corresponding node in both A and B.
There were no significant differences in the betweenness of the two networks (COVID-19 network: median 0.002; non–COVID-19 network: median 0.003;
In total, 41,903 (mean 1103) actions were performed with the EHRs of patients with COVID-19 and 44,131 (mean 1161) actions were performed with the EHRs of patients without COVID-19. There were no statistically significant differences in the number of actions performed with the EHRs of patients with and without COVID-19 (
Differences in the daily average patient staffing intensities of COVID-19 and non–COVID-19 critical care. A Bonferroni-corrected
Expertise category | Staffing intensity, mean (SD) | Staffing intensity, median (IQR) | ||||
COVID-19 critical care | Non–COVID-19 critical care | COVID-19 critical care | Non–COVID-19 critical care | |||
|
||||||
Hospital nurse practitioners: nurse practitioner | 1.81 (0.19) | 0.27 (0.10) | 1.76 (1.37) | 0 (0.18) | <.001 | |
Medical Center Easta: registered Nurse | 0.78 (0.08) | 0.21 (0.07) | 0.69 (0.70) | 0 (0.02) | <.001 | |
Cardiovascular intensive care unit: registered Nurse | 0.26 (0.05) | 0.01 (0.01) | 0.18 (0.37) | 0 (0) | <.001 | |
Internal medicine: nurse practitioner | 0.33 (0.05) | 0.06 (0.02) | 0.23 (0.30) | 0 (0.05) | <.001 | |
Surgical intensive care unit: registered nurse | 0.17 (0.04) | 0.03 (0.02) | 0.09 (0.29) | 0 (0) | .002 |
|
|
||||||
Medicine house staff: resident physician | 0.18 (0.06) | 1.56 (0.15) | 0 (0.07) | 1.62 (1.32) | <.001 | |
Emergency medicine: resident physician | 0.06 (0.03) | 0.27 (0.05) | 0 (0.05) | 0.16 (0.30) | <.001 | |
|
||||||
Medical Center Easta: technician | 0.23 (0.03) | 0.10 (0.04) | 0.22 (0.19) | 0 (0) | .007 | |
Hematology oncology: physician | 3.54 10-3 (2.87 10-3) | 0.22 (0.08) | 0 (0) | 0 (0.06) | .01 | |
Emergency medicine: physician | 0.27 (0.06) | 0.53 (0.10) | 0.11 (0.43) | 0.30 (0.92) | .02 | |
Pharmacy inpatient (central): pharmacist | 0.12 (0.03) | 0.21 (0.03) | 0.06 (0.14) | 0.15 (0.13) | .03 | |
Pharmacy inpatient (evening): pharmacist | 0.16 (0.03) | 0.24 (0.03) | 0.12 (0.28) | 0.21 (0.23) | .03 | |
Infectious disease: physician | 0.05 (0.03) | 0.18 (0.05) | 0 (0) | 0 (0.20) | .05 | |
Internal medicine: physician | 1.14 (0.13) | 0.60 (0.11) | 0.92 (1.12) | 0.53 (0.90) | .002 | |
Medical intensive care unit: registered nurse | 1.19 (0.13) | 0.95 (0.10) | 1.03 (0.91) | 0.85 (0.78) | .13 | |
Radiology: physician | 0.30 (0.04) | 0.38 (0.04) | 0.27 (0.34) | 0.37 (0.42) | .15 | |
Nephrology: physician | 0.14 (0.05) | 0.26 (0.10) | 0 (0) | 0 (0.05) | .21 | |
Allergy/pulmonary: physician | 0.80 (0.11) | 0.94 (0.12) | 0.65 (0.82) | 0.88 (0.73) | .21 | |
Pharmacy inpatient operations manager: pharmacist | 0.14 (0.02) | 0.21 (0.04) | 0.11 (0.13) | 0.14 (0.24) | .22 | |
Pharmacy inpatient satellite operating room: pharmacist | 0.20 (0.04) | 0.15 (0.03) | 0.11 (0.38) | 0.11 (0.23) | .28 | |
Respiratory care: respiratory therapist | 0.74 (0.11) | 0.89 (0.14) | 0.60 (0.87) | 0.80 (1.32) | .30 | |
Emergency services: registered nurse | 0.15 (0.05) | 0.17 (0.05) | 0.07 (0.18) | 0.09 (0.18) | .41 | |
Pharmacy inpatient (evening): pharmacy technician | 0.26 (0.05) | 0.28 (0.05) | 0.16 (0.36) | 0.19 (0.40) | .43 | |
Cardiovascular medicine: physician | 0.21 (0.04) | 0.18 (0.03) | 0.15 (0.37) | 0.16 (0.15) | .43 |
aMedical Center East is the building where we created the COVID-19 unit. Before the creation of the COVID-19 unit, nurses in this building cared for critically ill patients without COVID-19.
There was a larger number of internal medicine nurse practitioners, cardiovascular ICU registered nurses, and surgical ICU registered nurses who performed daily actions on the EHRs of critically ill patients with COVID-19 compared to the number of those who performed daily actions on the EHRs of patients without COVID-19. In contrast, the EHRs of patients without COVID-19 were managed by a larger number of resident physicians (ie, those with medicine and emergency medicine expertise). These differences were statistically significant (
We also found that expertise categories were not statistically different in terms of daily patient staffing intensity. Such categories included radiology physicians, nephrology physicians, pulmonary/allergy physicians, emergency medicine physicians, MICU registered nurses, and respiratory therapists.
There are no universal guidelines for HCW staffing in ICUs. To date, ICU staffing studies have focused on organization models (eg, open, closed, and hybrid models), and few have examined collaborations among HCWs. In this study, we used a novel method for learning about collaborations among HCWs and building corresponding networks. We measured eigencentrality and betweenness centrality to quantify the core and betweenness statuses of HCWs and identify several significant differences between the COVID-19 and non–COVID-19 network structures. Differences in the collaboration structures between the two networks mirrored those in intentional strategic planning structures across the health care system. For instance, there was a significant difference (
Beyond collecting data on basic strategic planning methods (ie, the reduction of the number of residents) in the management of critically ill patients with COVID-19, we also learned about the aspects of collaboration structures that are important for the management of teamwork but are not explicitly documented in existing ICU staffing plans. We found that internal medicine physicians and nurse practitioners in the COVID-19 collaboration structure were more active (ie, high eigencentrality or betweenness) than those in the non–COVID-19 collaboration structure. As shown in
Combining knowledge on connections among HCWs with their eigencentrality and betweenness values in the collaboration network can assist HCOs with designing and developing more specific staffing strategies, which can potentially improve care quality and patient outcomes. The network analysis methods and team structures that are depicted in our retrospective study can be used in a prospective setting. Our COVID-19 and non–COVID-19 networks can be used to identify the characteristics of a newly established or modified team. For instance, if a COVID-19 ICU has plans for creating a team to care for the increasing number of patients, the eigencentrality and betweenness centrality of each HCW and the HCW relationships that we learned about in our COVID-19 network can be used as evidence for identifying the characteristics of the newly created team. The team creators can evaluate the leadership (ie, eigencentrality), robustness (ie, betweenness), and familiarity (ie, the strength of the relationships between HCWs) of the newly established team. They can also dynamically add or remove an HCW from the created team and measure changes in leadership, robustness, and familiarity, which will help team creators with finding their desired team.
In this study, we did not investigate temporal networks or team dynamics, which are essential to HCOs that monitor and manage team dynamics. However, researchers can use the network analysis methodologies that were developed in our study to identify temporal networks, such as daily, weekly, or monthly networks. They can also use the sociometrics that we developed to quantify changes in temporal network structures. For instance, HCOs can use our network analysis methodologies to temporally measure the relationships among internal medicine physicians, nurse practitioners, and residents; and quantify the weekly, monthly, and yearly changes in these relationships.
There are several limitations in this study that should be recognized. These limitations serve as opportunities for further investigation. First, this study was based on a small number of critically ill patients with COVID-19. Although our sample had sufficient power for analyzing the differences in the eigencentrality, betweenness, and patient staffing intensities for several types of HCWs in COVID-19 and non–COVID-19 critical care, a larger volume of data is needed to obtain statistically meaningful results. Second, comorbidities could have impacted team structures; however, matching the comorbidities between patients with and without COVID-19 can lead to certain risks. According to our observations, matching comorbidities between the two cohorts will considerably enlarge the study window (ie, >3 years) for the non–COVID-19 cohort. However, non–COVID-19 care teams can drastically change over time, making the study of the non–COVID-19 team structures less meaningful. Therefore, in this study, we focused on the most important confounding factors (ie, age, gender, and the length of stay) that characterize team efficacy and may impact team structures. Additionally, our study's primary focus was to learn about team structures in COVID-19 and medical ICUs, which allowed for some degree of variance in comorbidities. Third, the characteristics of the COVID-19 structures that we learned about during this single-center study could provide HCOs with reference data for assessing their own COVID-19 ICU structures. However, our medical center is an institution that intentionally developed a nurse practitioner–centered organizational structure. This should be considered when interpreting our results and findings. To learn about general COVID-19 ICU collaboration structures, researchers need to conduct analyses that account for multiple HCOs. Fourth, there was a lack of standard terminology for characterizing HCO departments and the roles of HCWs. Although there are taxonomies for describing clinician specialties [
HCOs have been planning and refining their staffing strategies to provide more efficient and effective care to patients with COVID-19. However, there are few efficient methodologies for assessing the execution of collaboration structures in practice, especially those for assessing the cross-disciplinary connections among HCWs. In this study, we demonstrated how data on the use of a large academic medical center’s EHR system could be used to learn about the collaboration structures in COVID-19 critical care (ie, through network analysis methodologies). HCOs can use our network analysis approaches and data on eigencentrality, betweenness, and patient staffing intensities to characterize HCW roles in collaboration networks during the COVID-19 pandemic or future events. Furthermore, research on how HCWs are connected has created an opportunity for studying the relationships among team structures, care quality, and patient safety.
Distributions of the COVID-19 and non–COVID-19 groups' propensity scores.
Examples that illustrate eigencentrality and betweenness centrality.
Differences in the eigencentrality and betweenness of COVID-19 and non–COVID-19 structures.
Differences in overall patient staffing intensities between the COVID-19 and non–COVID-19 structures.
electronic health record
health care organization
health care worker
intensive care unit
medical intensive care unit
This study was partially supported by the National Library of Medicine of the National Institutes of Health (award number: R01LM012854). The content of this study is solely the authors' responsibility; it does not necessarily represent the National Institutes of Health's official views. The data sets that were used for the described analyses were obtained from Vanderbilt University Medical Center’s Research Derivative, which is supported by institutional funds and Vanderbilt Clinical and Translational Science Awards (grant number: ULTR000445) from the National Center for Advancing Translational Sciences and the National Institutes of Health. The Vanderbilt Institutional Review Board reviewed and approved this study (approval number: 200792).
The data sets that were generated and analyzed during this study are not publicly available because they include patients’ private information. However, the data sets can be obtained from the corresponding author upon reasonable request.
YC conceived the study idea, collected the data, analyzed the data, designed the methods, designed the experiment, evaluated and interpreted the results of the experiment, and wrote the manuscript. CY conceived the study idea, analyzed the data, designed the methods, designed the experiment, evaluated and interpreted the results of the experiment, and revised the manuscript. XZ analyzed the data, evaluated and interpreted the results of the experiment, and revised the manuscript. CG analyzed the data, evaluated and interpreted the results of the experiment, and revised the manuscript. EW, JC, DF, YG, MP, and BM evaluated and interpreted the results of the experiment and revised the manuscript.
None declared.