We screened for adult patients who were admitted to the COVID-19 ICU between March 17, 2020, and May 31, 2020. We matched each patient with COVID-19 with an adult patient without COVID-19 who was admitted to the medical ICU (MICU) between December 1, 2019, and February 29, 2020, via propensity score matching.

The propensity score was based on age, gender, and patients’ length of stay. The distribution of the COVID-19 and non–COVID-19 groups’ propensity scores is depicted in Multimedia Appendix 1. The Pearson correlation coefficient between the two distributions was 0.93; the associated P value was <.001. This proved that the variance in the confounding factors between the two patient groups was very small. We focused on patients who were alive at discharge because their hospital stays were relatively complete. This process yielded a sample of 76 critically ill, adult patients—38 with COVID-19 and 38 without COVID-19. In total, 3 patients with COVID-19 required multiple ICU stays. For this study, we randomly selected one stay for each of these patients. Table 1 provides a summary of the demographic characteristics, comorbidities, and outcomes of the investigated patients with and without COVID-19.

The study population had several notable aspects. First, we noticed that there was a disproportionate number of males. Second, while there were more self-reported White patients than patients of other races, the number of White patients in the COVID-19 group was substantially smaller than that in the non–COVID-19 group. Third, patients without COVID-19 had a high incidence of comorbidities; specifically, patients without COVID-19 exhibited the six comorbidities that are common in patients with COVID-19 (ie, those reported by the COVID-19–Associated Hospitalization Surveillance Network) [1]. Fourth, the majority of patients from the two groups (with COVID-19: 29/38, 76%; without COVID-19: 24/38, 63%) were discharged home.

The analysis consisted of two primary components. First, we used network analysis methods to learn about the HCW networks that were involved in the management of critically ill patients. Second, we statistically compared and contrasted the network structures in COVID-19 and non–COVID-19 settings.

We analyzed the actions that HCWs performed with patients’ EHRs to measure worker-worker connections. There are six types of HCW actions, including condition-related (eg, assigning a diagnosis), procedure-related (eg, intubation), medication-related (eg, prescriptions), note-related (eg, writing progress notes), order-related (eg, ordering laboratory tests), and measurement-related (eg, measuring respiratory rate) actions.

Research has shown that a 1-day window is enough to capture the meaningful, collaborative relationships among HCWs [20-23]. Therefore, we assumed that there was a connection between two HCWs who interacted with the same patient’s EHR on the same day. We built a network in which the nodes represented HCWs and the edges indicated the number of days that two HCWs performed actions on the EHRs of the same patients. We built one network for critically ill patients with COVID-19 and another for patients without COVID-19.

The nodes in the COVID-19 and non–COVID-19 networks were defined as follows:

To better interpret the networks, we used an HCW’s specialty (eg, respiratory care) and type (eg, respiratory therapist) to label each node. We combined these factors to define expertise (ie, “specialty: type”; eg, respiratory care: respiratory therapist). Expertise in the COVID-19 and non–COVID-19 networks were defined as follows:

In equations 1-4, Z and EXP were used to describe the composition of COVID-19 or non–COVID-19 networks.

In each network, we used two sociometrics—eigenvector centrality and betweenness centrality—to quantify an HCW’s core and betweenness status in the network, respectively. We used Gephi (ie, an open-source network analysis and visualization software package) [25] to calculate eigencentrality and betweenness centrality values.

Eigencentrality characterizes the degree to which an HCW is densely connected to other HCWs who are also densely connected with other HCWs. A high-eigencentrality HCW is likely to be a core person who actively works with other HCWs when performing actions on EHRs. An example HCW network with eigencentrality values is shown in Multimedia Appendix 2.

The betweenness centrality of an HCW refers to the number of shortest paths between two other HCWs that pass through the HCW in question. An HCW with a broad skill set who cares for a wide spectrum of patients could frequently be in a high-betweenness position. An example HCW network with betweenness centrality values is shown in Multimedia Appendix 2.

We investigated whether differences in the eigencentrality and betweenness of COVID-19 and non–COVID-19 critical care structures were statistically significant at the network and expertise levels. The network-level comparison was conducted by assessing the network as a whole (ie, COVID-19 vs non–COVID-19 networks), while an expertise-level comparison was conducted for each expertise (eg, internal medicine physicians). Since eigencentrality and betweenness are not Gaussian distributed, we conducted a Mann-Whitney U test with a significance level of α=.05. The tests for expertise included at least 8 HCWs and involved Bonferroni correction to account for multiple hypotheses.

We defined the set of inpatient stays in COVID-19 and non–COVID-19 settings as follows:

Since each inpatient stay (ie, s_i) can last for more than 1 day, we defined the jth day of a stay as s_i,j (ie, 1≤j≤l_i); l_i represents the last day of a patient’s hospital stay (ie, s_i). For s_i,j, we calculated the number of HCWs (ie, Ns_i,j,exp_k) in each expertise category (ie, exp_k) who interacted with the EHRs of patient i on day j. For each inpatient stay (ie, s_i), we calculated the average number of HCWs in each expertise category (ie, exp_k) who interacted with the EHRs of the same patient on each day, as follows:

In equation 7, los_i refers to the length of hospital stay (ie, the total number of hours between the start and end times of an inpatient stay divided by 24 hours). Since each inpatient stay may start and end at different times of the day,los_i may be different from l_i. Daily patient staffing intensity was defined as an expertise-level value (ie, ).

To learn about the differences in the daily patient staffing intensities of COVID-19 and non–COVID-19 critical care, we conducted a set of tests. Specifically, for each investigated expertise (eg, internal medicine nurse practitioners), we tested whether critically ill patients with COVID-19 required a significantly higher daily patient staffing intensity than critically ill patients without COVID-19. We focused on the 20 expertise categories with the highest mean daily staffing intensity values in COVID-19 and non–COVID-19 critical care and used the Mann-Whitney U test, which had a Bonferroni-corrected significance level of .05.

We also assessed the differences in the overall patient staffing intensities of COVID-19 and non–COVID-19 critical care in terms of the number of HCWs who were involved in the management of a patient. Overall staffing intensity was defined as the number of HCWs who interacted with the EHRs of a patient with or without COVID-19. Our hypothesis was as follows: critically ill patients with COVID-19 require a significantly higher overall staffing intensity than critically ill patients without COVID-19. We tested this hypothesis by using the Mann-Whitney U test, which had a Bonferroni-corrected significance level of .05.

The number of HCWs, types of HCWs, specialties, and expertise categories in COVID-19 and non–COVID-19 critical care was 759 and 1331, 24 and 24, 92 and 128, and 133 and 207, respectively. These values indicated that patients without COVID-19 required more expertise, specialties, and HCWs. A possible reason for this is that critically ill patients without COVID-19 were admitted to the MICU for a wide range of major conditions. With regard to patient-level values, COVID-19 and non–COVID-19 critical care consisted of 79.5 and 88.2 HCWs, 9.8 and 10.6 types of HCWs, 23.0 and 27.1 departments, and 29.2 and 34.0 expertise categories, respectively. The patient-level values for COVID-19 and non–COVID-19 critical care were highly similar.

Figure 1 illustrates the union of the 10 COVID-19 and non–COVID-19 critical care expertise categories with the largest proportion of HCWs. It can be seen that, aside from residents and registered nurses with myelosuppression expertise, the COVID-19 setting had higher percentages of different types of HCWs than the non–COVID-19 setting. These results demonstrate how the Vanderbilt University Medical Center assigned full-time, nontrainee HCWs to the task of managing critically ill patients with COVID-19 and reduced the number of residents during the COVID-19 pandemic.

There were no registered nurses with myelosuppression expertise in the COVID-19 setting. Upon further analysis, we found 4 patients with COVID-19 and cancer, but none were in need of invasive intervention at the time of their care.

Figure 2 presents the HCW networks in COVID-19 and non–COVID-19 critical care from eigencentrality and betweenness perspectives. In the figure, it can be seen that the majority of HCWs in the COVID-19 network are larger in size (ie, higher eigencentrality) than those in the non–COVID-19 network. This indicates that HCWs are much more highly and densely connected in the COVID-19 network than those in the non–COVID-19 network. We performed a test to measure the differences in eigencentrality between the COVID-19 and non–COVID-19 networks. The results indicated that the two networks had significantly different median eigencentrality values (COVID-19: 0.096; non–COVID-19: 0.057; P<.001).

After removing expertise categories that had less than 8 HCWs, we performed pairwise tests on the remaining 12 expertise categories. The results of these tests are provided in Table S1 in Multimedia Appendix 3. There were several notable findings. First, we observed that internal medicine physicians in the COVID-19 network had higher eigencentrality values than those in the non–COVID-19 network (P=.001). Figure 2 also shows the subnetworks of internal medicine physicians in the COVID-19 and non–COVID-19 networks from an eigencentrality perspective. From the figure, it can be seen that internal medicine physicians in the COVID-19 network were connected with each other, while those in the non–COVID-19 network were separated. Second, the resident physicians in the non–COVID-19 network had higher eigencentrality values than those in the COVID-19 network (P=.002). Figure 3 shows that there were many residents across the entire non–COVID-19 network. However, the number of residents was much smaller in the COVID-19 network.

There were no significant differences in the betweenness of the two networks (COVID-19 network: median 0.002; non–COVID-19 network: median 0.003; P=.22). However, nurse practitioners in the COVID-19 network had significantly higher betweenness values than those in the non–COVID-19 network (P<.001). The complete set of test results for expertise categories with nonsignificantly different betweenness values is provided in Table S2 in Multimedia Appendix 3. Figure 2 also shows the subnetworks of nurse practitioners in the COVID-19 and non–COVID-19 networks from a betweenness perspective. From the figure, it can be seen that nurse practitioners had a larger number of connections in the COVID-19 network than those in the non–COVID-19 network. In Figure 2, it can also be seen that nurse practitioners are in the central part of the COVID-19 network and serve as connective bridges between other HCWs. Given that betweenness reflects an HCW’s access to a wide spectrum of patients, a nurse practitioner in the COVID-19 collaboration structure can build connections among HCWs who are not directly connected. Internal medicine physicians and nurse practitioners were the core of the COVID-19 network. As shown in Figure 3, these two types of HCWs had a larger number of connections in the COVID-19 care setting than those in the non–COVID-19 setting.

In total, 41,903 (mean 1103) actions were performed with the EHRs of patients with COVID-19 and 44,131 (mean 1161) actions were performed with the EHRs of patients without COVID-19. There were no statistically significant differences in the number of actions performed with the EHRs of patients with and without COVID-19 (P=.32). The differences in the number of expertise categories and HCWs who performed actions on the EHRs of critically ill patients with and without COVID-19 were also not significant (expertise categories: P=.08; HCWs: P=.19). The complete set of results is provided in Table S3 in Multimedia Appendix 4.

Table 2 shows the differences in the patient staffing intensities of COVID-19 and non–COVID-19 critical care for each expertise category. The union of 20 COVID-19 and 20 non–COVID-19 expertise categories yielded 24 categories.

There was a larger number of internal medicine nurse practitioners, cardiovascular ICU registered nurses, and surgical ICU registered nurses who performed daily actions on the EHRs of critically ill patients with COVID-19 compared to the number of those who performed daily actions on the EHRs of patients without COVID-19. In contrast, the EHRs of patients without COVID-19 were managed by a larger number of resident physicians (ie, those with medicine and emergency medicine expertise). These differences were statistically significant (Table 2).

We also found that expertise categories were not statistically different in terms of daily patient staffing intensity. Such categories included radiology physicians, nephrology physicians, pulmonary/allergy physicians, emergency medicine physicians, MICU registered nurses, and respiratory therapists.

There are no universal guidelines for HCW staffing in ICUs. To date, ICU staffing studies have focused on organization models (eg, open, closed, and hybrid models), and few have examined collaborations among HCWs. In this study, we used a novel method for learning about collaborations among HCWs and building corresponding networks. We measured eigencentrality and betweenness centrality to quantify the core and betweenness statuses of HCWs and identify several significant differences between the COVID-19 and non–COVID-19 network structures. Differences in the collaboration structures between the two networks mirrored those in intentional strategic planning structures across the health care system. For instance, there was a significant difference (P<.001) in the number of resident physicians between the COVID-19 and non–COVID-19 structures because our medical center assigned full-time, nontrainee HCWs to the management of critically ill patients with COVID-19. This mirrors resident protection strategies that were implemented during the outset of the COVID-19 pandemic by the National Graduate Medical Education. Figure 3 shows the subnetworks of resident physicians in the COVID-19 and non–COVID-19 networks. It can be seen that the non–COVID-19 network has a larger resident network than the COVID-19 network, and the connections between residents are more complex than those in the COVID-19 network. This suggests that resident physicians are highly active with respect to the management of critically ill patients in a non–COVID-19 setting.

Beyond collecting data on basic strategic planning methods (ie, the reduction of the number of residents) in the management of critically ill patients with COVID-19, we also learned about the aspects of collaboration structures that are important for the management of teamwork but are not explicitly documented in existing ICU staffing plans. We found that internal medicine physicians and nurse practitioners in the COVID-19 collaboration structure were more active (ie, high eigencentrality or betweenness) than those in the non–COVID-19 collaboration structure. As shown in Figure 3, internal medicine physicians and nurse practitioners connected more frequently in the COVID-19 network than those in the non–COVID-19 network. This phenomenon suggests that they are core members in collaborations that relate to the management of critically ill patients with COVID-19.

Combining knowledge on connections among HCWs with their eigencentrality and betweenness values in the collaboration network can assist HCOs with designing and developing more specific staffing strategies, which can potentially improve care quality and patient outcomes. The network analysis methods and team structures that are depicted in our retrospective study can be used in a prospective setting. Our COVID-19 and non–COVID-19 networks can be used to identify the characteristics of a newly established or modified team. For instance, if a COVID-19 ICU has plans for creating a team to care for the increasing number of patients, the eigencentrality and betweenness centrality of each HCW and the HCW relationships that we learned about in our COVID-19 network can be used as evidence for identifying the characteristics of the newly created team. The team creators can evaluate the leadership (ie, eigencentrality), robustness (ie, betweenness), and familiarity (ie, the strength of the relationships between HCWs) of the newly established team. They can also dynamically add or remove an HCW from the created team and measure changes in leadership, robustness, and familiarity, which will help team creators with finding their desired team.

In this study, we did not investigate temporal networks or team dynamics, which are essential to HCOs that monitor and manage team dynamics. However, researchers can use the network analysis methodologies that were developed in our study to identify temporal networks, such as daily, weekly, or monthly networks. They can also use the sociometrics that we developed to quantify changes in temporal network structures. For instance, HCOs can use our network analysis methodologies to temporally measure the relationships among internal medicine physicians, nurse practitioners, and residents; and quantify the weekly, monthly, and yearly changes in these relationships.

There are several limitations in this study that should be recognized. These limitations serve as opportunities for further investigation. First, this study was based on a small number of critically ill patients with COVID-19. Although our sample had sufficient power for analyzing the differences in the eigencentrality, betweenness, and patient staffing intensities for several types of HCWs in COVID-19 and non–COVID-19 critical care, a larger volume of data is needed to obtain statistically meaningful results. Second, comorbidities could have impacted team structures; however, matching the comorbidities between patients with and without COVID-19 can lead to certain risks. According to our observations, matching comorbidities between the two cohorts will considerably enlarge the study window (ie, >3 years) for the non–COVID-19 cohort. However, non–COVID-19 care teams can drastically change over time, making the study of the non–COVID-19 team structures less meaningful. Therefore, in this study, we focused on the most important confounding factors (ie, age, gender, and the length of stay) that characterize team efficacy and may impact team structures. Additionally, our study's primary focus was to learn about team structures in COVID-19 and medical ICUs, which allowed for some degree of variance in comorbidities. Third, the characteristics of the COVID-19 structures that we learned about during this single-center study could provide HCOs with reference data for assessing their own COVID-19 ICU structures. However, our medical center is an institution that intentionally developed a nurse practitioner–centered organizational structure. This should be considered when interpreting our results and findings. To learn about general COVID-19 ICU collaboration structures, researchers need to conduct analyses that account for multiple HCOs. Fourth, there was a lack of standard terminology for characterizing HCO departments and the roles of HCWs. Although there are taxonomies for describing clinician specialties [26-28], these tend to neglect the nonphysicians who play vital roles in the management of patients. It is clear that common data models for department names and HCW types would improve the quality of our study and assist other institutions with using our methodology. Fifth, we assumed that two HCWs would have a connection when they performed actions on the EHRs of patients. Although such an assumption can help with identifying collaboration relationships between HCWs, it may also have resulted in the identification of many spurious relationships.

HCOs have been planning and refining their staffing strategies to provide more efficient and effective care to patients with COVID-19. However, there are few efficient methodologies for assessing the execution of collaboration structures in practice, especially those for assessing the cross-disciplinary connections among HCWs. In this study, we demonstrated how data on the use of a large academic medical center’s EHR system could be used to learn about the collaboration structures in COVID-19 critical care (ie, through network analysis methodologies). HCOs can use our network analysis approaches and data on eigencentrality, betweenness, and patient staffing intensities to characterize HCW roles in collaboration networks during the COVID-19 pandemic or future events. Furthermore, research on how HCWs are connected has created an opportunity for studying the relationships among team structures, care quality, and patient safety.