Using multiple user-centered design strategies [39-42], the Discharge Today Tool was iteratively developed from July 1, 2018, to July 31, 2019, and deployed to hospitalists and other clinical staff on March 5, 2019. This tool was designed to integrate with customizable EHR patient worklists used by most clinicians and staff members providing clinical care to hospitalized patients (Figure 1).

In the provider view, hospitalists may access the Discharge Today tool via the D/C Today? Primary column in their EHR patient list. Using this tool, hospitalists may document patient discharge readiness (definite today, possible today, tomorrow, in 24 to 48 hours, or in more than 48 hours) and if the hospitalist is waiting on any final care before the patient can be discharged. Via the partner view, the data collected by the Discharge Today tool is shared with ancillary and consulting clinicians in the Single—D/C Today—What are you waiting on?—Ancillary and the Single—D/C Today—What are you waiting on?—Consultant columns in their EHR patient worklists. The definitions for the discharge readiness statuses are as follows: Definite-very high probability that the patient will be discharged today unless there are unexpected changes during the day. For example, if you have a patient who is clinically ready for discharge but needs home oxygen set up, this patient would be considered a definite discharge, awaiting respiratory therapy. Possible-some probability that the patient could be discharged today. For example, if you have a patient with complex health conditions waiting for subacute nursing facility placement, this patient would be considered a possible discharge, awaiting placement. Tomorrow: very likely that the patient could be discharged tomorrow. In 24-48 hours: the patient is not going home today but will likely be discharged in the next 24 to 48 hours; >48 hours: very unlikely that the patient would be discharged within the next 48 hours.

Hospitalists caring for the patients are able to easily document discharge readiness (definite today, possible today, tomorrow, in 24-48 hours, or in >48 hours) [21] and whether the hospitalist is waiting on any final care before the patient can be discharged. The data collected by the Discharge Today tool are also disseminated via EHR patient worklists, which are EHR-based reports designed to summarize patient care for clinicians using the EHR, and via an automatic paging functionality directly from the EHR. Details can be found in our study describing stakeholder engagement and the user-centered design approaches applied [43].

Addressing communication challenges by improving the efficiency and accuracy of communication may reduce inefficiencies and errors in health care, including during the discharge process. The Discharge Today tool fosters flexibility and agility in communication, including asynchronous communication, feedback loop capabilities, different functionalities according to user role, and allowing for both formal and informal communication.

This study was conducted as a prospective, single-center, quasi-experimental, pre-post study designed to evaluate the effectiveness of the Discharge Today tool. The study was approved by the Colorado Multiple institutional review board as a quality improvement project and funded by a small pilot grant.

This study was conducted at the University of Colorado Hospital, a 678-bed tertiary care center with approximately 12,000 medicine discharges per year.

Hospitalist physicians and APPs were trained as they started on service and asked to use the Discharge Today tool every day that they were on service with all patients assigned to their team. Clinicians were asked to update patient discharge readiness statuses first thing in the morning and throughout the day as discharge readiness and needs evolved. Patients who were expected to be discharged >48 hours out only needed an update every 3 days as the tool would automatically unpopulate the patient status if unchanged after 3 days to ensure the most accurate and up-to-date information. Clinicians received a small incentive for participation (ie, coffee or other small tokens of gratitude that were funded by the small grant).

Patients were enrolled in this study as part of their regular hospitalization if they presented during the study period. Patients already in the hospital at the start of the pilot implementation period (March 5, 2019, to July 31, 2019) were excluded from the analysis. Patients admitted on or after March 5, 2019, and discharged on or before July 31, 2019, were assigned to the pilot implementation period.

All patient-level clinical and quality outcomes data queried from the hospital EHR data warehouse were collected as part of their hospitalization process. We queried data from the EHR data warehouse for any patient admitted to the hospital and assigned to a hospital medicine service during the preimplementation period (October 1, 2018, to March 4, 2019), the pilot implementation period (March 5, 2019, to July 31, 2019), and the postimplementation maintenance period (August 1, 2019, to December 31, 2019).

To assess adoption, we documented the number of users who added the tool to their patient worklists within the EHR. To assess both reach and implementation, we queried each time data were entered into the tool by a clinician, including discharge readiness status, when patients assigned a definite discharge status would be ready to be discharged, and what ancillary services or tasks might be needed, such as rehabilitation services, respiratory therapy, pharmacy, social work, care management, medical improvement, or consultant services (Table 1).

Surveys were conducted using REDCap (Research Electronic Data Capture)—a secure, web-based application for building and managing web-based surveys and databases [44]—to evaluate the usability of and experience with hospital medicine physicians, APPs, nurses, care management, and other clinical staff during the pilot implementation period. The complete survey results are reported in a study describing the stakeholder engagement and user-centered design approaches that we applied [43].

Primary outcomes for assessing the effectiveness of this tool were (1) time of day the physician entered the discharge order, (2) time of day the patient left the hospital, and (3) hospital length of stay. Secondary outcomes were (1) proportion of patients for whom a discharge order was entered before 11 AM and (2) proportion of patients discharged before 11 AM, both metrics commonly used to evaluate patient flow. We also queried our data warehouse for the type of patient (inpatient or observation patient), Charlson Comorbidity Index, type of team (physician alone, physician with APP, physician with resident, or physician with APP and resident), proportion of days in the hospital that discharge status was documented for each patient (0%-25%, 25%-50%, 50%-75%, or >75%), and the number of patients assigned to a team at 7 AM (starting morning census).

On the basis of the original planned interrupted time series design, to maximize feasibility against sample size, we allowed for approximately 20 weeks of data collection during each period; that is patients discharged during the preintervention period, patients discharged during the pilot intervention period, and patients discharged during the postintervention period. On the basis of data from 2017, we anticipated an average of approximately 140 discharges per week. However, to account for clustering within providers and teams, the analysis shifted to a mixed modeling approach. Although no post hoc power analysis was conducted, >4000 patients were discharged in each time period.

We estimated means and SDs for continuous variables when approximately normally distributed (as assessed by visual inspection of histograms), medians and IQRs when not, and frequencies for categorical variables. Descriptive statistics were computed for patient, clinician, and team characteristics.

Patient-level, clinician-level, and team-level covariates, hypothesized a priori to be associated with the time of discharge order, time of discharge, and hospital length of stay, were included in multivariable analyses. Models for discharge order time, actual discharge time, and hospital length of stay were adjusted for (1) type of patient, (2) Charlson Comorbidity Index, (3) teaching service, (4) staffed with an APP, (5) discharge to postacute care, (6) starting morning census per team, (7) team, and (8) physician (Table 1). The discharge order time and discharge time models were also adjusted for hospital length of stay.

We used linear mixed modeling for the analysis of the time of day the hospitalist physician entered the discharge order into the EHR, the time of day the patient left the hospital, and the hospital length of stay. We converted time to hours elapsed since midnight on a 24-hour clock for modeling. For our binary outcomes, specifically, whether a discharge order was entered before 11 AM and whether a patient was discharged before 11 AM, a generalized linear mixed model with logit link function and binary response distribution was used. The intervention period, that is preimplementation and pilot implementation, was the independent variable of interest. Team and discharging physicians were included as random effects in all models to account for correlation between patients cared for by the same team and the same physician. Given that hospital length of stay is right skewed, this variable was log-transformed to facilitate regression analysis. We reported a relative difference in hospital length of stay by exponentiating the coefficient, subtracting 1, and expressing the result as a percentage [45].

Secondary analyses were performed to determine whether potential effect modification was supported by the data. We hypothesized that the Discharge Today tool would help hospitalist physicians with a high number of patients on their team triage work and enter discharge orders more quickly. To test this hypothesis, we included an interaction term between the team starting morning census and intervention period, allowing for the intervention’s effect to depend on daily patient volume [21,46]. We also hypothesized that the Discharge Today tool might be more effective for teams staffed with an APP, allowing teams to triage and divide work more efficiently [47-49]. To test this hypothesis, we included an interaction term between whether a team was staffed with an APP and intervention period, allowing the intervention’s effect to depend on the presence of an APP.

Patients with missing data on any variables necessary for a specific analysis were excluded from that analysis. All statistical analyses were performed using SAS Enterprise Guide 8.1 (SAS Institute Inc).

During the preimplementation period—October 1, 2018, to March 4, 2019—4558 patients were discharged from 1 of 18 hospital medicine teams at the University of Colorado Hospital by 57 hospitalist physicians (Table 2). During the pilot implementation period—March 5, 2019, to July 31, 2019—4707 patients were discharged from 1 of 18 teams by 62 hospitalist physicians.

During the implementation period, 84.85% of the patients discharged were assigned a discharge status. The most common barriers identified were medical improvement, placement, subspecialty consults, physical therapy, and social work or care management (Table 3).

Of the 56 hospitalists who discharged a patient during the pilot implementation period, 46 (82%) used the tool for patients assigned to their teams. During the pilot implementation period, 352 users, including physicians, APPs, residents and medical students, nurses, physical and occupational therapists, care managers and social workers, and pharmacists, added the tool to their patient worklists. Of these users, 86% (48/56) of hospitalist physicians and 88% (29/33) of hospitalist APPs added the tool to their EHR patient lists. Physicians, APPs, residents, and medical students added the primary column in which they entered a discharge readiness status daily, and other clinical staff, including nurses, physical and occupational therapists, care managers and social workers, and pharmacists, added the read-only columns where the discharge readiness status entered by providers can be viewed. In addition, in some cases, the tool was added to shared patient worklists, which meant that >352 clinical staff were using the tool.

Hospital medicine physicians, APPs, nurses, care management, and other clinical staff reported in a survey conducted during the pilot implementation period that the tool did not adversely affect their workflow (21/24, 88% of the providers, and 34/34, 100% of clinical staff) and was helpful for managing the patient discharge process (32/34, 94% of clinical staff).

In both unadjusted effectiveness analysis and after adjusting for prespecified confounders, we did not find a significant reduction in the time of day the discharge order was entered into the EHR by the discharging physician during the pilot implementation period compared with the preimplementation period (Table 4).

In the secondary analyses for effect modification, we observed an interaction effect between intervention period and starting morning census (P=.07; Figure 2).

The time of day the discharge order was entered into the EHR by the discharging physician varied according to the number of patients assigned to a team at 7 AM each morning in the pilot implementation period compared with the preimplementation period.

Specifically, the time of day the discharge order was entered into the EHR by the discharging physician decreased by an additional 2.9 minutes per patient for every 1-patient increase in morning census during the pilot implementation period compared with each 1-patient increase in morning census during the preimplementation period. However, we did not find any evidence of effect modification for the intervention by the presence of an APP (Table 4).

Although in unadjusted and adjusted analyses the time of day the patients left the hospital for the pilot implementation period compared with the preimplementation period did not change significantly, we found, in secondary analyses conducted to investigate effect modification, that the average time of day the patients left the hospital decreased for every 1-patient increase in morning census for a given team during the preimplementation period compared with the pilot implementation period by 3.0 minutes (P=.07; Figure 3).

The time of day the patient left the hospital after being discharged varied according to the number of patients assigned to a team at 7 AM each morning in the pilot implementation period compared with the preimplementation period.

In addition, the average time of day the patients left the hospital decreased for teams staffed with an APP during the preimplementation period compared with the pilot implementation period by 19.1 minutes (P=.04; Table 4; Figure 4).

The time of day the patient left the hospital after being discharged varied according to whether a team was staffed with an advanced practice provider in the pilot implementation period compared with the preimplementation period.

In the unadjusted analysis, hospital length of stay did not change significantly. After adjusting for prespecified confounders, we observed a trend toward reduction in hospital length of stay for the pilot implementation period compared with the preimplementation period (decrease of 3.7%; P=.06). We did not observe significant changes in the length of stay from preimplementation to pilot implementation under different conditions, such as high-census days or presence of an APP on a patient team; that is, no significant interactions between the intervention period and these variables were detected (Table 4).

Neither of the secondary outcomes—proportion of patients for whom a discharge order was entered before 11 AM and proportion of patients discharged before 11 AM—was found to significantly improve after introduction of the Discharge Today tool in unadjusted analysis, after adjusting for prespecified covariates, or under different conditions (Table 4).

To test whether the effects of this tool persisted in a maintenance period during which stakeholder engagement efforts were curtailed, we compared the outcomes of the pilot implementation period with those of the postimplementation period using mixed effects models (Table 5). Adjusting for prespecified covariates, we observed a significant reduction in the time of day the discharge order was entered into the EHR for teams staffed with an APP during the postimplementation period compared with teams staffed with an APP during the pilot implementation period (an average decrease of 20.1 minutes per patient (95% CI –36.1 minutes to –4.0 minutes; P=.01; Figure 5).

The time of day the discharge order was entered into the EHR by the discharging physician varied according to whether a team was staffed with an advanced practice provider in the postimplementation (maintenance) period compared with the pilot implementation period.

However, no other outcomes improved significantly from the pilot implementation period to the postimplementation period.

The important findings of this work are as follows:

Other studies have described similar tools, such as the Red, Yellow, or Green Discharge tool [50] and the Kanban web-based application [51]. However, these tools were not integrated into the patient worklist, an EHR workspace that is commonly used across clinical staff, including physicians, APPs, residents, medical personnel, nurses, physical and occupational therapists, care managers and social workers, and pharmacists, thus enhancing the real time, multidisciplinary communication about discharge readiness. Recently, a similar tool was described in the pediatric setting, which was associated with an increase in the proportion of patients discharged before noon [52]. However, unlike our Discharge Today tool, this tool did not allow providers to document any tasks or clinical care required before the patient could be discharged. In addition, our Discharge Today tool allowed providers to note what time of day (before 11 AM, before 2 PM, or after 2 PM) a patient might be discharged.

Previous work has shown that hospital census and census on teams can affect the overall flow of hospitals [21,46]. In this pilot study, although the primary outcomes evaluated were nonsignificant in analysis, interactions between the number of patients assigned to a team in the morning or teams staffed with an APP and the intervention suggest that there may be effect modification at work such that the intervention is effective in certain subgroups or under certain conditions. After including starting morning census per team as an effect modifier, although nonsignificant, there was a reduction in the time of day the discharge order was entered into the EHR by the discharging physician and in the time of day the patient left the hospital. In addition, when teams were staffed with an APP, the use of this tool was associated with significantly earlier discharges beyond that seen for teams without an APP. Research has shown that discharging patients just 1 hour earlier alleviates hospital crowding and reduces access blocking [23,24]. Although we were not able to achieve the goal of discharging patients an hour earlier on average, the incremental gains from multiple solutions implemented across many patients may be additive to moving discharge times and could result in improvements in patient flow and hospital capacity.

Finally, during the maintenance period, when teams were staffed with an APP, discharge orders were entered significantly earlier by the discharging providers. Our APPs were early adopters of the tool and continue to be heavy users, which may have produced the observed improvements. We believe that these findings highlight the importance of APPs in the success of discharge initiatives. Although other studies have suggested that a multidisciplinary approach will improve the early discharge of patients [22,53,54], our study specifically investigates the effects of APP involvement. Our study suggests that APPs may be vital partners in work undertaken to improve the discharge process in an adult medicine population. A pilot study of a multidisciplinary team led by an APP and staffed by a pharmacist and nurse demonstrated a significant improvement in discharge times for patients seen by this team [55]. Similarly, previous research has shown that most providers do not prioritize discharges first as they are tending to other patients [21]; thus, using a team approach to patient care may be advantageous when working to improve throughput metrics.

Our results suggest that some effects of the tool continued even after robust stakeholder engagement efforts were reduced to periodic reminders. We observed a significant reduction in the mean time of day the physician entered the discharge order when a team was staffed with an APP over the reduction observed when a team was not staffed with an APP during the postimplementation period compared with the pilot implementation period. During this time, the hospital medicine triagist, an APP-staffed position, started using the tool for bed management, suggesting that APP use may have become more deliberate. Sustained improvement after demonstrating the effectiveness of an intervention is not often evaluated, likely because of constraints of time and available budget [56]; however, without consideration of the relevant contextual factors, evaluating whether an intervention has resulted in sustainable improvement may prove elusive [57].

Finally, our tool had high adoption and use rates, with relatively minimal incentives to do so. There were several features of our project that helped to improve adoption and use. Our stakeholder engagement process—both preimplementation and during the pilot implementation period—was robust, resulting in a product that was developed for and by frontline staff members and clinicians. In addition, the Discharge Today tool was integrated into the current workflow (ie, EHR worklists) and color coded, which serves as a visual prompt for both clinicians and frontline staff to use.

Given that this was a pilot study of this tool aiming to evaluate the user-centered design approach taken, adoption of the tool, and effectiveness in a sample of providers delivering care to hospitalized patients, the tool had not been fully scaled up across hospital settings, thus potentially limiting the effectiveness. Although our tool had high adoption rates with our target populations, it was challenging to fully implement it across all care teams across an entire hospital, and thus it took some time to scale. Since this pilot, an initiative to use the existing EHR applications to better support patient flow has been launched. As an aspect of this work, the Discharge Today tool has been integrated with other EHR functionalities to capture patients’ progress toward discharge, and any roadblocks to discharge were implemented. We suspect that as adoption continues and additional features are added, adoption will further scale, and perhaps larger effects on the desired outcomes could be seen. EHR tools intended to change clinician behavior require continuous iterative optimization and evaluation to realize their full potential.

Our study has a number of strengths. First, we describe a novel tool to communicate discharge readiness in real time to key stakeholders. Second, we had remarkable engagement by our clinicians and frontline staff members, with high use rates and overall positive feedback. Third, although our study was conducted at a single center, our sample included >4000 patients, almost 60 physicians, >40 APPs, almost 90 residents and medical students, and >160 frontline staff members during the pilot implementation period. Fourth, we have accounted in statistical modeling for the contextual factors that we hypothesized a priori could influence the effectiveness of the tool by including effect modifiers for the number of patients assigned to a team in the morning and a team staffed with an APP.

Our study also has several limitations. First, it was performed at a single university-affiliated academic hospital and was a quality improvement initiative using a pre-post study design; therefore, the results might not be generalizable to other types of institutions or other patient populations. Second, throughout our study period, we continued to optimize the tool, and thus the full effect of the tool may not have been realized at the end of the pilot implementation period. Third, for this analysis, we assessed both the discharge order time and the discharge time; however, we did not evaluate the circumstances around that gap (ie, when the patient was actually ready for discharge and any reasons for delays between the time the order was entered and the time the patient could leave). Future analyses would benefit from assessing whether the use of the Discharge Today tool closes the gap between when the discharge order is entered and when the patient is actually discharged. Fourth, there are most likely unknown confounders at work that we did not identify or include as adjustment factors. Fifth, although we did ask the providers to update the tool first thing in the morning and throughout the day as patient statuses changed, we did not ask that they otherwise change their workflow. Before, during, and after this study, there have been consistent institutional efforts asking providers to prioritize discharges first. It is possible that by asking providers to update the status, that alone could have resulted in improved discharge times regardless of the tool used; however, even with discharge-before-noon initiatives implemented at most places, <10% of hospitalists typically round on discharges first [21]. On the basis of previous literature and mixed successes around early-discharge initiatives, we believe that a multipronged approach is likely needed, including ensuring reasonable workloads, optimizing care team models, and improving communication processes [4,35,50,58]. This tool offers a potential component that is minimally intrusive and communicates across disciplines.

Finally, we were unable to account for other initiatives (eg, huddles held throughout the day to discuss patients who may be able to be discharged) intended to improve discharge times and lengths of stay that were taking place concurrently with our Discharge Today tool implementation.

We have described a unique, EHR-based approach to improving communication around discharge in real time with all care team members, regardless of their physical location in the hospital, that improves discharge times and lengths of stay. The Discharge Today tool allows for real time documentation and sharing of discharge statuses, and our results suggest that the tool may be useful for improving discharge times and lengths of stay, particularly if a team is staffed with an APP or possibly in higher-census situations.