This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on http://humanfactors.jmir.org, as well as this copyright and license information must be included.
Emergency departments (EDs) frequently care for individuals with opioid use disorder (OUD). Buprenorphine (BUP) is an effective treatment option for patients with OUD that can safely be initiated in the ED. At present, BUP is rarely initiated as a part of routine ED care. Clinical decision support (CDS) could accelerate adoption of ED-initiated BUP into routine emergency care.
This study aimed to design and formatively evaluate a user-centered decision support tool for ED initiation of BUP for patients with OUD.
User-centered design with iterative prototype development was used. Initial observations and interviews identified workflows and information needs. The design team and key stakeholders reviewed prototype designs to ensure accuracy. A total of 5 prototypes were evaluated and iteratively refined based on input from 26 attending and resident physicians.
Early feedback identified concerns with the initial CDS design: an alert with several screens. The timing of the alert led to quick dismissal without using the tool. User feedback on subsequent iterations informed the development of a flexible tool to support clinicians with varied levels of experience with the intervention by providing both one-click options for direct activation of care pathways and user-activated support for critical decision points. The final design resolved challenging navigation issues through targeted placement, color, and design of the decision support modules and care pathways. In final testing, users expressed that the tool could be easily learned without training and was reasonable for use during routine emergency care.
A user-centered design process helped designers to better understand users’ needs for a Web-based clinical decision tool to support ED initiation of BUP for OUD. The process identified varying needs across user experience and familiarity with the protocol, leading to a flexible design supporting both direct care pathways and user-initiated decision support.
Opioid use disorder (OUD) is an escalating public health crisis that has impacted all regions of the United States and represents a substantial portion of emergency department (ED) visits each year. An estimated 2.1 million people in the United States have OUD [
Buprenorphine (BUP), a partial opioid agonist often combined with an opioid antagonist, is a proven effective treatment for OUD that decreases mortality, withdrawal symptoms, craving, and opioid use [
Clinical decision support (CDS), computerized systems that offer patient-specific assessments or recommendations to clinicians, represents one approach to facilitating and accelerating the implementation process [
However, CDS faces its own challenges, including unintended consequences such as alert fatigue and increased cognitive load [
Across the fields of technology and human-computer interaction, building usable systems has been found to be essential to improve efficiency and reduce errors [
Our objective was to develop a pragmatic, user-centered CDS for ED-initiated BUP and referral to treatment for patients with OUD. The user-centered design process for the development of this tool is described here. We developed this CDS specifically for the purposes of a planned multisystem pragmatic trial to study the effectiveness of user-centered CDS on adoption rates of ED-initiated BUP [
From March to July 2018, we utilized a multiphase, user-centered design methodology for the formative design, development, and evaluation of the EMergency department-initiated Buprenorphine for opioid usE Disorder (EMBED) CDS intervention. Primary phases in this method included (1) needs assessment, (2) initial prototype design, (3) iterative design feedback, and (4) final prototype testing. Formative feedback sessions were approved by our institution’s institutional review board; given the minimal risk of the study and a protocol that did not involve the collection of participants’ private information, all participants gave verbal consent for participation.
Eligible participants included ED clinicians and key stakeholders (including administrative and information technology leaders and ED addiction counselors) from an urban academic level I trauma center with 103,000 patient visits per year. Recruitment for user feedback sessions focused specifically on attending physicians and residents in the second, third, or fourth year of postgraduate medical training. During a 4-month period from March to June 2018, a total of 26 unique participants offered feedback during iterative design, including 14 through informal sessions and 12 through formal user feedback sessions. In addition, 6 participants offered feedback on multiple versions of the design. Informal sessions were conducted in the ED or private administrative offices and lasted 10 to 30 min. Formal sessions were conducted in the Yale Center for Medical Simulation and were approximately 45 min in length. Formal user design sessions were conducted in parallel with both attending and resident physicians by a human factors researcher (JR).
Given our goal to rapidly increase adoption rates of ED-initiated BUP for a subsequent pragmatic trial, we elected to take a pragmatic approach to formative usability evaluation, as described by Mann et al [
The initial phase of design consisted of a focused discussion with key content and context experts as well as 3 ethnographic observation sessions of 2 to 5 hours in length. In the first 2 observations, the lead designer (MM) shadowed attending physicians in the ED. The third observation period focused on the processes of registration and the administration of patient flow through the waiting room and ED. Five 1-hour, individual interviews were then conducted with an ED drug and alcohol program counselor, a drug and alcohol treatment coordinator, an attending physician, and a resident. Interviews captured additional detail on workflow, roles, and user information needs.
After identifying potential users and their information needs, an initial low-fidelity prototype was designed. This prototype focused on key components necessary for implementing the ED-initiated BUP protocol, including modules to evaluate patients for OUD based on the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5) criteria [
With the initial static design complete, an interactive prototype was built in InVision (InVision, New York, NY). This prototype provided users with an interactive navigation and functionality experience. Feedback was gathered both through informal review and through formal user design sessions. Informal review included the distribution of electronic or print versions of all screens in the design to both attending and resident physicians. After verbal consent was obtained, each participant was oriented to the session format and read a case (
Final testing of the interactive InVision prototype consisted of formal user feedback sessions that proceeded until the design team reached a consensus that the prototype would exceed all users’ needs 80% of the time based on the 80/20 rule [
Overall, 4 key topics for design were identified in Phase 1 (
Activation of CDS tools was an early feedback topic. The initial design was an Epic (Epic Systems, Verona, WI) best practice alert (BPA;
A second area of concern for users (throughout all iterations) was avoiding workflow disruptions; they preferred that the tool take no longer than 2 to 5 min to use. Similarly, users highlighted the need for system flexibility to accommodate for the user’s experience level by allowing for decision support as needed as well as a direct care pathway selection with less support for more experienced clinicians.
Needs assessment at baseline and ethnographic observation results.
Needs/topics | How they were expressed |
Appropriate patient identification | Is it possible to have nurses identify patients with OUDa? |
Need to properly explain COWSb to patients, who may understand it as “dope sick” | |
Can discharge instructions for opioid abuse be a trigger to activate CDSc? | |
There needs to be advanced search terms to trigger the CDS system—BPAsd should not be the common denominator for analysis | |
Avoiding workflow disruptions | Avoid BPAs. They are intrusive and are rarely acted upon |
Sometimes, physicians do leave electronic health record to access MDCalc or clinical resources websites | |
Attending physicians usually do not have time for decision support. Better to tailor this toward residents and nurses | |
Entire intervention should take 2-5 mins to increase adoption | |
Streamlining CDS steps | Integrate COWS into the H & Pe template, with integrated decision support and order sets to determine the need for BUPf |
If a user is initiated for OUD diagnosis, then workflow should be streamlined and skip through the diagnostic criteria for OUD and go straight to treatment decision support | |
Understanding treatment process | Should patients be given a 4 mg or an 8 mg dosage? |
Need to have a short SBIRTg included in CDS to assess patient willingness to begin treatment | |
This is not the responsibility of our department but rather the substance abuse program | |
Patients are rarely in the right range of withdrawal to prescribe BUP. Need to have a system to allow them to return at an appropriate time to the EDh | |
Some patients may have a preference for suboxone versus methadone | |
Some providers may have completed the waiver process but may not yet be recognized for it | |
Should we have patients return to the ED for follow-up post BUP administration, using the 72-hour rule? |
aOUD: opioid use disorder.
bCOWS: Clinical Opioid Withdrawal Scale.
cCDS: clinical decision support.
dBPA: best practice alert.
eH & P: history and physical.
fBUP: buprenorphine.
gSBIRT: Screening, Brief Intervention, and Referral to Treatment.
hED: emergency department.
Initial prototype user interface mockup as Epic best practice alert.
Second prototype user interface with optional decision support. DSM: Diagnostic and Statistical Manual for Mental Disorders; OUD: opioid use disorder.
These needs informed the development of the first prototype that incorporated existing paper forms into 1 process. This first iteration presented step-by-step guidance through 6 sequential screens (
We created 5 major prototypes based on ongoing feedback.
Feedback on the initial prototype (described above) focused on streamlining the prototype. This feedback was used to inform the second prototype (
Users found that this second iteration still had too many steps and too much text. They expressed difficulty in locating the decision support elements. Specific suggestions included consolidating steps with more clarity in regard to navigating the treatment options by including a progress bar.
These suggestions led to a complete redesign of the CDS in prototype 3 (
This feedback informed the design of prototype 4 (
Third prototype user interface single-click care pathways. BUP: buprenorphine; COWS: Clinical Opioid Withdrawal Scale; DSM: Diagnostic and Statistical Manual of Mental Disorders; EHR: electronic health record; SL/PO: sublingual/by mouth.
Fourth prototype user interface with care pathways to columns. BUP: buprenorphine; COWS: Clinical Opioid Withdrawal Scale; DSM: Diagnostic and Statistical Manual of Mental Disorders; SL/PO: sublingual/by mouth.
The final prototype design goal was an intuitive, simple layout offering flexibility for direct treatment or user-initiated decision support. In response to navigation concerns, nonessential text was removed, and decision support was presented with blue buttons in the far right column, following the horizontal path for the DSM, COWS, and motivational interview (
Final prototype user interface with decision support moved to the right column. BUP: buprenorphine; COWS: Clinical Opioid Withdrawal Scale; DSM: Diagnostic and Statistical Manual of Mental Disorder; OUD: opioid use disorder.
Final information technology workflow for user-centered clinical decision support intervention based on formative evaluation. ED: emergency department; EMBED: EMergency department-initiated Buprenorphine for opioid usE Disorder; EHR: electronic health record; OUD: opioid use disorder.
We describe the iterative user-centered design process to develop a CDS for ED-initiated BUP. Across 4 phases with 5 major revisions and continuous iteration, we identified user needs for a flexible tool to support members of the care team who could be either experienced users or those new to ED-initiated BUP. Interactive feedback sessions identified key themes throughout the refinement process, including issues of navigation, overall design recommendations, considerations for workflow integration, and questions regarding the treatment protocol. Throughout the design process, how and where to activate decision support represented a key challenge. Early prototype versions provided
This study supports the use of user-centered design. Through both formal and informal feedback, we captured user needs and input that would not be captured in traditional CDS design processes that lack a needs assessment or formative evaluation. Expert review of content ensured accuracy, whereas feedback from users with varying levels of experience highlighted the need for flexible support. Comments from expert users emphasized the need for an option to directly launch the desired care pathway. Although a direct care pathway provides flexibility for expert clinicians, novice clinicians emphasized the need for more structured decision support and clarity in the diagnostic and treatment processes. Specifically, less experienced users welcomed the detailed criteria for OUD diagnosis and withdrawal assessment, instructions for conducting a motivational interview, and clarity regarding BUP dosing. This contrast in user needs presented a design challenge that highlights the importance of sampling participants across the range of user experience levels with the protocol supported by the CDS. A deliberate sampling of participants from earlier iteration sessions as well as new participants provided confirmatory feedback on how recommended changes were incorporated into the design. Finally, design relied on both user comments and existing standards for layout and use of specific design features such as color.
As the CDS supports a treatment pathway, the underlying workflow driving development was identified as clinician workflow. As such, clinicians were the primary users studied. Less focus on other members of the care team could represent a limitation—in particular, if the user population is broadened in implementation at the site where the design process was conducted or at other sites using the tool in the future. A number of users suggested a role for other nonclinician staff in identifying OUD patients in the ED. In particular, multiple clinicians mentioned that the COWS could be completed by a nurse. Therefore, the tool is designed with resources that can be used by or distributed to other members of the care team (eg, nurse, medical student, and addiction counselor). In this way, a nonclinician could still complete the diagnostic or withdrawal assessment, and the clinician could incorporate this assessment into their final care pathway selection.
Given the urgency of the opioid epidemic, we made a conscious decision to take a pragmatic approach to the design and formative evaluation of our intervention. Developing the CDS through a pragmatic approach instead of a traditional academic approach allowed for the rapid inclusion of user feedback in a shorter time frame [
This work represents the initial phase of a larger project for the development, implementation, and testing of the effectiveness of the CDS developed here. Design and user feedback sessions were conducted at a single site, though implementation will include multiple sites and could potentially interface with other vendors’ EHRs. Having a limited group of users engaged in design is practical and not unique to our work. However, we recognize that this introduces the potential for design features supporting local norms and processes that may not be generalizable. To mitigate this potential limitation, we sought feedback throughout the design process from external collaborators as well as guidance from a subject matter expert on ED management of substance use disorder.
Given the devastating toll of the opioid epidemic, this user-centered CDS was developed to give clinicians the tools necessary to engage more people suffering from OUD in effective treatment at a time when they may be particularly open to it [
Given these challenges to adoption, we perceived an opportunity to increase the likelihood of success by employing user-centered design to create the EMBED CDS intervention. Emerging literature supports this approach [
Notably, these studies have shown that including user feedback in the design phase leads to greater effectiveness and efficiency and, ultimately, to a sense of physician ownership of the CDS, which increases its immediate uptake and continued use. They have also highlighted rapid-cycle prototyping with user engagement throughout a design process [
This work describes the design and formative evaluation of a user-centered CDS for ED-initiated BUP. We add to the expanding literature on the design of user-centered CDS tools by describing the process and challenges of designing a flexible tool that supports both novice and expert clinicians in identifying appropriate patients and appropriate care pathways. Future work will include summative usability evaluation and pilot testing of the intervention to further optimize the tool for wide-scale implementation within existing ED workflows in a large pragmatic clinical trial across multiple health care systems. The aim of this subsequent pragmatic trial is to increase adoption of ED-initiated BUP for people suffering from OUD, thereby decreasing morbidity and mortality associated with opioid addiction. Users will also inform pilot implementation in a series of focus groups. Although early engagement of users supports the design process, we anticipate continued support of potential users will be equally important across the project life cycle.
User-centered design script.
Design iterations, feedback, and solutions.
best practice alert
buprenorphine
clinical decision support
Clinical Opioid Withdrawal Scale
Diagnostic and Statistical Manual of Mental Disorders
emergency department
EMergency department-initiated Buprenorphine for opioid usE Disorder
opioid use disorder
Research reported in this publication was supported within the NIH Health Care Systems Research Collaboratory by a cooperative agreement (UG3DA047003) from the National Institute on Drug Abuse of the NIH. This work also received logistical and technical support from the NIH Collaboratory Coordinating Center (U24AT009676). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
ERM, JMR, MM, and GD conceived and designed the work. All authors substantially contributed to the acquisition, analysis, and interpretation of the study data. JMR and OMA drafted the initial manuscript. All authors edited and approved the final version submitted for publication. ERM takes responsibility for the study as a whole.
None declared.