Chronic obstructive pulmonary disease (COPD) is one of the most common chronic respiratory conditions in the world [1]. COPD typically arises from long-term exposure to airway irritants, such as cigarette smoke or air pollution [2-5]. It causes nonreversible chronic obstruction of the airways, resulting in breathlessness, fatigue, and frequent chest infections [6-8]. These symptoms can make it difficult to engage in daily activities, such as leaving home, socializing, and getting dressed [9,10]. Exposure to respiratory infections, physical exertion, smoke inhalation, and environmental factors such as air pollution can worsen symptoms [4,11-13]. COPD is a considerable challenge for millions of people and many health care services around the world [14,15]. It is estimated that ≥200 million people have COPD worldwide [16], with approximately 16 million people in the United States and 1.2 million in the United Kingdom [16,17]. In the United Kingdom specifically, COPD generates ≥140,000 hospital admissions annually, with 97% of these admissions being for emergency care [17,18].

When caring for patients with COPD, health care professionals (HCPs) must make timely and informed decisions to treat patients effectively. Clinical decision-making is a complex process that involves using medical knowledge to make decisions about care [19]. Making informed clinical decisions about patients with chronic conditions such as COPD requires quick access to a range of information about the patient and their medical history [14,20,21]. Insight into symptoms, quality of life, medications, past interventions, and results of recent clinical tests can add important context to inform decisions [21,22]. For example, by knowing the frequency and context of a patient’s respiratory exacerbations (ie, flare-ups of their COPD), HCPs can suggest more personalized interventions that may be more effective.

However, data relevant for chronic condition care are heterogeneous and often buried across paper notes or electronic records or held by other HCPs involved in the patient’s care [14,22-24]. Effectively collating and visualizing data about a patient’s chronic condition has the potential to support timely and informed care decisions [23-25]. This presents an opportunity to explore how digital technology can be designed to provide timely data-driven support for HCPs. Digital technology, which provides data that support decision-making, is termed data-supported decision-making (DSDM) technology. Designing DSDM technologies to support demanding health care contexts requires us to work closely with HCPs to explore their needs and expectations. An appreciation of the broader complexities of health care work is also needed [23].

In response, our research aimed to actively engage HCPs in considering how DSDM technologies could support clinical decision-making in the context of COPD care. Through extensive engagement with HCPs, we identified a set of data interaction scenarios relevant to their practice. We then created an interactive web application as a tool to visualize these scenarios and facilitate discussion about how DSDM technologies might support their work.

Electronic health records [26], dashboards [27,28], and clinical decision support systems [29,30] are types of DSDM technologies used across health care. They present pertinent information to inform clinical decision-making. Dashboards are a prominent form of DSDM technology that can improve patient care [27,28,31-34]. Dashboards aggregate and visualize data in ways that produce insights to users. For instance, to support users to increase the number of patients undergoing health screening [31], identify possible high-risk medication prescribing scenarios [32], track in-patients in mental health wards [33], and effectively use patient-reported outcome data for cancer care [34].

While reviewing the literature on how DSDM technologies are designed and used in health care, we found that many studies focused on measuring the clinical effectiveness or quantified outcomes achieved using the technology [25,28,29,31,32,35,36]. Although these studies are crucial for establishing the quantitative impact of DSDM technologies on health care, they do not document the design process or provide detailed user insights about the technology. This knowledge is crucial to inform how DSDM technologies should be designed for real-world contexts from a human-centered perspective [37-40]. Collaborating with HCPs during the design process can unearth important sociotechnical and human-data interaction considerations required to build successful technologies [41-43]. Sociotechnical considerations investigate the design and implementation of systems based on technical and social dimensions [44]. Human-data interaction investigates how people interact with, interpret, and understand data [45,46]. Legibility, agency, and negotiability are key human-data interaction challenges [46]. Legibility refers to making data and algorithms transparent and comprehensible. Agency is the capacity to act on data and data implications. Negotiability relates to re-evaluating decisions about data and data processing as contexts change.

We found that a small number of studies have investigated the challenges involved in designing DSDM technologies for specific clinical contexts [47-49]. Bardram and Nørskov [48] and Sarcevic et al [47,49] took a user-centered approach to inform the design of context-aware dashboards for high-risk settings. Their prototypes focused on patient safety in operating theaters [48] and trauma resuscitation [47,49] and were evaluated with staff during a simulated clinical scenario. The researchers then revised how the data were presented to effectively support decision-making in these contexts, for instance, supporting dynamic information visualization in the fast-paced setting [48] and excluding audio feedback that could startle patients [49]. The nuances of clinical work were understood by engaging hospital staff during the design process, highlighting the value of partnering with end users when designing DSDM technologies [44,50]. However, there is more to learn about designing DSDM technologies outside the specific use case of high-risk settings [47-49]. Crucially, we need to explore wider everyday data interactions to inform the design of DSDM technologies in health care settings. This can enhance our understanding of the possibilities for DSDM technologies in health care.

Scenario-based design has been used in previous studies to evaluate health care technologies [43,48,49,51,52]. Scenarios, which are task-driven descriptions of work instances, focus on how a system can support human activities [51,53]. They are effective for the qualitative systematic evaluation of usability, suitability, and user experience of a technology or prototype [51,53]. This involves users completing tasks presented as scenarios on the proposed tool, presented as scenarios, and evaluating their experience.

Scenarios are particularly effective in eliciting detailed feedback from users without deploying a full system in clinical practice [52]. For example, Bardram [51] used scenarios to approach the redesign of an information system used in hospitals. Using scenarios allowed the hospitals’ existing activities to remain central to the design and evaluation, helping to focus on how the system could support both current and future activities. Scenarios are a creative thinking tool for envisaging how systems can support work and how it is organized [51]. Given our desire to capture rich details about how DSDM technologies could support HCPs with COPD care, scenario-based design was an appropriate method for this study.

This study explored how DSDM technologies could support HCPs in their decision-making regarding COPD care. We achieved this by presenting an exploration of a scenario-based research tool called Respire. Respire is an interactive web app that presents HCPs with data interaction scenarios to support their decision-making about their patients with COPD and service. We designed Respire with input from 9 respiratory HCPs over 2 years and subsequently explored the output with 11 respiratory HCPs (9 of which were involved in the design process).

Our findings uncover the challenges faced when HCPs interact with health care data in context. From this, we reveal novel insights and lessons regarding the design of DSDM technologies to support the real-world complexities of clinical decision-making. Our paper makes three main contributions: (1) we provide insights into how DSDM technologies can support respiratory care by exploring HCPs’ data needs; (2) we uncover key barriers that impact HCPs’ engagement with data for decision-making; and (3) we provide novel and translatable [50] design implications that inform the creation of future DSDM technologies for health care.

This study was divided into 3 stages. The first stage explores HCPs’ data needs related to COPD care, with a view to understanding how DSDM technology could support these requirements. The second stage involves the selection of key data requirements identified from the first stage and developing them into digital data interaction scenarios (presented in Respire). The third stage explores Respire with HCPs to understand how each data scenario could support their decision-making regarding COPD care.

This was a co-design study involving HCPs from 2 collaborating National Health Service organizations in North West England. Co-design involves embedding users in the design process, which is appropriate, given our desire to explore HCPs’ needs and experiences at each stage in detail [54]. The first organization we worked with, the hospital, has a respiratory ward with patients with COPD under the care of specialists. The specialists also visit respiratory outpatients in clinics, including those recently discharged from the hospital after an exacerbation. The second organization, community care, provides services that enable patients with COPD to manage their condition in the community. Services include routine clinics to assess a patient’s condition and management, pulmonary rehabilitation classes that use exercise and education to improve self-management [55], and home support services in which on-call specialized nurses support acute patients in their homes [56]. Patients are referred to community care by their general practitioner (GP) if they require advanced support or by the hospital to help stabilize their management after a hospitalization.

We have previously worked with HCPs from both organizations to explore their challenges with lack of access to data and effective visualizations for COPD care [24]. In a study by Tendedez et al [24], we found that (1) HCPs used multiple clinical systems that were inflexible, (2) existing data lacked required detail and quality, and (3) HCPs rarely had time for extensive training on clinical systems and needed intuitive user interfaces. Following Tendedez et al [24], we worked with them to explore how DSDM technology could support their needs by creating Respire. Respire is a web app designed to effectively visualize COPD data that are routinely collected across both organizations (contained in digital systems or paper notes). Crucially, it also aimed to visualize data that are not yet available in clinical practice to envision how decision-making could be supported in the future. The current and future data needs were identified through an iterative process.

We had no access to the organization’s data during this study. Therefore, Respire used test data sets that were created by the research team and based on the data requirements elicited during the study. A researcher experienced in large-scale hospital data advised on the content and structure of the test data sets. For instance, they advised on (1) the typical data fields that hospitals collect for patients with COPD and (2) the range of data within those fields. For scenarios that displayed medical data, we used web-based resources regarding medical readings to inform clinically realistic test values [57]. The researcher then checked that the test data sets we had produced were realistic for the purposes of the activity. The test data were created solely to populate Respire for this study; they were not intended to be used beyond this purpose. We stored the data in a MySQL database, which was read by Respire via a custom REST (Representational State Transfer) application programming interface.

An outline of the study methodology is presented in Figure 1.

This study involved 11 HCPs across both organizations (6 from community care). Each participant (except for C6 and H5) was involved in the design process. Snowball sampling was used to recruit participants, with HCPs informing their colleagues about the project [58]. Details of the participants is presented in Table 1, showing their experience in their current role and using clinical information systems.

This qualitative study followed an interpretivist approach that emphasizes the social construction of individuals’ knowledge based on their lived experiences [59]. This approach was appropriate because of the exploratory nature of this study, which focused on HCPs’ experiences providing COPD care. We used 2 techniques for data analysis.

Stages 1 and 2 used content analysis to determine the presence and frequency of specific themes within discussions [60]. Content analysis was selected because of the volume and nature of the data collected. We carefully read the transcripts and assigned codes to references to specific types of data, reasons the data are needed, and comments about data visualization. Stage 3 used inductive thematic analysis to analyze the interview data. We used bottom-up open coding to assign codes to the data at the sentence level based on what the data described. We then grouped those codes to create broader themes that described the entire data set [61]. This analytical approach was chosen given our desire to be more exploratory in stage 3, focusing on capturing the nuances of interacting with Respire.

The first stage involved 5 workshops to explore the data needs for Respire. There were 2 separate workshops with the hospital (H1, H3, and H4) and community HCPs (C7, C8, C9, C10, and C11) each before uniting in the final workshop. Sessions were organized to suit HCPs’ availability and lasted between 30 and 90 minutes in quiet rooms at the clinical sites. Plans for workshops 1 and 2 are provided in Multimedia Appendix 1. We analyzed workshop transcripts after each session.

The first workshop aimed to understand the data needs and develop a shared language between HCPs and researchers. The discussions focused on their patients with COPD and services. We asked, “What data do you want to see about your COPD patients/service?” and “How would you want to interact with that?” In addition to verbal discussions about data needs, to stimulate discussion, HCPs created basic sketches of how data might be visualized. Basic sketching was used as a visual communication tool, enabling researchers to understand HCPs’ mental model of how the data might be tangibly presented. After the sessions, we used the sketches to create wireframes of basic user interfaces and to complement the data analysis. We provided the wireframes in later workshops as stimulus materials.

Subsequent workshops explored the data needs in detail using the wireframes to structure discussions. The HCPs revised the way the data were visualized on the wireframes and refined the included data. For instance, they supplemented the tables with graphs and removed data that they no longer saw as a priority on reflection. We updated the wireframes after the session.

The final workshop gave HCPs from both organizations the opportunity to discuss one another’s wireframes. Two business intelligence staff members, who had been involved in earlier stages of the project [24], attended to share knowledge about the general existence of the included data within both organizations.

We reviewed the data needs captured in stage 1 and chose 5 key use cases to expand into data interaction scenarios for Respire (Figures 2-10). We decided to focus on the needs common to both organizations, as these appeared to be most impactful. For example, we created a scenario based on viewing a patient’s spirometry test result history, as this was an unmet need voiced by both organizations. Spirometry tests require the patient to blow into a device used to diagnose and monitor respiratory conditions. Both organizations discussed these data as being a prominent pain point in practice; thus, it was an important scenario to explore further.

Following this, we confirmed with H4 and C7 if the 5 chosen scenarios (Table 2) were an important focus. We then created Respire, which is an interactive digitization of the 5 data scenarios. Respire is a web app populated with test data to support dynamic interaction. During development, we met with available HCPs (H2, H3, H4, C7, C8, and C9) for feedback on the early versions. For example, ensuring that the wording of headings and the test data made sense for each scenario. Table 2 details the 5 shortlisted scenarios, including a description of the scenario and where the data contained within the scenario would be reported from. A snapshot of each scenario on Respire is shown in Figures 2-10.

The final stage explored Respire with 11 HCPs (Table 1). We gathered feedback about their interaction with the scenarios, exploring how they might support their decision-making regarding COPD care. A total of 11 one-to-one sessions were held in quiet rooms at both clinical sites, lasting between 60 and 90 minutes. HCPs were tasked with walking through each scenario and imagining that they had access to it in practice. We asked HCPs to interact with each scenario freely by exploring different tabs, reviewing and interacting with the visualizations, and examining the (test) data. They were asked to think aloud [62]. The semistructured session plan is contained in Multimedia Appendix 2. During the sessions, we asked, “How could the data presented to you in this format influence your decision-making?” “Are there any challenges that you could envisage when using this scenario?” “Who do you think needs to be involved in the collection and maintenance of this data to ensure it is useful?” These questions would provide insight into how HCPs envisage digital tools, such as Respire, might be used in practice.

After reviewing each scenario, HCPs rated (on a 7-point Likert scale, with 1 representing strongly disagree and 7 representing strongly agree) how realistic they perceived the digital data scenario to be (“the scenario responds in a way that you would expect when using a system to complete this task”) and its relevance to their job (“this scenario is something you would use in your role”). The former was asked to understand if the scenarios were presented realistically to inform them how well they could engage with them and respond to questions. The latter was asked to understand whether there were scenarios that were more relevant to the responsibilities of some HCPs compared with others. Both these responses would add further framing to the discussion.

To conclude the sessions, HCPs ranked the 5 scenarios against each other in order of usefulness (with a score of 1 being the most useful). This would help to discuss the respective strengths and weaknesses of each scenario in context with another.

This study received ethics approval from both Lancaster University’s Faculty of Science and Technology Ethics Committee and the Health Research Authority (reference: 17/HRA/3092). All participants were required to read an information sheet and sign an informed consent form before participation. All sessions were audio-recorded, with full permission from the participants.

The data requirements shared by both the hospital and community care were identified from the stage 1 workshop and are summarized in Textbox 1. They largely focused on (1) understanding the severity of a patient’s condition and (2) managing the demands of both health care services. A full list of the data requirements captured from the stage 1 workshop is provided in Multimedia Appendix 3.

The following sections outline the findings from the stage 3 interviews, including Likert questionnaires, scenario ranking, and qualitative feedback.

This study explored how DSDM technology could support COPD care. We achieved this by designing a scenario-based research tool with HCPs to understand human-data interaction for decision-making. DSDM technologies have clear potential to connect HCPs with pertinent data to inform decisions. However, we have unearthed important challenges and lessons relevant to the success of DSDM technologies in practice, which are of particular relevance to the human factors research community: (1) data recorded by HCPs may not be trusted for decision-making, (2) transparency about data sources is required to trust and understand data, (3) sporadic and subjective data generated by patients have value but create challenges for decision-making, and (4) HCPs require support to interpret and respond to new data and its use cases.

Data were considered most trustworthy when the HCP who recorded it was perceived as an expert in assessing COPD. Previous work has shown that the source of medical information determines its adequacy for use in decision-making [68-70]. Specifically, Cicourel [68] observed how the perceived credibility of medical information was based on social and professional hierarchies. For example, they found that diagnostic information from attending physicians was rarely challenged and perceived as more objective than that of medical students [68]. In our study, data recorded by the respiratory ward staff at the hospital (perceived as highly specialized in COPD) were considered the most trustworthy, whereas data generated by GPs (perceived as less specialized in COPD) were considered the least trustworthy.

It was easier to assess if the data were trustworthy when it was produced by a familiar colleague, enabling the HCP to assess the colleague’s skills and competencies. Jirotka et al [71] described this as “biographical familiarity,” a predicate for trust. They observed how mammogram readers became familiar with the strengths and weaknesses of their colleagues, affecting how they read the mammograms produced by different centers [71]. Similarly, our study shows how a lack of biographical familiarity impacts HCPs’ engagement with data from staff with unfamiliar competencies. In contrast, the hospital and community care trusted each other’s data, as they were familiar with one another’s competencies.

Awareness of how trust impacts the use of data across departments and organizations is important and impacts how data should be displayed on DSDM technologies. Respire showed the source of spirometry test results and exacerbation reports, which HCPs felt were crucial contextual metadata to emphasize. One possible way to support building trust with unfamiliar data could be through seals of approval or digital badges built into dashboard designs [72]. For example, a badge representing skill proficiency could be displayed next to entries from organizations that have had specific training in spirometry.

Knowing which data sources were populating Respire was important for assessing any limitations when using data to make decisions. The explicit mention of a system’s data sources is also important for building trust [73]. This emphasizes the need to make data sources transparent to users, addressing the human-data interaction challenges of legibility [46]. Certain data sources were perceived as unreliable, such as data from the coding department and the hospital data flag system. The unreliability of coded data has been explored in previous work [74-76], particularly the overlap of codes for a single clinical condition [74]. As the specificity of medical data is tailored to the original purpose of its collection, repurposing it requires additional details for the data to be usable in new contexts [77,78].

We argue that transparency about the data sources that populate DSDM technologies will enable HCPs to assess important contextual factors about the data. This supports the use of data in new contexts. Transparency can be achieved by labeling the data sources on the user interface and visually representing their reliability. For example, data from the coding department could have icons alongside it, which indicate that the code is a working diagnosis or overlaps with other respiratory conditions.

Despite the benefits of viewing patient-generated data, self-assessed data may be too variable for decision-making. Previous work has shown that it is challenging for patients with COPD to answer subjective questions about breathlessness and coughing [79]. To address this, patients may underreport symptoms unless there are large deviations from their baseline [79]. Unreliable reporting of symptoms impacts how data are interpreted by HCPs, which means that patients may not receive the care they require. This concern relates to the human-data interaction challenge of legibility, making data transparent and comprehensible [46].

We found that there are contexts in which subjectivity in patient-generated data is acceptable, such as clinical discussions about perceived symptoms and quality of life. Scenario 3 presented symptom data in a structured format, enabling HCPs to quickly pinpoint moments in time. Patient-generated data, in turn, becomes a useful resource for HCPs and patients to collaboratively identify personalized management strategies and goals [80]. Therefore, although quantitative symptom readings can address variability in patient reports [81], complete quantification of a patient’s chronic health experience removes an important perspective. A combination of quantitative and subjective data can provide a holistic view of a patient’s condition to support the development of personalized goals. However, patients may require support to understand their data in preparation for clinic visits to maximize the value of the co-interpretation process [80].

To support patients’ understanding of their data, digital technologies for self-monitoring could prompt them to input written context alongside symptom changes; for example, if symptoms deviate from a baseline. The written context prepares patients to discuss key moments in the clinic visit. However, in contexts where HCPs receive data remotely, HCPs may feel concerned if they deem themselves liable to address the content of patients’ free-text notes outside of clinic visits [82]. To mitigate this, when viewing patient-generated data remotely, typed notes could be inaccessible until the HCP interacts with the patient directly. Future work is needed to explore how to connect context to symptoms without causing these concerns in HCPs.

A notable challenge with patient-generated data is the perceived effort required to encourage patients to record data consistently so that health patterns can be identified [83,84]. Thus, sporadic data entry can mean that important insights are missed or rendered ambiguous [85]. For instance, sporadic data can cause challenges where complete data are required to predict exacerbations [86]. Similarly, sporadic data may indicate that a patient is too unwell to monitor their symptoms [85,87,88]. However, we found that sporadic data could have value depending on the use case.

Patients’ symptomatic days were key information for HCPs, as this required some action from them. Therefore, HCPs suggested that recording data about being unwell would be a valuable insight, despite the absence of recording asymptomatic days. Patients who prefer to reduce their time thinking about their condition may also prefer to record data only when symptomatic [10]. HCPs suggested that Respire could have a filter that only displayed patients who were symptomatic and required support. However, when applying filters to data sets, HCPs may inadvertently pay less attention to patients outside the filtered subset [27]. Therefore, filters applied to the views of patient-generated data should have alerts regularly reminding the user of the applied filter.

It should be highlighted that enabling patients and HCPs to discuss health improvements is important [89,90]. Therefore, we do not argue that asymptomatic days should not be tracked as they can provide a measure, and a reminder, of how frequently patients feel well. Rather, we have found in contexts where there is no hard requirement to record data each day, a focus on symptomatic days alone can provide value. Future work should further explore such use cases to identify the key opportunities for sporadic data.

Concerns about responding to data can impact HCPs’ desire to integrate data into their workflow [88,91-93]. This relates to the human-data interaction challenge of agency, regarding acting on data and its implications [46]. Patient-generated data present a novel opportunity to support decision-making. However, HCPs were concerned that they would be expected to instantly investigate (lack of) data and the work involved in meeting this expectation. They felt that patients may stop acting on their health concerns as they expected HCPs to closely monitor them. In addition, HCPs had no guidance on the investigation and interpretation of patient-generated data. Aligning expectations about responding to data is important for HCPs to use data in practice [88,93]. Bardram and Frost [88] observed similar challenges raised by nurses that were responding to low mood reported by patients with bipolar disorder. This challenge highlights how wider sociotechnical considerations influence how HCPs engage with data.

Future studies should explore aligning expectations and establishing processes for responding to patient-generated data to alleviate concerns. This can be achieved by working with HCPs to understand the patient segments [94] who they wish to receive data from, the data required, and the frequency of its collection. Following this, we can collaboratively design appropriate workflows, dataflows, and digital interfaces. Our study found 4 use cases for patient-generated data: (1) supporting discussions in clinic visits, (2) monitoring acute patients to detect deterioration, (3) temporary monitoring of patients discharged from the hospital, and (4) temporary monitoring of newly diagnosed patients. Each use case may benefit from different processes, data, and visualizations. Balancing HCPs’ data needs with patients’ expectations in different contexts can support an understanding of how these systems can work in practice.

This study has 2 important methodological limitations. First, the exploration of Respire consisted of the same HCPs who informed its design (except for H5 and C6). Involving the same HCPs in the design could have introduced a positive bias into the feedback, with participants potentially responding more positively to Respire [95]. The second limitation is that this research was undertaken with 2 National Health Service organizations in North West England. Their local context and ways of working have shaped our findings, which require acknowledgment when transferring the findings to other health care contexts [50].

By exploring data interaction scenarios with HCPs, we unearthed lessons and design implications for DSDM technologies in the context of COPD care. Although DSDM technologies can support HCPs, there are important human-data interaction and sociotechnical challenges that influence their design and deployment. These challenges are related to (1) trusting data for clinical decision-making, (2) navigating unreliable and incomplete data sets, and (3) interpreting and responding to new types of data. Further investigation of these challenges will enhance the design and deployment of effective DSDM technologies for health care. Although COPD was our area of focus, we argue that our findings have the potential to translate [50] to other areas where DSDM technologies might be used in health care.