Using Patient Portals to Improve Patient Outcomes: Systematic Review

Background: With the advent of electronic health record (EHR) systems, there is increasing attention on the EHR system with regard to its use in facilitating patients to play active roles in their care via secure patient portals. However, there is no systematic review to comprehensively address patient portal interventions and patient outcomes. Objective: This study aimed to synthesize evidence with regard to the characteristics and psychobehavioral and clinical outcomes of patient portal interventions. Methods: In November 2018, we conducted searches in 3 electronic databases, including PubMed, EMBASE, and Cumulative Index to Nursing and Allied Health Literature, and a total of 24 articles met the eligibility criteria. Results: All but 3 studies were conducted in the United States. The types of study designs varied, and samples predominantly involved non-Hispanic white and highly educated patients with sizes ranging from 50 to 22,703. Most of the portal interventions used tailored alerts or educational resources tailored to the patient’s condition. Patient portal interventions lead to improvements in a wide range of psychobehavioral outcomes, such as health knowledge, self-efficacy, decision making, medication adherence, and preventive service use. Effects of patient portal interventions on clinical outcomes including blood pressure, glucose, cholesterol, and weight loss were mixed. Conclusions: Patient portal interventions were overall effective in improving a few psychological outcomes, medication adherence, and preventive service use. There was insufficient evidence to support the use of patient portals to improve clinical outcomes. Understanding the role of patient portals as an effective intervention strategy is an essential step to encourage patients to be actively engaged in their health care. (JMIR Hum Factors 2019;6(4):e15038) doi: 10.2196/15038


Introduction Background
Since the enactment of the Health Information Technology for Economic and Clinical Health Act in 2009, a part of the American Recovery and Reinvestment Act, adoption of electronic health record (EHR) systems by hospitals has steadily increased. According to the 2019 Brief by Office of the National Coordinator for Health Information Technology [1], nearly 86% (9/10) of hospitals in the United States now have at least a basic EHR system (eg, patient demographics, problem lists, medication lists, and discharge summaries) [1]. In addition to growth in EHR adoption overall, hospital adoption of technology with advanced functionality has increased significantly. For example, hospital adoption of comprehensive EHR systems-which include the aforementioned basic functions plus more expanded functions such as computerized provider order entry (eg, laboratory tests, radiology tests, medications, consultation requests, and nursing orders), laboratory and diagnostic test result management, and decision support (eg, drug-drug interactions, clinical reminders, or drug dosing support)-has increased from 1.6% in 2008 to more than a third (40%) of US hospitals in 2015 [2].
An examination of 9 hospitals in the United States with a comprehensive EHR system revealed that the EHR systems facilitated patient safety and quality improvement through the use of checklists, alerts, and predictive tools and electronic prescribing and test ordering that reduce errors and redundancy [3]. Similarly, faster communication and streamlined processes through EHR systems led to improved patient flow and quality of care in outpatient cardiology practices [4] and primary care [4,5], although some exceptions exist. For example, a recent analysis [6] using a large registry of hospitalized patients with heart failure (N=21,222) failed to substantiate any association between EHR use and a set of outcomes including quality of care and 30-day postdischarge death or readmission. Similarly, a longitudinal observational study [7] involving 4 primary care clinics of 2242 patients with diabetes examined EHR messages sent among team members to pass patient care information and found that more frequent EHR message forwarding in primary care teams was associated with worse patient outcomes and higher medical costs.
Although the existing literature has much emphasis on clinician and system use of EHR, increasingly closer attention is being paid to the EHR system in terms of its use in facilitating patients to play active roles in their care via a portal-a secure Web-based site tied to an EHR that gives patients access to their health records, appointment scheduling, refill requests, or secure messaging with the health care team. For example, a recent state of the science review [8] examined patient experiences with portals. The review found that patients' interest and ability to use the patient portals was influenced by personal factors, such as age, ethnicity, education level, health literacy, health status, and role as a caregiver, and that provider endorsement was one of the most influential factors impacting patients' adoption of the patient portal [8]. In a realist review, Otte-Trojel et al [9] noted patient insight into personal health information, activation of information, interpersonal continuity of care, and service convenience as mechanisms of patient outcome improvements in 32 studies of patient portals published since 2003. A total of 2 systematic reviews [10,11] examined the effect of patient portals on clinical care and patient outcomes. Specifically, Ammenwerth et al [10] reviewed 4 controlled trials published between 1990 and 2011 and found quicker decrease in office visit rates and better adherence to treatment in the patient portal group, compared with a control group. They found no significant changes in health outcomes. Goldzweig et al [11] reviewed 46 studies of various designs (eg, randomized, nonrandomized, and qualitative studies) published between 1990 and 2013. They found that evidence was mixed about the effect of portals on health care utilization (eg, emergency room visits and hospitalizations); portal use was associated with improved outcomes for patients with chronic diseases such as diabetes, hypertension, and depression when used in conjunction with case management [11].

Objective
The field is rapidly evolving; however, none of the previous systematic reviews have comprehensively addressed the goals, types, and scope of the patient portal interventions and how these interventions are linked to patient outcomes. Given the rapid adoption of comprehensive EHR systems involving patient portals, a comprehensive systematic review on patient portal interventions is warranted. This study aimed to critically appraise evidence on the effects of patient portal interventions on clinical and psychobehavioral outcomes of patients. We examined the detailed characteristics of patient portal interventions and relevant patient outcomes. Our review systematically extends previous efforts by providing an understanding of (1) what constitutes patient portal interventions (scope and nature) and (2) how patient portal interventions achieve desired effects.

Review Design and Study Eligibility
We conducted a systematic review of research evidence designed to assess patient portal interventions. Studies were screened to assess their relevance to the purposes of our systematic review. Articles were included in this review if the study was (1) about patient portals, (2) published in the English language, and (3) included patient outcomes (either behavioral or clinical in nature). Studies were excluded if full texts were not available (eg, conference abstracts) because of its limited information addressing patient portal interventions and associated outcomes. Studies with no measured outcomes and quantitative designs were also excluded.

Search and Selection of Studies
The search was conducted in November 2018. Following consultation with a health science librarian, 3 databases-PubMed, EMBASE, and Cumulative Index to Nursing and Allied Health Literature-were searched. Search terms included the following: "Electronic Health Records" OR "Medical Records" AND electronic* OR computer* OR "electronic medical record" OR "electronic medical records" OR "electronic health record" OR "electronic patient records" OR "electronic patient record" OR "electronic health records" OR "EMR" OR "EPR" OR "EHR" OR "patient portal" AND "Patient Participation" OR "patient involvement" OR "patient engagement" OR "patient empowerment." A full search strategy with specific terms for each database can be found in Multimedia Appendix 1.
There were 2742 references that were retrieved from the electronic searches and imported into Covidence software. Of these, 744 duplicates were removed, and 1998 studies were selected for title and abstract screening. A total of 2 reviewers independently conducted an initial screening of titles and abstracts for relevance. In total, 1782 articles were excluded because they were irrelevant. A total of 2 reviewers independently evaluated 216 full-text articles to determine eligibility. Following this, 192 articles were excluded for the following reasons: wrong study design (n=88), not a research study (n=63), wrong intervention (n=23), wrong outcomes (n=16), and abstract only (n=2). All references were screened by 2 independent reviewers. Disagreements were resolved through consensus. A total of 24 articles met the inclusion criteria. Figure 1 provides details of the selection process.

Data Extraction
Relevant data were extracted by 2 authors using a standardized data extraction form developed by the authors. The following data were extracted from the included studies: first author, publication year, country, study design, study outcomes, measurement, setting, sample sizes, sample demographics, attrition rates, main findings, and patient portal intervention characteristics, including main goal of intervention, type, modality, dose and scope, and patient engagement metrics. An independent research assistant reviewed extracted data to check accuracy. Any discrepancies were resolved through discussions among all research assistants and authors.

Quality Appraisal
The selected studies were evaluated for quality, based on published quality rating scales to identify strengths and weaknesses in study methodologies and guide the interpretation and assessment of study findings. Specifically, 2 authors rated each study for its quality independently using the Joanna Briggs Institute quality appraisal tool [12]. Each research study's methodological characteristics were evaluated using the corresponding tool according to study design. A mixed method study [13] was assessed by using both cross-sectional and qualitative checklists. Studies were rated a 0 if they did not identify or include a component of the quality rating and a 1 if they did. Then, the total individual scores (numerator) were added up and divided by the total possible score (denominator) for the respective scale. Studies were rated high, medium, or low quality if they successfully addressed >66.6%, 33.4% to 66.6%, or <33.4% of the components, respectively. Studies were not excluded based on the quality appraisal. Interrater agreement statistics using percent agreement ranged from 66% to 100% (average 88%). Any discrepancies were resolved through team discussions. Tables 1 to 4 show consensual scores of quality assessment. Half of the studies included in this systematic review were of high quality [14][15][16][17][18][19][20][21][22][23]. Of the 10 randomized controlled trials (RCTs), 9 were of medium quality [24][25][26][27][28][29][30][31][32], and 1 was of high quality [33]. Common methodological issues observed in the RCTs had to do with a lack of concealment of allocation to treatment groups, such as nonblinding of participants to treatment assignment [28], nonblinding of those delivering treatment, [24,30,31] or nonblinding of outcome assessors to treatment [24][25][26][27][29][30][31]. Among the quasi-experimental studies, 6 out of 7 [14,16,18,19,21,22] were of high quality, and 1 was of low quality [34]. The low-quality study did not have a control group, did not report if the participants included from the 3 different sites were similar at baseline, did not describe and analyze the incomplete follow-up, and did not report the reliability of the outcome measures. In addition, this study did not have multiple measurements of the outcome both pre-and postexposure to intervention. Of the 6 cohort studies, 4 [15,17,20,23] were of high quality, whereas the remaining 2 [35,36] were of medium quality. These specific studies were of lower rating because of not identifying potential confounding variables or strategies to deal with the confounding variables. The mixed method study [13] was of high quality for its quantitative and cross-sectional methods and of low quality for its qualitative component. Table 1. Study quality ratings for randomized controlled trials.
Were participants analyzed in the groups to which they were randomized?  Table 2. Study quality ratings for quasi-experimental study.
Were the participants included in any comparisons receiving similar treatment/care, other than the exposure or intervention of interest?
Was there a control group?
Were there multiple measurements of the outcome both pre-and postintervention/exposure?
Was follow-up complete and, if not, were differences between groups in terms of their follow-up adequately described and analyzed?
Were the outcomes of participants included in any comparisons measured in the same way?
Were outcomes measured in a reliable way?
Was appropriate statistical analysis used? Table 3. Study quality ratings for cohort study.
Were the exposures measured similarly to assign people to both exposed and unexposed groups?  1 Do the conclusions drawn in the research report flow from the analysis, or interpretation, of the data?

Psychological and Behavioral Outcomes
Effects of patient portal interventions were tested in relation to a variety of psychological (eg, health knowledge, decision making, patient activation, and self-efficacy) and behavioral (eg, adherence and preventive service use) outcomes. Specifically, patient portal interventions were associated with a significant increase in patient knowledge of a health condition or topic in 4 studies [18,25,29,31]. Each of the 4 studies used patient report and a nonstandardized instrument to assess patient knowledge. Similarly, in a pilot RCT [31], patients in the intervention group reported significantly lower conflict in making decisions (measured by the Decisional Conflict Scale) and significantly higher preparation for making decisions (measured by the Preparation for Decision Making Scale). In contrast, 3 quasi-experimental studies reported no significant difference in patient activation [21,22] or patient-reported achievement of behavioral goals (eg, taking medications, healthy eating, being active, monitoring, taking medications, problem-solving, reducing risks, and healthy coping) [16] across the intervention and control group. One of the quasi-experimental studies that did not find a significant difference in patient activation [22] did find that participants in the intervention group were more likely to talk to their health providers about the health topic covered in the intervention. Finally, a quasi-experimental study [14] investigating the impact of the portal's secure messaging feature reported significantly higher self-efficacy (measured using the Diabetes Management Self-Efficacy Scale) and reports of a collaborative relationship (measured by a self-developed questionnaire) at 26 weeks.
The effects of patient portal interventions on behavioral outcomes were consistently positive. In a cohort study comparing portal users with non-portal users [20], portal users had significantly higher medical refill adherence. Similarly, a quasi-experimental study [34] investigating the impact of the OpenNotes feature of the patient portal reported proportionately higher medication adherence measured by patient report and analyzed with summary statistics. A retrospective cohort study [23] and an RCT [28] found that patient portal users were significantly more likely to engage in preventive health care including breast and colorectal cancer screening and Pap smear tests.

Clinical Outcomes
A total of 10 studies included in the review reported on clinical outcomes encompassing BP control [17,19,21,32,33], glycemic control [13,16,21,24,26,32,33], cholesterol control [16,21,30,32,33], and weight loss [30,32,33]. In a retrospective cohort study [17] comparing patient portal users with non-patient portal users, portal adoption was only associated with improved BP control in unadjusted models. A quasi-experimental study [19] found that the patient portal intervention was significantly associated with achieving BP control, compared with the control group. The intervention also included a remote, home-based telemonitoring program in addition to the patient portal [19]. An RCT that focused on a tailored patient portal for patients with uncontrolled diabetes and included BP control as a secondary outcome [33] found no significant differences between the intervention and control groups in BP control. Similarly, a quasi-experimental study [21] and a cluster randomized trial [32] found no significant difference in BP control between the intervention and control groups.
Glycemic control, as measured by hemoglobin A 1c (HbA 1c ), significantly improved at 6 months, compared with baseline, but the change at 12 months was nonsignificant in patient portal users compared with no patient portal users in both an RCT [33] and a quasi-experimental study [21]. A quasi-experimental study [16], an RCT [24], and a cluster randomized trial [32] also found no difference in glycemic control between the intervention and control groups. A mixed method study with no comparison group found that patient portal use was significantly associated with lower HbA 1c values [13]. In addition, an RCT [26] investigating patient portal use found that only the participants randomized to the patient portal who sustained regular use reported significantly lower diabetes distress (measured by the Problem Areas in Diabetes scale), which, in turn, was significantly linked to lower HbA 1c .
Effects of additional clinical outcomes including cholesterol and weight control were also mixed. For example, cholesterol control, measured by a low-density lipoprotein (LDL) level, was significantly improved in the intervention group of an RCT [33] but was not significantly improved in the intervention group of 2 quasi-experimental studies [16,21], an RCT [30], or a cluster randomized trial [32]. Finally, an RCT [30] and a cluster randomized trial [32] both reported that participants who received the patient portal intervention experienced significant weight loss. In contrast, an RCT [33] investigating a patient portal intervention tailored to patients with uncontrolled type 2 diabetes reported no significant difference in weight loss among the intervention group.

Principal Findings
To the best of our knowledge, this is the first systematic review that provides a critical appraisal of patient portal interventions with relevant patient outcomes. Although the patient portal interventions varied in their scope, methodology, and outcomes, evidence generally supported the use of patient portal interventions in improving health knowledge [18,25,29,31] and other psychological outcomes, such as decision making [31] and self-efficacy [14], and behavioral outcomes, such as medication adherence [20,34] and cancer screening [23,28]. Patient portal intervention was not effective in improving patient activation [21,22] or behavioral goal achievement [16]. Of particular note, the positive effects of patient portal interventions on medication adherence and cancer screening were consistent across the studies, regardless of the study design, including cohort study [20,34], quasi-experimental study [34], and RCT [28]. These findings suggest patient portal as a promising strategy to improve certain psychological outcomes and health behaviors via simple interventions such as individually tailored messages [28], registration of patients in the Web-based refill services [20,34], or open notes between the patient and the provider [34]. Nevertheless, these studies [20,28,31,34] included predominantly white, middle-aged, and English-speaking populations in their study samples. In addition, the studies reporting positive behavioral outcomes involved a very large sample size (>2000) for which even a small difference (eg, between-group difference of 2.4% in the proportion of patients up-to-date with cancer screening) [28] would result in a statistical significance. Future research is warranted to include patients with more diverse backgrounds (eg, racial/ethnic minorities, older patients, and individuals with limited English proficiency) and of adequate statistical power for testing of applicability and efficacy of patient portal interventions.
Patient portal interventions, overall, had little effects on clinical outcomes addressed in the studies included in the review. For example, of 5 studies in which BP was included as an outcome, only 1 [19] found improved BP control, whereas the other studies did not [21,33] or failed to identify any significant effect in adjusted models [17]. Similarly, less than half of the 7 studies [13,16,21,24,26,32,33] including glucose control as an outcome had a significant finding but either in a noncontrolled setting with no comparison group [13] or only for a short term (6 months) [21,33]. Effects of cholesterol control were also, overall, insignificant, as only 1 [33] of 5 studies had significant reduction in LDL. The overall lack of significant improvements in the clinical outcomes might be attributable to a number of methodological issues such as short-term follow-up or insufficient power to detect changes in outcomes [13,24,26,30]. More important, patient engagement with the portal interventions was not evaluated at all in more than one-third of the studies included in the review [15,17,18,20,23,26,28,31] nor was it systematically incorporated in the design and analysis of the portal interventions. As some studies, where discussed, generally indicated positive changes in patient behaviors or clinical outcomes for individuals with sustained engagement with the portal [21,36], future patient portal interventions should be expanded in scope to focus more on strategies to promote active engagement of patients with the portal.
There are methodological issues to be taken into consideration when interpreting the findings in this review. Although attrition ranged from 0% [18,31] to 71% [34], attrition greater than 20% was observed in more than one-third of the studies using a longitudinal study design [14,24,25,28,29,32,34,36]; another one-third did not report the number and/or reasons for participant withdrawals or dropouts [17,19,20,23,26,35]. Furthermore, 7 studies [14,16,18,19,21,34] used a quasi-experimental study design and, hence, were subject to threats to internal validity. A lack of concealment was also a common methodological issue noted in more than half of the RCTs [24][25][26][27][28][29][30][31]. Nonblinding of those delivering treatment or outcome assessors is likely to have led to the disclosure of group allocation or response bias, hence, threatening the internal validity of the results. Future studies should address these issues by concealing group assignments and separating data collection from intervention delivery. In addition, for reasons not explained in the studies examined, the studies conducted in the United States also lacked complete racial/ethnic diversity by including predominantly white, highly educated, and highly literate in the study samples [13,[15][16][17][21][22][23][24][25][26][27][28][31][32][33]35,36], and in some cases, such data were not reported [19,20,23,29,34]. The failure to include participants with diverse backgrounds in the sample of studies conducted in the United States limits the generalizability of the study findings. It is furthermore notable that patient portal intervention modalities included in this review involved a form of text messaging activities most often designed for those with high computer literacy skills [32]. Future studies need to include more diverse populations in the study sample such as nonwhites and individuals with limited English proficiency to account for the rapid increase of the populations and those with limited computer literacy. In addition, Future research needs to expand the nature and scope of the modalities in patient portal interventions beyond simple digital text messaging by using a more interactive way of engaging patients, such as using voice and video modalities.

Limitations
A number of limitations of this review should be noted. First, it is possible that we did not find all relevant articles in the literature. To avoid this, we conducted an extensive systematic electronic search using a compressive list of Medical Subject Heading terms, after consultation with an experienced health science librarian, in addition to hand searches of references of the identified studies. In addition, we did not include gray literature such as reports from organizations; hence, publication bias may exist. We included only articles written in English; therefore, the findings cannot be generalized to studies published in non-English languages. Finally, the studies included in the review used predominantly non-Hispanic white, highly educated, and highly literate individuals, limiting the generalizability of study results. Therefore, the findings from this review should be interpreted with caution.

Conclusions
Our review of 24 articles of various study designs shows that patient portal interventions can promote positive psychological outcomes for adults in outpatient [14] or primary care [25,29,31] or those in surgery department [18]; increase medication adherence among patients with HIV [20] or those in primary care [34]; and increase cancer screening among those in outpatient or primary care [28]. We were unable to find sufficient evidence to support patient portal interventions as an effective approach for improving clinical outcomes, as some of the included studies reported positive improvements in BP control [17,19], short-term glycemic control [13,21,33], cholesterol control [33], and weight loss [30], whereas others did not [16,21,24,30,32,33]. Although several methodological biases and weaknesses were noted in reference to the patient portal interventions included in this review, our findings suggest the need for more rigorous and continued evaluations of this approach for a broader range of outcomes and populations.

Introduction
Despite there being no legal restriction to abortion care in Canada, women who seek or have an abortion continue to experience stigma across the country. This has the potential to leave them feeling isolated and unsupported, and potentially prevents them from seeking follow-up care if needed. Therefore, innovative approaches for using information and communication technologies to achieve enhanced health service delivery, broadly known as digital health [1], is a way to address these issues. Digital health interventions in the form of hotlines, text messaging, and mobile applications have been shown to be safe, effective, and acceptable to women and providers for delivery of various aspects of abortion care [2][3][4][5][6][7]. Ensuring that an innovation is acceptable to the end-user and incorporating their voice throughout the research process is essential. Human-centered design is a methodology that implements an iterative participatory process by applying the needs of the end-users to the development of a given technology solution [8,9]. This methodology has been widely used for the design of innovations that generally affect behavior change.
Digital technology is changing the way we collect information and share and consume data. There is a growing momentum in the provision of resources for family planning, but specifically towards safe abortion care, in terms of the use of digital health interventions to address service delivery and legal barriers in various contexts. The importance of incorporating the end-user perspective's voice into the design and development of these interventions is crucial, as it has been noted that there are few mobile interventions that are truly effective and scalable [4,9]. Utilizing user-centered design for the development of a mobile tool that women can use to self-manage their care after a surgical abortion will lead to a higher likelihood that it will be acceptable and feasible to use and implemented to scale.
This study is Phase II of III. The findings from Phase I, which are published separately, were essential to Phase II [10]. The main objectives for this study included: (1) the development of a Web-based intervention based on the preferences and experiences of women who underwent an abortion as measured in the formative phase of the Feasibility and Acceptability of a Mobile Technology Intervention to Support Postabortion Care (FACTS) Study; (2) usability testing of the intervention to determine user-friendliness and appropriateness of the intervention; and (3) finalizing a beta-version of the Web-based intervention for pilot testing. Phase III of this three-phase study will determine acceptability and feasibility of the tool in a pilot prospective mixed-methods study. This study is the first in Canada to utilize user-centered design to develop a mobile intervention to support follow-up care after a surgical abortion.

Overview
The methods presented below are specific to the design, development, and usability testing of the intervention. A systematic visual depiction of each phase is provided in Figure  1.

Development and Design
Employing user-centered design, a systematic process was used to develop a mobile intervention based on the results from Phase I. This was conducted from September 2017 to January 2018. User-centered design is a methodology with "roots in a participatory process" and: provides a framework to understand and apply the needs of end-users to mHealth project development through a highly iterative process [9] We collaborated with University of British Columbia family planning experts to synthesize evidence and create a storyboard. A storyboard process is used to build a short narrative to visually plot elements of a prototype [8]. This process is allocated 60 minutes per session and uses a series of comic book style frames for drawing and highlighting the narrative of the mobile intervention [8]. The number of sessions is dependent on the complexity of the intervention. This is a useful step in the ideation process of design and development, and design companies use this process as an important early stage of user-centered design for health innovations.
We engaged various stakeholders, using implementation science principles, with the intent to assess the context that the intervention would be potentially applied to. This included stakeholders such as end-users, the hospital administration, funders, health care providers, family planning experts, and donors. In addition, we further engaged key rural stakeholders from Northern Health Prince George Hospital. Specifically, a focus group session was held with four providers from Prince George Hospital. The presentation of results from Phase I and a storyboarding session with the providers highlighted the facilitators and barriers to the provision of safe abortion care in rural British Columbia, particularly regarding follow-up support. These results were used to further refine content and design of the intervention.
By October 2017, an initial prototype for a comprehensive Web-based solution was developed with the support of a development partner and the findings from Phase I [10]. A scope document included the required key features for design and content.

Theoretical Framework
Like Phase I, the study design for Phase II was informed by the Technology Acceptance Model and the Theory of Reasoned Action [10][11][12]. Both these theories assess the perceived ease of use and usefulness of a system and individual's conduct based on their lived experiences, attitudes, and intention to engage in a behavior. As highlighted in Phase I, the study instruments for all three phases were developed using these theories based on validated survey tools [10].

Usability Testing
Recruitment of usability testing participants initially included contacting women from a database created at one of the abortion clinics in Vancouver that had a list of those who had consented to participating in future research. Due to limited response rates, we proceeded to utilize a social media recruitment strategy through two provincial and national reproductive and sexual health advocacy organizations, Action for Sexual Health Canada and Options for Sexual Health. This included Twitter and Facebook notifications. Eligible participants contacted the research coordinator and received a link to the website, a password and username, and a link to a survey. We did not collect demographic data.
A validated questionnaire adapted from the 2010 Post Study System Usability Questionnaire (PSSUQ) was used to assess participants' qualitative and quantitative feedback on usefulness, ease of use, privacy and security, content, visual layout, and general concerns [13]. Participants were recruited from a database of women who had consented to be contacted for future research at the CARE Clinic at British Columbia (BC) Women's Hospital. Participants were also recruited through social media advertising by national reproductive and sexual health organizations that used their respective Twitter accounts to share the link to the study website. Participants who were locally recruited conducted usability testing at BC Women's Hospital with researchers present. For those recruited through social media, participants received a link to the survey by email and details about how to access the website. A team of key stakeholders made up of obstetrics and gynecology specialists, family doctors, counsellors, nurses, and administrators provided feedback about the initial prototype of the intervention. Participants provided feedback on usefulness, ease of use, privacy and security, content, visual layout, and general concerns.

Data Analysis
During Phase 2, we performed descriptive data analysis. Results of the PSSUQ survey were reported in percentage (%). An official score was not calculated as the survey was adapted from the PSSUQ but was not used in its entirety. The adapted survey can be found in Multimedia Appendix 1.
This study was approved by the Children's and Women's Research Ethics Board (H16-02823).

Summary
Phase II participants for the storyboarding process included key stakeholders from Vancouver and Prince George Hospital. The health care providers (HCPs) had a median of 12 years (range: 1-20 years) of experience in family planning. These HCPs included: physicians, counsellors, nurses, and administrators. The development company selected for the study was a local software development group. We conducted 5 storyboarding sessions, which included the following: (1) family planning specialists in Vancouver; (2) rural providers in Prince George Hospital that included one family doctor and three specialist obstetrician/gynecologists; (3) five counsellors from an urban clinic in Vancouver; (4) a participant who had previously had an abortion and volunteered to participate; (5) a session with the investigators of this study; and (6) three senior administrative staff involved with one urban abortion clinic. Each session lasted between 60 to 90 minutes.

Key Stakeholders Engagement
Based on our stakeholder analysis we developed a communication strategy for engaging them, including the development of a facts sheet about the study, a website explaining the study, and standard presentations. The first step was to meet with each stakeholder and provide an orientation to the concept of a postabortion support tool using mobile technology. This was also an opportunity to further discuss their level of involvement for development and implementation of the intervention. Ongoing updates were provided with in-person meetings, telephone calls, and email bulletins. Table 1 highlights the key stakeholder groups, their respective area of influence or interest, the project phase, the engagement method, and the frequency with which they engaged with the development of myPostCare.ca.

Storyboarding
The formative research findings from Phase I were used to inform the creation of storyboards in collaboration with the research team and family planning experts based at the University of British Columbia. Two storyboards were created: (1) design; and (2) content for the mobile intervention. These storyboards took into consideration information based on the preferences that were elicited from the findings in Phase I [10]. The storyboard was reviewed in an iterative manner by the family planning experts and research team. It was also shared with members of the administration and allied health care providers at the three abortion clinics where recruitment for Phase I was conducted.

Development
Once the storyboard was completed, this was shared with a design and development company in Vancouver, British Columbia that is an expert in Web-based technologies for social marketing and behavior change and has experience working with the Ministry of Health in British Columbia. A step-by-step process was executed between the developer and the research team to build the prototype for the Web-based tool, which was a website that was accompanied by an email system. This came to be called myPostCare.ca. The steps of the process are highlighted in Table 2. The components of myPostCare.ca are highlighted in Table 3.

Usability Testing
As stated in the methods, user testing occurred both in person and remotely. Participants were given access to the website and after reviewing it completed an adapted version of the PSSUQ 2010 and provided qualitative feedback. There were 7 remote participants and 2 in-person participants.
The survey results adapted from the PSSUQ 2010 are available in Multimedia Appendix 2. Participants were satisfied with the usability of myPostCare.ca. Specifically, 62.36% "Strongly Agreed" and 28.69% "Somewhat Agreed" with the overall usability of the website. The PSSUQ reflects the overall usability of a website or app based on the respondent's experience. It has 3 subscores derived from subsets of 16 questions. Overall usability defined by the PSSUQ reflects system usefulness, information quality, and interface quality. Table 4 highlights the comments that participants shared and that were noted in the revisions of the prototype to prepare myPostCare.ca for the Phase III pilot study. Like Phase I, participants were accustomed to using some form of technology and were supportive of a Web-based tool to support follow-up care after an abortion. This was elicited from the key findings from potential users who completed the usability testing. The email notification system was developed in collaboration with family planning experts, physicians, and counsellors. Their expertise was used to specify what type of messaging would be appropriate at which time interval. This was complimented with results from Phase I of timing and content of email or text messaging. Two email streams were developed: one for participants who had an intrauterine device (IUD) inserted and another for those who did not have an IUD inserted. The emails were sent starting on the day of procedure (day 0) followed by every other day for one week and then weekly until day 28. The first week was focused on post-procedure signs and symptoms, and the next three weeks alternated between contraception counselling, emotional support, and overall sexual health information. The design of the email notifications was aligned with the design of myPostCare.ca. The content was developed by the primary investigator and reviewed by counsellors at the abortion clinics. The messaging was repurposed based on the social marketing expertise of our developer.

Primary Findings
myPostCare.ca is the first comprehensive Web-based postabortion tool in Canada and has the potential to be integrated as part of family planning services. It includes four interactive tools (Emotional Support Tool, Contraceptive Explorer, Postprocedure Care, and Sexual Health) that integrate automatic email notifications to provide support over the course of one month after the procedure. Integration of myPostCare.ca into clinical practice provides an opportunity to consider a new approach to supplement follow-up care specifically for abortions, but also women's health in general. We utilized user-centered design methodology, an iterative development process that was informed by input from key stakeholders such as patients, family planning experts, and administrators who are involved with abortion care [27][28][29]. This was crucial in developing a tool that responded to findings from Phase I [10].
Specifically, this phase demonstrated the importance of including the end users and key stakeholders in the design, development, and testing of a mobile intervention that services a population and deals with a health care issue that continues to be stigmatized. The formative research indicated essential information regarding women's interactions with technology, their needs and desires around follow-up and access to information, and their feedback on design, which was essential in the success of myPostCare.ca. An iterative design process was important to ensure that the research team was continually evaluating that myPostCare.ca realized the needs of the target users. Similar studies have successfully demonstrated that using this approach leads to a higher likelihood of implementation and scalability [3,[27][28][29].
We adopted a few theoretical frameworks, all of which use a comprehensive participatory approach to developing eHealth technologies. This was similarly done by Gilbert et al in the development of Get Checked Online, which is a Web-based sexually transmitted infection testing resource [29]. More specifically, integrating the Technology Acceptance Model and Theory of Reasoned action with the user-centered design methodology let us use a holistic approach to develop myPostCare.ca. According to the Technology Acceptance Model, perceived ease of use and perceived usefulness of a system are the two predominant indicators of system adoption [11,30]. Participants in our study were accustomed to using some form of technology, either mobile phones or computers, did not require acquisition of new skills, and were keen on the development of a technology-based tool to support follow-up care after an abortion. Importantly, myPostCare.ca will not eliminate structural barriers to comprehensive abortion care, and though it may not directly affect health behavior and decision-making, it may assist with making the delivery of abortion care more efficient, convenient, patient-centered, and accessible.
The limitations for this study include overall generalizability to other populations, small sample size for usability testing, loss to follow up, and recruitment bias. As it pertains to recruitment bias, those who consented to participate were likely individuals who were more engaged with technology, of a higher socioeconomic demographic, and were more likely to be early adopters of a digital health intervention to support abortion care. Though demographic data was not specifically collected for Phase II, this is based on the demographic data collected in Phase I [10]. In previous studies this has been noted as a digital divide, which suggests that though many developers of technology-based health interventions are optimistic about their impact, this needs to be balanced by the fact that the pattern of adoption is along social gradients [29]. New technologies like myPostCare.ca may further reinforce these social divides. Furthermore, abortion continues to be a stigmatized issue, which can be limiting for research since it can be a sensitive topic for most. In our study, it posed difficulties with recruitment and loss to follow-up. We assumed that lack of participant engagement may be associated with stigma about abortion, so we had to reevaluate our usability testing strategy regarding using social media platforms, which proved to be more successful as more participants were willing to engage anonymously at a distance. This recruitment strategy for abortion-specific studies is promising, particularly when thinking about diversifying the participants recruited and obtaining robust response rates for analysis.
Balancing these limitations are the strengths of this study, including: successful development of user-centered design elements, wide stakeholder engagement, diverse expertise on the research team, rigorous research methodologies, iterative design process, and development of the first Web-based postabortion tool in Canada, with the potential to expand it to other aspects of women's health (eg, miscarriage, gynecologic cancer care, sexual pleasure, and well-being).
Further research to evaluate acceptability and feasibility of myPostCare.ca and overall patient experience will be assessed in a prospective pilot mixed-methods study, which is Phase III of this three-phase study. In addition, as suggested in other Web-based literature [29], a health equity impact assessment with expert consultation and literature review may also help identify ways in which myPostCare.ca reinforces or alleviates health inequities in sexual health services.

Implications
By using user-centered design and rigorous key stakeholder engagement, there is potential for digital solutions for women's health to be implemented at scale. This study demonstrated that, by engaging end-users throughout the design of an intervention targeted to them, this provides insights and nuances that have implications for usability, acceptability, and feasibility to integration as a part of clinical care.

Conflicts of Interest
None declared.

Background
Although continuous patient monitoring is often thought to be reserved for intensive care units, the need for frequent assessment of vital signs is necessary in other clinical circumstances as well. Examples of these circumstances include during and after transfusion of blood products, and during and after transfusion of certain chemotherapies and immunotherapies [1,2]. Receiving these transfusions and infusions can result in untoward reactions that typically manifest as abnormal vital signs before or simultaneously with an adverse event (AE), also known as an infusion reaction [3][4][5]. For example, 2.1% of all recipients of blood products experience a transfusion reaction, some of which can be life threatening. In addition, administration of many anticancer drugs has a risk for infusion reactions [6]. Therefore, frequent vital sign monitoring during and after treatment is essential to prevent poor patient outcomes.
Even when intensive vital sign monitoring occurs, infusion reactions can go unrecognized. One reason may be inconsistent vital sign assessment [5,[7][8][9][10]. In one study, researchers found 27.4% (168/614) of nursing staff estimated respiration rate rather than measure it [11]. In many cases, physicians sent patients home after transfusion and infusion therapy with instructions to contact the clinic if the patient became unwell [6]. This leaves the possibility of late detection of infusion reactions open. These findings demonstrate that strategies for patient assessment during and following transfusion and infusion are suboptimal.
To improve the detection of infusion reactions, various interventions have been explored, such as additional training on identifying possible infusion-associated AEs for nurses [2]. Increasing nurses' knowledge of risk factors for infusion-associated AEs [12] has been implemented, in addition to guidelines to intensify monitoring of infusion recipients at higher risk. Process changes, such as standardized handoff forms [1] and clinical decision support systems [13], are other strategies used to improve safety of transfusions and infusions. Nonetheless, the challenge of adequate assessment during transfusion and infusions remains.
The evolution of digital health and biosensors has opened the possibility of an easier and more effective way of monitoring and analyzing vital signs in patients during and after the receipt of transfusions and infusions . Using these technologies, a health care professional may have greater insight into a patient's health status. One such product is the accelerateIQ and VitalPatch system. AccelerateIQ is an end-to-end clinical-grade system that collects data from wearable sensors (in this case VitalPatch). AccelerateIQ applies Food and Drug Administration-cleared artificial intelligence analytics to display indicators and data streams through a Web portal. Clinicians can then see any physiological changes indicative of clinical deterioration in patients outside of the acute setting (see Figure 1).

Objectives
The aim of this study was to evaluate whether the accelerateIQ and VitalPatch system is a practical way to monitor transfusion patients and whether patients find it acceptable to do so. We performed a pilot study to determine whether it was feasible to implement this system in the clinic and to support future studies, with a focus on wearability, usability of data, and safety. These pilot data will be used to inform a further definitive trial to optimize recruitment, treatment compliance, and follow-up protocols.

Study Design
This study was a three-arm, parallel, single center, observational, nonrandomized, open-label feasibility study. The study aimed to explore wearability, usability, and safety of use of the accelerateIQ and VitalPatch system in a population of transfusion and infusion patients.

Intervention
Participants in the study received standard care in addition to vital sign monitoring through the accelerateIQ and VitalPatch system. This system consists of the VitalPatch, a disposable adhesive patch biosensor that incorporates 2 surface electrodes with hydrogel and a thermistor on the bottom of the patch. A 4-day battery and an electronic module with an embedded processor, a microelectromechanical system tri-axial accelerometer, and a Bluetooth low-energy transceiver are also part of the sensor. The patch's sensors facilitate continuous, near real-time monitoring of heart rate, R-R interval, heart rate variability, respiratory rate, single-lead electrocardiography, skin temperature, body posture, fall detection, and activity. Data are sent via Bluetooth to a mobile phone (Samsung J327V, Android), which uploads the data over mobile data networks to the physIQ accelerateIQ cloud platform.
A universal smartphone was provided to maintain quality systems and standards in clinical and trial use. When the phone to patch distance exceeds blue tooth range, which is about 20 feet, the patch will store data and offload it when the phone is back in range. Raw data are then analyzed to extract further vital sign features and to detect vital sign anomalies in the patient physiological response. After that, the smartphone transmits data to the accelerateIQ data platform. physIQ operates in line with national standards for the privacy of health information and regulations for electronic records and signatures, pursuant to the Health Insurance Portability and Accountability Act of 1996 and the Code of Federal Regulations (CFR), respectively.
Patients wore a VitalPatch for a maximum of 12 days. The first patch was placed the week before the start of treatment and worn for 4 days to generate baseline data. The second patch was placed at the start of treatment (or the day before for transfusion patients), and a third patch, necessary because of battery life, was placed after 4 days by the patient at home (see Figure 2). The patch cannot be submerged in water, so while wearing the patch, patients could not swim or bathe. Showering was allowed.
A total of 2 nurses accessed accelerateIQ through a standard Web browser that displayed a patient dashboard. Within the dashboard, the nurses had access to data and analytics to determine if there were any abnormal reactions or potentially dangerous AEs associated with the treatment. The nurses also generally assessed the accuracy of monitoring but did not treat subjects based on portal data.

Outcome Measures and Data Collection
To understand the acceptability of use of the accelerateIQ plus VitalPatch system by patients and health care professionals, we assessed wearability and usability. Wearability was measured by a questionnaire, completed by patients at baseline, day 1, day 5, and day 8. The questionnaire was in the participant's native language of Dutch. Questions focused on the patient's evaluation of their experiences with the VitalPatch. They

Statistical Analysis
No sample size calculation was performed as this was an exploratory study. However, data were reviewed for trends. Descriptive statistics were used and means compared. AEs were analyzed by describing timing and extent of skin reaction.

Inclusion
From February to October 2018, 12 patients were enrolled in the study. Participants included 4 patients receiving red blood cell transfusions, 4 receiving proteasome inhibitors, and 4 receiving IV immunotherapy. These patients were receiving treatment for various diseases, including multiple myeloma, myelodysplastic syndrome, and non-Hodgkin lymphoma. A total of 83% (10/12) of the patients were aged 60 years and older, 83% (10/12) were male, and 92% (11/12) were non-Hispanic white (see Table 1). During the study, no abnormal reactions or AEs to transfusions and infusions occurred.

Wearability
Wearability was measured through patients completing an 11-question survey (see Table 2 and Multimedia Appendix 1). No difference in patient-reported outcomes (PROs) was observed when either the nurses or the patients applied the patch.
Patients considered wearing the patch pleasant (mean 6.7/10), and the patch remained in place (mean 8.1/10). Patients reported little discomfort (mean 2.2/10) and little trouble with sleeping because of the patch (1.6/10). Patients did not feel the patch was in the way (mean 2.3/10) and did not experience restrictions in daily activities because of wearing the patch (mean 1.3/10). Patients' reports of skin irritation did demonstrate a change over the patch-wearing period (mean of question 8 at baseline: 1.4; day 1: 2.3; day 5: 3.4). In fact, 3 patients withdrew from the study because of skin irritation. One of these dropouts had a case of relatively severe dermatitis with impacted skin integrity, the other 2 had mild discomfort with minor erythema. The mean of question 8 at day 8, however, normalized to 2.3. The dropout because of skin irritation caused a lower completion of PROs at day 8, biasing results. When withdrawals are excluded, the mean of question 8 at day 5 is comparable with that of day 8. This was also the case for prominent differences at question 3, question 5, question 7, and question 9 (corrected mean for withdrawal at day 5: 2.0, 1.7, 2.5, and 9, respectively). In addition to the questionnaire, connectivity issues between the patch and the phone were reported by 2 patients as bothersome (17%, 2/12). For one of the patients, this led to withdrawal from the study.

Usability
Usability was measured by a survey completed by 2 research nurses. Placement of the patch was reported to be easy (9/10). Initially, connecting the patch to the phone was at times troublesome but not insurmountable. The ability to monitor patients' vitals in this study was considered moderately pleasant (6.5/10). Research nurses considered data measured by the VitalPatch to be reliable (8/10).
A total of 4 patients (33%, 4/12) voluntarily withdrew from the study (3 because of skin irritation and 1 because of Bluetooth connection issues). Withdrawals because of skin issues were all reported when wearing the second patch. Withdrawals were evenly distributed over all 3 groups. Neither age, disease, therapy, or other demographic factors showed a trend with withdrawal or skin irritation.

Principal Findings
This feasibility study focused on the wearability and usability of the VitalPatch in the outpatient setting in patients receiving transfusions or infusions. We quantified the user experience among 12 patients wearing the patch for 12 days. Apart from skin irritation related to the patch in 3 patients, there was positive feedback on comfort and usability of the patch, and emphases on limited restrictions in daily activities because of the patch. Nurses reported ease of use and comfort with relying on data measured by the VitalPatch.
The dropout rate was high compared with earlier work by Selvaraj [14], where out of 70 patients wearing the HealthPatch for 50 days, 6 patients withdrew. Selvaraj also revealed skin irritation issues but also found the need to shave chest hair, personal lifestyle choices, frequent travels, and compensation as reasons for withdrawing from the study [14]. The main reason for dropout in our study was skin irritation. Potentially because our study group patients mainly consisted of patients with hematological malignancies, they were burdened so heavily by their disease and treatment that even relatively small inconveniences were enough to withdraw. Another explanation could be that patients needed more guidance in the first week of wearing the patch. Patients who wore the patch for the whole duration of the study, and thus had gained experience with it, reported little to no inconveniences. Small inconveniences, while missing guidance and experience with the patch, could discourage a patient enough to contribute to discontinuation of use.
Commercial use of the VitalPatch has revealed skin irritation in some subjects, which prompted physIQ's internal review of the VitalPatch and its relationship to skin irritation. It was discovered that the number of skin irritation reports increased (while statistically nonsignificant) after the introduction of Cavilon as a skin barrier. Therefore, Cavilon use has since ceased to be part of the patch placement instructions. Nonetheless, Cavilon use in this study might explain the dermal issues. Furthermore, we found no correlation in demographics, treatment, or other known parameters with skin irritation. Taking into account that the HealthPatch used in Selvaraj's trial [14] is a precursor of the VitalPatch, it is more plausible that the high dropout rate found in our study is simply caused by chance and the small sample size and not because of the product itself.

Strengths and Limitations
Sample size is a limitation of this study. Owing to the exploratory nature of the study, no sample size calculations and no statistical analysis have been applied. Another restriction is that dropouts did not complete the PRO survey at day 8. When comparing the mean values of the PRO survey at day 5 and day 8, it appears time improved patients' experience with the VitalPatch: PRO data at day 8 were clearly more favorable than those of day 5. However, when corrected for the withdrawals, the PRO means at day 5 and 8 remained the same. Thus, no changes in comfort and skin irritation were observed from day 5 onward. Furthermore, the fact that our cohort was predominantly comprised of older white men reduces its generalizability and is a limitation of this study.

Relationship to Previous Studies
Overall, comfort, wearability, and usability were comparable with the outcome of Selvaraj's trial, who also described encouraging feedback on patch-type biosensors for continuous home use [14]. Chan et al [15] looked into the accuracy and usability of measurements of the HealthPatch. These appear to be of comparable accuracy with those made by traditional, larger medical devices. They conclude their article noting that it is important that such a device must be easy to use if it is to be widely adopted. Our data demonstrated ease of use for patients and nurses alike.

Future Perspective
Besides aiding future studies, the VitalPatch has great potential for in-and outpatient health care. For instance, patients undergoing intensive treatment, such as those receiving an autologous stem cell transplantation, might benefit from continuous monitoring through quick registration and management of adverse events. Finally, in time, when enough experience with the VitalPatch has been gained, it may reduce nurses' workload by replacing frequent vital sign monitoring.

Conclusions
Among a variety of wearables [16], this report shows the VitalPatch to be an attractive device because of its wearability and low impact on daily activities in patients, therefore making it suitable for implementation in future studies.

Introduction
Globalization has mitigated a technological change in the health care sector. Thus, the application of digital technologies in medicine is becoming increasingly important. Internet-based health care applications include electronic health records, electronic prescriptions, and digital organization structures in the health care sector [1]. The introduction of digital technologies in medicine faces specific barriers such as end user acceptance [2]. Acceptance is understood to be the result of perception, a concluding evaluation, and a final decision, which leads to a specific attitude or voluntary action. The Technology Acceptance Model (TAM), developed in the late 1980s to study the use of digital technologies by employees, is a standard model to conduct acceptance research in the medical sector [3].
According to TAM, the user's intention to employ a new information system is influenced by his or her perception of its benefits and accessibility [2,4,5]. In other words, it identifies two main factors that determine the acceptance of an app: "perceived usefulness" and "perceived ease of use" [2,4]. The TAM is derived from the Theory of Reasoned Action, which aims to explain the behavior of users. It may serve to derive predictions about the end user acceptance and to evaluate and confirm already accepted apps [6,7]. The TAM has been developed continuously in the course of further research [2,8,9]. Because the previous theoretical models were incomplete, Lee and Coughlin (2015) developed a 10-factor model as an integrative approach to represent a total of 10 factors that may influence acceptance, namely value, usability, affordability, accessibility, technical support, social support, emotion, independence, experience, and confidence [10].
The acceptance of new technologies in medicine may facilitate access to health-related information or health care services and communication and may thereby significantly reduce errors and costs [2]. Identifying the crucial factors that constitute barriers may be decisive to ensure the acceptance of such innovative technologies [11]. It may also increase patient safety and ensure patient-centered care [12]. For example, an app such as an online patient record is directly linked to the improvement of Germany's health care system in terms of its integrated care, general practitioner-centered care, and outpatient specialist medical care. Thus, the goal of the implementation of such a record is its integration into the health care system, enabling appropriate and timely decision making and treatment [12].
The purpose of this study was to employ TAM to evaluate datasets pertaining to the use of digital technologies in medicine in order to determine their acceptance in Germany. The study was based on the ePatient Survey 2018, a market research assessment tool employed to evaluate the target groups of medical digital technologies. This survey entails questions on digital skills, user profiles, and possible apps and aims to generate a representative picture of the acceptance and spread of new digital technologies in medicine to derive respective practical recommendations for action [13].

Research Philosophy, Design, and Strategy
The ePatient Survey is one of the most comprehensive online surveys in the German-speaking region and has been conducted annually in the digital health sector since 2010 [13].
The survey uses standardized questionnaires to provide information about the prevalence, impact, and development of digital health apps. These questionnaires are distributed to patients across Germany through health-related websites, newsletters, and online communities. The participation is anonymous and voluntary. There are no defined exclusion or inclusion criteria other than participation in the online survey and completion of the entire questionnaire. The ePatient Survey 2018 was conducted between March 1 and May 1, 2018. A total of 37,589 participants were included, of which 9664 datasets were complete and a final 9621 were evaluated. The LimeSurvey software (Hamburg, Germany) was employed to conduct the survey. The evaluation of this survey assessed the responses to the following eight questionnaire items, with multiple answers allowed, where applicable: 1. Online programs and apps for health topics: How well were you able to handle them at the beginning? 2. What was the longest time you have ever used the same app or specific online disease, treatment, or health program? 3. Precisely how did your medication app help you? 4. How exactly did your online coaching program help you? 5. You said you have used an online coaching program for your illness or an app for your medications: Do you remember where you found out about this application? 6. Imagine that someone recommends an online program or app customized for your illness/treatment. From whom would you most like to receive it? 7. Imagine that you as a patient use an online program or an app that stores all your illness data for you at all times. With whom would you want to share this data (as a patient)? 8. Imagine that an app or an online program is tailored to you and your illness, including diagnostic and treatment data. From whom would you use such an application?
These questions were categorized according to TAM into questions assessing the "perceived use" and the "perceived ease of use."

Digital Health Care Apps
The first step was identification of the most common types of digital health care apps, which cover all information and communication technologies in the health sector including electronic health, mobile health, telemedicine, big data, and health apps [14]. The apps can be classified into seven types: health literacy promotion, analysis and knowledge, indirect intervention, direct intervention, case history documentation, organization and management, and purchasing and preventive [15]. The content of each category is listed in Table 1.

Sociodemographic Data
In the second step, sociodemographic data of the ePatient Survey 2018 was evaluated to allow for ranking of the evaluation of the respondents based on their age and educational level.

User Typology
The last step in the study was an assessment of the user typology and differentiation into healthy users, patients with acute conditions, and patients with chronic conditions and characterization of users versus nonusers.

Ethics
Ethical standards associated with social science research were applied to this research. This study aimed to interpret the data such that it reflected its original emphasis rather than the researchers' own preferences. Moreover, the study protected the privacy of the sources of any views expressed within the study interviews by conducting the survey in an entirely anonymous fashion. At no point were any personal data pertaining to the respondent's name or medical records obtained.

Sociodemographic Data
The average age of the respondents in the study was 59.7 years (SD 16 years). The majority of the respondents aged <40 years were female (n=6735, 70%). In contrast, 60% (n=5773) of the respondents aged >70 years were male [13]. In terms of the educational level, 41% (n=3945) of the respondents had a university or technical college degree, 40% (n=3848) had a high-school diploma ("Realschulabschluss") or a university-entrance diploma ("Abitur") without higher studies, and 18% (n=1732) had a certificate of secondary education ("Hauptschulabschluss"). In an all-German comparison, the academic rate in the survey was well above average, which did not significantly affect the study's validity [13]. In addition, 72% (n=6927) of the respondents had government insurance, while 11% (n=1058) had a private insurance and 17% (n=1636) had both.

User Typology
Long-term patients with chronic diseases accounted for the greatest proportion of users (n=4041, 42%); this group consisted of predominantly male patients with an average age of 63.3 years and no academic background. The majority of those with chronic diseases (n=2966, 73.4%) were receiving treatment at the time of the survey, with 51% (n=1513) of these patients taking medication, 19% (n=564) taking part in physical therapy, and 17% (n=504) receiving regular outpatient and inpatient treatment in a clinic.
Diseases for which digital apps were employed included primarily diseases of the locomotor system (n=1948, 27%), cardiovascular diseases (n=1876, 26%), and metabolic diseases (n=1448, 20%), followed by pain syndromes (n=938, 13%), psychiatric disorders (n=866, 12%), and ophtalmological diseases (n=721, 10%). The second largest group comprised patients with acute conditions (n=3175, 33%), with an average age of 56.6 years, and this group had a higher proportion of women (n=2000, 63%). Users in this category tended to be college graduates. Healthy users made up the smallest group (n=2405, 25%), with an average age of 58.6 years and no sex-relevant tendencies; this group had a significantly higher proportion of university/college graduates [13]. Most patients were receiving ongoing treatment during the survey and thus used the internet more frequently. The second most common phase was change of treatment or desire for a change; this phase was associated with slightly increased internet usage [13].
The intensity of digital medical app use was classified into nonusers, users, and intensive users. Nonusers were defined as never having used any digital medical app due to the lack of need for or interest in such apps. People in this category showed a limited willingness to share their data and a certain mistrust toward anybody but their physician. The average age in this group was 63 years, 4 years above the overall age average of 59 years. The level of education of people in this group was slightly lower than that in the user group [13]. The user group comprised people who regularly used digital apps for medical devices, medication, coaching, and obtaining a second opinion online. People in this group had a higher level of knowledge, a higher need for an online health record, and were generally more willing to share their personal data.
Of the respondents aged ≤40 years, the majority were women, while a higher proportion of men were observed in the age group of those aged >60 years. The average age in this group of 52 years was 7 years below the average age of 59 years. This user type had a slightly higher level of education than nonusers. Intensive users exhibited the greatest need for online medical records and consequently had the highest knowledge. The most frequently used apps in this group were, similar to the user group, those for medical devices, medications, coaching, and obtaining an online second opinion. Every second person in this group had completed an academic degree [13]. The average age was the same as that in the user group (52 years)-7 years below the overall age average. A higher age correlated with a higher proportion of men.
The evaluation results of the responses to the questionnaire items are summarized in Table 2. Evaluation of the responses Questions used in the study from the ePatient Survey 2018

Questions assessing the perceived usefulness according to the TAM a
In this scenario, 81% (n=7793) would like to share their data with their attending physician, 35% (n=3367) with their clinic, 28% (n=2693) with their health insurance provider, 13% (n=1251) with none of the above, and 5% (n=481) with the company producing their medication.
"Imagine that you as a patient use an online program or an app that stores all your illness data for you at all times. With whom would you want to share this data (as a patient)?" (value, social support, emotion, confidence) 37% (n=1607) of users stopped using it after only a few days, 20% (n=869) after a few weeks, 19% (n=825) after a few months, and 24% (n=1043) used it for ≥1 year.
"What was the longest time you have ever used the same app or specific online disease, treatment, or health program?" (experience, usability) 51% (n=306) of users said it helped them take their medication regularly, 27% (n=162) said it made no difference, 22% (n=132) saw somewhat of an improvement, 57% (n=342) saw an improvement in handling their medication, 29% (n=174) saw somewhat of an improvement, and 14% (n=84) did not see an improvement.
"Precisely how did your medication app help you?" (value, usability, accessibility, emotion, independence, experience, confidence) 33% (n=41) of online coaching program users said they were coping better with their illness in everyday life, 50% (n=63) saw somewhat of an improvement, and 17% (n=21) saw no improvement.
"How exactly did your online coaching program help you? -I cope much better with my illness and my everyday life with the illness." (value, usability, accessibility, technical support, social support, emotion, independence, experience, confidence)

Questions assessing the perceived ease of use according to the TAM
More than 50% (total n=4446) of users initially had minor to major difficulties operating health programs and apps, 46% (n=2045) stated that it was easy from the start, 39% (n=1734) stated that it required some experimentation and patience, and 15% (n=667) had major issues.
Online programs and apps for health topics: "How well were you able to handle it at the beginning?" (usability, accessibility, independence, experience, confidence) 68% (n=717) discovered the app by searching the internet themselves, 16% (n=169) received a recommendation for the app from their health insurance fund, 9% (n=95) from their physician, 8% (n=84) from family and friends, 8% (n=84) from magazines or the radio, and 5% (n=53) from their pharmacy.
"You said you have used an online coaching program for your illness or an app for your medications: Do you remember where you found out about this application?" (accessibility, independence) 72% (n=6927) would prefer to get this app from the attending physician, 40% (n=3848) from their health insurance provider, 20% (n=1924) would search for it by themselves on the internet, 15% (n=1443) would obtain it through a pharmacy, 13% (n=1251) from their hospital, and 8% (n=770) and 5% (n=481) from the company producing the medical device or medication, respectively. "Imagine that someone recommends an online program or app customized for your illness/treatment. From whom would you most like to get it?" (technical support, social support, emotion, experience) The majority of respondents said that they would use such an app if it came from their health insurance provider (55%, n=5292) or their physician's software (55%, n=5292). In addition, 23% (n = 2213) would use a governmental app, 12% (n=1155) one from an information technology provider in Germany and 6% (n=577), 5% (n=481), and 1.5% (n=144) from Google, Apple, and Amazon, respectively. "Imagine that an app or an online program is tailored to you and your illness, including diagnostic and treatment data. From whom would you use such an application?" (affordability, accessibility, technical support, social support, emotion, independence, experience, confidence)

Principal Findings
The decision to integrate new technologies in the health care industry depends on different factors common to patients, medical professionals, and health care providers. Knowledge on the attitude of potential users toward such technologies eventually determines their success on the market. In this study, we therefore assessed the frequency of use of different digital medical apps in Germany to characterize the current user population and identify the perceived usefulness and ease of use of such apps.
Our most important findings and a brief discussion of each finding are presented below.
The perceived usefulness of the apps was positively supported by the statement from most patients that such an app helped them regularly take their medication. Nonetheless, most users did not use the app for an extended period of time, with less than a quarter of the respondents continuing its use for a year or more. The acceptance is therefore apparently very short-lived and may limit the potential benefits (eg, long-term apps such as storage of the medical history). The most prominent perceived usefulness is the sharing of data stored in an app with their attending physician. In contrast, online coaching programs were not perceived as a helpful tool by most participants, with only one-third noting an improvement in their daily life after participating in such a program. If the users fail to see any long-term value in online coaching programs, the probability of acceptance likely decreases. Of note, the use of such programs has increased from 5% (n=200) in 2016 to 14% (n=1485) in 2018, likely due to increasing advertisement and a broader availability of such different programs.
In terms of the perceived ease of use, we discovered that users of medical digital apps are still facing minor or major difficulties in operating them. Furthermore, there appears to be a discordance between the preferred access to such apps (ie, the attending physician) and the actual way such apps are accessed (ie, the internet). At the same time, nonusers frequently showed a certain distrust toward people administering such apps other than their physicians. In addition, 72% (n=6927) would prefer to receive this app from the attending physician. This shows a clear divergence between the distribution channels actually used and the distribution channels desired.
Thus, the physician is clearly preferred to other parties (such as a hospital, health insurance fund, and medication manufacturer). User confidence is a crucial factor in the long-term acceptance of new technologies. There is generally a relationship of trust between the patient and the (family) physician that is sometimes built up over several years. To make the introduction of innovative technologies sustainable and promising for the user, it is advisable, based on current data, to have the attending physician function as the direct distribution channel.
An online health record system could facilitate the documentation of a patient's case history and prove to be cost-effective. Such a record system, the Siemens patient records system, was used in Austria as part of the digiLog project [16].
In this system, patient data were stored as a mobile electronic file and could be shared between the attending physicians and the patient [16]. However, the online health record has been receiving a low participation to date (n=241, 2.5%), possibly due to its pilot project status due to limited health insurance funds, making it accessible only to privately insured patients [13]. Its nationwide use requires the right framework, such as a basic social understanding of the benefits of such a record system and strategies for building and disseminating it [12].
In our study population, there was a low level of awareness of online patient record systems among female respondents and an apparent lack of interest in younger male respondents with a higher level of education. We speculate that a low level of awareness might prevent patients from using the available apps. An app's acceptance is generally linked to awareness of its capabilities. Hence, an increased promotion of digital medical apps, ideally by the person of trust, namely, the physician, could potentially increase their acceptance among patients.
Acceptance must be further differentiated according to sociodemographic factors and the types of the respective apps [17,18]. A recent study has shown relatively lower acceptance rates in older individuals [19]. Respondents in this study were 59 years old on average, with most of the respondents aged between 44 and 76 years. An extrapolation of our results to a younger population is therefore not possible, and we cannot judge the acceptance in younger adults in our cohort.
Several recent reports have addressed the acceptance of digital health technologies in specific cultural and socioeconomic settings. A pilot study in female respondents in rural Uganda has shown overall openness among the respondents to accept computer-assisted personal interviewing. The implementation of such a technology can, in the long run, be very beneficial for the community because it would provide a cost-effective and more easily accessible health care alternative [20]. It appears crucial that the anxieties and insecurities of both patients and medical professionals are considered to increase the chances for their acceptance [21].
Digital medicine has great potential to bring individualized health care solutions, but challenges for their successful implementation still exist. Many apps still need to be validated in clinical settings and, often, initial pilot studies may be underpowered [22]. Ethical challenges also need to be addressed, such as concerns about the confidentiality of personal data [23,24].
The survey revealed significant differences in the current spread, use, and perception of digital health technologies among users in Germany. Suitable framework conditions must be generated, particularly in the case of groups that currently still show a low level of acceptance of specific apps (including online patient records and online second opinions). Such conditions must take into account socioeconomic aspects and the use of apps by the younger generation as well as sex differences. Patients must also be informed about the capabilities of the app and a suitable offer must be made available.
The obvious limitation of our study is the average age of the respondents-59 years. This can most likely not be considered representative of the German population as a whole, which had a median age of 47.1 years in 2018 [25]. The reason for this high average age can be attributed to the type of channels through which the ePatient Survey was distributed, namely, health-relevant websites and newsletters, whose readership is typically older.

Conclusions
Our evaluation demonstrates the acceptance of medical digital technologies among a selected population group in Germany, primarily as tools to communicate with the attending physician. Our findings highlight the need to generate a framework for such technologies by increasing the knowledge on their existence and benefits and supporting them with respective funds from the health care providers.

Multiple Sclerosis, Diet, and Lifestyle
People with multiple sclerosis (MS), an autoimmune inflammatory disease in the central nervous system, experience individual and complex symptom patterns (eg, fatigue, cognitive impairment, walking difficulties, pain, bowel dysfunction, and bladder dysfunction) [1][2][3]. Symptom severity and fluctuations affect the perception of health and quality of life, and some people with MS identify diet and other lifestyle factors as triggers for daily symptom worsening [4][5][6]. A possible approach to investigate correlations among MS symptoms, diet, and lifestyle would be to collect daily patient-reported data on relevant variables. However, this kind of data collection relies heavily on daily manual registrations made by participants with MS.

Digital Data Collection
A digital, smartphone-based data collection has the potential to improve the participants' experience and, at the same time, reduce recall bias compared with a traditional pen and paper study design [7,8]. Furthermore, previous studies indicate that the use of digital and internet-based services is high in MS populations and, in some countries (eg, the United States), higher than the internet use in a general population sample [9][10][11].
However, heavy manual data registration place demands on the participants, and furthermore, studies testing digital patient portals, remote care services, and symptom management solutions all find that physical MS symptoms including vision impairment can cause barriers related to adoption if not addressed in the design and development stages [9,10,12]. Although studies investigating electronic health (eHealth) services and MS primarily focus on physical symptoms, only 1 study has investigated how cognitive symptoms and impairments might affect the use of eHealth technologies [12].

Electronic Health Literacy and Adoption
To achieve successful use, adoption, and value to both participants and researchers in a project with heavy manual data collection, it is necessary to design a useful digital tool with high usability to provide a good user experience to facilitate the adoption [13]. However, the relationship between usefulness and successful use is moderated by participants' eHealth literacy levels and the system demands on eHealth literacy [13,14]. eHealth literacy can be described as the competencies and skills needed to engage with eHealth tools, and Monkman's model and Kayser et al's expanded user-task-context matrix suggest that the design of a digital tool should match the eHealth literacy levels of the target population to ensure successful use and adoption [13,[15][16][17]. The eHealth literacy framework (eHLF) is a multifaceted, conceptual model with 7 distinct dimensions describing eHealth literacy as knowledge, skills, trust, motivation, and user experience with the system aspect. eHLF's dimensions are (1) ability to process information, (2) engagement in own health, (3) ability to engage actively with digital services, (4) feeling safe and in control, (5) motivation to engage with digital services, (6) having access to systems that work, and (7) digital services that suit individual needs [18].
Although the eHLF contains a dimension that focuses on motivation to use eHealth, a study in symptom management in MS concluded that adoption and completion in an eHealth-based randomized controlled trial (RCT) might have been improved by addressing willingness to participate [19]. This suggests that adoption and successful use in a digital data collection are also influenced by content and purpose. In Deci and Ryan's self-determination theory, extrinsic motivation involves the individuals' experience of competence and autonomy as well as relations to others to become motivated [20]. Although the eHLF covers aspects of this, eHealth literacy might be supplemented with additional dimensions focusing on the participants and their relationship with MS, diet, and lifestyle factors.
By complementing the eHLF with selected dimensions from the Health Education Impact Questionnaire (heiQ), we are able to cover aspects of competence and autonomy as well as relatedness. The heiQ is a validated and widely used patient-reported outcomes measure evaluating patient education.
The heiQ consists of 8 dimensions, which describes outcomes related to self-management behavior, and the dimensions have been found to capture aspects strongly related to empowerment [21].
We selected the following 3 dimensions that broaden the motivation aspect from the eHLF: positive and active engagement in life, self-monitoring and insight, and social integration and support [22].
We here report how we have used a combination of the theories of Monkman, eHLF, and heiQ in an analytical framework to identify factors that may be associated with motivation and adoption in a digital data collection relying on manual data reporting from participants with MS.
We use a sequential mixed methods design, which combines interviews, survey, and focus group interviews to gain in-depth knowledge through qualitative phases that are validated in a larger population in a quantitative format.

Study Design
In 2016, the Danish MS Patient Society established a research project, the KosMuS project, in collaboration with the University of Copenhagen. The aim was to explore potential correlations between diet and MS disease activity by inviting people with MS to register diet intake, lifestyle factors, and daily changes in MS for up to 100 days. The sequential mixed methods design that was used in this study consists of 3 phases. In phase 1, 15 semistructured interviews were conducted with people with MS. Of 3 categories, 2 identified in the phase 1 analysis were substantiated in a larger population in the phase 2 survey. Findings from phases 1 and 2 described eHealth literacy levels, health behavior, and attitude toward research participation in the participant population. All findings were discussed in 3 focus group interviews in phase 3 to explore how these would and could affect adoption and participation in the KosMuS research project. An overview of the study design is presented in Figure 1.
The study included people diagnosed with MS, Danish speaking, and aged older than 18 years. Severe cognitive impairment and aphasia were exclusion criteria for the interview-based data collections in phases 1 and 3.

Data Collection
This study's data collection was conducted together with a second part that explored how individuals with MS experience nutrition to affect their MS disease activity. Following data collection, the 2 parts were handled and analyzed separately. A detailed description of the data collection is presented together with the findings on diet and MS [6].

Phase 1: Semistructured Interviews
Guided by the eHealth Literacy Questionnaire and heiQ frameworks, interview guides were based on the 7 dimensions in eHLF and 3 dimensions from the heiQ. The dimensions were chosen by authors AK and LK through discussions that carefully examined the study aim compared with descriptors for each of the dimensions. The selection was furthermore based on other studies' experiences with using the concepts as frameworks for qualitative studies instead of scale constructs, which both eHLF and heiQ were originally developed for [23].
On the basis of the selected dimensions, authors AK and LK constructed 7 items for the interview guide. Each item and its subtopics for conversation cover 1 to 3 dimensions from eHLF or heiQ. In the interviews, the 7 items were followed by a short introduction to the KosMuS project and 2 items that invited participants to share their thoughts on project design. A total of 2 pilot interviews were conducted to test the interview guide and the framework's coverage. Pilot interviews were conducted by author AK, who interviewed 2 individuals with MS at the Danish MS Patient Society's premises in Valby. Following the interview, the 2 participants provided feedback on the items. Their interview responses and item feedback were combined with interviewer's notes and then discussed between all 3 authors with consideration to study aim, participants' experience, framework coverage, and the interviewer's role. All 9 items were kept in the interview guide, but several item adjustments were made to improve understandability, for example, from "Which barriers do you face when using technology?" to "Which challenges do you face when using technology in everyday life?" The pilot interviews are not included in the final dataset. A selection of items and their relation to the conceptual dimensions are found in Table 1.
A total of 15 semistructured interviews were conducted from July to August 2016. Participants were recruited through a post on the Danish MS Patient Society's Facebook page and from the Multiple Sclerosis Hospital in Haslev, Denmark [6]. The interviews were conducted by author AK and lasted 25 to 75 min. All interviews were audio recorded and transcribed by author AK, and all participants were given a pseudonym.

Phase 2: Survey
Of 3 categories, 2 identified in the phase 2 content analysis of interviews were further investigated in a survey. The survey format was chosen to investigate these findings in a larger population and a population that was less invested in the study than those who had contacted the research group or accepted an invitation for a face-to-face interview. The survey was distributed with a minimal time gap, and data were gathered to compare data with the findings from the interviews [24].
A draft for survey items was constructed by authors AK and LK based on the findings and overall study aim. The draft was closely scrutinized and adjusted in a meeting with all 3 authors. The authors focused on phrasing the items as closely as possible to the interview findings to ensure alignment [24]. A total of 14 items were constructed for the survey, and an excerpt is presented in Textbox 1. All 14 survey items can be found in Multimedia Appendix 1. The survey was constructed and managed digitally via SurveyXact [25]. It was sent out by email to a randomized sample of 1000 people from the Danish MS Society member database [6]. A reminder was sent to nonrespondents after 10 days.

Phase 3: Focus Group Interviews
The final phase of this study consisted of 3 focus group interviews that explored how findings from phases 1 and 2 could inform project design and development of the digital data collection tool.
The group dynamics in focus group interviews allows for an often deeper and richer data material, created through the social interaction facilitated by the interviewer [26]. All 3 interviews included an exercise, in which participants were asked to collaboratively rank and discuss reasons and incentives for participating in a digital health research project on nutrition and MS. The project would contain daily registrations on symptoms, diet intake, and lifestyle factors, which would have to be registered manually through a smartphone-based digital data collection tool.
The 3 focus group interviews were conducted in the Danish MS Society in Valby, Denmark (3, 4, and 5 participants in each) in January 2017 [6]. Participants were recruited from a list of people who had signed up for receiving updates on the KosMuS study. Interviews lasted 90 to 125 min and were recorded on Dictaphone and later transcribed by author AK [6].

Semistructured Interviews
Data were analyzed using content analysis [27]. This method allowed for a deductive coding of statements and topics related to eHealth literacy, empowerment, or health behavior dimensions while still leaving room for the inclusion of inductively identified categories. Interviews were coded in NVivo by author AK [28]. After coding of the first 3 interviews, the coding strategy was carefully reviewed and discussed with author LK. These 3 interviews were then recoded based on the adjusted coding strategy. Codes were printed out on paper and grouped into categories that were named. This work was conducted by author AK, followed by 2 sessions in which all 3 authors participated and reviewed codes and categories.

Survey
Basic descriptive statistics were used to identify response patterns [29]. Summarized responses were then compared with the earlier findings from the interviews.

Focus Group Interviews
The transcribed focus group interviews were deductively coded according to identified categories from the analysis of semistructured interviews. Coding was conducted in NVivo [28]. Coding strategy and excerpts of coded material were discussed with authors LK and LS. Code content was compared with the findings from previous phases, and the results were discussed between the authors. On the basis of this, the results were divided into factors that could encourage or discourage participation.

Ethical Considerations and Data Agency
In this study, no biological material or medical devices were used, and the participants were not subjected to any kind of diagnostics or treatment. Consequently, approval from the Danish National Committee on Health Research Ethics (Den Nationale Videnskabsetiske Komite) was not required, which is the case for all studies only involving interviews and questionnaires [30].
The study was registered and approved with the Danish Data Protection Agency (2016-41-4723).

Participants
A total of 5 men and 10 women, with a mean age of approximately 47 years (ranging from 28 to 69 years), accepted an invitation to an individual interview in phase 1. In the phase 3 focus group interviews, 2 men and 10 women with a mean age of approximately 50 years (range: 26-63 years) participated. Overall, 78.5% (334/425) of the survey participants were women, and the mean age among all survey participants was approximately 52 years (range: 19-77 years). The participants across all 3 phases represented the varying types and stages of MS, and they had, on average, been diagnosed for approximately 12 years. Each interview participant only participated in 1 interview activity, whereas the randomized sample used for the survey did not consider earlier participation. An overview of participants' sociodemographic distribution is shown in Table  2.
Overall, 7 of the interview participants in phase 1 and all 12 participants in phase 3 were invited from a list of individuals with MS who had signed up for updates on the KosMuS project on nutrition and MS. In addition, 8 of the interview participants and all survey participants were invited without having shown prior interest in KosMuS. Categories identified in the interview analysis (phase 1) were present among both those who signed up by themselves and those who participated after being invited. However, participants who had indicated interest in the project and considered themselves likely participants in the KosMuS project were less concerned of overcoming factors that might act as barriers.

Findings From Phase 1 Interviews and Phase 2 Survey
The content analysis of the phase 1 interviews identified the following 3 categories: (1) life with MS, (2) use of technology, and (3) participation and incentives. The first category represents the context of living with MS and how it affects everyday life. Category 2 relates to the everyday use of technology with regards to both skill and attitude toward the use. The final category contains participants' thoughts on and motivations for participation in research projects in general and the KosMuS project specifically.
In the following, each category is presented together with the findings from the survey in phase 2.
These are followed by a presentation of the findings from phase 3 focus group interviews, which are summarized focusing on factors that encourage or discourage adoption and participation.
An overview of categories identified in the content analysis can be found in Table 3, and survey results are summarized in Table  4.

Life With Multiple Sclerosis
The analysis of the semistructured interviews showed that all participants were affected by their MS in everyday life. Factors related to life with MS that were identified as potential influencers on the participation in a research project with digital data collections were divided into the following 3 subcategories: (1) physical and cognitive limitations, (2) disease management in everyday life, and (3) the social aspect and sharing with other. Table 3 provides an overview of each category. In the survey, 73.4% (n=245) participants answered that they experienced fatigue on a daily basis, and 78.2% (n=332) related daily changes in energy levels to MS. When responding to the statement "I feel limited because of my MS," 74.2% (n=315) answered agree or strongly agree, 84.3% (n=358) of the respondents said that they felt the need to ration their energy on days where they experienced MS symptoms, and 46.4% (n=197) said that they sometimes turn down arrangement or social events because of the MS. Table 4 provides an overview of survey results.
Although the analysis of the interviews indicated that participants found the disease-modifying drugs to be a necessary evil, less than a fourth (23.5%, n=100)) of the survey respondents agreed that taking medication affected their daily quality of life.
Survey responses supported the interview analyses with 89.0% (n=377) not only agreeing that social support was important to them in relation to their MS but also considered it their own responsibility to learn how to live with their MS (97.4% agree or strongly agree, n=416). The ambivalence of interacting with other people with MS was reflected in response to the statement "It means a lot to me to participate in networks (with other people with MS)," with 39.5% (n=168) responding agree or strongly agree, and 60.3% (n=257) disagree or strongly disagree.

Use of Technology
The analysis showed that participants all used technologies such as smartphones or computers in everyday life. Technologies were used to communicate with family and friends, as an aid (reminder and calendars), and to share experiences with others with MS. Although most participants had positive attitudes toward technologies, a few found it to be antisocial and frustrating to have to use.
In the survey, 79.4% (n=338) said that they used (digital) tools (eg, calendars and reminders) to help them in everyday life. These responses were in line with the findings from the interview analyses. Moreover, 75.8% (n=323) agreed to the statement "I make an effort to avoid that MS makes me appear different from others."

Participation and Incentives
Participants' reflections were mainly divided into thoughts on participation and motivations for participating. Primary motivations for participation included to contribute to research, an interest in the topic (here diet and nutrition), and to contribute and share own knowledge. Although participants were motivated to participate in a large-scale digital data collection, they expressed worries about the extensiveness of the registrations and the time consumptions. An overview of the results is found in Table 3.
In the following survey, when asked about if respondents could imagine participating in a project such as KosMuS, 20.7% (n=88) answered definitely; 8.8% (n=37) yes, depending on how good the app is; 27.4% (n=117) yes, depending on the workload; and 43.0% (n=183) answered that they would not be interested in participating.
Of those who were interested in participation, the main motivations for participation were listed as the contribution to research and the possibility of learning more about themselves and their MS. When asked to estimate an acceptable workload per day, participants' answer ranged from 4 to 60 min, with an average of 16 min.

Focus Group Interviews: Finding the Motivation
In each focus group interview, the KosMuS project was presented together with the findings from phases 1 and 2. Participants then discussed how the findings in each of the 3 categories could encourage or discourage participation in the project and adoption of the tool for digital data collection.
All 12 focus group interview participants considered themselves potential participants. However, the majority of participants emphasized that there were a number of conditions that would have to be met for them before they would enroll.
In the following sections, findings from the focus group interviews are grouped into reasons for not participating and motivations to participate.

Reasons for Not Participating
Some participants stated that they had plenty of time to spare in daily life, but for the majority of participants, the amount of time spent daily on registrations would be a crucial factor for their decision to enroll in the research project. Available time that could be allocated was affected by family, work, and parts related to living a life with MS symptoms. All participants agreed with the survey response that indicated a maximum of 15 min per day.
The results of the analysis indicated that when talking about the importance of daily time consumption in the project, participants wanted simple registrations that were convenient and not considered invasive in relation to their everyday life routines. Participants particularly emphasized the importance of convenience in a project such as KosMuS that stretches over several months. This also included being able to do everything from home: Participants were interested in easy registrations where it was okay to make estimates of portion size. In the third interview, they also emphasized the importance of this in relation to bad days with MS. Days where symptoms increased might be not only the most important ones to register but also the hardest for the participants to find the energy to do so.
Talking about nutrition and the hypotheses on MS being affected by nutrition, several participants raised the concern that projects such as KosMuS might attract people who are already interested in MS diets, and that people with normal eating patterns would choose not to participate because they did not feel that they were eating right: There might be some that live such unhealthy lifestyles that they don't want anyone to see or get involved. [ Participants emphasized that it would be important to provide clear information on what it takes to participate in the project and that communication before, during, and after participation would be crucial. Before the project communication should contain information on how to get started and during the project period, several participant discussions focused on the importance of knowing that someone was receiving the information they registered, and that it was a valuable contribution toward the project aim:

Motivation to Participate
A total of 7 participation incentives that had been identified in the interviews were handed out to each focus group and were in collaboration listed by importance to the participants (see results summary in Textbox 2).
When asked to rank participation incentives, the following 3 incentives were the highest ranked across the focus group interviews: to contribute to research, to learn more about myself and my MS, and to exchange experiences with other people with MS.
When describing the third-highest ranked incentive about exchanging information with other participants with MS, several individuals talked about good experiences, and that they considered it healthy in general to talk to others in the same situation. For others, the exchange of experiences was a way to learn about things that had helped others: Focus group 2 inserted an additional category Feedback that they described as getting feedback during the project and after the project had been concluded.
Participants, in general, agreed that weight loss would not be a personal motivation for participating in the research project, and to assist in weight loss was prioritized last among motivations in all 3 groups.

Principal Findings
Our findings show that when organizing and designing a tool for digital data collection in research projects for people with MS, there are disease-specific implications that are likely to affect the adoption and accessibility. Cognitive and physical symptoms related to MS such as vision impairment, tremors, and dizziness/fatigue might lower the accessibility if the digital tool is not suited to fit the needs caused by various MS symptoms. Despite limitations caused by MS symptoms, our findings indicate a high level of technology use in the Danish MS population, and participants in this study used smartphones for both everyday life communication and as MS aids-for example, reminders and alarms.
The adoption of a digital tool together with the research project itself is not affected by the disease itself. Family, friends, and peers with MS affect how individuals with MS use technologies and, particularly, smartphone-based solutions in everyday life. Medication and its side effects together with the uncertainty of the disease affect the willingness to participate in a project such as KosMuS.
Worries and reasons not to participate are primarily linked to the content and workload of the project and not so much the digital tool itself that provides the convenience and flexibility of not having to show up on particular times and places.
Our results indicate a positive attitude among people with MS toward participating in research. The primary incentive was the contribution to research and in the long run to contribute to new knowledge on how to better manage MS. This was closely followed by the wish to learn more about oneself and what affects one's own MS and to share experiences and advice with other people with MS.
For people with MS to enroll in a quite extensive research project such as KosMuS, the tool needs to be convenient and easy to use. It should be stated clearly what is expected of the participants, and our findings indicate that the communication between project coordinators and participants and the feedback to participants are equally important to participants compared with the design of the digital tool itself.

Previous Research
Few studies have investigated the implications of MS when designing eHealth or other technologies for an MS population. Atreja et al published a qualitative study in 2005 that informed on the design of a Web portal for individuals with MS [12]. The authors identified similar barriers to using a digital tool because of physical limitations caused by MS. However, the study was conducted before the introduction of smartphone technologies and large-scale social media. Although some of the limitations remain the same, Web 2.0 has changed accessibility, for example, smaller screens affect people with vision impairment, and small buttons on a touch screen affect people with fine motor skills impairment.
Although we have not been able to identify other studies investigating the motivation for participation in research among people with MS, Carroll et al explored motivations for participation in RCTs among people with pulmonary arterial hypertension [31]. Similarly, their results showed that major motivations were related to both personal interests (eg, getting better) and altruistic motives (eg, contributing to research). However, compared with our study, the motivational factors identified by Carroll et al are more focused on clinical aspects (eg, safety) than the learning experience (eg, learning more about one's own MS). This might be because of different research contexts (RCT vs observational digital data collection) or different diseases.
This study used dimensions from eHLF and the heiQ to investigate factors that affect the adoption and actual use of the eHealth tool as well as participants' willingness to participate in a project such as KosMuS. The dimensions covered in the interview guide are to an extent represented in the findings. Dimensions such as ability to actively engage with digital services (eHLF, dimension 3), digital services that suit individual needs (eHLF, dimension 7), engagement in own health (eHLF, dimension 2), and self-monitoring and insight (dimension 5, heiQ), are clearly reflected in the identified categories' content. On the contrary, feel safe and in control (eHLF, dimension 4) is less evident in the findings. The dimension was included in all interviews, but the topic did not spark any elaboration or clear opinion on data safety and trust among the participants. This might be because the dimension is not a matter of concern to the participants. The finding is in line with a study on eHealth literacy in a Danish outpatient clinic population that also observed less concern about eHLF's dimension 4 [32].

Methodological Considerations
Participants in all 3 phases of the study signed up voluntarily, and all focus group interview participants and half of the individual interview participants had already indicated an interest in the KosMuS project before being invited to interviews. This might indicate that parts of our study population have an existing interest in nutrition, research, and eHealth. Identified motivational factors in this study such as to exchange experiences with others with MS and to learn more about myself and my MS might not be applicable to potential participants who do not have an interest in nutrition or self-management.
Therefore, the data collection participants might also be more interested in nutrition and self-management, which may affect the collected data, for example, with under-/overrepresentation of different nutrition patterns.
A majority of participants in all 3 phases were female. This is consistent with the background MS population, in which more than 2 of 3 participants who diagnosed with MS are female [33,34].
Using a sequential mixed methods design, we have been able to continuously qualify and strengthen our results. However, we acknowledge that the researchers' subjectivity is an integrated part of the research process [35]. We have encouraged discussions of this matter in our frequent meetings and in both the processes of design and analysis. Using a framework for the interview, we risk guiding the participants too much, but the framework has also made it easier to address the introduction of elements into the interview guides, survey design, or analyses that were not in line with the original study aim.
There might also be underlying factors that those interested in nutrition and lifestyle are also the ones that are more affected by the side effects from medication. Although medication side effects were a big part of the interviews, only 23.5% (n=100) agreed to the statement that medication affects the quality of life negatively.

Recommendations
Our findings support the hypothesis that a system's design should be adjusted to meet the eHealth literacy levels of the user group [13]. In our study, we have used an analytical framework with a combination of dimensions from the eHLF and the heiQ. Future research should further explore how the combination of eHealth literacy and empowerment-related dimensions might assist the development and implementation of digital tools focused on disease management and other patient groups.
It is important to acknowledge that in projects similar to the one described here, the participant burden is high; that is, the project organizers need to contribute with extra work to avoid imbalance. The results from this study should be incorporated into the development of the eHealth tool for the KosMuS project. However, several of these findings are disease specific and not related to this particular project. When designing eHealth-based solutions for people with MS, factors related to the disease and living with the disease should be included in the design phase.

Conclusions
The interviews and survey in this study identified 3 categories that are important to address in the design of an eHealth-based research project in a population of individuals diagnosed with MS: life with MS, use of technology, and participation and incentives. The focus group interviews furthermore identified to contribute to research, to learn more about one's own MS, and to share experiences with others as main motivational factors for participation. These factors should be taken into consideration in the design of a study dependent on user-generated data in an MS population and in the development of a digital tool for data collection.