This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.
Although mobile health (mHealth) apps are increasingly being used to support patients with multiple chronic conditions (multimorbidity), most mHealth apps experience low interaction and eventual abandonment. To tackle this engagement issue, when developing an mHealth program, it is important to understand the social-behavioral factors that affect patients’ use behavior.
The aim of this study was to explore the social and behavioral factors contributing to patients’ use behavior of an mHealth app called the electronic Patient-Reported Outcome (ePRO). The ePRO app supports goal-oriented care delivery in interdisciplinary primary care models.
A descriptive qualitative study was used to analyze interview data collected for a larger mixed methods pragmatic trial. The original 15-month trial was conducted in 6 primary care teams across Ontario, Canada, between 2018 and 2019. The eligibility criteria for patients were being aged ≥60 years with ≥10 visits within the previous 12 months of study enrollment. For this analysis, patients were classified as long-term or short-term users based on their length of use of the ePRO app during the trial. The Social Cognitive Theory by Bandura was used to categorize social-behavioral factors that contributed to patients’ decision to continue or discontinue using the app.
The patient-provider relationship emerged as a key factor that shaped patients’ experiences with the app and subsequent decision to continue using the app. Other factors that contributed to patients’ decision to continue using the app were personal and social circumstances, perceived usefulness, patients’ previous experience with goal-related behaviors, and confidence in one’s capability. There was an overlap of experience between long- and short-term app users but, in general, long-term users perceived the app to be more useful and their goals to be more meaningful than short-term app users. This observation was complicated by the fact that patient health-related goals were dynamic and changed over time.
Complex patients’ use behavior of a goal-supporting mHealth app is shaped by an array of sociobehavioral factors that can evolve. To tackle this dynamism, there should be an emphasis on creating adaptable health technologies that are easily customizable by patients and able to respond to their changing contexts and needs.
ClinicalTrials.gov NCT02917954; https://clinicaltrials.gov/ct2/show/NCT02917954
Mobile health (mHealth) apps are being increasingly used to deliver care and support patients with chronic conditions [
The positive benefits of using apps and web-based platforms to support complex patients is documented in the current literature [
Attrition has been considered a major challenge in mHealth-based interventions [
The electronic Patient-Reported Outcome (ePRO) tool is both an mHealth app and a portal that enables goal-oriented care delivery by facilitating goal creation and monitoring by patients with complex chronic conditions working in collaboration with an interdisciplinary primary care team [
In a usability study of the ePRO trial, it was found that the app experienced gradual attrition of participants despite the tool scoring moderate usability [
We conducted a descriptive qualitative substudy drawing on patients’ interview data collected as part of a larger 15-month multisite pragmatic stepped-wedge trial of the ePRO tool [
A 2-stage sampling strategy was used to recruit participants for the study. First, we recruited family health teams (FHTs), and then we recruited patients within each FHT. FHTs are designed to provide integrated, multidisciplinary primary care and are typically led by physicians or nurse practitioners [
Quantitative data (surveys and chart audits) were collected from all 6 sites, whereas qualitative data were collected from 3 case sites [
The eligibility criteria for the recruited patients within the FHTs were being aged ≥60 years with ≥10 visits to the FHT within the previous 12 months. A total of ≥10 visits [
Using FHT electronic medical records, eligible patients were identified. The list of eligible patients was then given to FHT providers to assess whether the patients met the following additional criteria: (1) perceived willingness to engage in a conversation about goals of care, (2) ability to use a smartphone or tablet in English or having a caregiver who could do this on their behalf, (3) capability to provide consent to participate, and (4) willingness to complete surveys every 3 months thereafter until the trial concluded. Eligible patients were approached by their FHT staff (ie, care coordinators and administrators) and asked if they would be willing to speak to a research team member about the project. Recruitment occurred during a scheduled office visit or by phone. A detailed description of the recruitment procedure has been provided elsewhere [
Patients’ demographic information was collected through a survey at the beginning of the study. The first set of interviews was conducted at the midpoint of the trial, 4 to 6 months after the patients started using the app (the timing of the interviews depended on whether they were in the 12- or 9-month use group). The second round of interviews was conducted at the end of the trial. The purpose of the 2 sets of semistructured interviews was to explore patients’ overall experience with the ePRO intervention and how that experience changed over time. The semistructured interview guide addressed the following topics: (1) perception and experience of using the ePRO app, (2) patients’ relationship with their care team, (3) perception and experience of setting goals through ePRO, and (4) impact of ePRO on patients’ daily lives. Following the first set of interviews, the semistructured interview guide of the study was modified for the second set of interviews. Findings from the first set of interviews guided the iteration process for the semistructured interview guide and were decided by the research team members (FT [research coordinator], TA [graduate research assistant], JS [research coordinator], and CSG [research scientist and principal investigator with extensive qualitative research experience]).
The interviews were 25 to 40 minutes long and were conducted by 1 of 4 research team members (FT, TA, JS, and CSG). Each interview was audiotaped and transcribed using a commercial transcription service. Transcripts were checked for accuracy against recordings by a member of the research team.
Ethics approval was received from the University of Toronto Health Sciences Research Ethics Board (33944) and the research ethics boards of the 3 participating primary care practices. All patient participants provided informed verbal and written consent before initiation of study activities.
Multiple theories and frameworks have been used to explore the relationship between patients’ social-behavioral factors and mHealth or eHealth use [
This constitutes the dynamic and reciprocal interaction of person (individual with a set of learned experiences), environment (external social context, technology, and aids), and behavior (responses to stimuli to achieve goals). In SCT, these components—behavior, environment, and individual—are seen as acting bidirectionally.
This constitutes a person’s actual ability to perform a behavior through essential knowledge and skills.
This constitutes the level of a person’s confidence in their ability to successfully perform a behavior.
The internal and external responses to a person’s behavior affect the likelihood of continuing or discontinuing the behavior.
This constitutes the anticipated consequences of a person’s behavior. Outcome expectancies can be health-related or not health-related.
This theory was used to guide data analysis to explore how complex patients’ personal beliefs and attitudes and physical and social environmental factors affected their engagement pattern (long-term and short-term app use) with ePRO. Although SCT can be used as an explanatory framework, it was applied in this study to help categorize factors influencing use and relate those to engagement patterns. During the interview debrief sessions, memoing activities, and initial reading of the transcripts, the authors (TA, FT, and CSG) agreed that SCT demonstrated a fit with the interview data. As we chose SCT as the right analytical tool based on emerging interview data, we did not encounter the challenge of forcing data into categories.
A combination of 2 techniques was used to analyze the study data. In stage 1, the transcripts were inductively coded by 2 analysts (FT and TA). During the analysis, the research team met to discuss the identified codes and resolve any coding discrepancies. After coding 4 transcripts, the team decided that the coding scheme was appropriate. We reached data saturation after coding 12 transcripts. Data saturation was determined when no new codes emerged from the transcripts [
The first stage allowed us to see the social and behavioral factors related to use. However, to see how these factors related to each other and changed over time, we engaged in the second analysis stage of restorying.
Restorying is defined as the method of rewriting participants’ oral data temporally to draw a link between previous experience and subsequent experiences [
To restory patient data, 2 analysts (FT and TA) constructed a matrix of themes that distinguished between long-term and short-term app users (Table S1 in
To enhance the rigor of this study, the researchers undertook several strategies to increase the credibility and trustworthiness of the findings [
On the basis of patients’ app-automated use logs, patients were classified into 2 categories: long-term users and short-term users. Of the 22 interviewed individuals, 9 (41%) were short-term users and 13 (59%) were long-term users. Participants who did not use the ePRO app after initial onboarding or used it for <3 months were categorized into the “short-term user” group. By contrast, the participants who used the ePRO app for >3 months were categorized into the “long-term user” group. The 3-month cutoff period was determined because the app experienced a sharp decline in use at 3 months [
There were 44 study participants in the larger pragmatic trial, with 37 (84%) from the 3 case sites. Of the 37 patients who were invited to participate in the interviews, in total, 22 (59%) were interviewed. Of the 22 interviewed patients, 17 (77%) participated in both interviews, 3 (14%) participated in only the midpoint interviews, and 2 (9%) participated in the last interview only. A total of 41% (15/37) of the participants did not take part in the interviews because of scheduling issues, illness, being out of the country when the interview was scheduled, or not responding to interview requests.
The demographic information of the study participants can be found in
Demographic characteristics (N=37).
Variable | Interviewed participants (n=22) | Noninterviewed participants (n=15) | |
Age (years), mean (SD) | 75.1 (5.6) | 71.14 (6.5) | |
Sex—female, n (%) | 10 (45) | 5 (33) | |
Smartphone comfort level score, mean (SD)a | 2.17 (1.4) | 3.64 (1.4) | |
Number of chronic conditions, mean (SD) | 4.88 (2.1) | 3.07 (1.8) | |
|
|||
|
CAD $0 to $29,000 (US $0-$21,310.40) | 1 (5) | 4 (27) |
|
CAD $30,000 to $59,000 (US $22,045.30-$43,355.70) | 7 (32) | 4 (27) |
|
CAD $60,000 to $89,000 (US $44,090.60-$65,401) | 3 (14) | 3 (20) |
|
>CAD $90,000 (US $66,135.90) | 4 (18) | 3 (20) |
|
|||
|
Lower than high school | 2 (9) | 2 (13) |
|
High school | 2 (9) | 4 (27) |
|
Some college or university | 4 (18) | 3 (20) |
|
University (undergraduate or graduate) | 4 (18) | 5 (33) |
aThe range of the smartphone comfort level score is 1 to 5. A higher score indicates a higher comfort level with a smartphone.
The patient interviews revealed insights into the factors that influenced patients’ decision to continue or discontinue app use. When discussing their use of the ePRO app, patients identified what encouraged them to use the app, including factors relating to their social and clinical relationships, capability to use the app and perform goal-related activities, and their expected outcomes from the ePRO app.
Description of the themes.
Category and subcategory | Exemplary quotes | ||
|
Long-term user | Short-term user | |
Reciprocal determinisma | “They [care team] always know what to do with me, so there was no problem setting goals because they know that I am trying to be active and healthy. and I kept using it (ePRO app) daily because I know they (care team) are watching my data.” [Female, patient 18] | “I just did not know if anyone is looking at my data, there was no communication from you guys [research team] or my nurse or doctor here. There was no feedback for me about my data, so I felt like I am talking to the void when I was putting my information in. I would like to know if I was doing well or not. It would be helpful to talk to others (peers) about our goals, to see who else is doing the same thing as me and how they are feeling.” [Female, patient 16] | |
|
|||
|
Subtheme 2a—confidence and skills in goals | “When my dietician first asked what goal I wanted to set, I knew it would be tracking my everyday walk, I knew it would be easy to keep up at because I have been doing this for long time. But ePRO made me more accountable, I wanted that accountability. I liked how the device asked me if I have achieved my goal for that day. Clicking yes to that felt good and I kept doing that.” [Male, patient 7] | “Setting any goal was hard for me because my conditions flare up here and there and throws me off my routine. So I wasn’t sure how well I can keep up with the goals...I sprained my ankle in last winter so then I was off my walking for 5 weeks. Considering all these troubles, I didn’t work on my goals, and the app became redundant because what would I track. When the app asked Did I achieve my goal for the day, I did not want to keep saying no.” [Male, patient 2] |
|
Subtheme 2b—confidence and skills in technology | “I expected the app to have some direction for me about how I was doing on my goals, it was nice to see what I was accomplishing weekly basis. No complaints about the app, very easy to use...nothing complicated that anyone will have difficulty with...But I have used computer all my life for work so using this phone or any other phone is not a problem.” [Female, patient 3] | “The small fonts or buttons in this phone [ePRO] was trouble...but I thought I will get used to it (the phone) but did not at the end. I was sometimes working on my goals but could not record it on the phone, so I lost interest in the phone...then I forgot about my goals too because I was not tracking it or doing anything about it.” [Male, patient 21] |
|
Subtheme 2c—outcome expectancies | “The main reason I enrolled-I wanted to stay on track of my goals and feel healthier over time-I thought the app was helpful to keep me on track.” [Male, patient 1] | “When my doctor suggested this app, I did not know what to expect because there is nothing important, I need to work on, in my opinion anyway. My doctor suggested some goals but nothing very important...I could not make a purpose of it (ePRO).” [Male, patient 11] |
Use reinforcementc | “I was bedridden so [provider’s name] she was ‘gung-ho’ that I join her walking group for my recovery. And she said, “why don’t you try this new thing we are doing, this will be good for you?”. And She was right, it was nice to have the app because I know every Monday, I will have to say how many times I walked last week, so I tried to go out over weekends...She was there for me throughout, walking alongside me in every walking group.” [Female, patient 6] | “My doctor did not think ePRO was helping me that much, because both of us thought I am doing fine without it, everything [diabetic symptoms] was on track, so we decided maybe I do not need it.” [Male, patient 17] |
aThis domain refers to the dynamic relationship between individual, context, and behavior.
bThis domain refers to individuals’ confidence and skills in achieving their goals in the electronic Patient-Reported Outcome app and the perceived usefulness of the app.
cThis domain refers to the internal or external responses that encourage or discourage behavior change.
Elaine considers herself to be a healthy individual whose diabetes symptoms are well managed through diet and exercise. She thinks of herself as “lucky” to have great health care providers who have helped her manage her symptoms for the past 2 and a half years. She has multiple other chronic conditions such as chronic pain and hypertension, but controlling diabetes symptoms is her foremost priority as she heard it can affect her other conditions. At first, she joined the electronic Patient-Reported Outcome (ePRO) study because her dietician at the family health team encouraged her to do so (
Elaine considers herself technologically savvy. However, she experienced a few technological difficulties while using ePRO. The most challenging one was being logged out of ePRO after taking a break from the tool during Christmas time when she visited her family in Scotland for 15 days. After not using ePRO while she was away, Elaine was locked out of the app. After returning from her holidays, she contacted her dietician to resolve the issue (
Josh is a man aged 76 years with several concurrent chronic conditions, including diabetes, hypertension, and arthritis. Josh considers himself to have a fair understanding of his ailments and considers that his conditions are fairly well managed. Josh is the primary caregiver to his wife, who is ill. As a result of this caregiving role, Josh finds that he does not often have time to participate in social groups such as walking groups offered through his local community center (
In addition, Josh was hesitant to set a goal as he had never had a health-related goal before and was uncertain about whether he had the necessary skills or discipline to keep up with a specific goal (
At the beginning of the study, Josh completed his check-in questions regularly. Over time, Josh began checking in on the app less and less, eventually not using the technology at all. When the ePRO study team reached out to Josh, he stated that he forgot his password and was unable to log in to the ePRO platform, so he did not use it. Although Josh describes himself as “computer illiterate,” he found the ePRO app and web platform easy to use. Josh also found that, whenever he met with his health care provider, they did not discuss his goals but rather spoke about his medications and management of his conditions, resulting in goal setting becoming less of a priority (
A major decision that was made during the analysis was to collapse 3 SCT domains—goal efficacy, behavior capability, and outcome expectancies—into one as it was identified that patients’ confidence in their goal and technological skills was linked to the anticipated outcome of the ePRO app. Previous studies on goal-setting behavior have also identified that, in a real-world setting, individuals’ confidence in health-related goals is confounded by their outcome expectancies, capability and skill level for carrying out various goals and activities, and technological and health literacy [
In this section, we elaborate on the themes identified in the data according to the SCT domains. Some domains had richer information than others. For example, the themes related to reciprocal determinism, goal efficacy, and outcome expectancies had more nuanced data compared with the other 2 themes, which were behavior capability and use encouragement.
Reciprocal determinism focuses on the dynamic interaction between person-context behavior and the influence of this dynamic interaction on individuals’ behavior. As demonstrated in the long- and short-term user narratives as well as in
In total, 46% (6/13) of long-term users described their longstanding relationship with their primary care providers as being beneficial to setting meaningful goals:
I got lucked out with my providers, they will always know exactly how to deal with me and keep me out of the hospital, which is my main goal. My doctor knows that my nurse and dietitian here (primary care team) know that, so it was easy to set those goals to keep my blood sugar low.
Short-term users also described a good relationship with their providers. However, 44% (4/9) of short-term users described that their providers did not discuss the ePRO app during their clinic visits. Participants listed the following reasons for not discussing the ePRO app with their providers: clinicians’ heavy workload, not having enough time during the visit, feeling that it was unnatural to discuss the app during a regular clinic visit, and feeling that their goals were personal work and did not fall under providers’ responsibility. A participant described the following:
Dr. [physician’s name] is great, but he is really busy, so I did not want to waste his time talking about my walking schedule. He needs to check my blood pressure level; I would not bring up how many times I walked last month. Feels irrelevant for him to know that.
Another way the patient-provider relationship influenced app use was when patients faced any sort of technical error in using the app or had to modify their goals after the initial goal-setting process. Specifically, long-term users were more likely to reach out for support and tended to report more instances of connection with their providers regarding the ePRO app. Some of the common technical challenges were (1) being logged out of the app because of prolonged inactivity, (2) forgetting passwords, and (3) inability to modify goals based on patients’ needs. In terms of modifying goals, ePRO did not allow patients to modify their own goals, so primary care providers had to modify the goals for them. Therefore, when patients needed to modify their goals, they were uncertain about how to do that:
After they (government) changed the number of blood glucose tests I can do per week, my goal had to be changed because I wanted to test my glucose level daily but after they changed it, now I only test twice a week, but I still it report it on the phone just not daily. And my nurse over here changed it (frequency of reporting) for me.
When faced with these technical difficulties or needed modifications, patients either abandoned the ePRO app or reached out to their health care providers or research team to solve the issue. Most long-term users (7/13, 54%) chose the latter option:
I was locked out of the app when I was on vacation...after I got back, I contacted the dietician over here (care team), and she connected me to you guys. Everything got resolved within 2 days, I kept using it.
Short-term users, by contrast, decided to abandon the app and did not reach out for support when they faced similar technical difficulties:
It would be good if I could change my goals in the app because walking 5 km is what I set out to do at the beginning. It was too ambitious of a goal in this bad winter. I never reached 5 km, so I never had anything to report on the app...I did not reach out to my nurse practitioner, I guess I forgot about it (ePRO) for a while, and then I asked you (research team) to take it away.
Both long- and short-term users also reflected on the fact that their relationships with peers and their communities could influence their app use behavior. For example, a patient discussed that being able to communicate with their peers would be useful in understanding others’ experiences with the ePRO app:
Sometimes I felt that the app does not give me enough feedback. There could be more photos, a thumbs up if I did well. I’m a unique person so when I found I felt that way I thought, well I wonder if anyone else is feeling that way. So, communicating with other people that are using it without divulging your specific things would be nice.
Importantly, unexpected changes in these relational contexts also influenced patients’ use behavior, for example, a sudden transition to a caregiving role, a move away from social ties, or a divorce:
After my marriage fell apart, I moved to this area with my partner and I have to keep going back to the city to meet my friends, which makes it harder for me to meet people here. I am currently in an anxiety support group here, but I went off track with my other goals. I check the app (ePRO) sometimes but not regularly because I have nothing to report on.
Patients’ confidence, skills, and anticipated outcomes from the app influenced their use behavior. Although presented as distinct domains in SCT, data from this study suggest that the domains of goal efficacy, behavior capability, and outcome expectancies are linked.
The restorying work reveals these connections, which are best represented in the long- and short-term user narratives in
This subtheme demonstrates patients’ descriptions of how their confidence in their goals and their skills to achieve the goals influenced their ePRO use behavior. Previous goal-setting experience and familiarity with goal-related tasks influenced patients’ confidence in achieving the goals set in the ePRO app. Patients who had been working on a goal for a long time were more confident in their skills to achieve a goal. A total of 38% (5/13) of long-term users had already been working on a number of health-related goals before enrolling in the study and had been tracking their progress using electronic or paper-based tools such as calendars, wearable technologies, and handwritten notes. For these participants, the ePRO app was an additional electronic way to track their goals. These participants demonstrated confidence that they had the necessary skills to set appropriate goals and achieve them with the use of ePRO and, because they had the confidence and skills, they also had better outcome expectancy from the ePRO app:
I did pretty well in terms of crushing all my goals...because I already had the same goals, I was already continuing with the exercise program. So, it (ePRO goals) was just a continuation. I just kept up with the same tasks, swimming, walking that I was doing before joining your study.
By contrast, patients who did not have any previous goal-setting experience reflected on the fact that setting a meaningful goal was difficult for them. Consequently, their providers had to suggest some goals for them, but some patients found that those goals were not personally meaningful. In these cases, not having previous goal-setting experience negatively affected patients’ ability to set meaningful goals, which in turn affected their use behavior:
I’ve never had health goals before, so could not come up with one when they (health provider) asked me what I want to put in here (ePRO app). I got some kidney conditions, so my doctor suggested I set daily goals of drinking eight glasses of water and tracking them. I did not think I need to track it; I remember it anyway. I don’t need a phone to tell me I need to hydrate. I did not think the goal was anything important for me to track on a phone.
In terms of individuals’ confidence in achieving their goals, some long-term users (6/13, 46%) indicated that their traits, such as “will-power,” “self-discipline,” and “motivation,” boosted their confidence that they would be able to reach their goals:
It [achieving health goals] has nothing to do with the phone [ePRO app]. It has everything to do with the person. You have to be determined that you are going to walk. And you’re going to set your goal—you’re going to walk a block and you’re going to walk back. You have to have determination. You have to have the willpower to say, I’m going to do it and that’s it. ePRO is not going to do it for you, but it was good to have to see my progress. I thought it (ePRO) was a neat way to see how I am doing.
In addition, patients reflected on the fact that their confidence and skills in achieving a goal changed over time depending on their health. When patients felt that they were not able to achieve their goals because of health and life circumstances and they did not have “enough” to report on the app, they discontinued using it:
Initially, I set up my goal to go 3 miles walking every day. But after my surgeries and my accident, there was no way I could do it. I was barely getting out to walk my dogs. I was falling short every day and it made no sense for me to use the app, I just felt sad that it [ePRO] kept showing me I was not the go-getter anymore. I did not know how to pause it [ePRO].
Not surprisingly, patients who did not think that they had the necessary technological skills to use the ePRO app discontinued their use.
Several patients (14/22, 64%) discussed that they were technologically savvy enough to be able to use the app:
I found the app to be user-friendly, very clean, nothing too difficult, but I am good with computers and all that stuff, a tech-junkie. I use computers, phones, iPad all the time.
Some participants (4/22, 18%) stated that they needed help using the ePRO app as often the fonts were too small:
I never had to use the computer for my work so never learned it. Now I got muscular dystrophy, so the fonts were way too small for me, so I did not use the app at all. I used the app [ePRO] on my computer, but I am not very good at it. My wife must help me a lot. I cannot even send an email; she will just do it for me. I ended up not using it [ePRO on the computer] at all.
Patients described their anticipated outcomes from the ePRO app. Typically, for long-term users, ePRO seemed like a beneficial addition to their health. A long-term user described that, while enrolling in the study, they anticipated that ePRO would make them more accountable toward their goals:
I wanted to get off my oxygen tank, I do not want to lug this machine everywhere. So I need to drop some pounds...by walking, exercising...I thought this phone would show me how I am doing, am I doing it too much, am I getting any good.
By contrast, 33% (3/9) of the patients who were short-term users described that they discontinued using the app as they did not think that the app was “well-developed” to be implemented in the real world. Therefore, they did not think that the app would be a beneficial addition to their lives. A short-term user described their dissatisfaction with the functionality of the app:
I think that’s all [research on people taking control over their health] a great idea I just feel that the actual implementation isn’t as far advanced as it needs to be for it to work effectively, at least for me. I use my fitbit anyways to count my steps which is far better because that watch automatically counts my steps. I could not see any use for it [ePRO app] to work on my goals. I did not see any benefit for my health from it.
The use reinforcement domain of SCT suggests that internal and external factors such as internal satisfaction or external rewards can encourage or discourage individuals’ behavior change. In total, 38% (5/13) of long-term users reported that they felt a sense of accomplishment (ie, internal reward) when they were able to “check off” their goals in the ePRO app. The app had the following question—“did you achieve your goal yesterday?”—and patients had the option of reporting yes or no. Some patients (6/13, 46%) found this exercise rewarding:
Well, to be honest, the only thing it did was—I do it [check off the list], used to do it every Monday morning, and it focused me on not smoking. That was the motivation every Monday morning, you know.
Some short-term users (2/9, 22%) identified that they had already used many other legacy devices such as calendars, notebooks, cell phones, and glucose monitoring devices. These participants found reporting the same measures in 2 different tools to be redundant, and they did not think of the ePRO app as an important addition to their health-related goals:
I am an old school paper-pencil, calendar on refrigerator person, so that helps me to visualize my progress every day. I see them every day before breakfast, so I know what I had to do that day. The phone [ePRO] just stayed on my night table.
An unexpected external influence can be discouragement from providers. Among the 9 short-term users, 22% (2/9) of the participants reported receiving advice from their providers to discontinue the use of ePRO. The factors that contributed to providers’ discouragement were patients’ frail health, patients’ anxiety with the app regarding not being able to reach their goals, and changed health-related priorities:
My breathing issue has gotten worse in winter so I was not working on my goals anymore...When I told her [health provider] that I am worried about not reaching my goal, I feel anxious that I am not reaching my goal, she said “just forget about it [ePRO] for now, let’s get back you to feeling good first,” so I thought okay one thing off my list. I felt better.
The 2 narratives presented in
This study used descriptive qualitative methods and restorying analytic techniques to explore the social and behavioral factors contributing to patients’ use behavior of the ePRO tool. Study findings show that patient-provider relationships, patients’ social relationships, and patients’ personal circumstances play a central role in their decision to continue or discontinue the use of the ePRO app.
Leveraging SCT as a tool for data analysis, we were able to identify social-behavioral factors that contribute to patients’ decision to continue or discontinue app use, such as their social and environmental factors and relationships (domain 1); confidence and skills in using technology, confidence and skills in setting and achieving goals, and expected outcomes from the intervention (domain 2); and encouraging factors (domain 3). Study data reveal that the SCT constructs of goal efficacy and behavior capability are also importantly related as capability and skill influence perceived confidence in completing a task. This interrelationship makes sense theoretically. SCT suggests that performing a behavior successfully increases individuals’ confidence in their ability to accomplish goals as they believe that they have the skills to achieve goals through behavior change [
The stories show the themes of the interactions and links between concepts that the descriptive analysis could not. For example, an important interpretive theme that emerges from Josh and Elaine’s stories is that patients’ confidence and previous experience in goal setting influenced their capability and expectations from this goal-oriented intervention. Josh and Elaine approached their goals with varying degrees of experience, confidence, and attachment. For example, Elaine’s previous experience with goal setting helped her feel more competent and skilled in achieving future goals, which subsequently increased her intention to track goals through ePRO, whereas Josh’s lack of experience with goal setting made it challenging for him to make meaning of his goal, which translated into his reduced interest in tracking goals through ePRO.
Furthermore, the stories also show, in an interpretive manner, an important divergence in how long- and short-term users react to technical errors. App-related technical errors are ubiquitous, and many app-based interventions experience significantly high attrition after users experience an error [
Another important study finding that emerged from the interview data is the importance of meaningful goal setting for an effective behavior change intervention. Hence, when setting patients’ goals, a strong focus on patients’ perception of the meaningfulness or fit of the goal in their daily lives should be accounted for as this meaningfulness of the goal can influence not only behavior change but also patients’ adherence to a newly adopted technology [
The findings of this study support previous study findings that health technologies are often discontinued and abandoned when they lack features of meaningful customization and is not part of users’ already existing devices such as personal phones [
For example, previous studies with shorter follow-up periods have identified that factors such as health literacy, motivation, capabilities, social and environmental structures, and social support have an impact on mHealth engagement [
In the current chronic care paradigm, the task of goal management is often left to the patients [
The descriptive qualitative approach of this research allowed us to identify multiple social-behavioral factors that influenced patients’ enrollment in the study and subsequent discontinuation or continuation. In addition, by using a restorying method, the findings were interpretive, allowing for the identification of nuanced patterns and interrelationships between identified themes. Furthermore, the longitudinal timeline of the study (15 months) allowed us to explore the factors that contribute to patients’ use behavior in the long term, which is underexplored in the current literature [
Owing to scheduling conflicts or loss to follow-up of participants, we were not able to interview all of them at either time point. As a result, a potential limitation of the study is that those who participated in the interviews may be unique as compared with those who chose not to. However, the sample size was too small to assess whether the difference between the 2 groups was significant. However, the interviews that were conducted were in-depth and provided rich information. Furthermore, the patient population represented in this study was recruited from only 3 of the 6 FHTs involved in this study. It is possible that some additional findings may have been obtained by looking across all 6 sites. However, the sample in this study represented 59% (22/37) of the total participants in the larger study. As is the case with case study research, it is also possible that findings may not be transferable to other models of primary care such as community health centers or solo practice environments. Furthermore, the participant demography suggests that the study patient population was less complex and well resourced, meaning that, on average, patients had a low number of chronic conditions and high income and educational attainment levels, which might not be representative of general complex patients. Therefore, the findings of this study may not be transferable to patients living in resource-poor communities or who have lower income or education levels. In addition, the underrepresentation of low-income individuals is a common occurrence across multiple research studies and requires attention in study design to facilitate this population’s participation [
In many cases, mHealth or any health innovation will have expected impacts if people use it as intended. To better predict, explain, and increase the actual use of innovations, we need to understand why different target user groups continue or discontinue the use of an innovation. This study identifies that multilevel factors contribute to complex patients’ decision to continue or discontinue using a goal-oriented app. In addition, our findings show that there is a need for ongoing, productive patient-provider interactions to set and modify patients’ goals according to their changing health and social needs. Future research should consider patients’ social and behavioral contexts when implementing mHealth apps and similar technological interventions for complex patients.
Additional information about data analysis.
CONSORT-EHEALTH checklist (V 1.6.1).
electronic Patient-Reported Outcome
Family Health Team
mobile health
Social Cognitive Theory
This study was funded by the Canadian Institute for Health Research eHealth Innovation Partnerships Program (CIHR–143559). The funders did not have any involvement in the study design and execution, analyses and interpretation of the data, and review or approval of the manuscript for publication.
FT contributed to data collection, writing of the first draft, reviewing, and editing. TA contributed to data collection and analysis, writing of the first draft, reviewing, and editing. BM, ML, SWM, and RU contributed to writing (critical review) and editing. CSG conceived and designed the study and contributed to data collection, writing of the initial draft, reviewing and editing, and supervision.
BM holds a paid consultancy with the Scottish Government to provide advice on remote patient monitoring. However, BM has no ownership stake in the electronic Patient-Reported Outcome app. Therefore, we do not foresee any conflict of interest.