This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on http://humanfactors.jmir.org, as well as this copyright and license information must be included.
Translating research into practice, especially the implementation of digital health technologies in routine care, is increasingly important. Yet, there are few studies examining the challenges of implementing patient-facing digital technologies in health care settings.
The aim of this study was to report challenges experienced when implementing mobile apps for patients to support their postsurgical rehabilitation in an orthopedic setting.
A mobile app was tailored to the needs of patients undergoing rotator cuff repair. A 30-min usability session and a 12-week feasibility study were conducted with patients to evaluate the app in routine care. Implementation records (observation reports, issues log, and email correspondence) explored factors that hindered or facilitated patient acceptance. Interviews with clinicians explored factors that influenced app integration in routine care.
Participant completion was low (47%, 9/19). Factors that affected patient acceptance included digital literacy, health status, information technology (IT) infrastructure at home, privacy concerns, time limitations, the role of a caregiver, inconsistencies in instruction received from clinicians and the app, and app advice not reflective of patient progress over time. Factors that negatively influenced app integration in routine care included competing demands among clinicians, IT infrastructure in health care settings, identifying the right time to introduce the app to patients, user interface complexity for older patients, lack of coordination among multidisciplinary clinicians, and technical issues with app installation.
Three insights were identified for mobile app implementation in routine care: (1) apps for patients need to reflect their journey over time and in particular, postoperative apps ought to be introduced as part of preoperative care with opportunities for patients to learn and adopt the app during their postoperative journey; (2) strategies to address digital literacy issues among patients and clinicians are essential; and (3) impact of the app on patient outcomes and clinician workflow needs to be communicated, monitored, and reviewed. Lastly, digital health interventions should supplement but not replace patient interaction with clinicians.
The use of health and wellness applications have grown rapidly over the past years [
To date, the majority of literature focuses on implementing clinician-facing digital technologies in health care settings [
The shoulder joint is a ball-and-socket joint between the scapula (socket) and the humerus (ball). The rotator cuff is a group of four tendons that connect the muscles attached to the scapula to the humerus. The function of the rotator cuff is to rotate the ball in the socket and therefore, move the arm.
Tears of the rotator cuff are a common cause of shoulder pain and upper limb weakness [
To achieve the best results from surgery, patients should adhere to a strict postoperative rehabilitation protocol to prevent retearing of the tendon and regain maximum shoulder function (
To improve patient adherence with the rehabilitation protocol, a mobile app using the Healthy.me platform was developed for patient use. Full details of the Healthy.me platform are described elsewhere [
The aim of this feasibility study is to examine factors that facilitate or hinder the implementation of a patient-facing app in routine clinical care following rotator cuff surgery.
A 30-min usability session and a 12-week study were conducted with patients undergoing rotator cuff repair surgery to evaluate the usability, feasibility, and acceptance of the app to support the patient’s postoperative rehabilitation. A mixed-methods approach was used to incorporate the collection of quantitative app usage data, qualitative data through feedback from patients and clinicians, as well as implementation records taken by researchers during the study.
This evaluation was performed on patients attending for surgical treatment at MUH, Sydney, New South Wales (Australia), which is a private teaching and tertiary referral hospital. Personnel involved in the study included orthopedic surgeons, practice and ward nurses, health informaticians, software engineers, and research and administrative staff. Ethics approval was obtained from the Macquarie University Human Research Ethics Committee.
Patients booked for rotator cuff repair at MUH were eligible to participate in this study if they were in the age range of 40 to 65 years, English-speaking, in possession of an Internet-enabled iPhone or Android mobile phone, and intended to undergo surgical treatment for rotator cuff repair.
All participants received standard postoperative care. Participants were required to complete a 10- to 15-min questionnaire at their first preoperative visit and at their routine visits 2 weeks, 6 weeks, and 12 weeks postoperatively. Participants could comment further on a voluntary basis via email or telephone during the 12-week study period.
Eligible patients suitable for the study were initially recruited by the surgeon, practice nurse, or a research team member during the patients’ preoperative consultation at the orthopedic clinic. Patients could also be recruited by the ward nurse during their recovery in the ward after surgery.
Patients suitable for the study were initially recruited by the surgeon, practice nurse, or a research team member during the patients’ preoperative consultation at the orthopedic clinic. Patients could also be recruited by the ward nurse during their recovery in the ward after surgery.
Participants provided written informed consent. They were advised that they could cease app use at any time and return to standard care involving regular outpatient clinic visits. They were also given an email address and a mobile phone number to a research team member for queries, issues, or comments during the 12-week period.
A steering group with 5 representatives from orthopedics, consumer informatics, and software development was formed and met over 3 months for 2 hours every fortnight to codesign the app, formulate the study design, and compose educational content for patients. An internal usability study with 10 individuals was conducted with all major usability issues addressed before patient recruitment. Three meetings were also held with all clinicians involved to refine ways to introduce the app to patients before study commencement.
A mobile app was developed that contains information on the postoperative rehabilitation program. It contains (1) postoperative rehabilitation exercise videos; (2) important information and restrictions at different stages of the recovery; (3) contact information of the surgeon, practice nurse, and the research team; and (4) a
To improve adherence with the rehabilitation protocol, patients were encouraged to complete a 3-min questionnaire daily within the app (
Whereas the app is not available for public use, the app was installed on the mobile phones of participants in this feasibility study using the TestFlight platform on iPhone operating system (iOS, Apple Inc) devices. Participants with an Android device were provided with an URL, where participants could download the app directly from Google Play Store, following email validation by the research team.
Patient app usage was assessed using a system log that recorded time of app access, app features used, and the duration and frequency of use for each app feature. Descriptive statistics were computed for the usage data.
The Western Ontario Rotator Cuff Index is a standardized and validated questionnaire that quantifies pain, analgesic usage, and quality of life specific to rotator cuff disease [
Implementation records (ie, field notes recorded by researchers that detail the implementation process of the app) were collected during the study. These include participant observation reports, issues log, and email correspondence with participants. For participants recruited by the research team, observation reports were made in accordance to a predefined template (
Home page of the app developed to support patient postoperative rehabilitation after rotator cuff repair.
At the completion of the 12-week study, a member independent of the app development team conducted a semistructured interview with the clinicians involved to explore key issues that emerged from the implementation records. An interview schedule was developed in consultation with the research team (
Before clinician interviews, a series of cross-functional flowcharts showing a workflow were drawn for patients requiring rotator cuff repair at MUH, representing the interactions between four main settings: the patient’s home, general practitioner (GP), orthopedic clinic, and hospital (including ward nurse and physiotherapist). Standard flowchart symbols were used [
For the qualitative data analysis, audiotapes were transcribed verbatim. Two members (AL and VL) read through all interview transcripts and implementation records independently, following the constant comparative method and thematic analysis [
Participant completion rate was low (47%, 9/19). Participants were in the age range of 42 to 67 years (mean=55.4 years, standard deviation [SD]=8.6 years). One participant (aged 67 years) was above the age eligibility criteria but was included in the feasibility study because of his enthusiasm to participate in the study. Patient demographic data are shown in
Eight out of 9 participants installed the mobile app. System log showed that the mean duration of app usage was 46.9 days (SD=24.5 days, median=42.5 days, interquartile range [IQR]=29 days). A total of seven app features were monitored, namely, home page, daily questionnaire, exercise videos, rehabilitation information, surgery details, contact information, and pillbox. Access frequencies for each feature are outlined in
Patient workflow for a general practitioner (GP) referral to an orthopedic surgeon.
Patient workflow for the preoperative procedures required before surgery.
Patient workflow during a rotator cuff repair surgical procedure.
Patient workflow for postoperative reviews with an orthopaedic surgeon.
Patient feedback was obtained from three sources: observation reports at the preoperative meeting, issues log, and email correspondence collected during the study. Issues reported by patients or recorded by researchers are detailed in
Patient perspectives on the role of the app in their postoperative rehabilitation are illustrated below.
Questionnaires regarding rotator cuff recovery need to more accurately reflect the patient’s journey:
I felt that if the questions changed to reflect the recovery milestones I had reached, then the app would be more relevant.
I thought the questions were quite good for the first six weeks but needed to be changed for the second six weeks as they are less relevant and this affects the frequency of filling out the questions.
The inability of the app’s questionnaires to relate to more specific scenarios:
I think there is a fundamental question missing from this questionnaire, namely, is the operated arm dominant arm? This would greatly affect the answers to many of these questions.
The questions about pain might be more useful if they included a question about pain when exercising. I have very little pain when undertaking normal activities but during and after exercises I have a much higher level of pain.
Sleeping has also contributed some level of pain and stiffness as I have reverted from sleeping propped up on my back to sleeping on my non operated side (I use to always sleep on my operated side).
Clinician feedback was obtained from two sources: implementation records during the study and semistructured interview at poststudy. Issues reported by clinicians or recorded by researchers are reported in
Clinicians’ perspectives on factors that affected patient acceptance and app integration in routine care are outlined below. Some clinicians have also provided suggestions on remedies:
Lack of coordination among multidisciplinary clinicians about new changes from the app:
The rehab program we brought in for the app was an agreed standard program that I’m happy with but most physios elsewhere have got my old rehab schedule not the new rehab schedules. So it’s a matter of actively changing across the board places where they’re going to go and formalising the fact that we’ve changed our rehab program so I think it’s just a communication issue.
Deficient digital literacy experienced among patients:
If it’s the elderly, it’s negative [responses] because half of them don’t even know how to use the phone. The smartphone.
Firstly, the demographic, 40-65 has a higher proportion of “tech-ludites” There are people who use a smartphone but don’t use apps...Secondly, there are people who think they are good with apps and they were daunted...And there were a number of people who...once were shown the app, go...too many steps. If you come back and do the study in 20 years’ time, everyone will be okay with it.
Time investment involved through daily consultation with the app:
Some said that they were too busy, that they didn’t want to. That was probably the main [reason].
1. Population variation in digital health literacy
Patients not remembering iPhone operating system
Patients forgetting app password when attempting to log in.
Patients unfamiliar with Wi-Fi setup at home and expressed concern on how to access the app at home.
Patients with high digital health literacy did not express any difficulties with the app.
One patient expressed preference to install the app on their iPad, which was not on their person, resulted in a verbal account of instructions. That patient could complete installation successfully at home.
One patient expressed unfamiliarity with the app download process, which raised questions on whether the use of mobile apps for such patients is appropriate.
2. Patient state and their health status (eg, pain, fatigue, comorbidities, and their concerns)
Patients were disengaged during the usability study, possibly because of pain (confirmed by some patients verbally and observed through body language) or low digital health literacy, leading to unfamiliarity with app.
Patients’ preexisting orthopedic conditions or multiple injuries excluded their eligibility to use the app as the exercises were designed only for people with rotator cuff repair.
Patients were unaware of the extent of postoperative restrictions until informed by the app.
Patients seemed engaged throughout the session but started becoming impatient with the number of questionnaires.
One patient expressed concern regarding how his ability to work will be affected (as he operates machinery at work).
3. App needs to more accurately reflect patient recovery journey
Patients expressed that the app needs to be more reflective of patient recovery journey, such as questions changed to reflect recovery milestones, provision of advice for specific scenarios, and that some restrictions need to be relaxed in certain circumstances as patients progress over time.
4. Privacy concerns over data collection
Patients express concerns over privacy and sharing of information with institutions, such as insurance and workers’ compensation (eg, WorkCover).
5. Role of caregivers
Patients with low mobile phone literacy depend on caregivers to install the app. In one scenario, caregiver also did not display adequate digital literacy to comfortably install the app (eg, forgetting Apple ID and iOS password).
Patient reliance on caregiver leads to participant withdrawal from study as caregiver becomes unavailable to look after patient.
6. Time limitations
In-person meetings were carried out immediately after patients attended their first meeting with their surgeon, and some patients did not have the time to stay for an additional 20 to 30 min to meet with researchers about the study.
One recruited patient pulled out because of time investment required being too much at the usability study.
7. Credibility of app content
Patients asked if exercise information was approved by surgeons.
1. Lack of coordination among multidisciplinary clinicians
Surgeons have their own rehabilitation protocol. Designing this app was also a way to consolidate differences in approaches and attitudes among surgeons in standardizing a postoperative rehabilitation protocol.
However, some physiotherapists were not aware of the new rehabilitation protocol and thus, provided instructions to patients that were not consistent with those embedded in the app.
Similarly, one patient reported discrepancies between the app and clinician instructions at hospital discharge.
2. Deficient digital literacy skills among clinicians
Some clinicians noted that a self-perceived lack of confidence and skills in using new technologies may have hindered their willingness to introduce the app to patients.
3. Demanding workload and competing demands
Clinicians are often too busy (or have forgotten) to introduce the app to patients.
Introducing the app during the preoperative stages of a rotator cuff surgical patient:
I think the big issue would be adequate pre-op engagement and instruction.
Pre-operatively, because you need all that information pre-operatively. You need them to remember their pain levels and all that sort of thing and their discomfort. So it definitely would have to be started pre-operatively.
New app features to closely reflect patient recovery journey:
The trouble is that it runs for six weeks and the patients are doing the same exercises for six weeks and after two weeks, they know what they’re doing. Maybe we could look at… they get prompted every day for the first week or two but then for the next four weeks, maybe they get contacted less.
What I really think would be good is to have some sort of generated referral [reminder] when it’s time to have the scans done, I think that would be good...And maybe a booking system online with the app so then they’re given a date and they can choose the time that suits them so that it’s an automated [process].
Suggestions on ways to educate patients on how to use the app:
One thing that probably we didn’t think about in the app is a practice app. Something that says let’s pretend you’ve had something, let’s pretend you’re [at] day 5, these are the questions that you’ll be asked so that they get to play with it without it being recorded as part of their management but purely as their education.
When you download the app, download an instructional video that whenever in doubt, it can say this is what you got to do...
Other issues emerged regarding the organizational setting are reported in
1. Information technology infrastructure
Failure to connect to the local Wi-Fi at outpatient clinic or in hospital.
2. Timing of introducing the app to patients
Determining the optimum timing, persons of contact, and patient recruitment logistics across multiple health care settings is not simple.
The need to constantly remind clinicians to offer patients to partake in the study.
(For inpatients recruited postoperatively) Without knowing the app’s existence before surgery, patients may have the propensity to decline automatically to participate, impeding patient recruitment rate.
3. Training clinicians on digital literacy and informing on impact is important
Strategies to improve digital literacy and inform frontline clinicians on ways to integrate the app into routine care (and the subsequent impact on workflow and workload) are important.
1. Problems with app installation
Installation process for iPhone operating system
App installation depends on receiving an email, with patients not necessarily linking their email account to their device. (In addition, there is the inability to customize TestFlight emails or the email address they are sent from. These emails could be categorized as spam and may cause confusion as they originate from a name and email address unknown to the user).
2. Problems with app distribution
Issues with TestFlight hindering app distribution. For example, app expires on TestFlight after 60 days, where users must update the app. Users receive a notification detailing app expiration, which may have misled them to think that the study has ended.
Distribution issues may reflect poorly on app developers, despite this issue being unable to be resolved from the app developer team.
3. Pre- and postimplementation support
Troubleshooting of issues is more difficult to occur without an onsite technician.
Some recruited patients did not respond to initial emails or secondary follow-up emails.
Time spent providing technical support to and liaising with participants is significant, and appropriate resources should be allocated for this.
This feasibility study offers insights into the implementation of a mobile app for patients in an orthopedic setting. Our mixed-methods approach has identified factors that affected patient acceptance of the app ranging from patient-related factors (such as digital literacy and health status), contextual factors (eg, IT infrastructure at home, time limitations, and the role of their caregiver), personal concerns (eg, privacy over information sharing), to other factors (such as inconsistencies in instruction received from clinicians and the app, and app advice not reflective of patient progress over time). In parallel, factors that negatively influenced app integration in routine care include clinician-related factors (eg, competing demands, heavy workload, digital literacy, and lack of coordination among multidisciplinary team), organizational factors (eg, IT infrastructure in health care settings and optimal timing of introducing the app to patients), and issues related to the app (eg, user interface complexity for older patients and technical issues with app installation).
Whereas this study focused on rotator cuff repair, barriers experienced from patients, clinicians, and health care organizations (
The use of mobile health technology as an intervention to provide patient care is becoming increasingly common [
At the time of writing, we are only aware of the mobile app developed by Semple et al to monitor patient recovery at home after surgery [
Our study, however, only targeted patients from an older age bracket of 40 to 65 years and experienced a lower study completion rate (47%). It has been acknowledged that a reduced usage of technology by the older population can be attributed to a variety of reasons, including but not limited to a deficient understanding of the benefits of mobile apps to provide care, reluctance to gain digital literacy skills, and physical impairments leading to a lacking confidence in navigating through app features [
We identified three key challenges that impeded app uptake and integration during routine care and proposed suggestions to address them. These challenges may also be factors that contributed to the low participation or completion rate in this study.
Our findings indicated that advice from the app was not relevant to patients’ recovery journey on some occasions. Apps should provide advice that relates to specific patient scenarios, recovery milestones, and individual progress, intelligently adapting to the patient’s changing condition over time. They should also be designed and implemented with the patient in mind, considering that patients could be experiencing pain, fatigue, comorbidities, and concerns that could affect their decision to use (or not use) the app at any time point during their treatment journey.
Our study also found that identifying an optimal time to introduce the app to patients in routine care is not straightforward. Patients may not have been ready to participate when the study was first introduced at the preoperative stage. The lead up to surgery is often a challenging time for patients, as they are anxious regarding the surgery and often are making significant adjustments to their regular work and home routines to accommodate the time off required for the surgery. However, patients should still have an opportunity to learn about the app during the postoperative stage, even when they have declined the opportunity to use the app earlier in their treatment journey.
To encourage uptake, apps designed for postoperative recovery ought to be introduced as part of preoperative care, where there are opportunities for patients to learn and adopt the app during their postoperative journey. Starting the recruitment process earlier in the patient journey, as well as having multiple points for patients to learn (or remind them) about the app postoperatively, could potentially improve participant uptake.
Our findings revealed that some patients may not have the digital health literacy necessary to use the app as a form of postoperative care, although there were some exceptions. Digital literacy concerns among
Our study also found that digital literacy concerns among clinicians need to be addressed. There were occasions when clinicians did not feel confident enough to introduce the app to patients because of their self-perceived lack of familiarity with apps. For example, some clinicians felt uneasy helping patients set up the app or addressing any technical concerns during app installation. Strategies to improve digital literacy among frontline clinicians, to help them tackle common problems expressed by patients during app installation and usage, and ways to incorporate the app into their work routine are of utmost importance.
Overall, the level of time and resources required in providing digital literacy support to patients and clinicians can be intensive. A
Our study indicated that not all clinicians involved in patient care were aware of the app, resulting in inconsistencies in patient instructions received from some clinicians and the app. Communication with all clinicians involved is paramount. All parties need to be informed on how the app will impact on their clinic workflow, responsibilities, and patient communication. The overall impact of the app on patient outcomes and the flow-on effects on staff workload needs to be communicated, monitored, and reviewed. Ultimately, digital technologies should supplement and not replace patient interaction with clinicians.
Our study also found that clinicians were often too busy (or have forgotten) to introduce the app to patients because of their heavy workload and competing demands. The rotator cuff patient workflow is more complex than was originally anticipated (as indicated in
Several limitations need to be noted:
Participant completion rate (47%) was low, which limited study generalizability. However, we have identified a range of factors that hindered the implementation of patient-facing apps in routine care, which is the major focus of this report.
Participants were only recruited from MUH (an academic private hospital), where socioeconomic status of patients is likely higher than the general community. It has been previously reported that individuals with a higher socioeconomic status also have higher digital literacy rates [
Using alternative platforms to distribute the app (such as TestFlight) may have hindered uptake rates, as most participants were not familiar with this app download process. When working with participants who may have digital literacy concerns, using traditional platforms for app distribution rather than alternative platforms may improve uptake rates.
Although an analysis of the app usage system log revealed quantitative evidence of the most frequently used app features, this may not be reflective of the patients’ app experience. Unfortunately, no patient participants in this study were available for follow-up interviews. The app may not meet patients’ changing needs and expectations during their recovery, which may be factors contributing to the low uptake. Future studies could consider using a theoretical framework to guide study implementation. These frameworks include (but are not limited to) the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework [
The rehabilitation protocol that the app was designed upon was specific to patients with rotator cuff tears undergoing surgery, which may have limited the intake of participants, as well as the generalizability of the app to other conditions. A wider range of exercise protocols could be implemented in the app so that it complements a larger pool of patients recovering from different injuries and surgical procedures. Future studies could also consider recording patient adherence to in-person rehabilitation programs and examine whether it is a contributing factor to app adherence.
In addition, we did not recruit family members or caregivers to be participants of the study. As indicated in our study, caregivers have an important role to play in surgical patients’ postoperative recovery; future studies could consider recruiting patients’ caregivers as study participants to elicit their views.
The potential of mobile apps to support patient care is increasingly recognized, but they are still not routinely recommended by clinicians or integrated as part of standard care [
the importance for implementation to remain
to be inclusive of patients and clinicians of varying levels of digital literacy
the impact of the technology on patients and clinician workflow needs to be communicated, monitored, and reviewed.
Ultimately, digital health technology should supplement and not replace patient interaction with clinicians. Consumer, clinician, and service provider involvement are vital if mobile health is to fulfill its potential.
The science of implementing
Supplementary tables.
App development and features.
Daily patient questionnaire.
Consolidated Framework for Implementation Research
Exploration, Preparation, Implementation, and Sustainment framework
general practitioner
interquartile range
information technology
Macquarie University Hospital
Promoting Action on Research Implementation in Health Services framework
Reach, Effectiveness, Adoption, Implementation, and Maintenance framework
standard deviation
The authors would like to thank Jingbo Liu for his contributions to the software development of the Healthy.me app and to Joyce El-Haddad for her assistance in journey preparation. The authors also gratefully acknowledge the patients and clinicians (Sue Goldrick, Angela Sciberras, and Ruth Green) who assisted with and/or participated in the study.
This project was funded by a grant received from the National Health and Medical Research Council Centre of Research Excellence in Informatics and eHealth (APP1032664). AYSL was supported by the NSW Health Early-Mid Career Fellowship. The funding bodies did not have a role in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.
AYSL, DB, KP, and EC were involved in the study conceptualization. AYSL, KP, DB, and PM were involved in study design. KP, PM, and AYSL were involved in journey design. AYSL, KP, DB, and PM were involved in all design and development. PM and VSLL were responsible for data collection. AYSL and VSLL were responsible for data analysis. AYSL and VSLL were responsible for the first draft, and AYSL, EC, KP, DB, VSLL, and PM were responsible for the subsequent drafts.
The university, EC, and AYSL involved in this project could benefit from any commercialization of Healthy.me or its technologies.