Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 14.09.20 in Vol 7, No 3 (2020): Jul-Sep

This paper is in the following e-collection/theme issue:

    Original Paper

    Mobile App for Monitoring 3-Month Postoperative Functional Outcome After Hip Fracture: Usability Study

    1Department of Trauma Surgery, Ziekenhuisgroep Twente, Almelo, Netherlands

    2Biomedical Signals and Systems, Faculty of Electrical Engineering, Mathematics, and Computer Science, University of Twente, Enschede, Netherlands

    3Ziekenhuisgroep Twente Academy, Ziekenhuisgroep Twente, Almelo, Netherlands

    Corresponding Author:

    Merle A J Geerds, MD, MSc

    Department of Trauma Surgery

    Ziekenhuisgroep Twente

    Zilvermeeuw 1

    Almelo, 7609 PP

    Netherlands

    Phone: 31 0631345083

    Email: m.geerds@zgt.nl


    ABSTRACT

    Background: As a result of an aging population, there has been an increasing incidence of hip fractures worldwide. In the Netherlands, in order to improve the quality of care for elderly patients with hip fractures, the multidisciplinary Centre for Geriatric Traumatology was established in 2008 at the Department of Trauma Surgery at Ziekenhuisgroep Twente hospital (located in Almelo and Hengelo in the Netherlands).

    Objective: Though the Dutch Hip Fracture audit is used to monitor the quality of care for patients with fractures of the hip, only 30.7% of patients complete registration in the 3-month follow-up period. Mobile apps offer an opportunity for improvement in this area. The aim of this study was to investigate the usability and acceptance of a mobile app for gathering indicators of quality of care in a 3-month follow-up period after postoperative treatment of hip fracture.

    Methods: From July 2017 to December 2017, patients who underwent surgical treatment for hip fracture were recruited. Patients and caregivers, who were collectively considered the participant cohort, were asked to download the app and answer a questionnaire. Participants were divided into two groups—those who downloaded the app and those who did not download the app. A telephone interview that was based upon the Unified Theory of Acceptance and Use of Technology was conducted with a subset of participants from each group (1:1 ratio). This study was designated as not being subject to the Dutch Medical Research Involving Human Subjects Act according to the appropriate medical research ethics committees.

    Results: Of the patients and caregivers who participated, 26.4% (29/110) downloaded the app, whereas 73.6% (81/110) did not. Telephone interviews with the subset of participants (n=24 per group) revealed that 54.0% (13/24) of the group of participants who did not download the app had forgotten the study. Among the group who downloaded the app, 95.8% (23/24) had the intention of completing the questionnaire, but only 4.2% (1/24) did so. The reasons for not completing the questionnaire included technical problems, cognitive disorders, or patient dependency on caregivers. Most participants in the group who downloaded the app self-reported a high level of expertise in using a smartphone (22/24, 91.7%), and sufficient facilitating conditions for using a smartphone were self-reported in both groups (downloaded the app: 23/24, 95.8%; did not download the app: 21/24, 87.5%), suggesting that these factors were not barriers to completion.

    Conclusions: Despite self-reported intention to use the app, smartphone expertise, and sufficient facilitating conditions for smartphone use, implementation of the mobile app was infeasible for daily practice. This was due to a combination of technical problems, factors related to the implementation process, and the population of interest having cognitive disorders or a dependency on caregivers for mobile technology.

    JMIR Hum Factors 2020;7(3):e16989

    doi:10.2196/16989

    KEYWORDS



    Introduction

    As a result of an aging population, the global incidence of hip fractures has been increasing with an estimated 6.25 million per year expected by 2050 [1,2]. In the Netherlands, 19,000 patients with hip fractures are treated annually [3,4]. To improve the quality of care for elderly patients with fractures of the hip, the multidisciplinary Centre for Geriatric Traumatology was established in 2008 at the Department of Trauma Surgery at Ziekenhuisgroep Twente hospital (located in Almelo and Hengelo in the Netherlands). Approximately 300 hip fracture patients are treated annually in this center [4]. To improve the quality of care among patients with fractures of the hip nationwide, the Dutch Hip Fracture Audit was established in 2016. The Dutch Hip Fracture Audit [5] monitors quality of care using indicators for quality of hospital stay, 3-month functional outcome, and 1-year mortality. Some of these quality indicators have been formulated by the Health and Youth Care Inspectorate and are mandatory; living situation of the patient, prefracture mobility score, and the Katz Index of Independence in Activities of Daily Living score are currently gathered during scheduled 3-month follow-up visits to the outpatient clinic.

    The proportion of patients who register to provide information regarding functional recovery is poor; only 30.7% of Dutch Hip Fracture Audit registrations are completed [5]. Due to age or health-related factors, patients do not visit the outpatient clinic for their scheduled 3-month follow-up. Poor registration may result in a suboptimal monitoring of quality of care. In contrast, the 3-month registration was completed by 89.0% of the patients in the Centre for Geriatric Traumatology. This higher percentage was achieved by using an active telephone approach for patients who missed or canceled their outpatient appointments; however, the active approach was time consuming and inefficient. Mobile apps may offer an opportunity for improvement. Mobile app use to remotely monitor patients who have a low risk of postoperative complications has been investigated in multiple studies [6-10] which have concluded that mobile apps were useful for following up with patients who had a low risk of postoperative complication and with patients from 18 to 82 years of age who had undergone day-procedures. To our knowledge, no studies have investigated the use of mobile apps for the follow-up of patients with fractures of the hip.

    There has been ongoing worldwide interest in home telemonitoring to support the health and vitality of the community-dwelling elderly population which has led to promising strategies for improving health care and health management [11-13]. Despite interest in the use of home telemonitoring, the literature mostly consists of pilot or feasibility studies. Real-world use and acceptance of home telemonitoring in daily care in older patient populations have mainly been studied in patients with chronic heart failure and have shown high acceptance of the technology using a 12-month survey [14,15]. In order to further optimize mobile app use among the elderly, a supportive theoretical framework has been recommended for iterative design of app implementation and evaluation [16]. These recommendations encompass multidisciplinary approaches, focus on end-user ease of use, and suggest starting with usability and feasibility testing in simulation environments [16-18]. In addition, during implementation, variation in levels of interest and technological literacy should be taken into consideration, especially among older adults [16].

    The primary goal of this study was to investigate the real-world use of a mobile app for monitoring postoperative functional recovery after hip fracture. The secondary goals were to analyze mobile app usability and acceptance among elderly patients and their caregivers. Usability and acceptance were considered to facilitate conditions for use, but were not presumed to lead automatically to use.


    Methods

    App Development and Implementation

    The mobile app platform was developed by technical experts, is currently used, and has previously been used in studies of postoperative outcome with a high rate of use [1]. A multidisciplinary team of health care professionals and technical experts developed a proof-of-concept version of the app that included specific adjustments for an older population of patients. A digital questionnaire consisting of indicators of quality of care from the Dutch Hip Fracture Audit was developed to remotely monitor postoperative functional outcome at 3 months. This questionnaire was implemented in the mobile app, and the technology was pretested with 2 patients with fractures of the hip who had been chosen at random.

    Participant Recruitment

    Patients with a hip fracture who had undergone surgical treatment between July 2017 and December 2017 at the Centre for Geriatric Traumatology of the Department of Trauma Surgery at Ziekenhuisgroep Twente hospital were recruited to participate in the study and asked to download the app in addition to their regular 3-month outpatient visit (the recruitment process is summarized in Figure 1). The population of interest consisted of older adults, among whom information and communication technology literacy or low motivation to use technology may be factors that hinder implementation of a mobile app and which could suggest the need to focus on patient spouses in addition to the patients themselves [19]. For the purpose of this study, both patients and spouses who decided to participate were considered participants. During admission to the surgical ward of the Centre for Geriatric Traumatology, a nurse informed potential participants about the study, use of the app, and how to download instructions for using the app. After verbally providing informed consent, participants received an information leaflet and provided their email address for further information.

    Figure 1. Study design flowchart.
    View this figure

    One week later, participants received a code by mail to activate the questionnaire in the downloaded app. Completion of the questionnaire was restricted to a period between 12 weeks and 18 weeks after their operation. A push notification with a request to complete the questionnaire was sent to the participant 12 weeks after they had been discharged from the hospital. A push notification was also sent to the health care provider at 17 weeks for unfilled questionnaires.

    Completed questionnaires were saved in OpenLine (a specialized health care hosting center) in accordance with Dutch legislation with respect to security standards. The local researcher applied for the data from the hosting center. Participants were anonymized and coded using a study number without any reference to patient number or date of birth. Only the local researcher had access to the participant study numbers. All data were treated confidentially and saved to the secured hospital network with a password.

    Usability and Acceptance Questionnaire

    To investigate usability and participant acceptance of the mobile app, an interview questionnaire was developed (Multimedia Appendix 1) based upon the Unified Theory of Acceptance and Use of Technology [20]; the model investigates user intentions and usage behavior in technology systems [20].

    Two questions regarding participant recollection of the intended purpose of the study and feedback on the use of the app were added to the interview. These questions were added because we were interested in obtaining feedback on the app and on the duration of the interval from when the information was given (from July 2017 to December 2017) to when the telephone interview took place (in May 2018). A single researcher conducted all interviews. Participants were given the option to stop the telephone interview at any time.

    Data Collection

    Data were collected from the clinical charts of the patients who participated themselves or whose caregivers participated. Age, gender, type of fracture, American Society of Anesthesiologists physical classification status, Charlson Comorbidity Index [21], dementia, prefracture Katz Index of Independence in Activities of Daily Living score [22], prefracture mobility score, and prefracture living situation were recorded as baseline characteristics. In April 2018, the app usage data from the hosting center were collected. Participants were divided into two groups—those who downloaded the app (use group) and those who did not download the app (nonuse group). Mobile app usability and acceptance telephone interviews were conducted with participants who could be reached by telephone within 3 attempts. The number of participants in both groups was adjusted to the lowest number of participants accessible by telephone of either group (use group, n=24); therefore, in the nonuse group, 24 participants were selected randomly. Participant answers were fully transcribed in individual and anonymized Office Word (version 2007; Microsoft Inc) documents and saved on a secure hospital server.

    Data Analysis

    Statistical analyses were performed using SPSS software (version 22.0; IBM Corp). We used thematic analysis with a deductive theoretical approach to analyze the written answers to the recalled purpose of the study and feedback questions [23]. Identification of patterns and themes within the data was performed by one researcher, and a second researcher was consulted to reach agreement; the data were then coded by themes. Categorical data were analyzed using the chi-square test or Fisher exact test when appropriate (ie, Fisher exact test was used when frequency was less than 5). Functional outcomes were analyzed using two-tailed paired t tests. Continuous data were analyzed using two-tailed independent t tests. If significant differences were found in categorical variables with two or more subgroups, Pearson chi-square test was performed post hoc. P<.05 was considered statistically significant.

    Ethics

    This study was been designated as an observational study not subject to the Dutch Medical Research Involving Human Subjects Act by the appropriate medical research ethics committees.


    Results

    Patient Characteristics

    Categorical variables are described as numbers with corresponding percentages. Continuous variables are described as the mean with standard deviation, or for nonparametric data, as the median with interquartile range.

    Patient characteristics are shown in Table 1. Patients with fractures of the hip (N=110) were a mean age of 80.5 (SD 10.4) years and were 71.8% (79/110) female and 28.2% (31/110) male. No significant differences were found between those who downloaded the app and those who did not download the app for age (P=.21), gender (P>.999), type of fracture (P>.999), American Society of Anesthesiologists physical classification status (P>.999), Charlson Comorbidity Index (P>.999), dementia (P=.05), prefracture Katz Index of Independence in Activities of Daily Living score (P=.10), prefracture mobility score (P=.10), and prefracture living situation (P=.73).

    Table 1. Baseline patient characteristics.
    View this table

    App Use

    Of the participants (29/110, 26.4%) who downloaded the mobile app, only 1 (1/29, 3.4%) completed the app questionnaire.

    Interviewed Participants

    Characteristics

    Participants characteristics of those who participated in the telephone interviews are presented in Table 2. In the use group (the subset of the group who downloaded the app), 95.8% (23/24) self-reported as expert level, and 87.5% (21/24) participants in the nonuse group (the subset of the group who did not download the app) self-reported as expert level. The groups showed significantly differences for smartphone usage of 5 to 10 years (use: 0/24, 0.0%; nonuse: 8/24, 33.3%; P=.004) and more than 10 years (use: 22/24, 91.7%; nonuse: 15/24, 62.5%; P=.02).

    Table 2. Comparison of baseline characteristics between the use (participants downloaded the app) and nonuse (participants did not download the app) groups.
    View this table
    Questionnaire Results

    Questionnaire results are presented in Multimedia Appendix 2. Among the use group, 95.8% (23/24) of participants had the intention of completing the app questionnaire; 41.7% (10/24) of the nonuse group had the intention of downloading the mobile app. In the nonuse group, 54.2% (13/24) stated that they were not informed during admission at the hospital or by mail of the app; 4% (1/24) had no intention of downloading the app. Therefore, no difference in expectancy determinants were calculated between the groups, and no answers were considered as blank.

    Thematic Analysis

    A thematic analysis was conducted to evaluate patient recollection of the study’s purpose. Participant responses (transcribed excerpts are presented in Multimedia Appendix 3) resulted in five themes: functional monitoring, replacement of the outpatient appointment, evaluation of participant satisfaction, no idea or not sure, and other. Correct answers for patient recollection of the study’s purpose were defined as those classified within the themes of functional monitoring and future replacement of the outpatient appointment.

    The study purpose was correctly remembered by 62.5% (15/24) of the use group participants compared to only 20.8% (5/24) in the nonuse group; 50% (12/24) of the participants in the use group said that they did not receive a smart phone notification with the request to complete the questionnaire which suggested a suboptimal implementation process.


    Discussion

    Principal Findings

    Completion of 3-month mandatory functional monitoring is poor among patients with fractures of the hip, which may result in a suboptimal monitoring of quality of care. This single-center pilot study to investigate the use and to analyze the usability and acceptance of a mobile app for monitoring postoperative functional recovery after hip fracture revealed poor results for actual use of the mobile app despite high self-reported intention to use the mobile app, high self-reported expertise in using mobile apps, and conditions that facilitated the use of mobile apps. This suggests that participants had the goal of using the mobile app, but that better support was needed to properly implement the technology in health care.

    For many years, apps have been regarded as an alternative to paper questionnaires, but the use of apps may have difficulties as well, especially when implemented in a population of community-dwelling older patients [16]. This study demonstrated implementation difficulties; only 26.4% (29/110) participants downloaded the mobile app. This demonstrated that implementation of the app may have required that sufficient attention be given to education of the community-dwelling older patient users.

    The low percentage of app downloads could partially be explained by an inability of the patients or caregivers to correctly remember the information that was provided to them in the hospital possibly as a result of stress [24]. Receiving information in a state of stress has been associated with suboptimal information processing and reduced cognitive efficiency [25,26]; therefore, correct timing of information provision is essential. This study provided both oral and written information, but more emphasis should be given to written information or video instructions, as this has been shown to lead to better information retention [27]. Among participants who are elderly, an inverse correlation has been reported between age and recall of medical information which could also have influenced the findings of this study [24,28]. The 3-month time period between when the information was provided and when the questionnaire was to be completed which also could have negatively affected information recall and recollection of the study’s purpose.

    One participant completed the app questionnaire after downloading the app. This participant showed an active approach by contacting the app developers and completed the questionnaire with assistance from the developers.

    A high percentage of the participants (34/48, 70.8%) who were interviewed were caregivers who were first-degree relatives of the patient. Study information was provided independently of whether a caregiver was present at the time of information provision; therefore, it is possible that some first-degree relatives were not provided with the study information if they were absent during recruitment.

    The telephone interview findings demonstrated that many in the use group had the intention of completing the questionnaire. This indicates that those participants were motivated to complete the app questionnaire. In the nonuse group (11/24, 45.8%), participants remembered the study, and 10 out of the 11 intended to download the app. Given this result, there seems to be a good level of intention in both groups. Facilitating conditions, such as facilitated help, were high in both groups and were not a restrictive factor for app usage [29]. Some participants in the nonuse group (13/24, 54.2%) were unable to remember the study, and they could not complete the interview. Difficulties in patients or caregiver recollection of study information may have been influenced by the previously noted patient-related factors such as cognitive impairment, anxiety, or stress [24]. Approaching multiple caregivers when providing information and conducting the telephone interview may also be a reason for some participants reporting that they did not remember the study. Respondents (18/48, 41.6%) also reported technical problems. The app developers suggested the start-up phase of the app as a possible explanation for the technical problems. The developers also suggested that a lack of received notifications could have been as a result of participants not enabling the appropriate permissions for notifications when downloading the app. Providing help in the hospital with downloading of the app could assist with this issue. Another way to decrease the frequency of technical problems while also optimizing usability and acceptance would be to frequently evaluate the mobile app during the implementation process [16].

    Recommendations

    Findings revealed intention to use the mobile app, but very low actual usage. The use of a mobile app as it was implemented in this study was not feasible, but the study findings suggested a potential for use if implemented properly. First, technical issues should be solved, and a helpdesk should be made available. Second, it is recommended to involve participants in the development and implementation phases—doing so can optimize ease of use and acquiring feedback during implementation is a feasible goal. Third, information provision needs to be optimized in terms of timing and method of dissemination. It is important to supply additional information after discharge in order to prevent low download rates as a result of patient or caregiver stress during admission [27]. Written information, video instructions, or fact sheets are preferred to oral information [2,3]. Fourth, in studies involving caregivers, a single contact person is recommended.

    Limitations

    Selection bias in the downloading group represents a threat to validity, as patients or caregivers already intended to participate in the study by downloading the app.

    Conclusions

    The use of a mobile app to monitor 3-month postoperative functional outcome of hip fracture was low. Despite intention, expertise, and sufficient facilitating conditions for using smartphones, the implementation of the mobile app in this study was demonstrated to be infeasible. Reasons for this included a technical problem, the implementation process, and population of interest having cognitive disorders or a dependency on caregivers for mobile technology.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Telephone interview questionnaire.

    DOCX File , 14 KB

    Multimedia Appendix 2

    Questionnaire results.

    DOCX File , 21 KB

    Multimedia Appendix 3

    Excerpts from participant responses.

    DOCX File , 14 KB

    References

    1. Dhanwal D, Dennison E, Harvey N, Cooper C. Epidemiology of hip fracture: worldwide geographic variation. Indian J Orthop 2011 Jan;45(1):15-22 [FREE Full text] [CrossRef] [Medline]
    2. Cooper C, Campion G, Melton LJ. Hip fractures in the elderly: a world-wide projection. Osteoporosis Int 1992 Nov;2(6):285-289. [CrossRef]
    3. 3 CSB. Letsels. 2017. Blatter, Letsel informatie systeem 2017, Amsterdam, 2018   URL: https:/​/www.​veiligheid.nl/​.ibmmodres/​domino/​OpenAttachment/​Veiligheid/​Website.nsf/​FD80D963DD249926C125838C003DF481/​asset/​Kerncijfers [accessed 2018-05-27]
    4. Folbert ECE, Smit RS, van der Velde D, Regtuijt EMM, Klaren MH, Hegeman JHH. Geriatric fracture center: a multidisciplinary treatment approach for older patients with a hip fracture improved quality of clinical care and short-term treatment outcomes. Geriatr Orthop Surg Rehabil 2012 Jun;3(2):59-67 [FREE Full text] [CrossRef] [Medline]
    5. Jaarrappotage 2016: Dutch Hip Fracture Audit.   URL: https://dica.nl/media/993/DICA-2016-jaarverslag.pdf [accessed 2018-05-12]
    6. Higgins J, Semple J, Murnaghan L, Sharpe S, Theodoropoulos J. Mobile web-based follow-up for postoperative ACL reconstruction: a single-center experience. Orthop J Sports Med 2017 Dec;5(12):2325967117745278 [FREE Full text] [CrossRef] [Medline]
    7. Armstrong K, Coyte P, Semple J. The effect of mobile app follow-up care on the number of in-person visits following ambulatory surgery: a randomized control trial. Stud Health Technol Inform 2015;216:894. [Medline]
    8. Jaensson M, Dahlberg K, Eriksson M, Grönlund Å, Nilsson U. The development of the recovery assessments by phone points (RAPP): a mobile phone app for postoperative recovery monitoring and assessment. JMIR Mhealth Uhealth 2015 Sep 11;3(3):e86 [FREE Full text] [CrossRef] [Medline]
    9. Armstrong KA, Coyte PC, Brown M, Beber B, Semple JL. Effect of home monitoring via mobile app on the number of in-person visits following ambulatory surgery: a randomized clinical trial. JAMA Surg 2017 Jul 01;152(7):622-627 [FREE Full text] [CrossRef] [Medline]
    10. Jaensson M, Dahlberg K, Eriksson M, Nilsson U. Evaluation of postoperative recovery in day surgery patients using a mobile phone application: a multicentre randomized trial. Br J Anaesth 2017 Nov 01;119(5):1030-1038 [FREE Full text] [CrossRef] [Medline]
    11. Joe J, Demiris G. Older adults and mobile phones for health: a review. J Biomed Inform 2013 Oct;46(5):947-954 [FREE Full text] [CrossRef] [Medline]
    12. Kitsiou S, Paré G, Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. J Med Internet Res 2015 Mar 12;17(3):e63 [FREE Full text] [CrossRef] [Medline]
    13. Cosco TD, Firth J, Vahia I, Sixsmith A, Torous J. Mobilizing mHealth data collection in older adults: challenges and opportunities. JMIR Aging 2019 Mar 19;2(1):e10019 [FREE Full text] [CrossRef] [Medline]
    14. Clark RA, Yallop JJ, Piterman L, Croucher J, Tonkin A, Stewart S, CHAT Study Team. Adherence, adaptation and acceptance of elderly chronic heart failure patients to receiving healthcare via telephone-monitoring. Eur J Heart Fail 2007 Nov;9(11):1104-1111 [FREE Full text] [CrossRef] [Medline]
    15. Scherr D, Kastner P, Kollmann A, Hallas A, Auer J, Krappinger H, MOBITEL Investigators. Effect of home-based telemonitoring using mobile phone technology on the outcome of heart failure patients after an episode of acute decompensation: randomized controlled trial. J Med Internet Res 2009 Aug 17;11(3):e34 [FREE Full text] [CrossRef] [Medline]
    16. Matthew-Maich N, Harris L, Ploeg J, Markle-Reid M, Valaitis R, Ibrahim S, et al. Designing, implementing, and evaluating mobile health technologies for managing chronic conditions in older adults: a scoping review. JMIR Mhealth Uhealth 2016 Jun 09;4(2):e29 [FREE Full text] [CrossRef] [Medline]
    17. May CR, Finch TL, Cornford J, Exley C, Gately C, Kirk S, et al. Integrating telecare for chronic disease management in the community: What needs to be done? BMC Health Serv Res 2011 May 27;11(1). [CrossRef]
    18. Nundy S, Dick JJ, Goddu AP, Hogan P, Lu CE, Solomon MC, et al. Using mobile health to support the chronic care model: developing an institutional initiative. Int J Telemed Appl 2012;2012:871925 [FREE Full text] [CrossRef] [Medline]
    19. Heart T, Kalderon E. Older adults: are they ready to adopt health-related ICT? Int J Med Inform 2013 Nov;82(11):e209-e231. [CrossRef] [Medline]
    20. Venkatesh, Morris, Davis, Davis. User acceptance of information technology: toward a unified view. MIS Quarterly 2003;27(3):425. [CrossRef]
    21. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40(5):373-383. [CrossRef] [Medline]
    22. Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of illness in the aged. the index of ADL: a standardized measure of biological and psychosocial function. JAMA 1963 Sep 21;185:914-919. [CrossRef] [Medline]
    23. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
    24. Kessels RPC. Patients' memory for medical information. J R Soc Med 2003 May;96(5):219-222 [FREE Full text] [CrossRef] [Medline]
    25. Siddiqui MQ, Sim L, Koh J, Fook-Chong S, Tan C, Howe TS. Stress levels amongst caregivers of patients with osteoporotic hip fractures - a prospective cohort study. Ann Acad Med Singapore 2010 Jan;39(1):38-42 [FREE Full text] [Medline]
    26. Luethi M, Meier B, Sandi C. Stress effects on working memory, explicit memory, and implicit memory for neutral and emotional stimuli in healthy men. Front Behav Neurosci 2008;2:5 [FREE Full text] [CrossRef] [Medline]
    27. Blinder D, Rotenberg L, Peleg M, Taicher S. Patient compliance to instructions after oral surgical procedures. Int J Oral Maxillofac Surg 2001 Jun;30(3):216-219. [CrossRef] [Medline]
    28. Ley P. Memory for medical information. Br J Soc Clin Psychol 1979 Jun;18(2):245-255. [CrossRef] [Medline]
    29. Venkatesh, Thong, Xu. Consumer acceptance and use of information technology: extending the Unified Theory of Acceptance and Use of Technology. MIS Q 2012;36(1):157. [CrossRef]

    Edited by B Price; submitted 09.11.19; peer-reviewed by O Pearce, D Banks, D Gooch; comments to author 28.11.19; revised version received 23.03.20; accepted 17.04.20; published 14.09.20

    ©Merle A J Geerds, Wieke S Nijmeijer, J H Hegeman, Miriam M R Vollenbroek-Hutten. Originally published in JMIR Human Factors (http://humanfactors.jmir.org), 14.09.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on http://humanfactors.jmir.org, as well as this copyright and license information must be included.