This paper is in the following e-collection/theme issue:

Design and Usability of Clinical Software and EHRs (242) Adoption and Change Management of eHealth Systems (677) Usability of Apps and User Perceptions of mHealth (902) Electronic Health Records (971) mHealth in a Clinical Setting (582) Usability Evaluation Case Studies (182) Design and Formative Evaluation of Mobile Apps (664)

Published on in Vol 11 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/47755, first published .
Usability of an App for Medical History Taking in General Practice From the Patients’ Perspective: Cross-Sectional Study

Usability of an App for Medical History Taking in General Practice From the Patients’ Perspective: Cross-Sectional Study

Usability of an App for Medical History Taking in General Practice From the Patients’ Perspective: Cross-Sectional Study

Authors of this article:

Klara Albrink1 Author Orcid Image ;   Dominik Schröder1 Author Orcid Image ;   Carla Joos1 Author Orcid Image ;   Frank Müller1 Author Orcid Image ;   Eva Maria Noack1 Author Orcid Image
Article Authors Cited by Tweetations (2) Metrics

    Original Paper

    Department of General Practice, University Medical Center Göttingen, Göttingen, Germany

    Corresponding Author:

    Eva Maria Noack, PhD

    Department of General Practice

    University Medical Center Göttingen

    Humboldtallee 38

    Göttingen, 37073

    Germany

    Phone: 49 551 39 65745

    Email: evamaria.noack@med.uni-goettingen.de


    Background: A future shortage of physicians, especially in general practice, will result in an increasing workload for health care providers as a whole. Therefore, it is important to optimize patient-encounter processes to increase time efficiency related to visits. Utilizing digital tools to record patients’ medical histories prior to a consultation offers great potential to achieve this goal. The collected information can be stored into the practice’s electronic medical record, allowing for the general practitioner to review structured information of the patients’ complaints and related medical history beforehand, thereby saving time during the encounter. However, the low usability of new digital developments in this setting often hinders implementation.

    Objective: The aim of this study was to evaluate the usability of an app designed for medical history taking in general practice to capture the patients’ perspective.

    Methods: Between November 2021 and January 2022, we recruited 406 patients with acute complaints in one out-of-hour urgent care and seven general practice clinics. These study participants used the app during their waiting time and subsequently assessed its usability by completing the System Usability Scale (SUS), a robust and well-established 10-question survey measuring the perceived usability of products and technologies. Additionally, we collected general participant information, including age, sex, media usage, health literacy, and native language. Descriptive and inferential statistics were applied to identify patient characteristics associated with low or high SUS scores.

    Results: We analyzed data from 397 patients (56.7% female, 43.3% male). The mean total SUS score was 77.8 points; 54.4% (216/397) of participants had SUS scores of 80 points or higher, indicating high usability of the app. In a multiple linear regression predicting SUS score, male sex and higher age (65 years or older) were significantly negatively associated with the SUS score. Conversely, a higher health literacy score and German as the native language were significantly positively associated with the SUS score.

    Conclusions: Usability testing based on the SUS anticipates successful implementation of the app. However, not all patients will easily adapt to utilizing the app, as exemplified by the participants of older age in this study who reported lower perceived usability. Further research should examine these groups of people, identify the exact problems in operating such an app, and provide targeted solutions.

    Trial Registration: German Clinical Trials Register World Health Organization Trial Registration Data Set DRKS00026659; https://trialsearch.who.int/Trial2.aspx?TrialID=DRKS00026659

    JMIR Hum Factors 2024;11:e47755

    doi:10.2196/47755

    Keywords

    digitization; application software; usability; mHealth; history of present illness; medical history taking 


    As in many countries, demographic change is becoming evident in the German health care system, resulting in more complex, multimorbid patients [1] and a shortage of physicians [2]. Moreover, the proportion of older people in the population is rising steadily [3] and people tend to use medical services at a higher rate as they increase in age [4]. In Germany, one group that is particularly affected by this development are general practitioners (GPs) who are the first point of contact for patients requiring medical care and serve as the “gatekeepers” in the German health care system [5]. Approximately 80% of the German population aged 18 years and older are treated by a GP at least once a year [6]. A considerable number of GPs will retire in the upcoming years, resulting in 11,000 GP vacancies expected by 2035. These vacancies will disproportionately impact structurally weak and rural areas [7,8]. Without a sufficient workforce to replace the retired GPs and meet the greater demand for physicians, remaining GP workloads are expected to increase significantly within the next decade [9]. These developments challenge the German health care system at various levels and require attention to address the following key issues: future financing, improving allocation of resources, ensuring access to care, increasing efficiency and effectiveness of health care provision, and strengthened collaboration between providers [10].

    To streamline patient care in the upcoming years, it is of importance to optimize patient-encounter processes to increase time efficiency related to visits. In this respect, digital tools offer great potential to support GPs in patient management, documentation workload, and the collection of medical history before consultation.

    Digital tools designed to collect patients’ medical history can ensure that information is always collected and documented thoroughly in a structured manner and with consistent quality. As many conditions can be diagnosed via a thorough medical history [11,12], these tools can be helpful in maintaining quality of care when time constraints may lead to an otherwise superficial medical history.

    As part of our project titled “Digitally assisted information acquisition before medical consultation” (DASI), we developed an app for medical history taking in general practice settings. The app is used by the patient prior to the medical consultation, which could be either in the waiting area or at home. The collected information can be stored in the practice’s electronic medical record and eventually be transferred to the individual electronic patient file, which statutory health insurers in Germany have been entitled to use since 2021 [13]. In the electronic patient file, patient data such as medical reports, X-rays, immunization records, and other medical data can be stored and shared among medical providers involved in the care of a particular patient by using the telematics infrastructure [14].

    One advantage of the app is that the GP can review structured information of the patients’ complaints and related medical history before the encounter. This is particularly helpful for patients that are unknown to the provider, those with many complaints, or those who have a comprehensive medical history. These situations are especially prevalent in out-of-hour urgent care practices. Furthermore, the tool might help patients to reflect on their conditions and enable them to better address their needs when seeing the provider. In this way, we expect that the limited consultation time can be used more efficiently.

    Despite Germany’s progress in digitalization within the health sector, concerns remain about the limited usability of new digital tools, hindering their full implementation. More than half of German practices see low usability as a strong obstacle to digitalization [15]. The evaluation of a digital tool’s perceived usability is of special interest as it is a key determinant of performance for end users. Therefore, the aim of this study was to assess the usability of the app from the patients’ perspective and to identify features in need of improvement. The broader aim is to ensure that the app is suitable for implementation in everyday practice, considering that GPs treat a broad range of patients of all ages and various educational and cultural backgrounds [16].


    Study Design and Recruitment

    This was a cross-sectional study conducted in Germany in one out-of-hour urgent care practice and seven GP practices to assess the usability of an app designed for medical history taking in general practice settings.

    Software and Hardware

    The app was developed to take a medical history based on general medical complaints directly from the patients. While there are no international standards for the composition of a standardized patient history, this app was developed based on guidelines and health literature by medical experts from aidminutes GmbH (Hamburg/Buchholz in der Nordheide, Germany). For this study, the content and query structure were further refined for primary care (general practice and out-of-hour practices) by aidminutes GmbH in collaboration with experienced researchers from the Department of General Practice at the University Medical Center Göttingen, Germany. The app was designed to be used by patients in the waiting room before they see the doctor. Patients select one or several complaints and are then guided through a symptom-related questionnaire. In the sense of a branching logic, the app is adaptive to patient responses, which trigger further specific questions about the selected key complaints (eg, how and when a symptom started). Patients are also asked about preexisting conditions, previous treatments and surgeries, current medication, living habits, and chronic conditions in the family history. Information such as biological sex, height, weight, age, as well as the subjectively perceived severity of the complaints are inquired from all patients. More details can be found in the published study protocol [17].

    The app was designed to be intuitive for the user such that no prior knowledge or any kind of instruction for its use is necessary. The user interface was designed to be simple to follow and only one question is asked per screen. As the app is operated in the waiting area, sound and video output of an earlier version [18] was omitted due to data protection. The questions are phrased in plain language; medical terminology is avoided or otherwise explained. The questions are substantially comprised by single-choice or multiple-choice questions that can be answered by tapping but also include several data fields (for age, height, and weight) and slider-type questions (Figure 1). The color scheme was designed to ensure reading accessibility for patients who may be color blind. A zoom function can be used for users who may experience visual impairment.

    ‎
    Figure 1. Screenshots of the app for medical history taking in general practice showing different types of questions: (A) single-choice question; (B) multiple-choice question; (C) hybrid question (ie, patients can either select several options or negate all of them); (D) slider for questions including a ranking between items (depicted here as “How sick do you feel?”); (E, F) data entry field (here: “Please enter your age”); and (G, H) selection of a body region on a figure (depicted in figure: “Please mark on the figure where you are suffering from the problems”).

    As this is a web-based app, it relies on a permanent internet connection. For this study, the app ran on an iPad Mini 4 (Apple Inc, Cupertino, CA, USA) held in an upright position. Tablets were equipped with haptically and visually inconspicuous cases (dark grey polyurethane leather outside and microfiber inside).

    Setting

    In Germany, GP practices aim at providing preventive, acute, and rehabilitative health care with long-lasting patient-doctor relationships. Out-of-hours urgent care practices provide urgent medical care for acute but not life-threatening cases when other practices are closed. Urgent care practices are often staffed with doctors of various specialties and an established relationship of care between the patients and doctors is not common. These aspects can lead to challenges in efficiently obtaining an accurate medical history and identifying serious health problems. Although the app was designed for general practice, it is also suitable to be used in out-of-hour urgent care practices.

    Data Collection

    The recruitment of patients was carried out by three study nurses and took place from November 22, 2021, to January 12, 2022. Patients were approached by study nurses in the waiting room of the respective practices before seeing their GP.

    Patients meeting the following criteria were eligible to participate in the study: (1) seeking care in a participating practice because of acute somatic and/or psychological complaints, (2) at least 18 years old, and (3) consenting to participate in the study. Patients meeting the following exclusion criteria could not participate in the study: (1) younger than 18 years old (legal minor), (2) patients in an apparent emergency, (3) patients who required immediate medical treatment, and (4) patients who were unable to provide consent.

    After the study nurses obtained written informed consent, a tablet on which the app was run on was handed over to the study participants. Participants used the app to report their medical history without an introduction on how to navigate the app. Once finished, they were asked to answer questions on personally perceived usability, media usage, and further sociodemographic data, which were digitally attached to the medical history–taking document. The study nurse in charge was present to observe any problems study participants may have had with using the app and was available to answer questions about the app’s content and usability if specifically requested. Data were collected in an anonymized format without any personal information (eg, name or address) linking the results to each study participant. More detailed information on the data collection can be found in the study protocol [17].

    Ethical Considerations

    The Medical Ethics Committee of the University Medical Center Göttingen approved the study (approval number 26/3/21). A written informed consent form was collected from all patients before their inclusion in the study. Participating in the study was voluntary for patients. Patients could withdraw from participation without giving a reason at any time before they had completed the survey. Subsequently, their data could no longer be deleted because it could not be traced back to the individual.

    Measures

    The main outcome “usability” was measured using the System Usability Scale (SUS) [19], a commonly used instrument for this purpose [20]. The SUS was developed based on Standard ISO 9241-11 [21], in which usability is measured by the three main attributes of “effectiveness,” “efficiency,” and “satisfaction” [22,23]. Compared to other instruments, the SUS offers several advantages: (1) it can be analyzed quickly, (2) it is relatively easy to understand by academics from other disciplines [24], (3) it contains only 10 statements for easy completion, and (4) it can be used to evaluate almost any type of user interface [25]. We used the translated and validated German version of the SUS [26] and modified the statements to suit our purpose (see Multimedia Appendix 1).

    The SUS consists of 10 statements (Table 1), where statements 1, 3, 5, 7, and 9 are positively connoted and statements 2, 4, 6, 8, and 10 are negatively connoted [19]. The scores for these statements are therefore inverted when calculating the sum. The raters decide on the extent to which they agree or disagree to these statements on a 5-point Likert scale ranging from 0 (strongly disagree) to 4 (strongly agree). The final sum score is multiplied by 2.5, resulting in a score range of 0-100 with higher scores indicating better usability [19].

    Lewis and Sauro [27] developed a curved grading scale for SUS scores by comparing more than 200 industrial usability studies and using the percentile ranges, resulting in grades “C” (scores of 62.7-72.5), “B” (scores of 72.6-78.8), and “A” (scores of 78.9-100). As a SUS score of 80 proves an above-average user experience, it has become a common industrial goal. This threshold was therefore used for interpreting our results.

    Table 1. Items of the System Usability Scale (SUS) [19].
    ItemsEnglish version of the statement
    SUS 1I think that I would like to use this system frequently
    SUS 2aI found the system unnecessarily complex
    SUS 3I thought the system was easy to use
    SUS 4aI think that I would need the support of a technical person to be able to use this system
    SUS 5I found the various functions in this system were well integrated
    SUS 6aI thought that there was too much inconsistency in this system
    SUS 7I would imagine that most people would learn to use this system very quickly
    SUS 8aI found the system very cumbersome to use
    SUS 9I felt very confident using the system
    SUS 10aI needed to learn a lot of things before I could get going with this system

    aThe scores of negatively connoted SUS items were inverted when calculating the sum.

    Covariates

    Consultations in general practice are attended by patients of different ages and educational as well as cultural backgrounds, who have a different quantities of digital interactions in everyday life. To determine whether these factors have an influence on the personally perceived usability, we surveyed age, sex, media usage, health literacy, and native language. Information about age and sex were part of the app-taken medical history. In addition to the SUS, we asked patients about which digital media tools were available to them in everyday life (possible answers: cell phone/smartphone, computer/laptop/notebook, tablet, television, none, and others; multiple answers were possible) and how many hours a day they used digital media (possible answers: 0-≤1, 1-≤2, 2-≤3, 3-≤4, or 4 or more hours). We asked three questions concerning health literacy as a proxy for education attainment, given that educational achievement is the central determinant of health literacy [28]. Questions covering the three aspects of finding/accessing, evaluating/appraising, and understanding health-related information and content were derived from the European health literacy survey [29,30] adapted for the German language (HLS-GER 2 [31]). The HLS-GER 2 uses a predefined 4-point Likert scale.

    Statistical Analysis

    Data from the app were saved into a database and subsequently exported to a tab separated format for further analyses. Participants with two or more missing values of the SUS questionnaire were excluded from statistical analysis. In the case of one missing SUS response, we substituted the missing value with a neutral score of 2, as this method has been used with the SUS in previous research [32].

    Sociodemographic data are presented as number and percentage of patients for each categorical data point. Mean and SD were utilized for interval or ratio-scaled data, which has become a common industrial goal. Sociodemographic data were compared between participants with SUS scores <80 and ≥80 using the Fisher exact test for 2×2 tables or the Fisher-Freeman-Halton test for categorical variables and the Wilcoxon rank-sum test for continuous variables. A multiple linear regression was conducted using sex, age, native German language, health literacy score, media usage duration per day, sickness level of the participants, and number of stated complaints in the app as independent variables and the SUS score as the dependent variable. Additionally, the individual SUS items were compared according to sex, age (<65 years vs ≥65 years), German native language, and tablet usage with the Wilcoxon rank-sum test. Data are visually presented as boxplots and radar charts. All analyses were carried out using R (4.1.3 under a GNU license) with the packages fmsb [33], psych [34], tidyr [35], dplyr [36], and ggplot2 [37].


    Patient Characteristics

    We aimed to include approximately 400 patients for this study. This target was set to be able to form subgroups and to ensure that all types of patient complaints were included in our sample, including those selected on a limited basis. In total, individual data from 397 participants were included, with 5 participants having one missing SUS item. Figure 2 shows the flowchart of included patients and Table 2 shows the patients’ characteristics.

    ‎
    Figure 2. Flowchart of patient inclusion in the cross-sectional study capturing patients’ perceived usability of the app. SUS: System Usability Scale.
    Table 2. Characteristics of the participants of the cross-sectional study capturing patients’ perceived usability of the app (N=397).
    CharacteristicsValue
    Sex, n (%)

    		Female225 (56.7)

    		Male172 (43.3)
    Age (years), median (IQR)35.0 (25.0)
    Age group (years), n (%)

    		<30152 (38.3)

    		30-65223 (55.4)

    		65+22 (6.3)
    Native language German, n (%)328 (82.6)
    Devices used regularlya, n (%)

    		Smartphone389 (98.0)

    		Tablet210 (52.9)

    		Computer/notebook310 (78.1)

    		Television296 (74.6)
    Media usage duration per day (hours), n (%)

    		<289 (22.4)

    		2-4174 (43.8)

    		>4134 (33.8)
    Self-assessed health literacy, median (IQR)b

    		Understanding doctor2.0 (0.0)

    		Search and understand health information2.0 (1.0)

    		Evaluate health information1.0 (1.0)
    “How sick do you feel?”c, n (%)

    		I don’t feel sick32 (8.1)

    		Just a little70 (17.6)

    		Fairly226 (56.9)

    		Very61 (15.4)

    		Unbearably6 (1.5)

    aMultiple selection possible.

    bMeasured on a 4-point (0-3) Likert-scale (higher scores indicate higher health literacy levels).

    cPerceived severity of acute complaint.

    Usability for All Participants

    We found a mean total SUS score of 77.8 points, with 54.4% (216/397) of participants having SUS scores of 80 points or higher, indicating high usability of the app overall. Figure 3 shows boxplots of the individual items in which the scores were calculated for each statement. Irrespective of a positive or negative connotation, a higher score indicates a better result. The maximum score that can be achieved for each item is 10.

    ‎
    Figure 3. SUS items and sum scores for all study participants (N=397). SUS: System Usability Scale. *Scoring was inverted.

    Usability Stratified by Sociodemographic Factors

    We divided the sample into two groups with the cutoff at a SUS score of 80. Participants with a SUS score of at least 80 were significantly younger, reported higher levels of technology device usage, and higher levels of self-assessed health literacy compared to participants with a SUS score below 80 (Table 3).

    Table 3. Sociodemographic variables of study participants stratified by System Usability Scale (SUS) score.
    VariableSUS score<80 (n=181)SUS score≥80 (n=216)P value
    Sex, n (%).09a

    		Female94 (51.9)131 (60.6)

    		Male87 (48.1)85 (39.4)
    Age (years), median (IQR)38.0 (26.0)32.5 (22.0).002b
    Age group (years), n (%).003c

    		<3055 (30.3)97 (44.9)

    		30-65111 (60.2)112 (51.4)

    		65+15 (9.4)7 (3.7)
    Native language German, n (%)142 (78.5)186 (86.1).05a
    Devices used regularlyd, n (%)

    		Smartphone174 (96.1)215 (99.5).03a

    		Tablet84 (46.4)126 (58.3).02a

    		Computer/notebook136 (75.1)174 (80.6).22a

    		Television125 (69.1)171 (79.2).03a
    Media usage duration per day (hours), n (%).08c

    		<248 (26.5)41 (19.0)

    		2-469 (38.1)105 (48.6)

    		>464 (35.4)70 (32.4)
    Self-assessed health literacy, median (IQR)

    		Understanding doctor2.0 (0.0)2.0 (1.0).16b

    		Search and understand health information2.0 (0.0)2.0 (1.0).01b

    		Assess confidence of health information1.0 (1.0)1.0 (1.0).19b
    “How sick do you feel?”e, n (%).35c

    		I don’t feel sick11 (6.1)21 (9.7)

    		Just a little30 (16.6)40 (18.5)

    		Fairly102 (56.4)124 (57.4)

    		Very34 (18.8)27 (12.5)

    		Unbearably3 (1.7)3 (1.4)

    aFisher exact test.

    bWilcoxon rank-sum test.

    cFisher-Freeman-Halton test.

    dMultiple selection possible.

    ePerceived severity of acute complaint.

    A multiple linear regression predicting the SUS score was conducted, including sex, age, native German language, health literacy score, media usage duration per day, sickness level of the participants, and number of stated complaints in the app as independent variables (see Table 4). Age, sex, health literacy score, and German native language were significantly associated with SUS score. A higher age (t385=3.30, P=.001) and male sex (t385=1.98, P=.05) were negatively associated with SUS score, whereby a higher health literacy score (t385=2.83, P=.01) and German as a native language (t385=2.51, P=.01) were positively associated with SUS score.

    Table 4. Multiple linear regression predicting the System Usability Scale sum score.
    Variableβ (95% CI)P value
    Male sex (reference=female)–3.21 (–6.40 to –.02).05
    age (per year)–.17 (–.27 to –.07).001
    German not native language (reference=yes)–5.39 (–9.61 to –1.17).01
    Does not use tablet (reference=yes)–1.44 (–4.66 to 1.79).38
    Average daily media usage (reference=<2 h)

    		2-4 h3.21 (–.95 to 7.37).13

    		>4 h.01 (–4.46 to 4.48).99
    Health literacy score (scale 0-9)a1.48 (.45 to 2.51).01
    How sick do you feel? (score 1-5)b–1.05 (–2.92 to .82).27
    Number of stated complaints (1-11)–.98 (–2.11 to .15).09

    aHigher scores indicate a higher level of health literacy.

    bPerceived severity of acute complaint; higher values indicate a higher level of discomfort.

    Differences in Individual Items of the SUS

    Stratified according to sex, age, native language, and tablet usage (see Figure 4), significant differences were detected in SUS items 2 (“unnecessarily complex”), 4 (“need technical support”), 7 (“learn to use quickly”), 8 (“cumbersome to use”), and 10 (“needed to learn a lot”).

    In comparing female and male respondents, all statements were rated more positively by female participants, except for items 1 (“would use frequently”) and 4 (“need technical support”). Female participants also scored significantly higher than male participants on items 2 (“unnecessarily complex”) (mean 7.82 vs 7.11; P=.02), 7 (“learn to use quickly”) (mean 8.11 vs 7.53; P=.05), 8 (“cumbersome to use”) (mean 8.57 vs 8.08; P=.05), and 10 (“needed to learn a lot”) (mean 9.14 vs 8.63; P=.03).

    Respondents aged 65 years and older scored significantly higher on items 2 (“unnecessarily complex”) (mean 7.58 vs 6.50; P=.04), 4 (“need technical support”) (mean 8.72 vs 6.20; P<.001), and 10 (“needed to learn a lot”) (mean 9.05 vs 7.; P<.001) compared to their counterparts.

    German native language speakers scored significantly higher on items 4 (“need technical support”) (mean 8.73 vs 7.75; P=.001), 8 (“cumbersome to use”) (mean 8.62 vs 7.10; P<.001), and 10 (“needed to learn a lot”) (mean 9.11 vs 8.01; P<.001) relative to nonnative speakers.

    Lastly, patients who regularly use a tablet had significantly higher SUS scores on items 4 (“need technical support”) (mean 8.95 vs 8.11; P<.001) and 10 (“needed to learn a lot”) (mean 9.18 vs 8.64; P=.02) in comparison to those of participants who reported reduced levels of tablet use.

    ‎
    Figure 4. Mean System Usability Scale (SUS) score items (see Table 1) stratified according to sex (A), age (B), German as native language (C), and current tablet usage (D). bold: P<.05 (Wilcoxon rank-sum test).

    Principal Findings

    In this study, we evaluated the usability of an app in taking medical histories in general practice directly from patients using the SUS [19].

    The app achieved a mean SUS score of 77.9, which corresponds to a B+ grade on the curved grading scale [27] and represents a “better” product that does not necessarily need improvement [25]. Other medical devices, even those widely used at home, have lower SUS scores. Kortum and Peres [38] assessed the usability of home health care devices among students, thus representing relatively young, healthy, and well-educated participants. SUS scores for these devices ranged from 65 for an epinephrine injector to 67 for a pregnancy test kit and 81 for a thermometer, even with previous experience using these devices.

    To ensure patients can be active participants in the digital medical history–taking process, the app must be easy and intuitive to use without technical introduction or support. This importance is reflected in items 3 (“easy to use”), 4 (“need technical support“), 7 (“learn to use quickly”), and 10 (“needed to learn a lot”). Mean values between 7.8 and 8.9 for these items indicate that intuitive use has been successfully addressed in the development of our app. Item 1, assessing the frequency of app usage, scored the lowest (6.2), which can be explained by the app’s implementation solely in a medical setting and not utilized regularly in leisure time. As such, this finding is the least meaningful for our purpose.

    In a pilot study by Melms et al [39], a self-completed tablet-based digital questionnaire designed for collecting medical histories in an emergency department was found to score high with respect to perceived usability. The design and content were similar to those of our app; however, their questions were only based on the SUS, which does not allow direct comparison. Other comparable instruments, although also for emergency departments, have been tested for usability in pilot studies using self-developed satisfaction surveys [40,41], a single question, and researcher or staff documentation of a patient’s need for assistance [42]. In these studies, patients were mostly satisfied with the self-administered medical history–taking tools and reported good ease of use. Taken together, these results give hope that it is possible to design a medical history app that is perceived as user-friendly.

    Nonetheless, obstacles to implementing a digital tool in general practice settings can be multifaceted. Surprisingly, we found that sex was significantly associated with usability; female participants had significantly higher SUS scores than male participants. The fact that men scored higher than women for item 4 (“need technical support”) suggests that men felt more confident than women with using the app. Previous studies demonstrated that men tend to report overconfidence in their abilities, especially in fields with a male connotation [43], which computer science certainly represents [44]. Therefore, it is unclear whether men really would have needed less help or whether they overestimated themselves in their technical skills.

    Our study suggests that older people are more likely to have difficulties with the handling of such an app. This aligns with a study showing that from the retirement age of 65 years, digital media use among the German population begins to decline dramatically [45] and a positive attitude toward digitalization decreases with increasing age [46]. Older age has a negative impact on the broad usability score given to a user interface [25]. To that end, this study cannot definitively conclude if the older participants of this study actually perceived the app to be of relatively low usability or if their more negative attitude toward the benefit of new technologies prompted them to give lower scores. Due to the small sample size of participants aged 65 years and older, it is not possible to assert how older people in general would cope with the handling of the app. Since GPs are consulted predominantly by older people [47], further research should focus on app testing with older patients to obtain specific feedback, including suggestions for improvement.

    Having learned German as a native language was positively associated with a higher SUS score, although only patients with sufficient German language proficiency were included in the study [17]. This could be due to two different reasons: despite the app’s plain language, it is possible that some of the medical history questions or SUS items were not understood properly.

    Daily media use was not associated with the SUS score, which suggests that the app is designed to also ensure that people with limited digital experience do not feel overstrained with its operation.

    Limitations

    Despite our efforts, this study comes with several limitations. The number of older participants (ie, aged 65 years and above) was relatively low in comparison to their constituents in GP practice settings [47]. One potential reason could be a more pronounced skepticism toward digital tools in older generations, leading to an increase in refusal for participation in the study among older patients. However, as no screening lists were maintained, this is mere speculation. A screening list should be obtained in future studies to be able to characterize individuals who declined participation. Another consideration is that people with lower levels of digital media literacy use may not have agreed to participate in the study.

    Data collection was performed during the SARS-CoV-2 pandemic, which may have disproportionately impacted study participants as certain patient groups may have avoided seeing a doctor or were more likely to refuse to participate in the study to avoid unnecessary contact. This could have included especially vulnerable groups such as older people or those with multimorbid conditions.

    The Likert scale of the SUS questions shown with clickable singular dots was replaced by a slider on December 8, 2021. In the dot-based representation, it was compulsory to make an entry before continuing, whereas the slider was automatically set to the neutral center and could be shifted. This may have led to incorrectly rated items. For example, this may have occurred in instances of the internet faltering or the patient having double-clicked without noticing. Since there were repeated questions about the word “Inkonsistenzen” (SUS item 2), we replaced it by the more common synonym “Unstimmigkeiten” (English translation: discrepancies). Furthermore, hardware as well as the operating system may have influenced the evaluation of the personally perceived usability of a system [48]. For this study, iPad Minis with the iOS operating system were exclusively used. Therefore, possible differences in assessment related to the operating system and hardware are not part of this study.

    The SUS is able to classify the usability of a system but is unable to identify specific usability issues nor capture the usability of the system in its entirety. For a more in-depth usability evaluation, different methods could be used (eg, interviews and observations). During data collection, staff were able to observe usability problems. In their observations, multiple-choice, single-choice, and hybrid questions as well as the slider did not appear to cause any difficulties. In contrast, problems concerning the handling of the app arose when participants were required to input free-text entries (eg, age, height, weight). Further, some study participants were unclear on how to open and close the on-screen keyboard. Some participants also did not understand that the figure on which a pain or an injury could be assigned to a body region (see Figure 1E) could be rotated by clicking on an icon at the bottom left of the screen. This means that, for example, back pain may have been falsely reported as abdominal pain. Lastly, an unstable internet connection arose during data collection, which caused the app to be unresponsive intermittently. These factors may have influenced the SUS score.

    Conclusion

    The app examined in this study for medical history taking passes the usability test based on the SUS and appears to function on par with other digital tools that have become well-integrated in our everyday lives. However, not all people adapted equally well to the app. For successful implementation, all end users, regardless of age, technical affinity, health literacy, or preferred language, must be able to use such a tool. Only if that is attained, providing practical digital solutions can contribute to the efficient and effective delivery of health care services. Therefore, further research should focus on the identification of causes for difficulties of using the app as well as finding appropriate solutions.

    Acknowledgments

    We would like to thank Julie Ngo, MD for proofreading. We sincerely thank all patients for their participation. We are very grateful to the physicians and their teams who allowed data collection in their practices despite the SARS-CoV-2 pandemic and the extraordinary burden that came along for health care workers. We thank the team at aidminutes GmbH for their commitment and support. This research is funded by the German Innovation Fund (funding number 01VSF19050) of the Federal Joint Committee (G-BA). The funders have not influenced the design of this study and did not play any role during its implementation (eg, data collection and analysis, interpretation, and publication of the results).

    Data Availability

    The data sets used and analyzed during the study are not publicly available due to the decision of the research ethics board but can be obtained from the authors upon reasonable request within a data sharing agreement.

    Authors' Contributions

    KA wrote the original draft. KA and EMN were mainly responsible for writing the manuscript. KA, EMN, and DS conceived the study design and analyzed the data. FM and CJ revised the manuscript. All authors have read and approved the final version of the manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Original and German versions of the System Usability Scale (SUS) and the items customized for this study.

    DOCX File , 15 KB
    1. GBE kompakt 3(2) Demografische Alterung und Folgen für das Gesundheitswesen. Robert Koch Institut. 2012. URL: https:/​/www.​rki.de/​DE/​Content/​Gesundheitsmonitoring/​Gesundheitsberichterstattung/​GBEDownloadsK/​2012_2_Demografischer_Wandel_Alterung.​pdf?__blob=publicationFile [accessed 2024-01-02]
    2. Gesundheit in Deutschland. Gesundheitsberichterstattung des Bundes. 09 Welche Auswirkungen hat der demografische Wandel auf Gesundheit und Gesundheitsversorgung? Robert Koch-Institut. 2015. URL: https:/​/www.​rki.de/​DE/​Content/​Gesundheitsmonitoring/​Gesundheitsberichterstattung/​GBEDownloadsGiD/​2015/​09_gesundheit_in_deutschland.​pdf?__blob=publicationFile [accessed 2024-01-02]
    3. Die Bevölkerungsgruppe der älteren Menschen ab 65 Jahren 2021. Statistisches Bundesamt. URL: https:/​/www.​destatis.de/​DE/​Themen/​Querschnitt/​Demografischer-Wandel/​Aeltere-Menschen/​bevoelkerung-ab-65-j.​html [accessed 2022-05-19]
    4. Kürschner N, Weidmann C, Müters S. Who enrolls in a general practitioner model? The behavioral model of health services use and general practitioner-centered care in Germany. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. Feb 2011;54(2):221-227. [CrossRef] [Medline]
    5. Schnitzer S, Balke K, Walter A, Litschel A, Kuhlmey A. Do gatekeeping programs increase equality of health care in Germany? A comparison of the health care situation of participants and nonparticipants. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. Aug 2011;54(8):942-950. [CrossRef] [Medline]
    6. Prütz F, Rommel A, Thom J, Du Y, Sarganas G, Starker A. Inanspruchnahme ambulanter medizinischer Leistungen in Deutschland – Ergebnisse der Studie GEDA 2019/2020-EHIS. J Health Monitor. 2021;6(3):49-71. [CrossRef]
    7. GESUNDHEITSDATEN: Niedergelassene Ärztinnen und Ärzte werden immer älter. Kassenärztliche Bundesvereinigung (KBV). 2022. URL: https://gesundheitsdaten.kbv.de/cms/html/16397.php [accessed 2022-05-19]
    8. Beerheide R, Haserück A. Versorgungsstrukturen: Hausärztemangel angehen. Deutsches Ärzteblatt. 2021;118(38):A-1684 /B-1396.
    9. Schulz M, Czihal T, Bätzing-Feigenbaum J, von Stillfried D. Zukünftige relative Beanspruchung von Vertragsärzten – Eine Projektion nach Fachgruppen für den Zeitraum 2020 bis 2035. Versorgungsatlas. 2016. URL: https://www.versorgungsatlas.de/fileadmin/ziva_docs/67/VA-16-02-Bericht-Final_V3.pdf [accessed 2022-05-19]
    10. Blümel M, Spranger A, Achstetter K, Maresso A, Busse R. Germany: health system review. Health Syst Transit. Dec 2020;22(6):1-272. [FREE Full text] [Medline]
    11. Hampton JR, Harrison MJ, Mitchell JR, Prichard JS, Seymour C. Relative contributions of history-taking, physical examination, and laboratory investigation to diagnosis and management of medical outpatients. Br Med J. May 31, 1975;2(5969):486-489. [FREE Full text] [CrossRef] [Medline]
    12. Roshan M, Rao AP. A study on relative contributions of the history, physical examination and investigations in making medical diagnosis. J Assoc Physicians India. Aug 2000;48(8):771-775. [Medline]
    13. Bundesgesetzblatt Jahrgang 2020 Teil I Nr. 46, ausgegeben zu Bonn am 19. Gesetz zum Schutz elektronischer Patientendaten in der Telematikinfrastruktur/Patientendaten-Schutz-Gesetz (PDSG). Bundesrepublik Deutschland. Oct 2020. URL: https:/​/www.​bundesgesundheitsministerium.de/​fileadmin/​Dateien/​3_Downloads/​Gesetze_und_Verordnungen/​GuV/​P/​PDSG_bgbl.​pdf [accessed 2024-01-02]
    14. Patientendaten-Schutz-Gesetz. Bundesministerium für Gesundheit. URL: https://www.bundesgesundheitsministerium.de/patientendaten-schutz-gesetz.html [accessed 2024-01-02]
    15. Albrecht M, Otten M, Sander M. Praxisbarometer Digitalisierung 2021. KBV. URL: https://www.kbv.de/media/sp/IGES_PraxisBaroDigit2021_Kurzbericht.pdf [accessed 2022-05-19]
    16. Zapata BC, Fernández-Alemán JL, Idri A, Toval A. Empirical studies on usability of mHealth apps: a systematic literature review. J Med Syst. Mar 2015;39(2):1. [CrossRef] [Medline]
    17. Albrink K, Joos C, Schröder D, Müller F, Hummers E, Noack EM. Obtaining patients' medical history using a digital device prior to consultation in primary care: study protocol for a usability and validity study. BMC Med Inform Decis Mak. Jul 19, 2022;22(1):189. [FREE Full text] [CrossRef] [Medline]
    18. Müller F, Chandra S, Furaijat G, Kruse S, Waligorski A, Simmenroth A, et al. A Digital Communication Assistance Tool (DCAT) to obtain medical history from foreign-language patients: development and pilot testing in a primary health care center for refugees. Int J Environ Res Public Health. Mar 20, 2020;17(4):1368. [FREE Full text] [CrossRef] [Medline]
    19. Brooke J. SUS: A 'quick and dirty' usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability Evaluation In Industry. London. CRC Press; 1995;189-194.
    20. Lewis JR. The System Usability Scale: past, present, and future. Int J Hum Comput Interact. Mar 30, 2018;34(7):577-590. [CrossRef]
    21. Ergonomics of human-system interaction – Part 11: Usability: Definitions and concepts. International Organization for Standardization (ISO). 2018. URL: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en [accessed 2022-05-19]
    22. Abran A, Khelifi A, Suryn W, Seffah A. Usability meanings and interpretations in ISO standards. Softw Qual J. 2003;11(4):325-338. [CrossRef]
    23. Kortum P, Hebl M, Oswald FL. Applying usability measures to assess textbooks. Presented at: 58th Human Factors and Ergonomics Society Annual Meeting; October 27-31, 2014, 2014; Chicago, IL. [CrossRef]
    24. Banger A, Kortum P, Miller J. Determining what individual SUS Scores mean: adding an adjective rating scale. J Usability Stud. 2009;4:114-123. [FREE Full text]
    25. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the System Usability Scale. Int J Hum Comput Interact. Jul 30, 2008;24(6):574-594. [CrossRef]
    26. Gao M, Kortum P, Oswald FL. Multi-language toolkit for the System Usability Scale. Int J Hum Comput Interact. Aug 19, 2020;36(20):1883-1901. [CrossRef]
    27. Lewis JR, Sauro J. Item benchmarks for the System Usability Scale. J Usability Stud. 2018;13(3):158-167. [FREE Full text]
    28. Sørensen K, Pelikan JM, Röthlin F, Ganahl K, Slonska Z, Doyle G, et al. HLS-EU Consortium. Health literacy in Europe: comparative results of the European health literacy survey (HLS-EU). Eur J Public Health. Dec 2015;25(6):1053-1058. [FREE Full text] [CrossRef] [Medline]
    29. Sørensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z, et al. (HLS-EU) Consortium Health Literacy Project European. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health. Jan 25, 2012;12:80. [FREE Full text] [CrossRef] [Medline]
    30. HLS-EU CONSORTIUM. Comparative Report of Health Literacy in Eight EU Member States (Second Revised and Extended Version): The European Health Literacy Survey HLS-EU. Jul 22, 2014. URL: https:/​/www.​hsph.harvard.edu/​wp-content/​uploads/​sites/​135/​2015/​09/​neu_rev_hls-eu_report_2015_05_13_lit.​pdf [accessed 2022-05-19]
    31. Schaeffer D, Gille S, Berens E, Griese L, Klinger J, Vogt D, et al. Digital health literacy of the population in Germany: results of the HLS-GER 2. Gesundheitswesen. Apr 14, 2023;85(4):323-331. [CrossRef] [Medline]
    32. Downey RG, King C. Missing data in Likert ratings: a comparison of replacement methods. J Gen Psychol. Apr 1998;125(2):175-191. [CrossRef] [Medline]
    33. Package 'fsmb'. CRAN R project. 2023. URL: https://cran.r-project.org/web/packages/fmsb/fmsb.pdf [accessed 2023-12-29]
    34. Revelle WR. psych: procedures for personality and psychological research. Northwestern University Northwestern Scholars. 2022. URL: https:/​/www.​scholars.northwestern.edu/​en/​publications/​psych-procedures-for-personality-and-psychological-research [accessed 2022-05-19]
    35. tidyr: Tidy Messy Data. URL: https://tidyr.tidyverse.org/ [accessed 2022-05-19]
    36. dplyr: A Grammar of Data Manipulation. URL: https://dplyr.tidyverse.org/ [accessed 2023-12-29]
    37. Wickham H. ggplot2: Elegant graphics for data analysis, second edition. Cham. Springer; 2016.
    38. Kortum P, Peres SC. Evaluation of home health care devices: remote usability assessment. JMIR Hum Factors. Jun 05, 2015;2(1):e10. [FREE Full text] [CrossRef] [Medline]
    39. Melms L, Schaefer JR, Jerrentrup A, Mueller T. A pilot study of patient satisfaction with a self-completed tablet-based digital questionnaire for collecting the patient's medical history in an emergency department. BMC Health Serv Res. Jul 30, 2021;21(1):755. [FREE Full text] [CrossRef] [Medline]
    40. Arora S, Goldberg A, Menchine M. Patient impression and satisfaction of a self-administered, automated medical history-taking device in the Emergency Department. West J Emerg Med. Mar 2014;15(1):35-40. [FREE Full text] [CrossRef] [Medline]
    41. Benaroia M, Elinson R, Zarnke K. Patient-directed intelligent and interactive computer medical history-gathering systems: a utility and feasibility study in the emergency department. Int J Med Inform. Apr 2007;76(4):283-288. [CrossRef] [Medline]
    42. Herrick DB, Nakhasi A, Nelson B, Rice S, Abbott PA, Saber Tehrani AS, et al. Usability characteristics of self-administered computer-assisted interviewing in the emergency department: factors affecting ease of use, efficiency, and entry error. Appl Clin Inform. 2013;4(2):276-292. [FREE Full text] [CrossRef] [Medline]
    43. Jakobsson N, Levin M, Kotsadam A. Gender and overconfidence: effects of context, gendered stereotypes, and peer group. Adv Appl Sociol. 2013;3(2):137-141. [CrossRef]
    44. Friedrich JD, Hachmeister CD, Nickel S, Peksen S, Roessler I. Frauen in Informatik: Welchen Einfluss haben inhaltliche Gestaltung, Flexibilisierung und Anwendungsbezug der Studiengänge auf den Frauenanteil? CHE Arbeitspapier Nr 200. Centrum für Hochshulentwicklung (CHE). Apr 2018. URL: https:/​/www.​uni-bremen.de/​fileadmin/​user_upload/​sites/​zentrale-frauenbeauftragte/​Berichte/​CHE_AP_200_Frauen_in_Informatik.​pdf [accessed 2022-05-19]
    45. Digital skills gap. Initiative D21. 2021. URL: https://initiatived21.de/publikationen/digital-skills-gap [accessed 2022-05-19]
    46. Digitale Teilhabe lässt ab 80 Jahren deutlich nach. Initiative D21. 2021. URL: https://initiatived21.de/digitale-teilhabe-laesst-ab-80-jahren-deutlich-nach/ [accessed 2022-05-19]
    47. Bergmann E, Kalcklösch M, Tiemann F. Public health care utilisation. Initial results of the Telephone Health Survey 2003. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. Dec 2005;48(12):1365-1373. [CrossRef] [Medline]
    48. Calisir F, Cevikcan E, Camgoz AH. Industrial engineering in the big data era. Cham. Springer; 2019.

    ‎
    DASI: digitally assisted information acquisition before medical consultation
    GP: general practitioner
    HLS-GER 2: European health literacy survey adapted for the German language
    SUS: System Usability Scale

    Edited by A Kushniruk; submitted 31.03.23; peer-reviewed by A Simmenroth, R Marshall; comments to author 27.08.23; revised version received 10.10.23; accepted 20.11.23; published 05.01.24.

    Copyright

    ©Klara Albrink, Dominik Schröder, Carla Joos, Frank Müller, Eva Maria Noack. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 05.01.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.