Telemedicine has enabled convenient and effective visits; reduced unnecessary testing and referrals; maintained good perception of care; reduced travel costs and caregiver burden; and, not least, helped health care providers to manage an ever-increasing volume of information and relationships [1,2]. Growing use of smartphones and tablets has made mobile health (mHealth) apps promising tools for empowering and engaging people in the self-management of their own health [3]. mHealth tools create opportunities to deliver new forms of health care and to expand services without the need to increase the existing workforce. For example, medical apps can be used within various domains such as wellness management, behavior change, health data collection, disease management, self-diagnosis, and rehabilitation as well as act as an electronic patient portal and medication reminder [4,5], leading to greater time spent at home and fewer medical visits at the center [6]. Furthermore, they represent useful solutions for participants with chronic and progressive diseases, such as multiple sclerosis (MS), that require continuous assistance and care.

mHealth apps promise to offer alternative methods for enhanced real-time data capture to screen for, monitor, and treat the heterogeneous symptoms in MS, thus favoring a substantial transformation in traditional paradigms of medicine [7-13]. However, as the number of mHealth apps increases, the demand for scientific evaluation of these solutions is strongly recommended as well [14]. Despite the growing popularity of mHealth apps, the amount of usability reports does not correlate with the number of published digital health implementation studies [15]. For instance, while Salimzadeh and colleagues [3] found 104 MS-related apps in iTunes (Apple Inc) and Google Play (Google LLC), they noted that there was no corresponding evidence regarding the usability and utility of these solutions in people with MS. To be used in clinical practice, the validity and usability of mHealth tools should be tested. mHealth apps must be designed to ensure good usability, and they must be easy to use and able to reach their goals efficiently. As indicated by Wilson and Lankton [16], perceived ease of use and usefulness affect people’s intention to adopt mHealth devices. Generally, a mobile app is considered to have good usability when (1) it is efficient, (2) users have a positive opinion about the app, (3) it is easy to learn, (4) it is easy to remember even after users have not used it for a while, and (5) it has a low error rate [17].

The mHealth apps can be grouped according to the nature of the interaction between patients and health care providers in the app: interactive and stand-alone mHealth apps. In interactive mHealth apps, users can send and receive information from their health care providers or patients via the app in a synchronous or asynchronous modality. In stand-alone mHealth apps, users enter, collect, or store health information about themselves or other people, which are not directly sent to the user’s health care providers [18].

The most commonly used method for assessing the usability of electronic technologies are questionnaires. General and technology-independent questionnaires such as the System Usability Scale (SUS) [19] and the Post-Study System Usability Questionnaire [20] are usually used in usability studies of mHealth apps [15]. However, these questionnaires were created for general software systems and cannot reliably identify mHealth specific problems that may arise, for example, in health self-management or accessing health care services.

In this context, a new specific usability scale for evaluating the validity of mHealth apps was developed, the mHealth App Usability Questionnaire (MAUQ) [18]. The English version of MAUQ has been translated to various languages such as Malay [21], Chinese [22], Spanish [23], German [24] and French [25]. However, no literature was found reporting a translated version of the questionnaire in Italian, although it was used in 1 study on app usability [26]. Thus, this study aimed to translate and validate the English version of the mHealth App Usability Questionnaire into Italian (ita-MAUQ) in a sample of people with MS.

We specifically hypothesized that the ita-MAUQ would retain acceptable model fit in confirmatory factor analysis, acceptable levels of internal consistency, and test-retest reliability. We also hypothesized an acceptable construct validity, defined based on relations between the ita-MAUQ and another standardized usability scale, and differences in known-groups.

In total, 116 people with MS (female: n=74; mean age 47.2, SD 14 years mean EDSS 3.32, SD 1.72) were enrolled (see Table 1).

Results from CFA of the 18 items-MAUQ order structure, defined by Zhou and colleagues [18], indicated acceptable fit for a 3D scale: χ²/df=2.1, root mean square error of approximation 0.068, and comparative fit index 0.92.

The internal consistency of the overall ita-MAUQ and each subscale was good. The Cronbach α for the ita-MAUQ was 0.92, and those for the 3 subscales were 0.78 (ita-MAUQ_E), 0.89 (ita-MAUQ_I), and 0.87 (ita-MAUQ_U). Similarly, the average interitem correlation was between 0.41 and 0.53. These results align with published satisfactory thresholds for scale reliability [23,24].

To assess the test-retest reliability, 25 participants were required to complete the ita-MAUQ, 2 weeks apart. ICC was 0.84 (95% CI 0.66‐0.92) for ita-MAUQ total score, 0.66 (95% CI 0.36‐0.84) for ita-MAUQ_E, 0.88 (95% CI 0.75‐0.94) for ita-MAUQ_I, and 0.67 (95% CI 0.38‐0.84) for ita-MAUQ_U, showing good or moderate temporal stability.

There were no floor or ceiling effects for the ita-MAUQ total score (0% scored 18 and 6.9% scored 126). The floor effect for ita-MAUQ_E and ita-MAUQ_I subscales was again null, while for ita-MAUQ_U, only 1 participant had the worst possible score of 6 (0.9%). The ceiling effect for ita-MAUQ subscales was 38.8%, 36.2%, and 9.5%, respectively. No other previous study reported the measure of ceiling and floor effects of this instrument [22-24], and therefore a comparison cannot be established.

Spearman coefficients revealed significant correlations between ita-MAUQ total score; ita-MAUQ_E, ita-MAUQ_I, ita-MAUQ_U subscales; and SUS (85.43, SD 14.3; all P values <.05). Table 2 shows the results of the construct validity analysis.

Known-group analysis found no difference between people with MS with mild and moderate EDSS (all P values >.05), suggesting that ambulation ability, mainly detected by the EDSS, did not impact the ita-MAUQ scores (Table 3). Interestingly, statistical differences between female and male participants concerning the ita-MAUQ_E was found (P=.02; Table 4).

Digital solutions as mHealth apps promise to offer alternative methods for enhanced real-time data capture to screen for, monitor, and treat symptoms in MS. These solutions may fundamentally shift traditional paradigms of medicine [7-12]. To be used in clinical practice, the validity and usability of mHealth tools should be tested. Questionnaires are the well-known methods for usability testing, but developing a new one might require concerted effort by the members of a research team, extra cost, and a lot of time [55]. Thus, adaptation of established, appropriate, and available questionnaires with documented validity in other languages is recommended [55].

Thus, the aim of this study was to translate and validate the English version of the ita-MAUQ in a sample of people with MS. Overall, findings demonstrated that the novel translated questionnaire ita-MAUQ is a reliable and valid measurement tool to assess the usability of mHealth apps for people with MS. In this context, people with MS self-administered the ita-MAUQ after interacting with DIGICOG-MS, a novel mHealth app for cognitive self-assessment in MS.

Results indicated that the ita-MAUQ had good internal consistency and stability, as indicated by the Cronbach α coefficient of 0.92. This is in line with the original version of the MAUQ [18] and with other translations of the same questionnaire [23,24,56].

Worldwide, Spearman coefficients between ita-MAUQ total score, subscales, and SUS were statistically significant, proving good criterion and construct validity. However, correlation between the SUS and ita-MAUQ_U was found to be low (0.26). This was in line with the study by Zhou et al [18], in which correlation between MAUQ_U and the SUS was 0.383, reflecting that MAUQ_U is mainly about the usefulness of apps for health care, which is an aspect not covered by the SUS.

Compared to another previous study on MAUQ that did not perform test-retest [24], results revealed good or moderate temporal stability. Given that mHealth apps may allow continuous health care services over time, identifying valid methods to test whether a digital tool is reliable in different measurements is crucial.

The ceiling effect of the ita-MAUQ subscores could indicate that the proposed mHealth app was indeed easy to use for many people with MS. Even though the interaction with DIGICOG-MS was supervised by a neuropsychologist to provide adequate responses to any questions from participants, they were invited to navigate through the mHealth app autonomously, since no specific instructions on how to use other app functionalities were given. In this way, we interpreted this ceiling effect positively as a successful interaction.

In general, the known-groups validity of the instrument was shown by the comparison of ita-MAUQ total and subscale scores between people with MS with different disability levels. The results indicate no significant difference in people with MS with mild and moderate EDSS. Since EDSS mainly measured ambulation capacity, this suggests that such mHealth apps could be found usable for both people that need assistance during ambulation and those who are able to walk without any aid or support.

Concerning gender, earlier studies found that perceived ease of use and usefulness technology may differ by gender [5,52]. Our results are in line with a previous study that demonstrated that gender was associated with higher usability scores in female than male participants [57]. Interestingly, male and female people with MS had different scores in ita-MAUQ_E, suggesting that women are more likely than men to be influenced by effort expectancy and facilitating conditions [53]. These results can help developers to enhance the usability of their services for all users with different personal and clinical characteristics, since men and women still have different traits and societal roles, which may affect their perceptions and usage of technologies.

Our study has several limitations. First, as a single-center study, participants’ characteristics may limit the interpretation of our results. The study sample may be considered representative of those clinic-attending people with MS followed as outpatients in rehabilitation centers (ie, middle-age or older adults and with a longer disease duration) [58]. Thus, results may not generalize to other populations of participants with MS (eg, young and neo-diagnosed people). Second, in this validation study, we used DIGICOG-MS that was designed and developed to assess a specific symptom in MS, that is cognitive impairment. Given the high frequency of cognitive impairment in people with MS, it is reasonable to conclude that study participants who have experienced such a disturbing symptom tended to appreciate the mHealth app more compared to people with other neurological diseases. People with MS with cognitive impairment might find such digital tools particularly beneficial due to their potential to offer structured cognitive training and monitoring, which can enhance their daily functioning and quality of life [59]. Thus, it cannot be ruled out that the results of the ita-MAUQ validation would have been different with a clinical population with dissimilar characteristics.

Third, this study was conducted in a controlled clinical setting, allowing participants to familiarize themselves with both the novel technology and usability questionnaires’ items. While this approach may have some limitations in terms of generalizability to real-life scenarios and may influence participants’ engagement and perceived usability, having a facilitator available to assist, if needed, people with MS that could have problems with reading and interpreting the questionnaire items ensured they understand each question before responding. Furthermore, in this study we overlooked other key factors that may influence usability, such as assessing the importance of providing adequate training and continuous support to users or the role of previous experience with a similar technology; investigating how interface design that considering layout, navigation, and accessibility features makes the app user-friendly for participants with cognitive impairments; evaluating the concerns and preferences of users regarding data security and privacy, which are crucial for building trust and ensuring compliance with health regulations data security and privacy. Here, we did not collect additional feedback after the completion of both digital assessment and usability questionnaires from people with MS. Further study should include feedback sections where participants can indicate if they found any items difficult to understand, allowing for continuous improvement of the novel tools. Worldwide, incorporating these factors into future research can lead to the development of more effective, user-friendly, and impactful mHealth apps for people with MS.

In conclusion, the ita-MAUQ demonstrated high reliability and validity, and it might be used to evaluate the usability, utility, and acceptability of mHealth apps in people with MS. This finding is in line with previous validation of the MAUQ in different languages as Malay [21], Chinese [22], Spanish [23], German [24], and French [25], further confirming the cross-cultural validity, reliability, and adaptability of the MAUQ.