This app, Steady-MS, was developed in Android Studio 3.1.2 and was developed as an extension of a validated fall risk app for older adults, Steady [23]. Modifications were made to the questionnaire, balance tasks, and algorithm of Steady to apply specifically to the MS population. Steady-MS consists of 2 components: the first includes 25 questions targeting demographic information and MS symptoms (Multimedia Appendix 1). These questions include age, sex, past history of falls, type of MS, history of MS, the 12-item Multiple Sclerosis Walking Scale (MSWS-12) [24], and the short form of the Activities Balance Confidence Scale (ABC-6; Figure 1) [25]. These questions were specifically chosen because they are associated with falls in people with MS [17,26-28]. The second component, following the 25 questions, is a series of progressive balance tasks, in which the app guides users through 5 progressively difficult standing balance tasks. In the following order, the tasks are as follows: (1) eyes open, (2) eyes closed, (3) semitandem, (4) tandem, and (5) single leg. A text description and image guide users through each task (Figure 2). Each task takes 30 seconds, beginning with a 5-second countdown and the word start and ending with the word stop. The phone also vibrates at the start and end of each task. Users were instructed to hold the phone against their chest for the duration of the task to measure their postural sway. These tasks were chosen because worse performance on these tasks is associated with falls in people with MS [29,30]. After each task, users were asked to report if they (1) completed the task, (2) attempted but did not complete, or (3) did not attempt. Steady-MS measures postural sway by measuring acceleration in the mediolateral, anteroposterior, and vertical directions [19]. The Romberg ratio, the ratio between eyes open and closed, of the root mean square acceleration measured and recorded as the increased Romberg ratio, is associated with increased fall risk in people with MS [31]. The number of balance tasks completed, the root mean square Romberg ratio, and the responses to the 25-item questionnaire were inputted into a weighted algorithm and converted into a score ranging from 0 to 100, in which higher scores represent a higher risk for falls.

Steady-MS was also developed considering common MS symptoms that may influence usability. For instance, fatigue is a common symptom that affects approximately 70% of people with MS [32]. To prevent fatigue, we limited the total number of questions to 25 questions that were needed for the fall risk algorithm and asked only important additional questions (ie, MS duration and type of assisted device) that relate to falls. We also limited the balance tasks to 1 trial of the 5 tasks. Vision impairments are also an MS symptom affecting approximately 30% of people with MS and may influence reading questionnaires and instructions [33]. Therefore, the font size was at least 14, and we emphasized the high contrast between text and background. Cognitive impairment, including reduced processing speed and memory decline, affects between 40% and 70% of people with MS [34]. We aimed to prevent cognitive overload by presenting one set of instructions per screen and maintaining consistency throughout the app.

A total of 10 people with MS participated in 2 usability rounds. It has been recommended that small groups (n=5) are suitable for identifying usability issues [35]. People with MS (n=5) interacted with Steady-MS and identified usability issues. Using their feedback, we improved the design of the app, and then, another group of people with MS (n=5) interacted with the app to identify any additional usability issues. This iterative design approach centered around the user is most effective for identifying user challenges when developing health apps [21,22]. Inclusion criteria for participants included (1) physician confirmed diagnosis of MS, (2) age 18 years or older, (3) self-reported ability to use a touchscreen device, and (4) ability to stand independently for at least 1 minute. Individuals with neurological disorders other than MS were excluded from the study. All procedures were approved by the Institutional Review Board, and all participants provided informed consent before participation.

An iterative design evaluation process of videotaped semistructured interviews was used to determine the optimal usability of Steady-MS. Participants were presented with a smartphone (Samsung Galaxy S6) and asked to open the app and follow all instructions as they completed both the in-app questionnaires and balance tasks. Participants first completed these steps independently, with as little assistance as possible. They then completed the in-app tasks a second time, but this time thinking aloud and narrating their thoughts. They were also encouraged to discuss their likes, dislikes, and recommendations for improvement. After receiving their fall risk score, participants were also asked to identify and draw different graphics of how they wanted to receive their score, such as on a circular chart or linear scale.

Following the semistructured interview, participants completed the Systematic Usability Scale (SUS) to understand the overall usability of the app. The SUS is widely used to quantify the usability of user-machine interfaces, consisting of 10 standard questions on a 5-point Likert scale [36]. The SUS ranges from 0 to 100, with higher scores representing greater usability. Previous work has indicated that the average technology SUS score is 60, and scores of 80 or above indicate that users are more likely to recommend the device to others [37]. Participants also completed the Mobile Device Proficiency Questionnaire (MDPQ) to understand their general proficiency in using mobile devices. The MDPQ ranges from 5 to 40, with higher scores representing greater technological proficiency [38]. Participants then completed the Expanded Disability Status Scale, a self-reported measure of disability that ranges from 0 to 10, with higher scores indicating greater disability [39].

After the first iterative cycle, changes were made to the app design based on the issues identified from the interviews. The second cycle of semistructured interviews was performed on 5 new participants with MS. Owing to COVID-19 restrictions on in-person research, interviews in the second round were performed remotely. The procedures followed the same format as the first round; however, participants were delivered a smartphone with Steady-MS installed, and interviews were conducted over a video call. This format allowed us to understand how Steady-MS is used in the home environment.

All videotapes and field notes taken during the interviews were transcribed verbatim on a computer. Qualitative data from videotapes and field notes were analyzed to develop a coding system using MAXQDA (Version 12.3.3). On the basis of their content, data were assigned codes, and codes with similar content were grouped into themes. The codes and themes were reviewed and discussed by 2 researchers.

After the second round of interviews, transcript analysis and coding revealed 3 themes related to intuitive navigation, efficiency of use, and perceived value.

In the first iteration, the average SUS score was 95.5, with a standard deviation of 3.3. In the second iteration, the average SUS score was 95.5, with a standard deviation of 2.9. Although the SUS score did not change between iterations, this is likely because of a ceiling effect with a maximum score of 100. These high scores suggest that participants are likely to recommend Steady-MS to others [37].

The purpose of this study is to test and refine the usability of a fall risk health app for individuals with MS through a user-centered design approach. After the first round of usability testing, participants identified issues navigating through the app but reported that it was easy to use and found value in undergoing a fall risk assessment. We modified the app to improve navigation, and after the second round of testing, participants reported that the app was easy to navigate and could use the app on their own. These results, complemented with high SUS scores, suggest that Steady-MS is a usable health app that people with MS can use to self-assess their risk of falling.

Importantly, our results underscore the need for a user-centered design during the development of health apps. Indeed, the main issues identified from semistructured interviews were related to intuitive navigation, and a health app with poor navigation is unlikely to be used. These issues were related to understanding the entire instructions of a balance task (ie, the position of the feet and if eyes are open or closed). Cognitive impairment is a common symptom in people with MS [40], and the instructions for each balance task may result in cognitive overload in people with MS. To reduce cognitive overload, we improved the visuals and text to depict each balance task. Indeed, during the second round of testing, 4 participants completed all balance tasks correctly without asking for clarification. For future developments, it is important to consider the cognitive demands of people with MS to prevent cognitive overload.

Participants in both rounds of testing reported that they found the app clear, simple to use, and useful in learning their risk of falling. This suggests that people with MS can independently use carefully designed health apps such as Steady-MS and learn about their fall risk. Participants also reported that they value receiving a fall risk score because they can identify whether their score improves with exercise or declines with the onset of symptoms. Steady-MS offers the potential for people with MS to self-assess and self-monitor their fall risk using a smartphone. As MS symptoms fluctuate throughout the course of the disease, their risk of falls also changes [4]. Therefore, tracking and monitoring fall risk can help people with MS increase their awareness of their fall risk and take part in prevention strategies before a fall occurs. Unlike traditional fall risk assessments performed in clinics or laboratory-based settings, Steady-MS provides a tool to increase access to fall risk assessments that can be performed at home.

When developing future mHealth apps for people with MS, there are important aspects to ensure high usability. First, it is important to prevent cognitive overload in people with MS, as cognitive impairment is a common symptom of MS [40]. Within Steady-MS, cognitive impairment was found when participants were asked to follow 2 separate instructions for a balance task. Using clear visuals and simple text is important to avoid cognitive overload. Second, when presenting a score or number to people with MS, it is important to ensure that the score is easily understandable. Participants in the study reported that they preferred receiving a number because it was measurable, and they could track improvements over time. However, it is important that people with MS accurately interpret scores. When using a scale from 0 to 100, it was important to depict, both visually and through text, that 0 represents low risk and 100 represents high risk. These 2 guidelines can improve the development of future health apps to maximize the usability of people with MS. Third, using a user-centered, iterative approach in designing Steady-MS resulted in users effectively and efficiently understanding new instructions. This approach may also apply to other clinical populations with physical and cognitive impairments when designing a health app.

This is the first study to develop and test an MS fall risk mHealth app; however, there are also limitations to this study. The participants in this study had high mobile technology use and scored high on the MDPQ. Those with less technology experience may have additional usability issues that were not identified in this study. However, MS commonly affects younger adults, and more than 80% of people with MS own a smartphone [7]. Therefore, it is likely that a person with MS will already have mobile device experience. Additionally, while Steady-MS measures overall fall risk, it currently does not offer fall prevention strategies. Seeking treatment after understanding individual risk is the next step to prevent falls, and future steps should aim to include tailored fall prevention strategies and understand if people with MS adopt these strategies. Future steps should also understand the validity of the algorithm and display the results of individual components that contribute to fall risk. This may help guide people with MS with specific fall prevention strategies. Saving fall risk scores may also help people with MS monitor their changes over time. In addition, future work should aim to understand how fall risk apps such as Steady-MS can be incorporated into clinical care. Although Steady-MS was designed for use at home, integrating fall risk apps with clinical guidance in a safe manner can increase access to fall prevention strategies. This study of 10 participants is also a small sample size, and future steps should include a larger, diverse sample to understand usability needs across the heterogeneous MS population. Finally, although participants reported high perceived value in learning about their fall risk score and offered suggestions to improve how the displaying the score, future interviews, and studies should understand how to present individualized fall risk information to prevent negative, unintended consequences.

In conclusion, the purpose of this study is to determine the usability of a fall risk health app for people with MS. After one round of semistructured interviews, we made modifications to improve users’ intuitive navigation when answering their health-related questionnaires and performing 5 balance tasks. After a second round of interviews, users reported that the app was straightforward to use and easy to navigate and that they found value in learning about their fall risk. SUS scores averaged 95.5 after the second round of testing, suggesting high usability. These results supported the use of a fall risk app to provide people with MS a tool to self-assess and self-manage their fall risk. Moreover, these results underscored the importance of using a user-centered design approach to identify usability challenges when developing mobile apps for individuals with chronic diseases.