Abstract
Background: The Swedish health care system is undergoing a transformation. eHealth technologies are increasingly being used. The System Usability Scale is a widely used tool, offering a standardized and reliable measure for assessing the usability of digital health solutions. However, despite the existence of several translations of the System Usability Scale into Swedish, none have undergone psychometric validation. This highlights the urgent need for a validated and standardized Swedish version of the System Usability Scale to ensure accurate and reliable usability evaluations.
Objective: The aim of the study was to translate and psychometrically evaluate a Swedish version of the System Usability Scale.
Methods: The study utilized a 2-phase design. The first phase translated the System Usability Scale into Swedish and the second phase tested the scale’s psychometric properties. A total of 62 participants generated a total of 82 measurements. Descriptive statistics were used to visualize participants’ characteristics. The psychometric evaluation consisted of data quality, scaling assumptions, and acceptability. Construct validity was evaluated by convergent validity, and reliability was evaluated by internal consistency.
Results: The Swedish version of the System Usability Scale demonstrated high conformity with the original version. The scale showed high internal consistency with a Cronbach α of .852 and corrected item-total correlations ranging from 0.454 to 0.731. The construct validity was supported by a significant positive correlation between the System Usability Scale and domain 5 of the eHealth Literacy Questionnaire (P=.001).
Conclusions: The Swedish version of the System Usability Scale demonstrated satisfactory psychometric properties. It can be recommended for use in a Swedish context. The positive correlation with domain 5 of the eHealth Literacy Questionnaire further supports the construct validity of the Swedish version of the System Usability Scale, affirming its suitability for evaluating digital health solutions. Additional tests of the Swedish version of the System Usability Scale, for example, in the evaluation of more complex eHealth technology, would further validate the scale.
Trial Registration: ClinicalTrials.gov NCT04150120; https://clinicaltrials.gov/study/NCT04150120
doi:10.2196/64210
Keywords
Introduction
In the rapidly evolving landscape of global health care, the advent of eHealth technologies has emerged as a transformative force that promises innovative solutions to the multifaceted challenges faced by health care systems worldwide [,]. The Swedish health care system is currently transforming along these lines. The use of digital applications and other digital contact methods, collectively described under the term eHealth, is increasing. The World Health Organization (WHO) defines eHealth as “the use of information and communication technologies (ICT) for health.” These technologies include a wide range of systems interventions, applications, and devices such as mobile health and telehealth []. There is compelling evidence for the increasing influence of eHealth on the provision of health care globally today and how it is enhancing the efficiency and responsiveness of health systems to meet people’s needs and expectations []. It is essential to ensure that health technologies are designed appropriately to meet the needs of end users before deploying them as health interventions []. Employing robust evaluation methods to ensure high-level usability has been recognized as a crucial component of good practice for achieving this goal [].
Usability is defined as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” []. To determine the potential usability of a digital solution, the System Usability Scale (SUS) has been widely adopted as a standardized evaluative device. In 1996, Brooke [] published the SUS as an instrument that could easily measure usability. Since then, it has been used to evaluate computer systems, applications, and other digital solutions in a wide range of areas [-]. The SUS is a questionnaire, consisting of 10 items each scored on a 5-point Likert scale from “strongly disagree” to “strongly agree.” It is easy to administer and has been shown to generate results with good reliability and validity [,,]. The instrument is free, and no fees are required to use it []. With an international reach, it is the most widely used standardized questionnaire for the assessment of perceived usability today []. The SUS can be used to evaluate a wide range of usability in products, including digital applications, mobile devices, and web pages [-].
However, as psychometric properties are sample dependent, it is essential to evaluate the psychometric properties when using patient-reported outcome measure in new settings or populations []. The SUS has been translated into numerous languages such as Chinese [], Finnish [], French [], Hindi [], Indonesian [], and Polish []. It has undergone psychometric validation [], including in Arabic [], Danish [], Dutch [], German [], Italian [], Malay [], Persian [], Portuguese [], Slovene [], and Spanish []. The psychometric properties that have resulted from these studies show that adapted versions of the SUS are a reliable tool for usability assessments. However, a number of these studies adopted a general focus and examined only the total sum in the test. Only a small number of studies have tested the instrument at an item level [-], with none of them in Swedish. This emphasizes the need for a comprehensive testing of a Swedish version of the SUS on an item level. Determining the robustness is critical for ensuring that the measurement instrument has sufficient validity in the proposed context.
Lewis [] outlines the essential requirements for translating the SUS into multiple languages and conducting validations across diverse countries. In Sweden, one translation of the SUS has been published in a scientific journal [], although several unpublished versions exist []. However, the linguistic discrepancies between these translations give room for ambiguities, and a united translation with a rigorous psychometric testing in a Swedish context is warranted.
In 2016, an initial vision was for Sweden to be “the best in the world” in eHealth by 2025. However, a more realistic view today is that Sweden has made significant progress in this area. Within the European context, Sweden demonstrates a distinct approach to digitalization in health care, emphasizing collaboration and innovation to address specific challenges and opportunities [].
With this in mind, there is a great need for a nationally united SUS that has been rigorously tested and proven effective. A robust process of evaluating the psychometric properties of a Swedish SUS will foster participatory usability research and ultimately improve the quality of health care services on a broader scale. Although there are different Swedish translations of the SUS, there is still a lack of psychometric testing showing their robustness. Before any Swedish SUS can be recommended for use, both translation and psychometric evaluation of the instrument are necessary. Therefore, the aim of this study was to translate and psychometrically evaluate a Swedish version of the SUS.
Methods
Overview
This study consists of 2 steps. In step 1, the SUS was translated and adapted into Swedish. In step 2, the psychometric properties of the Swedish version were tested: data quality, scaling assumptions, acceptability, convergent validity, and internal consistency. The translation, adaptation process, and psychometric evaluation adhered to the COSMIN (Consensus-Based Standards for the Selection of Health Status Measurement Instruments) checklist [].
The SUS Instrument
The SUS instrument consists of 10 items (statements). The items are divided equally into positively and negatively worded statements. Requests for a response are graded on a scale of agreement ranging from strongly disagree (1) to strongly agree (5). The score is calculated as follows: for positively worded items (1, 3, 5, 7, and 9), the score is the position on the scale (1-5) minus 1, and for negatively worded items (2, 4, 6, 8, and 10), the score is the position on the scale minus 5. Individual item scores, therefore, can range from 0‐4. The sum of all 10 scores is then multiplied by 2.5, resulting in a total score ranging from 0‐100, with higher numbers representing greater usability [,].
To date, factor analysis has not been able to show conclusively whether the SUS consists of one factor (usability) or two (usability and learnability) []. Efforts to replicate these findings have led to the conclusion that addressing the instrument as 2 dimensional has no practical or theoretical interest. This study, therefore, treats the SUS as a unidimensional instrument of perceived usability [].
Translation and Adaption
According to Brooke’s [] original formulation of the SUS, no formal permission is needed for translation, and it can be used free of charge. Further, Brooke [] allows for the possibility to, in any version, exchange the wording of the scale to a word or expression suitable for the situation. The translation process adopted in this study was inspired by Beaton et al []. First, the original instrument was translated from English to Swedish by one of the researchers (CC), proceeding from the versions presented by Bangor et al [] and Lewis []. During the process, perceived difficulties and uncertainties were noted. This first version was then reviewed and discussed by the group, consisting of researchers familiar with eHealth, until consensus was reached. Second, an authorized translator, naive to the research field, carried out a back translation on the instrument and the notes taken during the research process. Finally, the research group reviewed all versions of the translation, with all notes attached, and finalized a second version of a Swedish SUS. This version was then compared to other SUS translations within the Scandinavian countries for content validity and the original version for expressions and conceptions that could have been culturally influenced. Following this last step, the group then decided upon a final version by consensus. The response options were structured in the same way as the original questionnaire, although the phrases “strongly disagree” and “strongly agree” were exchanged for the Swedish equivalents of “totally disagree” and “totally agree.”
Psychometric Evaluation
Sample
The evaluation of the psychometric properties was carried out in conjunction with a larger intervention study evaluating eHealth, the eChildHealth tablet study []. Parents of children with a range of illnesses and health conditions who were patients in the pediatric department of a level-3 hospital in the south of Sweden were invited to take part. A total of 66 parents to 52 unique children gave informed consent and were included in the eChildHealth tablet study. Of these parents, 62 provided information on the SUS, resulting in a total of 82 measurements.
Data Collection
Parents were introduced to an app on a tablet computer through which they could communicate with health care staff after their child had been discharged from hospital. The app made it possible to continue to communicate with hospital staff whom the parents knew well, through chat messages, sending photos, video calls, and predesigned questionnaires. Using a questionnaire, data on various aspects of eHealth were collected for each parent. The SUS was one aspect, and the eHealth Literacy Questionnaire (eHLQ) [] was another. A study-specific questionnaire was used to collect demographic data such as age and level of education. Between October 2022 and October 2023, the 66 parents were included in study. Data were collected after 1‐2 weeks of use and at a second time point for those participants who used a tablet for more than 1 month. These parents constitute the eligible participants for this study. A sufficient adequate sample size of approximately 80 measurements was based on recommendations from COSMIN [] and Beaton et al [].
Data Analysis
The psychometric properties of the SUS were analyzed with IBM SPSS Statistics for Windows (version 28.0, IBM Corp). Descriptive statistics (mean, SD, and percentage) were used to visualize participants’ characteristics along with data quality, scaling assumptions, and acceptability. Construct validity was evaluated by convergent validity, and reliability was evaluated by internal consistency.
For data quality, use within the clinical setting was determined by item nonresponse and missing scale scores, as they reflect the acceptance and understanding of a measure []. Data quality was determined as high if the percentage of missing data per item was low (<10% acceptable). In this study, there were missing data in 5 items (ranging from 1/81, 1% to 3/81, 4%), representing high data quality. Participants with more than 3 unanswered questions were excluded (n=1), while mean imputation was carried out for those with 1 or 2 missing items (n=6, all missing 1 item) []. This resulted in a total of 81 measurements being included in the final analysis.
Regarding scaling assumptions, the dimensionality of the SUS has been evaluated previously []. In accordance with these studies, this study assumes that the SUS is to be treated as unidimensional (all items measure the same construct). Instruments composed of Likert-scale items can be summarized if they have similar means and SDs. Furthermore, item-total correlations (the correlation between each individual item score and the total score) would indicate if all items contribute equally to the total score. In line with Hobart et al [], item-total correlations with values of r≥0.3 were regarded as sufficient for summing up the items to a total sum score.
To evaluate the acceptability of score distributions, ceiling and floor effects along with skewness were calculated. Ceiling and floor effects were regarded as present if they exceeded 90%, that is, the percentage of responses for the lowest and highest scores. Skewness statistics should preferably be within the range of −1 to +1 [].
Construct validity was explored through convergent validity. It was evaluated with the correlation between the high total sum of the SUS and one of the domains of the eHLQ [], with the hypothesis that it would correlate with the total score of the SUS. Both instruments were distributed to the parents at the same time points. The eHLQ has already been translated, adapted, and validated within a Swedish context []. The eHLQ consists of 7 domains across 35 items, with each domain being extractable and treatable as a separate scale. Responses to each domain on the eHLQ are recorded on a Likert scale from 1 (strongly disagree) to 4 (strongly agree). The eHLQ was designed to be used to understand and evaluate people’s interaction with digital health services [,]. The research group independently reviewed the eHLQ and, after discussion, decided to use domain 5, measuring the motivation to engage with digital services. The correlation between the instruments was assessed using the nonparametric correlation coefficient of Kendall Tau-b. With regard to the comparative instrument not being used in total (only 1 domain), the correlation was regarded as acceptable if it was moderate or greater (>0.3) [] and with the level of significance being P<.05.
Internal consistency—how items are related to each other—was explored according to the indicators recommended by Hobart et al []: corrected item-total correlations and Cronbach α. The cutoffs were >.04 for acceptable corrected item-total correlations and >.8 for Cronbach α [,]. The internal consistency was completed with an SEM to analyze the measurement error of the instrument. The SEM represents the smallest difference in scores and indicates a change on a group level. SEM was analyzed with SDbaseline × √1 – reliability and complemented with a CI 95% [].
Ethical Considerations
This study was conducted in accordance with the Declaration of Helsinki [] and approved by the Swedish Ethical Review Authority (2021‐05077). The invited parents were recruited through gatekeepers at the pediatric department. Information regarding the study was initially provided orally and then followed up by written information, before written informed consent was obtained by a study nurse. Data were handled confidentially, and participating parents were able to quit without any explanation or impact on the care their children received. Participation in the study was voluntary, and no financial or other form of compensation was provided to the participants.
Results
Translation and Adaption
Both translations (forward and backward) were similar and did not differ substantially. Overall, the conformity of phrasing was high, with some discrepancy for phrasing in statements 2 (I found the system unnecessarily complex) and 8 (I found the system very cumbersome to use). In statement 2, the word “complicated” was suggested in the back translation from the Swedish word “komplex.” In statement 8, the word “awkward” was suggested instead of the original word “cumbersome.”
All such discrepancies were discussed within the research group and a consensus was reached for the final version. The final step, comparing the Swedish version to other Scandinavian versions, generated no further changes. 
Table 1 shows the original English version and the final Swedish version of the SUS.
| Component | English version | Swedish version |
| Response alternatives |
|
|
| Statements |
|
|
aIn this study, the Swedish instruction was as follows: "Markera det alternativ som bäst beskriver din reaktion för applikation i surfplatta idag," meaning “Please indicate your agreement with the following statements, one at a time.”
Psychometric Evaluation of the Swedish Translation
A total of 62 individuals were included in this study. Of these, 20 answered the questionnaire, including the SUS, twice. This resulted in a total of 81 measurements included in the analysis. Demographic data for the participants are shown in Table 2.
| Demographic data | Values | |
| Age (years), median (range) | 33 (22‐52) | |
| Gender, n (%) | ||
| 41 (66) | ||
| 20 (32) | ||
| 1 (2) | ||
| Marital status, n (%) | ||
| 33 (53) | ||
| 28 (45) | ||
| 1 (2) | ||
| Education level, n (%) | ||
| 18 (29) | ||
| 42 (68) | ||
| 2 (3) | ||
| Born in Sweden, n (%) | ||
| 52 (84) | ||
| 6 (10) | ||
| 4 (6) | ||
| First language | ||
| 56 (90) | ||
| 6 (10) | ||
Data Quality
The percentage of missing data per item was low (ranging from 0/81, 0% to 3/81, 4%) across all items. A tendency could be seen for a higher percentage within the 3 highest steps of the scale, resulting in 5 items having 0% in the lowest points of the 5-point scale (Table 3).
| Item | Missing data, n (%) | Item frequency distribution, n (%) | ||||
| 0 | 1 | 2 | 3 | 4 | ||
| 1 | 0 (0) | 1 (1) | 5 (6) | 21 (26) | 36 (44) | 19 (23) |
| 2 | 1 (1) | 0 (0) | 4 (5) | 16 (20) | 9 (11) | 52 (63) |
| 3 | 0 (0) | 1 (1) | 0 (0) | 10 (12) | 24 (29) | 47 (58) |
| 4 | 2 (2) | 0 (0) | 0 (0) | 6 (7) | 16 (20) | 58 (71) |
| 5 | 0 (0) | 2 (2) | 2 (2) | 21(26) | 32 (39) | 25 (31) |
| 6 | 3 (4) | 2 (2) | 3 (4) | 21 (25) | 13 (16) | 40 (49) |
| 7 | 1 (1) | 0 (0) | 1 (1) | 4 (5) | 34 (42) | 42 (51) |
| 8 | 2 (2) | 1 (1) | 1(1) | 5 (6) | 10 (13) | 63 (77) |
| 9 | 0 (0) | 0 (0) | 0 (0) | 6 (7) | 28 (34) | 48 (59) |
| 10 | 0 (0) | 0 (0) | 1 (1) | 4 (5) | 14 (17) | 63 (77) |
Scaling Assumptions
Item means ranged from 2.82 to 3.70 (Table 4). The item-total correlations showed that each item contributed substantially to the total score with correlations ranging from 0.454 to 0.731 (Table 5), thus indicating that the scale can be summarized. The total sum for the SUS in the data ranged between 50 and 100 (mean 84, SD 13).
| Item | Score, mean (SD) | Skewness | |
| 1 | 2.82 (0.904) | −.550 | |
| 2 | 3.35 (0.964) | −1.094 | |
| 3 | 3.41 (0.800) | −1.490 | |
| 4 | 3.65 (0.618) | −1.585 | |
| 5 | 2.93 (0.940) | −.766 | |
| 6 | 3.09 (1.076) | −.876 | |
| 7 | 3.44 (0.652) | −1.038 | |
| 8 | 3.66 (0.762) | −2.682 | |
| 9 | 3.51 (0.633) | −.943 | |
| 10 | 3.70 (0.622) | −2.203 | |
| Item | Scale mean if item deleted | Corrected item-total correlations | Cronbach α if item deleted |
| 1 | 30.80 | 0.463 | .847 |
| 2 | 30.30 | 0.731 | .820 |
| 3 | 30.22 | 0.534 | .840 |
| 4 | 30.00 | 0.657 | .833 |
| 5 | 30.72 | 0.479 | .847 |
| 6 | 30.57 | 0.463 | .852 |
| 7 | 30.21 | 0.454 | .846 |
| 8 | 29.99 | 0.687 | .827 |
| 9 | 30.12 | 0.675 | .831 |
| 10 | 29.96 | 0.585 | .837 |
Acceptability
As indicated by the item mean score (range 2.82-3.70; Table 4) and item-frequency distribution (Table 3), the instrument showed acceptability. For 3 items (4, 8, and 10), the item frequency was above 70%, which is still within the acceptable range for the absence of a ceiling effect. Skewness statistics were below or near the acceptable range of −1 for a total of 7 items, thus indicating that the distribution was excessively skewed (Table 4).
Convergent Validity
Convergent validity was evaluated with the correlation between the total sum of the SUS and the total sum of domain 5 in the eHLQ. As expected, there was a positive correlation between the total score of the instruments (correlation coefficient 0.305), which was significant (P=.001) and supported the construct validity of the Swedish version of the SUS.
Internal Consistency and Measurement Error
Cronbach α for the scale was .852. Corrected item-total correlations were between 0.454 and 0.731, as shown in Table 5, indicating internal consistency for the different items. For all items except item 6 (I thought there was too much inconsistency in this system), the α value if the item deleted was lower than the Cronbach α for the scale. The SEM was 5.05 (95% CI –4.84 to 14.954) points for the Swedish version of the SUS.
Discussion
Principal Findings
This study presents a new Swedish version of the SUS that is psychometrically tested. This study seeks to establish the new Swedish version of the SUS as a reliable and valid instrument for assessing system usability. Overall, the psychometric testing showed high data quality, good scaling assumptions, high internal consistency, and fair convergent validity. Together, these analyses support the validity and reliability of the new Swedish version of the SUS.
The translation process of the new Swedish version of the SUS was executed incrementally, both within and outside the research team, involving an authorized translator who conducted a back translation. This method facilitated thorough scrutiny and comparison of the translation from multiple perspectives. The approach also helped reduce the risk of bias and improved the scale’s validity by incorporating multiple viewpoints [].
The psychometric testing regarding internal consistency showed that the Cronbach α values were satisfactory and that all items contributed to the instrument’s total score. This indicates that the Swedish translation of the SUS is a stable instrument to use in a Swedish context.
Convergent validity showed fair correlation between the SUS and domain 5 of the eHLQ, which measured the motivation to engage with digital services (I find that digital technology support me in taking care of my health). This supported our hypothesis that a generally positive attitude toward digital solutions would correlate with a high usability score. This hypothesis is also in line with previous research, which shows that positive expectations directed toward a product generate positive subjective usability ratings [].
There was an indication of a ceiling effect for 3 items on the Swedish version of the SUS (items 4, 8, and 10). They presented above 70% in item frequency distribution and a high mean total score, which is in line with studies indicating a generally high score for the SUS []. Previous studies have also shown ceiling effects of the SUS but only for the total score [,], thus the responses on the different items can not be justly compared between studies. This study, however, explores the SUS in much more detail and in line with recommended psychometric evaluations, as it explores the psychometric properties for each item separately [,]. Further, the distribution was excessively skewed. This could be an expected result of the nonnormal distribution, since negatively skewed variables are assumed to have a ceiling effect [,]. Also, the robustness of skewed distribution as the sole indicator of ceiling effect has been questioned [].
However, based on the skewness result of the items, the question arises as to whether the range of the response options is wide enough (5-point scale) and if the wording in the response options (“strongly disagree” to “strongly agree”) is a sufficient description. It could be interesting to expand the number of response options but, arguably, there could be obstacles with revising such a widely used and widespread instrument.
Regarding the high mean scores for both the total sum and the items of the SUS, it should be noted that the population had a mean age of 33 years and are, therefore, used to digital solutions. This could have influenced their experience of the app. Therefore, the high item scores of the SUS could also indicate a product that is perceived to have usability for this group. A recommendation for future studies would be to test the instrument in other contexts and with different age groups. This app was designed to support users who were in exactly the same situation as the participants in this study: parents in a specific situation. Going home with a child after hospitalization can be stressful; the parents in this study reported this eHealth solution as supportive []. The SUS has previously been tested for its sensitivity in different digital solutions and has shown a high scoring (ceiling effect) within the best-of-class products [].
The Swedish government has envisioned Sweden as a leading country of eHealth in the near future. It has declared the need for individual users to act as the cocreators of such solutions []. Within eHealth, there has already been a call for a participatory design to increase the equality of access to digital solutions []. Even though digital technology and eHealth are intended to enhance access to health care, for example, despite geographical distances. It may also result in the opposite, as people have different knowledge in using digital solutions. For example, younger people tend to use and access digital solutions differently than older people, which is why solutions should be adapted to the intended end users []. In this study, the population was younger, as previously discussed, which could have influenced the result. To have a project design of cocreation often means that end users are involved from the start of the project as collaborators. Coproduction, on the other hand, often involves the end users during the implementation phase []. Arguably, the end users are invaluable in all steps of the process, and the usability of products needs to be evaluated where instruments such as the SUS can be useful []. This study could therefore be regarded as enhancing the possible participation of end users in the future development of digital solutions and eHealth in a Swedish context.
In conclusion, this study indicates that this developed and psychometrically tested Swedish version of the SUS can be recommended for use within the Swedish adult context.
Limitations
There are some methodological challenges in this study. First, the sample could be regarded as small (N=62), although it was sufficient according to the recommendations [,]. A larger sample might enable different methods of analysis, such as Rasch analysis, which might offer deeper psychometric insights.
Second, the 1-year interval for data collection could be considered long in a rapidly changing world. This length of time could influence how the app was perceived. That said, the digital product was not changed to any great extent during this period, so the perception of usability should not have been influenced to any great extent. Additional data collection was carried out after COVID-19 restrictions were essentially withdrawn in Sweden. There is, however, always the possibility of perceptions fluctuating, as the product could be used in various ways according to the circumstances of the parents.
Third, there was an intention to pursue a data quality test for reproducibility as some of the participants answered the SUS twice. Regrettably, the test-retest sample (n=20) was regarded as too small in this study, and such analyses need to be considered in future studies. In addition, the use of imputation can be discussed. There was a low number of missing data, with a noncomplete frequency of only 9 responses in total. The percentage of missing data per item was also low across all items. This resulted in only 6 imputed scores across the items, and based on mean score for the item, this was regarded as not influencing the result noticeably.
Conclusions
This study presents a new Swedish version of the SUS. It is the first study to carry out a psychometric evaluation of a Swedish SUS, to establish the Swedish version of the SUS as a reliable and valid instrument for assessing system usability. Overall, the psychometric testing showed high data quality, good scaling assumptions, high internal consistency, and fair convergent validity, all of which support the validity and reliability of the new Swedish version. The results from this study are promising. They raise the possibility that the Swedish version of the SUS could be used to evaluate digital health solutions. To further strengthen the usability of the scale, we suggest additional analysis on data that evaluate more complex eHealth technology and include a wider participant age group, both younger and older.
Acknowledgments
We would like to express our sincere gratitude to the participants in the study. We would like to acknowledge Inger Kristensson Hallström, who passed away in 2024, for her extensive work, deep commitment to the field of pediatric nursing, and expertise dedicated to the eChildHealth program. Additionally, we wish to express our appreciation to Magnus C. Persson, for the design of the questionnaires in REDCap (Research Electronic Data Capture) and assisting in data transfer, as well as Desirée Lemmen and other gatekeepers for their assistance with participant inclusion and the distribution of questionnaires to the study participants. This study was funded by the Swedish Research Council for Health, Working Life and Welfare, Sweden (grant 2018-01399).
Authors' Contributions
All authors have made substantial contributions to all of the following: (1) the conception and design of the study, the acquisition of data, or the analysis and interpretation of data; (2) the drafting of this paper or its critical revision for important intellectual content; and (3) final approval of the manuscript version to be submitted.
Conflicts of Interest
None declared.
References
- Jacob C, Lindeque J, Klein A, Ivory C, Heuss S, Peter MK. Assessing the quality and impact of eHealth tools: systematic literature review and narrative synthesis. JMIR Hum Factors. Mar 23, 2023;10:e45143. [CrossRef] [Medline]
- Lennon MR, Bouamrane MM, Devlin AM, et al. Readiness for delivering digital health at scale: lessons from a longitudinal qualitative evaluation of a national digital health innovation program in the United Kingdom. J Med Internet Res. Feb 16, 2017;19(2):e42. [CrossRef] [Medline]
- Digital health. World Health Organization. 2024. URL: https://www.who.int/westernpacific/health-topics/e-health [Accessed 2024-07-01]
- Brown W III, Yen PY, Rojas M, Schnall R. Assessment of the Health IT Usability Evaluation Model (Health-ITUEM) for evaluating mobile health (mHealth) technology. J Biomed Inform. Dec 2013;46(6):1080-1087. [CrossRef]
- Zapata BC, Fernández-Alemán JL, Idri A, Toval A. Empirical studies on usability of mHealth apps: a systematic literature review. J Med Syst. Feb 2015;39(2):1. [CrossRef] [Medline]
- Dix A. Human-Computer Interaction. Pearson; 2003. URL: https://www.pearson.com/en-gb/subject-catalog/p/human-computer-interaction/P200000008804/9780130461094 [Accessed 2025-01-07] ISBN: 0130461091
- Brooke J. SUS: a “quick and dirty” usability. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability Evaluation In Industry. CRC Press; 1996:189-194. URL: https://www.taylorfrancis.com/chapters/edit/10.1201/9781498710411-35/sus-quick-dirty-usability-scale-john-brooke [Accessed 2025-01-07] ISBN: 9780429157011
- Brooke J. SUS: a retrospective. J User Exp. 2013;8(2):29-40. URL: https://uxpajournal.org/sus-a-retrospective/ [Accessed 2025-01-07]
- Hägglund M, Scandurra I. User evaluation of the Swedish patient accessible electronic health record: System Usability Scale. JMIR Hum Factors. Jul 27, 2021;8(3):e24927. [CrossRef] [Medline]
- Lewis JR. The System Usability Scale: past, present, and future. Int J Hum-Comput Interact. Jul 3, 2018;34(7):577-590. [CrossRef]
- Lewis JR, Sauro J. The factor structure of the System Usability Scale. In: Kurosu M, editor. Human Centered Design. HCD 2009. Lecture Notes in Computer Science, vol 5619. Springer; 2009:94-103. [CrossRef]
- Hobart J, Cano S. Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods. Health Technol Assess. Feb 2009;13(12):iii, ix-x, 1-iii, ix-x177. [CrossRef] [Medline]
- Wang Y, Lei T, Liu X. Chinese System Usability Scale: translation, revision, psychological measurement. Int J Hum-Comput Interact. Jun 14, 2020;36(10):953-963. [CrossRef]
- Raita E, Oulasvirta A. Too good to be bad: favorable product expectations boost subjective usability ratings. Interact Comput. Jul 2011;23(4):363-371. [CrossRef]
- Gao M, Kortum P, Oswald FL. Multi-language toolkit for the System Usability Scale. Int J Hum-Comput Interact. Dec 13, 2020;36(20):1883-1901. [CrossRef]
- Sharfina Z, Santoso HB. An Indonesian adaptation of the System Usability Scale (SUS). 2016. Presented at: 2016 International Conference on Advanced Computer Science and Information Systems (ICACSIS):145-148; Malang, Indonesia. [CrossRef]
- Borkowska A, Jach K, Borzemski L, Grzech A, Świątek J, Wilimowska Z, editors. Pre-testing of Polish translation of System Usability Scale (SUS). In: Information Systems Architecture and Technology: Proceedings of the 37th International Conference on Information Systems Architecture and Technology–ISAT 2016–Part I. Advances in Intelligent Systems and Computing, vol 521. Springer; 2017:143-153. [CrossRef] ISBN: 9783319465838
- Streiner DL, Norman GR, Cairney J. Health Measurement Scales: A Practical Guide to Their Development and Use. Oxford University Press; 2024. URL: https://global.oup.com/academic/product/health-measurement-scales-9780199685219?cc=us&lang=en& [Accessed 2025-01-07] ISBN: 9780199685219
- AlGhannam BA, Albustan SA, Al-Hassan AA, Albustan LA. Towards a standard Arabic System Usability Scale: psychometric evaluation using communication disorder app. Int J Hum-Comput Interact. Sep 2, 2018;34(9):799-804. [CrossRef]
- Hvidt JCS, Christensen LF, Sibbersen C, Helweg-Jørgensen S, Hansen JP, Lichtenstein MB. Translation and validation of the System Usability Scale in a Danish mental health setting using digital technologies in treatment interventions. Int J Hum-Comput Interact. May 8, 2020;36(8):709-716. [CrossRef]
- Ensink CJ, Keijsers NLW, Groen BE. Translation and validation of the System Usability Scale to a Dutch version: D-SUS. Disabil Rehabil. Jan 2024;46(2):395-400. [CrossRef] [Medline]
- Moorthy P, Weinert L, Harms BC, Anders C, Siegel F. German version of the mHealth App Usability Questionnaire in a cohort of patients with cancer: translation and validation study. JMIR Hum Factors. Nov 1, 2023;10:e51090. [CrossRef] [Medline]
- Borsci S, Federici S, Lauriola M. On the dimensionality of the System Usability Scale: a test of alternative measurement models. Cogn Process. Aug 2009;10(3):193-197. [CrossRef] [Medline]
- Mohamad Marzuki MF, Yaacob NA, Yaacob NM. Translation, cross-cultural adaptation, and validation of the Malay version of the System Usability Scale questionnaire for the assessment of mobile apps. JMIR Hum Factors. May 14, 2018;5(2):e10308. [CrossRef] [Medline]
- Dianat I, Ghanbari Z, AsghariJafarabadi M. Psychometric properties of the Persian language version of the System Usability Scale. Health Promot Perspect. Jul 12, 2014;4(1):82-89. [CrossRef] [Medline]
- Martins AI, Rosa AF, Queirós A, Silva A, Rocha NP. European Portuguese validation of the System Usability Scale (SUS). Proc Comput Sci. 2015;67:293-300. [CrossRef]
- Blažica B, Lewis JR. A Slovene translation of the System Usability Scale: the SUS-SI. Int J Hum Comput Interact. Feb 2015;31(2):112-117. [CrossRef]
- Sevilla-Gonzalez MDR, Moreno Loaeza L, Lazaro-Carrera LS, et al. Spanish version of the System Usability Scale for the assessment of electronic tools: development and validation. JMIR Hum Factors. Dec 16, 2020;7(4):e21161. [CrossRef] [Medline]
- SUS Svensk: System Usability Scale in Swedish. Rosenfeld Media. Jul 4, 2011. URL: https://rosenfeldmedia.com/sus-svensk-system-usability-scale-in-swedish/ [Accessed 2024-05-01]
- Ministry of Health and Social Affairs, Swedish Association of Local Authorities and Regions. Vision for eHealth 2025 – common starting points for digitisation of social services and health care. e-Hälsa 2025. URL: https://ehalsa2025.se/wp-content/uploads/2021/02/vision-for-ehealth-2025.pdf [Accessed 2024-10-01]
- Mokkink LB, Prinsen C, Patrick DL, et al. COSMIN study design checklist for patient-reported outcome measurement instruments. BMJ. Jul 2019. URL: https://gut.bmj.com/content/gutjnl/70/1/139/DC1/embed/inline-supplementary-material-1.pdf [Accessed 2025-01-07]
- Lewis JR. Measuring perceived usability: SUS, UMUX, and CSUQ ratings for four everyday products. Int J Hum-Comput Interact. Sep 14, 2019;35(15):1404-1419. [CrossRef]
- Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). Dec 15, 2000;25(24):3186-3191. [CrossRef] [Medline]
- Bangor A, Kortum PT, Miller JT. An empirical evaluation of the System Usability Scale. Int J Hum Comput Interact. Jul 29, 2008;24(6):574-594. [CrossRef]
- Kristensson Hallström I, Castor C, Hansson H, et al. eHealth as an aid for facilitating and supporting self-management in families with long-term childhood illness; development, evaluation, and implementation in clinical practice – a research programme. Clin Health Promot. 2018;13(1):e23002. URL: https://portal.research.lu.se/en/publications/ehealth-as-an-aid-for-facilitating-and-supporting-self-management [Accessed 2025-01-07]
- Kayser L, Karnoe A, Furstrand D, et al. A multidimensional tool based on the eHealth Literacy Framework: development and initial validity testing of the eHealth Literacy Questionnaire (eHLQ). J Med Internet Res. Feb 12, 2018;20(2):e36. [CrossRef] [Medline]
- Hobart JC, Riazi A, Lamping DL, Fitzpatrick R, Thompson AJ. Improving the evaluation of therapeutic interventions in multiple sclerosis: development of a patient-based measure of outcome. Health Technol Assess. Mar 2004;8(9):iii, 1-iii,48. [CrossRef] [Medline]
- Huisman M. Imputation of missing item responses: some simple techniques. Qual Quant. 2000;34(4):331-351. [CrossRef]
- Sjöström AE, Hajdarevic S, Hörnsten Å, Kristjánsdóttir Ó, Castor C, Isaksson U. The Swedish version of the eHealth Literacy Questionnaire: translation, cultural adaptation, and validation study. J Med Internet Res. Apr 12, 2023;25(1):e43267. [CrossRef] [Medline]
- Abma IL, Rovers M, van der Wees PJ. Appraising convergent validity of patient-reported outcome measures in systematic reviews: constructing hypotheses and interpreting outcomes. BMC Res Notes. Apr 19, 2016;9:226. [CrossRef] [Medline]
- Samuelsson M, Wennick A, Bengtsson M, Lydrup ML, Jakobsson J. Translation, cultural adaptation, and psychometric testing of the Supportive Care Needs Survey for partners and caregivers for Swedish family members of persons diagnosed with colorectal cancer. J Patient Rep Outcomes. Oct 11, 2023;7(1):100. [CrossRef] [Medline]
- World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human participants. JAMA. Oct 19, 2024. [CrossRef] [Medline]
- Polit DF, Beck CT. Nursing Research: Generating and Assessing Evidence for Nursing Practice. th ed. Wolters Kluwer; 2021. ISBN: 9781975154141
- Kortum PT, Bangor A. Usability ratings for everyday products measured with the System Usability Scale. Int J Hum Comput Interact. Jan 2013;29(2):67-76. [CrossRef]
- Ho AD, Yu CC. Descriptive statistics for modern test score distributions: skewness, kurtosis, discreteness, and ceiling effects. Educ Psychol Meas. Jun 2015;75(3):365-388. [CrossRef] [Medline]
- Lindkvist RM, Sjöström-Strand A, Landgren K, Johnsson BA, Stenström P, Hallström IK. “In A Way We Took the Hospital Home”-a descriptive mixed-methods study of parents’ usage and experiences of eHealth for self-management after hospital discharge due to pediatric surgery or preterm birth. Int J Environ Res Public Health. Jun 15, 2021;18(12):6480. [CrossRef] [Medline]
- Kirklies PC, Neumann O, Hohensinn L. Promoting digital equality in co-production: the role of platform design. Gov Inf Q. Mar 2024;41(1):101903. [CrossRef]
- Frennert S. Hitting a moving target: digital transformation and welfare technology in Swedish municipal eldercare. Disabil Rehabil Assist Technol. Jan 2021;16(1):103-111. [CrossRef] [Medline]
- Lember V, Brandsen T, Tõnurist P. The potential impacts of digital technologies on co-production and co-creation. Pub Manag Rev. Nov 2, 2019;21(11):1665-1686. [CrossRef]
- van Velsen L, Evers M, Bara CD, Op den Akker H, Boerema S, Hermens H. Understanding the acceptance of an eHealth technology in the early stages of development: an end-user walkthrough approach and two case studies. JMIR Form Res. Jun 15, 2018;2(1):e10474. [CrossRef] [Medline]
Abbreviations
| COSMIN: Consensus-Based Standards for the Selection of Health Status Measurement Instruments |
| eHLQ: eHealth Literacy Questionnaire |
| SUS: System Usability Scale |
| WHO: World Health Organization |
Edited by Andre Kushniruk, Elizabeth Borycki; submitted 13.07.24; peer-reviewed by Jessica S Berner, John Brooke; final revised version received 04.12.24; accepted 08.12.24; published 16.01.25.
Copyright© Helene Åvik Persson, Charlotte Castor, Nilla Andersson, Mia Hylén. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 16.1.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.