This paper is in the following e-collection/theme issue:

General Articles on Human Factors (8) Mobile Health (mhealth) (3224) Human Factors and Usability Case Studies (911) Usability of Apps and User Perceptions of mHealth (1102) mHealth for Wellness, Behavior Change and Prevention (3146) Mobile Health in Psychiatry (490) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1877)

Published on in Vol 12 (2025)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/65472, first published .
Usability and Quality of the JoyPop App: Prospective Evaluation Study

Usability and Quality of the JoyPop App: Prospective Evaluation Study

Usability and Quality of the JoyPop App: Prospective Evaluation Study

Authors of this article:

Ishaq Malik1 Author Orcid Image ;   Teagan Neufeld1 Author Orcid Image ;   Aislin Mushquash1 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    Department of Psychology, Lakehead University, 955 Oliver Road, Thunder Bay, ON, Canada

    Corresponding Author:

    Aislin Mushquash, PhD


    Background: Mental health difficulties are increasing among Canadian postsecondary students, and many face barriers to accessing mental health care. Mobile health smartphone apps for mental health reduce common barriers to care and improve student mental health outcomes. However, students’ engagement and use of mental health apps is low. Evaluating the usability and quality of mental health apps is essential not only for user engagement but also for safety and overall utility. Few mental health apps have undergone usability and quality evaluations, especially with measures explicitly designed for these apps. The JoyPop app is a resilience-building mental health app with evidence supporting its effectiveness for student mental health. It has yet to be evaluated using standardized measures of mental health app usability and quality, and the influence of usability and quality on use is unknown.

    Objective: We evaluated the usability and quality of the JoyPop app and the predictive importance of usability and quality, compared to other relevant user characteristics, in predicting intentions to use the app in the future (usage intentions).

    Methods: Participants (N=183) completed preapp measures assessing demographics and personality traits, then used the app for 1 week, and then completed postapp measures assessing the usability, quality, and use of the JoyPop app. Usability (overall; and subscales: ease of use, interface and satisfaction, and usefulness) and quality (objective, subjective, and perceived impact) were assessed with descriptive statistics. Multiple regression analyses tested the predictive importance of usability and quality on usage intentions after controlling for other user characteristics.

    Results: Participants rated the JoyPop app’s overall usability as “very good” (mean 5.63, SD 0.85). Participants rated the JoyPop app’s overall objective quality as “excellent” (mean 4.06, SD 0.54). Subjective quality ratings were good, with many participants (135/183, 73.8%) indicating they would recommend the app to others. Participants rated the app as having a moderate and helpful impact on their mental health and coping skills (mean 3.48, SD 0.88). In each regression model, usability (β=.56, P<.001) and quality (β=.52, P<.001) were the strongest predictors and predicted usage intentions over and above other user characteristics.

    Conclusions: Results align with prior research evaluating the JoyPop app and maintain that it is an engaging and high-quality mental health app that can support students. Findings provide important insight into the optimal design of mental health apps for students and inform adaptations to future iterations of the JoyPop app.

    JMIR Hum Factors 2025;12:e65472

    doi:10.2196/65472

    Keywords

    usability; quality; mobile health; mental health app; postsecondary students; uMARS; MAUQ; evaluation study; mental health; app; resilience; Canada; student; usefulness; satisfaction; regression analyses; coping; smartphone app; mHealth app; mHealth; users; user version of the Mobile Application Rating Scale; Mobile Application Usability Questionnaire 


    Background

    Mental health difficulties among Canadian postsecondary students are increasing [1-3]. Unaddressed mental health difficulties can exacerbate existing challenges and negatively impact academic and long-term health outcomes [4,5]. Concerningly, students face a multitude of barriers (eg, long wait times and financial constraints) to access mental health support [4,6], resulting in prolonged distress, difficulties for campus mental health services, and low future help-seeking behaviors [7,8]. Novel approaches that increase students’ access to mental health support are needed [9,10].

    Mobile health smartphone apps designed to improve mental health are promising tools to support students [10,11]. Mental health apps also expand service capacity, improve availability and access to support, and reduce stigma, cost, time, and travel-related barriers for students [12-14]. Mental health apps are especially promising for students because they use smartphones frequently and are open to using apps for health-related reasons [15,16]. Research examining the effectiveness and efficacy of mental health apps among students shows that apps can improve well-being and reduce stress, depression, and anxiety [17,18].

    The JoyPop App

    The JoyPop app is a mental health app designed to improve resilience through evidence-based features that promote emotion regulation and adaptive coping among youths and emerging adults [19,20]. The JoyPop app was co-designed by youths, researchers, and service providers [20]. The app’s transdiagnostic focus makes it a valuable addition to usual mental health supports and applicable to a diverse range of students [21,22]. The app is particularly relevant to students as many are undergoing critical life transitions, and timely interventions that promote resilience can have a long-lasting positive impact on well-being and mental health [23,24]. Details of each feature are provided in Figure 1, and the rationale for the inclusion of each feature is documented elsewhere [19,25].

    ‎
    Figure 1. Summary and highlights of features in the JoyPop app. LGBTQ+: lesbian, gay, bisexual, transgender, and queer.

    There is an increasing multimethod evidence base supporting the effectiveness and value of the JoyPop app as a complement to mental health services across diverse users in clinical and nonclinical settings [19,25-29]. Among students, using the JoyPop app is associated with improved emotion regulation, improved depressive symptoms, and reduced stress [19,29]. Additionally, students perceive that the app improves emotion regulation and facilitates opportunities to implement adaptive coping skills [28]. Multiple randomized controlled trials (with students more generally [30] and with youths waiting for mental health services [31]) evaluating the app’s effectiveness in improving emotion regulation and mental health outcomes are ongoing.

    Although evidence of a mental health app’s effectiveness is essential, it does not guarantee users will engage with or enjoy their experience with an app [15,32]. Student engagement and long-term uptake of mental health apps are typically low [33-35] and can reduce the overall positive outcomes associated with app use [32,36]. Engagement, safety, and overall utility are affected by how usable an app is perceived to be and a user’s perspective on the quality of an app [15,34,36]. Despite this, the majority of mental health apps available in major marketplaces have not undergone usability and quality evaluations [35,37,38].

    Usability

    Usability is defined as a “quality attribute that assesses how easy interfaces are to use” [39]. Evaluating mental health app usability can reveal factors (eg, functionality, feature usefulness, layout, and readability) that reduce engagement and use [35,36]. Mental health app usability studies show that students value mental health apps with a convenient and intuitive design, customization, anonymity, privacy, peer involvement, games, and options to access professional help [40,41].

    Most studies assessing usability include measures designed for general technologies [42,43], such as the System Usability Scale [44]. Applying general technology usability measures to mental health apps can provide some valuable information, but they miss identifying factors unique to mobile health apps, such as limited computation power, security and privacy challenges, and characteristics associated with small screens and portable devices [43,45]. Measures designed specifically for apps, such as the Mobile Application Usability Questionnaire (MAUQ) [43], are rarely used [42,46]. Usability evaluations with validated and standardized measures developed for apps are recommended [43,46].

    Quality

    Evaluating the quality (eg, effectiveness, credibility, reliability, and safety) of mental health apps is also essential. Users are more likely to use a mental health app if it is provided by a credible source, positively impacts their mental health, and has evidence of safety and effectiveness [32]. Without proof of quality, users may rely on subjective star ratings, which can be inconsistent with an app’s actual quality and clinical utility [47-49]. Claims of mental health app quality are not consistently assessed by regulatory bodies [49,50], and most apps lack evidence of quality and effectiveness, clinical input from professionals, and evidence-based features [51,52]. Poor-quality mental health apps can therefore pose significant risks to users, such as misdiagnosis, unknown side effects, and physical and mental harm [53].

    Several resources are now available to locate, assess, and regulate the quality of mental health apps, such as app rating guidelines and platforms [37,54,55]. For example, the Mobile Health App Database (MHAD) rating platform provides quality ratings for a wide variety of mental health apps to help guide users and health care providers in selecting quality apps [54,56]. The Mobile Application Rating Scale (MARS) [49] is the most popular app rating guideline [42] and allows experts in mental health to rate apps based on quality indicators from health-related fields [49]. The user version of the Mobile Application Rating Scale (uMARS) is also available to help consumers without training and expertise to rate the quality of mobile health apps [55].

    This Study

    In this study, we aimed to (1) evaluate the usability and quality of the JoyPop app and (2) test the relative predictive importance of usability and quality on usage intentions, compared to other user characteristics. We used questionnaires explicitly designed to evaluate mobile health apps [42,46] to ensure that important facets of usability and quality unique to these apps were captured [43,57]. We chose age, gender, neuroticism, and agreeableness as relevant user characteristics based on past research highlighting their influence on mental health app use [58-61]. Because of the extensive literature highlighting the importance of usability and quality in mental health app use [32,37,45], we hypothesized that usability and quality would significantly predict JoyPop usage intentions over and above the other user characteristics.


    Design

    We used a 1-week prospective design to ensure students had sufficient time to use the app thereby facilitating more informed app evaluations.

    Ethical Considerations

    The study was reviewed and approved by the Lakehead University research ethics board (Romeo #1470023). All participants provided informed consent before beginning the study, were under no obligation to participate, and were free to withdraw at any time without penalty. All data were deidentified. We compensated participants for their time via cash or e-transfer or bonus points toward an eligible psychology course for completing part 1 and part 3 of the study (CAD $10 [US $7.28] or 1 bonus point for part 1, CAD $20 [US $14.57] or 1.5 bonus points for part 3).

    Participants and Procedure

    Participants were eligible for the study if they were students at Lakehead University who spoke and read fluently in English. We recruited students throughout the fall (2023) and winter (2024) terms (N=183). To increase the representativeness of our sample, we used multiple recruitment methods (eg, posters and class emails), had no exclusionary criteria based on student characteristics (eg, year of study), and provided refurbished iPhones with the JoyPop app for students who did not have a suitable device of their own. The study had 3 parts. In part 1, students attended an orientation session where they provided informed consent, learned about downloading and using the app, and completed preapp measures (ie, demographics and personality traits). In part 2, the research team reminded participants (via text or email) to use the app twice daily for 1 week. In part 3, participants were emailed or texted a link to complete postapp measures (ie, usability and quality).

    Measures

    Demographics

    To describe the characteristics of the sample, we captured participant demographics (13 items) along with students’ experience with mental health apps (5 items) by adapting items from prior questionnaires assessing technology experience [62,63]. We used the age and gender of participants as control variables in our analysis.

    The Big Five Aspects Scale

    We used the Big Five Aspects Scale (BFAS) [64] to gather information on participant personality traits. The BFAS consists of 100 items rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Participants rate their level of agreement with how well brief statements generally describe how they are. The BFAS provides scores for the traditional Big Five domains and 2 aspects for each domain. We used scores for the neuroticism and agreeableness domains as control variables in our analysis. The BFAS shows excellent internal consistency and test-retest reliability in postsecondary samples [64-66]. The internal consistency of domains used in this study was excellent (neuroticism, α=.9; agreeableness, α=.82).

    MAUQ

    The MAUQ [43] assesses the usability of mobile health apps. We used the 18-item standalone patient version of the MAUQ in this study as it best suits the context and design of the JoyPop app. Overall usability includes 3 subscales: ease of use (5 items), interface and satisfaction (7 items), and usefulness (6 items). Participants rate their level of agreement with brief statements on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). We calculated the overall usability score by computing the mean across all items. We derived subscale scores by computing the mean of scores for relevant subscale items. Higher scores indicate greater usability. Consistent with prior research [67,68], we used descriptors based on quartiles derived from a maximum possible score of 7 on the MAUQ scales to facilitate reporting and interpreting results: 0‐1.75 (poor), 1.76‐3.5 (moderate), 3.51‐5.25 (good), and 5.26‐7 (very good). The MAUQ has adequate psychometric properties across samples, including among postsecondary students [43]. The internal consistency of the overall scale score and subscale scores in this study was strong (overall scale, α=.92; ease of use, α=.79; interface and satisfaction, α=.86; usefulness, α=.85).

    uMARS

    The uMARS is a simple and reliable tool for end users to comprehensively evaluate mobile health app quality via indicators of objective and subjective quality, and perceived impact on a target health behavior [55]. We used the 27-item uMARS to evaluate the quality of the JoyPop app from the user’s perspective. The 16-item “objective quality” scale of the uMARS allows users to rate app quality based on indicators derived from research and expert and clinician recommendations on a 5-point Likert scale ranging from 1 (inadequate) to 5 (excellent). The “objective quality” scale has 4 subscales: engagement (5 items), functionality (4 items), aesthetics (3 items), and information (4 items). We derived the overall and subscale scores by calculating the mean of relevant items (items rated as N/A were omitted). Based on item ratings and past research [69,70], we used descriptors for ranges of scores: 0‐1 (inadequate), 1‐2 (poor), 2‐3 (acceptable), 3‐4 (good), and 4‐5 (excellent), to facilitate the interpretation of results and comparisons with other mental health apps. The overall scale and subscale scores have good psychometric properties [55]. The internal consistencies in this study were good (objective quality, α=.89; engagement, α=.77; functionality, α=.75; aesthetics, α=.78; information, α=.79).

    The 4-item “subjective quality” subscale of the uMARS allows users to rate different indicators (eg, star ratings, whether it is worth recommending) of a mobile health app’s subjective quality on a 5-point Likert scale ranging from 1 to 5 (anchors vary across questions). The 6-item “perceived impact” subscale of the uMARS allows users to rate its impact on user knowledge, awareness, intention to change, help-seeking, behavior change, and attitudes on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). This subscale was designed so that the wording of the items could be adapted to fit the function (eg, to impact mental health) of an app. We calculated the subjective quality and perceived impact subscale scores by calculating the mean of the relevant items (higher scores indicate stronger quality and impact). The subjective quality scale and perceived impact scale have good psychometric properties [55]. The internal consistency in this study was good (subjective quality, α=.79; perceived impact, α=.93).

    Usage Intentions

    We used an item (“How many times do you think you would use the JoyPop app in the next 12 months if it was relevant to you”) from the subjective quality scale on the uMARS to assess participants’ intentions to use the app in the future (usage intentions). Participants responded on a 5-point scale ranging from 1 (none) to 5 (>50 times). (Although we tracked the number of days participants used the app across the study period, we did not include this as our primary app use outcome variable as there was insufficient variation; that is, the majority of participants used the app every day; see descriptive results below.)

    Statistical Analysis

    We used SPSS (version 29, IBM Corporation) for the analyses. We calculated and examined the descriptive statistics for demographics and personality traits to characterize the sample. We calculated descriptive statistics for each scale and item of the MAUQ and uMARS to evaluate the usability and quality of the JoyPop app. To test our hypotheses that usability and quality would predict app use over and above other user characteristics, we conducted 2 multiple regression analyses. The first model included overall usability as a predictor; the second model included overall objective quality as a predictor. Both models included age, gender, agreeableness, and neuroticism as covariates. We included all predictors together for each model because our focus was to examine the relative importance of quality and usability in predicting use.


    User Characteristics

    We present the characteristics of the sample in Table 1. On average, participants were aged 22.6 (SD 7.08; range 16‐56) years. Most participants were women (150/183, 82%), White (100/183, 54.6%) or Black (33/183, 18%), full-time students (156/183, 85.2%), and in their first (81/183, 44.3%) or second (54/183, 29.5%) year of university. The majority of participants reported no current or past use of mental health apps (148/183, 80.9%), and most did not currently have a mental health app installed on their phones/devices (158/183, 86.3%). Among participants who used mental health apps in the past (35/183, 19.1%), most used them for 0‐6 months (115/183, 62.8%). Descriptive statistics of relevant personality traits (ie, neuroticism: mean 3.18, SD 0.63; agreeableness: mean 4, SD 0.42) were consistent with prior research among students [64,66]. Most participants used the app every day of the study period (mean 4.69, SD 0.81).

    Table 1. User characteristics (N=183).
    Value, n (%)
    Gender
     Women150 (82)
     Men33 (18)
    Ethnicity
     White100 (54.6)
     Black33 (18)
     South Asian20 (10.9)
     Indigenous10 (5.5)
     Southeast Asian7 (3.8)
     Other (eg, Middle Eastern, South American, and Asian Canadian)13 (7.1)
    Country of birth
     Canada116 (63.4)
     Nigeria26 (14.2)
     India14 (7.7)
     Pakistan4 (2.2)
     Philippines3 (1.6)
     Other (eg, Italy, Barbados, Vietnam, Sri Lanka, Uganda, and China)20 (10.9)
    Program
     Nursing66 (36.1)
     Psychology62 (33.9)
     Education10 (5.5)
     Social work7 (3.8)
     Computer science7 (3.8)
     Biology7 (3.8)
     Other (eg, kinesiology, business, outdoor recreation, and political science)24 (13.1)
    Year of university
     181 (44.3)
     254 (29.5)
     326 (14.2)
     419 (10.4)
     >53 (1.6)

    Usability

    Participants rated the JoyPop app’s overall usability as “very good” (mean 5.63, SD 0.85; range 2.22‐7), and 69.9% (128/183) of the sample had scores in the “very good” range. Participants rated the ease of using the app as “very good” (mean 6.37, SD 0.67; range 3.2‐7), and 92.3% (169/183) of the sample reported scores in the “very good” range. Participants rated the interface and satisfaction with the app as “very good” (mean 5.7, SD 0.99; range 1.57‐7), and 68.9% (126/183) had scores within the “very good” range. Participants rated the usefulness of the app as “good” (mean 4.93, SD 1.17; range 1.5‐7), with 42.1% (77/183) of scores in the “good” range and 38.3% (70/183) of scores in the “very good” range. We found that all item scores (see Table 2) on the ease of use and interface and satisfaction subscales fell within the “very good” range, and all item scores on the usefulness subscale were in the “good” range.

    Table 2. Means and SDs of MAUQa subscale items.
    Subscale itemsMean (SD)
    Ease of use
     1. The app was easy to use.6.58 (0.66)
     2. It was easy for me to learn to use the app.6.64 (0.55)
     3. The navigation was consistent when moving between screens.6.37 (0.94)
     4. The interface of the app allowed me to use all the functions (such as entering information, responding to reminders, and viewing information) offered by the app.6.32 (1.04)
     5. Whenever I made a mistake using the app, I could recover easily and quickly.5.96 (1.20)
    Interface and satisfaction
     6. I like the interface of the app.5.75 (1.23)
     7. The information in the app was well organized, so I could easily find the information needed.6.29 (0.92)
     8. The app adequately acknowledged and provided information to let me know the progress of my action.5.50 (1.45)
     9. I feel comfortable using this app in social settings.5.84 (1.24)
     10. The amount of time involved in using this app has been fitting for me.5.59 (1.42)
     11. I would use this app again.5.42 (1.50)
     12. Overall, I am satisfied with this app.5.55 (1.50)
    Usefulness
     13. The app would be useful for my health and well-being.5.19 (1.58)
     14. The app improved my access to health care services.4.63 (1.62)
     15. The app helped me manage my health effectively.4.52 (1.55)
     16. This app has all the functions and capabilities I expected it to have.4.85 (1.71)
     17. I could use the app even when the internet connection was poor or not available.5.16 (1.31)
     18. This app provided an acceptable way to receive health care services, such as accessing educational materials, tracking my own activities, and performing self-assessments.5.22 (1.45)

    aMAUQ: Mobile Application Usability Questionnaire.

    Quality

    Participants rated the overall objective quality of the app as excellent (mean 4.06, SD 0.54; range 1.96‐5), with 60.7% (111/183) of participants rating app quality as “excellent.” Participants rated the app’s functionality (mean 4.47, SD 0.52; range 2.75‐5; 136/183, 74.3% in the “excellent range”), aesthetics (mean 4.17, SD 0.67; range 1‐5; 101/183, 55.2% “excellent”; 41/183, 21.4% “good”), and information (range 1‐5; 107/183, 58.5% “excellent”; 39/183, 21.3% “good”) as “excellent.” Participants rated the app’s engagement as “good” (range 1‐5; 74/183, 40.4% “good”; 34/183, 18.6% “excellent”).

    Regarding subjective quality (see Table 3), we found that many participants (144/183, 78.7%) would recommend the app to others. For our measure of app use, which asked how often participants would use the app in the next 12 months, 24% (44/183) reported 3‐10 times, 42.1% (77/183) reported 10‐50 times, and 19.1% (35/183) reported >50 times. Most participants (132/183, 72.1%) did not want to pay for the app. The most frequent star rating of the app was 4 stars (92/183, 50.3%), followed by 3 stars (62/183, 33.9%).

    Table 3. Means, SDs, and frequencies of “subjective quality” item responses (N=183).
    Item and response optionsResponses, n (%)Responses, mean (SD)
    Would you recommend the JoyPop app to people who might benefit from it?3.69 (1.22)
     Not at all; I would not recommend this app to anyone7 (3.8)
     There are very few people I would recommend this app to32 (17.5)
     Maybe, there are several people I would recommend this app to33 (18)
     There are many people I would recommend this app to49 (26.8)
     Definitely, I would recommend this app to everyone62 (33.9)
    How many times do you think you would use the JoyPop app in the next 12 months if it was relevant to you?3.55 (1.61)
     None19 (10.4)
     1 to 28 (4.4)
     3 to 1044 (24)
     10 to 5077 (42.1)
     >5035 (19.1)
    Would you pay for this app?1.9 (0.98)
     Definitely not82 (44.8)
     Probably not50 (27.3)
     Unsure40 (21.9)
     Probably yes9 (4.9)
     Definitely yes2 (1.1)
    What is your overall (star) rating of the app?3.54 (0.80)
     One3 (1.6)
     Two13 (7.1)
     Three62 (33.9)
     Four92 (50.3)
     Five13 (7.1)

    The overall perceived impact of the app on participants’ mental health and coping skills (see Table 4) was moderate (mean 3.48, SD 0.88; range 1‐5). Most participants rated the app as having a moderate impact (55/183, 30.1% of scores between ≥3 and <4) or a strong impact (45/183, 24.6% of scores ≥4). Many participants agreed or strongly agreed that the app had a positive impact on their awareness (107/183, 58.5%), behaviors (103/183, 56.3%), knowledge and understanding (103/183, 56.3%), intentions and motivation (106/183, 57.9%), and attitudes (100/183, 54.6%) related to mental health and coping skills. Most agreed or strongly agreed that the app had a helpful impact on their willingness to seek further help for mental health and coping skills if needed (115/183, 62.8%).

    Table 4. Descriptive statistics for perceived impact subscale items (N=183).
    ItemsResponse option frequencies, n (%)Response, mean (SD)
    Strongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agree
    Increased my awareness of the importance of addressing mental health and coping skills7 (3.8)28 (15.3)41 (22.4)78 (42.6)29 (15.8)3.51 (1.05)
    Increased my knowledge and understanding of mental health and coping skills10 (5.5)26 (14.2)44 (24)82 (44.8)21 (11.5)3.42 (1.04)
    Changed my attitudes toward improving mental health and coping skills9 (4.9)32 (17.5)42 (23)79 (43.1)21 (11.5)3.39 (1.06)
    Increased my intentions and motivation to address mental health and coping skills7 (3.8)26 (14.2)44 (24)80 (43.7)26 (14.2)3.49 (1.02)
    Would encourage me to seek further help to address mental health and coping skills6 (3.3)20 (10.9)42 (23)81 (44.2)34 (18.6)3.64 (1.01)
    Will improve my mental health and coping skills10 (5.5)19 (10.4)51 (27.9)86 (47)17 (9.3)3.44 (0.99)
    Total score—a————3.48 (0.88)

    aNot applicable.

    Usability, Quality, and Usage Intentions

    We present the results of each regression model examining the relative predictive importance of usability and quality on usage intentions in Table 5. We found evidence of heteroskedasticity for age, so we used robust standard errors to obtain unbiased estimates of the regression coefficients. Our first regression model tested the predictive importance of usability on usage intentions. Age (β=.15; P=.008) and gender (β=.12; P=.04) were both significant and positive predictors of usage intentions. Neuroticism and agreeableness were not significant predictors of usage intentions. In line with our hypothesis, usability significantly and positively predicted usage intentions over and above other user characteristics, and usability was the strongest predictor in the model (F5,177=15.69; P<.001; R2adjusted=0.29; β=.56; P<.001). Our second regression model tested the predictive importance of quality on usage intentions. Age was a significant and positive predictor of usage intentions (β=.17; P=.008). Gender, neuroticism, and agreeableness were not significant predictors of usage intentions. Consistent with our hypothesis, quality significantly and positively predicted usage intentions over and above other user characteristics and was the strongest predictor in the model (F5,177=12.58; P<.001; R2adjusted=0.24; β=.52; P<.001).

    Table 5. Multiple regression analysis predicting intention to use the JoyPop app in the future.
    PredictorsBaRobust SEβP value
    Model 1
     Constant0.141.05—b.90
     Gender–0.380.18.12.04
     Age0.030.01.15.01
     Neuroticism–0.020.13–.01.86
     Agreeableness–0.260.18–.10.15
     Usability0.760.10.56<.001
    Model 2
     Constant0.261.07—.81
     Gender–0.360.20.12.07
     Age0.030.01.17.01
     Neuroticism–0.060.14–.03.66
     Agreeableness–0.340.18–.12.06
     Quality1.120.17.52<.001

    aUnstandardized coefficient.

    bNot applicable.


    Usability and Quality of the JoyPop App

    We evaluated the usability and quality of the JoyPop app and assessed their impact on intentions to use the app in the future after controlling for other user characteristics. Participants rated the overall usability as “very good” and all indicators comprising usability subscales were in the “good” to “very good” range. The cumulative evidence strongly supports that JoyPop app users are comfortable and satisfied with using it, find it easy to learn and use, and view it as a helpful tool to support mental health and well-being in different contexts (eg, clinical and academic settings) [25-28]. Usability ratings for the JoyPop app from this study are also similar to ratings of other popular mental health apps [71]. These findings will support organizations and consumers who are considering offering or using the JoyPop app.

    Participants rated the JoyPop app’s overall objective quality, functionality, aesthetics, and quality of information as “excellent” and the app’s engagement as “good.” Results are consistent with past research on the JoyPop app, highlighting it as an engaging, high-quality mental health app with strong functionality, visual design, and helpful evidence-based content [25-28]. Results also indicate that the JoyPop app falls within the upper ranges for objective quality when compared to popular and research-supported mental health apps (eg, Calm, PTSD Coach, SuperBetter, and Destressify) rated by experts [54,72]. For example, across approximately 50 mental health apps rated using the MARS, the average objective quality was 3.54 (range 1.63‐4.75) [54]. Among 19 popular research-supported mental health apps, the average objective quality score was 3.52 (SD 0.71), and average subscale scores were 3.98 (SD 0.82) for engagement, 3.42 (SD 0.80) for functionality, 3.23 (SD 0.90) for aesthetics, and 3.47 (SD 0.69) for information [72].

    Response frequencies to items on the subjective quality scale also support the quality of the JoyPop app. Although these are subjective ratings, their importance is highlighted by the consistency between subjective consumer ratings of mental health apps and objective expert reviews of mental health apps [72]. Subjective quality results also showed that most participants would not pay for the app. This is not surprising, as research consistently shows that people are less likely to use apps that have associated costs [32,73]. Taken together, the high objective and subjective quality ratings of the JoyPop app across indicators of mobile health app quality, combined with its quality ratings compared to mental health apps reviewed by experts, strongly support its safety, credibility, helpfulness, and use for students. Future research will benefit from having experts evaluate the quality of the JoyPop app (eg, using the MARS) to validate this study’s findings and by compiling end-user quality evaluations of mental health apps to facilitate comparisons.

    Finally, more than half of the participants perceived that the JoyPop app positively impacted their mental health and coping skills in all areas queried. This finding further supports the overall quality of the app. The broad range of areas impacted by the JoyPop app is consistent with past research demonstrating its effectiveness and benefits among diverse samples of users and supports its overall ability to deliver users an effective experience [19,25-29]. Results from rigorously controlled trials examining effectiveness are still needed. However, results on the app’s perceived impact on mental health and coping skills can delineate whether future outcomes are meaningful and impactful, provide researchers with an understanding of the potential mechanisms and reasons why the app might be effective, and inform outcome measures that can be assessed in rigorously controlled trials (eg, measures of help-seeking and coping skills).

    Influence of Usability and Quality on Usage Intentions

    In our regression models, overall usability and objective quality were the strongest predictors of usage intentions and predicted usage intentions over and above age, gender, neuroticism, and agreeableness. Although we did not objectively measure app usage (eg, number of days used), behavioral intentions are a strong predictor of actual behavior, and user intentions are consistently shown to predict the use of digital health technologies [74-76]. Consequently, our results align with research demonstrating the substantial impact of quality and usability on engagement, long-term uptake, and use of mental health apps [32,35,37]. The importance of usability and quality in predicting participants’ intentions to use the JoyPop app in the future, along with insight into indicators influencing its quality and usability, provides integral information for JoyPop and other mental health app development teams in optimal app design to improve user experience. An iterative process can be used in which development teams have users rate the usability and quality of the app, determine specific indicators requiring improvement, adapt the app based on these indicators, and then re-evaluate the usability and quality among similar populations.

    Limitations

    This study has some important limitations. First, we used a convenience sample primarily comprised of women, which may reduce the generalizability of results to men and the broader student population. However, the demographics and diversity of our sample help support the generalizability of the results to students because this study’s demographics are similar to those found among students across Ontario [77]. Second, in our study, we requested that participants use the app twice a day over 7 days. This likely contributed to the limited variability in the number of days participants used the app and necessitated our use of a subjective measure of usage intention instead of actual use. To better capture objective usage metrics, studies conducted over longer periods without prompting participants to use the app are needed to establish the relative predictive importance of usability and quality on app use [57].

    Conclusions

    Mental health apps are promising tools for students’ mental health. This study evaluated the usability and quality of a mental health app (JoyPop) designed to improve resilience while examining the relative predictive importance of usability and quality in predicting intentions to use the app in the future. Participants rated the JoyPop app as having “very good” usability. Participants rated the app as having high overall quality because of its “excellent” objective quality, good subjective quality, and helpful impact on their mental health and coping skills. After accounting for user characteristics, usability and quality were the strongest predictors of participants’ intentions to use the JoyPop app in the future. Findings facilitate consistency and comparisons of mental health app usability and quality and contribute to the growing evidence base supporting the JoyPop app as an engaging and high-quality mental health app that can support students.

    Conflicts of Interest

    The senior author (AM) recently acquired intellectual property ownership rights for the JoyPop app (in June 2024). Prior to this time, intellectual property rights were owned by another researcher and institution. AM did not own any intellectual property rights when the study was developed, when data were collected, when data were analyzed, or when the initial draft of the manuscript was developed.

    1. Canadian reference group—executive summary, spring 2022. American College Health Association; 2022. URL: https:/​/www.​acha.org/​wp-content/​uploads/​2024/​07/​NCHA-III_SPRING_2022_CANADIAN_REFERENCE_GROUP_EXECUTIVE_SUMMARY.​pdf [Accessed 2025-06-05]
    2. Linden B, Stuart H. Post-secondary stress and mental well-being: a scoping review of the academic literature. Can J Commun Ment Health. 2020;39(1):1-32. [CrossRef]
    3. Linden B, Boyes R, Stuart H. Cross-sectional trend analysis of the NCHA II survey data on Canadian post-secondary student mental health and wellbeing from 2013 to 2019. BMC Public Health. Mar 25, 2021;21(1):590. [CrossRef] [Medline]
    4. Moghimi E, Stephenson C, Gutierrez G, et al. Mental health challenges, treatment experiences, and care needs of post-secondary students: a cross-sectional mixed-methods study. BMC Public Health. Apr 6, 2023;23(1):655. [CrossRef] [Medline]
    5. Wood D, Crapnell T, Lau L, Bennett A, Lotstein D, Ferris M, et al. Emerging adulthood as a critical stage in the life course. In: Halfon N, Forrest C, Lerner R, Faustman E, editors. Handbook of Life Course Health Development. Springer; 2018:123-143. [CrossRef]
    6. Dunley P, Papadopoulos A. Why is it so hard to get help? Barriers to help-seeking in postsecondary students struggling with mental health issues: a scoping review. Int J Ment Health Addiction. Jun 2019;17(3):699-715. [CrossRef]
    7. Jaworska N, De Somma E, Fonseka B, Heck E, MacQueen GM. Mental health services for students at postsecondary institutions: a national survey. Can J Psychiatry. Dec 2016;61(12):766-775. [CrossRef] [Medline]
    8. Ogrodniczuk JS, Kealy D, Laverdière O. Who is coming through the door? A national survey of self‐reported problems among post‐secondary school students who have attended campus mental health services in Canada. Couns Psychother Res. Dec 2021;21(4):837-845. [CrossRef]
    9. Kazdin AE. Addressing the treatment gap: a key challenge for extending evidence-based psychosocial interventions. Behav Res Ther. Jan 2017;88:7-18. [CrossRef] [Medline]
    10. e-Mental health in Canada: transforming the mental health system using technology. Mental Health Commission of Canada; 2014. URL: https:/​/www.​mentalhealthcommission.ca/​wp-content/​uploads/​drupal/​MHCC_E-Mental_Health-Briefing_Document_ENG_0.​pdf [Accessed 2025-06-21]
    11. Supporting mental health and well-being for post-secondary students. Mental Health Commission of Canada; 2020. URL: https://mentalhealthcommission.ca/wp-content/uploads/2021/05/PSS_Standard_Leaflet_eng.pdf [Accessed 2025-06-05]
    12. Becker TD, Torous JB. Recent developments in digital mental health interventions for college and university students. Curr Treat Options Psych. Sep 2019;6(3):210-220. [CrossRef]
    13. Price M, Yuen EK, Goetter EM, et al. mHealth: a mechanism to deliver more accessible, more effective mental health care. Clin Psychol Psychother. 2014;21(5):427-436. [CrossRef] [Medline]
    14. Wiljer D, Shi J, Lo B, et al. Effects of a mobile and web app (Thought Spot) on mental health help-seeking among college and university students: randomized controlled trial. J Med Internet Res. Oct 30, 2020;22(10):e20790. [CrossRef] [Medline]
    15. Bautista J, Schueller SM. Understanding the adoption and use of digital mental health apps among college students: secondary analysis of a national survey. JMIR Ment Health. Mar 22, 2023;10:e43942. [CrossRef] [Medline]
    16. Selected online activities by gender, age group and highest certificate, diploma or degree completed. Statistics Canada; Jul 20, 2023. [CrossRef]
    17. Choudhury A, Kuehn A, Shamszare H, Shahsavar Y. Analysis of mobile app-based mental health solutions for college students: a rapid review. Healthcare (Basel). Jan 16, 2023;11(2):272. [CrossRef] [Medline]
    18. Oliveira C, Pereira A, Vagos P, Nóbrega C, Gonçalves J, Afonso B. Effectiveness of mobile app-based psychological interventions for college students: a systematic review of the literature. Front Psychol. 2021;12:647606. [CrossRef] [Medline]
    19. MacIsaac A, Mushquash AR, Mohammed S, Grassia E, Smith S, Wekerle C. Adverse childhood experiences and building resilience with the JoyPop app: evaluation study. JMIR Mhealth Uhealth. Jan 4, 2021;9(1):e25087. [CrossRef] [Medline]
    20. Wekerle C. JoyPop: a resilience intervention for youth mental health. PageSuite. 2021. URL: https:/​/edition.​pagesuite-professional.co.uk/​html5/​reader/​production/​default.​aspx?pubname=&edid=b43e8d51-b6f4-48ca-9e31-d7936ed7a950 [Accessed 2025-06-05]
    21. Bakker D, Kazantzis N, Rickwood D, Rickard N. Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Ment Health. Mar 1, 2016;3(1):e7. [CrossRef] [Medline]
    22. Jadhakhan F, Blake H, Hett D, Marwaha S. Efficacy of digital technologies aimed at enhancing emotion regulation skills: literature review. Front Psychiatry. 2022;13:809332. [CrossRef] [Medline]
    23. Howard AL, Galambos NL, Krahn HJ. Paths to success in young adulthood from mental health and life transitions in emerging adulthood. Int J Behav Dev. Nov 2010;34(6):538-546. [CrossRef]
    24. Schulenberg JE, Sameroff AJ, Cicchetti D. The transition to adulthood as a critical juncture in the course of psychopathology and mental health. Dev Psychopathol. 2004;16(4):799-806. [CrossRef] [Medline]
    25. Malik I, Perez A, Toombs E, et al. Female youth and mental health service providers' perspectives on the JoyPop™ app: a qualitative study. Front Digit Health. 2023;5:1197362. [CrossRef] [Medline]
    26. Au-Yeung A, Marfatia D, Beers K, et al. Exploring the feasibility of a mental health application (JoyPop™) for Indigenous youth. Front Psychiatry. 2023;14:1269347. [CrossRef] [Medline]
    27. Kim K, Au-Yeung A, Dagher D, et al. Exploring the relevance of a psychology-based resilience app (JoyPop™) for Indigenous youth. Child Abuse Negl. Feb 2024;148:106343. [CrossRef] [Medline]
    28. Mushquash AR, Pearson ES, Waddington K, et al. User perspectives on a resilience-building app (JoyPop): qualitative study. JMIR Mhealth Uhealth. Jul 8, 2021;9(7):e28677. [CrossRef] [Medline]
    29. Maurer K, Kimyaci M, Konyk K, Wekerle C. Building resilience through daily smartphone app use: results of a pilot study of the JoyPop app with social work students. Front Digit Health. 2023;5:1265120. [CrossRef] [Medline]
    30. MacIsaac A, Mann V, Toombs E, et al. Promoting mental health and wellbeing among post-secondary students with the JoyPop app: study protocol for a randomized controlled trial. Trials. 2024;25:576. [CrossRef]
    31. MacIsaac A, Neufeld T, Malik I, et al. Increasing access to mental health supports for 18- to 25-year-old Indigenous youth with the JoyPop mobile mental health app: study protocol for a randomized controlled trial. JMIR Res Protoc. Jan 30, 2025;14:e64745. [CrossRef] [Medline]
    32. Jacob C, Sezgin E, Sanchez-Vazquez A, Ivory C. Sociotechnical factors affecting patients’ adoption of mobile health tools: systematic literature review and narrative synthesis. JMIR Mhealth Uhealth. May 5, 2022;10(5):e36284. [CrossRef] [Medline]
    33. Linardon J, Fuller-Tyszkiewicz M. Attrition and adherence in smartphone-delivered interventions for mental health problems: a systematic and meta-analytic review. J Consult Clin Psychol. Jan 2020;88(1):1-13. [CrossRef] [Medline]
    34. Melcher J, Camacho E, Lagan S, Torous J. College student engagement with mental health apps: analysis of barriers to sustained use. J Am Coll Health. 2022;70(6):1819-1825. [CrossRef] [Medline]
    35. Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid Based Ment Health. Aug 2018;21(3):116-119. [CrossRef] [Medline]
    36. Alqahtani F, Orji R. Usability issues in mental health applications. In: UMAP’19 Adjunct: Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization. Association for Computing Machinery; 2019:343-348. [CrossRef]
    37. Alon N, Torous J. Current challenges for evaluating mobile health applications. J Am Med Inform Assoc. Feb 16, 2023;30(3):617-624. [CrossRef] [Medline]
    38. Messner EM, Probst T, O’Rourke T. mHealth applications: potentials, limitations, current quality and future directions. In: Montag C, Baumeister H, editors. Digital Phenotyping and Mobile Sensing. Springer; 2023:321-334. [CrossRef]
    39. Nielsen J. Usability 101: introduction to usability. Nielsen Norman Group. Jan 3, 2012. URL: https://www.nngroup.com/articles/usability-101-introduction-to-usability/ [Accessed 2025-06-05]
    40. Montagni I, Tzourio C, Cousin T, Sagara JA, Bada-Alonzi J, Horgan A. Mental health-related digital use by university students: a systematic review. Telemed J E Health. Feb 2020;26(2):131-146. [CrossRef] [Medline]
    41. Oti O, Pitt I. Online mental health interventions designed for students in higher education: a user-centered perspective. Internet Interv. Dec 2021;26:100468. [CrossRef] [Medline]
    42. Hajesmaeel-Gohari S, Khordastan F, Fatehi F, Samzadeh H, Bahaadinbeigy K. The most used questionnaires for evaluating satisfaction, usability, acceptance, and quality outcomes of mobile health. BMC Med Inform Decis Mak. Jan 27, 2022;22(1):22. [CrossRef] [Medline]
    43. Zhou L, Bao J, Setiawan IMA, Saptono A, Parmanto B. The mHealth App Usability Questionnaire (MAUQ): development and validation study. JMIR Mhealth Uhealth. Apr 11, 2019;7(4):e11500. [CrossRef] [Medline]
    44. Brooke J. SUS: a “quick and dirty” usability scale. In: Jordan P, Thomas B, Weerdmeester B, editors. Usability Evaluation in Industry. Taylor & Francis; 1996:189-194. ISBN: 9780429157011
    45. Harrison R, Flood D, Duce D. Usability of mobile applications: literature review and rationale for a new usability model. J Interact Sci. 2013;1(1):1. [CrossRef]
    46. Inal Y, Wake JD, Guribye F, Nordgreen T. Usability evaluations of mobile mental health technologies: systematic review. J Med Internet Res. Jan 6, 2020;22(1):e15337. [CrossRef] [Medline]
    47. Kuehnhausen M, Frost VS. Trusting smartphone apps? To install or not to install, that is the question. In: 2013 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA). IEEE; 2013:30-37. [CrossRef]
    48. Singh K, Drouin K, Newmark LP, et al. Many mobile health apps target high-need, high-cost populations, but gaps remain. Health Aff (Millwood). Dec 2016;35(12):2310-2318. [CrossRef]
    49. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth. 2015;3(1):e27. [CrossRef]
    50. Huckvale K, Nicholas J, Torous J, Larsen ME. Smartphone apps for the treatment of mental health conditions: status and considerations. Curr Opin Psychol. Dec 2020;36:65-70. [CrossRef] [Medline]
    51. Kertz SJ, MacLaren Kelly J, Stevens KT, Schrock M, Danitz SB. A review of free iPhone applications designed to target anxiety and worry. J Technol Behav Sci. Jun 2017;2(2):61-70. [CrossRef]
    52. Sucala M, Cuijpers P, Muench F, et al. Anxiety: there is an app for that. A systematic review of anxiety apps. Depress Anxiety. Jun 2017;34(6):518-525. [CrossRef] [Medline]
    53. Albrecht UV, von Jan U. mHealth apps and their risks: taking stock. Stud Health Technol Inform. 2016;226:225-228. [CrossRef] [Medline]
    54. Neary M, Schueller SM. State of the field of mental health apps. Cogn Behav Pract. Nov 2018;25(4):531-537. [CrossRef] [Medline]
    55. Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and validation of the user version of the Mobile Application Rating Scale (uMARS). JMIR Mhealth Uhealth. 2016;4(2):e72. [CrossRef]
    56. Stach M, Kraft R, Probst T, et al. Mobile Health App Database – a repository for quality ratings of mHealth apps. In: 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS). IEEE; 2020:427-432. [CrossRef]
    57. Ng MM, Firth J, Minen M, Torous J. User engagement in mental health apps: a review of measurement, reporting, and validity. Psychiatr Serv. Jul 1, 2019;70(7):538-544. [CrossRef] [Medline]
    58. Antezana G, Venning A, Smith D, Bidargaddi N. Do young men and women differ in well-being apps usage? Findings from a randomised trial. Health Informatics J. 2022;28(1). [CrossRef] [Medline]
    59. Ervasti M, Kallio J, Määttänen I, Mäntyjärvi J, Jokela M. Influence of personality and differences in stress processing among Finnish students on interest to use a mobile stress management app: survey study. JMIR Ment Health. May 13, 2019;6(5):e10039. [CrossRef] [Medline]
    60. Gitlow L. Technology use by older adults and barriers to using technology. Phys Occup Ther Geriatr. Sep 2014;32(3):271-280. [CrossRef]
    61. Smail-Crevier R, Powers G, Noel C, Wang J. Health-related internet usage and design feature preference for e-mental health programs among men and women. J Med Internet Res. Mar 18, 2019;21(3):e11224. [CrossRef] [Medline]
    62. Jabour AM, Rehman W, Idrees S, Thanganadar H, Hira K, Alarifi MA. The adoption of mobile health applications among university students in health colleges. J Multidiscip Healthc. 2021;14:1267-1273. [CrossRef] [Medline]
    63. Venkatesh V, Thong JY, Xu X. Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Q. 2012;36(1):157. [CrossRef]
    64. DeYoung CG, Quilty LC, Peterson JB. Between facets and domains: 10 aspects of the Big Five. J Pers Soc Psychol. Nov 2007;93(5):880-896. [CrossRef] [Medline]
    65. DeYoung CG, Carey BE, Krueger RF, Ross SR. Ten aspects of the Big Five in the personality inventory for DSM-5. Personal Disord. Apr 2016;7(2):113-123. [CrossRef] [Medline]
    66. Ka L, R E, K W, G J, Lje B. Associations between facets and aspects of Big Five personality and affective disorders: a systematic review and best evidence synthesis. J Affect Disord. Jun 2021;288:175-188. [CrossRef]
    67. Mortezaei S, Rabiei R, Asadi F, Emami H. Development and usability evaluation of a mHealth application for albinism self-management. BMC Med Inform Decis Mak. Jun 13, 2023;23(1):106. [CrossRef] [Medline]
    68. Rezaee R, Ghaffari M, Rabiei R, Kavousi A, Rakhshanderou S. Design and usability evaluation of a mobile application for self-care among Iranian adolescents. BMC Public Health. Mar 25, 2024;24(1):892. [CrossRef] [Medline]
    69. Hoffmann A, Faust-Christmann CA, Zolynski G, Bleser G. Gamification of a stress management app: results of a user study. In: Marcus A, Wang W, editors. Design, User Experience, and Usability Application Domains HCII 2019 Lecture Notes in Computer Science. Springer; 2019. [CrossRef]
    70. LeBeau K, Huey LG, Hart M. Assessing the quality of mobile apps used by occupational therapists: evaluation using the user version of the mobile application rating scale. JMIR Mhealth Uhealth. May 1, 2019;7(5):e13019. [CrossRef] [Medline]
    71. Baumel A, Faber K, Mathur N, Kane JM, Muench F. Enlight: a comprehensive quality and therapeutic potential evaluation tool for mobile and web-based eHealth interventions. J Med Internet Res. Mar 21, 2017;19(3):e82. [CrossRef] [Medline]
    72. Lau N, O’Daffer A, Yi-Frazier JP, Rosenberg AR. Popular evidence-based commercial mental health apps: analysis of engagement, functionality, aesthetics, and information quality. JMIR Mhealth Uhealth. Jul 14, 2021;9(7):e29689. [CrossRef] [Medline]
    73. Huang HY, Bashir M. Users’ adoption of mental health apps: examining the impact of information cues. JMIR Mhealth Uhealth. Jun 28, 2017;5(6):e83. [CrossRef] [Medline]
    74. Khechine H, Lakhal S, Ndjambou P. A meta‐analysis of the UTAUT model: eleven years later. Can J Adm Sci. Jun 2016;33(2):138-152. [CrossRef]
    75. McEachan RRC, Conner M, Taylor NJ, Lawton RJ. Prospective prediction of health-related behaviours with the theory of planned behaviour: a meta-analysis. Health Psychol Rev. Sep 2011;5(2):97-144. [CrossRef]
    76. McEachan R, Taylor N, Harrison R, Lawton R, Gardner P, Conner M. Meta-analysis of the reasoned action approach (RAA) to understanding health behaviors. Ann Behav Med. Aug 2016;50(4):592-612. [CrossRef] [Medline]
    77. School attendance by visible minority and immigrant status: Canada, provinces and territories, census metropolitan areas and census agglomerations with parts. Statistics Canada; Nov 30, 2022. [CrossRef]

    ‎
    BFAS: Big Five Aspects Scale
    MARS: Mobile Application Rating Scale
    MAUQ: Mobile Application Usability Questionnaire
    MHAD: Mobile Health App Database
    uMARS: user version of the Mobile Application Rating Scale

    Edited by Andre Kushniruk; submitted 16.08.24; peer-reviewed by Alison O'Daffer, Jinghan Hu; final revised version received 13.04.25; accepted 18.04.25; published 04.07.25.

    Copyright

    © Ishaq Malik, Teagan Neufeld, Aislin Mushquash. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 4.7.2025.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.