Mental health difficulties among Canadian postsecondary students are increasing [1-3]. Unaddressed mental health difficulties can exacerbate existing challenges and negatively impact academic and long-term health outcomes [4,5]. Concerningly, students face a multitude of barriers (eg, long wait times and financial constraints) to access mental health support [4,6], resulting in prolonged distress, difficulties for campus mental health services, and low future help-seeking behaviors [7,8]. Novel approaches that increase students’ access to mental health support are needed [9,10].

Mobile health smartphone apps designed to improve mental health are promising tools to support students [10,11]. Mental health apps also expand service capacity, improve availability and access to support, and reduce stigma, cost, time, and travel-related barriers for students [12-14]. Mental health apps are especially promising for students because they use smartphones frequently and are open to using apps for health-related reasons [15,16]. Research examining the effectiveness and efficacy of mental health apps among students shows that apps can improve well-being and reduce stress, depression, and anxiety [17,18].

The JoyPop app is a mental health app designed to improve resilience through evidence-based features that promote emotion regulation and adaptive coping among youths and emerging adults [19,20]. The JoyPop app was co-designed by youths, researchers, and service providers [20]. The app’s transdiagnostic focus makes it a valuable addition to usual mental health supports and applicable to a diverse range of students [21,22]. The app is particularly relevant to students as many are undergoing critical life transitions, and timely interventions that promote resilience can have a long-lasting positive impact on well-being and mental health [23,24]. Details of each feature are provided in Figure 1, and the rationale for the inclusion of each feature is documented elsewhere [19,25].

There is an increasing multimethod evidence base supporting the effectiveness and value of the JoyPop app as a complement to mental health services across diverse users in clinical and nonclinical settings [19,25-29]. Among students, using the JoyPop app is associated with improved emotion regulation, improved depressive symptoms, and reduced stress [19,29]. Additionally, students perceive that the app improves emotion regulation and facilitates opportunities to implement adaptive coping skills [28]. Multiple randomized controlled trials (with students more generally [30] and with youths waiting for mental health services [31]) evaluating the app’s effectiveness in improving emotion regulation and mental health outcomes are ongoing.

Although evidence of a mental health app’s effectiveness is essential, it does not guarantee users will engage with or enjoy their experience with an app [15,32]. Student engagement and long-term uptake of mental health apps are typically low [33-35] and can reduce the overall positive outcomes associated with app use [32,36]. Engagement, safety, and overall utility are affected by how usable an app is perceived to be and a user’s perspective on the quality of an app [15,34,36]. Despite this, the majority of mental health apps available in major marketplaces have not undergone usability and quality evaluations [35,37,38].

Usability is defined as a “quality attribute that assesses how easy interfaces are to use” [39]. Evaluating mental health app usability can reveal factors (eg, functionality, feature usefulness, layout, and readability) that reduce engagement and use [35,36]. Mental health app usability studies show that students value mental health apps with a convenient and intuitive design, customization, anonymity, privacy, peer involvement, games, and options to access professional help [40,41].

Most studies assessing usability include measures designed for general technologies [42,43], such as the System Usability Scale [44]. Applying general technology usability measures to mental health apps can provide some valuable information, but they miss identifying factors unique to mobile health apps, such as limited computation power, security and privacy challenges, and characteristics associated with small screens and portable devices [43,45]. Measures designed specifically for apps, such as the Mobile Application Usability Questionnaire (MAUQ) [43], are rarely used [42,46]. Usability evaluations with validated and standardized measures developed for apps are recommended [43,46].

Evaluating the quality (eg, effectiveness, credibility, reliability, and safety) of mental health apps is also essential. Users are more likely to use a mental health app if it is provided by a credible source, positively impacts their mental health, and has evidence of safety and effectiveness [32]. Without proof of quality, users may rely on subjective star ratings, which can be inconsistent with an app’s actual quality and clinical utility [47-49]. Claims of mental health app quality are not consistently assessed by regulatory bodies [49,50], and most apps lack evidence of quality and effectiveness, clinical input from professionals, and evidence-based features [51,52]. Poor-quality mental health apps can therefore pose significant risks to users, such as misdiagnosis, unknown side effects, and physical and mental harm [53].

Several resources are now available to locate, assess, and regulate the quality of mental health apps, such as app rating guidelines and platforms [37,54,55]. For example, the Mobile Health App Database (MHAD) rating platform provides quality ratings for a wide variety of mental health apps to help guide users and health care providers in selecting quality apps [54,56]. The Mobile Application Rating Scale (MARS) [49] is the most popular app rating guideline [42] and allows experts in mental health to rate apps based on quality indicators from health-related fields [49]. The user version of the Mobile Application Rating Scale (uMARS) is also available to help consumers without training and expertise to rate the quality of mobile health apps [55].

In this study, we aimed to (1) evaluate the usability and quality of the JoyPop app and (2) test the relative predictive importance of usability and quality on usage intentions, compared to other user characteristics. We used questionnaires explicitly designed to evaluate mobile health apps [42,46] to ensure that important facets of usability and quality unique to these apps were captured [43,57]. We chose age, gender, neuroticism, and agreeableness as relevant user characteristics based on past research highlighting their influence on mental health app use [58-61]. Because of the extensive literature highlighting the importance of usability and quality in mental health app use [32,37,45], we hypothesized that usability and quality would significantly predict JoyPop usage intentions over and above the other user characteristics.

We used a 1-week prospective design to ensure students had sufficient time to use the app thereby facilitating more informed app evaluations.

The study was reviewed and approved by the Lakehead University research ethics board (Romeo #1470023). All participants provided informed consent before beginning the study, were under no obligation to participate, and were free to withdraw at any time without penalty. All data were deidentified. We compensated participants for their time via cash or e-transfer or bonus points toward an eligible psychology course for completing part 1 and part 3 of the study (CAD $10 [US $7.28] or 1 bonus point for part 1, CAD $20 [US $14.57] or 1.5 bonus points for part 3).

Participants were eligible for the study if they were students at Lakehead University who spoke and read fluently in English. We recruited students throughout the fall (2023) and winter (2024) terms (N=183). To increase the representativeness of our sample, we used multiple recruitment methods (eg, posters and class emails), had no exclusionary criteria based on student characteristics (eg, year of study), and provided refurbished iPhones with the JoyPop app for students who did not have a suitable device of their own. The study had 3 parts. In part 1, students attended an orientation session where they provided informed consent, learned about downloading and using the app, and completed preapp measures (ie, demographics and personality traits). In part 2, the research team reminded participants (via text or email) to use the app twice daily for 1 week. In part 3, participants were emailed or texted a link to complete postapp measures (ie, usability and quality).

We present the characteristics of the sample in Table 1. On average, participants were aged 22.6 (SD 7.08; range 16‐56) years. Most participants were women (150/183, 82%), White (100/183, 54.6%) or Black (33/183, 18%), full-time students (156/183, 85.2%), and in their first (81/183, 44.3%) or second (54/183, 29.5%) year of university. The majority of participants reported no current or past use of mental health apps (148/183, 80.9%), and most did not currently have a mental health app installed on their phones/devices (158/183, 86.3%). Among participants who used mental health apps in the past (35/183, 19.1%), most used them for 0‐6 months (115/183, 62.8%). Descriptive statistics of relevant personality traits (ie, neuroticism: mean 3.18, SD 0.63; agreeableness: mean 4, SD 0.42) were consistent with prior research among students [64,66]. Most participants used the app every day of the study period (mean 4.69, SD 0.81).

Participants rated the JoyPop app’s overall usability as “very good” (mean 5.63, SD 0.85; range 2.22‐7), and 69.9% (128/183) of the sample had scores in the “very good” range. Participants rated the ease of using the app as “very good” (mean 6.37, SD 0.67; range 3.2‐7), and 92.3% (169/183) of the sample reported scores in the “very good” range. Participants rated the interface and satisfaction with the app as “very good” (mean 5.7, SD 0.99; range 1.57‐7), and 68.9% (126/183) had scores within the “very good” range. Participants rated the usefulness of the app as “good” (mean 4.93, SD 1.17; range 1.5‐7), with 42.1% (77/183) of scores in the “good” range and 38.3% (70/183) of scores in the “very good” range. We found that all item scores (see Table 2) on the ease of use and interface and satisfaction subscales fell within the “very good” range, and all item scores on the usefulness subscale were in the “good” range.

Participants rated the overall objective quality of the app as excellent (mean 4.06, SD 0.54; range 1.96‐5), with 60.7% (111/183) of participants rating app quality as “excellent.” Participants rated the app’s functionality (mean 4.47, SD 0.52; range 2.75‐5; 136/183, 74.3% in the “excellent range”), aesthetics (mean 4.17, SD 0.67; range 1‐5; 101/183, 55.2% “excellent”; 41/183, 21.4% “good”), and information (range 1‐5; 107/183, 58.5% “excellent”; 39/183, 21.3% “good”) as “excellent.” Participants rated the app’s engagement as “good” (range 1‐5; 74/183, 40.4% “good”; 34/183, 18.6% “excellent”).

Regarding subjective quality (see Table 3), we found that many participants (144/183, 78.7%) would recommend the app to others. For our measure of app use, which asked how often participants would use the app in the next 12 months, 24% (44/183) reported 3‐10 times, 42.1% (77/183) reported 10‐50 times, and 19.1% (35/183) reported >50 times. Most participants (132/183, 72.1%) did not want to pay for the app. The most frequent star rating of the app was 4 stars (92/183, 50.3%), followed by 3 stars (62/183, 33.9%).

The overall perceived impact of the app on participants’ mental health and coping skills (see Table 4) was moderate (mean 3.48, SD 0.88; range 1‐5). Most participants rated the app as having a moderate impact (55/183, 30.1% of scores between ≥3 and <4) or a strong impact (45/183, 24.6% of scores ≥4). Many participants agreed or strongly agreed that the app had a positive impact on their awareness (107/183, 58.5%), behaviors (103/183, 56.3%), knowledge and understanding (103/183, 56.3%), intentions and motivation (106/183, 57.9%), and attitudes (100/183, 54.6%) related to mental health and coping skills. Most agreed or strongly agreed that the app had a helpful impact on their willingness to seek further help for mental health and coping skills if needed (115/183, 62.8%).

We present the results of each regression model examining the relative predictive importance of usability and quality on usage intentions in Table 5. We found evidence of heteroskedasticity for age, so we used robust standard errors to obtain unbiased estimates of the regression coefficients. Our first regression model tested the predictive importance of usability on usage intentions. Age (β=.15; P=.008) and gender (β=.12; P=.04) were both significant and positive predictors of usage intentions. Neuroticism and agreeableness were not significant predictors of usage intentions. In line with our hypothesis, usability significantly and positively predicted usage intentions over and above other user characteristics, and usability was the strongest predictor in the model (F_5,177=15.69; P<.001; R²_adjusted=0.29; β=.56; P<.001). Our second regression model tested the predictive importance of quality on usage intentions. Age was a significant and positive predictor of usage intentions (β=.17; P=.008). Gender, neuroticism, and agreeableness were not significant predictors of usage intentions. Consistent with our hypothesis, quality significantly and positively predicted usage intentions over and above other user characteristics and was the strongest predictor in the model (F_5,177=12.58; P<.001; R²_adjusted=0.24; β=.52; P<.001).

We evaluated the usability and quality of the JoyPop app and assessed their impact on intentions to use the app in the future after controlling for other user characteristics. Participants rated the overall usability as “very good” and all indicators comprising usability subscales were in the “good” to “very good” range. The cumulative evidence strongly supports that JoyPop app users are comfortable and satisfied with using it, find it easy to learn and use, and view it as a helpful tool to support mental health and well-being in different contexts (eg, clinical and academic settings) [25-28]. Usability ratings for the JoyPop app from this study are also similar to ratings of other popular mental health apps [71]. These findings will support organizations and consumers who are considering offering or using the JoyPop app.

Participants rated the JoyPop app’s overall objective quality, functionality, aesthetics, and quality of information as “excellent” and the app’s engagement as “good.” Results are consistent with past research on the JoyPop app, highlighting it as an engaging, high-quality mental health app with strong functionality, visual design, and helpful evidence-based content [25-28]. Results also indicate that the JoyPop app falls within the upper ranges for objective quality when compared to popular and research-supported mental health apps (eg, Calm, PTSD Coach, SuperBetter, and Destressify) rated by experts [54,72]. For example, across approximately 50 mental health apps rated using the MARS, the average objective quality was 3.54 (range 1.63‐4.75) [54]. Among 19 popular research-supported mental health apps, the average objective quality score was 3.52 (SD 0.71), and average subscale scores were 3.98 (SD 0.82) for engagement, 3.42 (SD 0.80) for functionality, 3.23 (SD 0.90) for aesthetics, and 3.47 (SD 0.69) for information [72].

Response frequencies to items on the subjective quality scale also support the quality of the JoyPop app. Although these are subjective ratings, their importance is highlighted by the consistency between subjective consumer ratings of mental health apps and objective expert reviews of mental health apps [72]. Subjective quality results also showed that most participants would not pay for the app. This is not surprising, as research consistently shows that people are less likely to use apps that have associated costs [32,73]. Taken together, the high objective and subjective quality ratings of the JoyPop app across indicators of mobile health app quality, combined with its quality ratings compared to mental health apps reviewed by experts, strongly support its safety, credibility, helpfulness, and use for students. Future research will benefit from having experts evaluate the quality of the JoyPop app (eg, using the MARS) to validate this study’s findings and by compiling end-user quality evaluations of mental health apps to facilitate comparisons.

Finally, more than half of the participants perceived that the JoyPop app positively impacted their mental health and coping skills in all areas queried. This finding further supports the overall quality of the app. The broad range of areas impacted by the JoyPop app is consistent with past research demonstrating its effectiveness and benefits among diverse samples of users and supports its overall ability to deliver users an effective experience [19,25-29]. Results from rigorously controlled trials examining effectiveness are still needed. However, results on the app’s perceived impact on mental health and coping skills can delineate whether future outcomes are meaningful and impactful, provide researchers with an understanding of the potential mechanisms and reasons why the app might be effective, and inform outcome measures that can be assessed in rigorously controlled trials (eg, measures of help-seeking and coping skills).

In our regression models, overall usability and objective quality were the strongest predictors of usage intentions and predicted usage intentions over and above age, gender, neuroticism, and agreeableness. Although we did not objectively measure app usage (eg, number of days used), behavioral intentions are a strong predictor of actual behavior, and user intentions are consistently shown to predict the use of digital health technologies [74-76]. Consequently, our results align with research demonstrating the substantial impact of quality and usability on engagement, long-term uptake, and use of mental health apps [32,35,37]. The importance of usability and quality in predicting participants’ intentions to use the JoyPop app in the future, along with insight into indicators influencing its quality and usability, provides integral information for JoyPop and other mental health app development teams in optimal app design to improve user experience. An iterative process can be used in which development teams have users rate the usability and quality of the app, determine specific indicators requiring improvement, adapt the app based on these indicators, and then re-evaluate the usability and quality among similar populations.

This study has some important limitations. First, we used a convenience sample primarily comprised of women, which may reduce the generalizability of results to men and the broader student population. However, the demographics and diversity of our sample help support the generalizability of the results to students because this study’s demographics are similar to those found among students across Ontario [77]. Second, in our study, we requested that participants use the app twice a day over 7 days. This likely contributed to the limited variability in the number of days participants used the app and necessitated our use of a subjective measure of usage intention instead of actual use. To better capture objective usage metrics, studies conducted over longer periods without prompting participants to use the app are needed to establish the relative predictive importance of usability and quality on app use [57].

Mental health apps are promising tools for students’ mental health. This study evaluated the usability and quality of a mental health app (JoyPop) designed to improve resilience while examining the relative predictive importance of usability and quality in predicting intentions to use the app in the future. Participants rated the JoyPop app as having “very good” usability. Participants rated the app as having high overall quality because of its “excellent” objective quality, good subjective quality, and helpful impact on their mental health and coping skills. After accounting for user characteristics, usability and quality were the strongest predictors of participants’ intentions to use the JoyPop app in the future. Findings facilitate consistency and comparisons of mental health app usability and quality and contribute to the growing evidence base supporting the JoyPop app as an engaging and high-quality mental health app that can support students.