This paper is in the following e-collection/theme issue:

Design and Usability of Consumer Health Tech and Home Monitoring Devices (218) Usability of Apps and User Perceptions of mHealth (974) Chatbots and Conversational Agents (531) mHealth for Wellness, Behavior Change and Prevention (2871) Quantified Self and Wellness (77) Formative Evaluation of Digital Health Interventions (2410) Mobile Health (mhealth) (2939) User Needs and Competencies (303)

Published on in Vol 11 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/59908, first published .
A New Research Model for Artificial Intelligence–Based Well-Being Chatbot Engagement: Survey Study

A New Research Model for Artificial Intelligence–Based Well-Being Chatbot Engagement: Survey Study

A New Research Model for Artificial Intelligence–Based Well-Being Chatbot Engagement: Survey Study

Authors of this article:

Yanrong Yang1 Author Orcid Image ;   Jorge Tavares1 Author Orcid Image ;   Tiago Oliveira1 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    Original Paper

    NOVA Information Management School (NOVA IMS), Universidade Nova de Lisboa, Lisboa, Portugal

    *all authors contributed equally

    Corresponding Author:

    Yanrong Yang, MSc

    NOVA Information Management School (NOVA IMS)

    Universidade Nova de Lisboa

    Campus de Campolide

    Lisboa, 1070-312

    Portugal

    Phone: 351 213 828 610

    Email: 20220021@NOVAIMS.UNL.PT


    Background: Artificial intelligence (AI)–based chatbots have emerged as potential tools to assist individuals in reducing anxiety and supporting well-being.

    Objective: This study aimed to identify the factors that impact individuals’ intention to engage and their engagement behavior with AI-based well-being chatbots by using a novel research model to enhance service levels, thereby improving user experience and mental health intervention effectiveness.

    Methods: We conducted a web-based questionnaire survey of adult users of well-being chatbots in China via social media. Our survey collected demographic data, as well as a range of measures to assess relevant theoretical factors. Finally, 256 valid responses were obtained. The newly applied model was validated through the partial least squares structural equation modeling approach.

    Results: The model explained 62.8% (R2) of the variance in intention to engage and 74% (R2) of the variance in engagement behavior. Affect (β=.201; P=.002), social factors (β=.184; P=.007), and compatibility (β=.149; P=.03) were statistically significant for the intention to engage. Habit (β=.154; P=.01), trust (β=.253; P<.001), and intention to engage (β=.464; P<.001) were statistically significant for engagement behavior.

    Conclusions: The new extended model provides a theoretical basis for studying users’ AI-based chatbot engagement behavior. This study highlights practical points for developers of AI-based well-being chatbots. It also highlights the importance of AI-based well-being chatbots to create an emotional connection with the users.

    JMIR Hum Factors 2024;11:e59908

    doi:10.2196/59908

    Keywords

    artificial intelligence–based chatbot; AI-based chatbot; mental well-being; intention to engage; engagement behavior; theoretical models; mobile phone 


    Overview

    According to the World Health Organization (2019), >80% of people worldwide face challenges in accessing mental health services [1]. This lack of access can be attributed to various factors, such as inadequate attention to health care, limited availability of medical resources, and the inability to afford the high costs of treatment [1,2]. Accessibility and scalability of mental health services need to be addressed [3].

    An artificial intelligence (AI)–based well-being chatbot can engage in conversations with humans in a relatively natural manner, offering companionship, emotional support, and guidance for emotional well-being [4]. Therapeutic well-being chatbots work by simulating how a mental health professional would treat a user [5], and companionship well-being chatbots facilitate or develop a social relationship with the user through chatting to alleviate and channel negative emotions, such as loneliness and irritability [6,7]. These new digital interventions provide considerable relief to individuals who need psychotherapeutic help but are plagued by a lack of time, space, or resources to access it [8-10]. Individuals who have interacted with these chatbots have expressed satisfaction with their experiences and have shown a positive attitude toward the future development of this technology [11]. These chatbots allow users to discuss private topics anonymously, effectively avoiding any feelings of shyness that may arise [12]. Well-being chatbots have also been used by professionals as an effective complementary tool to traditional face-to-face therapy [13,14]. In addition, they contribute positively to the dissemination of mental health knowledge and the promotion of healthy behaviors [11,15].

    A growing number of research findings support the idea that digital mental health interventions, for instance, well-being chatbots, reduce the risk of chronic diseases by improving patients’ psychosocial well-being and promoting other health behaviors [16-18]. They can help users overcome barriers to mental health support, and users can anonymously accept help from chatbots [19-21]. Scholars have taken notice of this phenomenon, and chatbot effectiveness, software design and development, use, and user satisfaction are being emphasized [22,23]. However, the problem of low engagement and high dropout rates between users and chatbots have not been prioritized, particularly in studying engagement behaviors through theoretical models. This will severely influence the user experience and effectiveness [3,8]. Exploring the factors influencing users’ engagement behavior with well-being chatbots is critical to comprehend and refine this association, to serve users better [8,24].

    This study aimed to investigate user intention to engage with well-being chatbots and engagement behavior by developing a new theoretical model that combines the theory of interpersonal behavior (TIB), diffusion of innovation (DOI), and trust. The goal is to understand the relationships among various factors and analyze their impact on the intention to engage and engagement behavior. We gathered data through a web-based survey to examine this model and identify the relationships between different factors. This research contributes to expanding the existing knowledge on theoretical models, particularly in the context of a human-centered digital mental health intervention. In addition, it will assist in designing, developing, and improving user-centered well-being chatbots; alleviating the problem of mental health medical resources; and helping to improve the overall well-being of the population. We have two research questions related to the objective of this study: (1) What factors influence users engaged with AI-based well-being chatbots? (2) How could AI-based well-being chatbot service be improved using the results of this study to improve users’ engagement and experience?

    Theoretical Background Rationale

    Published studies about adoption of AI-based well-being chatbots tend to focus on either emotional or technical components of this technology but not on a more integrated approach to study this new technology [25-30]. Particularly in digital health adoption, the most used theories, the technology acceptance model and unified theory of acceptance and use of technology, mostly focus on general technology adoption drivers [31,32]. Explaining the interaction of AI-based well-being chatbots with users goes beyond a simple technical interaction, it has been documented that they can create a psychological connection, like a friendship [33,34]. Therefore, we use the TIB, specifically its affect construct, to understand the relationship between a user and an AI-based well-being chatbot [29,30]. AI-based well-being chatbots are innovative technologies in the field of mental health care and personal well-being, and the application of DOI theory is beneficial for studying the factors that contribute to the adoption of AI-based well-being chatbots [35]. Trust is a key factor, particularly when dealing with personal and sensitive data [36,37], like the sharing process between the user and AI-based chatbots when it concerns mental health and personal well-being [38]. Without trust in the treatment intervention, the expected health outcomes between both parties may not be achieved [39]. The study brings these theories together through a new approach that combines relevant phycological factors for the adoption of AI-based well-being chatbots, which can be measured with the TIB and trust theory and the technical and innovation component of this new technology, which can be measured with the DOI theory.

    Engagement Behavior in Digital Mental Health Intervention

    Mental well-being is an increasingly important health topic of public concern. AI-based chatbots empower mental health and well-being through AI technology to provide emotional support to human beings [40]. AI-based chatbots enable user interaction based on text or voice support and complete corresponding tasks, recognizing users’ emotions and providing solutions [41]. The services of AI-based chatbots for mental well-being as a new digital mental health intervention to users are evolving, and it is crucial to study users’ engagement behavior.

    Engagement is a multidimensional concept that includes not only the formation of interest or adherence to a predefined plan, but also the development of trust, integration, and ongoing participation [42]. In this study, engagement behaviors are defined as the behaviors of users interacting with a well-being chatbot. The well-being chatbot serves as a new type of digital health intervention that provides users with mental health self-management and psychotherapy services [42]. Users’ engagement is an important factor, influencing the effectiveness of mental health interventions [43]. Research has shown that high engagement is associated with high intervention effectiveness [29,30,44,45]. In mental health treatment, participation in ongoing treatment is necessary for recovery [46,47]. A study of a digital mental health intervention found that >70% of users failed to complete all treatment modules and >50% withdrew before completing all treatment modules in general [29,48]. An analysis of mental health applications use showed that the average 15-day retention rate was only 3.9% [49]. Another study showed that mobile apps that emphasized user participation in design increased the effectiveness of interventions for depression and anxiety [50]. In a meta-analysis study of the impact of digital mental health engagement on mental health outcomes, users with higher levels of access showed substantial or moderate improvements in postintervention mental health outcomes [29]. This study explores the relationship between factors around engagement behaviors. This will help to uncover the insight of users’ willingness to engage and their engagement behaviors and improve the design capabilities and services of well-being chatbot.

    TIB Theory

    TIB was developed by Triandis [51] in 1977. It is similar to the ABC (attitude-behavior-context) model by Stern [52], combining internal and external factors, including affect, social factors, perceived consequences, habit, and facilitating conditions, to understand intended behavior [53]. In the context of engagement with well-being chatbot research, TIB is a well-suited theoretical model because well-being chatbots operate in a way similar to social software, where communication with users is accomplished through text dialogue and voice dialogue [54]. Users communicate trial experiences and results, and even recommend an AI-based chatbot to others [55]. The affect factor can seriously impact an individual’s willingness to communicate [56]. People will recommend their favorite products to each other, and this recommendation behavior will influence the individual’s intention [57]. Individuals past communication habit of using mobile apps will influence their willingness to use them [58]. If individuals frequently use instant messaging apps, they will be accustomed to this online communication method. TIB contains the above 3 critical aspects known as affect, habit, and social factors. Therefore, TIB is chosen as a theoretical basis for our model.

    DOI Theory

    DOI describes the process by which people embrace new ideas, use new products, and engage in new practices [59]. In general, only a few people have an attitude of developmental acceptance of new ideas and are willing to try them out and embrace them in the initial stages. As these people propagate them, gradually, more people begin to embrace them; the innovative idea or product thus diffuses through the population and eventually reaches saturation [60].

    DOI was proposed by Rogers [59] in 1995, it helps us to understand the characteristics of an innovation and what attracts users to it. According to Rogers’ research, 5 key features influence the adoption of an innovation: relative advantages, complexity, compatibility, observability, and trialability. Well-being chatbots are an innovative technology that has emerged in recent years, but they are not widespread in daily life. As their contribution to the mental health field, studying their dissemination among people leads to its understanding and acceptance by more people can contribute to human health and well-being. Therefore, extending the TIB model by adopting the properties of DOI is crucial. Among the 5 characteristics, observability can be considered equivalent to the combined effect of demonstrability and visibility [61]. Visibility was not used in this study because AI-based chatbot engagement was treated as a personal experience. Still, results demonstrability was used in our research model. Trialability was also not adopted because there was no evidence of whether the user had trialed a chatbot.

    Trust

    Trust is defined as the willingness of one party to accept the actions of another party, irrespective of the latter’s ability to control them [62]. The trust placed in machines is determined as the willingness of users to accept the information generated by machines and to adhere to their recommendations [63]. This indicates that one party intents to form a relatively secure attachment to the other, despite the potential for negative outcomes [64]. This represents a psychological mechanism that can reduce uncertainty and increase the likelihood of successful interaction with other entities within the environment [65]. Trust is a prerequisite for any social interaction and is instrumental in fostering collaboration and cooperation between individuals [66]. It serves as a key factor in successful transactions and establishing long-term relationships [67]. In the field of mental health, trust is of paramount importance in the relationship between patients and health care professionals [68,69]. The interactive behavior of users with chatbots for health purposes is analogous to that of patients with their doctors; the establishment of collaborative and cooperative relationships based on trust is conducive to the achievement of health objectives [70,71]. Therefore, trust represents a crucial element in the investigation of engagement with chatbots.

    Research Model and Hypotheses

    Following the theoretical rationale, TIB, DOI, and trust were combined to support the understanding of users’ intention to engage with AI-based well-being chatbots and engagement behavior in our research, as shown in Figure 1.

    ‎
    Figure 1. Research model. DOI: diffusion of innovation; H: hypothesis; TIB: theory of interpersonal behavior.

    Development of Hypotheses

    Perceived consequences are the positive or negative results of an action after it has occurred and the possibility of the outcome occurring [72]. When the perceived consequences are positive, the individual will be prompted to engage with the behavior to achieve advantages; when the perceived consequences are negative, the individual’s willingness to engage with the behavior will be reduced [53]. It had been validated to have a relevant effect on behavioral intention [53,73]. Well-being chatbots offer mental companionship and emotional support, contributing positively to users’ emotional and mental well-being [9]. Therefore, we assume that perceived consequences will be positively related to the intention to engage with well-being chatbots (hypothesis 1).

    Social factors are related to the extent to which people are influenced by others who are significant to them [51,73]. Individuals in a group or those observed by a group will comply with some of the unwritten rules within the group, and the likelihood that an individual will act in accordance with the group’s demands increases under the pressure of the group [51,74,75]. Social factors have been shown in several studies to positively influence individual behavioral intention [73,76-78]. In health care, the influence of social factors was examined and affirmed from multiple perspectives in a study of clinicians’ adoption of mHealth (mobile health) tools [79]. In chatbots that provide services in a social context, users decisions are influenced by perceptions of how those around them use these services [78]. Regarding the context of a well-being chatbot engagement, we hypothesize that social factors will be positively related to the intention to engage with well-being chatbots (hypothesis 2).

    Affect is used to describe the mental representation of internal bodily sensations associated with emotions, behaviors, or personality tendencies [80]. It is the purely emotional part of an individual’s attitude and contains positive or negative emotions, for instance excitement, joy, depression, and displeasure [81,82]. Affect has been shown to have an influence on behavioral intention in studies on information technology applications [76,83,84]. In the context of AI-based well-being chatbot engagement research, we assume that affect will positively influence individuals’ intention to engage with AI-based well-being chatbots. Thus, we propose that affect will be positively related to the intention to engage with well-being chatbots (hypothesis 3).

    Habit is a learned behavior, an automatic response to a steady stream of contextual cues [85], and it is regarded as a major influence on behavior [86]. A study has shown that an individual’s habits can predict future behavior to some extent [87]. Because the popularity of the internet as well as smartphones and the effectiveness of using digital interventions for health behaviors have been proven [51,53,54], this has caused health care apps to gradually become a way to optimize people’s daily health care behavioral habits [88]. As the well-being chatbot serves as a health care information system, we assume that habit will be positively related to engagement behavior (hypothesis 4).

    “Facilitating conditions” is a term that refers to objective elements in the environment that enable the easy execution of behavior [51]. In the IT context, it is defined as the resources necessary to support the use of a system, such as access to the internet or a smartphone [31]. Facilitating conditions have been identified as a key factor which influenced individuals’ behavior related to engagement [83]. Thus, we hypothesize that facilitating conditions will be positively related to engagement behavior (hypothesis 5).

    Trust has been recognized as one of the critical factors in human-robot interaction research [89,90]. Users’ trust in AI-based chatbots is based on the AI-based chatbot’s performance and services being dependable, trustworthy, and being able to assist in achieving the user’s intended purpose [37]. Developing and nurturing trust in the psychotherapy process to establish a good therapeutic relationship through engagement and ultimately effective treatment is crucial [91,92]. Meanwhile, trust was identified to have a major influence on the intention to act on eHealth websites [93]. Accordingly, trust influences users’ willingness to intent and engage with AI-based well-being chatbots [91-93]. So, we assume that trust positively influences an individual’s intention to engage AI-based well-being chatbots (hypothesis 6a) and trust positively influences users’ engagement behavior (hypothesis 6b).

    Compatibility refers to the extent to which the innovation matches the existing values and beliefs, previous experiences, and demands of potential users [59,94]. It provides a good indicator of how extensively an innovation complies with potential users’ lifestyles, needs, and preferences [60]. In previous research, compatibility was identified as one that influenced the intention to behavior [95]. Well-being chatbots meet the real-time needs of users [96-98], and chatbot mobile apps match the habits of smartphone users [99]. In this research, we assume that compatibility will be positively related to the intention to engage with AI-based well-being chatbots (hypothesis 7).

    Complexity is a measure of how difficult it is to understand and use an innovation [59]. It is a systematic form that is associated with almost all aspects of health care [100]. Complexity has been proven to have an impact on digital technology in health and well-being apps [101]. In another study on health care chatbots, complexity had a strong impact on the ability of chatbots to successfully provide health information and adoption behavior [100,102,103]. In this research, we assume that low complexity will be positively related to the intention to engage with well-being chatbots (hypothesis 8).

    Relative advantages is a term that refers to the degree to which an innovation is better than the object it replaces [94]. Innovation with greater relative advantage is beneficial for its diffusion [104]. It has been shown that an innovation will not be used if potential users believe that there is no comparative advantage in the adoption of the innovation over its earlier counterparts [105]. AI-based well-being chatbots are more empathetic than their earlier counterparts and even have memory functions, these advantages motivate users to interact and engage with them more [96,106-110]. Thus, we assume that relative advantages will be positively related to the intention to engage with AI-based well-being chatbots (hypothesis 9a) and relative advantages will be positively related to engagement behavior (hypothesis 9b).

    Results demonstrability is the degree to which innovative results are presented and disseminated [61]. Innovations will be more adopted if they generate demonstrably positive results; if the converse is the case, the chances of the innovation being adopted become lower [94]. Studies have shown that results demonstrability is a potential predictor of behavioral adoption [111]. AI-based well-being chatbots can serve users as an mHealth app. Thus, we assume that results demonstrability will be positively related to the intention to engage with AI-based well-being chatbot engagement (hypothesis 10).

    Intention to engage in a behavior is the most direct determinant of an individual’s behavior [112]. Exploring the relationship between intention to engage and engagement behavior helps to improve user experience and interaction effectiveness [9,113]. This also has a positive effect on the design of well-being chatbots in terms of enhancing user engagement [114,115]. Therefore, the intention to engage influences engagement and is an important factor in the study of user engagement with well-being chatbots. We assume that intention to engage with AI-based well-being chatbots will be positively related to engagement behavior (hypothesis 11).

    Age, gender, education and chronic disease status were implemented in the research model as control variables [116].


    Ethical Considerations

    Approval was obtained from the NOVA Information Management School Ethics Committee, NOVA University of Lisbon (INFSYS2023-5-257970). The procedures used in this study adhere to the tenets of the Declaration of Helsinki. All participants were aged at least 18 years, and informed consent was obtained from them. All data were collected anonymously, and participants were not compensated.

    Data Collection and Sample

    The questionnaire was developed in English on the Qualtrics platform. The survey was designed per the guidelines and the Checklist for Reporting Results of Internet E-Surveys (CHERRIES), which is presented in Multimedia Appendix 1 [117]. We explained to participants that participation was voluntary, and their data would be collected anonymously. We took measures to ensure that participants clearly understood what a well-being chatbot entails by introducing its concept and benefits at the start of the survey. Meanwhile, we described the functionality and use of an AI-based well-being chatbot. We engaged 2 experts and 2 colleagues to review and evaluate the questions to ensure that the topics were clear, relevant to the subject matter, and easy to understand. Once the questionnaire was finalized, a translator translated the questions into Chinese. Then, another translator was responsible for doing a back-translation and comparing it with the original English version to ensure accuracy [118]. Then, 40 participants were selected for pretesting to validate the questions’ understandability and the survey scale. No issues were reported that could indicate that the survey items were unreliable. Action was taken to prevent potential issues with single source and common source bias. The questionnaire was placed in 3 different web platforms to ensure the maximum coverage and avoid a single-source bias [117,119,120].

    We distributed the survey on 3 popular social media mobile apps: WeChat, Weibo, and Douban. WeChat is China’s most popular social media network, with 1.3 billion active users in 2022 [121]. Weibo is China’s second-largest social platform after WeChat, with 582 million active users at the end of the first quarter of 2022 [122]. Douban is an interest-oriented social network community with 75 million users as of 2020 [123,124]. Publishing the questionnaire across the 3 social media platforms will ensure fair data collection.

    The framework’s independent and dependent variable items were collected in a single questionnaire. We assessed if there was a clear understanding of what was being measured by the constructs, to avoid the risk of common source bias [119,120]. The aim was for the respondents to avoid using the same mental process or heuristics when replying to questions about different constructs [119,120,125]. The assessment of our pilot survey was that there was no reason for concern. For added precaution, additional features were incorporated in the final survey to enable psychological separation. While designing a survey, psychological separation should ensure that the measures of the different constructs are unrelated [120]. Different instructions for different sections of the survey were provided, and the sections of the survey that measure different constructs were physically separated [120].

    Finally, 256 valid replies from well-being chatbot users were collected from May to October 2023. The web-based survey did not impose any restrictions on participants other being an adult aged ≥18 years.

    Measurement

    The scales of all the variables in this study were produced concerning the relevant literature. Minor modifications were carried out according to the characteristics of AI-based chatbots. We used a 7-point scale to assess the variables from 1=“strongly disagree” to 7=“strongly agree.” The questionnaire with the measurement items and references for each variable are provided in Multimedia Appendix 2.

    Data Analysis

    The data were analyzed using the partial least squares structural equation modeling (PLS-SEM) approach using Smart-PLS (version 4.0) [126], which is suitable for analyzing and predicting complex models and nonnormally distributed data. PLS-SEM can also handle models that include both reflective and formative variables [127].

    Reflective and formative construct measurements were included in the research. In reflective measurement models, causality flows from the underlying construct to the indicator. In contrast, in formative measurement models, causality flows in the opposite direction, from indicators to constructs [128]. Reflective constructs measure entities with a series of positively correlated items [129,130]. In contrast, the formative construct is a singular construct which is constituted by the aggregation of multiple indicators without any a priori assumptions regarding the interrelationships between these elements [129,130]. Reflective and formative measurement models should be evaluated separately [128].


    Sample Characteristics

    Of the 256 valid samples, all had experience in using AI-based chatbot for mental health care. The participants’ average age was 30.9 years, and 55.9% of participants were younger than 30 years. The average age in other studies in China with the same scope as this study has ranged between 21 and 34.8 years [25-28]. The high proportion of young women was also present in demographic data from other studies, particularly studies on health technology adoption behaviors [109,131,132]. A recent Chinese study from 2023 showed that 77% of users of digital mental health technologies in China were female [28], which aligns with our study participants’ demographics. Approximately 91% of the participants held higher education degrees, which is more prevalent in innovation technology adoption studies [133,134]. However, the number of participants with chronic diseases was close to that of those without any disease, which is also reflected in the results of previous studies, which have found that chronic diseases have an impact on health applications [116]. The sample characteristics are shown in Table 1.

    Table 1. Demographic data (n=256).
    CharacteristicsParticipants, n (%)
    Age (years)

    		18-29143 (56)

    		30-4484 (33)

    		45-5913 (5)

    		≥6016 (6)
    Gender

    		Women187 (73)

    		Men69 (27)
    University education

    		Degree234 (91)

    		No degree22 (9)
    Chronic disease status

    		Yes111 (43)

    		No145 (57)

    Measurement Model

    Formative and reflective constructs were included in our model. They were measured separately. First, for reflective constructs, the construct items’ reliability was assessed by computing the value of each item. The loading values of all reflective construct items were above the threshold of 0.7, and they were accepted [135] and are listed in Multimedia Appendix 3. Then, we applied the Cronbach α reliability coefficient and composite reliability (CR) to measure their internal consistency. All Cronbach α and CR scores were above 0.7, and the model was proven to have good reliability [135]. Meanwhile, we examined convergence validity by assessing the average variance extracted (AVE); the value of AVE for each construct was >0.5 [135]. All detailed indicators for mean, SD, Cronbach α, CR, and AVE are shown in Table 2. We used the heterotrait-monotrait (HTMT) ratio as the main criterion to assess discriminant validity, following the latest guidelines recommendation [135-137]. The HTMT values were below the threshold value of 0.90 [135], thus confirming discriminant validity. The results are shown in Table 2. In addition, cross-loadings and the Fornell-Larcker criterion were also evaluated for discriminant validity, and the results also confirm discriminant validity (Multimedia Appendices 3 and 4).

    Table 2. Indicators of reflective constructs.
    ConstructValues, mean (SD)Cronbach αCRaAVEbHTMTc values





    		PCdAffectHabitFCeTrustCompatibilityComplexityRAfRDgITEhEBi
    PC4.726 (1.306).884.885.742—j——————————
    Affect4.638 (1.533).939.942.890.641——————————
    Habit4.333 (1.329).822.828.651.720.695—————————
    FC4.770 (1.269).846.846.684.684.602.573————————
    Trust4.461 (1.280).893.896.703.839.702.768.614———————
    Compatibility4.508 (1.344).851.853.771.725.746.732.666.767——————
    Complexity4.921 (1.292).896.900.763.690.679.648.811.697.791—————
    RA4.663 (1.242).849.855.689.816.707.768.676.842.805.752————
    RD4.620 (1.362).856.858.777.691.659.659.685.753.843.837.836———
    ITE4.575 (1.415).894.894.825.728.726.776.771.759.766.701.766.720——
    EB4.567 (1.362).896.896.763.779.723.803.653.827.758.662.760.736.897—

    aCR: composite reliability.

    bAVE: average variance extracted.

    cHTMT: heterotrait-monotrait.

    dPC: perceived consequences.

    eFC: facilitating conditions.

    fRA: relative advantages.

    gRD: results demonstrability.

    hITE: intention to engage.

    iEB: engagement behavior.

    jNot applicable.

    Social factors (SF) were measured as a formative construct in our research model [53]. We assessed the collinearity among indicators of the formative construct by calculating the variance inflation factor (VIF). The VIF values (Table 3) were below the cutoff value of 5 [135], which meant that there was no collinearity. Finally, we implemented a bootstrapping approach with 5000 resamples for identifying the statistical significance of each path. Social factor (SF) 1 and SF2 present statistically significant outer weights; SF3, SF4 and SF5 did not present statistically significant outer weights. Thus, we verified the SF3, SF4 and SF5 outer loading values, which were all >0.5 [135]. This aspect means that all SF items were relevant.

    Table 3. Indicators of formative construct.
    SFaVIFbOuter weightsP values (outer weights)Outer loadingsP values (outer loadings)
    SF12.386.224.04.826<.001
    SF22.411.421<.001.881<.001
    SF32.441.253.07.834<.001
    SF42.841.111.37.781<.001
    SF52.520.198.13.738<.001

    aSF: social factor.

    bVIF: variance inflation factor.

    In addition, we used Harman 1-factor test method to probe for common method variance (CMV).

    The total variance extracted by 1 factor was less than the recommended threshold of 50%. Hence, this data should not present any problem with CMV [120]. Afterward, the marker variable technique was adopted to assess the CMV, and an unrelated construct was defined as a marker variable to determine the relationship between it and each construct in the research model [125]. We obtained 0.055 (5.5%) as the maximum shared variance with other variables. Therefore, the value can be considered low [138]. After verification by 2 methods, it was concluded that the influence of CMV can be excluded from this study [120,125].

    Structural Model

    The structural model explained 62.8% variance in intention to engage, and 74% of variance in engagement behavior. Both R2 are regarded as high by the literature [135]. High R2 values indicate that our key target variables can be well predicted via the PLS path model [135]. Figure 2 shows the structural model results and identifies which latent variables are statistically significant.

    ‎
    Figure 2. Structural model results. DOI: diffusion of innovation; H: hypothesis; ns: nonsignificant; TIB: theory of interpersonal behavior. *P<.05; ** P<.01; ***P<.001.

    Regarding the intention to engage, affect (β=.201; P=.002), SF (β=.184; P=.007), and compatibility (β=.149; P=.03) were statistically significant. Hypotheses 2, 3, and 7 were supported. Perceived consequences (β=.110; P=.12), trust (β=.120; P=.11), complexity (β=.048; P=.47), and relative advantages (β=.120; P=.16) were not statistically significant; hence hypotheses 1, 6a, 8, and 9a were not supported.

    About engagement behavior, intention to engage (β=.464, P<.001), habit (β=.154; P=.01) and trust (β=.253, P<.001) were statistically significant, and facilitating conditions (β=−.002; P=.98), relative advantages (β=−.008; P=.91), and results demonstrability (β=.113; P=.10) were not statistically significant. Hypotheses 11, 6b, and 4 were supported, and hypotheses 5, 9b, and 10 were rejected, as shown in Table 4.

    Table 4. Hypothesized path analysis.
    HypothesisPathβP valuesSupported
    H1Perceived consequences → intention to engage.110.12No
    H2Social factors → intention to engage.184.007Yes
    H3Affect → intention to engage.201.002Yes
    H4Habit → engagement behavior.154.01Yes
    H5Facilitating conditions → engagement behavior−.002.98No
    H6aTrust → intention to engage.120.11No
    H6bTrust → engagement behavior.253<.001Yes
    H7Compatibility → intention to engage.149.03Yes
    H8Complexity → intention to engage.048.47No
    H9aRelative advantages → intention to engage.120.16No
    H9bRelative advantages → engagement behavior−.008.91No
    H10Results demonstrability → engagement behavior.113.10No
    H11Intention to engage → engagement behavior.464<.001Yes

    The PLSpredict algorithm was used to assess the framework predictive power. The method uses training and hold out samples to generate and evaluate predictions from PLS path model estimations [139,140]. The guideline recommendation was followed, and the number of folds was set to 10 [140]. This approach was done because it is possible to achieve a statistical power of 80% to detect minimum R2 values of 0.1 in the endogenous constructs in the structural model for a significance level of 1% [137,140]. The first parameter to be evaluated was the Q2predict of the indicators concerning our endogenous variables that was above 0, showing that the model demonstrates predictive power [140]. To evaluate the predictive magnitude, we compared the PLS-SEM study model with the naive linear regression model (LM) to see if it can outperform the LM benchmark [140]. Because the prediction errors distribution was considerably asymmetrical, with high kurtosis values (>1) [135], the mean absolute error was the more appropriate prediction statistic [140].

    The PLS-SEM analysis yielded lower prediction errors for most of the dependent variables’ indicators, as seen in Table 5; this indicates a medium predictive power for the study model [140]. When complex models are used, involving several theories, such as those explaining human behavior, R2 values higher than 0.5 can be regarded as substantial [135,137]. The model in this research study is complex, and achieving medium predictive power is challenging in such models [137,140]. Given that this research model shows a substantial R2 for a complex model and medium predictive power, it provides confidence in its use for real-world applications [135,137,140].

    Table 5. Prediction summary.
    IndicatorsQ2predictPLS_SEM_RMSEaPLS_SEM_MAEbLM_RMSEcLM_MAEdKurtosisSkewness
    EB1e.4311.165.8441.149.8102.029−.542
    EB2.4611.157.863f1.196.886.673−.048
    EB3.4901.114.8111.230.9131.550−.152
    EB4.5571.057.7761.157.8312.030−.413
    ITE1g.4851.128.8351.071.7852.374−.461
    ITE2.4721.135.8141.167.8692.782−.740
    ITE3.4521.156.8611.189.8841.862−.627

    aPLS_SEM_RMSE: partial least squares structural equation modeling root mean squared error.

    bPLS_SEM_MAE: partial least squares structural equation modeling mean absolute error.

    cLM_RMSE: linear regression model root mean squared error.

    dLM_MAE: linear regression model mean absolute error.

    eEB: engagement behavior.

    fMost relevant errors to define the model predictive power are highlighted in italics.

    gITE: intention to engage.


    Principal Findings

    The TIB was considered in this study, and it was extended with the DOI theory and trust to explore in depth the factors influencing users’ engagement behavior with AI-based chatbots for well-being. The new extended model was well explained, with R2 values of 62.8% for the variance in intention to engage and 74% for the variance in engagement behavior. RQ1 was answered in this study. AI-based well-being chatbots are gradually beginning to play an active role in the field of mental health. In this context, this study obtained important results, including affect, habit, SF, trust, compatibility and intention to engage as determinants influencing users’ engagement behavior. By extending the TIB to include important research variables in the model, a foundation was laid for future, related theoretical research and practice. The study’s results affirmed the significance of affect on users’ intention to engage. Our measurement of user affect contains both positive and negative emotions [141]. From the user’s perspective, there is a willingness to engage with the chatbot. It indicates that users have a positive impression of the well-being chatbot’s service and that users are inclined to deal with their emotions by engaging with a well-being chatbot. Users are beginning to be comfortable with the service as a mental health intervention that is available anytime, anywhere, without an appointment [142]. From the perspective of well-being chatbot characteristics, empathetic chatbots can understand users’ emotions and provide professional psychological counseling or companionship [96]; for instance, suggesting meditation, outdoor activities, or socializing with friends [7,143,144]. This proactive intervention on users’ emotions promotes users’ willingness to participate with sustained engagement behaviors.

    Habit has been extensively studied in previous research on health app use and engagement behavior [145]. In the current information age, smartphone use has become ingrained in people’s daily lives. Engaging with mobile apps for specific purposes has become a habit for many individuals [146]. This study confirmed that habits were an essential factor influencing users’ engagement behavior, aligning with other studies’ findings [147]. In addition, the significant influence of SF indicated that the opinions of friends, family, and medical professionals substantially impacted individuals’ intention to engage with well-being chatbots. This finding is consistent with previous studies [78].

    Furthermore, as AI-based well-being chatbots represent an innovative technology in a digital mental health intervention, proactively exploring methods for promoting their adoption is worthwhile. This research model incorporates features from the DOI framework to extend the TIB model. Among these features, compatibility was found to impact the intention to engage significantly, while the other 3 aspects did not demonstrate statistical significance. Because chatbots work like any other social software, the experience is consistent with past experiences [148,149]. They can be used easily and without extra effort, making users more willing to use them, which is also consistent with previous research [150].

    Trust, habit, and intention to engage were statistically significant for chatbot engagement. They are both internal (subjective) elements of individuals [151,152], whereas other factors had no statistically significant influence, such as facilitating conditions, relative advantages, and results demonstrability. It indicated that the user’s subjective sense of experience played a decisive role in engagement compared with other factors [153,154]. This suggests that a lack of user-centeredness in product design or a lack of information about mental health services in terms of content that meets users’ needs could directly reduce user engagement [155]. Consistent with the findings of this study, it further underlines the importance of user-centeredness.

    Potential explanations for the constructs that were nonsignificant are also addressed. The facilitating conditions hypothesis was not supported, aligning with previous research [32,156]. When engaging with an AI-based well-being chatbot, users’ ability to access smartphones, computers, or the internet, as well as their knowledge of how to use them, did not become barriers—most likely because our respondents were young and highly educated [157]. It could have been expected that complexity would be statistically significant; nonetheless, other studies of new technologies in health care, when complexity was evaluated as part of DOI, also obtained nonsignificant results [13,91,94,157-160]. A possible explanation, also supported by the literature, is that early adopters of new technologies, as in the case of our study, have a higher cognitive ability and are more accustomed to managing complexity, so they do not perceive complexity as an obstacle to using new technologies, including AI-based well-being chatbots [13,91,94,157,160]. Some research indicates that relevant advantages could influence users’ willingness to adopt new technologies in the initial implementation phase [161,162]. However, not all studies support this. The nonconfirmation of the relative advantages hypothesis in our research suggests that individuals focused more on the experiences and value derived from chatbots’ services rather than the direct benefits of outcomes [163]. The emotional component—affect—is much more relevant than technical factors, minimizing their impact on the model. Among younger people with higher level of education, perceived consequences show different influences on behavioral intention [164]. In our study, perceived consequences had a nonsignificant impact on the intention to engage with a well-being chatbot, suggesting that individuals become more focused on their engaging experience rather than on the positive or negative consequences.

    Age, gender, education, and chronic disease status as control variables were not statistically significant for the 2 dependent variables (intention to engage and engagement behavior) in the model. Our study respondents were young, were mostly highly educated, included a high proportion of women, and had a ratio of respondents with chronic diseases aligned with the literature [25-28,116]. In addition, early adopters exhibit behavior toward a technology that differs from those who adopt it later [94,157,160]. AI-based well-being chatbots are a new technology, currently being used by early adopters with no significant heterogeneity, which, at this early stage of implementation, does not contribute to significant results for the control variables.

    Theoretical Implications

    First, we provided an integrated perspective of TIB, DOI and trust to uncover the critical factors influencing chatbot intention to engage and engagement and how these factors influence individual decision-making. In previous studies, the technology acceptance model and unified theory of acceptance and use of technology were considered the most used and integrated information systems theories, and there were few relevant adoptions of TIB [165]. However, we believed that affect and habit in TIB were most appropriate for explaining AI-based well-being chatbot use behavior based on its characteristics. The new integrated model provides important theoretical support for future research on AI-based chatbots and other products.

    Trust is among the most important factors determining human intention to adopt smart technology products [166]. Adding trust to the research model explained users’ intention to engage and engagement behavior in a multidimensional way. The results of this study provided empirical evidence for the completeness of future AI information system application models. Nowadays, AI technology is being increasingly applied in diverse information systems and the autonomy of information systems has been enhanced. Theoretically validating the impact of trust on AI products is imperative.

    Practical Implications

    The results of our study model can provide guidance on better implementing AI-based well-being chatbots to increase their adoption. The triangulation between users, companies that develop AI-based well-being chatbots, and mental health practitioners is highly relevant and should be considered during the system’s development and real-world use. In our model, affect had a statistically significant impact on the intention to engage with AI-based well-being chatbots. It is crucial for designers and developers to recognize the influence of users’ affect on their willingness to engage with AI-based well-being chatbots. Affect describes a wide range of feelings, both positive and negative [72]. Developers should address the needs of users in different emotional states during the design and development process while simultaneously enhancing the system’s ability to recognize emotions and build emotional connections between chatbots and users [167,168]. Because affect is a complex dimension that transcends the purely IT aspects of the chatbot, developers should seek the support of mental health practitioners from the early stages of these systems’ development. Habit is also a statistically significant construct, so providing good support services should be a key area of focus for AI-based well-being chatbot companies, as they contribute to user experience and help maintain users’ habits of continuous engagement with the chatbot. According to our model results, SF also contribute to the intention to engage AI-based well-being chatbots. It suggests that individuals relevant to users can influence the adoption of AI-based well-being chatbots [51]. This indicates that mental health practitioners may influence users’ intention to engage with AI-based well-being chatbots. For companies developing AI-based well-being chatbots, it is important to engage with mental health practitioners. Developers could also leverage social media to promote their AI-based well-being chatbots, considering the relevance of SF.

    Trust was another statistically significant construct in the model. Establishing user trust is pivotal for driving participation behavior, and devising strategies to cultivate user trust requires careful consideration. They should consider introducing mental health practitioners into the system development process [169]. The incorporation of mental health practitioners enhances the efficacy and competitive advantage of chatbots, while reinforcing user trust and fostering their intention to engage. In some current practices, mental health experts have already been involved in the chatbot development process, with positive outcomes [170].

    Another relevant construct in the model is compatibility. It is recommended that users be included in the entire life cycle of the AI-based well-being chatbot, as this allows a comprehensive understanding of users’ habits and lifestyles, thereby facilitating product compatibility [8,171]. By communicating with users in greater depth, it is possible to gain insights into their interactions with the chatbot and to establish trust. The emotional connection between AI-based well-being chatbots and their users represents the core value of this technology, requiring designers and developers to prioritize a user-centered approach in their work [172].

    Although this research did not cover older people or those with special needs, future developments specifically targeting these groups should account for the fact that they may not have access to certain resources or may have lower digital literacy [173]. In these cases, the constructs of complexity and facilitating conditions, which were nonsignificant in this research, should play a critical role. Less complex systems and providing the right resources should increase the adoption of AI-based well-being chatbots within these groups.

    The involvement of mental health practitioners and users, integrating and applying their feedback throughout the AI-based well-being chatbot’s development, is essential for successfully implementing the chatbot’s entire life cycle. It is crucial to use AI-based well-being chatbots for therapeutic purposes under the guidance of qualified professionals to prevent misuse, which could potentially result in risky behaviors [174,175]. Figure 3 provides a graphical representation of the suggested practical implications. RQ2 about improving user experience and engagement from a practical point of view, leading to enhanced service levels was answered.

    ‎
    Figure 3. Suggested practical implications.

    Limitations and Future Research

    Our study used a convenience sample for which an online, multiplatform collection approach with a large coverage of the Chinese population was used to prevent single-source bias [120]. Still, the approach was not entirely random because we only posted the questionnaire, making it available, and we did not send messages to all platform users. In addition, the data source was from only one country. Future studies could use probabilistic sampling approaches. Access to large databases of AI-based well-being chatbot users could support studies with random sampling. Future research should expand the data sources to include participants from different countries and ethnicities, as well as special target populations (eg, people diagnosed with depression, older people). A multicountry approach as the next step may be used to evaluate if the findings are generalizable. Future studies could also collect the independent and dependent variables in different moments in time to reduce the probability of common-source bias [120]. Collection of real use data from the AI-based well-being chatbots could also be an advantage in future studies. Specific criteria for measuring engagement, such as the number of minutes of participation in using the chatbot, the number of logins, and the number of completed modules were not counted [176]. A future study may also explore engagement with AI-based well-being chatbots from mental health practitioners’ perspective, as well as conduct qualitative research or quantitative research.

    Conclusions

    AI-based well-being chatbots provide users with emotional support to help alleviate conditions such as loneliness and anxiety. They are an effective solution to the lack of resources for mental health care. Exploring the factors affecting their use carries great significance. This paper extended past models by using DOI and trust theory, based on TIB. It proposed an integrated model that effectively explained the factors affecting individuals’ acceptance and engagement with AI-based well-being chatbots. Among them, affect, habit, and trust play vital roles. The important theoretical role of the TIB model in the context of chatbots was validated. In addition, recommendations for the design of well-being chatbots were presented. For example, human-centered design concepts, attention to ethical issues, and building trust through characterization have important practical implications.

    Acknowledgments

    This work was supported by national funds through Fundação para a Ciência e a Tecnologia, under the project UIDB/04152/2020, Centro de Investigação em Gestão de Informação (MagIC Information Management Research Center/NOVA Information Management School).

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Checklist for Reporting Results of Internet E-Surveys (CHERRIES) checklist.

    PDF File (Adobe PDF File), 117 KB
    Multimedia Appendix 2

    Study questionnaire.

    PDF File (Adobe PDF File), 100 KB
    Multimedia Appendix 3

    Outer loadings and cross-loadings.

    PDF File (Adobe PDF File), 174 KB
    Multimedia Appendix 4

    Fornell-Larcker criterion.

    PDF File (Adobe PDF File), 158 KB
    1. The WHO special initiative for mental health (‎2019-2023)‎: universal health coverage for mental health. World Health Organization. 2019. URL: https://iris.who.int/handle/10665/310981 [accessed 2023-06-16]
    2. Mojtabai R, Olfson M, Sampson NA, Jin R, Druss B, Wang PS, et al. Barriers to mental health treatment: results from the National Comorbidity Survey replication. Psychol Med. Aug 2011;41(8):1751-1761. [FREE Full text] [CrossRef] [Medline]
    3. Jabir AI, Lin X, Martinengo L, Sharp G, Theng YL, Tudor Car L. Attrition in conversational agent-delivered mental health interventions: systematic review and meta-analysis. J Med Internet Res. Feb 27, 2024;26:e48168. [FREE Full text] [CrossRef] [Medline]
    4. MacNeill AL, MacNeill L, Doucet S, Luke A. Professional representation of conversational agents for health care: a scoping review protocol. JBI Evid Synth. Feb 2022;20(2):666-673. [CrossRef] [Medline]
    5. Gray-Burrows KA, Willis TA, Foy R, Rathfelder M, Bland P, Chin A, et al. Role of patient and public involvement in implementation research: a consensus study. BMJ Qual Saf. Oct 2018;27(10):858-864. [FREE Full text] [CrossRef] [Medline]
    6. Brandtzaeg PB, Skjuve M, Følstad A. My AI friend: how users of a social chatbot understand their human–AI friendship. Hum Commun Res. Jul 2022;48(3):404-429. [CrossRef]
    7. Xie T, Pentina I, Hancock T. Friend, mentor, lover: does chatbot engagement lead to psychological dependence? J Serv Manag. 2023;34(4):806-828. [CrossRef]
    8. Sadasivan C, Cruz C, Dolgoy N, Hyde A, Campbell S, McNeely M, et al. Examining patient engagement in chatbot development approaches for healthy lifestyle and mental wellness interventions: scoping review. J Particip Med. May 22, 2023;15:e45772. [FREE Full text] [CrossRef] [Medline]
    9. Kettle L, Lee YC. User experiences of well-being chatbots. Hum Factors. Jun 2024;66(6):1703-1723. [CrossRef] [Medline]
    10. Vaidyam AN, Wisniewski H, Halamka JD, Kashavan MS, Torous JB. Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can J Psychiatry. Jul 2019;64(7):456-464. [FREE Full text] [CrossRef] [Medline]
    11. Ahmed A, Ali N, Aziz S, Abd-alrazaq AA, Hassan A, Khalifa M, et al. A review of mobile chatbot apps for anxiety and depression and their self-care features. Comput Methods Programs Biomed Update. 2021;1:100012. [CrossRef]
    12. Nelekar S, Abdulrahman A, Gupta M, Richards D. Effectiveness of embodied conversational agents for managing academic stress at an Indian University (ARU) during COVID-19. Br J Educ Technol. Dec 20, 2021;53(3):491-511. [CrossRef]
    13. Baines R, Bradwell H, Edwards K, Stevens S, Prime S, Tredinnick-Rowe J, et al. Meaningful patient and public involvement in digital health innovation, implementation and evaluation: a systematic review. Health Expect. Aug 2022;25(4):1232-1245. [FREE Full text] [CrossRef] [Medline]
    14. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19-32. [CrossRef]
    15. Peuters C, Maenhout L, Cardon G, De Paepe A, DeSmet A, Lauwerier E, et al. A mobile healthy lifestyle intervention to promote mental health in adolescence: a mixed-methods evaluation. BMC Public Health. Jan 02, 2024;24(1):44. [FREE Full text] [CrossRef] [Medline]
    16. Chatterjee A, Prinz A, Gerdes M, Martinez S. Digital interventions on healthy lifestyle management: systematic review. J Med Internet Res. Nov 17, 2021;23(11):e26931. [FREE Full text] [CrossRef] [Medline]
    17. Widmer RJ, Collins NM, Collins CS, West CP, Lerman LO, Lerman A. Digital health interventions for the prevention of cardiovascular disease: a systematic review and meta-analysis. Mayo Clin Proc. Apr 2015;90(4):469-480. [FREE Full text] [CrossRef] [Medline]
    18. Fadhil A, Gabrielli S. Addressing challenges in promoting healthy lifestyles: the al-chatbot approach. In: Proceedings of the 11th EAI International Conference on Pervasive Computing Technologies for Healthcare. 2017. Presented at: PervasiveHealth '17; May 23-26, 2017; Barcelona, Spain. [CrossRef]
    19. Escriva Boulley G, Leroy T, Bernetière C, Paquienseguy F, Desfriches-Doria O, Préau M. Digital health interventions to help living with cancer: a systematic review of participants' engagement and psychosocial effects. Psychooncology. Dec 2018;27(12):2677-2686. [CrossRef] [Medline]
    20. Haque MD, Rubya S. An overview of chatbot-based mobile mental health apps: insights from app description and user reviews. JMIR Mhealth Uhealth. May 22, 2023;11:e44838. [FREE Full text] [CrossRef] [Medline]
    21. Belen-Saglam R, Nurse JR, Hodges D. An investigation into the sensitivity of personal information and implications for disclosure: a UK perspective. Front Comput Sci. Jun 30, 2022;4:1-22. [CrossRef]
    22. Guerreiro MP, Angelini L, Rafael Henriques H, El Kamali M, Baixinho C, Balsa J, et al. Conversational agents for health and well-being across the life course: protocol for an evidence map. JMIR Res Protoc. Sep 17, 2021;10(9):e26680. [FREE Full text] [CrossRef] [Medline]
    23. Diederich S, Brendel AB, Morana S, Kolbe L. On the design of and interaction with conversational agents: an organizing and assessing review of human-computer interaction research. J Assoc Inform Syst. 2022;23(1):96-138. [CrossRef]
    24. Al-Shafei M. Navigating human-chatbot interactions: an investigation into factors influencing user satisfaction and engagement. Int J Hum Comput Interact. Jan 12, 2024:1-18. [CrossRef]
    25. Qin Z, Ng S, Wu W, Zhang S. What Chinese women seek in mental health apps: insights from analyzing Xiaohongshu user posts during the COVID-19 pandemic. Healthcare (Basel). Jun 28, 2024;12(13):1297. [FREE Full text] [CrossRef] [Medline]
    26. Shan Y, Ji M, Xie W, Zhang X, Qian X, Li R, et al. Use of health care chatbots among young people in China during the Omicron wave of COVID-19: evaluation of the user experience of and satisfaction with the technology. JMIR Hum Factors. Jun 09, 2022;9(2):e36831. [FREE Full text] [CrossRef] [Medline]
    27. Aziz M, Erbad A, Almourad MB, Altuwairiqi M, McAlaney J, Ali R. Did usage of mental health apps change during COVID-19? A comparative study based on an objective recording of usage data and demographics. Life (Basel). Aug 19, 2022;12(8):1266. [FREE Full text] [CrossRef] [Medline]
    28. Zhang X, Lewis S, Chen X, Berry N, Bucci S. Technology use and attitudes towards digital mental health in people with severe mental health problems: a survey study in China. Front Psychiatry. 2023;14:1261795. [FREE Full text] [CrossRef] [Medline]
    29. Gan DZ, McGillivray L, Han J, Christensen H, Torok M. Effect of engagement with digital interventions on mental health outcomes: a systematic review and meta-analysis. Front Digit Health. 2021;3:764079. [FREE Full text] [CrossRef] [Medline]
    30. Neimeyer RA, Kazantzis N, Kassler DM, Baker KD, Fletcher R. Group cognitive behavioural therapy for depression outcomes predicted by willingness to engage in homework, compliance with homework, and cognitive restructuring skill acquisition. Cogn Behav Ther. 2008;37(4):199-215. [CrossRef] [Medline]
    31. Venkatesh V, Thong JY, Xu X. Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Q. 2012;36(1):157-178. [CrossRef]
    32. Tavares J, Oliveira T. Electronic health record patient portal adoption by health care consumers: an acceptance model and survey. J Med Internet Res. Mar 02, 2016;18(3):e49. [FREE Full text] [CrossRef] [Medline]
    33. Pentina I, Hancock T, Xie T. Exploring relationship development with social chatbots: a mixed-method study of replika. Comput Human Behav. Mar 2023;140:107600. [CrossRef]
    34. Zhu Y, Wang R, Pu C. "I am chatbot, your virtual mental health adviser." What drives citizens' satisfaction and continuance intention toward mental health chatbots during the COVID-19 pandemic? An empirical study in China. Digit Health. 2022;8:20552076221090031. [FREE Full text] [CrossRef] [Medline]
    35. Reinhardt R, Hietschold N, Spyridonidis D. Adoption and diffusion of innovations in health care. In: Gurtner S, Soyez K, editors. Challenges and Opportunities in Health Care Management. Cham, Switzerland. Springer; 2015:211-221.
    36. Saglam RB, Nurse JR, Hodges D. Privacy concerns in chatbot interactions: when to trust and when to worry. In: Proceedings of the 23rd HCI International Conference. 2021. Presented at: HCII 2021; July 24-29, 2021; Virtual Event. [CrossRef]
    37. Liu K, Tao D. The roles of trust, personalization, loss of privacy, and anthropomorphism in public acceptance of smart healthcare services. Comput Human Behav. Feb 2022;127:107026. [CrossRef]
    38. Zhao S, Cao Y, Cao H, Liu K, Lv X, Zhang J, et al. Chinese version of the mHealth app usability questionnaire: cross-cultural adaptation and validation. Front Psychol. 2022;13:813309. [FREE Full text] [CrossRef] [Medline]
    39. Wilkins CH. Effective engagement requires trust and being trustworthy. Med Care. Oct 2018;56 Suppl 10 Suppl 1(10 Suppl 1):S6-S8. [FREE Full text] [CrossRef] [Medline]
    40. Jiménez FR, Voss KE. An alternative approach to the measurement of emotional attachment. Psychol Market. Apr 10, 2014;31(5):360-370. [CrossRef]
    41. Grové C. Co-developing a mental health and wellbeing chatbot with and for young people. Front Psychiatry. 2020;11:606041. [FREE Full text] [CrossRef] [Medline]
    42. Andersson G, Titov N. Advantages and limitations of internet-based interventions for common mental disorders. World Psychiatry. Feb 2014;13(1):4-11. [FREE Full text] [CrossRef] [Medline]
    43. Bijkerk LE, Oenema A, Geschwind N, Spigt M. Measuring engagement with mental health and behavior change interventions: an integrative review of methods and instruments. Int J Behav Med. Apr 2023;30(2):155-166. [FREE Full text] [CrossRef] [Medline]
    44. Glenn D, Golinelli D, Rose RD, Roy-Byrne P, Stein MB, Sullivan G, et al. Who gets the most out of cognitive behavioral therapy for anxiety disorders? The role of treatment dose and patient engagement. J Consult Clin Psychol. Aug 2013;81(4):639-649. [FREE Full text] [CrossRef] [Medline]
    45. Hundt NE, Amspoker AB, Kraus-Schuman C, Cully JA, Rhoades H, Kunik ME, et al. Predictors of CBT outcome in older adults with GAD. J Anxiety Disord. Dec 2014;28(8):845-850. [FREE Full text] [CrossRef] [Medline]
    46. Becker MP, Christensen BK, Cunningham CE, Furimsky I, Rimas H, Wilson F, et al. Preferences for early intervention mental health services: a discrete-choice conjoint experiment. Psychiatr Serv. Feb 2016;67(2):184-191. [CrossRef] [Medline]
    47. Dixon LB, Holoshitz Y, Nossel I. Treatment engagement of individuals experiencing mental illness: review and update. World Psychiatry. Feb 2016;15(1):13-20. [FREE Full text] [CrossRef] [Medline]
    48. Karyotaki E, Kleiboer A, Smit F, Turner DT, Pastor AM, Andersson G, et al. Predictors of treatment dropout in self-guided web-based interventions for depression: an 'individual patient data' meta-analysis. Psychol Med. Oct 2015;45(13):2717-2726. [FREE Full text] [CrossRef] [Medline]
    49. Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res. Sep 25, 2019;21(9):e14567. [FREE Full text] [CrossRef] [Medline]
    50. Wu A, Scult MA, Barnes ED, Betancourt JA, Falk A, Gunning FM. Smartphone apps for depression and anxiety: a systematic review and meta-analysis of techniques to increase engagement. NPJ Digit Med. Feb 11, 2021;4(1):20. [FREE Full text] [CrossRef] [Medline]
    51. Triandis HC. Interpersonal Behavior. Pacific Grove, CA. Brooks/Cole Publishing Company; 1977.
    52. Stern PC. New environmental theories: toward a coherent theory of environmentally significant behavior. J Soc Issues. Dec 17, 2002;56(3):407-424. [CrossRef]
    53. Pee LG, Woon IM, Kankanhalli A. Explaining non-work-related computing in the workplace: a comparison of alternative models. Inf Manag. Mar 2008;45(2):120-130. [CrossRef]
    54. Singh B, Olds T, Brinsley J, Dumuid D, Virgara R, Matricciani L, et al. Systematic review and meta-analysis of the effectiveness of chatbots on lifestyle behaviours. NPJ Digit Med. Jun 23, 2023;6(1):118. [FREE Full text] [CrossRef] [Medline]
    55. Park D, Kim H. Determinants of intentions to use digital mental healthcare content among university students, faculty, and staff: motivation, perceived usefulness, perceived ease of use, and parasocial interaction with AI chatbot. Sustainability. Jan 03, 2023;15(1):872. [CrossRef]
    56. Zarrinabadi N, Lou NM, Darvishnezhad Z. To praise or not to praise? Examining the effects of ability vs. effort praise on speaking anxiety and willingness to communicate in EFL classrooms. Innov Lang Learn Teach. Jun 17, 2021;17(1):88-101. [CrossRef]
    57. Flavián C, Akdim K, Casaló LV. Effects of voice assistant recommendations on consumer behavior. Psychol Market. Nov 30, 2022;40(2):328-346. [CrossRef]
    58. Yang R, Wibowo S, Mubarak S. An investigation into domestic violence victims' adoption of chatbots for help-seeking: based on the UTAUT2 and health belief models. In: Proceedings of the PACIS 2023. 2023. Presented at: PACIS 2023; July 8-12, 2023; Nanchang, China.
    59. Rogers EM. Diffusion of Innovations, Fourth Edition. New York, NY. Free Press; 2010.
    60. Rogers EM. Diffusion of innovations: modifications of a model for telecommunications. In: Stoetzer MW, Mahler A, editors. Die Diffusion von Innovationen in der Telekommunikation. Berlin, Heidelberg. Springer; 1995.
    61. Moore GC, Benbasat I. Development of an instrument to measure the perceptions of adopting an information technology innovation. Inf Syst Res. Sep 1991;2(3):192-222. [CrossRef]
    62. Mayer RC, Davis JH, Schoorman FD. An integrative model of organizational trust. Acad Manag Rev. Jul 01, 1995;20(3):709-734. [CrossRef]
    63. Hancock PA, Billings DR, Schaefer KE, Chen JY, de Visser EJ, Parasuraman R. A meta-analysis of factors affecting trust in human-robot interaction. Hum Factors. Oct 2011;53(5):517-527. [CrossRef] [Medline]
    64. McKnight DH, Chervany NL. Trust and distrust definitions: one bite at a time. In: Falcone R, Singh M, Tan YH, editors. Trust in Cyber-societies. Berlin, Germany. Springer; 2001.
    65. Lukyanenko R, Maass W, Storey VC. Trust in artificial intelligence: from a Foundational Trust Framework to emerging research opportunities. Electron Markets. Nov 28, 2022;32(4):1993-2020. [CrossRef]
    66. Misztal B. Trust in Modern Societies: The Search for the Bases of Social Order. Cambridge, UK. Polity Press; 2013.
    67. Koehn D. Should we trust in trust? Am Bus Law J. Aug 27, 2007;34(2):183-204. [CrossRef]
    68. Du L, Xu J, Chen X, Zhu X, Zhang Y, Wu R, et al. Rebuild doctor-patient trust in medical service delivery in China. Sci Rep. Dec 15, 2020;10(1):21956. [FREE Full text] [CrossRef] [Medline]
    69. Chipidza FE, Wallwork RS, Stern TA. Impact of the doctor-patient relationship. Prim Care Companion CNS Disord. 2015;17(5):10.4088/PCC.15f01840. [FREE Full text] [CrossRef] [Medline]
    70. Seitz L, Bekmeier-Feuerhahn S, Gohil K. Can we trust a chatbot like a physician? A qualitative study on understanding the emergence of trust toward diagnostic chatbots. Int J Hum Comput Stud. Sep 2022;165:102848. [CrossRef]
    71. Powell J. Trust me, I'm a chatbot: how artificial intelligence in health care fails the Turing test. J Med Internet Res. Oct 28, 2019;21(10):e16222. [FREE Full text] [CrossRef] [Medline]
    72. Triandis HC. Values, attitudes, and interpersonal behavior. Nebr Symp Motiv. 1980;27:195-259. [Medline]
    73. Woon IM, Pee LG. Behavioral factors affecting internet abuse in the workplace – an empirical investigation. In: Proceedings of the SIGHCI 2004. 2004. Presented at: SIGHCI 2004; December 10-11, 2004; Washington, DC. URL: https://aisel.aisnet.org/sighci2004/5/
    74. Madden TJ, Ellen PS, Ajzen I. A comparison of the theory of planned behavior and the theory of reasoned action. Pers Soc Psychol Bull. Feb 01, 1992;18(1):3-9. [CrossRef]
    75. Ajzen I. From intentions to actions: a theory of planned behavior. In: Kuhl J, Beckmann J, editors. Action Control: From Cognition to Behavior. Berlin, Heidelberg. Springer-Verlag; 1985:11-39.
    76. Chang MK, Cheung W. Determinants of the intention to use internet/WWW at work: a confirmatory study. Inf Manag. Nov 2001;39(1):1-14. [CrossRef]
    77. Fanta GB, Pretorius L. Sociotechnical factors of sustainable digital health systems: a system dynamics model. Health Policy Technol. Mar 2023;12(1):100729. [CrossRef]
    78. Haoyue LL, Cho H. Factors influencing intention to engage in human–chatbot interaction: examining user perceptions and context culture orientation. Univ Access Inf Soc. Jan 29, 2024. [CrossRef]
    79. Jacob C, Sanchez-Vazquez A, Ivory C. Social, organizational, and technological factors impacting clinicians' adoption of mobile health tools: systematic literature review. JMIR Mhealth Uhealth. Feb 20, 2020;8(2):e15935. [FREE Full text] [CrossRef] [Medline]
    80. Barrett LF, Bliss-Moreau E. Affect as a psychological primitive. Adv Exp Soc Psychol. 2009;41:167-218. [FREE Full text] [CrossRef] [Medline]
    81. Alexander R, Aragón OR, Bookwala J, Cherbuin N, Gatt JM, Kahrilas IJ, et al. The neuroscience of positive emotions and affect: implications for cultivating happiness and wellbeing. Neurosci Biobehav Rev. Feb 2021;121:220-249. [FREE Full text] [CrossRef] [Medline]
    82. Diener E, Suh EM, Lucas RE, Smith HL. Subjective well-being: three decades of progress. Psychol Bull. Mar 1999;125(2):276-302. [CrossRef]
    83. Chin WY, Chua HN. Using the theory of interpersonal behavior to predict information security policy compliance. In: Proceedings of the Eighth International Conference on eDemocracy & eGovernment. 2021. Presented at: ICEDEG 2021; July 28-30, 2021; Quito, Ecuador. [CrossRef]
    84. Moody GD, Siponen M. Using the theory of interpersonal behavior to explain non-work-related personal use of the internet at work. Inf Manag. Sep 2013;50(6):322-335. [CrossRef]
    85. Gardner B, Abraham C, Lally P, de Bruijn GJ. Towards parsimony in habit measurement: testing the convergent and predictive validity of an automaticity subscale of the Self-Report Habit Index. Int J Behav Nutr Phys Act. Aug 30, 2012;9:102. [FREE Full text] [CrossRef] [Medline]
    86. Sheeran P, Gollwitzer PM, Bargh JA. Nonconscious processes and health. Health Psychol. May 2013;32(5):460-473. [CrossRef] [Medline]
    87. Orbell S, Hodgkins S, Sheeran P. Implementation intentions and the theory of planned behavior. Pers Soc Psychol Bull. Sep 1997;23(9):945-954. [CrossRef] [Medline]
    88. Badawy SM, Thompson AA, Kuhns LM. Medication adherence and technology-based interventions for adolescents with chronic health conditions: a few key considerations. JMIR Mhealth Uhealth. Dec 22, 2017;5(12):e202. [FREE Full text] [CrossRef] [Medline]
    89. Lee JD, See KA. Trust in automation: designing for appropriate reliance. Hum Factors. 2004;46(1):50-80. [CrossRef] [Medline]
    90. Montague E. Validation of a trust in medical technology instrument. Appl Ergon. Oct 2010;41(6):812-821. [FREE Full text] [CrossRef] [Medline]
    91. Brown JE, Halpern J. AI chatbots cannot replace human interactions in the pursuit of more inclusive mental healthcare. SSM Mental Health. Dec 2021;1:100017. [CrossRef]
    92. D'Alfonso S. AI in mental health. Curr Opin Psychol. Dec 2020;36:112-117. [CrossRef] [Medline]
    93. Sillence E, Blythe JM, Briggs P, Moss M. A revised model of trust in internet-based health information and advice: cross-sectional questionnaire study. J Med Internet Res. Nov 11, 2019;21(11):e11125. [FREE Full text] [CrossRef] [Medline]
    94. Scott SD, Plotnikoff RC, Karunamuni N, Bize R, Rodgers W. Factors influencing the adoption of an innovation: an examination of the uptake of the Canadian Heart Health Kit (HHK). Implement Sci. Oct 02, 2008;3:41. [FREE Full text] [CrossRef] [Medline]
    95. Hofstede G, Bond MH. The Confucius connection: from cultural roots to economic growth. Organ Dyn. Mar 1988;16(4):5-21. [CrossRef]
    96. Inkster B, Sarda S, Subramanian V. An empathy-driven, conversational artificial intelligence agent (Wysa) for digital mental well-being: real-world data evaluation mixed-methods study. JMIR Mhealth Uhealth. Nov 23, 2018;6(11):e12106. [FREE Full text] [CrossRef] [Medline]
    97. Ortega‐Ochoa E, Arguedas M, Daradoumis T. Empathic pedagogical conversational agents: a systematic literature review. Brit J Educ Technol. Dec 06, 2023;55(3):886-909. [CrossRef]
    98. Zhang X, Shah J, Han M. ChatGPT for fast learning of positive energy district (PED): a trial testing and comparison with expert discussion results. Buildings. May 27, 2023;13(6):1392. [CrossRef]
    99. Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid Based Ment Health. Aug 2018;21(3):116-119. [FREE Full text] [CrossRef] [Medline]
    100. Nason R. Challenges of implementing complexity in healthcare. Healthc Manage Forum. Nov 2023;36(6):368-372. [FREE Full text] [CrossRef] [Medline]
    101. Buckingham SA, Walker T, Morrissey K, Smartline project team. The feasibility and acceptability of digital technology for health and wellbeing in social housing residents in Cornwall: a qualitative scoping study. Digit Health. 2022;8:20552076221074124. [FREE Full text] [CrossRef] [Medline]
    102. Wang X, Luo R, Liu Y, Chen P, Tao Y, He Y. Revealing the complexity of users’ intention to adopt healthcare chatbots: a mixed-method analysis of antecedent condition configurations. Inf Process Manag. Sep 2023;60(5):103444. [CrossRef]
    103. Biro J, Linder C, Neyens D. The effects of a health care chatbot's complexity and persona on user trust, perceived usability, and effectiveness: mixed methods study. JMIR Hum Factors. Feb 01, 2023;10:e41017. [FREE Full text] [CrossRef] [Medline]
    104. Vagnani G, Volpe L. Innovation attributes and managers' decisions about the adoption of innovations in organizations: a meta-analytical review. Int J Innov Studies. Nov 2017;1(2):107-133. [CrossRef]
    105. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581-629. [FREE Full text] [CrossRef] [Medline]
    106. Zhou L, Gao J, Li D, Shum HY. The design and implementation of Xiaoice, an empathetic social chatbot. Comput Linguist. Mar 01, 2020;46(1):53-93. [CrossRef]
    107. Ghandeharioun A, McDuff D, Czerwinski M, Rowan K. EMMA: an emotion-aware wellbeing chatbot. In: Procedings of the 8th International Conference on Affective Computing and Intelligent Interaction. 2019. Presented at: ACII 2019; September 3-6, 2019; Cambridge, UK. [CrossRef]
    108. Curtis RG, Bartel B, Ferguson T, Blake HT, Northcott C, Virgara R, et al. Improving user experience of virtual health assistants: scoping review. J Med Internet Res. Dec 21, 2021;23(12):e31737. [FREE Full text] [CrossRef] [Medline]
    109. Ben-Zeev D, Chander A, Tauscher J, Buck B, Nepal S, Campbell A, et al. A smartphone intervention for people with serious mental illness: fully remote randomized controlled trial of CORE. J Med Internet Res. Nov 12, 2021;23(11):e29201. [FREE Full text] [CrossRef] [Medline]
    110. Lee M, Ackermans S, Van As N, Chang H, Lucas E, IJsselsteijn W. Caring for Vincent: a chatbot for self-compassion. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 2019. Presented at: CHI '19; May 4-9, 2019; Glasgow, Scotland. [CrossRef]
    111. Menzli LJ, Smirani LK, Boulahia JA, Hadjouni M. Investigation of open educational resources adoption in higher education using Rogers' diffusion of innovation theory. Heliyon. Jul 11, 2022;8(7):e09885. [CrossRef] [Medline]
    112. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. Dec 1991;50(2):179-211. [CrossRef]
    113. Entenberg GA, Dosovitsky G, Aghakhani S, Mostovoy K, Carre N, Marshall Z, et al. User experience with a parenting chatbot micro intervention. Front Digit Health. Jan 11, 2022;4:989022. [FREE Full text] [CrossRef] [Medline]
    114. Agapie E, Chang K, Patrachari S, Neary M, Schueller SM. Understanding mental health apps for youth: focus group study with Latinx youth. JMIR Form Res. Oct 18, 2022;6(10):e40726. [CrossRef] [Medline]
    115. Jakob R, Harperink S, Rudolf AM, Fleisch E, Haug S, Mair JL, et al. Factors influencing adherence to mHealth apps for prevention or management of noncommunicable diseases: systematic review. J Med Internet Res. May 25, 2022;24(5):e35371. [FREE Full text] [CrossRef] [Medline]
    116. Kontos E, Blake KD, Chou WY, Prestin A. Predictors of eHealth usage: insights on the digital divide from the health information national trends survey 2012. J Med Internet Res. Jul 16, 2014;16(7):e172. [FREE Full text] [CrossRef] [Medline]
    117. Eysenbach G. Improving the quality of web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. Sep 29, 2004;6(3):e34. [FREE Full text] [CrossRef] [Medline]
    118. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health. 2005;8(2):94-104. [FREE Full text] [CrossRef] [Medline]
    119. Fielding NG, Lee RM, Blank G. The SAGE Handbook of Online Research Methods. Thousand Oaks, CA. SAGE Publications Ltd; 2017.
    120. Podsakoff PM, MacKenzie SB, Lee JY, Podsakoff NP. Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol. Oct 2003;88(5):879-903. [CrossRef] [Medline]
    121. Liang X, Yan M, Li H, Deng Z, Lu Y, Lu P, et al. WeChat official accounts' posts on medication use of 251 community healthcare centers in Shanghai, China: content analysis and quality assessment. Front Med (Lausanne). Jun 12, 2023;10:1155428. [FREE Full text] [CrossRef] [Medline]
    122. Zeng L, Li RY, Zeng H. Weibo users and Academia's foci on tourism safety: implications from institutional differences and digital divide. Heliyon. Dec 10, 2022;9(3):e12306. [FREE Full text] [CrossRef] [Medline]
    123. Bao Z, Zhu Y. Understanding online reviews adoption in social network communities: an extension of the information adoption model. Inf Technol People. Sep 2023. (forthcoming). [CrossRef]
    124. Chou Y, Lin W. Blockbuster or flop? Effects of social media on the Chinese film market. SSRN J. Jul 25, 2023:1-41. [CrossRef]
    125. Lindell MK, Whitney DJ. Accounting for common method variance in cross-sectional research designs. J Appl Psychol. Feb 2001;86(1):114-121. [CrossRef] [Medline]
    126. Hair JJ, Sarstedt M, Hopkins L, Kuppelwieser V. Partial least squares structural equation modeling (PLS-SEM): an emerging tool for business research. Eur Bus Rev. Feb 2014;26(2):106-121. [CrossRef]
    127. Ringle CM, Sarstedt M, Straub DW. Editor's comments: a critical look at the use of PLS-SEM in "MIS Quarterly". MIS Q. Mar 2012;36(1):iii-xiv. [CrossRef]
    128. Freeze R, Raschke RL. An assessment of formative and reflective constructs in IS research. In: Proceedings of the Fifteenth European Conference on Information Systems. 2007. Presented at: ECIS 2007; June 7-9, 2007; St. Gallen, Switzerland.
    129. Bollen K, Lennox R. Conventional wisdom on measurement: a structural equation perspective. Psychol Bull. 1991;110(2):305-314. [CrossRef]
    130. Edwards JR, Bagozzi RP. On the nature and direction of relationships between constructs and measures. Psychol Methods. Jun 2000;5(2):155-174. [CrossRef] [Medline]
    131. Carl E, Witcraft SM, Kauffman BY, Gillespie EM, Becker ES, Cuijpers P, et al. Psychological and pharmacological treatments for generalized anxiety disorder (GAD): a meta-analysis of randomized controlled trials. Cogn Behav Ther. Jan 2020;49(1):1-21. [CrossRef] [Medline]
    132. Seegan PL, Miller MJ, Heliste JL, Fathi L, McGuire JF. Efficacy of stand-alone digital mental health applications for anxiety and depression: a meta-analysis of randomized controlled trials. J Psychiatr Res. Aug 2023;164:171-183. [CrossRef] [Medline]
    133. Al-Somali SA, Gholami R, Clegg B. An investigation into the acceptance of online banking in Saudi Arabia. Technovation. Feb 2009;29(2):130-141. [CrossRef]
    134. Viana Pereira F, Tavares J, Oliveira T. Adoption of video consultations during the COVID-19 pandemic. Internet Interv. Mar 2023;31:100602. [FREE Full text] [CrossRef] [Medline]
    135. Hair JFJ, Hult GT, Ringle CM, Sarstedt M. A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM). Thousand Oaks, CA. SAGE Publications; 2014.
    136. Henseler J, Ringle CM, Sarstedt M. A new criterion for assessing discriminant validity in variance-based structural equation modeling. J Acad Mark Sci. Aug 22, 2014;43:115-135. [CrossRef]
    137. Hair JF, Risher JJ, Sarstedt M, Ringle CM. When to use and how to report the results of PLS-SEM. Eur Bus Rev. 2019;31(1):2-24. [CrossRef]
    138. Johnson RE, Rosen CC, Djurdjevic E. Assessing the impact of common method variance on higher order multidimensional constructs. J Appl Psychol. Jul 2011;96(4):744-761. [CrossRef] [Medline]
    139. Ringle CM, Wende S, Becker JM. SmartPLS 4. SmartPLS GmbH. URL: http://www.smartpls.com [accessed 2024-10-13]
    140. Shmueli G, Sarstedt M, Hair JF, Cheah JH, Ting H, Vaithilingam S, et al. Predictive model assessment in PLS-SEM: guidelines for using PLSpredict. Eur J Mark. 2019;53(11):2322-2347. [CrossRef]
    141. Bradburn NM, Noll CE. The Structure of Psychological Well-Being. New York, NY. Aldine Publishing Company; 1969.
    142. Wilson L, Marasoiu M. The development and use of chatbots in public health: scoping review. JMIR Hum Factors. Oct 05, 2022;9(4):e35882. [FREE Full text] [CrossRef] [Medline]
    143. McKay FH, Wright A, Shill J, Stephens H, Uccellini M. Using health and well-being apps for behavior change: a systematic search and rating of apps. JMIR Mhealth Uhealth. Jul 04, 2019;7(7):e11926. [FREE Full text] [CrossRef] [Medline]
    144. Su Z, Figueiredo MC, Jo J, Zheng K, Chen Y. Analyzing description, user understanding and expectations of AI in mobile health applications. AMIA Annu Symp Proc. Jan 25, 2021;2020:1170-1179. [FREE Full text] [Medline]
    145. Huang CY, Yang MC, Huang MC. An empirical study on factors influencing consumer adoption intention of an AI-powered chatbot for health and weight management. Int J Performability Eng. 2021;17(5):422-432. [CrossRef]
    146. Hsiao CH, Chang JJ, Tang KY. Exploring the influential factors in continuance usage of mobile social apps: satisfaction, habit, and customer value perspectives. Telemat Inform. May 2016;33(2):342-355. [CrossRef]
    147. Bhattacherjee A. Understanding information systems continuance: an expectation-confirmation model. MIS Q. Sep 2001;25(3):351-370. [CrossRef]
    148. Okey OD, Udo EU, Rosa RL, Rodríguez DZ, Kleinschmidt JH. Investigating ChatGPT and cybersecurity: a perspective on topic modeling and sentiment analysis. Comput Secur. Dec 2023;135:103476. [CrossRef]
    149. Ogilvie L, Prescott J, Carson J. The use of chatbots as supportive agents for people seeking help with substance use disorder: a systematic review. Eur Addict Res. 2022;28(6):405-418. [FREE Full text] [CrossRef] [Medline]
    150. Rahimi B, Nadri H, Lotfnezhad Afshar H, Timpka T. A systematic review of the technology acceptance model in health informatics. Appl Clin Inform. Jul 2018;9(3):604-634. [FREE Full text] [CrossRef] [Medline]
    151. Spitzberg BH. Issues in the development of a theory of interpersonal competence in the intercultural context. Int J Intercult Relat. 1989;13(3):241-268. [CrossRef]
    152. Ibrahim A, Knox K, Rundle-Thiele S, Arli D. Segmenting a water use market: theory of interpersonal behavior insights. Soc Mark Q. Nov 19, 2017;24(1):3-17. [CrossRef]
    153. Kuhail MA, Thomas J, Alramlawi S, Shah SJ, Thornquist E. Interacting with a chatbot-based advising system: understanding the effect of chatbot personality and user gender on behavior. Informatics. Oct 10, 2022;9(4):81. [CrossRef]
    154. Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med. Jun 2017;7(2):254-267. [FREE Full text] [CrossRef] [Medline]
    155. Vaghefi I, Tulu B. The continued use of mobile health apps: insights from a longitudinal study. JMIR Mhealth Uhealth. Aug 29, 2019;7(8):e12983. [FREE Full text] [CrossRef] [Medline]
    156. Yuan S, Ma W, Kanthawala S, Peng W. Keep using my health apps: discover users' perception of health and fitness apps with the UTAUT2 model. Telemed J E Health. Sep 2015;21(9):735-741. [CrossRef] [Medline]
    157. Tavares J, Oliveira T. New integrated model approach to understand the factors that drive electronic health record portal adoption: cross-sectional national survey. J Med Internet Res. Nov 19, 2018;20(11):e11032. [FREE Full text] [CrossRef] [Medline]
    158. Yi MY, Jackson JD, Park JS, Probst JC. Understanding information technology acceptance by individual professionals: toward an integrative view. Inf Manag. Apr 2006;43(3):350-363. [CrossRef]
    159. Oliveira T, Thomas M, Baptista G, Campos F. Mobile payment: understanding the determinants of customer adoption and intention to recommend the technology. Comput Hum Behav. Aug 2016;61:404-414. [CrossRef]
    160. Rogers EM. Diffusion of Innovations. Washington, DC. Free Press; 2003.
    161. Min S, So KK, Jeong M. Consumer adoption of the Uber mobile application: insights from diffusion of innovation theory and technology acceptance model. J Travel Tour Mark. Sep 04, 2018;36(7):770-783. [CrossRef]
    162. Al-Rahmi WM, Yahaya N, Alamri MM, Alyoussef IY, Al-Rahmi AM, Kamin YB. Integrating innovation diffusion theory with technology acceptance model: supporting students’ attitude towards using a massive open online courses (MOOCs) systems. Interact Learn Environ. Jun 20, 2019;29(8):1380-1392. [CrossRef]
    163. Zhang X, Yu P, Yan J, Ton A M Spil I. Using diffusion of innovation theory to understand the factors impacting patient acceptance and use of consumer e-health innovations: a case study in a primary care clinic. BMC Health Serv Res. Feb 21, 2015;15:71. [FREE Full text] [CrossRef] [Medline]
    164. Li H, Novack DH, Duke P, Gracely E, Cestone C, Davis T. Predictors of medical students' ethical decision-making: a pilot study using the Theory of Interpersonal Behavior. Patient Educ Couns. May 23, 2020:S0738-3991(20)30292-5. (forthcoming). [CrossRef] [Medline]
    165. Jacob C, Sanchez-Vazquez A, Ivory C. Understanding clinicians' adoption of mobile health tools: a qualitative review of the most used frameworks. JMIR Mhealth Uhealth. Jul 06, 2020;8(7):e18072. [FREE Full text] [CrossRef] [Medline]
    166. Kaur D, Uslu S, Rittichier KJ, Durresi A. Trustworthy artificial intelligence: a review. ACM Comput Surv. Mar 31, 2023;55(2):1-38. [FREE Full text] [CrossRef]
    167. Thieme A, Belgrave D, Doherty G. Machine learning in mental health: a systematic review of the HCI literature to support the development of effective and implementable ML systems. ACM Trans Comput Hum Interact. Aug 17, 2020;27(5):1-53. [CrossRef]
    168. Gao J, Yan D, Dong M. Black-box adversarial attacks through speech distortion for speech emotion recognition. EURASIP J Audio Speech Music Process. Aug 17, 2022;2022. [CrossRef]
    169. Coghlan S, Leins K, Sheldrick S, Cheong M, Gooding P, D'Alfonso S. To chat or bot to chat: ethical issues with using chatbots in mental health. Digit Health. 2023;9:20552076231183542. [FREE Full text] [CrossRef] [Medline]
    170. Fitzpatrick KK, Darcy A, Vierhile M. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Ment Health. Jun 06, 2017;4(2):e19. [FREE Full text] [CrossRef] [Medline]
    171. Maenhout L, Peuters C, Cardon G, Compernolle S, Crombez G, DeSmet A. Participatory development and pilot testing of an adolescent health promotion chatbot. Front Public Health. Nov 11, 2021;9:724779. [FREE Full text] [CrossRef] [Medline]
    172. Koulouri T, Macredie RD, Olakitan D. Chatbots to support young adults’ mental health: an exploratory study of acceptability. ACM Trans Interact Intell Syst. Jul 20, 2022;12(2):1-39. [CrossRef]
    173. Frishammar J, Essén A, Bergström F, Ekman T. Digital health platforms for the elderly? Key adoption and usage barriers and ways to address them. Technol Forecast Soc Change. Apr 2023;189:122319. [CrossRef]
    174. Abd-Alrazaq AA, Rababeh A, Alajlani M, Bewick BM, Househ M. Effectiveness and safety of using chatbots to improve mental health: systematic review and meta-analysis. J Med Internet Res. Jul 13, 2020;22(7):e16021. [FREE Full text] [CrossRef] [Medline]
    175. Pandey S, Sharma S, Wazir S. Mental healthcare chatbot based on natural language processing and deep learning approaches: Ted the therapist. Int J Inf Technol. Jun 02, 2022;14(7):3757-3766. [CrossRef]
    176. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res. Aug 05, 2011;13(3):e52. [FREE Full text] [CrossRef] [Medline]

    ‎
    ABC: attitude-behavior-context
    AI: artificial intelligence
    AVE: average variance extracted
    CR: composite reliability
    DOI: diffusion of innovation
    HTMT: heterotrait-monotrait
    ITE: intention to engage
    mHealth: mobile health
    PLS-SEM: partial least squares structural equation modeling
    SF: social factors
    TIB: theory of interpersonal behavior
    VIF: variance inflation factor

    Edited by A Kushniruk, E Borycki; submitted 27.04.24; peer-reviewed by CL Gan, Y Shahsavar; comments to author 07.07.24; revised version received 25.08.24; accepted 13.09.24; published 11.11.24.

    Copyright

    ©Yanrong Yang, Jorge Tavares, Tiago Oliveira. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 11.11.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.