Globally, the prevalence of mental illness is rising and health care systems are under increasing pressure [1]. In Australia, individuals face barriers to mental health care, including financial costs and extended wait times for appointments, while practitioners are struggling to meet demand [2,3]. In this context, artificial intelligence (AI) is a potential game changer, offering 24/7 support for individuals, thus alleviating pressure on mental health practitioners [4,5]. Current models engage users in therapeutic conversations, with many using “rule-based” scripted responses and predefined dialog structures [6]. This technology differs from generative AI chatbots, such as ChatGPT (OpenAI), which generate original content and provide lifelike interactions that closely imitate humans.

Generative AI can offer more empathetic emotional support in comparison to rule-based chatbots [4,7-9], making it more effective in counseling conversations and more acceptable to clients. A recent qualitative study exploring the efficacy of ChatGPT in supporting 24 individuals with mental health conditions found that more than 50% of participants found ChatGPT to be empathetic and 80% said the tool helped to manage symptoms [10]. However, participants also raised concerns regarding data privacy and the risk of personal data theft, uncertainty in the reliability of the information shared and the quality of training data, and a lack of understanding of cultural determinants of mental health.

Generative AI chatbots may also provide improved access to mental health information and psychoeducation. Preprint and peer-reviewed studies assessing the quality of psychoeducational information generated by ChatGPT on depression, anxiety, psychosis, and substance abuse found that it was accurate, clear, clinically useful, relevant, and empathetic [11-13]. Furthermore, studies have found generative AI chatbots to be effective in supporting adherence to psychological treatments [14] and delivering positive psychology interventions [15,16]. These studies indicate that generative AI chatbots may improve individuals’ access to information and basic therapeutic exercises, allowing clinicians to focus on more complex tasks, such as diagnosis and treatment.

Studies have identified a variety of uses of AI for practitioners, including research, documentation, administering psychological assessments, clinic management, and triaging [5,17-19]. Generative AI could enhance these functions by offering faster processing speeds, more comprehensive client histories, and higher empathy [20,21]. However, like rule-based AI models, studies have found generative AI chatbots are prone to mistakes, with one study finding that ChatGPT included incorrect information in 36% of client documents in a medical setting [21]. Another study evaluated ChatGPT’s performance in conducting mental health assessments and found that, while responses for simple cases were acceptable, recommendations for more complex cases were inappropriate and potentially harmful [22]. These findings highlight the risks associated with AI, including hallucinations and misinterpretations.

Generative AI may also be capable of more sophisticated functions, such as recommending diagnoses and treatment options. Bartal et al [23] found that ChatGPT was able to accurately identify posttraumatic stress disorder (PTSD) following childbirth by analyzing individuals’ birth stories, achieving high sensitivity (85%) and specificity (75%), aligning with commonly accepted benchmarks of reliability [24,25]. Similarly, another case study demonstrated that ChatGPT can identify treatment-resistant schizophrenia and suggest appropriate treatment options [26]. Despite these innovative applications, some argue that there is a need for significant improvement before AI is adopted in mental health care [27].

Clinicians appear cautious about the use of AI and feel more education and training are required [28,29]. In a systematic review of AI in mental health care, Rogan et al [28] found that most studies highlighted a need for clinician training, emphasizing that “AI will only be useful if clinicians understand and feel comfortable using it”. Similarly, in a survey of 138 American psychiatrists, Blease et al [29] found that 90% of participants agreed that clinicians need more support to understand generative AI. Qualitative responses were mixed, with some participants expressing uncertainty and concerns relating to potential harm and others overestimating the readiness of generative AI to deliver clinical tasks.

While practitioners have expressed a need for more training and support, many may already be using AI tools for internal purposes. Blease et al [29] found that more than half of psychiatrists surveyed had used generative AI chatbots to answer clinical questions. A survey of 86 Australian mental health professionals found that 43% had used AI and that ChatGPT was the most commonly used tool [19]. Likewise, a systematic review preprint on the use of generative AI in health care found that the technology is “heavily applied” for documentation, research, and administration among medical practitioners [30].

Research examining practitioners’ use of generative AI chatbots with clients is more limited. One study found strong support among clinicians to use chatbots with clients, with 80% of participants reporting they would be either very likely (24%) or somewhat likely (56%) to recommend AI mental health chatbots to clients within the next 5 years [31]. Participants may have been more supportive as they were asked to consider their future likelihood and, therefore, the potential for evolution of both the technology and their understanding.

This study aims to explore which functions of generative AI chatbots mental health practitioners think are most useful, and to what extent they would recommend them to clients. In addition, this study will investigate the impact of exposure to a generative AI chatbot on participants’ likelihood of recommending chatbots to clients.

This study uses both quantitative and qualitative data to address 3 research questions (RQ) aligned with corresponding hypotheses. These RQs address key gaps in the literature, including limited insight into which functions practitioners prefer, a lack of empirical data on their willingness to recommend generative AI chatbots to clients, and minimal understanding of how exposure to the technology might influence those attitudes.

RQ1 explores practitioners’ views on the most useful functions of generative AI chatbots in mental health care. It is hypothesized that functions such as generating case notes (hypothesis 1), supporting session planning (hypothesis 2), and identifying literature (hypothesis 3) will be prioritized. Previous research suggests clinicians favor using AI for internal tasks like documentation and research [29,30].

RQ2 examines the likelihood of mental health practitioners recommending a generative AI chatbot to clients. It is hypothesized (hypothesis 4) that fewer than 50% will do so, as practitioners typically have low technology literacy [28,29,32].

RQ3 investigates whether practitioners are more likely to recommend an AI chatbot after a demonstration, with the prediction (hypothesis 5) that exposure to a demo will significantly increase their likelihood to recommend it. Research indicates that familiarity with health care technologies improves clinician adoption [33].

Participants were recruited using a purposive, convenience sampling method. Digital flyers were distributed on forums including the Australian Psychological Society (APS), Australian Association of Psychologists Inc (AAPi), through Melbourne University channels, and with friends and family. Individuals who were not working in a client-facing, mental health role were excluded from the study.

In total, 23 individuals participated in the study, and all were working in Australia. Sample demographics are presented in Table 1.

The study team devised interview questions (Multimedia Appendix 1) to explore practitioners’ perspectives on generative AI chatbots in mental health care. The interviews were of 45 minutes and conducted over Zoom (Zoom Video Communications, Inc.) between May and August 2024. Open-ended, Likert-scale, and ranking questions were used to collect both qualitative and quantitative data. Interviews were recorded and transcribed using AI software, Notta (Notta Inc), to support qualitative analysis.

Participants were asked to select their top 3 “most useful” functions of generative AI chatbots from a list of 11 options and explain their reasoning. The uses were:

An option to select “Other” was offered and subsequently removed from the analysis as it was not selected by any participants.

Participants were asked, “How likely are you to recommend a chatbot to a client?” on a Likert scale of 1 (Highly unlikely) to 4 (Highly likely). This question was asked both before (T1) and after (T2) viewing a demonstration of a generative AI chatbot.

The demonstration was an 11-minute screen recording of a generative AI chatbot that had been built using GPT-4o (OpenAI) for the purposes of this research. During the demonstration, a fictional client (“Saman”) interacted with the chatbot to illustrate a range of use cases, from triaging, psychological assessment, clinic onboarding, counseling, and completing a gratitude exercise. The demonstration also showed Saman’s fictional therapist using the chatbot to generate case notes, identify literature relevant to Saman’s profile, and plan future sessions (Multimedia Appendix 2).

Although the demonstration featured a chatbot built using GPT-4o, participants were asked to rate and reflect on generative AI chatbots for mental health support more generally, rather than the specific tool presented. Only the text-based conversational capacity of GPT-4o was used, with safety filters (mechanisms that monitor and restrict the model’s output to reduce risk of harm), and real-time context (allowing the chatbot to access external data and information) active by default. These features were not explicitly evaluated but were considered reflective of current generative AI chatbot capabilities.

A thematic analysis was conducted following Braun and Clarke’s [34] 6-phase framework. Analysis began with familiarization, where transcripts were reviewed to identify initial ideas. Themes were then generated based on recurring concepts, iterated, and refined to ensure they captured core insights. Finally, qualitative themes were considered alongside the quantitative results to identify key research findings. Where participants are quoted, a reference is provided with their participant identification number (#), gender (men or women), and age.

All participants provided informed consent and no compensation was offered. Participants were informed in advance that participation was voluntary, and they could withdraw at any time without consequence. Data were stored securely, and participant confidentiality was secured by anonymizing all data. Ethics approval was obtained from the University of Melbourne Human Research Ethics Committee (#28479).

Thematic analysis produced 9 qualitative themes (QTs), each aligned to a particular RQ, as shown in Figure 1. These themes are reported in further detail throughout the following section.

Proportions of participants who selected each use in their top 3 are presented in Figure 2.

A binomial test was conducted to assess whether Generating case notes was selected in participants’ top 3 choices at greater than chance levels. The chance probability was calculated at 0.25, based on the probability of a participant selecting an option, acknowledging that they had 3 choices and could not select the same option twice (P(selected)=1 - [1- 1/11] x [1 - 1/10] x [1- 1/9]). The results were significant, P=.001, indicating that Generating case notes was selected at greater than chance levels. A binomial test indicated that Support with session planning was not selected in participants’ top 3 choices at greater than chance levels (0.25; P=.86). A binomial test indicated that Identifying literature and Suggesting literature was not selected in participants’ top 3 choices at greater than chance levels (0.25; P=.10).

The distribution of participants’ likelihood of recommending a chatbot to a client at T2 is presented in Figure 3. Data from T2 (postdemonstration) was used for the analysis, as it best reflected participants’ informed perspectives of generative AI chatbots. One participant responded, “I don’t know,” and their data were excluded from the quantitative analysis.

A binomial test was conducted to determine whether participants would be likely to recommend a generative AI chatbot to a client (defined as a response of 3 or 4 on the 4-point Likert scale). The proportion of participants likely to recommend was 55% (12/22), which was not significantly different from the 50% threshold (P=.74; odds ratio 1.20 [95% CI 0.47-3.17]). A Bayesian binomial test provided moderate evidence for the null hypothesis that more than 50% of participants would be likely to recommend a chatbot (BF₀₁=5.24).

A one-tailed paired samples t test was conducted to assess whether there was a statistically significant increase in the likelihood of recommending a chatbot from T1 (mean 2.09 [SD 1.07]) to T2 (mean 2.64 [SD 0.79]). Results were significant, t₂₁=3.20; P=.002, with a mean difference of 0.55. The effect size (d=0.68) was medium-to-large [35], suggesting a meaningful increase in recommendation likelihood.

A Bayesian paired samples t test provided strong evidence in favor of the alternative hypothesis (r=0.707; BF₁₀=20.08), indicating that participants were more likely to recommend a chatbot after the demonstration (T2) compared to before (T1).

This study found that practitioners appear to prefer administrative uses of generative AI chatbots, such as generating case notes, and are uncertain about recommending chatbots to clients. While exposure to a demonstration may increase the likelihood of recommending a chatbot, concerns regarding client safety and risk persist.

The first hypothesis (hypothesis 1) was supported. Generating case notes was selected as the most useful function of generative AI chatbots at greater than chance levels. The second (hypothesis 2) and third (hypothesis 3) hypotheses were not supported. Support with session planning and Identifying and suggesting literature were not selected at greater than chance levels. The fourth hypothesis (hypothesis 4), that less than 50% of participants would be likely to recommend a chatbot to a client, was not supported. The fifth hypothesis (hypothesis 5) was supported; participants were significantly more likely to recommend a chatbot to a client postdemonstration, and Bayesian analysis provided strong evidence for this increase.

The results indicate a preference among practitioners for using generative AI chatbots for simple, administrative tasks. Quantitative and qualitative data highlight that practitioners see value in generative AI chatbots for drafting and summarizing case notes. Participants expressed enthusiasm for the potential administrative efficiencies that using a chatbot would allow more time to focus on client-facing work. This aligns with previous studies where practitioners reported using generative AI tools for documentation and research [19,29,30]. This preference is also evident in the influx of AI-based tools for case note documentation in the market, such as Heidi (Heidi Health), Upheal (Upheal), and Autonotes (Autonotes AI, LLC) [36-38].

There is; however, some nuance in the types of administrative tasks practitioners are willing to use generative AI chatbots for. While descriptive data showed that Identifying and suggesting literature was the second most frequently selected use, it was not selected at greater than chance levels. Similarly, there was a lack of support for Session planning. Participants emphasized the importance of human oversight, indicating a lack of trust in generative AI. This aligns with findings from Blease et al [29], where psychiatrists were cautious of using generative AI for tasks that require clinical expertise and raised concerns about accuracy. As such, while practitioners may not be willing to delegate more complex administrative tasks to generative AI chatbots, this study suggests that there is potential for the technology to be applied in a supporting capacity.

This study found no clear preference for or against recommending a chatbot to clients, with uncertainty evident in both quantitative and qualitative results. Both the frequentist and Bayesian analyses reflected statistical uncertainty regarding the true proportion of participants who were likely to recommend a chatbot to clients. Qualitative analysis highlighted several ethical concerns, ranging from professional liability to confidentiality and data security, and a lack of familiarity with the technology. This aligns with previous research, which found that practitioners often harbor ethical concerns and lack education on the appropriate use of generative AI tools in practice [29,32].

This uncertainty contrasts with Sweeney et al [31], where 80% of practitioners were likely to recommend a chatbot to a client. This variance may be explained by the difference in how questions were phrased. In Sweeney et al [31] study, participants were asked whether they would recommend a chatbot “in the next five years,” allowing practitioners to consider how the technology, and their understanding of it, may evolve. In contrast, the present study focused on immediate likelihood, which may have biased participants toward considering current limitations. Participants also expressed optimism about the future of generative AI, suggesting they may have been more supportive if asked to consider their likelihood of recommending a chatbot in the future.

A unique insight from this study was practitioners’ support for conditional use of generative AI chatbots, depending on factors unique to the client. Qualitative responses indicated that participants were more inclined to recommend chatbots for low-risk, less complex cases (ie, stress, sleep issues, and life transitions) compared to high-risk (ie, suicidal intent) or trauma-related cases. The existing literature on the appropriate client profiles for generative AI chatbots presents mixed views. Some propose that AI is appropriate for use with clients with complex clinical histories, on the basis that the technology can analyze and track data on, for example, genetic markers and behavioral patterns [39]. Others suggest that AI chatbots may be most appropriate in prevention and psychoeducation in low-risk populations [40]. Ultimately, like any mental health treatment, the success of generative AI tools will depend not only on the technology, but also on its fit for the client.

This study found that participants’ likelihood of recommending a generative AI chatbot to a client increased significantly after a demonstration. Qualitative insights provided depth to the quantitative results, as participants explained that the demonstration had increased their understanding of how chatbots can be applied in mental health care. This result aligns with earlier findings of a positive correlation between health care practitioners’ knowledge of generative AI chatbots and positive attitudes toward the technology [41]. Similarly, previous studies have found that negative perceptions of AI among mental health professionals may be due to minimal exposure [29,42].

Alternatively, participants’ increased recommendation likelihood postdemonstration may be reflective of novelty bias, where individuals tend to exhibit increased enthusiasm toward a new or unfamiliar technology or idea [43]. Qualitative data suggests that for 22 out of 23 participants (96%), the demonstration was the first time they had seen a generative AI chatbot used in a mental health setting. In a similar study by Liu et al [16], it was observed that participants responded more positively to a chatbot trained to deliver positive psychology interventions during initial interactions.

Surprisingly, despite the quantitative increase in recommendation likelihood, qualitative responses indicated that the demonstration heightened client safety concerns for some participants. Of the ten participants who reported an increased recommendation likelihood, 4 described being more concerned about the risks. This suggests that a greater understanding of the associated risks of using generative AI chatbots may play a role in adoption among practitioners. This aligns with results from Zhang et al [42], where participants described the importance of understanding the risks of AI to “clinician buy-in” and adoption of the technology in practice. While an examination of potential risks was beyond the scope of this study, this finding underscores the importance of providing evidence-based, objective education to practitioners.

Several limitations of this study should be addressed. First, convenience sampling may have introduced selection bias, as individuals who were more supportive of AI technologies may have been more likely to participate, leading to an overrepresentation of positive attitudes. Future studies should apply random sampling methods to reduce selection bias, ensure representativeness, and improve external validity [44]. In addition, participants may have been aware that the interviewers created the chatbot demonstration, introducing demand characteristics and potential bias toward more favorable responses. The use of independent researchers to administer questionnaires may assist in mitigating potential bias in future research.

A primary focus of this research was to gather in-depth qualitative data to allow for greater exploration of practitioners’ perspectives. Due to project constraints and the time required of participants, the study sample size was small (N=23). While this number is appropriate for qualitative thematic research [45], it limited the statistical power and reliability of the quantitative analyses [46]. No a priori power calculation was conducted, as the study was primarily exploratory and qualitatively focused. In addition, the sample was Australian, predominantly female (17/23, 74%) and mostly psychologists and counselors (13/23, 56%). Caution should be exercised when generalizing these findings to other health systems or sociocultural contexts. In the future, researchers should recruit larger, more representative samples in different cultural contexts, while still maintaining the qualitative analysis that is crucial for understanding the nuanced perspectives of practitioners.

Participants were asked to rate their likelihood of recommending a chatbot to a client on a 4-point Likert scale. While the absence of a neutral midpoint encouraged participants to share their perspectives, it may have forced practitioners to choose a position that they did not fully support.

This study did not consider recent developments in generative AI chatbot technology, such as voice recognition or more sophisticated emotional detection capabilities [47]. Researchers may consider partnering with AI developers to test emerging technologies to ensure that studies remain relevant, while also allowing for practitioner and user feedback on tools before they are released to market.

For practitioners, the findings suggest that generative AI chatbots could be valuable in alleviating their administrative burden. However, the study also highlights the need for careful consideration as to how AI is applied and the need for practitioners to improve their understanding of AI chatbots, particularly as their clients may already be using them [22].

Educational institutions and industry associations play an important role in ensuring practitioners have appropriate AI skills and literacy and are prepared to deliver services in a field where technology is playing an increasingly important role [48]. Practical, evidence-based guidelines for the use of AI tools should address concerns expressed in this study regarding liability, client safety, and ethics, while also enabling effective use of technology to support positive mental health outcomes. Practitioners should also be informed of emerging research so as to help validate their decision-making and points of view.

For AI developers, insights from this study may support iteration and refinement of generative AI chatbots for mental health care. For example, developers should ensure practitioners’ concerns regarding client safety, data security, and accuracy are addressed in AI tools and service agreements. This study also highlights the need for developers to engage practitioners, as well as users, throughout the design process to support safe and effective technology implementation.

Generative AI chatbots have significant potential to play an important role in the delivery of mental health care. In the immediate term, their greatest value may be in administrative, nonclient-facing tasks, to alleviate practitioner workload and allow greater attention to be directed to therapeutic work. This study highlighted the complexity and nuance of practitioners’ perspectives on generative AI chatbots. While there is interest and optimism, there is a lack of familiarity with AI tools and uncertainty regarding client-facing uses. Furthermore, exposure to AI technology may play an important role in supporting adoption among practitioners.

Future research should continue to explore practitioners’ views on administrative and client-facing uses with larger, more representative samples to enhance the reliability and generalizability of findings. It would also be beneficial to explore hybrid models and the benefits of therapist-AI collaboration. Such evidence could inform objective guidelines to allow practitioners to make informed decisions on their use of generative AI chatbots. Ultimately, this study provides valuable contributions as to “how,” rather than simply “whether,” generative AI chatbots may be integrated into mental health care, guiding more thoughtful, stakeholder-informed implementation.