In terms of BMI, 42% of the Swiss population is overweight or obese [1]. To reach a sustainable weight reduction, restrictive and malabsorptive bariatric surgeries are one of the most effective methods [2,3]. Therefore, the number of procedures has more than quadrupled in the last 20 years [4]. To achieve a successful outcome of bariatric surgery, patients need to be provided with broad knowledge of food intolerances, dumping syndrome, and protein intake [5-10]. Therefore, patients need to be informed in detail before bariatric surgery to know what to expect and what kind of nutritional and behavioral changes must be made after the surgery [5]. To seek help for addressing these problems, patients use a variety of sources, such as websites [7,8]. These patients are in great need of satisfying and reliable answers to all their open questions [7,8,11,12]. In this regard, accessibility to and regular contact with a registered dietician are of utmost importance because they have been shown to remain the main source of reliable information, advice, and support for patients [7,8,11]. Preoperative dietetic counseling shows a positive effect on the outcome of bariatric surgery and benefits for weight loss [13,14]. In addition to preoperative counseling, the Swiss Society for the Study of Morbid Obesity and Metabolic Disorders highly recommends regular postoperative nutritional assessment and counseling [15]. Patients seem to need easy access to in-between dietetic consultations [7].

Recent findings highlight the potential of novel artificial intelligence (AI)–based technologies, such as mobile phone apps and web-based platforms, in improving patient support and weight loss after bariatric surgery [7,16,17]. Versteegden et al [17] showed that eHealth platforms used postoperatively, with topics such as information dissemination regarding obesity and bariatric surgery, can lead to significantly greater weight loss at 1 and 2 years postoperatively. In addition, there is a specific recommendation for combining accessible information for patients with obesity in electronic and nonelectronic media [18]. A recent study [19] showed good acceptance and usability of a smartphone app for postoperative care for bariatric surgery. This program was based on a standardized questionnaire, which patients completed in the app periodically, as well as reminders and push notifications to take supplements and engage in physical activity. In general, web-based health information is a support for patients and can potentially lead to more productive conversations with health care professionals, as frequently asked questions (FAQs) can already be answered before a visit [20]. Furthermore, it is an opportunity to provide evidence-based support for patients who do not require an expensive and time-consuming visit with a health care professional but nevertheless need information and advice in between visits with the responsible dietician [21,22].

People have access to information in all areas around the clock. Incorrect information about nutrition can easily be found in chat rooms on social media, and the amount of information on the internet can be overwhelming for patients [6,20,23,24]. Evidence-based health bots (HBs) could potentially fill the gap in providing assistance and information while preventing patients from consuming incorrect information on other internet-based platforms [12,24]. Current evidence on health chatbots and AI shows that they mainly focus on nutritional and neurological disorders [25], physical activity [26,27], and mental health [27]. Future research studies should address the concern of the lack of data about the acceptability and usability of patient-centered eHealth tools among patients [12,20,23,28-30]. Although the possible benefits of an HB are seen in creating more time for dieticians to focus on behavioral and individual support, as simple knowledge questions can be cleared by the HB [12,17,21,22,24], no studies are available on the use of an HB in the dietetic treatment of patients in the bariatric setting. Therefore, this study aimed to explore how patients receiving bariatric care rate the overall usability, benefits, risks, strengths, and weaknesses of and trust in an HB for nutrition-related questions before and after a bariatric intervention. The second aim was to evaluate how patients receiving bariatric care rated the quality of the answers generated by the HB to their nutrition-related questions [31].

The elaborated knowledge corpus was developed based on patient documents from the collaborating clinic and FAQ sheets from various bariatric centers in Switzerland. Two feedback loops, the incorporation of the collected feedback, and 3 fine-tuning iterations were carried out during the development of the HB. The feedback loops were conducted with the help of experienced nutritionists. The technology used was Hugging Face [32], which is an AI specialized in recognizing same sentences. This model was designed to compute sentence embeddings for English and German texts. The question that is entered in the HB by the user is compared with the questions in the model. The question that is most similar to it is used, and its answer is communicated to the user. This means that AI does not learn the questions but just simply hand overs questions and answers. The latest HB version was tested in a patient study (Beyeler, M. unpublished data, 2022) with 161 questions and showed the following outcomes: 85 (52.8%) questions were answered correctly by the HB, and 76 (47.2%) questions were not answered satisfactorily. Of these 76 questions, 36 (47%; 22.4% of the total questions) were not included in the knowledge corpus and, therefore, could not be answered, and 40 (53%; 24.8% of the total questions) questions were included in the knowledge corpus but provided with a nonmatching answer. In Figure 1, an example of an HB-generated answer is presented.

A study evaluating the usability and performance of an HB was conducted, in which quantitative and qualitative methods are applied independently [33-35]. The study was conducted via face-to-face interviews with patients with obesity in the preoperative and postoperative settings, which took approximately 45 minutes. The study took place in a bariatric center. First, a short qualitative interview with 4 questions was conducted. The second task was the testing of the HB, wherein the participants asked the HB nutrition-related questions. Regarding the following predefined categories, which correspond to the structure of the HB’s knowledge, at least 1 question per category should have been asked per person: postoperative diet plan, mealtime rhythm, protein, dumping syndrome, liquids, food tolerance, vitamins, digestion, quantity of food, and others. In “others,” the participants were free to ask any other bariatric nutrition–related questions. Participants were also encouraged to ask more than 1 question per category to be able to generate a higher quantity of questions, which could possibly be included in a further development cycle. The questions’ content and wording were generated by the participants. The satisfaction with the answers of the HB had to be evaluated after each question. After the testing phase, participants completed a web-based questionnaire with 46 items. At the end, another qualitative interview with 8 questions was conducted. The study procedure is illustrated in Multimedia Appendix 1.

In all, 12 participants were recruited from September 2021 to January 2022 at a specialized bariatric center in Berne, Switzerland. Potential patients who entered the bariatric center had their first appointment with a specialized medical physician, followed by various medical clarifications, including nutritional counseling from a dietician. In this counseling session, the patients were asked whether they were interested in participating in the study. In case of consent, the potential study participants were contacted by the research team for an appointment and to clarify their questions.

Eligible participants were defined as adults (aged ≥18 y) with obesity (BMI≥35 kg/m²) from Switzerland who were planning to undergo a Roux-Y gastric bypass or sleeve gastrectomy bariatric surgery in the next 3 months or who had undergone one of the mentioned surgeries within the last 2 months. As comparable usability studies with 7 to 21 participants achieved a high detection rate, we decided to select a sample of 10 to 14 participants with an equally distribution of patients before surgery and patients after surgery [31,33,36-39]. Participants were selected based on the need for bariatric surgery (Roux-Y gastric bypass or sleeve gastrectomy) according to the Swiss Society for the Study of Morbid Obesity and Metabolic Disorders criteria [40]. In addition, potential participants had to be proficient in German, as the HB was available only in the German language. Furthermore, patients must have had at least 1 preoperative dietetic counseling session. This ensures basic knowledge about bariatric nutrition among participants, which is helpful for getting ideas about what questions to ask the HB [5,10,11]. For participants after surgery, the time frame for the survey was up to 2 months after surgery, as the HB’s knowledge base was primarily developed for this period because most adaptations to the patient’s diet must be made within the first 2 months after surgery [10,11]. Patients with obesity who received conservative or drug-related weight reduction therapy were excluded.

As a usability study bears only very minimal risks for the participants, no ethics approval was required [41], as confirmed by the Business Administration System for Ethics Committees, which rejected jurisdiction (Business Administration System for Ethics Committees–Nr: Req-2021-00952). Therefore, this study was not registered at ClinicalTrials.gov.

All individuals participated voluntarily and did not receive monetary compensation. They were free to withdraw their participation at any time. An informed consent form, which included information about the study aim and methodology, was signed by the participants before participation. Other than the inclusion criteria, there was no collection of health-related data in this study.

The study consisted of a qualitative part, which was conducted by MB using 4 questions at the beginning of each session with the participant and 8 questions at the end of the session. These items were specifically developed for this study and are presented in Textbox 1. The interviews aimed to gain deeper insight into participants’ perceptions of the HB. In addition, the topics “perception,” “strengths,” “weaknesses,” and “further development” were explored, which could be better embedded in an interview than in a questionnaire. After the first patient interview, small adaptation to the interview questions were made for improvement. The 2 interview sequences were recorded with a smartphone and named under the participant’s assigned ID as part 1 or 2. The audio recordings were then saved locally on a laptop for further processing and deleted from the smartphone. With the support of the f4transkript (audiotranskription) software, MB created semantic content transcripts from the interviews according to the simple transcription rules of Kuckartz [42]. Subsequently, a summary inductive content analysis according to Mayring [43] was performed. This step was performed manually, and the data were entered into an Excel (Microsoft Corp) database. The focus of this further processing was on summarizing and paraphrasing the transcripts, with the goal of concentrating the content and formulating summarized answers by topics.

The questionnaire for quantitative data collection was built in a web-based survey tool called UmfrageOnline, which is available only through encrypted connections [44]. The questionnaire was divided into 2 segments. The first segment consisted of the System Usability Scale (SUS) [45] validated in German, which is the main spoken language at the location of the survey’s execution [46]. The SUS consists of 10 items and is one of the most widely used standardized usability questionnaires [47,48]. The answers are ranked on a 5-point Likert scale [49], with positive and negative formulations alternating to prevent response bias [47,50]. Because the SUS did not cover all topics of interest for this study, a second part of the questionnaire was created. A total of 4 frequently used usability and acceptability questionnaires—the Telehealth Usability Questionnaire [51]; Service User Technology Acceptability Questionnaire [52,53]; Usefulness, Satisfaction, and Ease of Use Questionnaire [54]; and Post-Study System Usability Questionnaire [55]—were selected and evaluated according to the research question as well as the HB functionalities. After removing redundant and duplicate items, 28 of the total 92 items were selected and used in the questionnaire. According to the usability study by Li et al [56], 2 items each from the categories “intention to share information” and “intention to seek information” were also added [56]. According to the categories used in the abovementioned questionnaires, the final items were assigned to the following dimensions: usability (3 items), usefulness (6 items), user-friendliness and learnability (6 items), interface quality (4 items), reliability (1 item), satisfaction (4 items), risks (2 items), benefits (2 items), intention to share (2 items), and intention to seek (2 items). Similar to most of the used sources, the answer options of the questionnaire were presented on a 7-point Likert scale [51-57]. Furthermore, 4 demographic questions, namely those on sex, age, highest level of education, and digital ability, were included at the end of the questionnaire [36-38,58].

To obtain quantitative data about the participant’s satisfaction with the answers the HB provided in the testing, participants were asked to rank each answer. A 5-point Likert scale was included right below the answer, with the following options: very good (1), good (2), acceptable (3), bad (4), and very bad (5). Participants were asked to rate the answers according to their personal satisfaction.

Data processing and statistical analysis were performed in R Studio (version 3.6.1; R Studio Inc), with attached base packages GlobalEnv, tools:rstudio, package:stats, package:graphics, package:grDevices, package:utils, package:datasets, package:methods, Autoloads, and package:base [59]. To determine the SUS score, which ranges from 0 to 100, each answer option was assigned a number from 0 to 4, taking the positive or negative formulation of the question into account. All items were summed up, and the resultant was multiplied by 2.5 [47]. The interpretation of the SUS score was based on the study by Bangor et al [60], with the highest score being 100 [60]. For the SUS, 1 patient was excluded from the evaluation because they got confused with the questions phrased alternately positive and negative. For the remaining part of the questionnaire, the participant ensured that the questions were read carefully and was able to answer correctly.

The second part of the questionnaire was analyzed through descriptive statistics of each item, namely mean and SD. The response options ranged from 1 to 7, with 1 representing “strongly disagree” and 7 representing “strongly agree.” Each dimension in the questionnaire (eg, benefits) was presented separately, with mean and SD calculated for each dimension [61]. The internal consistency of the dimensions with at least 3 items was analyzed using Cronbach α [62]. The dimensions “usability” (Cronbach α=.87), “usefulness” (Cronbach α=.92), “user-friendliness and learnability” (Cronbach α=.91), and “satisfaction” (Cronbach α=.95) showed very good values (raw Cronbach α>.8), and “interface quality” showed an acceptable value (Cronbach α=.61). To explore a possible correlation between digital affinity and the different categories, Spearman correlations were calculated [63]. Owing to the small cohort size, the mean values of all categories were compared between the before surgery and after surgery groups using the nonparametric Mann-Whitney U test [63].

The number and percentage of questions asked in the HB that fell under each category, as well as for the received score from 1 (very good) to 5 (very bad) were calculated.

Table 1 presents an overview of the characteristics of the 12 participants included in this study. Among the 12 patients, 8 (67%) were female, and the majority (n=10, 83%) were aged between 18 and 49 years. Education was evenly distributed. For self-assessed digital affinity, which was scored 1 (none) to 10 (expert), the mean score was 6.9 (SD 1.98). Of the 12 participants, 7 (58%) were in the preoperative phase, and 5 (42%) were in the postoperative phase.

The Mann-Whitney U test (Table 2) showed no significant difference between the before surgery and after surgery groups in scores for any item, including the SUS (P=.06; the P values ranged from .13 to >.99), so these groups were combined as 1 sample group for the analyses. The median SUS score in the study was 90 out of 100, and the mean SUS score was 87 (SD 12.5). Both values are classified as “excellent” [60]. The range of the 11 individual scores was from 57.5 to 100. The other dimensions are listed in Table 3. The dimension “usability” showed the highest mean value, with 6.47 (SD 1.16) out of 7 points on the Likert scale. The highest per item mean value of 6.75 (SD 0.87) was reached by the item “The HB is simple and easy to understand” (“interface quality”). In the same dimension, the item “The HB can do everything I want it to do” scored the lowest, with a mean of 4.75 (SD 1.76; “interface quality”). All dimensions showed high means, ranging from 6.47 (SD 1.16) for usability to 5.28 (SD 2.02) for usefulness, showing the positive perceptions of the participants. The dimension “risk” was worded negatively, so the score 1 is the highest possible score, and 7 is the lowest possible score; it showed a low risk with a mean of 1.58 (SD 1.56). The items “The HB meets my needs” and “The HB can do everything I would want it to be able to do” scored the lowest, with mean values of 4.75 (SD 2.18 and SD 1.76, respectively). No significant correlations were observed between digital affinity and the measured acceptability dimensions with P values ranging between .41 and .86 (Multimedia Appendix 2).

Patients asked most questions in the liquids category, followed by the dumping syndrome category. The possible ratings for the HB’s answers ranged from 1 (very good) to 5 (very bad). If the topic “other” was excluded, the average score of all ratings was 2.3 (SD 0.4). The “dumping syndrome” category had the most ratings of 1 (“very good”; 13/19, 68%). If the ratings 1 (very good) and 2 (good) are combined, the “protein” category received the best ratings, with 83% (10/12) of the rated answers receiving a 1 or 2. The answers of the HB on questions about “food tolerance” were rated as having the lowest quality, with the most ratings of 5 (3/11, 27%) and the most ratings of 4 (bad) and 5 (very bad) combined (5/11, 45%). In the “others” category, patients asked questions about preoperative nutrition, the allowance of specific food groups and ingredients, and blood glucose and sugar intake. An overview of the ratings is displayed in Table 4.

The second part of the questionnaire, although based on several proven-useful English-language questionnaires, was not tested for the quality criteria with the exact composition that it had in this study. The fact that this was a cross-sectional study is seen as a limitation in the methodology. The author’s presence during the study may have caused some bias owing to participants wanting to portray themselves well, and limited openness or honesty may lead to less critical responses [71]. In addition, the sample of 12 participants is relatively small for the statistical analyses of the questionnaires. However, for a usability test, the sample size is sufficient for the first cycle of the iterative process [72,73].

The use of an HB might be a promising approach to address nutrition-related questions in everyday clinical practice. However, there are also potential risks, which must be considered. When patients use an AI-based digital information tool without surveillance by a health care professional, there is a certain risk of misunderstanding or misinterpretation the provided answer [74]. Furthermore, the HB for patients with obesity only covers bariatric nutrition–related questions. Thus, any other comorbidities that require nutritional adaptations are not considered, and patients need to be made aware of this. Another potential risk is digital exclusion. People with low literacy, cognitive impairment, or no access to digital tools should not be at a disadvantage [20,75]. Therefore, the HB must be easy and intuitive to use, and high-quality traditional health care must remain accessible [20].

The HB has a great potential for further development. The next steps include the improvement of the accuracy of the answers, expansion of the topics, and improvement of the presentation of the answers and the design. Subsequently, a further review with a similarly large sample of potential users is needed. A randomized controlled trial with a larger sample would be needed to analyze potential benefits, such as better patient care or improved nutrition knowledge in patients in practice. Beyond exploring the short-term use of an HB around bariatric surgery, exploring more extensive use after surgery would be needed. Nutrition questions might change over the years, and an HB that supports patients in this trajectory could be a preventive tool for weight gain after surgery [76].

Existing interventions using conversational agents focus more on healthy lifestyle behaviors and less on health care setting with patients [77,78]. A recent review showed that chatbot interventions are supportive for physical activity behavior, fruit and vegetable consumption, sleep duration, and sleep quality [77]. Therefore, chatbots also offer the potential to support health care delivery in an efficient, appealing, and personalized manner. This should be explored in areas where lifestyle or behavioral changes are prescribed as part of the treatment, such as rehabilitation and dietetics, and to promote patient compliance. In the future, capturing health professionals’ perceptions of the HB and their willingness to use it in the medical setting would be important. To implement new technologies in patient care, health professionals’ opinions are just as relevant as patients’ opinions. The last hurdle for the use of HBs in practice would their financing and certification as medical devices.

In this study, the strengths of an HB supporting nutritional care for patients with obesity, such as its satisfactory usability and provision of nutrition information, were determined. Weaknesses were identified in the accuracy of the response of, limited knowledge of, and design of the HB.