Abstract
Background: Antimicrobial resistances pose significant challenges in health care systems. Clinical decision support systems (CDSSs) represent a potential strategy for promoting a more targeted and guideline-based use of antibiotics. The integration of artificial intelligence (AI) into these systems has the potential to support physicians in selecting the most effective drug therapy for a given patient.
Objective: This study aimed to analyze the feasibility of an AI-based CDSS pilot version for antibiotic therapy in sepsis patients and identify facilitating and inhibiting conditions for its implementation in intensive care medicine.
Methods: The evaluation was conducted in 2 steps, using a qualitative methodology. Initially, expert interviews were conducted, in which intensive care physicians were asked to assess the AI-based recommendations for antibiotic therapy in terms of plausibility, layout, and design. Subsequently, focus group interviews were conducted to examine the technology acceptance of the AI-based CDSS. The interviews were anonymized and evaluated using content analysis.
Results: In terms of the feasibility, barriers included variability in previous antibiotic administration practices, which affected the predictive ability of AI recommendations, and the increased effort required to justify deviations from these recommendations. Physicians’ confidence in accepting or rejecting recommendations depended on their level of professional experience. The ability to re-evaluate CDSS recommendations and an intuitive, user-friendly system design were identified as factors that enhanced acceptance and usability. Overall, barriers included low levels of digitization in clinical practice, limited availability of cross-sectoral data, and negative previous experiences with CDSSs. Conversely, facilitators to CDSS implementation were potential time savings, physicians’ openness to adopting new technologies, and positive previous experiences.
Conclusions: Early integration of users is beneficial for both the identification of relevant context factors and the further development of an effective CDSS. Overall, the potential of AI-based CDSSs is offset by inhibiting contextual conditions that impede its acceptance and implementation. The advancement of AI-based CDSSs and the mitigation of these inhibiting conditions are crucial for the realization of its full potential.
doi:10.2196/66699
Keywords
Introduction
Sepsis infections caused by pathogens with antimicrobial resistance (AMR) represent a significant global challenge in health care [,]. In 2017, there were 48.9 million new cases of sepsis and 11 million deaths related to sepsis, accounting for 19.7% of all global deaths []. In Germany, sepsis incidence increased by an average of 5.7% per year, from 280 cases in 2010 to 370 cases in 2015 per 100,000 individuals []. A recent meta-analysis indicated that the 30-day mortality rate for sepsis in Germany was estimated to be 26.5%, which is consistent with the observed rates in North America and Europe []. Furthermore, over 1.27 million deaths per year are attributed to AMR worldwide. In 2019, there were 9650 deaths attributable to AMR (mortality rate of 5 per 100,000) and 45,700 deaths associated with AMR (mortality rate of 22 per 100,000) in Germany []. Recent data from Germany show heterogeneous trends in AMR proportions of infected patients underscoring the urgent need for enhanced infection prevention measures to limit AMR spread []. Inappropriate antibiotic prescribing represents a significant contributing factor []. In particular, the prolonged or improper use of nonspecific, broad-spectrum antibiotics is highly problematic, as these antibiotics provide symptomatic relief but also facilitate the development of resistance in other bacteria [,].
Clinical decision support systems (CDSSs) provide a solution for promoting targeted and guideline-based antibiotic prescribing [-]. CDSSs adopt a variety of forms and are integrated into routine practice. Despite their diverse nature, they share a common objective: to assist medical professionals in identifying the most appropriate form of therapy for each patient, based on existing data and established guidelines, through the use of programmed algorithms or artificial intelligence (AI) [,]. In the context of CDSSs, previous studies have demonstrated a decrease in antibiotic prescription rates, more rapid initiation of appropriate antibiotic therapy for patients, and improved clinical outcomes, such as reduced mortality, increased antibiotic-free days, and fewer medical complications [-]. However, CDSSs do not always improve clinical practice []. For example, there is still insufficient evidence to conclusively show positive effects on therapy duration, dosage, or adherence to clinical guidelines [,,].
Despite the growing body of research on AI-based CDSSs in intensive care units (ICUs) [,], there is a notable gap in the implementation of AI tools in routine care in general [] and the availability of AI-based CDSSs for sepsis in the German health care system. The KINBIOTICS (translated as “AI-based decision support for antibiotic therapy”) project, funded by the German Federal Ministry of Health, aimed to train AI-based algorithms and improve the prediction of suitable antibiotics for sepsis infections on the basis of a comprehensive data set. In addition to the development of an AI-based CDSS, a cross-sectoral resistance observatory and a new rapid test for sequencing the antibiotic genome were developed [].
Engaging clinicians in the design and development of CDSSs is often suggested as a strategy to enhance the alignment between the system and the needs of its users [-]. Interviews and expert groups are approaches used for preimplementation clinician involvement []. Insights into clinicians’ views provide valuable information on the barriers and facilitators that affect their willingness to adopt and use CDSSs in practice []. By examining clinicians’ perceptions, the study seeks to inform targeted strategies to improve the CDSS’s design, usability, and relevance, thereby promoting more effective and widespread adoption in clinical practice.
This study aimed to perform a use case analysis of a pilot version of the AI-based CDSS within an ICU setting. The objective was to analyze the clinical decision-making processes, and adopt a more comprehensive perspective on the factors that facilitate or hinder the future implementation of such a system.
Methods
Reporting is based on the Standards for Reporting Qualitative Research (SRQR) []. The SRQR checklist is provided in .
Study Design and Setting
A 2-stage base model (random forest at both stages) was developed for the initial therapy using patient data, laboratory data, and clinical data. Subsequently, the model was optimized using a variety of parameters, including the number of decision trees and tree depth (the model specifications and results will be published separately once the AI-based CDSS model has been finalized). The developed model was trained and tested on data from 1 of the 3 participating clinics and then evaluated for robustness using data from a second clinic. Although the performance of the underlying AI model was insufficient in terms of accuracy, specificity and sensitivity at the time of the evaluation, it was crucial at this time to gain insights into the decision-making mechanisms of the physicians. This was undertaken in order to scrutinize the variables included in the initial model and, where necessary, to supplement or adjust them. Accordingly, the preliminary prototype was not modified during the interview phase. However, subsequent to the interviews, the model underwent alterations and is still under development.
The pilot version of the AI-based CDSS was evaluated in a 2-stage semistructured qualitative process []. With regard to feasibility, physicians were each shown 5 exemplary sepsis patient cases in a desktop version of the CDSS. Cases were selected randomly from the evaluation dataset and were identical for each interviewee. In addition to basic patient information, this contained relevant vital parameters, and the treatment recommendation determined by the AI-based CDSS (
Figure 1). This antibiotic therapy recommendation was presented to the physicians both for the admission situation to the ICU (initial treatment; the start of antibiotic therapy) and for a possible therapy correction at the time the microbiological findings were available (re-evaluation; usually 48-72 hours after start of antibiotic therapy). The physicians were informed during the individual interviews that the CDSS is still in a pilot status and that the recommendations are likely not yet reliable. The physicians were asked to assess and evaluate the intensive care situation at both time points as well as the AI-based model recommendation for antibiotic therapy. In addition to the plausibility of the content of the therapy recommendations, the design and layout of the pilot version were evaluated.
Following the individual interviews, physicians from each clinic were interviewed again as part of a focus group interview. The theoretical focus here was on technology acceptance of AI-based CDSSs and the identification of challenges and conductive conditions with regard to future implementation.
Expert Selection
Expert interviews were conducted with intensive care physicians from the 3 clinical centers involved in the project at the University Medical Center East Westphalia-Lippe. The interviews were conducted on a voluntary basis and the potential participants were selected by the respective project partners in the 3 clinical centers.
Data Collection
Between October and November 2023, the face-to-face expert interviews and focus groups were conducted. The interview guideline (
Textbox 1) for the focus group interviews was developed on the basis of the extended Unified Theory of Acceptance and Use of Technology (UTAUT 2) [,]. The UTAUT has recently being used to explain technology adoption among health care practitioners’ intention to use AI-based CDSSs [].
- Opening question
- What were the objectives of today’s meeting?
- Questions about the tool
- Please describe your impressions of the KINBIOTICS user interface.
- What information may have been lacking in your medical decision-making process that led to the prescription of antibiotics?
- Please indicate which aspects you consider to be beneficial for a potential application.
- What additional functions would be beneficial to include in the KINBIOTICS user interface to ensure effective utilization in routine clinical practice?
- Questions regarding the use of artificial intelligence (AI) in general
- To what extent do digital tools contribute to the processes undertaken in your day-to-day work?
- Please indicate which digital applications you currently use in your professional activities.
- Please indicate which digital health applications you use in your private life.
- Please indicate your opinion on the utilization of AI, and provide your expectations regarding the implementation of AI in healthcare, both in general and on your ward.
- Please describe your previous experience of contact with AI and clinical decision support systems.
- What is the prevailing attitude in your clinical setting, on the ward and among your professional colleagues with regard to the utilization of artificial intelligence in everyday medical procedures?
- In the hypothetical scenario of utilizing such a tool like KINBIOTICS in tomorrow’s clinical decision-making process, what potential challenges might be identified?
- Questions about future utilization of a clinical decision support system
- Please describe the benefits you perceive in the utilization of AI and clinical decision support systems in your everyday professional practice.
- In order to apply such a system in medical decision-making, what prerequisites must be met?
- Please indicate whether you identify any additional aspects that should be addressed for use in everyday clinical practice.
The interviews were recorded by audio and subsequently transcribed. After complete transcription, the audio recordings were irrevocably deleted in accordance with the data protection policy.
Data Analysis
The data analysis of the transcribed interview material was conducted deductively and inductively based on categories according to Kuckartz (2018) []. The procedure for the individual interviews primarily followed evaluative content analysis, while the focus groups were analyzed in terms of content structure. To gain as much knowledge as possible, the transcripts of the individual interviews were also analyzed with regard to possible content-structuring findings. The transcripts were analyzed anonymously using the qualitative analysis software MAXQDA (version 2022, VERBI) []. The interview analyses and category assignment were carried out independently by 2 researchers (JAD and MK). In the case of discrepancies, a third researcher (DL) was consulted. Categories were adapted or reformulated by the interdisciplinary research team (JAD, MK, DL, SE, and WG).
Ethical Considerations
Informed consent was obtained before the interviews. A positive ethics vote (November 22, 2021) from the ethics committee of the Medical Association of Westphalia-Lippe and the Westphalian Wilhelms University of Münster was received for the entire project (No. 2021-699-f-S). The interview transcripts were anonymized. The interviewees did not receive any compensation; participation was on a voluntary basis.
Results
Sample
A total of 19 individual interviews and 3 focus group interviews were conducted. The distribution of respondents with regard to gender, age, and professional experience was relatively balanced, with 10 (53%) of the respondents identifying as male. In each case, 7 (37%) respondents were aged 30-40 years, and 40-50 years. In total, 5 (26%) respondents had less than 5 years or 11‐20 years of professional experience, 4 (21%) had between 5 and 10 years, and 3 (16%) respondents indicated that they had more than 20 years of professional medical experience. The duration of the expert interviews ranged from 17 to 43 minutes, while the average duration of the focus group interviews was 1 hour.
Feasibility of the AI-Based Clinical Decision Support System
The approval or rejection of the AI-based antibiotic recommendations was heterogeneous and primarily dependent on 2 factors: first, the plausibility of the AI-based individual therapy (change) recommendations (refer to quotes 1 and 2). It should be noted that not every detailed recommendation was already plausible or medically advisable due to the pilot status of the CDSS respective example cases.
Quote 1:The system now suggests piperacillin/tazobactam. Of course, this is a very far-fetched suggestion for primary antibiotics when it comes to designing a calculated antibiotic therapy. For me, this would not be the first drug of choice for patients who arrive at the hospital as primary patients, who have been treatment-naïve so far and have not yet received antibiotic therapy [...].
[Expert 5, Clinical Center 3]
Quote 2:So, [piperacillin/tazobactam]. Well, difficult [...] with a completely normal PCT [Procalcitonin], I would find it difficult [...] to start directly with such a broad antibiotic.
[Expert 8, Clinical Center 2]
Quote 3:[...] So, [...] such a recommendation, I’m very sure, will be malpractice in the next three to five years [...]. This reflects the reality, but this is the worst thing that can happen, to treat a Staphylococcus aureus bacteremia with [piperacillin/tazobactam], that is an absolute no-go.
[Expert 4, Clinical Center 1]
Second, the evaluation of the recommendations is dependent upon the level of professional experience (quote 3). Accordingly, the more advanced the professional experience, the more confident a recommendation was to be approved or rejected. However, the data indicate that the majority of respondents expressed consensus regarding the CDSS recommendation in each case study (
Table 1).
| CDSS recommendation for initial treatment | CDSS recommendation for re-evaluation | |||
| Agreeing with the recommendation, n (%) | Rejecting the recommendation, n (%) | Agreeing with the recommendation, n (%) | Rejecting the recommendation, n (%) | |
| Use case 1 | 14 (78) | 4 (22) | 12 (67) | 6 (33) |
| Use case 2 | 11 (61) | 7 (39) | 1 (5) | 18 (95) |
| Use case 3 | 3 (16) | 16 (84) | 2 (11) | 16 (89) |
| Use case 4 | 15 (83) | 3 (17) | 14 (78) | 4 (22) |
| Use case 5 | 13 (76) | 4 (24) | 12 (71) | 5 (29) |
A suggestion was made that the pilot CDSS should be expanded to include the integration of combination therapies. The vital parameters and patient information should be included as variables in the statistical model for predicting therapy to include the suspected focus of infection. In addition, it was noted that the data provided for the development of the model and the case studies in the pilot version were incomplete in many instances (Table 1). The physicians encountered significant challenges in evaluating the CDSS recommendation when the fields for data that could be viewed in the application were left empty (quote 4).
Quote 4:Here again the PCT [Procalcitonin] is missing, that would make it easier for us. We only have [...] indirect information that we have an infection here [...] measured by the thrombocytopenia, and then [...] I can get along with actually using piperacillin and tazobactam primarily here when it comes to the calculated antibiotic therapy [...].
[Expert 5, Clinical Center 3]
Quote 5:The [application] is more explanatory than my phone. So, everything is fine in that respect.
[Expert 3, Clinical Center 3]
From the physicians’ perspective, the primary factor influencing the acceptance of the AI-based CDSS was the time required for completion of the process. If the system is able to produce an evident medication recommendation in a shorter time than the physicians can achieve, a high willingness to use it was reported. There was unanimous feedback that the design and layout of the CDSS are pleasant and easy to understand (quote 5). Furthermore, the majority of respondents indicated a preference for a mobile version of the system for use in everyday clinical practice. This preference was again justified by the availability of digital resources.
Facilitating and Inhibiting Factors for the Implementation of an AI-Based Clinical Decision Support Systems
The focus group interviews showed that while there is already strong foundational support for digital tools and applications in diagnostics and therapy as part of routine clinical practice, the use of more advanced applications, especially those based on AI, was limited to isolated cases and primarily on a project basis. Previous experiences with CDSSs had a significant impact on expectations regarding the quality of the CDSS, the effort required for integration into everyday medical practice, and the intention to potentially use such a system (quotes 6 and 7).
Quote 6:I’m thinking about the echocardiography, [...] where the AI has become so good that we let it take over. We do the examination ourselves. And yet the module helps us with that. That’s how I imagine it [here] too.
[Expert 2, focus group 2]
Quote 7:Yes, for the colonoscopy [...] we once had [a CDSS]. But the human was faster than the AI. So in that respect [...] rather not.
[Expert 3, focus group 3]
Nevertheless, the majority of respondents indicated a high level of willingness to adopt AI-based CDSSs (quotes 8 and 9). The option of re-evaluation included in the application was considered to be particularly beneficial.
Quote 8:I actually found the idea compelling. Because in the early days of my clinical work, even more so than now, the question always arises: am I doing it right now?
[Expert 1, focus group 2]
Quote 9:[T]he field is so complex [...] that’s why I think a support tool that works well is very helpful.
[Expert 3, focus group 1]
As previously noted, the acceptance and implementation of digital solutions is largely dependent on the amount of time available. In addition, the majority of respondents indicated that they are consistently seeking tools that facilitate efficient support in routine clinical practice. Consequently, the respondents exhibited an intrinsic motivation to use CDSSs, which was observed across all age groups and genders, but was particularly pronounced in the “30‐40 years” and “40‐50 years” age groups.
In contrast with the physician’s technology openness as a beneficial factor, the level of digitization in the clinics was found to be heterogeneous. Furthermore, the physicians’ assessment indicated that there was significant potential for improvement in all 3 hospitals (quote 10). The statements indicated that the level of digital infrastructure, including stable internet coverage, exhibited considerable variation not only between the clinics but also between individual wards within the clinics. Overall, there was a consensus that better equipment in terms of digital devices and applications was needed (quote 11). The lack of comprehensive digital and cross-sectoral availability of relevant patient health information was also identified as a barrier to the implementation of AI-based CDSSs and efficient medical treatment in general. This demonstrated that the degree of digitization in everyday clinical practice affects the perceived feasibility of implementing the CDSS and is the most significant barrier to the beneficial factors.
Quote 10:So, it’s really funny that you can still secure good pens. Because we do so much paper based.
[Expert 2, focus group 2]
Quote 11:The problem is that there are far too many individual components that all [...] communicate with each other via interfaces. And in the process, values are lost or the data transfer doesn’t work. And then you end up standing there and yes, the computer hangs.
[Expert 4, focus group 2]
Furthermore, the existing data basis for the AI-based CDSS model was identified as a significant challenge. The participants were aware that, in addition to the aforementioned lack of digitized patient information, the heterogeneous quality of previous antibiotic administration practice also had an influence on the predictive ability of the recommendation from the CDSS. Furthermore, an increased documentation and justification effort in the event of potential deviation from the CDSS recommendation was also seen as a barrier. However, all participants were aware that such a system would be implemented as a supportive and not a replacement measure, and this was not criticized.
Two relevant recommendations for improvement were derived from the interviews. First, the majority of respondents indicated that applicable guidelines on antibiotic therapy should be incorporated into the specification of the underlying statistical models. Second, there was a recurring suggestion that, alongside the further development of the presented system, the capability to diagnose sepsis infections should also be integrated into a CDSS. Once again, the time to correct diagnosis and subsequent appropriate treatment was identified as the main driver. Suggestions for adapting the tested CDSS comprised the inclusion of quantities and suggestions for the duration of antibiotic administration, in addition to the therapeutic agents themselves (quote 12).
Quote 12:So, in an ideal world, [...] [AI] can be a huge step forward as a decision-making tool. But [A] the right information has to flow in. And [B] the decisive recommendation [...] is already anticipated here [...]. So, the first step: can this be sepsis? That is an important step. [...] In the second step, a recommendation for initial therapy is helpful [...], yes.
[Expert 4, focus group 1]
Discussion
Principal Findings
Overall, this use case analysis shows various barriers and facilitators for the implementation of AI-based CDSSs for antibiotic therapy in sepsis (Textbox 2). The findings of the expert and focus group interviews indicate that the potential of digitization and, in particular, AI-based tools is expected and viewed in a favorable light by the majority of the physicians. This finding is in line with other research which indicates that clinicians have a predominantly positive perception of AI systems [-]. Similarly, the willingness to use these tools is high []. In this context, the plausibility of the prediction and the potential time savings are of particular importance. It is evident that the existing contextual conditions represent the most significant obstacle to future implementation. In addition to the basic technical equipment in the clinics, other factors, such as a stable, comprehensive internet supply and the necessary digital availability of data, were also identified as potential barriers to future implementation. Accordingly, the question arises as to what extent clinics are equipped with the digital infrastructure to fully leverage the benefits of AI-based CDSSs. At this point it has to be noted that the digital maturity of German hospitals appears to be comparatively limited []. This can be attributed, at least in part, to a lack of financial resources [,]. It remains to be seen to what extent hospitals use the transformation fund provided by the German Hospital Future Act and how this can enhance the conditions for the implementation of AI systems.
Barriers
- Overall
- Low degree of digitization in everyday clinical practice.
- Lack of comprehensive digital and cross-sectoral data availability.
- Negative previous experiences with clinical decision support systems.
- Use Cases
- Heterogeneous quality of previous antibiotic administration practice limits predictive ability of the artificial intelligence–based recommendations.
- Increased effort to document and justify potential deviations from the clinical decision support system’s recommendation.
- The confidence to approve or reject artificial intelligence–based antibiotic recommendations depends on the level of professional experience.
Facilitators
- Overall
- Potential time savings.
- Physician’s technology openness or intrinsic motivation.
- Positive previous experiences with clinical decision support systems.
- Use Cases
- Re-evaluation of clinical decision support system recommendations is considered to be particularly beneficial.
- Pleasant design and layout of the clinical decision support system (easy to understand).
Recommendations
- Incorporation of guidelines on antibiotic therapy into the specification of the statistical models.
- Clinical decision support systems should include the integration of combination therapies.
- Statistical model for predicting antibiotic recommendations should include vital parameters and further patient information to identify the suspected focus of infection.
- Besides sepsis treatment, the capability to diagnose sepsis infections should also be integrated into a clinical decision support system.
- Preference for a mobile version of the clinical decision support system.
The medication recommendations by the CDSS were not yet perceived as reliable due to the limited and biased data available for training the AI system. Often, a very broad standard therapy (piperacillin/tazobactam) was recommended. This was mainly due to the fact that this medication occurred considerably more frequently within the learning data set than all other antibiotics. According to the experts, this approach is not absolutely mandatory and sometimes maybe counterproductive in terms of calculated antibiotic therapy. Younger physicians in particular tended to agree with the recommendations. This is a sign that a CDSS for the most common “standard medication” might not be beneficial if not detrimental. This can be confirmed by the physicians’ statements that they would like decision support primarily for medication combinations and for rare antibiotic agents or rare sources of infection. In view of the goal of avoiding the development of resistance by reducing the use of broad-spectrum antibiotics, the current pilot version of the AI system would possibly even promote this risk of further resistance. To avoid this, current guidelines should be incorporated into the model design in addition to retrospective primary data.
The respondents’ call for the upstream integration of sepsis diagnostics in addition to the further development of the existing CDSS appears to be purposeful. A recent study has confirmed initial successes in this regard []. Although no clear correlations between sociodemographic characteristics and the willingness to use CDSSs were found in the present evaluation, differences in the approval or rejection of the AI-based CDSS recommendations depending on professional experience were identified. This finding is supported by Lambert et al []. It is recommended that future studies investigate the degree to which professional experience and other sociodemographic variables exert an influence on technology acceptance and the willingness to use CDSSs. In this regard, using the UTAUT would be appreciated [].
Generally, the following challenges associated with the use of AI in health care should not be overlooked. For example, automation bias, in which health care providers may develop an overreliance on AI recommendations, potentially reducing their clinical vigilance and leading to oversights in patient care, needs to be considered []. Furthermore, AI-driven CDSSs may unintentionally propagate biases present in training datasets, resulting in discriminatory recommendations that may disadvantage certain demographic groups []. Another key concern is the ambiguity and complexity of liability; it remains unclear who is legally responsible if an AI-influenced decision contributes to a negative patient outcome []. Furthermore, previous research has highlighted the potential negative impact of a lack of user involvement in the development of CDSSs on the willingness to use them, even when the system quality is optimal []. One of the key strengths of the study is the early involvement of future users. The participatory approach highlights the need for involvement. Consequently, given the significant lack of engagement with CDSS interventions reported within the literature [], future studies should consider the involvement of users from the earliest stages of development [,,]. The findings of the present study confirm that in addition to personal factors, the plausibility of the recommendations is the most decisive factor determining the willingness to use the tool []. Further development of the pilot version or similar systems is therefore crucial for implementing and exploiting the potential benefits of AI. Furthermore, the feedback from the physicians participating in this study provided valuable insights into additional variables that were subsequently integrated into the further development of the CDSS, following the completion of the interviews.
Limitations
Several limitations must be acknowledged in the evaluation of the use case presented in this study. First, the efficacy of the CDSS was clearly limited due to the rather small amount of data used for training. The findings suggest that the CDSS may offer less benefit for physicians during the initial therapy phase compared with its use in the context of therapy re-evaluation. At this stage, it was not feasible to accurately predict the most appropriate antibiotic, one that would minimize treatment changes and side effects, based on the currently available data. The acceptance of AI-based interventions heavily depends on the plausibility and relevance of therapy recommendations, which ultimately impacts their efficacy. Therefore, further development and refinement of AI-based CDSSs are essential, along with an expansion of the digital data infrastructure.
In addition, it is important to consider that the evaluation was based on a qualitative interview study, which comes with inherent limitations. The voluntary nature of participation may introduce self-selection bias, potentially leading to an overestimation of the participants’ openness to technology and their acceptance of AI-based CDSSs. This limitation suggests that the findings may not be fully generalizable to the broader population of health care professionals.
Conclusion
This pilot study of an AI-based CDSS indicates that further development is needed to achieve the original goals of minimizing switches in antibiotic prescriptions and reducing the reliance on broad-spectrum antibiotics. Relying solely on retrospective data from past care practices does not seem to be an effective strategy for meeting these objectives. In addition, it is essential to address the key challenges identified in this study to enable the successful integration of AI into clinical settings. While the respondents generally showed a positive attitude toward the use of AI-based CDSSs, a more comprehensive evaluation of technology acceptance among health care professionals can only be conducted when a version with realistic, guideline-compliant recommendations is available. Early engagement of users in the development process proves beneficial, not only for refining the CDSS but also for fostering acceptance among medical practitioners.
Acknowledgments
The authors would like to thank all members of the KINBIOTICS project and in particular the contact persons at the 3 hospitals (Evangelisches Klinikum Bethel, Klinikum Bielefeld Mitte, and Klinikum Lippe) for their support in conducting the interviews. Special thanks go to all the physicians who took part in the interviews and to Inga Heißenberg, who helped to prepare and conduct the interviews. Furthermore, we would like to thank Dr Timothy Mc Call for his support in translating the original quotations. The KINBIOTICS project was funded by the German Federal Ministry of Health (grant ZMVI1-2520DAT930). The funding organization was not involved in the data collection, evaluation, analysis, or writing of the manuscript.
Conflicts of Interest
None declared.
References
- Kumar NR, Balraj TA, Kempegowda SN, Prashant A. Multidrug-resistant sepsis: a critical healthcare challenge. Antibiotics (Basel). Jan 4, 2024;13(1):46. [CrossRef] [Medline]
- GBD 2021 Antimicrobial Resistance Collaborators. Global burden of bacterial antimicrobial resistance 1990-2021: a systematic analysis with forecasts to 2050. Lancet. Sep 28, 2024;404(10459):1199-1226. [CrossRef] [Medline]
- Rudd KE, Johnson SC, Agesa KM, et al. Global, regional, and national sepsis incidence and mortality, 1990-2017: analysis for the Global Burden of Disease Study. Lancet. Jan 18, 2020;395(10219):200-211. [CrossRef] [Medline]
- Fleischmann-Struzek C, Mikolajetz A, Schwarzkopf D, et al. Challenges in assessing the burden of sepsis and understanding the inequalities of sepsis outcomes between National Health Systems: secular trends in sepsis and infection incidence and mortality in Germany. Intensive Care Med. Nov 2018;44(11):1826-1835. [CrossRef] [Medline]
- Bauer M, Groesdonk HV, Preissing F, Dickmann P, Vogelmann T, Gerlach H. Mortality in sepsis and septic shock in Germany. Results of a systematic review and meta-analysis. Anaesthesist. Aug 2021;70(8):673-680. [CrossRef] [Medline]
- The burden of antimicrobial resistance in G7 countries and globally: an urgent call for action. Robert Koch-Institut. 2022. URL: https://edoc.rki.de/handle/176904/9864;jsessionid=BB6B616866626A2E3B56758E5A1C5C8B?locale-attribute=en [Accessed 2025-02-15]
- Rödenbeck M, Ayobami O, Eckmanns T, Pletz MW, Bleidorn J, Markwart R. Clinical epidemiology and case fatality due to antimicrobial resistance in Germany: a systematic review and meta-analysis, 1 January 2010 to 31 December 2021. Euro Surveill. May 2023;28(20):2200672. [CrossRef] [Medline]
- Zilberberg MD, Shorr AF, Micek ST, Vazquez-Guillamet C, Kollef MH. Multi-drug resistance, inappropriate initial antibiotic therapy and mortality in Gram-negative severe sepsis and septic shock: a retrospective cohort study. Crit Care. Nov 21, 2014;18(6):596. [CrossRef] [Medline]
- Pradipta IS, Sodik DC, Lestari K, et al. Antibiotic resistance in sepsis patients: evaluation and recommendation of antibiotic use. N Am J Med Sci. Jun 2013;5(6):344-352. [CrossRef] [Medline]
- Kalın G, Alp E, Chouaikhi A, Roger C. Antimicrobial multidrug resistance: clinical implications for infection management in critically ill patients. Microorganisms. Oct 16, 2023;11(10):2575. [CrossRef] [Medline]
- Neugebauer M, Ebert M, Vogelmann R. A clinical decision support system improves antibiotic therapy for upper urinary tract infection in a randomized single-blinded study. BMC Health Serv Res. Mar 6, 2020;20(1):185. [CrossRef] [Medline]
- Van Dort BA, Penm J, Ritchie A, Baysari MT. The impact of digital interventions on antimicrobial stewardship in hospitals: a qualitative synthesis of systematic reviews. J Antimicrob Chemother. Jun 29, 2022;77(7):1828-1837. [CrossRef] [Medline]
- Xu R, Wu L, Wu L, Xu C, Mu T. Effectiveness of decision support tools on reducing antibiotic use for respiratory tract infections: a systematic review and meta-analysis. Front Pharmacol. 2023;14(1253520):1253520. [CrossRef] [Medline]
- Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3(17):17. [CrossRef] [Medline]
- Graafsma J, Murphy RM, van de Garde EMW, et al. The use of artificial intelligence to optimize medication alerts generated by clinical decision support systems: a scoping review. J Am Med Inform Assoc. May 20, 2024;31(6):1411-1422. [CrossRef] [Medline]
- Rittmann B, Stevens MP. Clinical decision support systems and their role in antibiotic stewardship: a systematic review. Curr Infect Dis Rep. Jul 24, 2019;21(8):29. [CrossRef] [Medline]
- Rutten JJS, van Buul LW, Smalbrugge M, et al. An electronic health record integrated decision tool and supportive interventions to improve antibiotic prescribing for urinary tract infections in nursing homes: a cluster randomized controlled trial. J Am Med Dir Assoc. Mar 2022;23(3):387-393. [CrossRef] [Medline]
- Hartman EAR, Groen WG, Heltveit-Olsen SR, et al. Implementation of a tailored multifaceted antibiotic stewardship intervention to improve antibiotic prescribing for urinary tract infections in frail older adults (ImpresU) in four European countries: a process evaluation alongside a pragmatic cluster randomized controlled trial. Trials. Oct 18, 2024;25(1):691. [CrossRef] [Medline]
- Laka M, Milazzo A, Merlin T. Can evidence-based decision support tools transform antibiotic management? A systematic review and meta-analyses. J Antimicrob Chemother. May 1, 2020;75(5):1099-1111. [CrossRef] [Medline]
- Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. Apr 2, 2005;330(7494):765. [CrossRef] [Medline]
- Catho G, Sauser J, Coray V, et al. Impact of interactive computerised decision support for hospital antibiotic use (COMPASS): an open-label, cluster-randomised trial in three Swiss hospitals. Lancet Infect Dis. Oct 2022;22(10):1493-1502. [CrossRef] [Medline]
- Hong N, Liu C, Gao J, et al. State of the art of machine learning-enabled clinical decision support in intensive care units: literature review. JMIR Med Inform. Mar 3, 2022;10(3):e28781. [CrossRef] [Medline]
- Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med. Nov 2018;24(11):1716-1720. [CrossRef] [Medline]
- Weinert L, Müller J, Svensson L, Heinze O. Perspective of information technology decision makers on factors influencing adoption and implementation of artificial intelligence technologies in 40 German hospitals: descriptive analysis. JMIR Med Inform. Jun 15, 2022;10(6):e34678. [CrossRef] [Medline]
- KINBIOTICS [Website in German]. URL: https://kinbiotics.de [Accessed 2024-11-07]
- Khairat S, Marc D, Crosby W, Al Sanousi A. Reasons for physicians not adopting clinical decision support systems: critical analysis. JMIR Med Inform. Apr 18, 2018;6(2):e24. [CrossRef] [Medline]
- Ji M, Yu G, Xi H, Xu T, Qin Y. Measures of success of computerized clinical decision support systems: an overview of systematic reviews. Health Policy Technol. Mar 2021;10(1):196-208. [CrossRef]
- Carroll C, Marsden P, Soden P, Naylor E, New J, Dornan T. Involving users in the design and usability evaluation of a clinical decision support system. Comput Methods Programs Biomed. Aug 2002;69(2):123-135. [CrossRef] [Medline]
- Newton N, Bamgboje-Ayodele A, Forsyth R, Tariq A, Baysari MT. Does involving clinicians in decision support development facilitate system use over time? a systematic review. Stud Health Technol Inform. Jun 22, 2023;304:11-15. [CrossRef] [Medline]
- Westerbeek L, Ploegmakers KJ, de Bruijn GJ, et al. Barriers and facilitators influencing medication-related CDSS acceptance according to clinicians: a systematic review. Int J Med Inform. Aug 2021;152:104506. [CrossRef] [Medline]
- O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. Sep 2014;89(9):1245-1251. [CrossRef] [Medline]
- Döring N. Datenanalyse. In: Döring N, editor. Forschungsmethoden Und Evaluation in Den Sozial- Und Humanwissenschaften [Book in German]. Berlin, Heidelberg: Springer Berlin Heidelberg; 2023:587-766. [CrossRef] ISBN: 978-3-662-64761-5
- Venkatesh V, Thong JYL, Xu X. Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Q. 2012;36(1):157. [CrossRef]
- Venkatesh V, Thong J, Xu X, University of Arkansas, Hong Kong University of Science and Technology, The Hong Kong Polytechnic University. Unified theory of acceptance and use of technology: a synthesis and the road ahead. JAIS. 2016;17(5):328-376. [CrossRef]
- Dingel J, Kleine AK, Cecil J, Sigl AL, Lermer E, Gaube S. Predictors of health care practitioners’ intention to use AI-enabled clinical decision support systems: meta-analysis based on the unified theory of acceptance and use of technology. J Med Internet Res. Aug 5, 2024;26:e57224. [CrossRef] [Medline]
- Kuckartz U. Qualitative Inhaltsanalyse: Methoden, Praxis, Computerunterstützung 4. Auflage. Weinheim [Book in German]. Basel: Beltz Juventa; 2018. ISBN: 9783779936824
- MAXQDA. URL: https://www.maxqda.com/ [Accessed 2025-02-20]
- van der Meijden SL, de Hond AAH, Thoral PJ, et al. Intensive care unit physicians’ perspectives on artificial intelligence-based clinical decision support tools: preimplementation survey study. JMIR Hum Factors. Jan 5, 2023;10:e39114. [CrossRef] [Medline]
- Maassen O, Fritsch S, Palm J, et al. Future medical artificial intelligence application requirements and expectations of physicians in German university hospitals: web-based survey. J Med Internet Res. Mar 5, 2021;23(3):e26646. [CrossRef] [Medline]
- Pecqueux M, Riediger C, Distler M, et al. The use and future perspective of artificial Intelligence-a survey among German surgeons. Front Public Health. 2022;10:982335. [CrossRef] [Medline]
- Hennrich J, Kauffmann AL, Buck C, Eymann T. Künstliche intelligenz in der hausärztlichen versorgung. In: Pfannstiel MA, editor. Künstliche Intelligenz Im Gesundheitswesen: Entwicklungen, Beispiele Und Perspektiven [Book in German]. 2022:413-430. [CrossRef] ISBN: 978-3-658-33596-0
- Geissler A, Hollenbach J, Haring M, Amelung VE, Thun S, Haering A. A nationwide digital maturity assessment of hospitals – results from the German DigitalRadar. Health Policy Technol. Sep 2024;13(4):100904. [CrossRef]
- Burmann A, Fischer B, Brinkkötter N, Meister S. Managing directors’ perspectives on digital maturity in German hospitals-a multi-point online-based survey study. Int J Environ Res Public Health. Aug 6, 2022;19(15):9709. [CrossRef] [Medline]
- Steinbach D, Ahrens PC, Schmidt M, et al. Applying machine learning to blood count data predicts sepsis with ICU admission. Clin Chem. Mar 2, 2024;70(3):506-515. [CrossRef] [Medline]
- Lambert SI, Madi M, Sopka S, et al. An integrative review on the acceptance of artificial intelligence among healthcare professionals in hospitals. NPJ Digit Med. Jun 10, 2023;6(1):111. [CrossRef] [Medline]
- Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. J Am Med Inform Assoc. 2012;19(1):121-127. [CrossRef] [Medline]
- Gianfrancesco MA, Tamang S, Yazdany J, Schmajuk G. Potential biases in machine learning algorithms using electronic health record data. JAMA Intern Med. Nov 1, 2018;178(11):1544-1547. [CrossRef] [Medline]
- Price WN 2nd, Gerke S, Cohen IG. Potential liability for physicians using artificial intelligence. JAMA. Nov 12, 2019;322(18):1765-1766. [CrossRef] [Medline]
- Rawson TM, Moore LSP, Hernandez B, et al. A systematic review of clinical decision support systems for antimicrobial management: are we failing to investigate these interventions appropriately? Clin Microbiol Infect. Aug 2017;23(8):524-532. [CrossRef] [Medline]
- Neff MC, Schaaf J, Noll R, et al. Initial user-centred design of an AI-based clinical decision support system for primary care. Stud Health Technol Inform. Jan 25, 2024;310:1051-1055. [CrossRef] [Medline]
- Panicker RO, George AE. Adoption of automated clinical decision support system: a recent literature review and a case study. Archives of Medicine and Health Sciences. 2023;11(1):86-95. [CrossRef]
Abbreviations
| AI: artificial intelligence |
| AMR: antimicrobial resistance |
| CDSS: clinical decision support system |
| ICU: intensive care unit |
| SRQR: Standards for Reporting Qualitative Research |
| UTAUT: Unified Theory of Acceptance and Use of Technology |
Edited by Andre Kushniruk; submitted 02.10.24; peer-reviewed by Deepika Sharma, Godwin Denk Giebel, Trutz Bommhardt; final revised version received 22.11.24; accepted 31.12.24; published 04.03.25.
Copyright© Juliane Andrea Düvel, David Lampe, Maren Kirchner, Svenja Elkenkamp, Philipp Cimiano, Christoph Düsing, Hannah Marchi, Sophie Schmiegel, Christiane Fuchs, Simon Claßen, Kirsten-Laura Meier, Rainer Borgstedt, Sebastian Rehberg, Wolfgang Greiner. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 4.3.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.