Our intervention was designed as an SSI [24,25]. SSIs are structured, stand-alone activities intended to produce immediate effects on targeted psychological outcomes. Typically lasting 10‐20 minutes, they offer a time-efficient way to promote reflection, reappraisal, or symptom relief, making them suitable for a wide range of populations. In addition to offering rapid support, SSIs can serve as a gateway to more comprehensive mental health care by helping individuals build insight or skills that increase their willingness to seek further support [8]. Prior research has demonstrated their effectiveness in addressing negative thought patterns, reducing stress, and alleviating symptoms of anxiety and depression in both clinical and nonclinical settings [8,22,24,25].

Unlike multisession interventions, which typically unfold over multiple weeks through repeated check-ins, skill-building exercises, and progressive assignments [12,29], SSIs are completed in a single sitting without requiring prior sessions or follow-up engagement. While multisession programs rely on cumulative exposure and gradual skill development over time, SSIs are designed to deliver focused support within a bounded interaction. Prior work illustrates how SSIs can compress core therapeutic elements into a single experience. For example, Schleider and Weisz [30] tested a brief computer-based SSI in which participants engaged with psychoeducational content and a short reflective writing task within one sitting, and observed reductions in depression and anxiety symptoms at follow-up. More recently, Sharma et al [20] demonstrated that cognitive reframing can be supported through structured, single-session interactions with language models. In contrast to multisession formats that distribute these elements across time, SSIs integrate learning, reflection, and application into a single, self-contained interaction.

We drew particular inspiration from a prior SSI reported by Bhattacharjee et al [8]. The activity was designed to be easily referenced or practiced by individuals when encountering negative thoughts and emotions in their everyday lives. Their work adapted core concepts from CBT, specifically thought records and behavioral chaining [31,32], into a guided digital reflection activity. In CBT, thought records involve documenting a situation; the thoughts, emotions, and behaviors it evokes; and then generating alternative thoughts [31]—behavioral chaining focuses on tracing the sequence from trigger to reaction to identify patterns [32]. Both techniques help individuals become more aware of their internal responses and prepare for similar situations in the future by promoting insight and reflection. Given that cognitive reappraisal is not inherently context-specific, our activity was similarly designed to be broadly applicable across a range of stressful situations, with the workplace serving as the deployment context for this study.

Our activity follows a similar structure, consisting of 11 questions. The activity was developed through an iterative process involving researchers with expertise in human-computer interaction (HCI), psychology, cognitive science, generative AI, and DMH. Members of the research team had prior experience designing DMH interventions both with and without the use of AI and have published extensively in leading HCI, psychology, and AI venues. The first set of prompts (Questions 1‐7) operationalizes the cognitive model underlying CBT thought records, in which a situation gives rise to automatic thoughts that in turn drive emotional and behavioral responses [31,32]. Moving from the situation (Q1) to its most troubling element (Q2), to automatic thoughts (Q3-Q4), to feelings (Q5), to behaviors (Q6), and finally to an integrated trigger → thought → feeling → behavior summary (Q7) mirrors the standard structure used to build awareness of the links among cognition, affect, and action before any restructuring is attempted [15]. Questions 8‐11 then enact cognitive reappraisal as defined in the stress-appraisal and emotion-regulation literatures [3,5]. Q8 corresponds to examining the evidence for and proportionality of an appraisal, a core component of cognitive restructuring. Q9 draws directly on the challenge-versus-threat distinction in transactional models of stress appraisal [4], inviting users to reframe a stressor as a challenge to be met rather than a threat to be avoided. Q10 reflects reappraisal in Gross's process model of emotion regulation, in which the meaning of a situation is reinterpreted to alter its emotional impact [5]. Q11 supports generalization and consolidation, encouraging users to carry the reappraisal into future situations [3]. Reappraisal comes last in the sequence by design: examining and reframing an appraisal is difficult until the cognition, emotion, and behavior tied to the trigger have been laid out, which is the work of Questions 1-Q7.

Table 1 shows the full sequence of prompts and their intended purpose. After completing all steps, the chatbot was instructed to provide a brief summary of the conversation and clearly state that the structured part has concluded.

Motivated by recent advances in LLM-based interventions that support context-aware support [17,22,33], we adapted the sequence in Table 1 to allow content to align more closely with users’ specific circumstances. To implement this, we used a prompt engineering approach. The prompt was developed through an iterative process by the same research team. The complete system prompt is provided in Section A in Multimedia Appendix 1. We structured the prompt using GPT-4o with the following components:

The chatbot was implemented as a text-based conversational agent and was embedded directly within the Qualtrics survey platform. It did not have a named persona or profile picture. The chatbot opened each session with a fixed introductory message welcoming the participant and describing the nature of the interaction (Section A in Multimedia Appendix 1). The language style was conversational and reflective, with a temperature setting of 0.7 to allow for naturalistic variation in responses while maintaining coherence. A screenshot of the interface is provided in Section C in Multimedia Appendix 1.

This study was approved by the Microsoft Research Institutional Review Board (IRB) through the Microsoft Research Ethics Review Program under ERP ID 10893. Participants were invited to engage in an SSI in which they reflected on a workplace stress situation through a conversation with a chatbot.

The study was disseminated through a one-time email invitation, and participants could complete the activity at any time of their choosing after receiving it. The intervention was designed as a single, one-time activity to be completed within a continuous sitting; participants were not able to return to complete additional sessions. The chatbot guided them through a series of questions designed to support reflection on their experience. Participants were encouraged to respond genuinely to each prompt. At the end of the interaction, the chatbot indicated that the conversation was complete, after which participants proceeded to the next page. They were informed in advance that the conversation would likely involve around 15 conversational turns, based on the 11 core questions programmed into the chatbot, along with potential follow-up and clarification prompts. Section C in Multimedia Appendix 1 demonstrates an example conversation between the chatbot and a user. All components of the study, including the chatbot interaction, were completed within the Qualtrics platform.

To assess the impact of the SSI, participants completed a set of self-report questions both before and after the session. To evaluate short-term changes following the intervention, we selected measures that capture both immediate subjective stress responses and theoretically relevant appraisal-related constructs. Specifically, we assessed situational stress intensity, stress mindset, perceived demand, and perceived resources, as these reflect how individuals interpret and respond to stressful situations. All responses were recorded on a 5-point Likert scale, where 1 indicated low agreement or intensity (eg, “strongly disagree” and “very low”) and 5 indicated high agreement or intensity (eg, “strongly agree” and “very high”). The measures included:

A single question assessing participants’ stress level in relation to a workplace situation [8]. We selected this brief measure to capture immediate subjective stress relevant to the focal scenario and to minimize burden within a single-session study.

Eight statements capturing beliefs about the potential enhancing or debilitating effects of stress (eg, “Experiencing this stress facilitates my learning and growth;” “The effects of this stress are negative and should be avoided”) [38]. These included both positively and negatively framed statements. We included this measure because cognitive reappraisal may shift how individuals interpret the meaning and consequences of stress.

A rating of how demanding the stressor felt [36,39]. This measure was included as an exploratory outcome grounded in the job demands-resources model [39].

A rating of the participant’s perceived ability to cope with the stressor [36,39]. Alongside perceived demand, this measure captured whether the intervention shifted participants’ sense of their own coping capacity [39].

Situational stress intensity served as the primary outcome measure, as it most directly captures the intended effect of the reappraisal activity. The remaining measures—stress mindset, perceived demand, and perceived resources—were exploratory and included to provide a broader characterization of participants’ stress-related experiences before and after the intervention.

To gather user feedback about the intervention, participants also responded to open-ended questions about how the activity affected their stress levels and their view of the situation, as well as what aspects they liked or disliked. All open-ended questions were provided immediately at the end of the session, within the same Qualtrics survey. These questions included the following:

All open-ended questions were mandatory except for the improvement suggestion question, which was optional. All 100 participants provided responses to the 4 mandatory questions, and 66 provided responses to the optional improvement suggestion question.

Because the study used a one-time, survey-based format, participants who did not complete the study did not submit responses and therefore were not included in the final sample. We were unable to assess dropout rates among participants who may have started but not completed the study.

As a feasibility study, our objective was to examine whether engagement with an LLM-based chatbot for cognitive reappraisal could lead to observable improvements in stress-related outcomes, such as situational stress intensity and stress mindset. Early-stage feasibility studies serve a distinct role in intervention research [27]; they help determine whether the conceptual approach is worth pursuing, whether users engage meaningfully with the system, and whether measurable changes occur in theoretically relevant constructs. Demonstrating even modest improvements under these conditions can support the potential viability of LLM-based reappraisal as a foundation for more controlled investigations. To contextualize these quantitative findings, we complement them with qualitative analyses that explore how users experienced and interpreted the chatbot’s support, providing insight into the underlying mechanisms and perceived value of the interaction.

We conducted paired Wilcoxon signed-rank tests to assess pre-post differences across the 4 measures described in the “Procedure” section. We applied the Benjamini-Hochberg (BH) procedure to account for multiple comparisons.

To complement user-reported scores, we analyzed linguistic features of user-chatbot conversations. Each conversation was divided into 3 segments (T1=beginning, T2=middle, and T3=end), and we examined how predicted stress and sentiment varied across them using 3 approaches.

A pretrained Robustly Optimized BERT Pretraining Approach (RoBERTa)–based model [40] (BERT: Bidirectional Encoder Representations from Transformers) that outputs a probability distribution over positive and negative sentiment for each conversation segment.

A pretrained RoBERTa-based model [41] estimates stress probability ranging from 0 to 1 for each conversation segment.

A prompting-based approach that assigns a stress score to each user message using a structured prompt informed by prior work on LLMs as evaluators [42-46]. The rater outputs a score from 1 to 5, where 1=no observable stress and 5=high stress (characterized by pronounced emotional strain or difficulty coping). The rating criteria were developed through a review of prior literature and iterative design, enabling finer distinctions in stress intensity. Further details about the prompt design are provided in Section D in Multimedia Appendix 1.

While these methods provide useful approximations of users’ affective and psychological states, we recognize that their outputs can be shaped by modeling assumptions and may miss contextual nuance. We therefore interpret them as supportive evidence for our quantitative analyses rather than as definitive indicators of users’ internal experiences.

For the qualitative analysis, we first anonymized and cleaned participants’ responses, then conducted a thematic analysis [47]. The first author independently reviewed all participant responses and developed an initial codebook using an open-coding method. This process reflected an iterative approach that integrated both deductive and inductive techniques: the analysis of RQ1 was guided by prior work on stress reappraisal interventions, while RQ2 was approached inductively, allowing themes to emerge directly from the data. The first author then used axial coding to refine categories and develop higher-order themes. Throughout the analysis, the codes and themes were regularly discussed with other members of the research team to ensure multiple perspectives were considered. The development of themes was reviewed through detailed collaborative discussions. This approach is consistent with qualitative coding practices used in prior studies [48,49].

Participants reported significant reductions in situational stress intensity following the intervention (mean 0.29, SD 0.83, r_rb=0.54; P=.002) and significant improvements in stress mindset (mean 1.70, SD 4.37, r_rb=0.44; P=.002). The latter reflects a shift toward viewing stress as more enhancing than debilitating. Perceived resources showed improvement (mean 0.17, SD 0.83, r_rb=0.32; P=.07), and perceived demand showed a reduction (mean 0.12, SD 0.83, r_rb=0.22; P=.17), although these changes were not statistically significant. Table 2 shows these results.

We examined the progression of average negative sentiment and stress scores across conversation tertiles (T1-T3) for all 3 classifiers, followed by nonparametric statistical analyses. Results are presented in Figure 1 and Table 3. Across all 3 classifiers, users’ messages exhibited a consistent pattern: sentiment became more positive, and indicators of stress decreased from the beginning to the end of the interaction. Friedman tests revealed statistically significant differences across tertiles for each classifier. Pairwise Wilcoxon tests showed significant changes between T1 and T3, as well as between T2 and T3, in all 3 cases. These findings suggest that as users engaged with the chatbot, their expressions of negative sentiment and stress declined over time, aligning with the observed reduction in self-reported stress scores.

Participants in general highlighted the value of the chatbot’s step-by-step approach, noting that the ordered prompts helped them organize thoughts that might otherwise remain scattered. P55 described the session as “a good exercise in laying out the issue, my responses, and getting to why it’s that way,” and P42 emphasized how the sequence of questions encouraged structured analysis rather than a quick vent. Several also echoed that the structure offered a clear path through the conversation, making it easier to track what had been covered and where to focus next.

Participants often remarked that the value of the interaction lay in how it helped them arrive at their own insights through deliberate unpacking of the stressor. They acknowledged that the structured opportunity to express themselves brought a sense of relief. The act of “thinking aloud” through writing or guided dialogue was also perceived as familiar and useful. Drawing a comparison to journaling, P15 reflected:

Some participants also appreciated that the structured format helped shift their focus from problems to potential solutions. They noted that the prompts encouraged them to slow down and articulate their thoughts more deliberately. As P89 explained:

These comments suggest that participants valued not just the content of the activity, but the structured way in which it guided their thinking.

The shift observed in stress mindset scores, toward viewing stress as more enhancing than debilitating, was echoed in participants’ qualitative comments. Several described how the conversation helped them reassess their situation from a new angle. P79 shared that it “shifted my perspective just enough to see it more as a challenge than a frustration,” while P21 noted that the chatbot offered an alternative way of thinking when they felt mentally stuck, helping them move beyond unproductive thought patterns.

The conversation logs further reflect this reappraisal process. P4, who began the session feeling overwhelmed and unsupported by colleagues, wrote at Q9 that “perhaps the anger and anxiety are the result of repeated cycles and maybe I need to consider I am burnt out on the role”—a reinterpretation that emerged through their own responses to the structured questions. P65, working through anxiety about executing quickly on a difficult technical project, arrived by Q10 at a reframe of the work as “an exploration, an open mindset that allows me to explore boundaries,” shifting from perceiving the situation as a threat to viewing it as an opportunity for learning.

However, not every participant experienced dramatic shifts in perspective. P30, for example, remarked that “while it did not change my viewpoint 100%, it was nice to write it out,” suggesting that even partial reframing can be helpful. Others, including P15 and P48, described subtle changes in how they interpreted or emotionally responded to the stressor. These comments highlight the potential of the interaction to support gradual cognitive shifts, helping participants reinterpret their stressors and relate to stress in a more constructive way. It is worth noting that these reframings were generated by participants themselves through the structured reappraisal questions; the chatbot’s role was to guide and facilitate this process rather than to supply alternative interpretations directly.

Beyond perspective shifts, some participants described how the dialogue helped translate reflection into concrete action. P66 appreciated “the structured approach to break down the root of my stress and ideate on solutions,” while P83 noted that the activity helped them “channel stress into action.” P70 shared that although their stress level did not change, the activity “helped to structure my approach to the situation,” and P76 commented that the conversation “helped lead me to some useful options to help me fix my problem.” These comments suggest that, for some participants, the value of the activity lay not only in processing emotions and perspectives but in arriving at a clearer sense of what to do next.

The conversation logs further illustrate how this process unfolded during the later stages of the activity. Participants who initially described interpersonal tension or coordination difficulties sometimes progressed toward specific action plans by the final reappraisal prompts. For example, P73, who began the session feeling overwhelmed by disorganized planning and strained colleague relationships, later described intentions to foster better communication and establish more regular check-ins with their team before the next planning cycle. Similarly, P61, who entered the conversation frustrated by a difficult team dynamic, moved toward articulating concrete steps such as involving a new team member in daily discussions and investing in a stronger working relationship. P59, who entered the session anxious about presenting to a large group of external customers in a nonnative language and worried about losing the audience’s attention, arrived by the final prompts at a concrete plan to use auto-captioning technology in future presentations and to build in pauses for audience questions. These examples suggest that the structured questioning sequence could help some users move from diffuse stress toward more actionable coping plans.

Some participants described the chatbot as helpful in recognizing or affirming their existing efforts and emotional responses. Even when the chatbot did not introduce new ideas, it was still seen as useful for reinforcing what participants were already feeling or doing. As P16 explained:

Others similarly appreciated moments where the chatbot reflected back their current state of mind. P74 noted:

P82 shared that “the chatbot acknowledged my efforts as productive” in dealing with a difficult situation, and P92 mentioned it “affirmed steps planned.”

For a few, this sense of reinforcement was seen as limited. As P46 commented, “Lots of validation. It might have been better if it had challenged me.” Still, for many participants, the interaction was valued for offering a sense of confirmation rather than redirection.

Several participants appreciated the chatbot’s nonjudgmental tone, describing it as a space where they could express themselves freely. For some, this quality made the interaction feel less pressuring than conversations with people. P42 remarked that it “allowed me to vent, without fear of judgment,” while P65 shared, “I liked that it was objective.”

Some others described the experience as supportive precisely because it did not involve being evaluated. P66 emphasized the benefit of having “an unbiased conversation” that helped them “focus on a task at hand.” This perception of the chatbot as nonjudgmental contributed to some participants’ willingness to open up or think through the situation more deliberately.

Some participants described instances when the chatbot’s responses felt disconnected from the realities of their situation. In particular, a few noted that the interaction seemed to assume ideal conditions, such as collaboration or personal control, that did not reflect their experiences. As P36 explained:

P94 similarly expressed that the prompts did not adequately reflect the complexity of their stressor, stating, “I felt that I needed to double down and explain more about the stressful situation, to say it’s not that easy.”

Some participants also described discomfort with having to revisit a stressful situation in detail, particularly when they had already discussed it elsewhere. P59 remarked, “I already had to explain the situation to myself, then my team, and now this bot wants to know about it too, causing me to relive the situation in my mind.” Others felt that, despite going through the interaction, there was no tangible outcome. As P24 noted, “I dislike the fact that it goes nowhere and there is no action to be done.” These reflections suggest that for some participants, retelling the situation without any follow-up or resolution made the experience less satisfying.

As discussed in the previous sections, participants noted that the structured nature of the chatbot’s prompts helped them consider alternative perspectives and identify potential solutions. However, several participants also reflected on how this structure may have come at the expense of a more natural, conversational flow. P6 remarked that “it wanted to ask certain questions as opposed to engage in conversation,” highlighting a perception that the interaction was being guided more by a fixed sequence than by mutual exchange.

Some participants further sensed that the chatbot was following a script, which contributed to a feeling of artificiality. As P75 put it, the interaction felt “not human,” due to what appeared to be a scripted, predetermined flow. While certain elements of the structure, such as follow-up questions or requests for clarification, were appreciated by some, they were not universally welcomed. For example, P94 noted that these follow-ups “helped pull out some important information,” but others, like P96, found them repetitive and frustrating:

Together, these responses underscore a tension between the benefits of a structured, goal-directed dialogue and the desire for a more natural and responsive conversational experience.

Some participants expressed a desire for more questions to allow them to elaborate on their situation—particularly when their stressors were complex or nuanced. P7 noted the difficulty of condensing their experience, remarking, “It’s the writing it down, trying to cover the important topics with brevity. Tough nut to crack.” Similarly, P57 wished for a longer “getting to know each other” period to help the chatbot better personalize its responses.

Conversely, some participants felt that even the current level of interaction was somewhat extended. P12 commented that the exchange lasted “much longer than I cared for.” They expressed that the number of questions, often accompanied by clarifications and follow-ups, made the experience feel more effortful than necessary. This concern was particularly salient among participants dealing with less intense stressors, where a longer interaction felt disproportionate. As P23 explained, “My stress trigger was basic and didn’t need to have that many turns.” These perspectives reveal a core design tradeoff: while deeper context can enable more tailored and relevant responses, excessive questioning may lead to user fatigue or disengagement in some cases.

Although the study did not explicitly ask about participants’ attitudes toward AI, some participants shared comments that reflected preexisting skepticism about the use of AI and technology for well-being support. Several participants indicated that their discomfort stemmed not from the content of the conversation, but from their underlying beliefs about what makes an interaction feel helpful. P29 shared that “the fact that it was a chatbot made it annoying,” and P37 expressed discomfort with “the fact that it was not a real person.” These comments suggest a reluctance to view nonhuman agents as appropriate partners for discussing personal or emotional matters. P51 further expressed mistrust toward the system, stating that they do not trust a software entity that cannot feel emotions.

These views extended to how participants interpreted the chatbot’s attempts at empathy. Some reacted negatively to what they perceived as artificial emotional cues. P14 stated, “The whole mimicking empathy is annoying,” and P41 described the interaction as “trying too hard, like mimicking empathy but not quite getting there.” In contrast, others responded more positively to the chatbot’s empathic tone. P44 noted that the message felt “kind,” even though they knew it was automated. P58 similarly shared, “I could tell it was a bot, but it still felt like it was trying to be understanding.” These contrasting reactions highlight how users’ preconceptions about AI—and particularly their expectations around empathy from AI—impacted their perceptions of AI support.