Workplace emotional well-being is a pressing concern, as increasing numbers of employees suffer from burnout and chronic stress [1,2]. These issues lead to additional workplace problems, including absenteeism, reduced productivity, and mental health concerns [3]. Current mental health tools and employee assistance programs (EAPs) have expanded significantly in scope, evolving from a focus on mental health and substance use to include broader aspects of emotional well-being, such as family health and workplace stress [4]. However, the effectiveness of EAPs has been mixed. A 2017 systematic review concluded that EAPs benefit functioning and presenteeism but have mixed effects on absenteeism and engagement [5]. Due to the continued shortage of mental health workers, novel strategies are necessary to meet the growing demand for employee emotional support [6].

The integration of digital technology has enabled more immediate, cost-effective, and accessible forms of employee support, such as digital peer support (DPS), to be incorporated into traditional counseling-focused EAP infrastructures [7]. DPS offers an innovative approach to complement traditional EAPs, providing support on demand, in real time, by peers with shared experiences rather than clinicians, thereby promoting social connectedness, significant sentiment improvements, and emotional regulation [7,8]. However, how employees engage with DPS has yet to be studied.

Demographic variables such as age, gender, culture, technology use, and internet access significantly affect the engagement and effectiveness of digital interventions [9-12]. Barriers such as limited technological proficiency, especially among older adults or individuals with low digital abilities, may reduce user engagement and limit program success [13]. Prior research indicates that women are more likely than men to seek mental health support and use digital mental health tools [14-17]. These differences highlight the importance of tailoring support strategies to the needs of different groups. While mobile apps and chatbots using cognitive behavioral techniques are effective in reducing symptoms of depression and anxiety, evidence shows that digital tools are more effective when supplemented with human interaction, whether through peer support or moderated participation [9].

Additionally, promoting awareness of and trust in EAP resources through culturally sensitive education has been linked to enhanced utilization among diverse employee populations [13,18]. Addressing these sociobehavioral challenges through user-centered design, sensitive outreach, and anonymous technology can improve accessibility and inclusivity across demographic groups [13,19].

Research shows that DPS improves emotional well-being, self-efficacy, and social connection and empathy, all of which contribute to positive organizational outcomes [14]. DPS has emerged as a critical modality for delivering emotional health and wellness services through technological platforms [7,20]. It encompasses both synchronous and asynchronous support delivered via chat, forums, video, mobile apps, and other digital communication tools, allowing users to flexibly seek mental health support [20,21]. It offers higher accessibility than more traditional services, as most individuals own a smartphone, including those from lower socioeconomic backgrounds or with serious mental illness [22-24]. Additionally, DPS has fewer barriers than in-person services, which may involve scheduling, transportation, mobility, or discomfort with face-to-face interactions [19,25], as well as tele- or mobile clinical mental health services that may require insurance, appointments, and delays due to formal clinical assessments before care can begin. As a result, DPS facilitates broader participation, particularly among populations that are underserved or less likely to engage in typical care [19,25].

In addition to accessibility, DPS services can be available 24/7, allowing continuity of care and reinforcing self-efficacy and emotional regulation, thereby filling temporal gaps left by typical mental health services [7]. Peer-delivered digital interventions have been shown to improve both psychiatric and medical self-management by empowering individuals to actively participate in their care within a supportive, nonclinical environment [20,21,25]. Reviews of DPS programs, including those for public safety workers, highlight reductions in stress and trauma [26-28], while workplace-based engagement is associated with increased belonging, accountability, and a sense of community [20,21,25]. DPS is also unique in its ability to integrate lived experience into its design, development, and delivery [21,29,30]. The involvement of peers and their lived experiences increases the relevance and acceptability of interventions [31].

The timing of user engagement is affected by a variety of sociobehavioral factors. Users who seek DPS after work hours may experience heightened levels of emotional distress, loneliness, insomnia, or anxiety, and may engage in more spontaneous or emotionally charged exchanges, whereas interactions during business hours tend to be more structured [7,20,32]. Additional barriers such as privacy concerns, stigma, and work-hour constraints can also reduce engagement during regular business hours, highlighting the need for after-hours availability of support [18,33]. Engagement patterns are also affected by demographic variables such as age, gender, employment status, education level, and housing conditions [34,35]. Preferences for specific technologies, such as text-based versus visual platforms, may vary depending on users’ mental health conditions and comfort levels [36]. For instance, some individuals may avoid video-based support and prefer text-only interactions, seeking privacy, cultural alignment, or emotional safety [37].

DPS has also demonstrated measurable improvements in emotional well-being. Services report rapid reductions in sadness (57.5%) and loneliness (55.0%), with sentiment changes emerging within minutes of engagement [8]. These benefits may be understood through relational frameworks such as social penetration theory, which suggests that trust and intimacy in relationships deepen over time, especially when individuals feel secure in their anonymity and in control over self-presentation [38]. These dynamics are often heightened during after-hours use, when users may feel more emotionally vulnerable and open, forming meaningful peer connections [32,38]. Taken together, these findings point to the critical role of timing, sociobehavioral context, and technological preferences in shaping DPS engagement.

A central mechanism of DPS effectiveness is resource sharing, which promotes mental health literacy, reduces stigma, and encourages help-seeking behaviors [27,28]. Sharing success stories and testimonials has been shown to increase employee engagement within EAPs, supporting a workplace culture that values mental health and well-being [27]. Workplace-related resource sharing also enhances coping, particularly in managing remote work, and reduces isolation [26-28]. DPS also offers timely, subclinical support during a shortage of behavioral health clinicians, helping users access resources when needed without waitlists [6,27]. Access to tailored resources may be especially valuable for underserved or marginalized populations, helping to reduce equity gaps in mental health care [13].

This study examines how sociobehavioral factors, such as the timing of support access, levels of perceived emotional distress, and in-session resource sharing, shape employee engagement and sentiment outcomes within a DPS environment. By analyzing user data across 5 large employers, we aim to identify key trends that can inform the design and implementation of digital emotional support services within workplace well-being programs.

This study examines how timing and in-session support influence sentiment outcomes within an anonymous, synchronous DPS service, Supportiv. Specifically, we investigate (1) overall sentiment changes across users of DPS; (2) sociobehavioral differences between users who accessed DPS during versus outside business hours; (3) differences in sentiment outcomes based on the timing of service use (during vs outside business hours); and (4) the impact of targeted in-session resources on users’ sentiment changes.

Supportiv is an anonymous, synchronous, small-group, live-chat DPS service that is available on demand, 24/7. Employees can access the service through links from their employer portals, health plans, EAPs, or related channels. This study analyzes data from employees whose organizations offered Supportiv in addition to their existing EAP programs.

Users voluntarily access Supportiv’s anonymous peer-to-peer support service through an informed self-selection process. Engagement begins with a user-initiated response to the free-text prompt, “What’s your struggle?” Before accessing services, individuals are required to review and actively consent to the Terms of Service and Privacy Policy, which include consent for the use of their inherently anonymous data—anonymous from the point of collection—for research and service improvement purposes. No personally identifiable information, personal health information, or Health Insurance Portability and Accountability Act (HIPAA)–protected health information is collected.

To reduce the risk of indirect user identification through free-text content, all transcripts were audited using automated methods followed by manual processing before analysis. Potential personal identifiers, including names, locations, or contact information, as well as contextual details that could allow for identification, were omitted from the text. Data were stored and analyzed in deidentified form, and access was limited to authorized personnel. All data handling complies with General Data Protection Regulation (GDPR) and California Consumer Privacy Act (CCPA) requirements, including lawfulness, fairness, transparency, data minimization, purpose limitation, and integrity and confidentiality.

The platform is designed according to data protection by design and by default principles. To mitigate the risk of indirect identification in free-text content, all chats are continuously processed through automated filtering systems that remove explicit identifiers such as names, contact information, social media handles, websites, and physical addresses. In addition, trained human moderators review chats in real time and remove any remaining potentially identifying or narrowing contextual information as needed. These safeguards are applied universally across all chats as part of routine platform operations, not solely for research purposes. Access to research data is restricted to authorized personnel. Together, these safeguards reduce the risk of direct or indirect identification and ensure compliance with applicable privacy regulations.

Supportiv uses artificial intelligence (AI)–driven natural language processing (NLP) to match users with other concurrent users experiencing similar life challenges, forming small, synchronous groups of no more than 5 peers. Each group is facilitated in real time by a trained subclinical human moderator. Users are automatically placed into a chat session that may include other participants and the moderator. Moderators facilitate peer-to-peer discussion, maintain psychological safety by enforcing community guidelines, and share contextually relevant, useful resources identified through AI to match users’ expressed needs. Moderators also guide collaborative problem-solving and, in cases of crisis, promptly refer users to professional crisis services, ensuring comprehensive support.

During the study period, 40,431 total chat sessions were recorded between January 2023 and December 2024. This retrospective observational study analyzed 24,818 anonymous, live user chat sessions from 13,879 employees after excluding sessions with fewer than 3 messages. Data were securely stored in the Supportiv database. Users consisted of employees from 5 large companies that implemented Supportiv’s digital peer-to-peer support service to complement their existing EAPs. The main objective was to evaluate the service’s impact on employee well-being through sentiment outcomes derived from chat session transcript analysis. The primary outcomes were within-session sentiment changes across 8 key dimensions: loneliness, sadness, stress, anxiety, depression, despair, helplessness, and optimism. Secondary descriptive outcomes included engagement metrics, use patterns, and resource-sharing outcomes. In addition to anonymous chat content, limited metadata, such as user location at the city level, derived from nonidentifying technical data, were available to support contextual analysis and geographic segmentation. No personally identifiable information or personal health information was collected.

As this study used a large language model (LLM) to identify sentiment scores associated with user language in DPS live chats, several ethical considerations must be addressed [39]. Concerns related to privacy are mitigated by the use of anonymous text data and the absence of user data collection. Additionally, sentiment scores were used to assess within-session emotional benefits rather than to make clinical diagnoses. All chats were facilitated by trained subclinical human moderators to support accurate sentiment interpretation and constructive DPS sessions. This retrospective study utilized operational data and did not involve direct human participant research. It was determined to be exempt from human participant research review by the Alpha Institutional Review Board (protocol IRBEMP042025).

Each participant was matched to a chat within approximately 30 seconds of responding in free text to a single prompt: “What’s your struggle?” This response, referred to as the “user-chat-session struggle,” determined whether the user was placed into a live, ongoing small-group chat discussion on the same topic or into a 1-on-1 session with a moderator, to which others with similar struggles could later be added. Each chat could include up to 5 users and a trained subclinical professional moderator.

Moderators facilitated discussions using motivational interviewing–based OARS (open-ended questions, affirmations, reflective listening, and summaries) techniques, including open-ended questions, affirmations, reflective statements, summarizations, and structured information sharing [40]. These strategies supported conversation flow, encouraged engagement, and promoted collaborative problem-solving. To maintain continuity, moderators provided a summary for new participants joining mid-session. Moderators played a key role not only in guiding discussions and fostering a psychologically safe environment but also in reviewing AI-recommended resources, such as practical articles and informational materials tailored to users’ expressed needs, to ensure contextual sensitivity before sharing them with users.

Each user chat was analyzed independently, even if the same user participated in multiple sessions. These individual sessions were referred to as “user-chat sessions.” Although some users returned for multiple sessions, the service’s anonymous nature required that each chat experience be analyzed independently, without linking sessions to the same individual. This approach ensured that each interaction was assessed in isolation, providing a clearer and more user-focused analysis of sentiment changes per intervention.

To analyze the target sentiments (loneliness, sadness, stress, anxiety, depression, despair, helplessness, and optimism), OpenAI’s LLM model, GPT-4o-mini, was leveraged using a few-shot learning technique. This method involves presenting the model with a limited number of labeled examples within the prompt to guide its outputs [41]. Previous research has demonstrated that this approach matches or exceeds the performance of state-of-the-art models [8]. Despite agreement with expert human ratings established in previous work, the LLM-derived sentiment scores should not be regarded as psychometric instruments for clinical diagnoses. These scores are indicators of expressed emotional content rather than measures of clinical symptom severity.

A user session was defined by the initial “user-chat-session struggle,” consisting of a brief free-text response of up to 300 characters, along with all subsequent messages exchanged during the chat. When a user joined a new chat session at a later time, they submitted a new struggle prompt, initiating a separate and independent session that was analyzed separately. Sentiment changes were tracked for each user within a session. Every message was assessed for sentiment intensity on a 1-10 scale, where a score of “1” indicated low intensity, “5” indicated moderate intensity, and “10” indicated high intensity for the relevant sentiment. This scale provided sufficient granularity to detect subtle variations while maintaining interpretability. The use of sentiment scoring with this model has been previously validated and reported in a prior study [8] (Multimedia Appendix 1), demonstrating strong agreement with expert human assessments. The model’s performance is continuously monitored for false positives and false negatives, and flagged misclassifications are reviewed to iteratively fine-tune the model. Importance sampling is employed to prioritize messages with high uncertainty and those near the model’s decision boundary. This strategy allows human experts to review and further refine the model’s detection of language that may involve linguistic nuances, such as indirect, metaphorical, or sarcastic expressions.

Examples illustrating scores of 1, 5, and 10 are provided in Multimedia Appendix 1. Messages not suitable for sentiment analysis, such as clarifying or irrelevant statements, were excluded (eg, “Or a skill,” “What do you mean,” “At what time?”). To reduce misclassification, the model was instructed to refrain from assigning scores when insufficient information was available to infer sentiment content.

Due to the DPS service’s voluntary live chat model, validated multiitem self-report instruments such as the 9-item Patient Health Questionnaire (PHQ-9), 7-item Generalized Anxiety Disorder (GAD-7), or 10-item Perceived Stress Scale (PSS-10) were not administered to users. Although ecological momentary assessments and visual analog scales can support in-the-moment measurement, they are typically used in structured research protocols and rely on repeated prompts and user incentives. By contrast, peer support environments prioritize user autonomy and natural conversational flow; therefore, repeated prompts may be disruptive. Additionally, many symptom instruments rely on self-report, which is prone to bias, social pressure, reduced introspection, distortion, and response fatigue. In this context, LLM-assigned sentiment scores were used as conversationally embedded measures rather than validated instruments. These scores were based on the language used by participants. Our previous study demonstrated that this approach is comparable to state-of-the-art models [8]. Additionally, rather than indicating diagnostic severity, sentiment scores were used to assess within-session changes. This is consistent with prior research using NLP to evaluate digital interventions at scale [42-44]. These scores were not intended to represent diagnostic severity; instead, they provide scalable indicators of expressed emotional content to assess within-session sentiment changes. Future validation work may assess how NLP-derived indicators align with standardized measures (such as PHQ-9, GAD-7, and PSS-10).

To ensure a meaningful assessment of sentiment trends, specific criteria were applied when selecting chat sessions for analysis. Only conversations in which users engaged beyond a single message were included, as single-message interactions did not provide sufficient data to track sentiment changes over time.

For each user chat session, 8 emotions were screened to determine whether the session contained an interpretable sentiment change. To ensure the analysis focused on sessions with meaningful sentiment expression, a threshold score of 5 was established. For negative emotions (loneliness, sadness, stress, anxiety, depression, helplessness, or despair), sessions were included if the initial score was greater than 5, whereas for the positive emotion (optimism), a score below 5 was required. The initial emotional state was defined using the highest sentiment score from the user’s first 3 messages, recognizing that individuals often begin conversations with brief or casual exchanges before revealing more meaningful emotional content and underlying struggles. This approach captured users who eventually expressed their emotional concerns while filtering out sessions lacking sufficient sentiment depth for analysis.

To evaluate sentiment states over time across user chat sessions, the sentiment data were standardized to allow comparison despite variation in conversation duration. Each conversation was normalized to a relative scale (from 0 to 1) using bins of size 0.01. If a message with a sentiment score was present within a bin, the score was assigned to that bin. This normalization process allowed sentiment trends to be compared across conversations.

After aggregating the binned data for each sentiment, a linear trend line was fitted. The slope of this line indicated the direction and magnitude of sentiment change, reflecting the service’s effectiveness in supporting well-being. The goodness of fit was evaluated using the coefficient of determination (R²). Finally, CIs were calculated to represent the standard error of the data, indicating variability around the mean for each conversation duration bin.

Propensity matching was used to create comparable cohorts for each research question. Users within the cohorts were matched by training a logistic regression classifier using the following features: user-chat-session topic, device type, user country, number of users in the chat, conversation duration, engagement rate (number of characters typed by the user per unit of conversation time), and initial sentiment score. This matching ensured a high level of alignment between users in the 2 cohorts at the start of the chat by incorporating both demographic and behavioral metrics.

To ensure meaningful matching, conversation duration and engagement rate were transformed into categorical bins using predefined ranges. For conversation duration, conversations in each cohort were divided into the following bins based on length (in minutes): “0-5,” “5-10,” “10-20,” “20-45,” “45-60,” and “>60.” Similarly, for engagement rate (measured as the number of characters typed by the primary user per unit of time), bins were created to categorize conversations based on characters typed per unit time: “0-10,” “10-20,” “20-30,” “30-40,” “40-50,” and “>50.”

The logistic regression model produced a propensity score for each user chat session, representing the likelihood of assignment to one cohort. We used 25% of the SD of all propensity scores as the caliper (radius of search) for matching. For each user chat in 1 cohort, the closest match in the other cohort was identified using a k-nearest neighbor classifier within the caliper. Propensity matching was conducted separately for each trend analyzed.

The topics for user chat struggles were categorized into the following themes (alphabetically): anxiety, caregiving/parenting, depression, financial/social determinants of health, identity, loneliness, mental health conditions, physical health/medical, relationship issues, social connection and belonging, stress, suicide/self-harm, trauma, and work/productivity/burnout. These categories were identified through manual inspection of chats to determine common themes across the population.

To gain a comprehensive understanding of employee users and characterize their engagement with the service, several key components were analyzed. First, although the service is anonymous, users may voluntarily provide demographic information and optional feedback at the end of a chat session. Analysis of these data offers valuable insights into the study population.

The primary target group consisted of employees from 5 large employers across varied industry sectors, including retail, media, and health care. As these organizations differed in job roles and workforce structure, engagement patterns were analyzed across heterogeneous contexts, enhancing ecological validity and reducing bias. User engagement patterns were examined throughout the day to determine whether usage varied between business and nonbusiness hours. For this analysis, business hours were defined as Monday through Friday from 8:00 AM to 5:00 PM local time, while all other periods, including evenings, nights, and weekends, were categorized as nonbusiness hours.

A core advantage of the Supportiv service is its ability to deliver highly personalized resources tailored to users’ stated struggles and needs. Supportiv enables moderators to share coping tools, self-help materials, and other resources, as well as links to relevant employee benefits or community resources, specifically tailored to a user’s needs based on context. These resources include various formats such as articles, videos, audio files, tools, worksheets, exercises, community resources, and client-specific materials. Resource sharing is driven by a 2-stage AI recommendation system. The AI model retrieves relevant items that match users’ expressed needs from a resource library and then ranks the retrieved items to ensure that the most appropriate and valuable options are prioritized. Moderators review and select from this AI-curated list before sharing resources with users. The analysis examined the types of resources recommended by AI and shared by moderators, as well as how users interacted with them.

Lastly, although all target users were employees utilizing Supportiv through sponsored links (via their EAPs, employee benefits portals, or other employer-specific URLs), moderators may refer employees to additional relevant EAP services based on their expressed needs. These EAP-related metrics were analyzed to better understand how employees used integrated support services and how DPS functioned within the organizational benefits system.

Statistical analyses were conducted to (1) compare differences between groups for specific features, (2) evaluate the significance of predicted sentiment trends, and (3) compare sentiment improvement metrics (slopes) between target cohorts. Statistical significance was defined as P<.05, and all tests were 2-tailed. Independent contractors were engaged to develop the analysis plan and conduct statistical analyses to minimize commercial bias. Effect sizes for pairwise comparisons were computed using Cohen d.

All analyses were performed using Python 3.9.6 (Python Foundation) with the following libraries: pandas 2.3.3, NumPy 2.0.2, SciPy 1.13.1, scikit-learn 1.6.1, statsmodels 0.14.6, matplotlib 3.9.4, and seaborn 0.13.2.

We utilized a topic-modeling framework based on LLM-driven categorization to assign topics and subtopics to text data. After reviewing the conversations, a list of topics was manually defined. All texts were then processed using GPT-4o-mini with a structured prompt to categorize them into the predefined categories. Classification accuracy was manually verified, and, when necessary, categories were corrected manually.

A total of 40,431 chat sessions were recorded during the study period. Sessions containing only 1 or 2 messages were excluded, as they lacked sufficient context to evaluate sentiment trends or changes over time. After applying these criteria, 24,818 chat sessions from 13,879 employees with at least three messages were included in the analysis.

Users often reported more than 1 concern during their chat sessions. The most common struggles were relationship issues (7531/24,818, 30.34%), trauma (2663/24,818, 10.73%), workplace concerns (2650/24,818, 10.68%), stress (2074/24,818, 8.36%), loneliness (1757/24,818, 7.08%), depression (1570/24,818, 6.33%), and financial/social determinants of health (1565/24,818, 6.31%). Additional struggles included support system challenges (1160/24,818, 4.67%), anxiety (1098/24,818, 4.42%), mental health conditions (723/24,818, 2.91%), identity concerns (638/24,818, 2.57%), caregiving/parenting (617/24,818, 2.49%), physical health/medical concerns (399/24,818, 1.61%), and suicide/self-harm (373/24,818, 1.50%).

On average, users spent 37.97 (SD 35.51) minutes in peer group chats. The average number of messages sent by the primary user was 24.42 (SD 29.45). Variation in chat duration, as reflected by time spent and number of messages, indicates that the analysis includes both users who engaged in extended conversations and those who participated for shorter periods but met the minimum inclusion criteria. To better capture user participation intensity, a metric termed “characters per chat duration” was calculated, with an average of 70.65 (SD 1394.24) characters per minute. The high SD in this metric may be attributed to differences in users’ communication styles. While some users type long, paragraph-style messages, others send multiple short messages consisting of only a few words at a time.

Figure 1 illustrates the number of users throughout the day. User activity decreases during late-night hours, increases steadily as business hours begin, and peaks toward the end of the day before declining at night. Notably, user engagement was higher outside business hours, suggesting that employees may be more likely to use DPS during their personal time, when they are away from job responsibilities and have more space to share their thoughts and concerns without distraction.

This study offers novel insights into the role of timing, in-chat behaviors, and structural barriers in shaping user engagement and sentiment outcomes in a DPS service offered as a stand-alone service or integrated into EAPs. The findings advance understanding of when and how employees use DPS services, how these interactions guide sentiment trajectories, and which factors affect access to mental health support.

Despite the potential of DPS to fill critical gaps in care, broader EAP engagement remains limited due to persistent systemic barriers. The most pressing among these is a widespread lack of awareness about the scope and availability of EAP services, leading to underutilization even in organizations with robust offerings [49]. Research shows that, without sustained promotional efforts, EAP usage rates remain below 10%, while programs with active education and engagement campaigns have reported utilization rates closer to 30% [50,51]. Additionally, internalized stigma may deter individuals from seeking help [52]. Employees may interpret help-seeking as a signal of personal failure or professional inadequacy, especially in high-performance cultures. National surveys indicate declining employee comfort with employer-sponsored mental health services, with only 52% of employees reporting comfort accessing EAP resources in 2022, down from 64% in 2021 and 67% in 2020 [53]. Addressing these gaps requires shifts at the organizational level, including visible leadership endorsement of mental health resources, normalization of mental health care, and integration of DPS into wellness initiatives. These strategies can help develop EAPs from crisis tools into preventive support systems and increase their perceived legitimacy [54].

Beyond timing, additional sociobehavioral factors also influenced engagement with DPS. Although many users identified as female and reported White ethnicity, as expected for mental health services [55], DPS attracted a larger proportion of male and Black, Indigenous, and People of Color users than is typically observed in such services. The age distribution also reflected a broad user base, with a median age of 36 years and a mean of approximately 45 years. The accessibility and anonymity of DPS may offer a solution for adults who are underserved by traditional mental health care. These patterns highlight the broader demographic reach of DPS compared with typical services. The flexible, anonymous nature of DPS mitigates certain barriers associated with traditional mental health care, such as time constraints, privacy concerns, and stigma. Users were more likely to engage outside work hours, likely due to an increased sense of psychological safety for vulnerable self-disclosure when outside of work environments. This pattern supports theories of readiness to disclose, which suggest that individuals are more likely to seek help in environments they perceive as emotionally and socially safe [56,57]. Therefore, timing is not only an operational variable but also a sociobehavioral signal of trust, readiness, and psychological safety. Designing digital services that accommodate different levels of readiness would better enable users to engage based on their comfort and needs.

The most commonly discussed topic, by a clear margin, was relationship issues, which appeared in nearly one-third of all live chat sessions. However, differences in topic prevalence emerged when inflows from and outflows to EAP sources were examined. Most frequently, users accessed DPS services through their EAP (inflows) to discuss anxiety, loneliness, physical or mental health, and suicide or self-harm (Table 2). By contrast, users who sought DPS services and were subsequently referred to their EAP programs (outflows) most frequently discussed anxiety, depression, and caregiving or parenting (Table 3). Resource distribution across discussion topics showed that sessions centered on relationship issues received the greatest number of resources, reflecting the high volume of conversations on this topic (Table 4). Trauma, workplace concerns, and stress followed, aligning with their discussion frequency.

Significant improvements in sentiment scores were observed across all investigated sentiment states. The trajectories for loneliness, sadness, stress, anxiety, depression, despair, helplessness, and optimism revealed consistent and statistically significant improvements across all 8 dimensions (Figure 3). Notably, optimism showed the most pronounced change (n=23,491, ~77%; R²=0.938; P<.001), indicating that DPS use was associated with increased feelings of hopefulness and support. The greatest reductions in negative sentiment states were observed in loneliness (n=11,492, 46%; P<.001), stress (n=13,928, 46%; P<.001), and sadness (n=10,741, 45%; P<.001), suggesting that DPS use was associated with nearly halving users’ initial distress.

Consistent with previous research [7], user engagement revealed that nearly two-thirds of sessions occurred outside business hours. This high level of engagement after hours may reflect limited access to traditional in-person or typical tele-mental health support, as well as employees’ greater openness to sharing personal struggles during periods of privacy or reduced work-related stressors.

Users discussed different topics during business hours compared with outside work hours. During business hours, the most common topics were relationship issues, workplace concerns, financial or social determinants of health, trauma, and stress, whereas outside work hours, relationship issues were discussed most frequently, followed by trauma, workplace concerns, stress, and loneliness. Although concerns such as relationship issues were prevalent throughout the day, these differences suggest that users tend to focus more on workplace concerns and social determinants of health during the business day, while outside work hours they are more likely to explore more personal issues such as trauma, stress, and loneliness.

Differences in user activity based on time of use also emerged. Users were more likely to engage for longer periods (Figure 2A), send a higher number of messages and typed characters (Figures 3C and 3D), and discuss more topics (Figure 2E) outside business hours. By contrast, users typed faster during work hours (Figure 2B), suggesting that time constraints related to work responsibilities may limit engagement. Overall, users were able to participate in deeper conversations outside work hours, indicating the need for services that are accessible beyond the typical 9-to-5 window.

Baseline sentiment states differed by time of engagement between the 2 cohorts (Table 5). Users who engaged during business hours reported significantly higher baseline stress, potentially due to situational demands (eg, work or other daytime stressors) [58,59]. By contrast, users who accessed DPS after business hours reported higher baseline levels of depression and loneliness, aligning with research suggesting increased social isolation during evening and nighttime hours, when there are fewer opportunities for social interaction [60,61]. Meanwhile, other sentiment outcomes did not differ by time of day, suggesting that they may be more stable and less influenced by situational factors [62].

These findings suggest 2 distinct sentiment profiles: engagement during business hours is characterized by acute, reactive distress, whereas after-hours engagement reflects more chronic vulnerability. Improvements in sentiment states support these profiles. Depression, helplessness, and loneliness were reduced at higher rates with DPS use outside work hours, suggesting deeper personal engagement (Figure 4). Comparable reductions in anxiety, despair, sadness, stress, and optimism across the 2 groups indicate that these dimensions may be less affected by timing [58-61]. Furthermore, DPS supported psychological relief independent of timing, reinforcing the need for around-the-clock accessibility.

Another significant factor influencing sentiment improvements was resource sharing within DPS sessions. Sessions that included at least one resource, such as coping strategies or other psychoeducational materials, were associated with greater positive sentiment changes in anxiety, depression, despair, helplessness, loneliness, stress, and optimism (Figure 5). This finding aligns with the peer support literature, which emphasizes the therapeutic value of shared knowledge and validation of distress [63]. Peer support has been described as “a system of giving and receiving help founded on key principles of respect, shared responsibility, and an agreement of what is helpful” [64]. In digital formats, where users may initially feel disconnected or skeptical, resource sharing may help deepen engagement, enhance perceived value, and reinforce feelings of self-efficacy.

Interestingly, users who did not receive a resource showed slightly greater improvements in sadness, although this difference was minimal. This pattern may suggest that individuals with higher sadness were less likely to seek additional support materials, or that focusing on problem-solving interventions may temporarily heighten awareness of one’s struggles. These nuances highlight the importance of tailoring support strategies to individuals, suggesting that resource sharing may be most impactful for those navigating more acute sentiment states, while others may benefit more from affirmation and increased social connection.

While the study provides valuable insights, several limitations should be noted. The observational design prevents causal inference regarding the relationship between session characteristics and sentiment outcomes. An important limitation is that the primary outcome—sentiment change—was based on LLM-derived sentiment scores rather than clinical psychometric instruments. Although prior work and post hoc analyses have shown alignment between LLM-derived sentiment scores and expert human ratings, this study did not examine the psychometric properties of the sentiment scoring approach used. Given the large dataset, human annotation and validation were not conducted. However, post hoc analyses on a subgroup of conversational transcripts confirmed alignment between the model and expert human interpretation. Therefore, the reported sentiment score changes are not diagnostic or absolute measures of emotional change, but rather model-estimated shifts in sentiment within DPS sessions. Despite their relevance in understanding how DPS may support users and potentially encourage engagement with additional clinical care, these results should not be interpreted as indicators of absolute sentiment severity and are not equivalent to self-reported or clinician-scored outcomes. They do not reflect psychiatric symptom severity.

Additionally, limited demographic data restrict analyses of how variables such as age, gender, cultural background, or job role may shape engagement patterns or session efficacy. Although more than half of the users who reported demographic information identified as female and White—a pattern common across digital mental health platforms—DPS attracted a higher proportion of male and Black, Indigenous, and People of Color users than is typically observed. However, the limited demographic data prevented detailed subgroup analyses of sentiment outcomes. Future research should adopt randomized controlled trials or mixed methods designs to identify causal mechanisms underlying DPS effectiveness, while also prioritizing more comprehensive demographic reporting to better capture the service’s impact across diverse populations.

Demographic variables were analyzed for only 8647 out of 24,818 (34.84%) conversations with users who voluntarily disclosed their information. The optional disclosure of personal information introduced selection bias, limiting generalizability to a broader population. A large proportion of individuals may have preferred to withhold information due to privacy, stigma, or other vulnerability-related concerns, and individuals from certain demographic backgrounds may be more likely to face stigma or distrust the health care system, potentially skewing the findings. Therefore, these results may not fully represent the demographic distribution of the broader population; however, they provide valuable insight into user characteristics. The analyzed subset remains sizable and informative regarding the distribution of users in real-world digital mental health settings.

Longitudinal tracking could also reveal whether sentiment improvements persist over time and whether early engagement predicts long-term EAP utilization or mental health outcomes. Moreover, segmenting user data by demographic and occupational variables could clarify subgroup differences, such as among shift workers, caregivers, parents, or underrepresented racial and ethnic groups. Further investigation of digital literacy and privacy preferences as moderating factors could help guide the design of more inclusive digital support systems. Finally, further exploration of resource sharing is warranted. Examining resource types, relevance, and user feedback could help refine the resources provided. Such analyses could also inform best practices for moderator facilitation and content curation within DPS, optimizing its function as a bridge between nonclinical and clinical support systems.

The implications of these findings for workplace mental health strategy are substantial. Organizations can benefit from prioritizing awareness-raising efforts for EAP offerings through multimodal, repeated communications, especially during employee onboarding and at moments of organizational stress [54]. Embedding DPS as a core EAP component could expand reach and reduce barriers to support. Leadership endorsement, testimonials, and peer ambassador programs may help reduce stigma and foster a culture of openness [54]. Moreover, training moderators in sentiment attunement and evidence-based resource sharing may enhance user engagement and, in turn, increase the perceived benefits of DPS in improving sentiment state changes [9]. Integrating DPS within EAPs is a scalable approach that could reduce stigma associated with mental health care and improve employee well-being. This approach aligns well with a modern workforce that requires immediate, accessible, and flexible support tools.

This study underscores the importance of 24/7 access to mental health support, especially for employees who experience emotional distress outside standard business hours. However, because the primary measures were based on LLM-derived sentiment scores, conclusions about effectiveness remain exploratory. Around-the-clock DPS can bridge gaps in care when traditional services are unavailable, helping prevent crisis escalation and demonstrating organizational commitment to employee well-being. To improve EAP utilization, organizations should invest in consistent awareness efforts and destigmatize help-seeking. Leadership endorsement, integration of mental health into everyday communications, and collaboration with employee resource groups can broaden reach and foster a culture of support.

In-session resource sharing demonstrates the value of a hybrid approach that combines AI recommendations with trained peer support moderators to deliver targeted, practical tools. Incorporating psychoeducational resources within sessions may increase user engagement, emotional regulation, and self-efficacy. Furthermore, tailoring resources to users’ expressed needs in real time influences their effectiveness.

Finally, effective mental health strategies must be inclusive and flexible, adapting to diverse schedules, privacy needs, and digital comfort levels. By aligning accessibility with flexibility, employers can offer effective digital mental health interventions to employees. Overall, this study contributes to the growing evidence positioning DPS as an effective, scalable, and equitable tool for mental health across a diverse user base.