Social networking sites (SNSs) can serve multiple purposes for individuals with mental health difficulties [1]. These include facilitating self-disclosure of mental health disorders, acting as discursive spaces, and offering exposure to both harmful and helpful mental health–related content [2,3]. However, several research studies have identified relationships between SNS use and development or exacerbation of mental health symptoms, including depression, anxiety, psychosis, eating disorder, self-harm, and suicide behaviors [4,5].

With approximately 1 in 8 people globally living with a mental health condition [6] and considering the multifaceted role SNSs can have in shaping mental health experiences, platforms have a significant responsibility to support and guide users in managing their well-being. SNSs currently use a variety of moderation and safety techniques to regulate their online communities, including blocking and removing content, banning users, outlining community guidelines, and providing mental health resources, through automatic and manual processes [7].

Despite this, concerns remain about the effectiveness of SNS safety practices [8]. For instance, users engaging with self-harm and suicide material online have reported actively avoiding online help during crises, including dismissing pop-ups and support links [9]. Furthermore, SNS users engaging with self-harm and suicide content online have criticized platform inconsistencies regarding content removal and the communication of decisions around this, reporting that these moderation actions can lead to feelings of stigmatization and harm [8,10]. In addition, studies have found that due to strict censorship of mental health–related terms, platform users may turn to “algospeak,” where they adopt alternative codewords or phrases to avoid having content removed or banned by moderation systems [11], or to circumnavigate search filters [12,13]. Another study, which explored how people use platforms and technology to support their mental health [14] found that users wanted the provision of mental health support on SNSs to be more nuanced and trustworthy, with clear, evidence-based guidance. Concerns have also been raised more generally about users’ ability to read and understand SNS policies and guidance. One study identified significant variation between community guidelines across platforms, highlighting how this can negatively impact user safety and trust [15]. Another study stated that users’ contract requirements with SNSs whereby they state they have read and agreed to the terms of service (ToS) is “the biggest lie on the internet” [16].

These findings highlight possible inefficiencies in current moderation and safety mechanisms used by SNSs and significant potential risks, particularly for vulnerable users, including exposure to harmful content, inadequate crisis responses, and possible disengagement from regulated spaces in favor of less-moderated ones. The full extent of such moderation and safety challenges across SNSs remains unclear, highlighting the need for more in-depth explorations of user experiences. The introduction of the Online Safety Act (OSA) [17] in the United Kingdom, which aims to tighten safety regulations and standards for digital platforms, further emphasizes this need for comprehensive data on current SNS practices and their effectiveness, to fully assess and address issues.

The walkthrough method is a valuable tool for understanding user experiences on these online platforms [18,19]. It is a structured ethnographic approach examining application and platform features, settings, and interactions by “touring” the platform interactively. The method focuses on understanding how platform design and resources potentially encourage and impact user behavior. For instance, in their paper on a walkthrough of the dating platform Tinder, Gillett [20] identified several user safety issues, including the problematic placement and design of the reporting icon. The author proposed that these design choices could significantly influence user behavior, leading to potential unintended interactions with harmful content and adverse effects on user well-being. This study demonstrated how the walkthrough method can be used to reveal critical aspects of user experience that might otherwise be overlooked.

Using the walkthrough method, this study aimed to systematically identify how SNSs manage mental health content and user well-being through moderation, safety features, resources, and policies and to explore the potential impact of these platform attributes on users. By identifying platform successes and areas for improvement, the study aimed to contribute to the creation of safer and more supportive digital environments.

This study used an ethnographic walkthrough method to systematically and critically assess the approaches of various SNSs in moderating mental health content and promoting user well-being. Researchers actively participated in the platforms as observers to evaluate their features, guidance, and resources.

This study adhered to ethical guidelines for internet-mediated research [21]. Researchers ensured that their participation did not interfere with the functioning of the platforms, mislead other users, or go against platform regulations. Confidentiality was maintained by avoiding the recording of personally identifiable information, such as user-generated names or specific images. No interventions were made in response to harmful content observed, ensuring that the user experience and platform responses remained unaltered. No financial compensation was provided to researchers as they undertook the research as part of their routine professional role. There were well-being procedures in place for researchers, and they were able to withdraw from conducting the walkthrough checklists if they wished.

Ethics approval was received from the Faculty of Health Sciences Research Ethics Committee at the University of Bristol (approval number 17972).

As researchers in the field of mental health and digital technology, we acknowledge that our personal and professional experiences shape our perspectives and approach to this study. Researchers ZH and LB are both White women from the United Kingdom. ZH identifies as a digital native, reflecting a comfort with digital technology and early adoption of various online platforms throughout her life. In contrast, LB is a digital immigrant.

We selected 4 SNS platforms for evaluation. TikTok and Instagram were selected due to their status as two of the most widely used social media platforms in the United Kingdom, particularly among adolescents [22,23]. Tumblr was chosen for its established reputation as a significant SNS platform, offering a contrasting perspective to the newer platforms. In addition, although Tumblr has less representation in the current research literature and discussion around online safety [23], media reports indicate a recent shift in their user base toward the Gen Z population [24,25]. Tellmi was chosen as a smaller platform specializing in mental health support and discussion among young people, which offered an opportunity to evaluate a platform specifically designed with the mental health of users in mind, providing a useful comparison to SNSs that prioritize broader aspects of social interaction. For this study, Facebook was excluded from our SNS evaluation despite it being the most accessed platform in the United Kingdom. This was because of its lower popularity among younger UK users [23] and its likely similarities with Instagram, given their shared ownership by Meta.

To conduct our walkthroughs, we used 2 checklists (Multimedia Appendices 1 and 2) to systematically evaluate each platform’s approach to moderating mental health content and supporting user well-being. Checklists were developed from previous research findings [8,26] and industry guidelines for suicide and self-harm content [27]. To conduct the walkthrough checklist, ZH used a new iOS device and LB used a web browser. Table 1 depicts the platform areas that the checklist focused on, and Table 2 shows the stages of use that were observed and documented through field notes and screenshots.

Data were analyzed using a thematic analysis approach [29,30]. Initially, ZH reviewed the field notes, observations, and screenshots, applying a deductive approach to generating codes based on the checklist format. These codes were then further refined through an iterative process. The codes were then organized into groups, based on similarities in the issues, ideas, or patterns they addressed. These grouped codes were then defined as themes that represented meaningful constructs of the coded data. Data for each theme can be viewed in a comparative table, by platform (Multimedia Appendix 3).

Data were analyzed to understand the current provisions by platforms regarding mental health–related content moderation and user well-being. The themes we defined are summarized in Table 3.

Platform resources for mental health support varied significantly. Instagram and TikTok provided a range of internal resources on self-harm, suicide, and disordered eating, including pages with practical advice, educational content, and emergency procedures that were written in consultation with credible external organizations. Uniquely, TikTok also published a page focused on creator burnout, and Tellmi allowed users to complete and track scores in mental health quizzes. In addition, Instagram hosted a variety of resources around mental health that had been developed and published by external services and charities, which users were able to download. Similarly, Tellmi directed users to a wealth of resources hosted externally. Both Instagram and TikTok provided a somewhat overwhelmingly wide range of guidance and policy pages directed at young people and their parents or guardians, again providing information on well-being support and practical online advice. Tumblr only provided 1 brief page for users that aimed to help with mental health needs, giving specific advice on what to do in response to immediate harm. On this page, Tumblr also gave contact details for a couple of UK-based mental health organizations. Tellmi and Instagram offered a broader range of relevant organizations, including those focused on mental health, online harms, and intersectional factors such as sexuality and religion. Notably, TikTok, Instagram, and Tumblr often featured US-centric resources across help and safety pages, which sometimes excluded other countries.

Platforms generally communicated with their users using clear and professional language, suitable for their audiences. However, Tumblr’s casual style veered into potentially patronizing territory:

All platforms occasionally relied on nondefinite language regarding processes, which could potentially confuse users and create opportunities for inconsistent practices. When conveying information, Instagram and TikTok were reasonably text heavy, although attempts were made to use more mixed media in youth-centered areas of the platform. Tellmi used images and short descriptions, likely in an attempt to keep younger users engaged. Tumblr was more effective at using a mix of media—such as illustrations, GIFs, and videos—throughout their information and guidance pages aimed at all audiences.

Tellmi organized its resources and help pages in a single directory, and Tumblr, across a few pages, resulting in a concise and user-friendly experience. In contrast, Instagram and TikTok disseminated information about mental health policies, guidance, and resources across various centers, with Instagram often redirecting users to additional Meta sites as well. Although this approach offered multiple valuable resources and ways to discover them, it complicated user navigation and increased information burden and repetition.

Within the app interface, Tumblr, TikTok, and Instagram provided a trigram “hamburger” icon for access to most tools, pages, and policies, whereas Tellmi did not use familiar icons along a navigation bar. On the browser, access to all relevant documentation was available in the footer of the platform home page. In addition, several platforms functioned differently depending on device, occasionally restricting direct access to self-moderation options on browsers. Finally, Instagram’s use of hidden links—accessible only through specific paths—and TikTok and Instagram’s occasional reliance on blog posts for self-moderation tool descriptions limited accessibility for users.

This study extended our understanding of how a selection of SNSs manage user safety and well-being. Using walkthrough analysis, we observed both accomplishments by platforms and challenges they face in balancing content management, moderation, and user empowerment, leading to the development of preliminary recommendations for the industry (Textbox 1). Findings raised some critical concerns regarding the effectiveness of existing practices and the need for further research.

Previous studies have shown that application onboarding periods are critical in various contexts, including assisting new users’ engagement with features [31,32], and helping patients with mental health disorders manage expectations and build confidence and proficiency with technology [33,34]. However, findings from this study showed that platforms were limited in their education of users during initial engagements, particularly lacking clear direction to “community guidelines” and “self-moderation” tools. Such a passive approach may contribute to users feeling less informed about platform features and potentially susceptible [35].

Despite this, there was some evidence of platforms using nudges—subtle prompts designed to positively influence user behavior [36]—during this period. This included TikTok’s data protection and safety videos for younger teenagers, Instagram and Tumblr’s prompts for users to follow accounts, personalizing recommendations, and Tellmi’s information regarding its limitations in terms of crisis support. These nudges served as proactive ways of engaging users with platform mechanisms, leveraging early interactions to educate or apply settings that could benefit users in the future [36], though there were some limitations, TikTok’s instructional videos could be overwhelming, especially given that online users perceive learning about privacy settings as burdensome [37]. In addition, targeting younger teenagers excluded other users who may need the guidance. Furthermore, TikTok and Instagram allowed users to skip their processes, which, although encouraging autonomy, could result in users being less informed [36]. Tumblr did not offer the same skipping option, but findings indicated that the recommendation system offered options to engage with potentially harmful content, exposing broader moderation issues [38]. This suggests that while platforms have some capacity to engage users through nudges, they may be missing a valuable opportunity for early intervention around safety features and content moderation for their respective user bases. In addition, recommendation system inadequacies need to be considered significant in the safety of new or susceptible users.

Instagram and TikTok offered comprehensive internal mental health–related support, including resources on suicide, self-harm, and disordered eating, as well as areas for younger or parent and guardian audiences. Integrating evidence-based resources internally and matching platform branding may be an effective approach for capturing users’ attention through familiarity and perceived reliability [39,40]. However, the volume of mental health–related resources often led to repetition, resulting in the redundancy of information and potentially user fatigue [41]. Given evidence that individuals with severe mental illness struggle to understand complex content or navigation, platforms should aim to avoid overwhelming users with excessive psychoeducation, safety, and well-being content [42,43]. Furthermore, the presence of broken links and spelling mistakes in some of these sections emphasized poor maintenance, highlighting potential concerns around the platforms’ commitment to providing high-quality, up-to-date resources [44].

Instagram also gave access to several mental health–related resources produced by other organizations, and Tellmi provided similar links to external content in various media forms. This approach allowed platforms to offer a wider range of resources, including those addressing intersectional factors in mental health, without needing to produce the content themselves, thereby improving efficiency. However, directing users away from internal platform resources may have drawbacks, including the need for monitoring external updates and possible inconsistencies in resource branding that could confuse users [44].

Following searches for harmful content, all platforms provided contact information for mental health organizations and emergency services. However, these details were often absent in other mental health–related platform spaces, missing opportunities for preventive intervention [45]. In addition, except for Tellmi, which is for young people in the United Kingdom, platforms overrepresented US resources. This reflects a broader issue of US-centric content bias on global platforms that may marginalize non–US users by limiting accessibility to relevant information [46].

Design decisions on platforms, particularly those impacting usability and accessibility to features, were also considered. Tumblr, TikTok, and Instagram used the “hamburger” icon for access to settings, policies, and resources, although some features were directly accessible during platform use. While the hamburger icon is widely recognized by experienced users, it may appear hidden or unclear to those without prior online knowledge [47]. Tellmi used a ring-bound notepad symbol for their “directory” resources page, which may be less recognizable. Research shows that users with mental health disorders often find abstract icons challenging, highlighting potential usability barriers associated with these icon designs [48].

Study findings indicated that while major platforms have implemented some “self-moderation” strategies to comply with guidelines in the OSA [17], significant gaps remain in how they promote user empowerment, particularly for those engaging with mental health–related content or experiencing mental health difficulties. For instance, TikTok and Instagram introduced screen break tools to encourage user control, but these tools allowed users to return after short periods or to be easily overridden, potentially undermining their effectiveness in favor of user retention. This reinforces criticisms that commercial interests may outweigh ethical considerations in social networking design [49].

In contrast, Tumblr’s limited self-moderation tools and weak content moderation increased users’ potential exposure to harmful content, while Tellmi, although also offering fewer self-moderation tools, compensated with robust content moderation offering better avoidance from unsafe material. However, the supportive nature of interactions on Tellmi could create a sense of responsibility among users, potentially resulting in excessive engagement that current “self-moderation” tools do not address [50]. In addition, the absence of Tellmi moderators overnight could result in user migration to other platforms, leading to riskier engagements elsewhere [51]. Alternatively, TikTok’s “refresh your for you feed” feature was a novel self-moderation tool that empowered users and allowed for continual platform engagement, highlighting benefits for both users and SNSs. However, as with other self-moderation tools, its effectiveness relied on users recognizing the need for online behavior change and having the capacity to act, which may be challenging for individuals experiencing poorer mental health or with lower metacognition skills [26].

This highlights a critical limitation found across platforms—the lack of proactive intervention to raise awareness about “self-moderation” tools. These findings align with the survey by Ofcom video-sharing platform [52], which found that only 42% of UK-based video-sharing platform users were aware of platform safety mechanisms. Along with minimal prompts directing users to tools, our findings revealed insufficient education on their functionality and usefulness. There was also a significant lack of information on tool adoption or any evidence that they underwent user testing, raising questions about their efficacy. This is especially relevant considering a recent research by Bright et al [35] indicating users’ dissatisfaction with online safety technology. These findings suggest platforms may be superficially complying with OSA requirements rather than genuinely committing to empowering users. They also indicate a broader failure across the industry to address the importance of digital literacy surrounding “self-moderation” tools.

Platforms in this study also attempted to improve user safety by implementing default feature restrictions. However, default sensitivity or mature content controls often lacked clarity on whether they addressed mental health–related content or the types of content covered under each level, potentially confusing users [8]. On TikTok and Tumblr, the strictest restrictions were largely applied until users reached the age of 18 years, suggesting platforms may assume age as a qualifier of user efficacy. This overlooks the risks of removing protective measures for users who have not developed the skills or awareness needed to manage their safety independently [53]. Alternatively, Instagram, via its “teen accounts” approach allowed teenaged users to modify defaults with parental approval, encouraging a more open dialogue between users and their families. However, this approach shifts responsibility from platforms to parents. These measures also fail to address prevalent issues of age falsification among young users [54] and parents permitting their children to use social media before reaching the minimum age [52,55]. In contrast, Tellmi stood out by allowing all users to adjust their sensitivity content controls, prioritizing user agency over imposed controls [53].

Despite a lack of proactive user empowerment strategies, the changes made by platforms to improve user safety have likely been driven by the introduction of recent regulations such as the Digital Services Act in Europe [56] and the OSA in the United Kingdom [17]. These frameworks have specifically prompted platforms to implement changes that focus on the safety of young users and the moderation of dangerous or harmful content. However, the effectiveness of regulation implementation remains unclear [57,58]. For instance, an initial case study of Instagram’s “teen accounts” revealed that young people using the feature remain exposed to inappropriate content [59].

In addition, while platforms adapt to meet the technical requirements of regulations, they likely remain unable to overcome long-standing limitations in moderating harmful content without further support or guidance from legislators. Current regulations also overlook variability between types of SNS platforms, including differences in resource availability and community make-up. This lack of insight risks the introduction of regulation that may inadvertently suppress or disempower online users. Country-specific regulations further fail to account for workarounds by users, such as the use of virtual private networks, allowing them to bypass geographical restrictions and access platforms from countries where regulations do not exist. This suggests that although platforms may implement changes in response to legislation, ongoing challenges in user safety and content moderation could remain unaddressed due to gaps in regulatory focus and scope.

Our findings also showed that platforms often implied it was the user’s responsibility to identify and report harmful content, which may be problematic for susceptible users [8] and those unaware of reporting mechanisms [60]. In addition, Instagram’s claim that reported content would feature less for the reporter could discourage the behavior, as users who engage with mental health–related content may fear limiting their own access to resources or information [26]. Observations also revealed that transparency reports on harmful content removal, when available, were difficult for users to locate and interpret. Improving their visibility could alleviate user pressure by showing that much disallowed mental health–related content is automatically flagged, but the continued reliance on user reporting for content that evades moderation systems may undermine confidence.

Platforms did make efforts to streamline reporting, with Instagram and TikTok offering feedback mechanisms and Tellmi providing direct communication with human moderators. However, it remains unclear whether the reporter or reportee receives any support after harmful content is reported, and the absence of feedback or mental health resources for reporters may discourage action [8]. Furthermore, if reported users are not offered appropriate resources, platforms may inadvertently punish individuals for sharing mental health struggles, reinforcing stigma [61]. This highlights the need for greater transparency around platform reporting procedures and clearer communication about the resources and support available to those whose content is reported.

All platforms gave some indication of the types of mental health content that were prohibited. However, while TikTok and Instagram included in-depth examples of allowed and disallowed content, other platforms lacked similar specificity. Furthermore, guidelines often overlooked mental health–related terminology, and there was minimal mention of rules around mental health topics unrelated to suicide, self-harm, or disordered eating. This lack of clarity could lead to subjective interpretations, resulting in self-censorship or unintentional posting of harmful material. Furthermore, only Instagram addressed the allowance of content depicting self-harm scars, a significant oversight by other platforms given that users have previously criticized moderation practices regarding this issue [26]. In addition, consequences for posting harmful content were frequently vague, with platforms using ambiguous language to describe repercussions. While this may allow for nuanced moderation, it risks confusing users and deterring them from posting around their mental health, potentially perpetuating feelings of isolation and stigma [62].

Harmful mental health content was not immediately visible on TikTok, Instagram, or Tellmi. However, on Tumblr, it was available following recommendation choices, posing higher risks to susceptible users [63]. After searching common mental health–related terms, all platforms other than Tellmi provided help resources, as expected for Tellmi as a platform focused on mental health discussion. In some instances, sensitive content overlays were used on harmful images on Instagram, which may empower users by giving them the opportunity to make informed decisions about engagement, encouraging metacognitive skills [64]. Although, if similar messages are presented repeatedly, it may result in desensitization [65] and could increase curiosity among susceptible users [26]. Furthermore, we found the application of overlays on Instagram was inconsistent, which may reduce their effectiveness.

Notably, one algospeak term resulted in content on Tellmi. Although this content was not harmful, its inclusion by human moderators indicated an acceptance of the language. This presents a broader issue in mental health content moderation where censoring common mental health terms can normalize algospeak terms as substitutes, effectively reverting platforms to their original state concerning mental health language moderation [11]. Similarly, mental health–related algospeak terms bypassed moderation on TikTok, Tumblr, and Instagram and resulted in harmful content. This highlights the danger that algospeak terms can more readily expose users to risky content, and their use instead of common mental health terms may exclude users from finding and sharing genuine information and support [12,13].

On Instagram and Tumblr, some user biographies featured the phrase “block don’t report.” This showed an online culture where users actively attempted to circumnavigate content moderation [66]. This direction to others may indicate that those engaging with mental health content believe users are responsible for their own safety through “self-moderation,” rather than relying on the platform to provide protection. It may also reflect a community norm, which could result in newer members feeling pressured to conform and avoid reporting. At the same time, the statement highlights the existence of a space where users feel empowered to express themselves by sharing content. This highlights an ongoing tension for platforms between allowing users to feel empowered through self-expression and ensuring their safety.

Our work found no evidence that platforms evaluated the effectiveness of their self-moderation tools and guidance relevant to user mental health and well-being. This included assessing the usability, accessibility, and function of mental health–related resources and tools as well as the acceptability and practicality of moderation procedures. Notably, there was a lack of evidence of user input and testing, potentially raising concerns about the current efficacy and value of existing measures among users. If such evaluations are being conducted internally, platforms should be encouraged to share these insights publicly to enhance transparency, adding to user confidence and supporting learning across the industry. Future research evaluating these features remains essential to ensure that platforms meet users’ needs and do not promote ineffective or even risky features or guidance.

While this study did expose gaps in the current availability and accessibility of mental health and well-being resources and functions across a selection of SNS platforms, there remains a spectrum of platforms further research should aim to investigate. This includes safety and well-being resources, features, and guidance available on social media, dating, communication, e-commerce, pornography, mental health, and gaming platforms. In addition, research is needed into newly developed SNSs such as those including artificial intelligence chatbot integration plus smaller online communities that are less likely to undergo media or governmental attention.

This study resulted in a set of preliminary recommendations for industry, which give potential ways for platforms to address mental health–related content and moderation concerns. However, these recommendations need further validation and refinement, with stakeholder input, to ensure their applicability and effectiveness.

Due to the rapidly evolving nature of online platforms, we acknowledge that the data collected may become outdated. This research aims to capture a moment in time, and it is possible and hopeful that platforms have since updated their practices to address some of the issues highlighted and align with our recommendations. Nevertheless, we believe these observations remain valuable in understanding user experiences. In addition, the recommendations provided are likely to be relevant for similar emerging or popular platforms.

We also understand the limitations of using the walkthrough method in this study. This approach relies on subjective interpretations of researchers’ observations, which may have led to biases. Although the checklist approach allowed the researchers to be systematic in their investigation of the platforms, it did not represent the real-world experience of a user, limiting the ecological validity. To mitigate these limitations, we included both a digital native and a digital immigrant researcher in an attempt to understand different interpretations and perspectives of platforms. However, as a next step, research should engage directly with users to reflect on their experiences of online mental health–related content moderation and safety.

Tellmi was included in this study as a smaller UK-based SNS, characterized by its user-generated content and social interaction around mental health. We recognize that Tellmi is not equivalent to Instagram, TikTok, or Tumblr in terms of target user populations, content diversity, and platform functionalities. However, given its direct focus on mental health and younger users, we considered Tellmi valuable as a comparator to platforms not specifically related to mental health in evaluating their management of user features, guidance, and resources. Through our analysis, we identified notable practice differences and approaches across all platforms that helped to refine our recommendations.

These findings indicate that while platforms use some successful strategies, they are not effectively addressing the mental health moderation and well-being needs of all users. While larger platforms that undergo governmental scrutiny may introduce changes, these can seem superficial, appearing to prioritize user retention and engagement over meaningful safety intervention. Smaller platforms which face less scrutiny could also present risks due to weaker automated moderation systems resulting in access to harmful content or challenges in managing high user traffic, causing potential engagements with alternative less-moderated platforms. Using human moderation before material is posted may be the most effective approach to avoiding harmful content availability but is likely unfeasible for larger, global platforms due to scalability issues. It also raises concerns about limiting online freedoms, which could have negative consequences for users, particularly for those intentionally sharing harmful content. Platforms should focus on empowering users by educating them about safety mechanisms early, giving them agency over their decisions, and providing clear information regarding content moderation. In addition, while platforms have a responsibility toward the safety of young users, they should also consider other susceptible users, such as those with mental health disorders, during their moderation policy decisions. This study also highlights the need for improvements in the scope and global relevance of government regulations to better address user needs on platforms. Further research is essential to understand the effectiveness of existing interventions used by platforms, including self-moderation tools. User involvement is also necessary for meaningful improvement.