The National Institutes of Health institutional review board exempted this study on October 11, 2019, under category 2, research that only includes interactions involving educational tests, survey procedures, interview procedures, or observation of public behavior (title 45 of the Code of Federal Regulations, part 46.10(d)(2)), and category 3, research involving benign behavioral interventions (title 45 of the Code of Federal Regulations, part 46.10(d)(3)). ICF International’s institutional review board exempted this study on November 19, 2019, under category 2 and approved an amendment on February 26, 2020. All participants verbally consented to take part in the study. Participants received US $150 as compensation. Participants were assigned numeric identification numbers that were used during the focus group discussions and in the transcripts.

We partnered with UserWorks, Inc, to recruit a convenience sample of 38 young adults with low SES who smoked. Individuals with low SES are a priority for smoking cessation efforts, with high prevalence of cigarette smoking among adults with low incomes (18.3%) and those without a high school diploma or who passed the General Educational Development test (20.1% and 30.7%, respectively) [38].

Eligibility criteria included being aged 18 to 29 years; having a low SES, as evidenced by being neither a 4-year college graduate nor a college enrollee [39-41]; being a current smoker who smoked at least 100 cigarettes in their lifetime and reported smoking every day or some days; willingness to quit within 6 months; not currently using any smoking cessation aids or noncigarette combustible tobacco products; being a smartphone owner; and speaking English. Recruitment took place between January 2020 and April 2020. The sample size was deemed sufficient to achieve data saturation, and none of the participants withdrew from the study [42,43]. The 32-item COREQ (Consolidated Criteria for Reporting Qualitative Research) checklist can be found in Table S1 in Multimedia Appendix 1 [44].

We conducted 12 virtual focus groups lasting approximately 1.5 hours each on GoTo Meeting using a topic guide informed by the second unified theory of acceptance and use of technology (Multimedia Appendix 1) [45,46]. In this paper, we focus solely on Quit Journey’s message library and just-in-time support feature, which are under development. The moderator described the concept behind the on-demand message library and just-in-time support feature in 8 focus groups and presented mock app pages with placeholder messages and 5 sample tweets in 4 focus groups. For example, one sample tweet read the following:

All sample tweets can be found in Note S1 in Multimedia Appendix 1. In total, 31.6% (12/38) of the participants were involved in 2 focus groups, one in which the moderator presented the concept behind utilizing user-generated content in Quit Journey and one in which the moderator presented app mock pages with placeholder user-generated content. Our work on perceptions of smoking cessation apps and of Quit Journey and its individual features has been published elsewhere [47-51; Wakeman M, unpublished data, April 2025].

We conducted dry runs to ensure that the sessions ended on time. TG, a male user experience strategist, moderated the discussions. EL, a female strategic communications and marketing project director, was a backup moderator and recorded notes. TG, EL, and SE-T were the only nonparticipants present during the focus group discussions, and none had a previous relationship with the participants. Participants were informed that the moderators were unaffiliated with the research group that commissioned the study. The focus groups were audio recorded and auto-transcribed by GoTo Meeting. Three members of our staff verified these transcripts against the audio files. Transcripts and findings were not returned to participants for feedback.

We adopted a deductive thematic approach to data analysis [52]. After an initial review of the transcripts, we developed codes and corresponding themes based on 5 constructs from the second unified theory of acceptance and use of technology: effort expectancy, facilitating conditions, hedonic motivation, performance expectancy, and social influence [45,46]. Furthermore, we coded the transcripts for negative, neutral, and positive sentiment; design concepts; suggestions for improvement; and Quit Journey app features [49-51]. Briefly, effort and performance expectancies were defined as “perceived ease or effortfulness” and “perceived usefulness or helpfulness” of mobile apps and their features, respectively. Facilitating conditions were defined as “factors that can aid or impede [their] uptake or use,” whereas social influence referred to “perceived importance of significant others’ recommendations and approval” in considering use of mobile apps and their features. Hedonic motivation was defined as “perceived fun, pleasure, or enjoyment (or lack thereof) associated with [their] use.” Negative and positive sentiment captured statements that indicated “a sense of disapproval, criticism, or skepticism” or “a sense of approval, praise, or certainty” about mobile apps, respectively. Neutral sentiment captured remarks that “did not have positive or negative tone, contained equal number of positive- and negative-toned remarks, or were conditional in nature.” The design concept code captured statements related to the visual depiction and aesthetics of mobile apps or their features. Finally, the suggestion code captured statements that were “concerned with improvements, modifications, or adjustments” to mobile apps or their features, whereas the app feature code captured the Quit Journey app feature being discussed (eg, carbon monoxide tracking; Wakeman M, unpublished data, April 2025) [47-51]. Quotes that did not fit under a technology acceptance construct but still conveyed a particular sentiment were coded as “not applicable.”

We introduced new codes if the content in the transcripts did not fit the predefined ones. In coding the transcripts, we applied the predominant theme in each quote even if it was not mentioned explicitly (ie, in response to a moderator’s question). When one theme was contingent on another, we coded the quote for the underpinning theme. We followed a multicoding approach where multiple domains (eg, technology acceptance and sentiment) could be applied to a single quote but only 1 code could be selected from each domain (eg, negative, neutral, or positive).

MW and LT independently coded the focus group transcripts. Using the ATLAS.ti qualitative software (version 8; ATLAS.ti Scientific Software Development GmbH), we calculated 2 intercoder agreement measures: the Krippendorff c-alpha and Krippendorff cu-alpha. The first, a measure of separating relevant and irrelevant content (ie, whether coders identified texts of similar locations and lengths), was 0.82 [53]. The second, a metric related to semantic domain reliability (ie, whether coders coded for the presence or absence of groups of related themes), was 0.66 for technology acceptance, 0.70 for sentiment, and 0.94 for app features [53]. MW, LT, and SE-T discussed discrepancies and resolved them.

Sample characteristics are shown in Table 1, and detailed participant characteristics can be found in Table S2 in Multimedia Appendix 1.

User-generated content was an acceptable source of smoking cessation support messages. Of 108 extracted quotes, roughly half (n=56, 51.9%) reflected a positive sentiment (Table 2). Discussions largely focused on performance expectancy (n=37, 34.3%), followed by effort expectancy (n=7, 6.5%), hedonic motivation (n=7, 6.5%), and facilitating conditions (n=1, 0.9%). All quotes appear verbatim in Tables S3 and S4 in Multimedia Appendix 1.

Most quotes on the usefulness of user-generated content for smoking cessation (25/37, 67.6%) overlapped with positive sentiment. Reasons for its perceived usefulness included providing nonrepetitive, authentic content from people “going through the same thing as you” (P35). User-generated content was also seen as a validation of the intended audiences’ emotions and experiences while imparting a sense of community and social support:

Participants emphasized the benefits of knowing the experiences of others, which could improve their self-efficacy and motivation to quit:

Quotes with a negative sentiment (9/37, 24.3%) captured skepticism about the relevance of others’ success stories or their motivating impact on individual participants’ quit journeys:

Negative sentiment also reflected participants’ reactions to sample tweets that were perceived as sarcastic or ironic in tone or those that referenced the adverse health effects of smoking:

Participants perceived Quit Journey’s tweet-based message library as both easy and fun to use. Quotes reflecting effort expectancy, or the perceived ease of using the message library, and hedonic motivation, or the perceived enjoyment of using the message library, were generally positive (6/7, 85.7% and 4/7, 57.1%, respectively):

Discussions rarely focused on conditions that could facilitate or impede the use of a Twitter-based message library or just-in-time support messages. For example, one participant had no privacy concerns, noting that they would be comfortable with the app collecting their ratings of messages to personalize message content as the app learned their preferences.

An additional 105 quotes focused on participants’ suggestions related to message content, timing, and frequency. First, to maintain the authenticity of user-generated content (ie, ensuring that app users know that the messages were authored by real people), participants suggested that messages be attributed to the poster’s name, Twitter handle, or online platform. Others suggested that the app’s library maintain the visual presentation of Twitter messages to signal the messages’ authenticity. Finally, participants suggested that the tweets should be unedited or unaltered, including emojis and any spelling or grammatical errors unless they jeopardized the meaning of the message. If edited, it should be minimal to aid comprehension:

Participants emphasized that sarcastic, ironic, or negative tweets should not be included in the app’s message library:

Regarding push messages, participants suggested tailoring the messages based on age, smoking behavior, and ratings of previous messages, among other factors. Furthermore, participants emphasized the importance of being able to customize the timing and frequency of notifications. For example, although feedback was mixed, most participants preferred to receive support messages before a craving occurred. Participants suggested that they would be receptive to 1 to 10 messages per day, with 3 messages per day being the most common:

This study showed that user-generated content was an acceptable source of support messages in Quit Journey, a smoking cessation intervention targeting individuals with low SES who smoke. The perceived usefulness of user-generated content reflected core characteristics of effective health messaging, including their brevity, relatedness to target populations, and narrative-like nature, while avoiding pitfalls common to expert-designed messages, such as monotony, repetitiveness, and didacticism [3,6]. Suggestions to improve user-generated content for use in Quit Journey’s message library and just-in-time support feature included refraining from overly negative or sarcastically toned messages and tailoring the messages to users’ characteristics, time-varying needs, and message preferences [3]. Cumulatively, these results suggest that we can leverage user-generated content to build Quit Journey’s message library. More broadly, user-generated content marks a departure from the limited number of top-down, expert-designed messages commonly used in health interventions, which can be subject to persuasion resistance and message fatigue [8,9]. Moreover, the use of user-generated content exponentially increases the pool of candidate messages, which allows for the tailoring of message content, frequency, and timing to users’ needs. This is particularly true for technology-based smoking cessation interventions, where the capture of time-varying psychological, physiological, and contextual factors is increasingly feasible [54,55]. Our proposed approach of relying on user-generated content is applicable to practically all health conditions and behaviors with relevant publicly available online content, including addictive and lifestyle behaviors [56,57].

User-generated content includes elements of effective health messages, including its brevity and narrative-like nature, which can improve message persuasiveness and effectiveness in promoting behavior change [30,31]. Indeed, our participants attributed the usefulness of user-generated content to its realism and role-modeling from others who have attempted to quit, successfully or otherwise. For example, one participant noted their aversion to “computer-generated” messages (P11), whereas another noted how these messages could improve self-efficacy, saying that “if someone else can do it, maybe...you can do it too” (P21). These positive perceptions may also reflect involvement with message creators and trust in the health information provided as personal stories by those perceived to be similar to them and with firsthand experiences [27,29,58]. Evidence shows that peer messages and emotional and narrative content have been used to effectively treat nicotine dependence and are associated with greater engagement with tobacco cessation interventions [32,59]. Notably, tweets are similar in length to SMS text messages, which have proven successful as smoking cessation interventions [60]. While user-generated content has been shown to influence nonhealth decision-making (eg, purchasing decisions) [61,62], future research should examine its persuasive effect on health behaviors, identify mechanisms of its effect (eg, self-efficacy and perceived norms), and compare its effectiveness to other content generation approaches (eg, expert-crafted messages and artificial intelligence conversational agents or chatbots [63]).

Open-source content negates message fatigue associated with message repetition [8,9]. A typical smoking cessation intervention is 6 to 8 weeks long and includes a fixed number of support messages (eg, the National Cancer Institute’s SMS text messaging intervention SmokefreeTXT) [64]. An intervention recipient is bound to receive the same messages during the intervention once they have exhausted the number of messages dedicated to a particular purpose (eg, craving support). Furthermore, individuals who smoke often attempt to quit several times before they are successful [65]. This means that they will be exposed to the same messages if they reset their quit day or re-enroll in an intervention. Additionally, tailoring messages to individuals’ characteristics and time-varying factors as they undergo an intervention quickly becomes a challenge [66]. This could be one of the reasons why many smoking cessation interventions use only basic tailoring strategies (eg, addressing an intervention recipient by their name) [67]. A large pool of candidate smoking-related messages curated from Twitter or other platforms can facilitate tailoring of intervention message content.

Our results show that the appeal of user-generated content as a source of cessation support messages was dependent on message content and attributes. Participants generally preferred positive messages over those perceived as sarcastic or ironic, consistent with previous research [68]. Furthermore, perceptions were mixed on messages that focused on the adverse health effects of smoking. These discussions reflect debates on the use of fear appeals (ie, messages that include a threat) and gain- versus loss-framed messages (ie, messages that emphasize benefits of compliance with message recommendations or losses associated with noncompliance, respectively) in health interventions [69,70]. There is evidence suggesting that fear appeals can be effective in promoting smoking cessation [71-73], including among individuals with low SES who smoke [2]. It is noteworthy that the persuasiveness of a message depends on various factors, such as the behavior in question (eg, one time vs repeated and prevention vs detection), audience characteristics (eg, readiness for change), and other message elements (eg, inclusion of self-efficacy statements). For example, evidence shows that gain-framed messages can be slightly more effective in promoting some preventive behaviors, including smoking cessation [74,75]. Additionally, there are unintended and ethical consequences of fear appeals and loss-framed messages and questions about their long-term effects on behavior [76]. Taken together, feedback from our participants and existing literature suggests a need to identify the underlying theoretical concepts conveyed in user-generated messages and investigate the effect of different types of user-generated messages on behavioral outcomes at various stages of a smoking cessation intervention.

Participants’ suggestions centered on message attribution and content and the frequency and timing of push messages. They emphasized the appeal of attributing the messages in Quit Journey to their actual authors. This is consistent with research showing that message source can increase trust and operate as a cue to persuasion and attitude change [27,77]. Relatedly, participants noted the need for the messages to be authentic and human-authored, suggesting that the tweets retain their conversational tones and even grammatical errors. Finally, participants emphasized the need to tailor message content to personal and dynamic factors [66,78,79]. For instance, participants suggested several tailoring factors, including cravings, mood, and stage of quitting. Furthermore, participants suggested that, on average, they would be open to receiving up to 3 daily push notifications but preferred to have the ability to customize the frequency and timing of these messages. While these requests are common [80], they defeat the purpose of just-in-time support [10]. A compromise could be to allow users some control over the number of daily messages they want to receive and giving them options to customize other aspects of the intervention (eg, app aesthetics).

To our knowledge, our approach to curating publicly available user-generated content in a smoking cessation mobile intervention (and in health interventions generally) is the first of its kind [22,54]. In this study, we demonstrated initial acceptance of this approach among the target population of our smoking cessation intervention, Quit Journey. Study strengths include a diverse group of participants, with no more than 50% being from one race and/or ethnicity. Limitations include the reliance on educational level as a sole indicator for SES [39-41]. The focus groups were held virtually due to the COVID-19 pandemic, which may have impacted engagement in discussions. Additionally, the pandemic had mixed effects on smoking behaviors [81], which consequently could have affected participants’ perceptions of our smoking cessation mobile app and its components, including the proposed user-generated message library to support smoking cessation. Participants were only shown 5 example tweets, which were not representative of the wide range of smoking-related tweets that could have elicited different reactions from participants. Furthermore, because the message library and just-in-time support features of Quit Journey were still under development, participants were presented with mock pages with placeholder messages. Participants were not representative of all individuals who smoke, such as older individuals, some of whom may not be as receptive to social media content as the young adults in our study [82]. Finally, due to scheduling conflicts, focus groups had unequal numbers of participants.

This study provided evidence of the acceptability of user-generated content as a source of support messages in smoking cessation interventions. Implementing this new approach to developing health messages requires additional research to select and evaluate user-generated content with the target audience before these messages are integrated into a behavioral intervention. Although we focused on smoking cessation, our proposed approach to health message development can be applied to other health conditions and risky behaviors to accommodate increasing needs for content-diverse, relatable, and short messages that can support individual users and their dynamic needs during behavior change.