Published on in Vol 10 (2023)

Preprints (earlier versions) of this paper are available at, first published .
Development of a Digital Behavioral Intervention to Reduce the Consumption of Sugar-Sweetened Beverages Among Rural Appalachian Adults: Multiphased, Human-Centered Design Approach

Development of a Digital Behavioral Intervention to Reduce the Consumption of Sugar-Sweetened Beverages Among Rural Appalachian Adults: Multiphased, Human-Centered Design Approach

Development of a Digital Behavioral Intervention to Reduce the Consumption of Sugar-Sweetened Beverages Among Rural Appalachian Adults: Multiphased, Human-Centered Design Approach

Original Paper

1Department of Public Health Sciences, School of Medicine, University of Virginia, Christainsburg, VA, United States

2Department of Psychiatry and Neurobehavioral Sciences, School of Medicine, University of Virginia, Charlottesville, VA, United States

Corresponding Author:

Jamie Zoellner, RDN, PhD

Department of Public Health Sciences

School of Medicine

University of Virginia

16 East Main Street, Suite 101

Christainsburg, VA, 24073

United States

Phone: 1 4349624488


Background: To avoid the low engagement and limited efficacy of digital behavioral health interventions, robust human-centered design (HCD) processes are needed.

Objective: The primary objective of this study was to describe a flexible, step-by-step HCD process to develop digital behavioral health interventions by illustrating iSIPsmarter as an example. iSIPsmarter is a digital intervention for reducing the consumption of sugar-sweetened beverages (SSBs) that comprises 6 internet-based cores metered out over time to deliver the program content, an integrated SMS text message strategy to engage users in reporting SSB behaviors, and an electronic cellular-enabled scale for in-home weighing. The secondary objective is to illustrate the key components and characteristics of iSIPsmarter that resulted from the HCD process.

Methods: The methods were guided by the Model for Internet Interventions and by best practices in HCD and instructional design processes (eg, rapid prototype development and think-aloud protocol). The 3-phased (ie, contextual, prototype testing, end user testing phases) process followed in this study included a series of 13 semistructured one-on-one interviews with 7 advisory team participants from the targeted Appalachian user group. The interviews were content coded by 2 researchers and then deductively coded to the suggested areas of digital behavioral health interventions.

Results: The participants provided rich perspectives pertaining to iSIPsmarter’s appearance, behavioral prescriptions, burdens, content, delivery, message, participation, and assessment. These inputs included requests for built-in flexibility to account for varying internet and SMS text message accessibility among users; ideas to resolve the issues and problems encountered when using the prototypes, including those related to navigation and comprehension of content; ideas to enhance personalized feedback to support motivation and goal setting for SSB consumption and weight; and feedback to refine the development of realistic and relatable vignettes. The participants were able to interact with multiple prototype drafts, allowing researchers to capture and incorporate feedback related to the iSIPsmarter dashboard, daily SSB and weight diaries, action planning, core content, interactions, and vignettes.

Conclusions: Using scientific models and established processes is critical for building robust and efficacious interventions. By applying an existing model and HCD and instructional design processes, we were able to identify assumptions and address the key areas of the iSIPsmarter intervention that were hypothesized to support users’ engagement and promote behavior change. As evidenced by the rich feedback received from the advisory team members and the resulting iSIPsmarter product, the HCD methodology was instrumental in the development process. Although the final iSIPsmarter content is specific to improving SSB consumption behaviors among adults in rural areas, the intent is that this HCD process will have wide applications in the development of digital behavioral health interventions across multiple geographic and behavioral contexts.

JMIR Hum Factors 2023;10:e41262




The availability of digital behavioral health interventions has surged in recent years. However, when deployed in research trials and disseminated in real-world practice, the uptake of and engagement with many digital interventions are often lower than desired [1-3]. In turn, this results in the unrealized potential of both immediate and sustained health outcome improvements among the intended users [3]. Although numerous factors may contribute to low uptake, one of the most important is a suboptimal fit between the characteristics of the technology and the needs, skills, and context of the user. To adequately address these complex interrelationships, the development of digital behavioral health interventions should include a theory-driven, iterative, and human-centered design (HCD) process.

Importantly, there are a number of models and frameworks available to guide the digital intervention development processes [4-9]. The Model for Internet Interventions [4] is a key example that has been used as a basis for the development of many digital health programs [10-13]. This model can help researchers distinguish and operationalize various components of digital behavioral health interventions and identify the relationships among the components. More specifically, this model posits that to explain behavior change across digital behavioral health interventions, it is necessary to consider design-related components, areas, and elements, including user characteristics, environment, intervention content, level of intervention support, and targeted outcomes [4]. Furthermore, the Model for Internet Interventions highlights 8 main areas that comprise the digital health application used to deliver the intervention (ie, appearance, behavioral prescriptions, burdens, content, delivery, message, participation, and assessment). To improve the likelihood of digital health applications meeting the needs and requirements of users, HCD and instructional design processes should be applied when considering and manipulating these 8 areas.

Although this and other models and frameworks provide helpful principles and guidelines [4-9], they are not intended to be a step-by-step prescription for digital intervention development. A scoping review of 160 papers regarding research activities for participatory eHealth development processes identified a variety of methods and products [14]. However, there was no evidence of an optimal single-step approach for developing digital behavioral health intervention applications [14]. The findings from this narrative review highlighted the importance of researchers and developers selecting the most appropriate objectives for and methods for developing the context and user characteristics of their digital behavioral health interventions.

Despite the flexibility in methodological processes, the application of HCD and instructional design processes in the development of digital behavioral health interventions is consistently recommended [15]. The International Organization for Standardization defines HCD as “an approach to systems designs and development that aims to make interactive systems more usable by focusing on the use of the system and applying human factors and usability knowledge and techniques” [16]. Interdependent design activities include understanding and specifying the context of use, specifying the user requirements, producing design solutions, and evaluating the design. In addition, the International Organization for Standardization posits that six requirements must be met for an HCD process: (1) the design is based upon an explicit understanding of users, tasks, and environments; (2) users are involved throughout design and development; (3) the design is driven and refined by user-centered evaluation; (4) the process is iterative; (5) the design addresses the entire user experience; and (6) the design team includes multidisciplinary skills and perspectives [16].

Complementary to HCD concepts, instructional design processes involve designers executing cycles of continuous formative evaluation to ensure that the intervention meets the users’ needs, prior knowledge, and experience [15]. These instructional design processes should involve setting measurable learning objectives or performance requirements, assessing the users’ achievement of the targeted outcomes, and revising the program components until the desired outcomes are achieved [15,17-20]. Ultimately, the ability of digital behavioral health interventions to achieve sustained engagement and desired behavioral outcomes is enhanced when using scientific frameworks along with proven and context-specific HCD and instructional design approaches during the development process.

Appalachian Digital Environment and Sugar-Sweetened Beverage Context

Although HCD processes are important across different contexts, the need for them is magnified in rural regions, including Appalachia. Historically, extending digital interventions into the Appalachian region has been hindered by digital divide [21,22]. However, similar to other rural American communities, Appalachia is making great strides in narrowing the digital divide and closing the rural-urban gap in home broadband internet connection and smartphone ownership [23-26]. However, little is known about how rural Appalachian adults engage with digital behavioral health interventions. As such, human-centered and instructional design processes are especially important for interventions targeting this and similarly underserved regions where access to digital behavioral health interventions has been limited.

In addition to the geographic and digital divide context of Appalachia, the behavioral context of sugar-sweetened beverage (SSB; eg, soda or pop, sweet tea, sports and energy drinks, and fruit drinks) consumption is noteworthy. SSBs are the single largest source of calories in the US diet and account for approximately 7% of the total energy intake of the US adults [27]. In Appalachia, SSB intake is disproportionately higher and accounts for an average of 14% of the total energy intake—twice as high as national estimates [28]. Consistent with both national and Appalachia-specific data showing that SSB is the largest contributor to added sugar intake [27-29], excessive SSB consumption is undeniably pervasive in Appalachia. However, there are only a few known SSB consumption–specific behavioral interventions that have used digital technologies or applied HCD processes [30-33].


The primary objective of this study was to describe a flexible, step-by-step approach to and an HCD process for developing digital behavioral health interventions by illustrating iSIPsmarter as an example. iSIPsmarter is a technology-based behavior and health literacy intervention aimed at improving SSB consumption behaviors among Appalachian adults. The secondary objective is to illustrate the key components and characteristics of iSIPsmarter that resulted from the HCD process. Although iSIPsmarter’s content is specific to improving SSB consumption behaviors among Appalachian adults, the intent is that the HCD process will have wide applications in the development of digital behavioral health interventions across multiple geographic and behavioral contexts.


The development process for the digital behavioral health intervention iSIPsmarter was guided by the Model for Internet Interventions [4] and by best practices in HCD [16] and instructional design processes [17-20], which is being evaluated in a randomized controlled trial (Trial Registration: NCT05030753). Figure 1 illustrates the conceptual integration of this model and these processes, along with key definitions. Specifically, the development process of iSIPsmarter included 13 semistructured interviews across three nonsequential iterative phases: (1) contextual, (2) prototype testing, and (3) end user testing phases.

This formative, flexible, and step-by-step approach allowed the advisory team participants to provide insights at all phases of the development process and interact with multiple prototype drafts developed by the content development team (CDT).

The following sections describe the considerations for the CDT, advisory team participants, adaptation context, iSIPsmarter intervention, and adaptation process. Subsequently, the 3 phases of data collection and the data analysis strategy are described.

Figure 1. Conceptual overview and integration of the Model for Internet Interventions and best practices in human-centered design processes.

Ethical Considerations

This program evaluation project involved activities that do not represent human participant research and, therefore, did not require submission to the institutional review board (Human Research Protection Program Standard Operating Procedures [34]).

The Role of CDT

The development of digital behavioral health interventions should be conducted by a multidisciplinary CDT [15,16,35]. This includes subject matter experts, behavioral psychologists, web designers, instructional designers, and developers and programmers.

The multidisciplinary CDT involved in the development of iSIPsmarter included 4 doctoral-level researchers: 2 (50%) SSB content experts who had led prior SSB behavioral intervention trials in the targeted Appalachian region, 1 (25%) expert on digital behavioral health technology interventions, and 1 (25%) instructional design expert. This team was further supported by research staff experienced in nutrition, public health, and digital health development and implementation, including those with expertise in the design and creation of robust user interface and user experience. For the duration of this HCD process, the team met weekly to collaboratively draft advisory team interview guides with clear objectives, develop prototypes, and respond to feedback from the participants. Likewise, the CDT also wrote content for the iSIPsmarter cores, developed media-rich interactions and videos, drafted personas and vignettes, and developed an integrated internet-based platform.

Advisory Team Participants

When developing digital behavioral health interventions, the selection of advisory team members is an important decision. Ideally, the participants should represent the intended end user of the intervention in terms of demographic and cultural characteristics, health literacy and digital literacy skills, geographic location, and patterns of the targeted behaviors [16].

The advisory team members involved in the development and design of iSIPsmarter were a convenience sample of prior SIPsmartER participants who resided in the Appalachia region. The advisory team members also had to have internet access and the ability to review the iSIPsmarter program materials on the web. These members were diverse in terms of age (21 to 60 years), gender, and socioeconomic status. Of the 15 prior SIPsmartER participants invited, 7 (47%) joined the advisory team. These participants were involved throughout the intervention design and development processes, with the first interview conducted in February 2019 and the final interview in December 2021. In total, the advisory team members were compensated with up to US $200 in electronic gift cards for their participation.

Adaptation Context

iSIPsmarter is a technology-based behavior and health literacy intervention aimed at improving SSB behaviors among Appalachia adults [36]. iSIPsmarter was adapted from the evidence-based intervention SIPsmartER [37]. SIPsmartER is a 6-month intervention that includes 3 in-person small-group classes, 1 live teach-back call, and 11 interactive voice response (IVR) calls; the intervention also comprises a 12-month maintenance phase that includes monthly IVR calls (refer to the database Evidence-Based Cancer Control Programs for details [38]). The classes cover key behavioral content and action planning, whereas the IVR calls engage users in reporting SSB intake and personal action planning. Several formative research projects among Appalachian adults initially guided the development of SIPsmartER [39-41]. The content and strategies of SIPsmartER are guided by the theory of planned behavior and health literacy, numeracy, and media literacy concepts [42-46]. In brief, SIPsmartER was found to be effective at reducing and maintaining SSB consumption behaviors through a full-scale randomized controlled trial and a pilot dissemination and implementation trial conducted in collaboration with the Virginia Department of Health districts in Appalachia, Virginia [29,37,47-58]. Although SIPsmartER had been highly successful in reducing SSB consumption, transitioning from its original structure to a fully digital modality presented an opportunity to focus on optimizing scalability and reach.

At the onset of developing iSIPsmarter, the CDT had ample experience understanding and intervening on SSB consumption in the targeted region and had over a decade’s worth of rich qualitative, observational, and experimental data from the previous SIPsmartER trials. As such, content and behavior change techniques related to SSB consumption were relatively well established when embarking on the digital intervention development. However, little was known about the opportunities, barriers, and access to technology in the targeted population for intervention delivery or data collection purposes. Similarly, weight self-monitoring and the use of cellular-enabled scales to encourage in-home weighing had not been previously explored in the targeted population and region. Therefore, the intention of the iSIPsmarter adaptation process was largely focused on changing the mode of delivery and adding key content around weight while maintaining other core components and the cultural relevance for the intended Appalachia target audience.

iSIPsmarter Description

In its final form, iSIPsmarter comprises 6 internet-delivered interactive cores, which are metered out sequentially over time, with a new core becoming available 1 week after completion of the previous core; an integrated SMS text message strategy to engage users in tracking and reporting SSB consumption behaviors; and an electronic cellular-enabled scale for in-home weighing [36]. At the end of cores 2 to 6, the users evaluate their SSB and weight diary data and set a personalized SSB consumption and weight action plan for the upcoming week. After the sixth core is completed, there is a recurring maintenance core where users can continue behavioral tracking and personalized action planning. To assist with mastering key content and behavioral strategies, the cores also include PDF resources that users can view or print. iSIPsmarter is highly interactive and contains a media-rich format of text, audio, graphics, animation, interaction, and video. Vignettes are woven throughout the intervention, which include stories (based on the experiences of past SIPsmartER participants) to model and describe situations related to setting goals, behavior changes, resolving barriers, and encountering slips. Finally, iSIPsmarter includes a stepped care engagement strategy (ie, human-supported text messages, followed by phone calls if needed) to provide encouragement, technical assistance, and strategies to promote core completion. The adaptation processes, HCD approach, and findings leading to the final content and structure of iSIPsmarter are further detailed in the subsequent sections.

Areas of Adaptation and the Digital Health Intervention Development Process

The Model for Internet Interventions highlights 8 areas to consider when developing digital behavioral health interventions (Figure 1) [4]. These areas are defined and illustrated within the adaptation context for iSIPsmarter.

Content and Messages

Content refers to the actual intervention information and may be the single most important component of the program [4]. Focus is placed on the style and source of messages, which are theorized to impact user engagement and other mechanisms of change, including knowledge acquisition and motivation. [4]. Similar to SIPsmartER, iSIPsmarter is grounded in the Theory of Planned Behavior and health literacy, numeracy, and media literacy concepts. In addition, the scientifically grounded core content, key learning objectives, and behavior change techniques of the digital iSIPsmarter intervention are similar to those of the original evidence-based SIPsmartER intervention [36,37]. The main difference between the two is the addition of evidence-based weight self-monitoring and weight-related strategies to iSIPsmarter. Therefore, the development process focused heavily on weight-related content and messages. In addition, all iSIPsmarter content was written using clear communication strategies and with the goal of achieving <eighth grade reading level. Specifically, all user-facing core content pages were entered into the Readable website to determine the readability score using the validated Flesch-Kincaid Grade Level formula, which considers sentence length and word length [59]. iSIPsmarter passages were revised in up to 2 rounds to improve readability (ie, reduce sentence length and reduce multisyllabic words). This process resulted in 90% of the iSIPsmarter core content pages meeting the criteria of <eighth grade level.

Delivery and Participation

Delivery refers to the form in which the intervention content is disseminated and includes text, audio, illustrations or graphics, animations, video, and vignettes or stories or testimonials [4]. Participation is focused on the program’s ability to engage and involve the user in the intervention [4]. Within each iSIPsmarter core, videos and interactions were developed using gamification principles to encourage the users to engage in learning and practicing key behavioral strategies and techniques [60]. The intervention content prioritized both interactions focused on skill development and personalized reinforcement to drive behavior change (refer to Table 1 for details). To replicate the role modeling and observational learning that occurred during the SIPsmartER small-group classes, personas and vignettes were developed for iSIPsmarter. Personas are defined as user archetypes that represent the characteristics of future users or actual people from a targeted group [14]. Vignettes are narrative stories that illustrate key situations and real-life scenarios faced by individuals and problem behaviors that the intervention aims to improve. The development of personas and vignettes was a multistep and iterative process. First, based on the CDT’s experience with past SIPsmartER participants and intended users of the digital iSIPsmarter intervention, 9 personas were identified and developed to represent the range of traits and SSB consumption change patterns that the CDT witnessed among SIPsmartER participants. These personas detailed numerous user characteristics, such as (1) demographics and family or social characteristics, (2) SSB consumption change patterns and barriers, (3) weight-related patterns and barriers, (4) motivation level and perceived behavioral control, and (5) the use of planned digital technology components. Then, journey maps were created to share with the advisory team participants. These journey maps were single-page snapshots modeling each persona’s potential path through iSIPsmarter, including experiences, successes, and challenges with improving SSB consumption and weight behaviors. Feedback was solicited from the advisory team participants. On the basis of the participants’ insights, the personas were narrowed down and further refined. Next, vignettes (ie, narrative stories) were developed for each persona to model and describe situations related to setting goals, behavior changes, resolving barriers, and encountering slips. Finally, vignettes were mapped to key content through each of the iSIPsmarter cores and several of the interactions.

Table 1. Final iSIPsmarter overview: asset summary, user objectives, interactions or video descriptions, printable PDFs, and user summaries.
CoreOverview and number of assetsUser objectivesInteractions or videos description
Core 1: getting ready
  • Core content screens: 26
  • Action planning screens: N/Aa
  • Interactions: 4
  • Videos: 3
  • Vignettes: 16
  • Printable documents: 3
  • See how iSIPsmarter works and what to expect
  • List my personal reasons for joining iSIPsmarter
  • Discover what counts as a sugary drink
  • Recognize portion sizes of sugary drinks
  • Recall my typical sugary drink patterns
  • Track my sugary drinks and weight for my iSIPsmarter Diaries
  • Interactive questions to raise awareness on SSBb availability, SSB costs, and the amount of sugar in SBBs
  • Interactive content to show successful results from the previous SIPsmartER intervention
  • Sorting game to recognize and practice what counts as an SSB
  • Sorting game to recognize the portion size of SSBs
  • Three short videos to highlight the importance of tracking SSB consumption, the importance of tracking weight, and how to use iSIPsmarter to track SSB consumption and weight
Core 2: making a plan
  • Core content screens: 29
  • Action planning screens: 22
  • Interactions: 4
  • Videos: 1
  • Vignettes: 12
  • Printable documents: 8
  • View the recommendations for sugary drinks
  • Recognize the health risks of too many sugary drinks
  • Explore red-light, yellow-light, and green-light drink categories
  • See the health benefits of non-sugary drinks
  • Evaluate my weight and see a healthy weight range
  • Set a personal Action Plan to help meet my sugary drink and weight goals
  • Interactivity to rate the importance of and confidence in decreasing SSB consumption, with personalized feedback
  • Interactive game to realize the amount of sugar in SSBs and equivalent sugar packets per day
  • Interactive body map to recognize the health risks and key health facts associated with the consumption of too many SSBs
  • Video to learn about how the consumption of too many SSBs impacts the body and leads to health risks over time
  • Interactivity to illustrate the connection between the consumption of SSBs and weight over time
Core 3: using numbers
  • Core content screens: 23
  • Action planning screens: 8-12
  • Interactions: 1
  • Videos: 1
  • Vignettes: 9
  • Printable PDFs: 3
  • Recognize my calorie and energy balance needs
  • Identify my limits on added sugars
  • Apply information from food labels to identify sugary drinks
  • Set a personal Action Plan to help meet my sugary drink and weight goals
  • Video to highlight the key components and application of the nutrition facts label, which includes grams of added sugars, servings per container, serving size, and ingredient list
  • Four-part interactivity to apply one\'s skills in reading nutrition labels, identifying different names for sugars, and sorting drinks using the nutrition facts label and to learn tips that can be applied when a drink does not have a nutrition facts label
Core 4: balancing choices
  • Core content screens: 33
  • Action planning screens: 8-12
  • Interactions: 1
  • Videos: 0
  • Vignettes: 9
  • Printable documents: 6
  • Explore red-light, yellow-light and green-light food categories
  • See how red-light drinks and foods can create imbalance in my diet
  • Recognize how to plan ahead to balance my choices
  • Practice reducing my red-light drink and food choices
  • Discover the health benefits of physical activity
  • Set a personal Action Plan to help meet my sugary drink and weight goals
  • Two-part interactivity to identify red-light foods that are consumed often and select red-light foods that can be removed, replaced, or reduced
Core 5: thinking critically
  • Core content screens: 26
  • Action planning screens: 8-12
  • Interactions: 2
  • Videos: 0
  • Vignettes: 10
  • Printable documents: 2
  • Recognize my sugary drink choices may be swayed by marketing
  • Identify common marketing tactics used to sell sugary drinks
  • Discover the goal of sugary drink companies is to make me spend money
  • Evaluate the costs of sugary drinks
  • Analyze, critique, and modify sugary drink ads
  • Set a personal Action Plan to help meet my sugary drink and weight goals
  • Interactivity to recognize that, in the context of SSB marketing, people’s attention is the product
  • SSB commercial with interactive questions and feedback to help the participants see through marketing and think critically about advertisements
Core 6: staying on track
  • Core content screens: 23
  • Action planning screens: 8-12
  • Interactions: 1
  • Videos: 0
  • Vignettes: 12
  • Printable documents: 2
  • Evaluate my sugary drink and weight program goals
  • Learn how to handle slips in my red-light sugary drink and food choices
  • Focus on strategies to prevent prolonged relapses
  • Check my understanding of key iSIPsmarter concepts
  • Set a personal Action Plan to help me stay on track with my changes
  • A 25-item interactive quiz to help the participants review and check their understanding of the key iSIPsmarter concepts in cores 2 to 6

aN/A: not applicable.

bSSB: sugar-sweetened beverage.

Behavioral Prescriptions and Assessments

Behavioral prescriptions instruct the user on how to address the targeted problem. They are designed to increase commitment and boost adherence and may include, for example, behavioral contracts as well as automated and personalized prompts (eg, emails and SMS text messages) [4]. Assessment refers to the ability to measure the needs of the user, personalize the program, and provide tailored content and recommendations [4]. In the case of iSIPsmarter, the personalized action planning process, behavioral monitoring, and personalized feedback loop were transitioned from the original format of small-group classes and IVR calls in SIPsmartER to a fully digital format. The web development team built iSIPsmarter on a proprietary software platform called the Research Infrastructure Containing eHealth (RICE) interventions, developed for building digital health programs. The RICE platform integrates all aspects of a digital intervention for a seamless user experience and research administration. iSIPsmarter provides an opportunity to extend the technological capability of the intervention infrastructure by incorporating SMS text message and sensor integration, specifically, the BodyTrace scale (BodyTrace, Inc). Daily SMS text message prompts are sent to encourage users to report the of ounces of SSBs consumed the previous day. In addition, an electronic cellular-enabled BodyTrace scale is provided for weight data collection. Users are encouraged to step on the scale daily, with a minimum threshold of 3 days per week, to receive personalized feedback. The integration of the internet-based platform, SMS text messages, and cellular-enabled scales created an integrated experience, whereby users can log into the internet-based platform and view their progress through the cores, along with synced personal diary data (SSB consumption and weight) on their iSIPsmarter dashboard. When appropriate, gamification principles are also applied when providing tailored feedback and recommendations (eg, cues when cores are complete and encouraging feedback when SSB and weight diaries are entered) [60].


Burdens are specific to the intervention content and can include problems related to use, such as poor application navigation and program length [4]. In iSIPsmarter, understanding the burden of the users was prioritized when developing features associated with diary tracking, dashboard navigation, the action plan process. Given that goal setting, planning, self-monitoring, and feedback are among the most important behavior change techniques, understanding and resolving potential user burdens with the digital content was especially important. In addition, attention was paid to the overall length of each core and to balancing the amount of content with the projected user fatigue.


Appearance refers to the look and feel of the application and can include the use of color, page or screen layout, organization of content, and screen size [4]. For iSIPsmarter, a graphic designer developed key images and icons with a consistent theme and style. These graphics were applied throughout the intervention, including all core screens and printable documents.

Additional Components of the Digital Health Intervention Development Process

The Model for Internet Interventions (Figure 1) also illustrates other design-related components and elements (eg, support, user characteristics, environment, and digital health intervention use) [4]. Many of these components were considered in the iSIPsmarter development process. As an example, the model describes supports (eg, email, phone, and face-to-face) that can directly impact users’ adherence to digital health applications. During the development and design of iSIPsmarter, interviews were conducted with the advisory team members to assess their understanding and perceptions of program reminders and gather inputs on the timing and content of stepped care messages.

Data Collection


The iSIPsmarter components were developed using a flexible and iterative 3-phased HCD process. This process included semistructured interviews with the advisory team participants. For each interview, the CDT collaboratively developed key objectives and designed interview guides during weekly team development meetings. One of the researchers drafted the initial questions based on the focus area and agreed upon objectives. Subsequently, other team members provided feedback until a final version was agreed upon. All the interview guides used open-ended questions and probes. The interview content was prioritized by the current development activities. Therefore, it was possible to follow an ongoing, iterative, HCD process that matched the steps and pace of the program development. One or 2 researchers led the audio-recorded interviews. The interviews were completed over telephone and via Zoom (Zoom Video Communications Inc). The sessions ranged from 45 to 60 minutes.

Contextual Phase

The contextual phase of the HCD process involves gathering information from the intended users regarding their behaviors and their context and requirements for using the technologies through which the digital health interventions will be implemented [5,16,18]. Given the CDT’s ample experience intervening in SSB consumption among frequent consumers in the targeted region and a deep understanding of many of the components of the Model for Internet Interventions (ie, user characteristics, mechanisms of change, and behavior change) [4], the 4 interviews comprising the contextual phase largely focused on important technology-related factors associated with potential iSIPsmarter use as well as research evaluation components.

The initial advisory team meeting explored participants’ technology ownership and use, internet availability and use, and perceived benefits of and barriers to receiving a digital behavioral health intervention. The 3 subsequent contextual interviews assessed the participants’ understanding and perception of program reminders and gathered feedback on the timing and content of stepped care messages. Likewise, the participants’ experiences with daily weighing on scales and perceptions toward using cellular-enabled scales for a digital health intervention were evaluated. Finally, the participants’ feedback on the research outcome data components that were assessed was solicited via an internet-based survey, telephone calls, and cellular-enabled scales, all of which were completed by the participants in the comfort of their homes.

Prototype Testing Phase

Prototype testing includes the visual representation of the to-be-developed technology, where the CDT explored different concepts and possible solutions with the intended users. Testable prototypes can take many forms, such as paper prototypes, mock-ups, and wireframes (eg, a skeletal framework of an interface, usually a website or other applications) [5,16,18]. Prototype testing is a critical phase for iteratively addressing and responding to user feedback across the suggested areas of digital behavioral health interventions (ie, appearance, behavioral prescriptions, burdens, content, delivery, message, participation, and assessment) [4].

The iSIPsmarter prototypes developed in this phase were paper- and web-based sketches that illustrated planned scenarios. The participants were sent links to the web-based prototypes, which included preprogrammed pages that displayed various feedback points as the participants moved through the content. This intentional rapid prototyping allowed the participants to interact with multiple prototype drafts and allowed the researchers to incorporate iterative feedback from the participants. The think-aloud method, which is a common approach to assessing the usability of digital health interventions and involves the participants verbally narrating their thoughts when completing a task related to the prototypes, was used. The researchers guided the process and asked the participants to complete specific tasks. Open-ended questions allowed for a robust understanding of usability and functionality. The interviewers documented where the participants encountered problems and difficulties using the prototypes.

This phase included 6 interviews to assess the participants’ comprehension and experience and the overall prototype functionality. The prototypes largely focused on the design and content of the iSIPsmarter user dashboard, daily SSB diary, action planning, and vignettes. The prototypes of the iSIPsmarter user dashboard and daily SSB diary were shown to the participants multiple times. The participants were asked to interact with the dashboard prototypes to collect data on their comprehension of the dashboard interface. This included navigating the dashboard to start a core (lesson) and add a daily SSB diary. Diary prototypes were also tested to collect data on the users’ understanding of adding daily SSB ounces. In addition, the participants were presented with SMS text message screenshots to assess their comprehension of daily SSB consumption tracking features. Similarly, prototypes of SSB and weight action plans were presented to the users to inform the development of a key intervention behavior change technique, personalized goal setting. Moreover, as previously mentioned, the journey maps were shared with the advisory team participants, and feedback was solicited to help narrow down, refine, and develop the vignettes associated with the personas.

End User Testing Phase

In this phase, the advanced functioning prototype or beta versions of the application are exposed to the end users and evaluated [16,18]. This phase can be carried out in a controlled laboratory setting, yet it is typically more useful when the intervention is field tested in the user’s own environment.

For iSIPsmarter, once the prototypes transitioned into the web-based RICE platform, the program-enabled website underwent numerous rounds of internal review by the CDT for clarity of content, flow, and transitions and for addressing programmatic bugs. Subsequently, the end user testing phase began. The advisory team participants were given access to the iSIPsmarter website in their own environment. They completed the cores, including the embedded interactions, videos, and vignettes. Furthermore, they tracked their SSB intake and weight using the SMS text message feature and electronic cellular-enabled scale, respectively. A total of 3 semistructured interviews explored the usability and functionality of and user experience and satisfaction with the iSIPsmarter website.

Data Analysis

After each round of interviews, the researchers who conducted the interviews reviewed the audio transcripts and created interview summary documents that summarized each participant’s response to each question. Following each interview series, these summary documents with screenshots were shared with the CDT to incorporate feedback into the ongoing intervention content and programmatic development.

The analysis and data interpretation process involved several steps [61,62]. First, the 2 researchers who conducted the interviews reviewed the summary documents and independently identified key takeaway statements for each of the 13 interview rounds. Then, they met to build consensus and finalize the interview-level takeaway statements. Second, 2 researchers independently examined and deductively summarized the interview-level takeaway statements as higher-level findings and incorporated them into all 3 HCD phases, as aligned with the Model for Internet Interventions. During this second step, they also used CDT meeting minutes and artifacts from the development process (eg, prototype versions and drafts of core content) to inform higher-level findings. Again, the 2 researchers met to build consensus on the overarching phase-level findings. Finally, to illustrate the phase-level findings, key quotes from the advisory team members were extracted from the transcripts.

Multimedia Appendix 1 illustrates the goals of the 13 semistructured advisory team interviews along with key interview-level takeaways. The phase-level findings, as aligned with the Model for Internet Interventions, are further summarized below.

Contextual Phase

Interviews 1, 3, 11, and 12 focused on the contextual and technology-related aspects of intervention delivery and data collection. The following key findings emerged:

  • To promote participation, the participants described the need for built-in flexibility to account for varying levels of internet and SMS text message accessibility (digital health intervention use).
  • The participants reported the need for accountability and personalized assessments.
  • Plans for stepped care contacts were viewed as a helpful and important intervention component (support) that could boost adherence and core completion (behavioral prescription).
  • The participants saw value in a weight monitoring component to promote reinforcement (participation) and personalization (assessment), yet barriers to weighing varied based on user characteristics and environmental factors, such as limited cellular service to transmit weights.
  • The intervention enrollment procedures were easy to understand and complete.

These findings are illustrated by several key quotes from the advisory team members. One of the members expressed that they face difficulty in responding to SMS text messages owing to a lack of signal but that they would be able to email:

...I can receive her message but...I couldn’t answer her as prompt[ly] as I should or whatever. I’d have to find a place where I could get enough signal to send her back a reply. But with my home computer, if I get on there and have an email that I need to check, I can go ahead, you know, and go through and do it.

In reference to monitoring weight on the iSIPsmarter dashboard, one of the members said the following:

I think it’s a good cut back on your sugary drinks...if you’re able to see your weight up there.

Prototype Testing Phase

The prototype testing phase allowed the CDT to iteratively address and respond to the user feedback. Interviews 2 and 8 focused on the dashboard and diaries; interviews 5, 6, and 7 were dedicated to the action planning concepts; and interviews 4, 5, and 6 focused on the personas. The final dashboard illustrated in Figure 2 contains a few examples of key intervention components, and a sample action plan is shown in Multimedia Appendix 2. For the persona-focused interview, 9 persona journey maps were shared with the advisory team participants, one of which is illustrated in Figure 3. On the basis of the participants’ insights into and reflections on the journey maps, the 9 initial personas were narrowed down to 6 and then further refined.

The main phase 2 findings are summarized as follows:

  • The participants felt that flexibility in diary tracking methods and resources (eg, SMS text message, web-based tracking, drink cards, and paper diaries) would enable high engagement and participation among the users in tracking their SSB consumption and weight (behavioral prescriptions). For example, the participants who prefer paper diaries could first log their daily diary data via pencil and paper. Then, they could log into their dashboard weekly and back enter their diary data. Alternatively, the participants who prefer the digital methods could respond to the daily SMS text messages or email messages.
  • Through multiple iterations, the dashboard became easy to navigate (low burden), with clear content, and contained helpful tailored user information (assessment), including visual cues to signify the completion of the core and diary tasks (appearance).
  • The action planning process was easy to navigate (low burden); the content and delivery features were clear; a personalized and tailored feedback was perceived to be helpful when setting and monitoring SSB and weight goals (behavioral prescriptions, assessment, and messages).
  • Weight-related messages and content, including barriers and strategies, were relatable and easily understood.
  • The vignettes were perceived as realistic and relatable, indicating an effective delivery approach for conveying key content.

Highlighting different patterns in SSB consumption behaviors among the vignettes was identified as a helpful messaging approach to improve the personalization and tailoring of content.

The following key advisory team member quotes support the findings:

In reference to the ability to navigate the action planning process, one of the members said the following:

Everything else was pretty easy to understand. It was straightforward. It wasn’t lengthy as far as, like, a whole bunch of text that you needed to read. It provided really good examples for people to go by.

In reference to the personas being realistic and relatable, the following was said:

Kim because...She’s a single mom working full time. Busy schedule. Kind of just drink most of the day because she was too busy to eat...That was basically my story. So,...I can totally relate to that one. Wanted to be a healthy role model for her kids.
Figure 2. Final dashboard screenshot.
Figure 3. Example journey map shared with the advisory team participants.

End Users Testing Phase

Interviews 9, 10, and 13 focused on the end user testing. Table 1 highlights the final iSIPsmarter overview that was evaluated in this phase, including the asset summary, the user objectives, and a description of interactions or videos. The cores and interactions were developed and programmed in a manner that allows the users to go back and review the content and repeat the interactions to master skill development. Related to action planning, the underlying programmatic structure in cores 3 to 6 and the maintenance core are identical. However, the personalized feedback loops change based on the user’s diary data, progress toward their prior goals, and their new goals. The final SSB message bank includes 14 barriers and 80 strategies, and the weight-related message bank includes 13 barriers and 115 strategies. Therefore, each time a user completes an action plan, the personalized feedback has the potential to look very different (Multimedia Appendix 2).

On the basis of phase 2 findings, the CDT fully developed 6 vignettes and mapped key content through each of the iSIPsmarter cores and several interactions. Table 2 illustrates a final vignette, along with several examples of how the vignette is integrated across the cores.

The following phase 3 findings emerged when the participants were allowed to access the programmed intervention in their own environment:

  • The overall appearance of iSIPsmarter was well received, including layout, organization, iconography, graphics, visuals, and color use.
  • The participants reported high satisfaction with how the messages and content were delivered, including interactions, animations, videos, vignettes, and illustrations.
  • Few burdens were reported, as the participants found the cores to be enjoyable, easy to navigate, user friendly, not overly text heavy, and of an acceptable length. Minor bugs and glitches were identified and resolved before launching the intervention.
  • Participation was enhanced by the built-in flexibility for diary tracking.
  • The behavioral prescriptions and personalization of assessments were well received, particularly the action planning process and automated emails and SMS text messages for reminding the participants to track their diaries.

A few key quotes from the advisory team member have been illustrated to support these findings. In reference to the interaction aimed at teaching and reinforcing the amount of sugar in drinks, one of the members said the following:

When I clicked on...the container that you drink out of the most...and it showed you the little packets of sugar out there. I was like, Oh my gosh, when I seen that it really hit and I thought, No way do you need that.

In reference to the personalized feedback, the following was expressed:

[I]t’s always good to visualize because, you know, it’s hard to kind of think back to what I did you know last Thursday. But on something like this where I can see, OK, I had 12 ounces as last Thursday, I can kind of pull back in memory of the exactly OK, what I did those days that either led me to only have this amount or led me to end up drinking this amount. So it...put those numbers into context a lot better than it would if you just said is average, this is your high this your low.

In reference to the overall iSIPsmarter user experience, the following was said:

...[T]he website itself—that’s the thing that stood out the most to me. It’s well designed. It’s easy to navigate. It’s not too complicated. Everything’s well-organized...Everything is explained well, and it’s simple and concise and to the point. I like the level of the language that’s used. There’s not a whole lot of like jargon or extensive terminology or anything like that that might shy people away from participating. I think the readability of it is really well.
Table 2. An example vignette summary from iSIPsmarter.
Name and imageProfileExample stories throughout the cores
  • Beth is a 53-year-old married homemaker and part-time church secretary. She has two grown kids who live away from home. She cares for her elderly father
  • Sugary drink pattern: Total ounces=64 ounces per day
  • 16 ounces frappe, three 16 ounces glasses of sweet tea throughout the day
  • Reasons for drinking sugary drinks: likes the taste; enjoys making sweet tea for family and friends; it’s a stress reliever
  • Physical activity: sedentary-has a gym membership but rarely goes; wants to be more active but has a hard time fitting it all in while caring for her father
  • Health status: just told she has diabetes and has gained 20 pounds in a year
  • Reason for joining iSIPsmarter: wants to manage the diabetes so she can be healthy, travel with her husband and take care of her family
  • Quote: “It’s just part of who I am”
  • Core 1—Texting Makes Tracking Easy: The daily text makes tracking my sugary drinks easy. I simply respond to the text with my total sugary drink ounces from yesterday. Also, I like seeing everything in one place when I sign in to iSIPsmarter.
  • Core 2—Getting My Diabetes Under Control: I was shocked when my doctor told me I had diabetes. My doctor told me getting to a healthier weight and making better food and drink choices were important to manage my diabetes. I started thinking about my health more. I know I have to take care of myself to keep taking care of my family. I saw that I drank too many sugary drinks and learned its impact on my diabetes and weight. I knew cutting back on sugary drinks was going to be hard. Sweet tea and my frappes were just part of my routine.
  • Core 3—Getting Help from Others to Stay Accountable: I work part time for my church, and the staff often goes out to eat together. When eating out with my coworkers, it was harder to stick to my sugary drink goals. I told my co-worker Mary about iSIPsmarter and my goals and asked her to help me. Getting her support, helped me stay accountable.
  • Core 5—Could be Saving Money for College Tuition: My husband and I are helping our son with college costs so he doesn’t finish school with too many student loans. We are always looking for ways to save money. The frappes I was getting every day were $5 each day or $1,820 a year! This amount of money will definitely help cover some of his college costs.

Principal Findings

This paper presents a flexible, step-by-step approach to and an HCD process for developing digital behavioral health interventions using iSIPsmarter as an illustrative example. By applying the Model for Internet Interventions [4] as well as best practices in HCD [16] and instructional design processes [17-20], we have been able to clarify assumptions and address key areas of the iSIPsmarter intervention that were hypothesized to support participants’ engagement and promote behavior change. As evidenced by the rich feedback received from the advisory team members, the human-centered methodology was instrumental in our development process. Likewise, the value of our robust process is exemplified by our resulting user-informed, high-quality products (eg, iSIPsmarter core components, vignettes, dashboard, and personalized action plan). Our approach can be interpreted within the context of the findings of recent narrative scoping review by Kip et al [14], in which a variety of methods (eg, interviews, focus groups, questionnaires, card sorting, and usability testing) and products (eg, prototypes, personas, and behavior change strategies) used for participatory eHealth development processes have been highlighted. Similar to the conclusions of Kip et al [14], our approach illustrates the importance of researchers and developers using the most appropriate methods to match their objectives and user characteristic context.

In terms of the iSIPsmarter development process, some of the biggest challenges faced when transitioning from the original group class structure to the digital structure include (1) replicating engagement provided by participant-to-participant and participant-to-instructor communication and relationships; (2) balancing the demands and cognitive load of the behavior change content (especially planning for the potential low health literacy skills of our targeted users) and trying to mimic visual, experiential, and hands-on class-based activities; (3) building flexibility to account for varying levels of internet and SMS text message accessibility among users; (4) creating a seamless user experience by integrating internet, SMS text messages, and sensor information; (5) operationalizing all aspects of the intervention content for digital delivery; and (6) automating all elements while ensuring that all permutations were considered. Similar challenges have been highlighted in a few other digital behavioral health intervention adaptation papers [14]. However, these issues are often overlooked and underreported in the literature, which may partly explain the low uptake of digital behavioral health interventions as well as the suboptimal fit between the characteristics of the technology and user needs. Simply converting evidence-based content traditionally delivered face-to-face to a web-based or digital format, without adequate attention to these and other challenges, is an insufficient approach to engaging the intended users or promoting improved health outcomes. Researchers and developers should rely on the established models and frameworks, such as the Model for Internet Interventions [4-9], to help anticipate and guide key decisions when embarking on the development of digital behavioral health interventions.

Although several behavioral interventions targeting SSB consumption reduction among adults have been developed and evaluated [30], only a few have used scalable digital approaches. Moreover, only one other known adult-focused digital intervention targeting SSB consumption reduction has applied user-centered methodologies in the formative phases of intervention development [31]. Similar to our study, the study by Tonkin et al [31] focused on disadvantaged and nonurban adults, and its findings revealed the importance of understanding the available technology and patterns of its use as well as participants’ preference for stories, role modeling, and gamification, which foster engagement with the intervention. In addition, digital weight self-monitoring has become a cornerstone of many weight-related behavioral interventions, and greater adherence to self-monitoring is associated with better outcomes [63-65]. Unfortunately, consistency and disengagement in digital weight self-monitoring are known to be problematic [66]. Although several studies have investigated experiences of self-monitoring at the conclusion of interventions [67,68], there is a dearth of published studies that have applied HCD processes in formative intervention development stages to understand and build-in behavioral strategies to address potential personal and environmental barriers to digital self-monitoring [69].

On the basis of our experiences with iSIPsmarter, we offer 6 broad considerations for other teams developing or adapting digital behavioral health interventions (Textbox 1).

In terms of study implications, the efficacy of iSIPsmarter in reducing SSB consumption in rural Appalachian adults is currently being evaluated in a randomized controlled trial that includes a 2-group design (iSIPsmarter vs static Participant Education website) with 4 assessment points (Clinical Trial Registry: NCT05030753) [36]. When efficacy and other summative data are available, they will provide additional insights to inform the potential value of applying an HCD process to build iSIPsmarter as well as identify future areas of study.

Six recommendations for the development or adaptation of digital behavioral health interventions.
  1. Assemble a multidisciplinary team of experts and end users: similar to our iSIPsmarter experience, the value of multidisciplinary team science and participatory processes in the development of digital behavioral health interventions is largely supported by other studies [15,16,18,35]. Our multidisciplinary content development team brought together expertise in the areas of nutrition content, behavior change, and rural health and worked alongside experts in digital behavioral health interventions, software engineering, instructional design, and user-interface design. Likewise, involving advisory team members with lived experiences in the targeted Appalachia region and with previous involvement in the SIPsmarter intervention brought immense value to the adaptation process. By applying human-centered design (HCD) principles [16], the advisory team members critically responded to iterative prototype versions, which helped shape key intervention decisions.
  2. Support efficient communication and decision-making processes among teams: anticipating diverse feedback among different stakeholders, coordinating efficient communication among subteams, and finding a compromise are imperative to efficiently advance the HCD process [16,18]. For example, in our study, we coordinated communication and cooperation among 3 different subteams (ie, content, technology, and advisory teams) working to develop and advance iSIPsmarter. In some instances, the advisory team requirements were different and contradictory to one another and to the requirements of the content development team.
  3. Define areas of adaptation at the onset of the process: Similar to the adaptation of any behavioral intervention, the adaptation of digital behavioral health interventions can be driven by several distinct purposes. For example, the Adaptome [70] describes 5 potential sources of intervention adaptations: core components, culture, mode of delivery, target audience, and service settings. We adapted iSIPsmarter from the evidence-based SIP smart ER trial with the clear goals of preserving the core components, cultural aspects, and the intended rural Appalachia target audience. This allowed us to concentrate on the mode of delivery and add key content around weight management. For example, we were able to focus end user feedback on potential digital divide concerns in Appalachia [21-26]. This feedback guided us to build iSIPsmarter with flexible features intended to enhance the likelihood of engaging in web-based cores, SMS text messages, and in-home weighing using cellular-enabled scales.
  4. Apply the available models and frameworks to guide digital intervention development processes: Despite the promise and increased availability of digital behavioral health interventions, rapid disengagement and small effect sizes remain problematic [3]. Although evidence-based behavioral content is essential, it is only one of a multitude of factors that must be considered when developing or adapting digital behavioral health interventions. By applying the available models and frameworks to guide digital intervention development processes, researchers can identify and operationalize comprehensive components that affect the engagement and impact of digital behavioral health interventions [4-9].
  5. Clearly define instructional design goals to guide the HCD process: To improve the likelihood of digital health applications meeting the needs and requirements of the users, instructional design objectives (eg, learning, affective, cognition, or psychomotor objectives) should be applied in the HCD process [15,16]. For example, in the iSIPsmarter interviews, we were interested in evaluating the participants’ knowledge, problem-solving skills, attitudes, and values associated with completing certain tasks (eg, navigating the dashboard, completing an action plan, and engaging with an interaction). By having clear instructional design objectives, we were able to better understand and modify iSIPsmarter features to meet the needs of the users and support the achievement of improved sugar-sweetened beverage consumption behaviors.
  6. Allocate appropriate resources and time to successfully execute HCD processes: The time and resources required to develop or adapt digital health interventions can vary widely; however, they are often underestimated. For example, our iterative 3-phased HCD process was nearly an 18-month process. Researchers and developers should carefully consider and anticipate investment in robust HCD processes, as these processes are likely critical to the long-term uptake, engagement, and impacts of most digital health interventions.
Textbox 1. Six recommendations for the development or adaptation of digital behavioral health interventions.


Overall, 3 key limitations of this study should be considered. First, generalizability may be limited by the relatively small sample of advisory team members, the targeted rural Appalachia region, and SSB-specific content. Second, qualitative interviews can result in socially desirable responses. However, efforts were made by our trained interviewers to minimize this potential limitation by probing and clarifying the participant responses. This approach resulted in the participants providing thoughtful and constructive critiques of iSIPsmarter. Third, although we applied the Model for Internet Interventions, we limited the focus and scope of our manuscript to highlight the 8 main areas that comprise digital health applications [4]. Future efforts could focus more broadly on describing the other design-related components and elements that guided the iSIPsmarter adaptation process. Despite these potential limitations, we hope that the illustrated processes, scientific frameworks, and context-specific instructional design methodology will have wide applications in the development of digital behavioral health interventions across multiple geographic and behavioral contexts.


Our process emphasizes the value of researchers and developers applying the existing models and frameworks as well as best practices in HCD and instructional design processes in digital intervention development processes. Together, the complementary skills of the CDT and advisory team members were invaluable in the iSIPsmarter adaptation process. The importance of the contextual and iterative prototype testing phases was largely reinforced by the overwhelming positive feedback received in the user testing phase. By illustrating iSIPsmarter content, we have highlighted the user-informed, high-quality products that resulted from our robust HCD process.


The authors would like to acknowledge the time and dedication of our advisory team participants as well as members of our technology team, including Gabe Heath, Nicole Le, Ella Hoeppner, Ian Terrell, Alan Lattimore, Brendan Murphy, and Rom DuPlain. This study was funded by the National Institutes of Health, National Institute on Minority Health and Health Disparities (R01MD015033). The National Institutes of Health was not involved in the design of this study or writing of this manuscript.

Authors' Contributions

JZ, LR, and KP were responsible for the acquisition of funding. JZ, LR, AR, CF, KP, and MH were engaged in all aspects of the 3-phased development process. AR and CF led the advisory team interviews and initial data coding. JZ and AR prepared the first draft of the manuscript. All the authors critically reviewed, edited, and approved the final version of the manuscript.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Goals and outcomes of the 13-series user-centered design process.

DOCX File , 33 KB

Multimedia Appendix 2

Example of a user’s action plan.

PNG File , 541 KB

  1. Baumel A, Yom-Tov E. Predicting user adherence to behavioral eHealth interventions in the real world: examining which aspects of intervention design matter most. Transl Behav Med 2018 Sep 08;8(5):793-798. [CrossRef] [Medline]
  2. Ritterband LM, Thorndike FP, Morin CM, Gerwien R, Enman NM, Xiong R, et al. Real-world evidence from users of a behavioral digital therapeutic for chronic insomnia. Behav Res Ther 2022 Jun;153:104084 [FREE Full text] [CrossRef] [Medline]
  3. Szinay D, Jones A, Chadborn T, Brown J, Naughton F. Influences on the uptake of and engagement with health and well-being smartphone apps: systematic review. J Med Internet Res 2020 May 29;22(5):e17572 [FREE Full text] [CrossRef] [Medline]
  4. Ritterband LM, Thorndike FP, Cox DJ, Kovatchev BP, Gonder-Frederick LA. A behavior change model for internet interventions. Ann Behav Med 2009 Aug;38(1):18-27 [FREE Full text] [CrossRef] [Medline]
  5. van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res 2011 Dec 05;13(4):e111 [FREE Full text] [CrossRef] [Medline]
  6. Mohr DC, Lyon AR, Lattie EG, Reddy M, Schueller SM. Accelerating digital mental health research from early design and creation to successful implementation and sustainment. J Med Internet Res 2017 May 10;19(5):e153 [FREE Full text] [CrossRef] [Medline]
  7. Yardley L, Morrison L, Bradbury K, Muller I. The person-based approach to intervention development: application to digital health-related behavior change interventions. J Med Internet Res 2015 Jan 30;17(1):e30 [FREE Full text] [CrossRef] [Medline]
  8. Oinas-Kukkonen H, Harjumaa M. Persuasive systems design: key issues, process model, and system features. Commun Assoc Inform Syst 2009;24. [CrossRef]
  9. Hekler EB, Klasnja P, Riley WT, Buman MP, Huberty J, Rivera DE, et al. Agile science: creating useful products for behavior change in the real world. Transl Behav Med 2016 Jun;6(2):317-328 [FREE Full text] [CrossRef] [Medline]
  10. Smith A, Bamgboje-Ayodele A, Butow P, Klein B, Turner J, Sharpe L, iConquerFear Community Advisory Group, et al. Development and usability evaluation of an online self-management intervention for fear of cancer recurrence (iConquerFear). Psychooncology 2020 Jan 10;29(1):98-106. [CrossRef] [Medline]
  11. Draucker CB, Martsolf DS, Crane S, Romero L, McCord AL. Designing an internet intervention for emerging adults who experience troubled relationships. Arch Psychiatr Nurs 2017 Jun;31(3):296-301 [FREE Full text] [CrossRef] [Medline]
  12. Ritterband LM, Thorndike FP, Ingersoll KS, Lord HR, Gonder-Frederick L, Frederick C, et al. Effect of a web-based cognitive behavior therapy for insomnia intervention with 1-year follow-up: a randomized clinical trial. JAMA Psychiatry 2017 Jan 01;74(1):68-75. [CrossRef] [Medline]
  13. Manne S, Heckman C, Kashy D, Ritterband LM, Thorndike FP, Lozada C, et al. Randomized controlled trial of the mySmartSkin web-based intervention to promote skin self-examination and sun protection among individuals diagnosed with melanoma. Transl Behav Med 2021 Jul 29;11(7):1461-1472 [FREE Full text] [CrossRef] [Medline]
  14. Kip H, Keizer J, da Silva MC, Beerlage-de Jong N, Köhle N, Kelders SM. Methods for human-centered eHealth development: narrative scoping review. J Med Internet Res 2022 Jan 27;24(1):e31858 [FREE Full text] [CrossRef] [Medline]
  15. Hilgart MM, Ritterband LM, Thorndike FP, Kinzie MB. Using instructional design process to improve design and development of Internet interventions. J Med Internet Res 2012 Jun 28;14(3):e89 [FREE Full text] [CrossRef] [Medline]
  16. British Standards Document BS EN ISO 9241-210 Ergonomics of human-system interaction. Human-centred design for interactive systems. BSI. 2019.   URL: [accessed 2022-06-13]
  17. Kinzie M, Cohn W, Julian M, Knaus W. A user-centered model for web site design: needs assessment, user interface design, and rapid prototyping. J Am Med Inform Assoc 2002;9(4):320-330 [FREE Full text] [CrossRef] [Medline]
  18. Harte R, Glynn L, Rodríguez-Molinero A, Baker PM, Scharf T, Quinlan LR, et al. A human-centered design methodology to enhance the usability, human factors, and user experience of connected health systems: a three-phase methodology. JMIR Hum Factors 2017 Mar 16;4(1):e8 [FREE Full text] [CrossRef] [Medline]
  19. Lewis C. Using the "thinking Aloud" Method in Cognitive Interface Design. New York City, Albany, New York: IBM Thomas J. Watson Research Division; 1982.
  20. Nielsen J. Thinking aloud: the #1 usability tool. Nielsen Normal Group. 2012 Jan 15.   URL: [accessed 2021-02-22]
  21. Behringer B, Friedell G. Appalachia: where place matters in health. Prev Chronic Dis 2006 Oct;3(4):1 [FREE Full text]
  22. Marshall J, Thomas L, Lane N, Holmes M, Arcury T, Randolph R, et al. Health disparities in appalachia. Appalachian Regional Commission.   URL: [accessed 2019-04-16]
  23. Vogels E. Some digital divides persist between rural, urban and suburban America. Pew Research Center.   URL: https:/​/www.​​fact-tank/​2021/​08/​19/​some-digital-divides-persist-between-rural-urban-and-suburban-america/​ [accessed 2019-04-29]
  24. Middleton G, Henderson H, Evans D. Implementing a community-based obesity prevention programme: experiences of stakeholders in the north east of England. Health Promot Int 2014 Jun 07;29(2):201-211. [CrossRef] [Medline]
  25. Internet Service Providers in Virginia. BroadbandNow.   URL: [accessed 2019-04-29]
  26. Commonwealth Connection homepage. Commonwealth Connection.   URL: [accessed 2022-07-28]
  27. Kit B, Fakhouri T, Park S, Nielsen S, Ogden C. Trends in sugar-sweetened beverage consumption among youth and adults in the United States: 1999-2010. Am J Clin Nutr 2013 Jul;98(1):180-188. [CrossRef] [Medline]
  28. Yuhas M, Hedrick V, Zoellner J. Consumption of added sugars by rural residents of Southwest Virginia. J Appalach Health 2020;2(3):53-68 [FREE Full text] [CrossRef] [Medline]
  29. Hedrick VE, Davy BM, You W, Porter KJ, Estabrooks PA, Zoellner JM. Dietary quality changes in response to a sugar-sweetened beverage-reduction intervention: results from the Talking Health randomized controlled clinical trial. Am J Clin Nutr 2017 Apr 01;105(4):824-833 [FREE Full text] [CrossRef] [Medline]
  30. Vargas-Garcia E, Evans C, Prestwich A, Sykes-Muskett B, Hooson J, Cade J. Interventions to reduce consumption of sugar-sweetened beverages or increase water intake: evidence from a systematic review and meta-analysis. Obes Rev 2017 Nov 18;18(11):1350-1363 [FREE Full text] [CrossRef] [Medline]
  31. Tonkin E, Jeffs L, Wycherley TP, Maher C, Smith R, Hart J, et al. A smartphone app to reduce sugar-sweetened beverage consumption among young adults in Australian remote indigenous communities: design, formative evaluation and user-testing. JMIR Mhealth Uhealth 2017 Dec 12;5(12):e192 [FREE Full text] [CrossRef] [Medline]
  32. Beun RJ, Luiten C, Verbeek C, Poelman MP. A rationale for a gamified e-coach application to decrease the consumption of sugar sweetened beverages. Front Digit Health 2020 Jan 18;2:564529 [FREE Full text] [CrossRef] [Medline]
  33. Yuhas M, Porter KJ, Brock DP, Loyd A, McCormick BA, Zoellner JM. Development and pilot testing of text messages to help reduce sugar-sweetened beverage intake among rural caregivers and adolescents: mixed methods study. JMIR Mhealth Uhealth 2019 Jul 30;7(7):e14785 [FREE Full text] [CrossRef] [Medline]
  34. University of Virginia.   URL: [accessed 2023-01-24]
  35. Ritterband LM, Gonder-Frederick LA, Cox DJ, Clifton AD, West RW, Borowitz SM. Internet interventions: in review, in use, and into the future. Professional Psychol Res Pract 2003 Oct;34(5):527-534. [CrossRef]
  36. Zoellner JM, Porter KJ, You W, Reid AL, Frederick C, Hilgart M, et al. Study protocol for iSIPsmarter: a randomized-controlled trial to evaluate the efficacy, reach, and engagement of a technology-based behavioral intervention to reduce sugary beverages among rural Appalachian adults. Contemp Clin Trials 2021 Nov;110:106566 [FREE Full text] [CrossRef] [Medline]
  37. Zoellner J, Chen Y, Davy B, You W, Hedrick V, Corsi T, et al. Contemp Clin Trials 2014 Jan;37(1):43-57 [FREE Full text] [CrossRef] [Medline]
  38. SIPsmartER. NIH National Cancer Institute.   URL: [accessed 2022-06-28]
  39. Zoellner J, Krzeski E, Harden S, Cook E, Allen K, Estabrooks PA. Qualitative application of the theory of planned behavior to understand beverage consumption behaviors among adults. J Acad Nutr Diet 2012 Nov;112(11):1774-1784 [FREE Full text] [CrossRef] [Medline]
  40. Zoellner J, Estabrooks PA, Davy BM, Chen Y, You W. Exploring the theory of planned behavior to explain sugar-sweetened beverage consumption. J Nutr Educ Behav 2012 Mar;44(2):172-177 [FREE Full text] [CrossRef] [Medline]
  41. Zoellner J, Cook E, Chen Y, You W, Davy B, Estabrooks P. Mixed methods evaluation of a randomized control pilot trial targeting sugar-sweetened beverage behaviors. Open J Prev Med 2013 Feb 01;3(1):51-57 [FREE Full text] [CrossRef] [Medline]
  42. Ajzen I. From intentions to actions: a theory of planned behavior. In: Action Control. Berlin, Heidelberg: Springer; 1985.
  43. Health literacy. Centers for Disease Control and Prevention. 2022.   URL: [accessed 2022-06-28]
  44. Health literacy. Office of Disease Prevention and Health Promotion. 2021.   URL: [accessed 2021-02-28]
  45. Aufderheide P. Part II: Conference Proceedings and Next Steps. Washington DC: Communications and Society Program of the Aspen Institute; 1993.
  46. Golbeck AL, Ahlers-Schmidt CR, Paschal AM, Dismuke SE. A definition and operational framework for health numeracy. Am J Prev Med 2005 Nov;29(4):375-376. [CrossRef] [Medline]
  47. Zoellner JM, Hedrick VE, You W, Chen Y, Davy BM, Porter KJ, et al. Effects of a behavioral and health literacy intervention to reduce sugar-sweetened beverages: a randomized-controlled trial. Int J Behav Nutr Phys Act 2016 Mar 22;13(1):38 [FREE Full text] [CrossRef] [Medline]
  48. Hedrick V, Davy B, Myers E, You W, Zoellner J. Changes in the healthy beverage index in response to an intervention targeting a reduction in sugar-sweetened beverage consumption as compared to an intervention targeting improvements in physical activity: results from the talking health trial. Nutrients 2015 Dec 04;7(12):10168-10178 [FREE Full text] [CrossRef] [Medline]
  49. Davy BM, Jahren AH, Hedrick VE, You W, Zoellner JM. Influence of an intervention targeting a reduction in sugary beverage intake on the δ13C sugar intake biomarker in a predominantly obese, health-disparate sample. Public Health Nutr 2016 Jun 14;20(1):25-29. [CrossRef]
  50. Bailey AN, Porter KJ, Hill JL, Chen Y, Estabrooks PA, Zoellner JM. The impact of health literacy on rural adults' satisfaction with a multi-component intervention to reduce sugar-sweetened beverage intake. Health Educ Res 2016 Aug 11;31(4):492-508. [CrossRef] [Medline]
  51. Porter K, Chen Y, Estabrooks P, Noel L, Bailey A, Zoellner J. Using teach-back to understand participant behavioral self-monitoring skills across health literacy level and behavioral condition. J Nutr Educ Behav 2016 Jan;48(1):20-6.e1. [CrossRef] [Medline]
  52. Zoellner JM, Porter KJ, Chen Y, Hedrick VE, You W, Hickman M, et al. Predicting sugar-sweetened behaviours with theory of planned behaviour constructs: outcome and process results from the SIPsmartER behavioural intervention. Psychol Health 2017 May 06;32(5):509-529. [CrossRef] [Medline]
  53. Estabrooks P, You W, Hedrick V, Reinholt M, Dohm E, Zoellner J. A pragmatic examination of active and passive recruitment methods to improve the reach of community lifestyle programs: the talking health trial. Int J Behav Nutr Phys Act 2017 Jan 19;14(1):7 [FREE Full text] [CrossRef] [Medline]
  54. Zoellner JM, You W, Estabrooks PA, Chen Y, Davy BM, Porter KJ, et al. Supporting maintenance of sugar-sweetened beverage reduction using automated versus live telephone support: findings from a randomized control trial. Int J Behav Nutr Phys Act 2018 Oct 04;15(1):97 [FREE Full text] [CrossRef] [Medline]
  55. Porter KJ, Chen Y, Lane HG, Zoellner JM. Implementation of media production activities in an intervention designed to reduce sugar-sweetened beverage intake among adults. J Nutr Educ Behav 2018 Feb;50(2):173-9.e1. [CrossRef] [Medline]
  56. Porter KJ, Thomson JL, Zoellner JM. Predictors of engagement and outcome achievement in a behavioural intervention targeting sugar-sweetened beverage intake among rural adults. Public Health Nutr 2019 Dec 04;23(3):554-563. [CrossRef]
  57. Zoellner JM, Porter KJ, You W, Estabrooks PA, Perzynski K, Ray PA, et al. The reach and effectiveness of SIPsmartER when implemented by rural public health departments: a pilot dissemination and implementation trial to reduce sugar-sweetened beverages. Transl Behav Med 2020 Aug 07;10(3):676-684. [CrossRef] [Medline]
  58. Porter KJ, Brock DJ, Estabrooks PA, Perzynski KM, Hecht ER, Ray P, et al. SIPsmartER delivered through rural, local health districts: adoption and implementation outcomes. BMC Public Health 2019 Sep 18;19(1):1273 [FREE Full text] [CrossRef] [Medline]
  59. Be Readable. Readable.   URL: [accessed 2022-06-28]
  60. Floryan M, Ritterband L, Chow P. Principles of gamification for internet interventions. Transl Behav Med 2019 Nov 25;9(6):1131-1138. [CrossRef] [Medline]
  61. Watkins DC. Rapid and rigorous qualitative data analysis. Int J Qual Method 2017 Jun 08;16(1):160940691771213. [CrossRef]
  62. Gale RC, Wu J, Erhardt T, Bounthavong M, Reardon CM, Damschroder LJ, et al. Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implement Sci 2019 Feb 01;14(1):11 [FREE Full text] [CrossRef] [Medline]
  63. Burke LE, Wang J, Sevick MA. Self-monitoring in weight loss: a systematic review of the literature. J Am Diet Assoc 2011 Jan;111(1):92-102. [CrossRef] [Medline]
  64. Patel ML, Wakayama LN, Bennett GG. Self-monitoring via digital health in weight loss interventions: a systematic review among adults with overweight or obesity. Obesity (Silver Spring) 2021 Mar;29(3):478-499. [CrossRef] [Medline]
  65. Patel ML, Brooks TL, Bennett GG. Consistent self-monitoring in a commercial app-based intervention for weight loss: results from a randomized trial. J Behav Med 2020 Jun 08;43(3):391-401. [CrossRef] [Medline]
  66. Carpenter CA, Eastman A, Ross KM. Consistency with and disengagement from self-monitoring of weight, dietary intake, and physical activity in a technology-based weight loss program: exploratory study. JMIR Form Res 2022 Feb 18;6(2):e33603 [FREE Full text] [CrossRef] [Medline]
  67. Burke LE, Swigart V, Warziski Turk M, Derro N, Ewing LJ. Experiences of self-monitoring: successes and struggles during treatment for weight loss. Qual Health Res 2009 Jun 13;19(6):815-828. [CrossRef] [Medline]
  68. Hartmann-Boyce J, Boylan A, Jebb SA, Aveyard P. Experiences of self-monitoring in self-directed weight loss and weight loss maintenance: systematic review of qualitative studies. Qual Health Res 2019 Jan 09;29(1):124-134. [CrossRef] [Medline]
  69. Das A, Svanæs D. Human-centred methods in the design of an e-health solution for patients undergoing weight loss treatment. Int J Med Inform 2013 Nov;82(11):1075-1091. [CrossRef] [Medline]
  70. Chambers DA, Norton WE. The Adaptome: advancing the science of intervention adaptation. Am J Prev Med 2016 Oct;51(4 Suppl 2):S124-S131 [FREE Full text] [CrossRef] [Medline]

CDT: content development team
HCD: human-centered design
IVR: interactive voice response
RICE: Research Infrastructure Containing eHealth
SSB: sugar-sweetened beverage

Edited by A Kushniruk; submitted 28.07.22; peer-reviewed by E Radwan, I Gabashvili; comments to author 04.10.22; revised version received 06.12.22; accepted 11.12.22; published 01.02.23


©Jamie Zoellner, Annie Reid, Kathleen Porter, Christina Frederick, Michelle Hilgart, Lee Ritterband. Originally published in JMIR Human Factors (, 01.02.2023.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.