Development and Usability of REACH: A Tailored Theory-Based Text Messaging Intervention for Disadvantaged Adults With Type 2 Diabetes

Introduction

Overview

Currently, at least one in three people will develop type 2 diabetes mellitus (T2DM) in his or her lifetime [1]. People with diabetes are at higher risk of critical health complications including kidney failure, heart disease, and stroke [1]. More than 20% of health care spending in the United States goes toward people with a diagnosis of diabetes [1]. Racial and ethnic minorities are more likely than non-Hispanic whites to have a diagnosis of T2DM [1] and, once diagnosed, have more diabetes-related complications [2], hospitalizations [3], and premature death [4].

People with T2DM can take medication, eat healthily, exercise, and test blood glucose levels to achieve optimal glycemic control [5] and, in turn, prevent diabetes complications [6] and premature mortality [7]. However, the initiation and maintenance of these self-care activities is challenging [8], and rates of self-care adherence are low among adults with T2DM [8,9]. Adherence rates are even lower among racial and ethnic minorities [10-12] and persons of low socioeconomic status (SES) [13] owing in part to financial difficulties and misconceptions about diabetes and self-care (eg, believing they do not have control over their diabetes, believing medication is not important) [8,14-16].

Mobile phone–based interventions using text messaging are a practical approach for improving medication adherence among low-SES, racial and ethnic minorities with T2DM. More than 90% of US adults own a mobile phone [17]. Although smartphones are used less among individuals with diabetes [18], lower income [19], and lower education [19], text messaging does not require a smartphone and is the most common activity among all mobile phone users, used equally across SES, race, and ethnicity strata [20,21].

Text messaging interventions are improving the self-care and glycemic control of adults with diabetes [22-24], but few have been tested with disadvantaged populations in the United States [25]. Two prior text messaging interventions [26,27] improved glycemic control in low-SES samples but not relative to a control group. A third text messaging intervention [28] improved glycemic control among a racially diverse sample but this sample had relatively high SES. Each of these interventions use text messages to address barriers to self-care, but none identifies and addresses each user’s unique barriers. Such barriers vary from person to person with T2DM [29-32], requiring a tailored user experience.

Tailoring text messages to a user’s unique adherence barriers can address issues most applicable to him or her, such as limited diabetes knowledge, negative beliefs about medication (eg, fear of side effects), or limited financial resources [16]. Although interventions cannot easily target a person’s SES, they can enhance one’s problem-solving ability to address financial barriers and other modifiable barriers [14,16]. We developed the MEssaging for Diabetes (MED) intervention that sends tailored text messages addressing user-specific barriers to adherence and text messages assessing adherence [33]. After 3 months of MED among disadvantaged adults with T2DM, users’ barriers were reduced and barrier reduction was associated with improved glycemic control [34]. Furthermore, MED users were highly engaged, responding to 84% of daily assessment messages, and engagement did not differ by sex, race, income, health literacy, or duration of diagnosed diabetes [35]. MED’s findings are consistent with reviews suggesting text messaging interventions with personally relevant, tailored content are more engaging [36] and effective [37] than those without tailored content.

Additionally, there is mixed evidence as to whether theory-based interventions are more effective than atheoretical interventions [38,39]; this is in part due to interventions not extensively applying theory and using theories that are inappropriate for behavior change [38]. The Information-Motivation-Behavioral skills (IMB) model suggests that adherence to a behavior depends on behavior-specific knowledge, personal and social motivation, and behavioral skills [40]. The IMB model is empirically validated among a wide range of diverse samples of adults with T2DM, including samples with low SES [41-43], and explains more than 40% of the variance in their medication adherence [43]. Text messaging interventions for T2DM self-care are rarely based on health behavior theory [24,44,45]. Of the few text messaging interventions tested among disadvantaged populations with T2DM, only 2 mention using a theory-driven approach [27,28]; however, the extent to which theory was applied in these interventions is unclear.

Objective

We developed Rapid Education/Encouragement And Communications for Health (REACH), a tailored, IMB model–based text messaging intervention. We performed 3 rounds of usability testing with adults with T2DM receiving care from Federally Qualified Health Centers (FQHCs) to identify and address any content and functionality issues before evaluating REACH’s effects on self-care and glycemic control.

Methods

REACH Intervention Development

REACH was developed with MEMOTEXT, an algorithmic communications and data management platform supporting personalized user outputs and inputs via short message service (SMS); interventions using this platform have been tested with diverse patient populations with different health conditions who found them acceptable, engaging, and whose adherence improved >30% [46,47]. We worked with MEMOTEXT to develop REACH based on our experience developing and testing MED [33-35].

REACH Content Development

Similar to MED, we created tailored text messages addressing barriers to medication adherence common in our target population; however, REACH addresses more barriers to adherence than MED and barriers map onto the IMB model. To develop REACH content, we first conducted a thorough review of published studies reporting medication adherence barriers among adults with T2DM. In October 2014, we searched for studies in PubMed using terms from each of 3 categories: (1) medication adherence (ie, diabetes medication, medication adherence, medication nonadherence, medication compliance), (2) barriers (ie, barriers, challenges, problems), and (3) diabetes or type 2 diabetes. Terms were intralinked with “OR” and “AND.” There were no restrictions on year of publication. We then searched references cited in eligible articles and articles citing relevant articles by hand. Experts in diabetes medication adherence on our team (authors SK and CYO) ensured our search captured meaningful articles. We reviewed all studies identifying barriers to diabetes medication adherence among adults diagnosed with T2DM and documented the reported barriers and race and ethnicity of the sample. Across 30 studies, we identified 68 barriers to taking medications and 7 barriers to taking insulin. We then sorted and collapsed similar barriers, resulting in 31 medication-related and 5 insulin-specific barriers. Finally, we tagged each of the 36 barriers to the IMB model’s information, motivation, or behavioral skills constructs [40], and content experts drew on identified studies to develop text messages addressing each barrier (Table 1).

Users of MED wanted text messages providing information specific to their prescribed medications. Therefore, the REACH team’s clinical pharmacist and nurse practitioner identified and classified available oral hypoglycemic agents, insulin, and noninsulin injectable drugs (glucagon-like peptide-1 receptor agonists). They then developed regimen-specific text messages on how to handle missed doses, manage medication side effects, administer medication, and store and discard medication for each class of medication.

MED users also recommended adding messages promoting other self-care behaviors (in addition to medication adherence) and inspirational messages [67]. In response, the REACH team’s dietitian/diabetes educator developed text messages with tips promoting adherence to healthful eating, physical activity, and self-monitoring of blood glucose (SMBG). These messages were developed with the goal of providing general diabetes nutrition, exercise, and SMBG statements that are applicable to people with diabetes (vs specific instructions or information that should be determined in a one-on-one consult). Therefore, guidelines for development of these messages were to generate content providing concrete and practical diabetes information applicable to most adults with T2DM. We also developed inspirational text messages to encourage the initiation and maintenance of self-care efforts (eg, “Remember that you have the power every day to make progress toward improving your health!”) and ensured all messages were contextually appropriate (eg, referenced local resources, avoided mention of things such as gym memberships).

After developing all content, the REACH team’s health communication experts reviewed and edited text messages to be readable and understandable (ie, written at the sixth-grade reading level, avoided complex terms and jargon, and health literacy appropriate). Finally, a digital content developer shortened messages (≤160 characters) and ensured consistent tone across messages and appropriateness for digital delivery.

REACH Functionality Development

With the help of MEMOTEXT, we developed functionality to optimize the REACH user experience, making it more personalized and interactive than MED. MED users received a daily text message assessing whether they took their medication that day. Users responded to this message frequently [35] and said it served as a reminder to take their medication [67]. We retained this feature in REACH but made the experience more interactive. For example, if users respond “no,” they receive a follow-up message asking why they did not take their medication with response options to encourage reflection on reasons for nonadherence (Figure 1).

MED users received feedback on their adherence (ie, aggregated responses to daily adherence assessment messages) via an interactive voice response (IVR) call. Although users enjoyed receiving adherence feedback, most said the IVR call was a nuisance [67]. They were also less likely to answer calls than respond to text messages [35]. Therefore, REACH delivers adherence feedback via a weekly text message instead of a weekly IVR call. Feedback reflects participants’ adherence for the past week and for the prior week and delivers an encouraging message tailored to whether adherence improved, declined, or stayed the same.

Finally, MED users wanted to change the times they received text messages, so REACH allows for flexibility in text message timing. Users determine a preferred window of time to receive self-care promoting text messages and indicate their bedtime for receiving adherence assessment text messages. Participants are able to adjust message timing throughout the intervention by contacting the REACH Helpline (described below).

MEMOTEXT tailors, schedules, and sends text messages using participant data received through an application programming interface (API). At enrollment, research assistants enter participants’ survey responses and electronic health record (EHR) data into REDCap, a secure, Web-based application designed to support data capture for multisite studies [68,69]. REDCap data are then transferred to MEMOTEXT via the API. MEMOTEXT tailors the messages addressing medication adherence barriers by ranking participants’ self-reported barrier scores (see Measures section) and sending messages addressing each user’s 4 highest-ranked barriers. In instances of a tie, the system randomly selects among the tied barriers. MEMOTEXT also tailors regimen-specific messages based on each user’s prescribed diabetes medication taken from the EHR.

We describe each REACH component in Table 2. REACH users receive 2 daily text messages: (1) a text promoting self-care—either tailored to user-identified barriers to medication adherence or nontailored to promote another self-care behavior—and (2) a text assessing medication adherence for that day. Users also receive a weekly text message with medication adherence feedback based on responses to daily assessment texts. Furthermore, after users have their hemoglobin A_1c (HbA_1c) level tested during study enrollment, they receive a text message providing directions on how to access their HbA_1c test result; users can either log on to a Health Insurance Portability and Accountability Act (HIPAA)–compliant webpage hosted by MEMOTEXT or, if they do not have access to Internet, call the REACH Helpline. Finally, users have access to the REACH Helpline for research-, technical-, and medication-related questions. When users leave a voicemail on the Helpline, a REACH team member returns their call within one business day. Figure 2 illustrates the REACH user experience with example text messages.

‎View this figure

‎View this figure

Usability Testing

Sample and Recruitment

Using flyers, interest cards, and referrals from clinic staff, we recruited participants from FQHCs in Nashville, Tennessee. Eligible participants had a T2DM diagnosis, were currently prescribed at least one daily diabetes medication, were responsible for taking their diabetes medication (ie, a caregiver did not administer medication), had a mobile phone with text messaging, were at least 18 years of age, could speak and read English, and provided a social security number (necessary to process compensation). Exclusion criteria included an existing diagnosis of dementia, auditory limitations, an inability to communicate orally, and an inability to receive, read, or send a text message as determined by trained research assistants.

Data and Procedures

The Vanderbilt University Institutional Review Board approved all study procedures before enrollment. Research assistants met with interested patients to describe the study and verify eligibility. In a private room at the patient’s clinic, research assistants administered a brief cognitive screening instrument [70] and sent a test text message to each patient to assess whether he or she could see, read, and successfully respond to the message. If a patient passed this screener, research assistants obtained informed consent before verbally administering survey measures. A clinic phlebotomist performed a blood-drawn HbA_1c test. Research assistants accessed participants’ EHRs to confirm a T2DM diagnosis, collect the type and quantity of prescribed diabetes medication, and the study HbA_1c test result.

During each testing round, participants experienced REACH for 2 weeks and then completed a semistructured phone interview that qualitatively assessed their user experience. Following each round, research assistants collected user feedback, the REACH team resolved content- and research-related issues, and MEMOTEXT resolved technical issues before the next round. Participants received up to US $54 for completing the enrollment survey (US $20), replying to assessment messages (US $1/day), and completing the phone interview (US $20).

Measures

Sample Characteristics

We collected self-reported age, sex, race, ethnicity, income, education (ie, years in school), insulin status, and diabetes duration (ie, years since a diabetes diagnosis). We also asked about comfort with mobile phones and text messaging and used validated survey instruments to capture additional information.

Barriers to Medication Adherence

Respondents rated how much each barrier in Table 1 (written as statements, eg, “I’m not sure what my diabetes medicine is supposed to do”) gets in the way of taking their diabetes medication from 1=“not at all” to 10=“a lot.” Each item maps onto a single IMB model–based barrier.

REACH Engagement

We measured engagement with system-collected responses to daily adherence assessment texts, the frequency of REACH Helpline calls, and the frequency of accessing HbA_1c test results via the website. We calculated engagement with assessment text messages by dividing each participant’s number of responses by the total number of messages sent to him or her. MEMOTEXT tracked Helpline calls and HbA_1c website use.

User Feedback

Likert-type items assessed ease of understanding and helpfulness of the REACH intervention elements. Open-ended items assessed what users did and/or did not like, asked how and why an element was or was not helpful, and elicited suggestions for improving the REACH user experience. Table 3 presents items used to elicit user feedback.

Analyses

We calculated descriptive statistics with SPSS Statistics version 23 (IBM Corp). Interviews were audio-recorded and transcribed verbatim by an external transcription service. Questions and responses were pasted into REDCap under each interview question, organized by testing round, and then exported to Excel. We undertook a pragmatic approach to analyze participant feedback quickly between rounds to support changes to the intervention in a timely fashion. Between rounds, a member of the research team (LAN) read interview transcripts to manually categorize participants’ feedback by intervention component. We then looked across participants’ comments for each intervention component to identify areas for improvement and to ascertain the overall tone and message of the users’ feedback for each component.

Results

Participant characteristics

An average of 12 participants experienced REACH each testing round, totaling 36 participants (Table 4). The average age of the participants was 52.4 (SD 9.5) years, 63% (22/35) were a racial or ethnic minority, 39% (14/36) had less than a high school degree or equivalent, and 67% (22/33) had an income less than US $35,000. The average HbA_1c level was 8.2% (SD 2.2%); 64% (23/36) of the participants had suboptimal glycemic control (HbA_1c≥7.0%). Across rounds, the most frequently reported barriers to medication adherence were forgetting to take doses (56%, 20/36 users reported this barrier with an average score of 5.2, SD 3.0, of 10), the high cost of medication (44%, 16/36 users; mean score 6.2, SD 2.7, of 10), and believing that taking medication is unpleasant (42%, 15/36 users; mean score 5.2, SD 2.6, of 10). The most commonly reported insulin-specific barrier was feeling embarrassed to take insulin in public (35%, 6/17 users who were prescribed insulin; mean score 4.5, SD 2.4, of 10).

All 36 participants completed an exit interview. Overall, participants said REACH was helpful and gave favorable feedback on each intervention element. Participants reported preferences and technical issues requiring iterative improvements between testing rounds. Below, we describe this feedback, our iterative changes by intervention element, followed by changes in our research processes.

Daily Text Message Promoting Self-Care

Across rounds, participants rated the ease of understanding and helpfulness of the daily self-care text message, on average, 9.6 (SD 0.7) and 9.3 (SD 1.4) on a scale of 1-10, respectively. Participants appreciated that these messages were simple and without medical jargon. Participants said inspirational messages made them feel supported and not alone in living with diabetes and motivated them to take more initiative with self-care. Messages with self-care tips and information were helpful because they either provided a useful reminder or communicated something new.

Interviewer: Why did you read those messages? What made you want to?

Participant: They was [sic] helpful. Some things I didn't know [sic]. They helped me understand a lot of stuff because I didn't understand. [37-year-old, African American male]

Many participants valued reminders to care for their diabetes when they otherwise might not think about it:

I thought these messages were very helpful. I get so busy in the day that I don’t [even] take time to eat. And then when I get a text, [I realize] oh, wow, I need to do something. That really helps a lot. I wish I had somebody who did that for me all the time. [59-year-old, white male]

Despite the overall positive feedback about self-care messages, some participants had concerns. For example, a round 2 participant said a message provided a suggestion for remembering to take medications, without providing the steps for carrying out the suggestion. A few participants said some messages implied a problem when they did not have one (eg, “Struggling to take your diabetes medications every day? Talk to a loved one about what is getting in your way.”). To address such concerns, we revised all problematic text messages between rounds 2 and 3 to provide additional context and be less presumptuous (see Table 5 for examples of problematic and revised text messages).

Daily Text Message Assessing Adherence

Across rounds, participants responded to 96% of adherence assessment text messages and, on average, rated the helpfulness of these messages 9.1 (SD 2.1) out of 10. Assessment messages helped remind participants to take their medications and maintain their routine. One participant commented on these messages’ emotional and social support:

[The texts] keep you on task about what you should do...especially if someone doesn't have anybody around. You know it's kind of like having a family member around to remind you, “Hey, you should take your meds.” These [texts] make you feel like someone cares or is concerned about your health and makes sure you're taking care of yourself. So I think that's very helpful. [48-year-old, African American male]

Several participants with optimal glycemic control (HbA_1c <7%) said these messages were not particularly helpful because they routinely took their medication and rarely missed doses. Nonetheless, these participants endorsed the value of these messages for others recently diagnosed with diabetes and/or newly prescribed medication who do not have an established routine.

Round 1 participants complained about needing to respond to assessment messages several times before the system would accept their response. Upon viewing system-collected data, we learned that participants used different variations of “Yes” to respond (eg, “Yup” or “Yeah”). Therefore, we expanded the acceptable response options representing “Yes” and “No” between rounds (see Table 5).

Weekly Text Message Providing Feedback

Across rounds, participants rated the helpfulness of the weekly feedback messages, on average, 8.5 (SD 2.7) out of 10. Round 1 participants had two concerns with the weekly adherence feedback text message. First, many participants felt these messages were wordy and confusing. Because the message provided numerical information about the number of days a participant took his or her medication in both the past and the prior week, the content was difficult to read and interpret. We simplified feedback messages by including only the number of adherent days from the past week, but we indicated whether adherence had improved, stayed the same, or declined with an encouraging statement (Table 5).

Second, some round 1 participants complained their feedback underreported their adherence. This was due, in part, to the limited number of accepted responses to the assessment message (described above). However, we also learned the system was not counting responses received after midnight on the day it sent this message, so we extended the response time window (Table 5). In round 1, participants rated the helpfulness of feedback messages 8.2 (SD 3.0) out of 10. After revising feedback text messages and resolving functionality issues, participants in subsequent rounds rated the helpfulness of these messages 8.8 (SD 2.2) out of 10.

Hemoglobin A1c Text Message

Across rounds, very few participants accessed their HbA_1c test result. One participant logged into the HbA_1c website and 2 participants called the REACH Helpline to get the result over the phone. Participants’ most common reason for not using either option was that they learned their HbA_1c test result from their clinic before receiving the HbA_1c text message. When asked their opinion about accessing their result with the HbA_1c text message, some participants appreciated this convenience, whereas others preferred their health care provider contact them with the result. On the basis of this feedback and the feedback from providers who preferred delivering and individually interpreting HbA_1c test results, we reduced our interpretation of the HbA_1c test result on the website (Table 5) and over the phone.

REACH Helpline

We received 22 voice mails on the REACH Helpline (12 research-related, 8 technical-related, and 2 medication-related voice mails) from 11 participants (8 of whom called more than once). When we asked the other 25 participants about the Helpline, most said they simply did not need it but thought they might use it if the program lasted longer.

Research Processes

On the basis of participant feedback and lessons learned by research staff, we made several changes to our research process. One change involved modifying how we administered the barrier assessment. Initially, research assistants asked participants to rate how much each of the barrier items gets in the way of taking their medication by reading each item aloud sequentially and asking for a rating. After round 1, research assistants reported some participants became disinterested/disengaged when completing this assessment and 20% reported no barriers despite having suboptimal HbA_1c levels. Therefore, after round 1, we changed the barrier assessment to a two-stage process. The first stage is a card-sorting task in which participants sort cards with barrier statements printed on them (see Measures section) into piles labeled “Sometimes” or “Never” based on whether or not the barrier applies to them. Next, research assistants ask participants to rate the degree to which each barrier placed in the “Sometimes” pile applies to them from 1=“a little” to 10=“a lot.” Before the two-stage process, round 1 participants reported a total of 62 barriers. After implementing the two-stage process, round 2 and round 3 participants reported 87 and 92 barriers, respectively. According to research assistants, participants in rounds 2 and 3 were more engaged during the barrier assessment process than participants in round 1.

We also modified the instructions provided during enrollment. Round 1 participants did not know they could change the timing of their text messages, so, in subsequent rounds, we clarified that participants could call the REACH Helpline at any point to request a time change. Also, during round 1, many participants sent unprompted responses (eg, “Thanks” or “OK, I will”) to the text messages promoting self-care, suggesting they thought a person sent these messages. Therefore, we revised our informed consent to make clear that a computer was sending text messages and not a person. For additional safeguarding, MEMOTEXT monitors all text message responses and notifies the REACH team if any text message requires follow-up.

Discussion

Principal Findings

Text messaging interventions provide an opportune platform for extending the delivery of tailored diabetes education and support; however, few have been designed for and tested among disadvantaged persons with T2DM [27,28]. We developed REACH—a tailored text messaging intervention designed to overcome user-specific medication adherence barriers and support other self-care behaviors—and tested its usability among patients with T2DM who were representative of the population REACH is designed for (ie, racially diverse, low SES, more than 25% limited health literacy). Participants who experienced REACH for 2 weeks had favorable opinions and responded frequently to daily text messages. We learned participants’ concerns/preferences, technical issues, and problems with our research process that we then fixed between each testing round, improving REACH in preparation for an evaluative trial.

Overall, participants were satisfied with REACH and provided favorable ratings for each of its elements. Text messages provided emotional/social support, reminded participants to engage in self-care activities, and helped them keep their self-care routine on track. In a similar 4-week study, Dick et al [71] assessed the usability of a text messaging program (SMS-DMCare) for improving T2DM self-care among African Americans. Participants provided ratings comparable to REACH regarding SMS-DMCare’s ease of use and provided similar interview feedback (eg, messages were helpful by reminding participants to take medication amid the demands of their daily lives) [71]. REACH’s text message engagement was higher than SMS-DMCare’s engagement [71], which may be due to REACH’s tailored content and/or personalized adherence feedback.

Usability studies often rely on survey- or questionnaire-based feedback [72], which may overlook much of what participants like or do not like about a system and how to improve it. Georgsson and Staggers [73] endorse using multiple data sources to identify and address usability issues. We collected both quantitative and qualitative feedback and system-collected data to fully understand users’ experience, improve our programmatic content and functionality, and resolve technical problems. REACH users who were adults from racially diverse and low-SES groups expressed concerns with the phrasing and wordiness of some messages, so we improved them. Furthermore, unanticipated feedback from research assistants and clinic staff was instrumental in refining our research process.

Through this multiple data source approach to usability testing, we made several improvements to REACH and enriched the user experience. We revised text messages to be more comprehensive, clear, and consistent with participants’ preferences. We limited our interpretation of HbA_1c test results to be more respectful of provider-patient relationships. We also improved REACH’s functionality to ensure the system recognizes and records participants’ attempts to interact with the intervention. Finally, we improved our assessment for capturing participants’ adherence barriers and modified our informed consent to ensure participants know how to use each intervention component and that a computer, not a person, sends all text messages.

There are several limitations to this study. First, participants experienced REACH for 2 weeks. Therefore, feedback and engagement may not be representative of participants experiencing REACH for longer periods. Furthermore, we compensated participants US $54 for their participation in the 2-week usability testing and feedback interview. By providing this incentive, we sought to adequately compensate participants for time and travel to the enrollment appointment and to offset mobile phone costs associated with text messages and the phone interview. This compensation may have inflated engagement with the intervention, but it was important for usability testing that participants actually use the intervention to be able to provide meaningful feedback. Despite this compensation, few participants accessed the REACH Helpline and HbA_1c website, making it difficult to gain insight on how participants felt about these elements and their functionality. Furthermore, participants may have been reluctant to provide critical feedback owing to study compensation, social desirability, or associating the study with their clinic. Additionally, because we were interested in specific questions concerning each intervention element, we composed our own feedback interview items and do not have validity and reliability information to report. Finally, although our sample size far exceeded the targeted enrollment for qualitative (at least 5) and quantitative (at least 20) usability testing [74,75], our sample was still too small to examine differences in opinions by participant characteristics.

Conclusions

Usability testing is imperative for ensuring that effects identified during efficacy trials are due to the intervention as intended and not due to errors in understanding or using the intervention [72]. Moreover, involving disadvantaged adults in usability testing may reveal preferences and concerns unique to this population. Iterative usability testing of the REACH intervention using multiple data sources revealed shortcomings in content, functionality, and research processes that we addressed before evaluating its effects on adherence and glycemic control in a randomized controlled trial. The REACH randomized controlled trial will assess the intervention’s effectiveness by recruiting patients from FQHCs and comparing outcomes between patients who did and did not experience REACH. To gain knowledge about REACH’s potential for implementation in clinic settings as a supplement to usual care, we will compensate participants for completing study assessments but will not provide them mobile phones or mobile phone plans. Although users experienced and commented on REACH specifically, our usability testing process and findings are applicable to other technology-delivered health interventions for disadvantaged populations. Specifically, informed consent should affirm that participants understand an intervention’s functionality, capabilities, and automation. Additionally, intervention content should provide enough information to be useful and avoid implying that a user is experiencing a specific issue; this is challenging while maintaining brevity necessary for digital content. Finally, our findings emphasize the importance of using multiple methods and sources of data to identify and resolve usability issues.