Men and women who were overweight or obese with Android smartphones (BMI 25-49.9 kg/m²; N=9) were recruited in South Carolina for a 2-month weight loss pilot intervention to test usability and provide feedback to be used in the refinement of the smartphone app. Participants were recruited via worksite listserv announcements, flyers, and newspaper advertisements. Exclusion criteria included not having an Android phone, previous stroke or heart attack diagnosis, diagnosis of diabetes and using insulin or oral medications to control diabetes, BMI outside the range of 25.0-49.9 kg/m², unable to attend required meetings, unable to access the Internet using a computer for completing assessments, having a psychiatric illness, receiving treatment for drug or alcohol dependency, having an eating disorder, participating in another weight loss program, being pregnant or planning on becoming pregnant during the study, and breastfeeding. Participants were excluded for endorsing any of the first 4 items on the revised Physical Activity Readiness Questionnaire (PAR-Q) [15,16]: (1) informed by a doctor that they have a heart condition and should only participate in physical activities approved by a doctor, (2) feeling chest pain when engaging in physical activity, (3) experiencing chest pain in the past month when not engaging in physical activity, and (4) ever losing their balance and becoming dizzy or ever losing consciousness. If participants reported a bone or joint problem that could be made worse by participating in physical activity (item 5 of the PAR-Q) or taking blood pressure medication (item 6 of the PAR-Q), they were required to have a physician consent form completed to participate in the study. Participants received US $30 for completion of assessments at baseline, US $15 after the 1-month focus group, and US $15 after the 2-month follow-up weight assessment.

Participants attended 4 in-person meetings: an orientation session to learn about the study and complete baseline dietary assessments; a training session to learn how to download and use the Social POD app and podcasts and to collect baseline height and weight measurements; a mid-study focus group at 1 month to provide feedback regarding the usability of the Social POD app, provide suggestions for improvement, and collect 1-month weight measurements; and an end-of-study session to provide 2-month weight measurements. All participants provided written consent. This study was approved by the University of South Carolina institutional review board.

Participants were instructed to track their total calories from all meals, snacks, and beverages consumed; minutes of intentional physical activity; and body weight each day for the duration of the 2-month usability study. Participants were instructed to use MyFitnessPal or LoseIt, free commercial diet-tracking apps with extensive nutrient databases, to look up calorie information for food and beverages consumed. Participants were then asked to transfer total calories from each meal and snack consumed for the day to the Social POD app. Screenshots of the tracking features are shown in Figure 1.

Participants received within-app notifications at certain times throughout the day to remind them to self-monitor (promoting self-regulation) diet, minutes of physical activity, and total body weight each day. Participants could view a history of all dietary, activity, and weight information entered on the within-app history screen. Participants could view weight entered on a weight graph. Participants who were frequently using the Social POD app (users who entered information in the app in the past 48 hours) were prompted via notifications to send encouraging messages to other group members who had not entered data in the app over the previous 48 hours (infrequent users). Messages were sent from frequent users by clicking the notification to send a message to an infrequent user, selecting one of three options listed on the message selection screen, and clicking Send. Screenshots of the message log, message selection, and history screen of the Social POD app features are shown in Figure 2.

This study is novel in that it used theory-based messages designed by the researchers to help re-engage infrequent participants over the course of the 2-month intervention. The participants were matched to provide support to one another based on principles of recommender systems used by some websites and applications (eg, Amazon and Netflix) [17-20], which filter information to match users based on user history or preferences [21]. Frequent users were randomly matched to provide support (by sending a theory-based message) to help re-engage infrequent users in this intervention. Social Cognitive Theory (SCT) [22,23] was used as a framework to design user-to-user messages that targeted social support [13], self-efficacy [22,23], and outcome expectations [24] regarding self-monitoring behavior (ie, targeting self-regulation) [23] of diet, physical activity, and weight. SCT is the belief in the reciprocal relationship between cognitions, environmental influences, and behavior [22,25]. Constructs from SCT were selected for this study based on results from a previous Internet-based weight loss intervention, which found that targeting these specific constructs led to healthier diet and physical activity behaviors and resulted in a reduction of body weight among study participants [26]. The interventionists created messages for each of the three social construct theories and prompted frequent users to select 1 of 3 messages to send to an infrequent user. Table 1 provides examples of each message type as well as the SCT construct targeted.

Participants were provided with 3 20-minute, evidence-based weight loss podcasts each week. Podcasts were uploaded to the study website, and participants were sent a reminder that a new podcast was available every Monday, Wednesday, and Friday via email. Participants were instructed to listen to the 3 podcasts within the week but could listen at a time and place of their choosing. Podcasts were informed by SCT and provided participants with a range of weight loss topics. Podcast topics included nutrition and exercise information, an audio diary tracking the weight loss progress with challenges experienced by a male and female character, and a weight loss soap opera depicting the challenges of overcoming social barriers to weight loss, with a goal-setting activity related to weight loss at the end of each episode. Specific information regarding the development and testing of these podcasts in previous interventions can be found elsewhere [12,27].

Participants completed online questionnaires assessing demographic characteristics. Height was measured at baseline assessment only using a Seca 213 (Hamburg, Germany) calibrated stadiometer. Weight was measured at baseline and 2 months using a Seca 869 calibrated digital scale accurate to 0.01 kg. Dietary measures were completed by participants using the National Cancer Institute’s Automated Self-Administered 24-Hour Dietary Recall (ASA24) [28]. Each participant completed the ASA24 online for 1 unannounced weekday and weekend day at baseline and 2 months. The following previously validated questionnaires were completed by participants at baseline and 2 months unless otherwise specified: the Paffenbarger Physical Activity Questionnaire to determine calories expended during activity [29], the 20-item Weight Efficacy-Lifestyle Questionnaire to measure participant self-efficacy [30], the 44-item Big Five Inventory (baseline only) as a measure of personality characteristics [31], and the Sallis Social Support Scale [32] and Physical Activity Social Influence Scale as measures of social support from family and friends (modified to include online social networks) [33]. Objective measures of frequency and duration of app use were collected unobtrusively by a secure network server following the intervention (at 2 months). All baseline measurements were collected prior to the administration of the weight loss intervention.

Participants attended 1 of 2 focus group sessions at 1 month to provide feedback regarding their use of the intervention components (Social POD app and podcasts). The focus groups were conducted by a trained focus group facilitator, and questions were asked using a semistructured interview guide developed by the investigators that included prescripted and spontaneous probes designed to elicit information from participants regarding their experience with the Social POD app. Questions were designed to assess participant satisfaction and dissatisfaction with the weight loss podcasts; the Social POD app including the tracking, prompting, and messaging features; and the weight graph and history features for future revisions. Participants were also asked if they would like incentives for using the app, and if so, how they would like an incentive system to be structured. Information regarding aesthetics and appearance of the app was solicited as well.

Focus group sessions were taped using an audio recorder, and all participants were instructed to use their study identification number to protect confidentiality during the focus groups. A trained graduate research assistant was present at the focus group sessions to take field notes during the sessions. The focus group facilitator completed detailed memos following each midstudy focus group session to document salient themes from participant discussions. Recordings from the 1-month focus group were transcribed verbatim by the research assistant and were cross-referenced with recordings for accuracy.

During the study (weeks 1 through 7) participants completed brief weekly online surveys each Friday to report the number of podcasts listened to and the number of days they used the Social POD app that week. Participants also reported any problems experienced using the Social POD app during the past 7 days and any suggestions they had to improve the app. If participants reported problems or suggestions for the Social POD app, they were then asked to write a detailed explanation.

All qualitative data were collected and analyzed between February and August 2014 using NVivo for Mac Beta 2014 statistical software (QSR International) to extract a combination of a priori and emergent themes from the focus groups to guide the revision of the Social POD app. Open-ended responses from weekly surveys regarding problems experienced and suggestions for improvements to the Social POD app were coded as well. Stata version 13.1 (StataCorp) software was used to detect statistical significance for all quantitative data analysis. Paired samples t test was used to test for statistically significant differences from pre- to posttest for weight, calories consumed, and calories expended during reported physical activity. The effect size for weight loss was also calculated. Correlation between total app entries and percent weight loss at 2 months was examined. Correlation between total days of objectively measured app use and total days of self-reported app use (via weekly questionnaires) by participants was examined. Fisher exact test was used, due to the small sample size, to detect significant differences in user-to-user message selection among participants categorized into subgroups based on amount of weight loss (dichotomized into groups of high versus low weight loss at the median split of percent weight loss). The sample size for this study was determined for qualitative analysis based on expert recommendations for mHealth usability testing studies [34,35]; therefore, this study was not powered to detect significant changes in within-group weight. Changes in weight and weight-related behaviors, however, are presented.

A total of 34 individuals inquired about the study, 15 individuals qualified to participate, and 9 completed baseline measures and received the intervention. Major reasons why participants did not qualify included not having an Android phone (9/19, 47%) and meeting medical exclusion criteria (3/19, 16%). Most participants (7/9, 78%) attended the 1-month focus group sessions (2 infrequent app users did not attend), and all participants completed the poststudy questionnaire and unannounced dietary recalls and had their body weight measured poststudy. The response rate for weekly surveys was 62% (39/63). Baseline demographics of the study sample are presented in Table 2.

Major themes that emerged during focus group discussions are reported below with illustrative quotes from participants. These themes included Social POD app functionality, improvements to the Social POD app and podcasts, additions to the Social POD app, incentive and goal system for the Social POD app, and satisfaction with the Social POD app and podcasts.

A news feed will be developed for the Social POD app for participants to view the progress of other users with weight loss-related goals. Participants will be able to send others encouragement for achieving these goals (eg, logging 30 minutes of exercise, logging diet and weight) through the news feed, targeting positive reinforcement [22]. Other revisions to the next iteration of the Social POD app will include the ability to earn points on a Point Tracker within the app for self-monitoring diet, physical activity, and weight and for sending others encouragement through the news feed. Points will be redeemed for study-provided prizes at the end of the pilot RCT, targeting reinforcement [22]. A weight loss competition among participants, as suggested by one participant, will not be integrated into the revised version of the Social POD app to minimize the potential risk of harm to some participants who are not achieving weight loss goals as quickly as others. Integrating a database of food and beverages similar to commercial diet tracking apps to view the nutrient content of commonly consumed items was recommended and will be incorporated into the calorie tracking features of the Social POD app to facilitate self-regulation [23]. A more extensive list of activities will be incorporated to the physical activity tracker in the revised Social POD app, also promoting self-regulation [23]. The color scheme of the app will be modified to include colors that are brighter and more eye-catching, and the option to add an avatar on the home screen will be incorporated. User-to-user messages will be revised to include more encouraging statements to better re-engage infrequent users with the Social POD app. Planned revisions for the podcasts include rerecording the podcast episodes to improve the sound quality and volume across segments within the episodes.

Participants lost a mean of 0.94 kg (SD 2.22). Differences in mean participant weight before (91.48 kg, SD 19.08; 95% CI 76.82-106.15) and after (90.55 kg, SD 20.01; 95% CI 75.17-105.93) the 2-month intervention were not statistically significant (P=.24, d=.05, r=.02). Participants reported expending a mean of 171 kcal/day (SD 153) during intentional physical activity following the intervention as compared to 138 kcal/day (SD 139) at baseline, yet this was not a statistically significant difference (P=.57). Participants reported consuming significantly fewer calories following the intervention (1570 kcal/day, SD 508) than before (2384 kcal/day, SD 993, P=.01).

Mean number of Social POD app entries over the course of the 2-month usability study was 77.2 (SD 73.8, 95% CI 17.66-133.68) with a minimum of 0 and a maximum of 219 entries. Messages were sent by frequent users targeting social support most often (32/68, 47%), followed by self-efficacy (28/68, 41%) and outcome expectations (8/68, 12%). There was not a statistically significant correlation between total app entries and percent weight loss over the course of the intervention (r=.49, P=.19). There was no difference in the type of message selected (self-efficacy, social support, and outcome expectation) between those participants who were successful at weight loss as compared to those who were less successful (defined at median split in percent weight loss) (P=.79).

On the weekly surveys, participants reported listening to an average of 2.24 podcast episodes (SD 1.50; minimum 0, maximum 6) per week, and they reported using the Social POD app an average of 4.5 days (SD 2.25; minimum 0, maximum 7). There were 7 reports of problems using the Social POD app by 3 participants over the course of the 2-month study. Reported problems from weekly surveys included spontaneous crashing (2 times) and notifications linking with the incorrect screen in the Social POD app (1 time); there were no explanations for the other reported problems on the weekly surveys.

There were 2 participants with no objective measure of Social POD app use over the course of the study. A participant reported using the Social POD app 1 day during the study, and she specified on the weekly survey that she did not use the app because she did not have enough time. The other participant without objective app use data reported on the weekly surveys that she used the app a total of 21 days during the study. The total number of days participants used the app (as objectively measured by the app) and total number of days of self-reported app use by participants via weekly surveys was highly correlated (r=.87, P<.01), indicating that self-report was a reliable measure of app engagement in this study.

While there has been much work in the area of mobile app development for health, there is currently little published research in the area of development and testing of new mobile apps for weight loss among adults who are overweight and obese. There are several recent studies documenting the development and testing of new apps for weight loss in adolescents [36], apps for increasing physical activity and reducing screen time among adolescent males [37], and apps for predicting the risk of childhood obesity among infants [38]. Several recent studies report on the development and testing of apps for modification of diet and physical activity among the young adult population [39-41].

In a similar mixed-methods usability study, Morrison and colleagues used a computer-based weight loss program in conjunction with an Android mobile app to improve participant goal setting and motivation to achieve weight management goals among young adult participants over a 4-week period [39]. This app also offered participants the opportunity to set notifications, similar to those in the Social POD app, as reminders to use the diet and physical activity diary features. Participants could choose when and if they would like to receive notifications in an effort to improve the usability of and satisfaction with the mobile app [39]. It was found that only about half of the participants used the self-monitoring features of the app over the course of this usability testing [39]. While 1 participant in the Social POD app study noted that the notifications could be “annoying,” others found the notifications, which were preset by study coordinators, a helpful reminder to self-monitor their behaviors and weight and said that they otherwise would not have performed this task. Prior research has also demonstrated that weight loss is improved when self-monitoring activities are performed in real-time and proximal to the target behavior [42]. This indicates that mHealth apps could better help users self-monitor health-related behaviors using a notification system that cannot be eliminated by participants to remind them throughout the day and at times in which the behaviors typically occur.

In a qualitative study examining the desired features of weight loss apps among young adults, Tang and colleagues found that participants valued the opportunity to move beyond strictly tracking their eating behaviors and wanted a way to integrate other features, such as behavioral weight loss goals [40]. Other mobile apps have also incorporated a goals feature. Morrison and colleagues instructed participants to set their own goals and track their goal achievement progress using their mobile app and found that participants most frequently accessed informational content using their app (eg, food lists); fewer participants used the goal setting and monitoring features of their mobile app [39]. As some participants are less likely to use app components that require initial set-up (eg, setting goals or notifications), including some type of preset behavioral goals for users to achieve could help promote user engagement and motivation with these mHealth apps.

In their qualitative study documenting the development and prototype testing of an app promoting change in eating and activity behaviors to reduce weight gain among young adults, Hebden and colleagues received feedback requesting positive reinforcement for performing desired behaviors (eg, eating healthy foods or engaging in physical activity) [41]. This is similar to participant suggestions for more opportunities to provide and receive praise in the Social POD study. Including opportunities for users to give and receive praise for performing targeted health behaviors could be another necessary component to help establish and maintain health behavior change within the context of mHealth interventions.

The color scheme of the Social POD app will be updated with brighter colors to better appeal to users as suggested in the Social POD participant focus groups. Tang and colleagues conducted focus groups with young adult participants and also found that the perceived attractiveness of an app was an important consideration for participant satisfaction and maintaining engagement with weight loss apps [40]. Functionality issues similar to those found in the notification system of the Social POD app were found in a study conducted by Morrison and colleagues where a participant reported not receiving notifications during testing of a new mobile app for goal setting and self-monitoring of diet and activity among a small sample of adults [39].

Ensuring that all components of mHealth apps are functioning properly over time is an integral part of the usability testing process and of great importance in remotely delivered behavioral health interventions. Following usability testing, it was imperative to prioritize the changes that were made to the Social POD app prior to the pilot RCT. Correcting functionality issues and developing a newsfeed and incentive systems took priority; an avatar to personalize the app was not incorporated, and, while a nutrient database was added, it was not as extensive as originally hoped.

Because the purpose of this study was to test the first iteration of the Social POD app prior to the pilot RCT, it was not adequately powered to detect statistically significant differences in pre-post scores. The time period of the study was limited and at 2 months may have been too short to detect significant differences in participant pre-post body weight. The Social POD app is currently only available for the Android operating system. Despite the fact that the Android phone is the most prevalent cell phone in the United States [43], the fact that other smartphones (eg, iPhone) were excluded may reduce the generalizability of the findings. Direct questioning (versus open-ended questioning) was used to solicit participant opinions regarding the addition of an incentive system, which could have resulted in bias. The sample size of this study was also very small, at just 9 participants and only 1 male, and was therefore not a representative sample of all potential users of the Social POD app. Because the sample size was small, message selection results could be skewed toward the message type that frequent participants preferred. Participants were instructed to use a commercial database to identify the caloric content of food and beverages consumed and use of this database could have contributed to the change in weight observed following this intervention.

One strength of this study was the use of a mixed-methods design, which included participant focus groups to obtain usability and functionality feedback and suggestions for improvements to the Social POD app prior to the pilot RCT. Obtaining feedback through the weekly surveys was another strength, given the potential for some participants to refrain or modify comments during focus groups based on social desirability bias. Another strength of this study was the iterative testing of the Social POD app to uncover and resolve functionality issues prior to the pilot RCT [34]. While the total number of app entries was not statistically significantly correlated with percent weight loss, a correlation coefficient of r=.49 represents a fairly large effect size [44]. Furthermore, mean difference in calories expended during intentional physical activity from pre- to posttest was also not significantly different, but this represented a small effect and could be greater if tested among a larger sample and over a longer time period [44]. While the sample size was small in this usability study, the minimum percentage of problems identified during testing of mobile apps increased from 55% to 85%, respectively, when the sample size increased from 5 participants to 10 [45]. The reach of mHealth interventions such as this has the potential to be even greater than traditional face-to-face interventions, and even small changes in weight have the potential to impact public health outcomes and reduce disease risk [46]. A reduction in body weight as little as 1 kg, as seen in this study, has been associated with a 16% reduction in type 2 diabetes risk [47] demonstrating that this type of mHealth intervention is a scalable way to deliver a weight loss program with beneficial reduction of disease risk. The comparison of objective and subjective reports of app use is another strength of this study.

The Social POD app provides an innovative way to encourage self-monitoring of dietary intake, weight, and physical activity while encouraging frequent users to provide social support to infrequent users. Although more time is needed for development, this mHealth intervention can be disseminated broadly for many years and to many individuals without the need for additional intensive in-person hours. The Social POD app should be tested in a larger clinical trial for a longer length of time to determine if changes in participant weight, calories consumed, and calories expended during physical activity are improved.