JMIR Publications

JMIR Human Factors

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 08.05.17 in Vol 4, No 2 (2017): Apr-Jun

This paper is in the following e-collection/theme issue:

    Original Paper

    A Web-Based Graphical Food Frequency Assessment System: Design, Development and Usability Metrics

    1Biomedical Engineering, School of Biological Sciences, University of Reading, Reading, United Kingdom

    2Hugh Sinclair Unit of Human Nutrition and Institute for Cardiovascular and Metabolic Research, Department of Food and Nutritional Sciences, University of Reading, Reading, United Kingdom

    Corresponding Author:

    Rodrigo Zenun Franco, MSc

    Biomedical Engineering

    School of Biological Sciences

    University of Reading

    Whiteknights Campus

    Reading,

    United Kingdom

    Phone: 44 118 378 7668

    Fax:44 118 378 7668

    Email:


    ABSTRACT

    Background: Food frequency questionnaires (FFQs) are well established in the nutrition field, but there remain important questions around how to develop online tools in a way that can facilitate wider uptake. Also, FFQ user acceptance and evaluation have not been investigated extensively.

    Objective: This paper presents a Web-based graphical food frequency assessment system that addresses challenges of reproducibility, scalability, mobile friendliness, security, and usability and also presents the utilization metrics and user feedback from a deployment study.

    Methods: The application design employs a single-page application Web architecture with back-end services (database, authentication, and authorization) provided by Google Firebase’s free plan. Its design and responsiveness take advantage of the Bootstrap framework. The FFQ was deployed in Kuwait as part of the EatWellQ8 study during 2016. The EatWellQ8 FFQ contains 146 food items (including drinks). Participants were recruited in Kuwait without financial incentive. Completion time was based on browser timestamps and usability was measured using the System Usability Scale (SUS), scoring between 0 and 100. Products with a SUS higher than 70 are considered to be good.

    Results: A total of 235 participants created accounts in the system, and 163 completed the FFQ. Of those 163 participants, 142 reported their gender (93 female, 49 male) and 144 reported their date of birth (mean age of 35 years, range from 18-65 years). The mean completion time for all FFQs (n=163), excluding periods of interruption, was 14.2 minutes (95% CI 13.3-15.1 minutes). Female participants (n=93) completed in 14.1 minutes (95% CI 12.9-15.3 minutes) and male participants (n=49) completed in 14.3 minutes (95% CI 12.6-15.9 minutes). Participants using laptops or desktops (n=69) completed the FFQ in an average of 13.9 minutes (95% CI 12.6-15.1 minutes) and participants using smartphones or tablets (n=91) completed in an average of 14.5 minutes (95% CI 13.2-15.8 minutes). The median SUS score (n=141) was 75.0 (interquartile range [IQR] 12.5), and 84% of the participants who completed the SUS classified the system either “good” (n=50) or “excellent” (n=69). Considering only participants using smartphones or tablets (n=80), the median score was 72.5 (IQR 12.5), slightly below the SUS median for desktops and laptops (n=58), which was 75.0 (IQR 12.5). No significant differences were found between genders or age groups (below and above the median) for the SUS or completion time.

    Conclusions: Taking into account all the requirements, the deployment used professional cloud computing at no cost, and the resulting system had good user acceptance. The results for smartphones/tablets were comparable with desktops/laptops. This work has potential to promote wider uptake of online tools that can assess dietary intake at scale.

    JMIR Hum Factors 2017;4(2):e13

    doi:10.2196/humanfactors.7287

    KEYWORDS



    Introduction

    Food frequency questionnaires (FFQs) are a commonly used tool for dietary assessment, and paper-based FFQs have been used for decades in the field of human nutrition [1-2]. An FFQ consists of a list of food and drink items, and for each item, an individual indicates their typical consumption frequency and portion size, based on their dietary intake for a given reference period (eg, the past month). The list of foods is based on the most frequent foods in the region and typically has around 100 items. Consumption frequencies are normally indicated using categories described in text (eg, 1 per day). Portion sizes can be indicated by selecting text-based categories (eg, small, medium, or large) or by selecting the closest match from a selection of portion-size photographs of actual foods [3]. There have been studies published on the validity of FFQs in different countries, in both paper-based and digital versions [4-13]. FFQs are frequently used in epidemiological (ie, population) studies as they are inexpensive to process, can be self-administered, and are relatively quick for participants to complete [14-15]. However, they are also prone to reporting bias; the consumption of healthy foods has been overestimated using this method [16-17].

    FFQs have traditionally been delivered using a pen-and-paper format, but there is a burden associated with this format for study participants, health professionals, and investigators. The digitalization of nutrition assessment methods has excellent potential to save time and resources, is preferred by participants [18], and is more suitable for large-scale studies. Other online dietary assessment methods such as the 24-hour recall [19-21] claim better accuracy than FFQs. However, the motivations for investigating online FFQs include that they are easier to replicate technically than these other methods, which often require a much larger food database and more complex technologies such as text search functionality; FFQs may also be more suitable for certain applications including online personalized nutrition interventions [22]. Although some Web-based FFQs have been developed in recent years, they have not been used widely in this format as yet, and there are few published results in terms of user acceptability of online FFQs.

    In order to facilitate the dissemination of online FFQs, it is important that the scientific and public health communities have open and free access, not only to the final results of validation studies but also to the design, architecture, development, and deployment of scalable, replicable, and secure tools. Furthermore, interdisciplinary collaboration and shared understanding between the health and technical communities is important for furthering research in this field, and as such, it is appropriate that studies also report their work from the perspectives of multiple disciplines. Therefore, this paper presents both the technical design of a Web-based graphical food frequency assessment system and results from user testing, with an aim of making a contribution to the wider uptake of digital FFQs.

    The online FFQ described in this paper was designed and developed for the Eat Well Kuwait project (EatWellQ8, www.eatwellq8.org), which aims ultimately to investigate whether Web-based personalized nutrition (based on dietary intake and anthropometrics) is as effective as face-to-face communication of personalized nutrition in Kuwait. The project is a collaboration between the University of Reading and the Dasman Diabetes Institute in Kuwait City [23]. The first stage of this project focused on the design and development of the Web-based FFQ, and a validation study is currently under way to compare the online FFQ with the current paper version of a Kuwaiti FFQ and a 4-day weighed food record.

    Objectives

    Overview

    This paper aims to make a contribution to the wider uptake of digital FFQs by describing the rationale, design, implementation, administration, and user feedback of a Web-based graphical food frequency assessment system. Online FFQs are not yet being used widely, and this is due in part to a variety of technical challenges. This section summarizes some of the technical considerations relevant to facilitating wide deployment of online FFQs.

    Reproducibility

    With a view to decreasing completion time and thereby increasing user acceptability, the list of food items in an FFQ normally includes only the most common foods in a region, divided into food groups (fruits, vegetables, etc). As these food lists and their related portion size images vary by location, it is useful to have either a customizable central system or an easily replicable system to help ensure that locally applicable FFQs for different regions can be created easily. Ideally, this system should be inexpensive in order to mitigate financial constraints that could block deployment. Furthermore, any need for technological expertise in customization and administration could hinder reproducibility, so it is important to design these aspects with ease-of-use in mind.

    Scalability

    One of the drivers for developing online dietary assessment methods is the potential to support population-level studies. When operating at this large scale, there is a potential to see high peaks in the system traffic, which are not easily handled. This is an important requirement to be considered in the system architecture.

    Mobile Friendly

    The need to consider deployment on mobile devices and tablets is more and more relevant, considering an increase in the market share of smartphones and tablets as compared with desktops and laptops [24]. The delivery of an FFQ via tablets and smartphones presents particular challenges. For example, due to screen size constraints, it is difficult to present all the portion sizes (usually between 3 and 7 images) on the screen simultaneously. The layout and interaction design has the potential to influence participant responses or increase the task completion time and requires careful consideration.

    Security

    Population studies often store sensitive data, since they usually collect medical information together with personal details. In this scenario, it is important to provide authentication and authorization features, protect the database from unauthorized access, and communicate with the database using a secure protocol.

    Usability

    Empirical data on system usability is important for enabling evidence-based decisions in the design and improvement of further systems. The system should build in the ability to collect metrics such as completion time and usability surveys.


    Methods

    Technical Design

    Overview

    The design of the EatWellQ8 food frequency questionnaire considered the main requirements described in the previous section and assessed and compared these with the main advantages and disadvantages of the currently most-used Web architectures and technologies.

    The requirements showed that the system was not intense computationally, pointing to the possibility of using a modern Web architecture called single-page application (SPA) [25]. In this paradigm, all the necessary code (HTML, Cascading Style Sheets, and JavaScript) is retrieved in a single load, and the updates in the view are managed by the code running in the browser. The JavaScript framework for creating SPA proposed by Google is called AngularJS, which is entirely client-side (ie, browser only) [26].

    An SPA architecture creates the possibility of using static hosting for delivering the code and media files (eg, food images in this project), which is much cheaper than dynamic hosting (ie, servers) and removes any need for server maintenance.

    Beside the static hosting, there were three basic requirements that needed to be fulfilled: user authentication, user authorization, and a secure database. After analyzing several major cloud-computing providers (ie, Amazon Web Services, Google, IBM, and Microsoft), it was clear that the typical Web app architecture could be delivered by any of them. One particular service that stood out during this comparison was Google Firebase for its particular focus in providing the most essential features for developing Web and mobile apps in a very affordable way, which has attracted more than 400,000 developers worldwide. Its main features are a real-time database, user authentication, and static hosting [27].

    Reproducibility

    Since data collection and retention standards are different around the world, a customizable central system may face some practical difficulties for implementation. This was one of the main reasons for choosing to create an easily replicable system using cloud-computing services, which are accessible worldwide.

    Data is stored in a JavaScript Object Notation (JSON) document in the Firebase database. In order to facilitate food list modification by nontechnical administrators, the original food table was created as an Excel spreadsheet (Microsoft Corp). The cells were then concatenated (using Excel’s concatenate function) into comma-separated values text, which was then converted to JSON (using a online converter such as convertcsv.com). The JSON was then imported to Firebase. The following object shows a food item structured in JSON, illustrating its human-readable format:

    “foods” : [ { “arabic” : “Broccoli in arabic”,“english” : “Broccoli”, “id” : 0 }, ....]}
    Scalability

    Using an SPA approach combined with a Firebase database, all the processing is transferred to the client (browser), which can easily handle simple interactions and functions for rendering the pages. The Firebase Spark Plan (free) can support 100 simultaneous connections with the database (this increases to unlimited simultaneous connections with the Flame Plan which, at the time of writing, costs US $25/month) using a secure https protocol and deliver the pages and images via its global Content Delivery Network [27].

    Mobile Friendly

    In order to design a Web app that can be used readily on mobile devices, the design was based on Bootstrap, a highly popular responsive Web framework. It is open source and has built up a big developer community since its launch in 2011 [28].

    The Bootstrap functionalities that played important roles in our implementation were the responsive navigation bar and the modal component; the former creates an adjustable navigation bar that converts into a hamburger icon on small devices, and the latter displays a pop-up window on top of a current page (this was used to be able to display food portion images using the entire screen).

    Security

    Firebase provides a complete authentication feature. Among the possible authentication providers (Facebook, Google account, etc), the email and password combination was enough for this project, although others could also be provided as alternatives. Firebase enables the use of AngularJS combined with its product via the AngularFire library. It provides a 3-way binding between the HTML, the JavaScript, and the database. This means that any modification in one of these parts can be propagated to the other two. For example, a modification of one value in the database triggers an update in the website. This feature becomes even more powerful when different systems are connected to the same real-time database, enabling users to switch between a website and a mobile app, for example, with their data synchronized between the two. Best practices in terms of authentication and page routing are provided by Firebase in the AngularFire Seed, a small open-source project that contains the implementation of the basic features (log in, password reset, data binding, etc) that were used in this project.

    Besides the authentication feature, Firebase provides Security Rules for defining authorization. Every time a user authenticates, an internal variable (auth) is populated with user information (eg, user unique ID). Using a simple JavaScript-like syntax, authorization was defined in order to prevent unexpected access. The following rules exemplify how to block access (read/write) to new objects and only allow authenticated users to access their own FFQ results:

    {“rules”: {“.read”: false, “.write”: false,
    “ffq”: {“$user”: {“.read”: “auth.uid === $user”,“.write”: “auth.uid === $user” }}}}

    Another important security aspect is communication between the browser and the database. Firebase uses https, which requires encryption in the communication between the browser and Firebase. If a custom domain is desired for the deployment (eg, https://eatwellq8.org), it will be necessary to configure the Domain Name Server according to the records provided by Firebase.

    EatWellQ8 Food Frequency Questionnaire

    The EatWellQ8 FFQ contains 146 food items (including drinks), adapted from the European Prospective Investigation into Cancer Study [29] and Food4Me FFQs [4] to reflect a Kuwaiti diet. The food names are shown in both English and Arabic. For each item, users indicate consumption frequency during the last month by selecting from 1 of 8 options: “never or less than 1 per month,” “1 to 3 per month,” “2 to 4 per week,” “5 to 6 per week,” “1 per day,” “2 to 3 per day,” “4 to 6 per day” and “more than 6 per day” [4]. Due to the number of options, the selection was implemented via a select element (drop-down list), which is expanded on mobile devices. In order to speed up the completion time, the default choice was set to the first option (“never or less than 1 per month”), so that participants could simply skip an item if they did not consume that specific food item (Figure 1).

    Users indicated portion size by selecting from 1 of 3 photographs of actual food portion sizes (Figure 2). Other studies have investigated various options to enable users to specify food portion sizes from photographs, including selecting from 1 of 8 portion size photographs [30] and a combination of having 3 portion size photographs to select from combined with 4 radio buttons to indicate portion sizes that were bigger/smaller than those depicted in the photos [31]. For the current system, the decision to present 3 portion size photos was based partly on prior (unpublished, from the FFQ described elsewhere [31]) user data indicating that photos are far more frequently selected than radio button options without an associated photograph and an aim of presenting all the photos to users in an efficient manner even on small screen sizes. Portion size photographs are sometimes labeled using descriptive labels of the portion sizes: for example, small, medium, or large. In our study, the photographs are presented without any labels to avoid potentially biasing the users in their choices. Each time a user selects a food frequency, the appropriate portion images are automatically presented to the user; this is implemented in a pop-up window using the modal component described earlier. After the portion size has been selected, the users’ selections are presented as “Size A,” “Size B,” or “Size C” (see Figure 1).

    Although participants are encouraged to complete the FFQ in one sitting/session, it is important to offer the possibility to save the FFQ, in case the user is interrupted or loses Internet connection temporarily. Hence, each food selection is saved individually (after the portion size selection), and the user has the option to retrieve the FFQ of a particular day when returning to the system. A timestamp (format yyyy-mm-dd) is saved together with each FFQ entry in the database, after formatting the JavaScript Date Object, in order to check the existence of an entry for that specific day.

    Usability Metrics

    To enable collection of data on system usability and use, the system included a usability survey and also logged usage data. The usability survey was presented after completion of the food frequency questionnaire. A modified version of the System Usability Scale (SUS) [32], originally defined by Brooke [33], was used to assess the user acceptance of the online FFQ. The SUS consists of 10 questions alternating between positive and negative statements, with 5 possible responses from “strongly disagree“ to “strongly agree.” The statements relate to a range of aspects of system use, such as complexity, ease-of-use, and learnability. Each participant’s responses are then scored, providing an overall SUS score between 0 and 100. After this stage, the overall usability of the system is evaluated via a general question, “Overall, I would rate the user-friendliness of this system as,” with the following options: “worst imaginable,” “awful,” “poor,” “fair,” “good,” “excellent,” and “best imaginable.” An additional question (“Have you found difficulties in some part of the system?”) was also presented. In the case of a positive answer, a textual description of the difficulties was requested.

    Collecting usage data involved storing browser information and logging user interactions with the system. Details of the browser were collected via the JavaScript Navigator Object. This object is not intrusive, is supported by all major browsers, and contains information such as browser name, platform, version, and language.

    In terms of logging user interactions, the system logged timestamps on actions completed during the completion of the FFQ (eg, opening and closing of the portion size selection screen) using the JavaScript Date Object, which contains the time in milliseconds since the beginning of the year 1970 [34]. The timestamps were analyzed for the total time spent completing the FFQ, calculated based on the first and last click interaction with the FFQ. As the system allowed users to stop partway through the FFQ and to return to it within the same day, in order to measure only the periods in which the volunteers were actively engaged in using the system, time intervals greater than 60 seconds without any click interactions were considered interruptions (ie, period of inactivity) and subtracted from the total completion time.

    Figure 1. Food items and frequencies presented by the system.
    View this figure
    Figure 2. Portion sizes presented by the system.
    View this figure

    The EatWellQ8 Web-based food frequency questionnaire was deployed in January 2016 as part of a validation study comparing the online FFQ against a preexisting paper version of a Kuwaiti FFQ and a 4-day weighed food record. The study was subject to ethical review according to the procedures specified by the University of Reading Research Ethics Committee (UREC 15/50) and by the Diabetes Institute’s International Scientific Advisory Board and Ethics Review Committee (RA-2015-018) and was given favorable ethical opinions for conduct.

    Because the usability study was being performed in parallel with the EatWellQ8 validation study, participant recruitment and eligibility criteria were set by the requirements of the wider study. Participants were recruited in Kuwait as part of the EatWellQ8 study without financial incentive. Recruitment was conducted via the Internet, posters, and social media or word of mouth, mainly from the higher education institutions in Kuwait, during 2016. Volunteers were requested to create an online account on the study website and to complete a screening questionnaire to determine their eligibility to participate in the study. Participants with chronic diseases (eg, diabetes), food allergies or food intolerances, or not within the age range (18-65 years) were not eligible to participate in the study.

    Figure 3. Food frequency questionnaire completion time for all participants (n=163) and by gender (93 female, 49 male).
    View this figure

    Results

    A total of 235 participants created accounts in the system, of which 163 completed the FFQ. Of those 163 participants, 142 reported their gender (93 female, 49 male) and 144 reported their date of birth (mean age of 35 years, range from 18-65 years).

    Regarding the devices participants used to complete the FFQ, 69 participants used a laptop/desktop computer, 87 used a smartphone, 4 used a tablet, and 3 devices/browsers did not return their JavaScript Navigator Object correctly and hence the device information is not available.

    The mean completion time for all FFQs (n=163), excluding periods of interruption, was 14.2 minutes (95% CI 13.3-15.1 minutes). Female participants (n=93) completed in 14.1 minutes (95% CI 12.9-15.3 minutes) and male participants (n=49) completed in 14.3 minutes (95% CI 12.6-15.9 minutes) (Figure 3). Participants using laptops or desktops (n=69) completed the FFQ in an average of 13.9 minutes (95% CI 12.6-15.1 minutes) and participants using smartphones or tablets (n=91) completed in an average of 14.5 minutes (95% CI 13.2-15.8 minutes) (Figure 4). Out of the 163 FFQs, 71 were completed without any interruptions (ie, there was no gap of more than 60 seconds without any interaction). Considering the 146 food items, the volunteers spent on average 5.84 seconds per food item. As the system collects timestamps just before the portion image presentation (ie, after the frequency selection) and when they are selected (ie, click on the portion image), it was possible to calculate the mean time spent in the portion size selection (4.18 seconds per food item) and by subtraction the rest of the time (1.66 seconds per food item) was considered spent on the frequency selection component of the task. For items where the frequency was “never,” no explicit selection was required.

    Regarding the portion size selection, we did not have the timestamp required to separate the time required for image loading from the time required by participants to decide on and select a photo due to the fact that this information cannot be captured by the Web app. However, informal testing with a good Internet connection showed that the pop-up is rendered with the 3 images (around 150 KB in total) in less than 1 second.

    For all participants, the usability survey was presented after completion of the FFQ. Of the 141 who elected to complete the usability survey, 125 reported their gender (80 female, 45 male) and 124 reported their date of birth (mean age of 36 years, range from 18-65 years). The median SUS score (n=141) was 75.0 (interquartile range [IQR] 12.5) for all the participants, and of the 125 who reported their gender, the results were 72.5 (IQR 12.5) for female (n=80) and 75 (IQR 11.25) for male (n=45) (Figure 5). Products with a SUS score higher than 70 are considered to be good [35-36]; this is discussed further in the Discussion section. No significant differences were found between genders or age groups (below and above the median) for SUS or completion time. Considering only participants using smartphones or tablets (n=80), the median was 72.5 (IQR 12.5), slightly below the SUS median for desktops and laptops (n=58), which was 75.0 (IQR 12.5). Users’ ratings on the overall user-friendliness of the system (based on the question “Overall, I would rate the user-friendliness of this system as”) were predominantly “good” and “excellent” (Figure 6).

    In the final question (“Have you found difficulties in some part of the system?”), 126 volunteers answered “no” and 15 answered “yes.” Further examination of the participants who provided comments (n=13) showed that their responses were more related to the process (eg, “too long and detailed,” “repeated questions,” “gets boring,” and “time consuming”) rather than fundamental problems with the system. Only 3 participants reported fundamental problems and they were related to the portion size pop-up in smartphones. Selected comments related to the usability of the system follow:

    The portion size pop-up aspect of the FFQ became a bit tedious. I think it might be slightly more user-friendly if the portion pictures are posted on the website rather than in pop-up form.
    The pictures were great and really were on spot with the amounts difference.
    It was not clear for me when choosing the portion/size if there was more than a, b, and c. By using mobile it was not easy at all to scroll down the size option.
    Figure 4. Food frequency questionnaire completion time for all devices (n=163) and by device (69 laptops/desktops, 91 smartphones/tablets).
    View this figure
    Figure 5. System Usability Scale of the food frequency assessment system by the study participants (n=141) and presented by female (n=80) and male (n=45) for those who reported gender (n=125).
    View this figure
    Figure 6. Overall user evaluation of the food frequency assessment system by the study participants (n=141).
    View this figure

    Discussion

    Principal Findings

    Participants gave the EatWellQ8 system a median SUS score of 75.0 (IQR 12.5). Kurtom and Bangor measured popular services and products and reported a SUS average of 70.14, including Microsoft Excel (54.4), Amazon (79.0), and an automated teller machine (80.5) [35-36]. Products with a SUS score higher than 70 are considered to be good [35]. When using this scale, it is useful to compare results within the same category. A very recent study published the SUS results of an online 24-hour recall system designed and developed during the project myfood24 [37]. For an adult population, it resulted in a SUS median of 68 (IQR 40) for the beta version, and a SUS median of 80 (IQR 25) for the live version. No similar results have been published for online FFQs, but the SUS median from this study indicates good design and user acceptability. We acknowledge potential for selection bias, which could not be quantified. This is further supported by participants’ positive responses relating to the overall quality of the system (Figure 6). We observed similar completion times and SUS medians for completing the FFQ on smartphones/tablets when compared with laptops/desktops, which indicates a good responsive design.

    Although retrospective dietary assessment methods such as the FFQ and 24-hour recall require less effort from users than prospective methods using similar technologies (eg, Web-based food diaries), completion times of around 14 minutes for completing the FFQ in full can still be a barrier if participants are not engaged with the study objectives. The challenge of engaging participants to complete data collection could potentially be addressed by providing personalized online feedback, acting as a reward to incentivize participants to complete the FFQ. A newer version of the EatWellQ8 system is currently under development with the ability to provide personalized feedback, which may further improve user satisfaction and interest for investing this amount of time to complete the FFQ.

    Conclusions

    We have designed and deployed an online FFQ in a way that encourages reproducibility and is available to be used in other studies, using the same cloud services, for free. In this way, we hope to make a contribution to the wider uptake of digital FFQs and to make more widely accessible their benefits in terms of time and resource savings and suitability to support large-scale studies.

    The FFQ we have developed is a responsive website that has been tested on smartphones and tablets using two major mobile operating systems (iOS and Android). It addresses security requirements using features provided by Google Firebase, a cloud-based real-time database service. The user rating of this version from 141 participants was good (75 out of 100, using the SUS), and the completion time calculated from 163 FFQs (14.2 minutes) seems to be acceptable but with room for improvement. This paper is an important landmark in encouraging the research community to publish technical designs and usability information of online dietary assessment methods.

    Acknowledgments

    The authors thank the participants in this study for their time and feedback. Thanks to the Food4Me (food4me.org) consortium for use of their portion size photographs. Thanks to the Dasman Diabetes Institute for their collaboration during the study in Kuwait. Rodrigo Zenun Franco is sponsored by CNPq (National Counsel of Technological and Scientific Development) from the Brazilian government via the Science without Borders program.

    Authors' Contributions

    All authors contributed to the study conceptualization and design. BAA was responsible for the EatWellQ8 ethical approval and recruitment. RZF was responsible for system design, development and data analysis, and article draft. RF, JAL, and FH were responsible for the final writing, critical revision, and approval of the paper. The research data from this study can be accessed by contacting the corresponding author.

    Conflicts of Interest

    None declared.

    References

    1. Bingham SA, Gill C, Welch A, Day K, Cassidy A, Khaw KT, et al. Comparison of dietary assessment methods in nutritional epidemiology: weighed records v. 24 h recalls, food-frequency questionnaires and estimated-diet records. BJN 2007 Mar 9;72(04):619-643. [CrossRef] [Medline]
    2. Mulligan AA, Luben RN, Bhaniani A, Parry-Smith DJ, O'Connor L, Khawaja AP, et al. A new tool for converting food frequency questionnaire data into nutrient and food group values: FETA research methods and availability. BMJ Open 2014 Mar 27;4(3):e004503 [FREE Full text] [CrossRef] [Medline]
    3. Kristal AR, Kolar AS, Fisher JL, Plascak JJ, Stumbo PJ, Weiss R, et al. Evaluation of Web-based, self-administered, graphical food frequency questionnaire. J Acad Nutr Diet 2014 Apr;114(4):613-621 [FREE Full text] [CrossRef] [Medline]
    4. Fallaize R, Forster H, Macready AL, Walsh MC, Mathers JC, Brennan L, et al. Online dietary intake estimation: reproducibility and validity of the Food4Me food frequency questionnaire against a 4-day weighed food record. J Med Internet Res 2014;16(8):e190 [FREE Full text] [CrossRef] [Medline]
    5. González CR, Bayo MJ, Meneu BT, García SP, Martínez-Monzó J. Design of a self-administered online food frequency questionnaire (FFQ) to assess dietary intake among university population. Nutr Hosp 2011;26(6):1440-1446 [FREE Full text] [CrossRef] [Medline]
    6. Welch AA, Luben R, Khaw KT, Bingham SA. The CAFE computer program for nutritional analysis of the EPIC-Norfolk food frequency questionnaire and identification of extreme nutrient values. J Hum Nutr Diet 2005 Apr;18(2):99-116. [CrossRef] [Medline]
    7. Labonté M, Cyr A, Baril-Gravel L, Royer M, Lamarche B. Validity and reproducibility of a Web-based, self-administered food frequency questionnaire. Eur J Clin Nutr 2012 Feb;66(2):166-173. [Medline]
    8. Verger EO, Armstrong P, Nielsen T, Chakaroun R, Aron-Wisnewsky J, Gøbel RJ, MetaCardis Consortium. Dietary assessment in the MetaCardis Study: development and relative validity of an online food frequency questionnaire. J Acad Nutr Diet 2016 Dec 23. [CrossRef] [Medline]
    9. Delisle NC, Henriksson H, Alexandrou C, Bergström A, Bonn S, Bälter K, et al. Validation of an online food frequency questionnaire against doubly labelled water and 24 h dietary recalls in pre-school children. Nutrients 2017 Jan 13;9(1) [FREE Full text] [CrossRef] [Medline]
    10. Vereecken CA, De Bourdeaudhuij I, Maes L. The HELENA online food frequency questionnaire: reproducibility and comparison with four 24-h recalls in Belgian-Flemish adolescents. Eur J Clin Nutr 2010 May;64(5):541-548. [CrossRef] [Medline]
    11. Matthys C, Pynaert I, De KW, De HS. Validity and reproducibility of an adolescent Web-based food frequency questionnaire. J Am Diet Assoc 2007 Apr;107(4):605-610. [CrossRef] [Medline]
    12. Beasley JM, Davis A, Riley WT. Evaluation of a Web-based, pictorial diet history questionnaire. Public Health Nutr 2009 May;12(5):651-659 [FREE Full text] [CrossRef] [Medline]
    13. Apovian CM, Murphy MC, Cullum-Dugan D, Lin P, Gilbert KM, Coffman G, et al. Validation of a Web-based dietary questionnaire designed for the DASH (dietary approaches to stop hypertension) diet: the DASH online questionnaire. Public Health Nutr 2010 May;13(5):615-622 [FREE Full text] [CrossRef] [Medline]
    14. Willett W. Food frequency methods. In: Nutritional Epidemiology (Monographs in Epidemiology and Biostatistics). New York: Oxford University Press; 2012.
    15. Falomir Z, Arregui M, Madueño F, Corella D, Coltell O. Automation of food questionnaires in medical studies: a state-of-the-art review and future prospects. Comput Biol Med 2012 Oct;42(10):964-974. [CrossRef] [Medline]
    16. Kowalkowska J, Slowinska MA, Slowinski D, Dlugosz A, Niedzwiedzka E, Wadolowska L. Comparison of a full food-frequency questionnaire with the three-day unweighted food records in young Polish adult women: implications for dietary assessment. Nutrients 2013 Jul 19;5(7):2747-2776 [FREE Full text] [CrossRef] [Medline]
    17. Cade JE, Burley VJ, Warm DL, Thompson RL, Margetts BM. Food-frequency questionnaires: a review of their design, validation and utilisation. Nutr Res Rev 2004 Jun;17(1):5-22. [CrossRef] [Medline]
    18. Benedik E, Koroušić SB, Simčič M, Rogelj I, Bratanič B, Ding EL, et al. Comparison of paper- and Web-based dietary records: a pilot study. Ann Nutr Metab 2014;64(2):156-166. [CrossRef] [Medline]
    19. Delve J, Simpson E, Adamson AJ, Poliakov I, Olivier P, Foster E. Comparison of INTAKE24 (an online 24hr dietary recall tool) with an interviewer-led 24 hr recall method in 11–16 year olds. Proc Nutr Soc 2015 Apr 15;74(OCE1). [CrossRef]
    20. Subar AF, Kirkpatrick SI, Mittl B, Zimmerman TP, Thompson FE, Bingley C, et al. The automated self-administered 24-hour dietary recall (ASA24): a resource for researchers, clinicians, and educators from the National Cancer Institute. J Acad Nutr Diet 2012 Aug;112(8):1134-1137 [FREE Full text] [CrossRef] [Medline]
    21. Liu B, Young H, Crowe FL, Benson VS, Spencer EA, Key TJ, et al. Development and evaluation of the Oxford WebQ, a low-cost, web-based method for assessment of previous 24 h dietary intakes in large-scale prospective studies. Public Health Nutr 2011 Nov;14(11):1998-2005. [CrossRef] [Medline]
    22. Celis-Morales C, Livingstone KM, Marsaux CF, Macready AL, Fallaize R, O'Donovan CB, Food4Me Study. Effect of personalized nutrition on health-related behaviour change: evidence from the Food4me European randomized controlled trial. Int J Epidemiol 2016 Aug 14. [CrossRef] [Medline]
    23. EatWellQ8 Study Website. 2016.   URL: https://eatwellq8.org/ [accessed 2017-01-09] [WebCite Cache]
    24. Deloitte. There’s no place like phone: half of Irish consumers check smartphones in the middle of the night. 2016.   URL: https://www2.deloitte.com/ie/en/pages/about-deloitte/articles/Theres-no-place-like-phone.html [accessed 2017-04-27] [WebCite Cache]
    25. Balasubramanee V, Wimalasena C, Singh R, Pierce M. Twitter bootstrap and AngularJS: Frontend frameworks to expedite science gateway development. 2013 Presented at: IEEE International Conference on Cluster Computing (CLUSTER); 2013; Indianapolis. [CrossRef]
    26. Google. AngularJS API Docs. 2017.   URL: https://docs.angularjs.org/api [accessed 2017-01-09] [WebCite Cache]
    27. Google. Google Firebase Features. 2017.   URL: https://firebase.google.com/features/ [accessed 2017-01-09] [WebCite Cache]
    28. Bootstrap. The world's most popular mobile-first and responsive front-end framework. 2017.   URL: http://getbootstrap.com/ [accessed 2017-01-09] [WebCite Cache]
    29. Bingham SA, Welch AA, McTaggart A, Mulligan AA, Runswick SA, Luben R, et al. Nutritional methods in the European Prospective Investigation of Cancer in Norfolk. Public Health Nutr 2001 Jun;4(3):847-858. [Medline]
    30. Timon CM, Forster SE, Barker ME, Godfrey AJ, Hwang F, Williams EA. A comparison of younger v. older adults' ability to estimate food portion sizes. Proc Nutr Soc 2011 Aug 15;70(OCE3). [CrossRef]
    31. Forster H, Fallaize R, Gallagher C, O'Donovan CB, Woolhead C, Walsh MC, et al. Online dietary intake estimation: the Food4Me food frequency questionnaire. J Med Internet Res 2014 Jun 09;16(6):e150 [FREE Full text] [CrossRef] [Medline]
    32. Finstad K. The System Usability Scale and non-native English speakers. J Usability Stud 2006;1(4):185-188.
    33. Brooke J. SUS—a quick and dirty usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability Evaluation in Industry. Boca Raton: CRC Press; 1996:4-7.
    34. ECMA International. ECMAScript 2015 Language Specification—ECMA-262 6th Edition. 2015.   URL: http://www.ecma-international.org/ecma-262/6.0/ [accessed 2017-01-09] [WebCite Cache]
    35. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum-Comput Int 2008 Jul 30;24(6):574-594. [CrossRef]
    36. Kortum PT, Bangor A. Usability ratings for everyday products measured with the system usability scale. Int J Hum-Comput Int 2013 Jan;29(2):67-76. [CrossRef]
    37. Carter MC, Albar SA, Morris MA, Mulla UZ, Hancock N, Evans CE, myfood24 Consortium. Development of a UK online 24-h dietary assessment tool: myfood24. Nutrients 2015 May 27;7(6):4016-4032 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    EatWellQ8: Eat Well Kuwait
    FFQ: food frequency questionnaire
    IQR: interquartile range
    JSON: JavaScript Object Notation
    SPA: single-page application
    SUS: System Usability Scale


    Edited by G Eysenbach; submitted 09.01.17; peer-reviewed by M Hoevenaar-Blom, V Andreeva; comments to author 22.03.17; revised version received 04.04.17; accepted 06.04.17; published 08.05.17

    ©Rodrigo Zenun Franco, Balqees Alawadhi, Rosalind Fallaize, Julie A Lovegrove, Faustina Hwang. Originally published in JMIR Human Factors (http://humanfactors.jmir.org), 08.05.2017.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on http://humanfactors.jmir.org, as well as this copyright and license information must be included.