Background: Digital health research encompasses methods from human-computer interaction and health research.
Objective: This paper aims to describe how these methods were combined to develop HeLP-Diabetes: Starting Out, a web-based structured education program for people newly diagnosed with type 2 diabetes.
Methods: The development process consisted of three phases: initial design for effectiveness, optimization for usability, and in the wild testing in the National Health Service with people newly diagnosed with type 2 diabetes, and further revisions. We adopted an iterative user-centered approach and followed steps from the human-computer interaction design life cycle and the Medical Research Council guidelines on developing and evaluating complex interventions.
Results: The initial design process resulted in an 8-session program containing information and behavior change techniques targeting weight loss, being more active, and taking medication. The usability testing was highlighted at an early stage, where changes needed to be made to the language and layout of the program. The in the wild testing provided data on uptake of and barriers to use. The study suggested low uptake and completion of the program, but those who used it seemed to benefit from it. The qualitative findings suggested that barriers to use included an expectation that the program would take too long. This informed refinements to the program.
Conclusions: The use of interdisciplinary methods resulted in an iterative development process and refinements to the program that were based on user needs and data on uptake. The final intervention was more suitable for a definitive evaluation than the initial version. The description of our approach informs other digital health researchers on how to make interventions more sensitive to user needs.
Interdisciplinary Research Methods
Research on digital health interventions (DHIs) brings together the human-computer interaction (HCI; which includes software engineering) and health (encompassing biomedical, behavioral, and social sciences). The research methods used in HCI, such as health research, are largely empirical (eg, experimental designs, surveys, and focus groups). However, health research tends to use a sequential approach, based on the methods used in pharmacological drug development, culminating in a randomized controlled trial to determine its effectiveness . In HCI research, there is more emphasis on proximal (interaction) and distal (effects) outcomes, and the need to iteratively design and test an intervention until it is deemed to be accessible and useful by the user [ - ]. Acceptability and usability are crucial to digital health researchers, because the effectiveness of DHIs relies on being used (at the individual level), and the population impact depends on reaching a high proportion of the target population. The use of iterative methods common to HCI allows DHIs to be optimized until they are likely to achieve sufficient acceptability to ensure adequate reach, uptake, and use to achieve effectiveness and cost-effectiveness [ ]. A decision about whether to proceed to a definitive randomized trial can then be made.
The Medical Research Council (MRC) has published guidelines for health researchers researching complex interventions to help them adopt appropriate methods. The 2006 MRC framework suggested a nonlinear approach to the development and evaluation of complex interventions, with four key stages: (1) development, (2) feasibility and piloting, (3) evaluation, and (4) implementation .
These 4 stages involve using evidence and theory to develop complex interventions, then testing them with a series of pilot studies aimed at key design uncertainties, before moving on to an exploratory and then a definitive evaluation . MRC best practice guidelines is that definitive evaluation should only be undertaken once (1) the intervention and its delivery package reach a degree of stability, (2) any further development would be relatively minor, (3) there is reasonable confidence that the intervention could be implemented with high fidelity, and (4) there is a reasonable likelihood that the intervention will lead to improved health outcomes or equivalent outcomes at lower cost [ ].
Iterative development and evaluation are also features of the HCI development life cycles. A life cycle is the sequence of activities that occurs from the initial concept, through to the eventual phasing out and replacement . The process purposefully cycles through several designs, incrementally improving the design until the final product is reached [ ]. A key aspect is to be user-centered and involve users throughout the design process. This allows designers to understand people in the contexts in which they live, work, and learn, and consequently how to design products that fit easily into users’ everyday lives [ ]. Standard frameworks for HCI and usability have been developed that recommend an iterative design process with an emphasis on the continuous identification of user requirements, testing the intervention against these requirements, respecifying user requirements, and retesting [ , ]. These processes inform each other and are repeated in each design cycle ( ).
The challenges of interdisciplinary work across HCI and health have been highlighted by Pagliari  and Blandford et al [ ]. One of the key issues is that, although both the MRC and the HCI life cycle approaches are iterative, the HCI life cycle is located entirely in the development phase of the MRC framework, as illustrated in .
This paper focuses on this development phase, with the aim of highlighting the importance of careful user-centered design and early-stage usability testing before further evaluation. We have reported how we tackled this, by describing the 3 phases of development and describing the methods from HCI and the methods from health that we incorporated at each phase. The aim of reporting our methods is to guide other researchers in developing similar interventions. Historically, complex interventions, such as diabetes self-management programs, have not been well described [, ]. Better reporting would improve the understanding of causal mechanisms and increase the collective knowledge of how and why interventions work or not. This would, in turn, facilitate learning among researchers and the development of more effective interventions [ ].
Our approach was also informed by guidelines on evaluating DHIs , which defines the research questions forming the basis of an evaluation of a DHI, and the issues that are particularly salient to DHIs rather than complex interventions as a whole. The guidelines on DHI evaluation was particularly relevant during the early stages of design when we needed to identify the health needs, target population, and causal model for the intervention.
There are other examples where the process of combining interdisciplinary methods have been documented [- ]. The person-based approach developed by Yardley et al [ ], for example, reports a development process involving qualitative interviews with a wide range of people from the target user population at every stage. Insights from users are then used to modify the intervention to make it more persuasive, feasible, and relevant. In keeping with the person-based approach, we collected insights from users and used these insights to modify the intervention at both the usability testing and in the wild testing stages.
Background to the Intervention
Guidelines for evaluating DHIs recommend starting by defining the problem to be addressed, namely the health need that the DHI is intended to address and the population who could benefit from the DHI. For HeLP-Diabetes: Starting Out (HDSO), the health need that is being addressed is the provision of a structured education for people with type 2 diabetes mellitus (T2DM). T2DM is an international priority, affecting approximately 425 million people worldwide. T2DM places a considerable burden on patients in terms of premature morbidity and mortality and on health services, in terms of cost. Both these burdens can be reduced by structured self-management education, which can improve patient knowledge, self-care behaviors, metabolic control, psychological outcomes, and health care costs [- ]. In the United Kingdom, T2DM affects an estimated 3.8 million people aged >16 years (8.6% of the population of this age group) [ ] and accounts for approximately 10% of the total National Health Service (NHS) budget [ ].
It is an NHS policy that all patients diagnosed with T2DM are offered structured education . General practitioners (GPs) in England are remunerated through the Quality and Outcomes Framework (QOF) for referring newly diagnosed patients to suitable programs, with the suitability of the program determined by the accreditation by the Quality Institute for Self-Management Education (QISMET) [ , ]. Despite this incentivization, the uptake of structured education is poor (8.3% uptake in 2016 [ ]). The reasons for this low uptake include difficulties with the current dominant model of structured education which is group-based and can be difficult for people who work, have caring responsibilities, or dislike groups. Our team had already developed a web-based self-management program (HeLP-Diabetes) [ ], which was shown to be effective and cost-effective [ , ]. HeLP-Diabetes is a website with over 560 pages that provide self-management support for patients from diagnosis to death. The content is broken down into 8 sections, including information about understanding and treating diabetes, an interactive health record, news and research, and a forum and help page [ ]. Engagement was also encouraged with regular emails and text that contained links to topical content within the website (eg, information regarding influenza vaccinations in winter) [ , ].
However, QOF payment and QISMET accreditation require a structured program (with a clear curriculum and learning goals and modules to work through in a linear fashion) aimed at newly diagnosed patients. HeLP-Diabetes was not structured (people have access to the website, without following a linear pathway), and it was not aimed at newly diagnosed patients but at patients at all stages of their diabetes journey. Therefore, we decided to develop a web-based structured course that could gain QISMET accreditation and meet the QOF requirements. The established courses that the GPs could refer patients to and gain QOF remuneration were all group-based and face-to-face; thus, a web-based structured course would provide an alternative that could potentially bypass some of the barriers to uptake described earlier.illustrates the key differences between the HeLP-Diabetes website and the HDSO-structured course described in this paper.
|Feature||HeLP-Diabetes||HeLP-Diabetes: Starting Out|
|Target user||People with T2DMa at any stage||Newly diagnosed people with T2DM|
|Size||8 sections, with 560 pages||5 sections, with selected content from HeLP-Diabetes|
|How the intervention was delivered||Nonlinear—people could access any part of the website and dip in and out as they pleased||Linear—people worked through modules one by one, and were given access to the next module once they completed the previous one|
|Curriculum||No curriculum—a wide breadth of information was available, and people could choose which topics to access depending on interest||Spiral curriculum—people worked through a series of modules and added to the knowledge they gained from previous modules in a spiral fashion|
aT2DM: Type 2 diabetes mellitus.
The interdisciplinary development process required the skills of a multidisciplinary team of patients, GPs, diabetes nurse specialists, and health and HCI researchers. KP and JR formed part of the HeLP-Diabetes team, and KP and SP formed part of the HDSO team. EM led both teams.
Aim and Objectives
This paper aims to describe how we combined the methods described in the MRC and HCI guidelines, using the development of the HDSO program as a worked example. The three stages of development we undertook were as follows: (1) phase 1—initial design, (2) phase 2—optimizing for usability, and (3) phase 3—in the wild testing and further revisions.
For each stage of development, we have described the methods from HCI, the methods from health, and how we combined the two. The evaluation of the final intervention for feasibility, acceptability, and impact is described elsewhere .
Phase 1: Design for Effectiveness
Methods From HCI: Establishing User Requirements for HeLP-Diabetes as a Precursor for HDSO
The first steps of the HCI design process involve understanding the user context and requirements. This took place during the development of the HeLP-Diabetes website, before the development of the HDSO-structured program.
Understanding the contexts in which people live, work, and learn allows designers to develop products that fit easily into users’ everyday lives. Products that are easy to use are more likely to be acceptable to patients and more widely taken up. Extensive work in establishing the requirements of patients with T2DM went into the development of HeLP-Diabetes, the precursor to HDSO, and has been reported by Dack et al . User requirements were conceptualized as features that would make people want to use the interventions (wants) and features needed to help improve health outcomes (needs). The HeLP-Diabetes team conducted focus groups with patients and health professionals (health professionals facilitated engagement with the program) to collect this information.
The content identified as necessary in the focus groups was integrated by the design team and then reviewed by a participatory design group consisting of patients with T2DM. The content went through several iterations and was put through usability testing. Usability testing is commonly used in software engineering and HCI research. It has been described as “representative users attempting representative tasks in representative environments, on early prototypes or working versions of computer interfaces” . Usability testing aims to find flaws in the interface that need improvement and to make products more sensitive to users’ needs at an early stage of development [ ]. There is a wide range of techniques used in usability testing, including questionnaires, think-aloud observation, and interview-based techniques [ ]. Usability testing for HeLP-Diabetes involved users thinking aloud while undertaking prespecified tasks (eg, finding specific information or using one of the self-monitoring tools), a technique common and unique to HCI [ ]. Usability testing helped to optimize the navigation and interactive features of HeLP-Diabetes. Selected content from HeLP-Diabetes were used to develop the HDSO program, informed by evidence, theory, and modeling.
Methods From Health
Systematic reviews of web-based diabetes self-management interventions [- ] have found that the most effective components are (1) prompting of self-monitoring of behavioral outcomes, (2) provision of information on consequences of behavior, (3) barrier to identification or problem solving, (4) feedback on performance, and (5) interaction with health care professionals via the internet [ , , ]. This evidence was combined with the theory regarding long-term condition self-management to determine the necessary components of the program. This theory is discussed in the next section.
Theory and Causal Modeling
The aim of the structured program was not only to impart knowledge but also to empower and encourage people newly diagnosed with T2DM to improve their self-efficacy (self-confidence in self-management) and emotional well-being by learning about living a healthy lifestyle, making the most of the NHS and staying motivated. There were many theories and theoretical models which related to the aims of the program. These included the Corbin and Strauss model for the work of living with a long-term condition  and behavior change theories. The Corbin and Strauss model was chosen because of its holistic approach to diabetes self-management and fit with education theory about multidimensional learning. Increasingly, learning has been construed as being multidimensional and involving the body, emotions, spirit, and the mind [ ]. The Corbin and Strauss model for the work of living with a long-term condition also emphasizes the need to address the emotional aspects of disease and identity issues. Corbin and Strauss identified 3 sets of tasks involved in self-management [ ] from qualitative work on the perception of patients about their long-term conditions. These are conceptualized as follows [ ]:
- Medical management: adopting healthy behaviors (eg, not smoking, exercising regularly, and eating healthy food), working with health professionals (eg, keeping appointments and following instructions), and taking medicines.
- Emotional management: addressing the negative emotions associated with being diagnosed with a long-term condition.
- Role management: coming to terms with the disruption to one’s sense of self, including adjusting to the patient role and managing the impact of one’s diagnosis on relationships with friends, family, and colleagues.
Behavior change theories were used because they can help predict how and when behavior change occurs . Behavior change techniques (BCTs) are the strategies used in an intervention to promote behavior change [ ]. They can be designed using behavior change theories. Interventions that use more theory-based BCTs have been found to have larger effect sizes compared with interventions that use fewer techniques in studies of digital health behavior change interventions [ ].
Guidelines on evaluating DHIs  recommends identifying the necessary components of an intervention (including BCTs) by establishing a credible causal pathway for the intervention; thereby, linking evidence and theory to the intended outcomes. We linked the 3 self-management tasks identified in the Corbin and Strauss model to the intended outcomes of the intervention (improved knowledge, self-efficacy, and emotional well-being) using a causal modeling approach. The causal model for HDSO is illustrated in .
In the long term, the proximal outcomes of knowledge, motivation, self-efficacy, and emotional distress combine to enable behavior change and better glycemic control. We opted not to measure long-term outcomes but focus on the short-term outcomes, including registration, use, and change in these 3 proximal outcomes.
How HCI and Health Methods Were Combined
Findings from the work on user requirements (HCI) and evidence, theory, and causal modeling of the intervention (health) were combined to select appropriate BCTs and develop the content, format, and structure of the program.
Phase 2: Optimize for Usability
Methods From HCI
The usability testing of the HDSO program was conducted using questionnaires emailed to 5 patient volunteers. The questionnaires were written by the GPs and diabetes specialist nurses in the HDSO team and included the following four items: (1) the title of the session (eg, easy to understand and relevant to content), (2) the information contained in the session (eg, appropriate quantity of information, encouraging tone, and relevant links to information from other sources), (3) the visual design of the session (eg, readability of font, ease of finding access to videos, and the next section of the program), and (4) any other specific suggestions.
Methods From Health
Usability testing formed part of the early phase work recommended by the MRC and was undertaken to test the initial design with target users, before the exploratory in the wild testing with patients in the NHS.
How HCI and Health Methods Were Combined
Combining the phased approach advocated by the MRC and making changes to the program based on the results of 1 round of usability testing meant that the early development of the program was iterative. This was a key strength of this research. We did not immediately proceed to evaluation after the initial design of the program but undertook cycles of testing, refinement, retesting, and further refinement until we were more confident that the intervention fulfilled user needs and did not need major changes. This was recommended by both the MRC guidelines used by health researchers and the HCI lifecycle models used by HCI researchers. The refinements made to the design of the program were based on the results of the usability testing (and therefore based on user needs and experience).
The purpose of the next stage of testing and refinement was to evaluate the design of the program against user requirements. This was undertaken using in the wild testing.
Phase 3: In the Wild Testing and Further Revisions
Methods From HCI
Research in the wild is a term used for research conducted in natural settings. It is increasingly used in HCI to understand how people react to and integrate technologies in their everyday lives over a period [, ]. In situ studies are more likely to reveal the behaviors people adopt and the problems they encounter when they use an intervention at home, at work, or elsewhere. The advantage of this is that they provide greater external validity than experimental studies, where participants are more aware of how they are expected to behave. Another advantage is that in the wild studies nearly always provide unexpected findings regarding what humans do when confronted with a digital intervention; these can be the most informative findings [ ]. In addition to usability testing, this provided an extra way of testing interventions against user requirements (as suggested in the HCI design cycle).
Methods From Health
The aim of the in the wild testing was to understand more about how patients were using the HDSO program in their everyday lives, including their experiences and views of the problems they encountered in using the program. Therefore, mixed methods were used in this study. Quantitative data were collected on the number of patients registering for and completing the program, patient characteristics, and changes in questionnaire scores. The questionnaires administered were the Problem Areas in Diabetes measuring diabetes-related distress , and the diabetes management self-efficacy questionnaire [ ]. Questionnaires were included in the web-based program at the start and end of the course. Qualitative methods were used to explore the patient experiences and views regarding using the program.
Qualitative methods are used in both health and HCI research, but there are some important differences in the approaches used. For example, the locus of expertise differs. In health and social science research, researchers typically start with their own expertise rather than the user’s expertise, and the design interventions that (it is hoped that) users will engage with . In HCI research, the user is assumed to be an expert in what they do and what they need. Digital health research has adopted more user-centered approaches to address the challenge of low uptake and adherence, with a focus on understanding and accommodating the perspectives of the people who will use the intervention [ ]. In health and social science, a less formative (developmental) and more summative (cumulative) approach is often taken, so that there is less focus on early outcomes in the developmental stages and more focus on the impact of the final intervention. Emphasis is placed on conducting interviews of sufficient depth and duration. Interviews in HCI research use methods that are more common to industry and are driven by time and resources. Rapid user experience studies with smaller sample sizes are conducted at several stages during product development.
How HCI and Health Methods Were Combined
Methods from HCI and health were combined to conduct the in the wild testing of the HDSO program. The setting for the in the wild testing was GP practices. This was the natural setting for this study as referrals to structured education for T2DM patients occurs in primary care. Practices in 2 London boroughs that had taken part in a HeLP-Diabetes implementation study and practices in 2 London boroughs that were interested in commissioning the HDSO program participated in the study. The program was offered to these practices for free as an alternative to established face-to-face diabetes structured education courses that were already commissioned.
The study was submitted to the Health Research Authority (HRA) for NHS Research Ethics Committee (REC) for review. Secondary analysis of information collected as part of normal care was excluded from the REC review by the HRA as long as the patients were not identifiable . Therefore, the collection of data on registrations, completed sessions, and questionnaire scores were permissible, as the data were automatically pseudonymized with a numerical identifier. Patients were informed on registration that anonymized data were collected by the program and used anonymously for ongoing service development.
A total of 15 practices agreed to offer the HDSO program to patients for the study. The program was offered to patients as an NHS service; therefore, there were no formal inclusion and exclusion criteria. Practices were informed that the target population of the intervention was adults (aged ≥18 years) with T2DM diagnosed in the last 9 months and asked to offer the program to everyone in this population. Practices were asked to identify eligible patients by running a search of the electronic medical records. Practices were sent registration packs to mail out to eligible patients. A total of 322 packs were mailed out. The registration pack contained information about the HDSO program (including that it was being offered as an NHS service as an alternative to face-to-face courses), how to register, and a reply slip. Patients interested in using the program returned a reply slip to the HDSO administrator with their contact details. Patients were then telephoned by the HDSO administrator who collected baseline demographic data (which was pseudonymized and added to the data collected automatically by the HDSO program) and created a username and password for the program. The HDSO administrator also confirmed whether they were happy to take part in research interviews and securely sent SP the ID numbers of all the patients who agreed. The username and password for the HDSO program were then emailed to the patient, along with information on who to contact if there were any problems.
We used qualitative telephone interviews with patients to explore their experience of using the program. We took an HCI approach of rapid data collection and used the data to inform optimization before further evaluation.
One of the members of the HDSO team (SP) contacted patients who registered for the HDSO program but did not start or complete it. We were unable to contact patients who did not register for the program, as they did not provide us with their details or consent to be contacted. The patients who did not start or complete the program were contacted, because we were particularly interested in the problems encountered with the program. The telephone calls were semistructured and lasted approximately 10 minutes. Questions included “What would help you to use the program more regularly?”
The interviews were carried out by telephone by SP, and written notes were taken rather than audio-recording and transcribing because the data needed to be collected and analyzed quickly to inform the program optimization. Note-taking is a recognized form of recording , and although it has the disadvantage of not capturing every word verbatim, the researcher mitigated this by noting down verbatim quotes where they were particularly pertinent.
Ethical approval was obtained from the HRA (reference number: 159488). Data on registrations, completed sessions, and questionnaire scores were excluded from the HRA REC review, because of a clause that states that secondary analysis of information collected as part of normal care is excluded from REC review by the HRA, as long as the patients are not identifiable .
Phase 1: Initial Intervention Components and Content
Establishing User Requirements for HeLP-Diabetes as a Precursor for HDSO
Results from the focus groups showed that patients needed help in managing the complexities of living with diabetes, such as managing the impact that irregular working hours had on diet and blood sugar, impact on relationships and social life, and support in dealing with the profound negative emotions caused by the diagnosis, which included anger, guilt, shame, and despair. The tools to help them manage these tasks included high quality, detailed information, personal stories from other people with similar experiences, and quizzes to test knowledge and provide feedback. Health professionals had similar perceptions of patient needs. Both patients and health professionals wanted HeLP-Diabetes to be interactive and visual (with quizzes, videos, and images), to be easy to use, and have a positive tone .
These results were combined with the results of the usability testing to create HeLP-Diabetes, a website containing 560 pages of information divided into 8 sections, which patients at any stage of their illness journey, could dip in and out of.
Content, Structure, and Format of HDSO
We used selected content from the HeLP-Diabetes website (informed by the causal modeling process) to construct the HDSO-structured program, which was needed to meet the QISMET and QOF requirements described in the introduction.
The causal modeling process helped us postulate that information and BCTs targeting healthy eating, weight loss, activity levels, smoking, alcohol consumption, and medication intake, would help users achieve the intended outcomes. Modules targeting these behaviors were therefore selected from the HeLP-Diabetes website to be integrated into the HDSO. The modules contained BCTs, including goal-setting, action-planning, self-monitoring, and feedback on performance . These BCTs are based on the self-regulation theory [ ], which states that our major self-regulative mechanism functions through (1) the self-monitoring of behavior, its determinants, and its effects; (2) the judgment of behavior concerning the person and place; and (3) effective self-reaction.
In addition to the BCTs from HeLP-Diabetes, self-assessment questionnaires and feedback were added as new components to the HDSO-structured course. The questionnaires assessed self-efficacy (self-confidence) in self-management, diabetes-related distress, and diabetes knowledge. These were positioned in the course in weeks 1 and 8 (before and after the program), thereby allowing users to reflect on the change in their scores.
Personalized emails were also added as new components to the HDSO-structured course. These were added to encourage motivation and engagement. A systematic review by Alkhaldi et al  on the effectiveness of prompts to increase digital interventions found that studies reported borderline small-to-moderate positive effects of technological strategies, including emails, to improve the use of interventions. Resource implications and mindfulness of our ultimate goal being HDSO delivered at scale across the NHS meant that emails were chosen as a cost and time-effective strategy for providing users with reminders.
A curriculum was needed to structure the content and components of the program and to achieve accreditation as a structured course. By identifying relevant theory (as suggested by the MRC guidelines), we decided that the program would follow a spiral curriculum based on the Harden and Stamper spiral curriculum model . This model proposes that there should be an “iterative revisiting of topics, subjects or themes throughout the course.” The idea is that topics are not just repeated, but that knowledge and understanding should be deepened each time. The learner’s competence should increase with each visit until the overall aim is achieved [ ].
The qualitative work that was conducted to establish user requirements for HeLP-Diabetes showed that users wanted information to be presented using text, images, and videos. These formats were therefore used to present information in the HDSO program and included videos of others living with diabetes. The text was written for people with a reading age of 12 to correspond with 80% of the population in the United Kingdom .
The result was an 8-session program containing information presented as text, images, and videos, and BCTs including goal-setting, action-planning, self-monitoring, and feedback on performance. Each session was designed to take approximately 40 to 50 minutes to complete and for people to complete 1 session per week. A screenshot of the HDSO program showing a video giving an introduction to T2DM is shown in. The 8 sessions of the program and each of their parts are listed in .
|Session title and content||Session components|
|Week 1—getting started|
|An introduction to diabetes||Information|
|Eating well for diabetes||Information|
|Taking control||Information and quizzes|
|Becoming more active||Information, physical activity goal-setting task, and videos of people’s stories|
|Handling feelings||Information and videos of people with diabetes|
|Week 3—improving my health and well-being|
|Protecting my body and mind||Information|
|Making changes||An exercise for reflecting on the quizzes in week 2, and setting SMARTa goals for diet, medication, activity, drinking, and other health behavior changes|
|Understanding my moods||Videos of people with diabetes|
|Working with diabetes||Information|
|Week 4—taking control of my diabetes|
|Making the most of the National Health Service||Information, videos of people talking about their interaction with the National Health Service, and a link to the health record in HeLP-Diabetes where users can record appointments|
|Update my goals and plans||A review of SMART goals set in week 3|
|Managing my moods||A reflection on the results of the mood quizzes in week 3, and a set of “Mood Tools,” including “Living Life to the Full,” a package developed by clinical psychologists using principles from cognitive behavioral therapy|
|My social life||Information and videos of people’s stories|
|Week 5—medication and lifestyle|
|Medication||Information, videos about the challenges and benefits of medications, and an interactive “My medicines” list|
|Review my goals and plans||A review and update of goals set in week 3|
|How to fix almost everything||An opportunity to revisit the mood tools used in week 4|
|Week 6—reducing my risks|
|Reducing the risks of heart attack and strokes||Information|
|Looking after my feet||Information|
|Review my goals and plans||An opportunity to review and update SMART goals|
|Living with diabetes||Videos of people talking about how they became used to having diabetes and an opportunity to revisit mood tools|
|Week 7—working with my health care team|
|My diabetes review||Videos about people’s experiences of diabetes care|
|Review my goals and plans||Review and update SMART goals|
|Week 8—celebrating success and planning for the future|
|Self-assessment||Opportunity to repeat the self-assessment questionnaires from week 1|
|Looking after my diabetes||Opportunity to prepare a care plan|
|Moving on: the end of the beginning||Information about staying motivated and reading about diabetes in the media|
aSMART: specific, measurable, achievable, realistic, and time bound.
Phase 2: Usability Testing
Responses to the questionnaires described earlier were reviewed by the team. The relevant changes were made where there was agreement among responses from the patient volunteers. Where there was disagreement in responses from the patient volunteers, the suggested changes were discussed among the team members, and a consensus decision was made as to whether to make the changes. Changes were made to the language and layout of each session of the program, including making text and titles clearer and easier to understand for users.
Examples of questionnaire responses and changes made to the program, as a result, are given in, demonstrating how the progression of the development of the program was grounded in user needs.
|Timeline||Questionnaire item||Response from patient volunteer||Design solutions|
|Week 1||Design||“Subtitles detailing content for each section would be helpful eg. Self-management 1 - Taking Control; Self-Management 2 - Getting Physical (Becoming more active); Self-management 3 - Handling Emotions (managing feelings?)”||Subtitles were added for each section of each module.|
|Week 2||Content||“First 3 pages good with 2 very useful video clips. Page 4 ‘Advice about increasing physical activity’ too much detail. Too many peoples stories at the end.”||Advice about physical activity was condensed, and the number of videos of people’s stories was reduced.|
|Week 3||Content||“Good, perhaps too much detail (contacts, addresses etc.) for sexual problems - could this be a website link? Level of detail might be off-putting for newly diagnosed.”||Contact details for support and advice organizations were removed and website links were added and signposted instead.|
We made changes to the design and content of the program based on the questionnaire responses. This ensured that the development of the program was grounded in user needs. The changes included clearer subtitling, advice about physical activity made more concise, a more appropriate number of videos of people’s stories included, and links to support and advice organizations added.
Phase 3: In the Wild Testing
During the study, 24 people registered for the HDSO program. Quantitative data were collected on program use, questionnaire scores, and characteristics. Of the 24 people registered, 3 (13%) people completed the program, 13 (54%) people started the program but did not complete it, and 8 (33%) people did not start the program. The data suggested low uptake and completion, but those who used it seemed to benefit from it (mean self-efficacy in self-management scores and diabetes knowledge scores increased).
The telephone interview responses were analyzed using a thematic analysis approach, and a list of barriers to completing the program emerged from the data. These are listed inwith illustrative quotes.
|Lack of time to start or complete the program|
|Expectation that completing the program would take too long|
|Ambivalence about starting|
|Feeling of content not being relevant to some users|
These themes were used to inform the refinement and optimization of the program as discussed in the next section. The ideas were followed up in subsequent interviews conducted as part of the evaluation of the final intervention and reported elsewhere .
Design Solutions Resulting From the In the Wild Testing
The themes identified from the interviews suggested that there were patient and program factors which influenced program use. Patient factors such as ambivalence were difficult to address. However, we were able to shorten the program and provide users with quicker access to the program with web-based registration.
Following a discussion among the HDSO team, the following changes were agreed upon:
- Reducing the number of sessions in the program: evidence from systematic reviews of engagement with digital behavior change interventions [ ] and research on adult web-based learning [ ] suggests that participants disengage if the intervention is perceived as too long or overly complicated. The decision about what content to retain and what to remove was made after discussions with the diabetes specialist nurses in the HDSO team who were trained educators and experienced in delivering face-to-face structured education courses. To determine what content to retain and what to remove, we discussed the data from the user experience interviews, reviewed the guidelines on T2DM management [ ], and examined the curriculum closely. The 8 sessions were cut down to 4 sessions, with a fifth bonus session available at the end. Despite comments about its irrelevance to newly diagnosed patients, all aspects of the management of T2DM were retained because it was considered important to give people a good overview and understanding of the types of treatment they might receive in the future. Topics including managing my diabetes when I’m ill, working with diabetes, and driving with diabetes were taken out of the main course and moved to the fifth bonus session. The final 4-session intervention is described elsewhere [ ] and contains the following sessions: getting started, self-management, improving my health and well-being, and taking control of my diabetes.
- Reducing the number of questionnaires: we decided to reduce the number of questionnaires from 3 to 2 by removing the AdKnowl questionnaire. The AdKnowl (knowledge) questionnaire [ ] was removed because it was significantly longer and more time-consuming than the other 2 questionnaires. Patient feedback suggested that they found the questionnaire burdensome and off-putting; the evidence we found from systematic reviews of diabetes self-management education programs suggests that there is a lack of a consistent positive relationship between knowledge and glycemic control and that factors other than knowledge are needed to achieve long-term behavior change [ ]. Therefore, we prioritized the changes in distress and self-efficacy.
- Web-based self-registration: it was decided to change to web-based registration to save time and to make it easier for patients to access the program quickly. The self-registration page included a demographic questionnaire, which allowed for the collection of baseline data. Telephone support from the HDSO team was still available for those who had difficulty registering on the web or using the program.
We decided to offer the program to everyone with T2DM and not just people who were newly diagnosed. The HDSO program was developed in line with the national clinical guidelines for GPs advising them to offer patients with T2DM structured education at and around the time of diagnosis . However, we knew from the National Diabetes Audit that not all patients were offered structured education at the time of diagnosis, and of those who were offered it in 2016-2017, only 7.1% attended [ ]. Therefore, many patients with T2DM who were not newly diagnosed have not received structured self-management education and are in need of it. In addition, data on the incidence and prevalence of T2DM in the United Kingdom show that T2DM prevalence rates have more than doubled between 2000 and 2013, but incidence rates have increased more slowly [ , ]. This suggests that there are more people being diagnosed younger and living longer rather than new diagnoses, which consequently suggests that it would be possible to recruit people who were not newly diagnosed to the HDSO program than people who were newly diagnosed. We decided to offer the program to everyone with T2DM and collect data on the duration since diagnosis. This allowed us to compare completion rates between newly diagnosed and non–newly diagnosed patients.
We have described the stages of the development of a web-based structured education program for people newly diagnosed with T2DM, HDSO. Methods from HCI and health research were used in combination at every stage. Methods from HCI put more emphasis on understanding user requirements and determining uptake. The in the wild testing allowed us to identify low completion rates, which were not picked up in the usability testing because it was conducted with highly motivated patient volunteers. The methods from health and the MRC framework for complex interventions emphasized the impact of the intervention. This helped us to understand the potential effectiveness of the intervention. The iterative development process that we went through with the intervention, in contrast to traditional piloting and feasibility studies conducted in health research , resulted in an intervention that was more stable and appropriate for a definitive trial than the earlier iteration. Proceeding to a trial too early can be problematic because trials do not detect whether the lack of intervention effect is due to implementation failure or genuine ineffectiveness [ ]. Randomized controlled trials also fail to permit iterative improvements to the design and updates to technology [ ].
Comparison With Previous Work
The existing reporting of complex behavior change interventions is limited, and this prevents successful replication of successful interventions . Reviews of web-based T2DM self-management interventions have reported extreme heterogeneity of interventions [ ] and poor descriptions of the theoretical bases and active ingredients of the interventions [ ]. This makes it difficult for researchers to identify and understand successful intervention components and to be able to design and implement successful interventions. The field of digital health research and web-based diabetes self-management is evolving rapidly, and it is important for future research that lessons can be learned from existing studies. This description of the development and content of HDSO helps add to the understanding of how and why web-based interventions for diabetes self-management (and other long-term conditions) work and can be used to inform future research in this area. In addition to describing the intervention, this study also adds to the understanding of how interdisciplinary methods from health and HCI can be used to develop a DHI. Previous studies by Blandford et al [ ] and Pagliari [ ] have described the challenges in using interdisciplinary research in the development and evaluation of DHIs, and this paper illustrates some of the concepts described in the literature using the example of a web-based diabetes self-management program.
Strengths and Limitations
The strength of this research is the use of methods from both health and HCI. We combined the theory of living with long-term conditions and a user-centered design to understand and meet user requirements. This meant that users of different age groups, education levels, and ethnic backgrounds could use the intervention, as demonstrated by the analysis of the usage data from subsequent studies . A weakness of the interdisciplinary approach in this study was the emphasis on time and resources when conducting qualitative interviews with program users. This meant that we conducted rapid user experience studies at several stages, with smaller sample sizes and limited depth and duration of interviews.
Another weakness of our approach was patient involvement. More extensive patient involvement could have been used in the design of the interview guide for the telephone interviews in the user experience study and data analysis. The patient volunteers could also have been asked to make their own suggestions for refinements to the first iteration of the program, instead of relying solely on the data.
This paper describes how interdisciplinary methods can be used to develop a web-based structured education program for people newly diagnosed with T2DM. Methods were combined from human-computer research and health research. The reporting of the development processes for DHIs needs to continue, especially when interdisciplinary methods are used, for researchers to be able to learn from each other and create user-centered interventions.
The authors would like to thank the participants and professionals who contributed to the development of the HeLP-Diabetes: Starting Out program, particularly Orla O’Donnell, Helen Gibson, and Rebecca Owen.
Funding for a percentage of the website development and a percentage of hosting and maintenance of the website was provided by the National Institute for Health Research School for Primary Care Research (project reference: 280).
The views expressed are those of the authors and not necessarily those at the National Institute for Health Research or the Department of Health and Social Care.
SP, FH, and EM contributed to this research proposal. EM, SP, JR, and KP were involved in the development of the intervention. SP obtained ethical approval, recruited patients, collected and analyzed the data, and drafted the manuscript. All authors, including NN, reviewed and edited the manuscript. All authors approved the final version of the manuscript.
Conflicts of Interest
EM was the Managing Director of the HeLP-Digital Community Interest Company during the period of the research, a not-for-profit social enterprise established to roll out HeLP-Diabetes. She was not remunerated for this role, and there was no financial gain from the publication of this paper.
- Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance. Developing and evaluating complex interventions: the new Medical Research Council guidance. Br Med J 2008 Sep 29;337:a1655 [FREE Full text] [CrossRef] [Medline]
- Blandford A, Gibbs J, Newhouse N, Perski O, Singh A, Murray E. Seven lessons for interdisciplinary research on interactive digital health interventions. Digit Health 2018 May 03;4:2055207618770325 [FREE Full text] [CrossRef] [Medline]
- Elwyn G, Kreuwel I, Durand MA, Sivell S, Joseph-Williams N, Evans R, et al. How to develop web-based decision support interventions for patients: a process map. Patient Educ Couns 2011 Feb;82(2):260-265. [CrossRef] [Medline]
- Klasnja P, Consolvo S, Pratt W. How to evaluate technologies for health behavior change in HCI research. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2011 Presented at: CHI '11: CHI Conference on Human Factors in Computing Systems; May 7 - 12, 2011; Vancouver BC Canada p. 3063-3072. [CrossRef]
- Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med 2016 Nov;51(5):843-851 [FREE Full text] [CrossRef] [Medline]
- Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: new guidance 2006. Medical Research Council. 2006. URL: https://mrc.ukri.org/documents/pdf/complex-interventions-guidance/ [accessed 2021-12-27]
- Dix A, Finlay J, Abowd G, Beale R. Human-Computer Interaction. New York: Pearson; 2004.
- Rogers Y, Sharp H, Preece J. Interaction Design: Beyond Human-Computer Interaction Third Edition Ed. Hoboken, New Jersey, United States: John Wiley & Sons; 2013.
- Calp MH, Akcayol MA. The importance of human-computer interaction in the development process of software projects. Int J Inform Technol Emerg Technol 2015 Nov 25:48. [CrossRef]
- Maguire M. Methods to support human-centred design. Int J Hum-Comput Stud 2001 Oct;55(4):587-634. [CrossRef]
- Pagliari C. Design and evaluation in eHealth: challenges and implications for an interdisciplinary field. J Med Internet Res 2007 May 27;9(2):e15 [FREE Full text] [CrossRef] [Medline]
- Editorial. Current issues and future directions in psychology and health: towards a cumulative science of behaviour change: Do current conduct and reporting of behavioural interventions fall short of best practice? Psychol Health 2007 Dec;22(8):869-874. [CrossRef]
- Riley BL, MacDonald J, Mansi O, Kothari A, Kurtz D, vonTettenborn LI, et al. Is reporting on interventions a weak link in understanding how and why they work? A preliminary exploration using community heart health exemplars. Implement Sci 2008 May 20;3(1):27 [FREE Full text] [CrossRef] [Medline]
- Michie S, Fixsen D, Grimshaw JM, Eccles MP. Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci 2009 Jul 16;4(1):40 [FREE Full text] [CrossRef] [Medline]
- Yardley L, Morrison L, Bradbury K, Muller I. The person-based approach to intervention development: application to digital health-related behavior change interventions. J Med Internet Res 2015 Jan 30;17(1):e30 [FREE Full text] [CrossRef] [Medline]
- O'Cathain A, Croot L, Sworn K, Duncan E, Rousseau N, Turner K, et al. Taxonomy of approaches to developing interventions to improve health: a systematic methods overview. Pilot Feasibility Stud 2019 Mar 12;5(1):41 [FREE Full text] [CrossRef] [Medline]
- O'Brien N, Heaven B, Teal G, Evans EH, Cleland C, Moffatt S, et al. Integrating evidence from systematic reviews, qualitative research, and expert knowledge using co-design techniques to develop a web-based intervention for people in the retirement transition. J Med Internet Res 2016 Aug 03;18(8):e210 [FREE Full text] [CrossRef] [Medline]
- White BK, Martin A, White JA, Burns SK, Maycock BR, Giglia RC, et al. Theory-based design and development of a socially connected, gamified mobile app for men about breastfeeding (Milk Man). JMIR Mhealth Uhealth 2016 Jun 27;4(2):e81 [FREE Full text] [CrossRef] [Medline]
- Curtis KE, Lahiri S, Brown KE. Targeting parents for childhood weight management: development of a theory-driven and user-centered healthy eating app. JMIR Mhealth Uhealth 2015 Jun 18;3(2):e69 [FREE Full text] [CrossRef] [Medline]
- Deakin T, McShane CE, Cade JE, Williams RD. Group based training for self-management strategies in people with type 2 diabetes mellitus. Cochrane Database Syst Rev 2005 Apr 18(2):CD003417. [CrossRef] [Medline]
- Norris SL, Engelgau MM, Narayan KM. Effectiveness of self-management training in type 2 diabetes: a systematic review of randomized controlled trials. Diabetes Care 2001 Mar 01;24(3):561-587. [CrossRef] [Medline]
- Ellis SE, Speroff T, Dittus RS, Brown A, Pichert JW, Elasy TA. Diabetes patient education: a meta-analysis and meta-regression. Patient Edu Counsel 2004 Jan;52(1):97-105. [CrossRef]
- Steinsbekk A, Rygg L, Lisulo M, Rise MB, Fretheim A. Group based diabetes self-management education compared to routine treatment for people with type 2 diabetes mellitus. A systematic review with meta-analysis. BMC Health Serv Res 2012 Jul 23;12(1):213 [FREE Full text] [CrossRef] [Medline]
- Diabetes Prevalence Model 2016. Public Health England. URL: https://www.diabetes.org.uk/professionals/position-statements-reports/statistics/diabetes-prevalence-2016 [accessed 2021-12-27]
- Hex N, Bartlett C, Wright D, Taylor M, Varley D. Estimating the current and future costs of Type 1 and Type 2 diabetes in the UK, including direct health costs and indirect societal and productivity costs. Diabet Med 2012 Jul;29(7):855-862. [CrossRef] [Medline]
- Type 2 Diabetes in adults: management. NICE guideline NG28. National Institute for Health and Care Excellence. 2015. URL: https://www.nice.org.uk/guidance/ng28 [accessed 2021-01-31]
- DSME Standard. Quality Institute for Self-Management Education. 2010. URL: https://www.qismet.org.uk/ [accessed 2021-02-14]
- 2016-17 General Medical Services (GMS) contract and Quality and Outcomes Framework (QOF). NHS Digital. 2017. URL: https://digital.nhs.uk/data-and-information/publications/statistical/quality-and-outcomes-framework-achievement-prevalence-and-exceptions-data/quality-and-outcomes-framework-qof-2016-17 [accessed 2021-12-27]
- National Diabetes Audit, 2017-18 : Care Processes and Treatment Targets short report. NHS Digital. 2018. URL: https://files.digital.nhs.uk/E6/369B83/National%20Diabetes%20Audit%202017-18%20Short%20Report%2C%20Care%20Processes%20and%20Treatment%20Targets.pdf [accessed 2021-12-27]
- Dack C, Ross J, Stevenson F, Pal K, Gubert E, Michie S, et al. A digital self-management intervention for adults with type 2 diabetes: combining theory, data and participatory design to develop HeLP-Diabetes. Internet Interv 2019 Sep;17:100241 [FREE Full text] [CrossRef] [Medline]
- Murray E. Development, evaluation and implementation of a computer-based self-management programme for people with type 2 diabetes. NIHR Journals Library. 2017. URL: https://www.journalslibrary.nihr.ac.uk/programmes/pgfar/RP-PG-0609-10135/#/ [accessed 2021-12-30]
- Li J, Parrott S, Sweeting M, Farmer A, Ross J, Dack C, et al. Cost-effectiveness of facilitated access to a self-management website, compared to usual care, for patients with type 2 diabetes (HeLP-Diabetes): randomized controlled trial. J Med Internet Res 2018 Jun 08;20(6):e201 [FREE Full text] [CrossRef] [Medline]
- Alkhaldi G, Modrow K, Hamilton F, Pal K, Ross J, Murray E. Promoting engagement with a digital health intervention (HeLP-Diabetes) using email and text message prompts: mixed-methods study. Interact J Med Res 2017 Aug 22;6(2):e14 [FREE Full text] [CrossRef] [Medline]
- Poduval S, Marston L, Hamilton F, Stevenson F, Murray E. Feasibility, acceptability, and impact of a web-based structured education program for type 2 diabetes: real-world study. JMIR Diabetes 2020 Jan 06;5(1):e15744 [FREE Full text] [CrossRef] [Medline]
- Lazar J. Web Usability: A User-Centered Design Approach. Boston, MA: Addison-Wesley; 2006.
- Rubin J, Chisnell D, Spool J. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Hoboken, New Jersey, United States: John Wiley & Sons; 2008.
- Yen P, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012;19(3):413-422 [FREE Full text] [CrossRef] [Medline]
- Nielsen J, Clemmensen T, Yssing C. Getting access to what goes on in people's heads?: reflections on the think-aloud technique. In: Proceedings of the Second Nordic Conference on Human-computer Interaction. 2002 Presented at: NORDICHI02: NORDICHI 2002, Proceedings of the Second Nordic Conference on Human-computer Interaction; October 19 - 23, 2002; Aarhus Denmark p. 101-110. [CrossRef]
- Pal K, Eastwood SV, Michie S, Farmer AJ, Barnard ML, Peacock R, et al. Computer-based diabetes self-management interventions for adults with type 2 diabetes mellitus. Cochrane Database Syst Rev 2013 Mar 28(3):CD008776 [FREE Full text] [CrossRef] [Medline]
- Pereira K, Phillips B, Johnson C, Vorderstrasse A. Internet delivered diabetes self-management education: a review. Diabetes Technol Ther 2015 Jan;17(1):55-63. [CrossRef] [Medline]
- Ramadas A, Quek K, Chan C, Oldenburg B. Web-based interventions for the management of type 2 diabetes mellitus: a systematic review of recent evidence. Int J Med Inform 2011 Jun;80(6):389-405. [CrossRef] [Medline]
- van Vugt M, de Wit M, Cleijne WH, Snoek FJ. Use of behavioral change techniques in web-based self-management programs for type 2 diabetes patients: systematic review. J Med Internet Res 2013 Dec 13;15(12):e279 [FREE Full text] [CrossRef] [Medline]
- Corbin JM, Strauss A. Unending Work and Care: Managing Chronic Illness at Home. San Francisco: Jossey-Bass; 1988.
- Merriam SB. Adult learning theory for the twenty-first century. New Directions for Adult and Continuing Education 2008 Jun;2008(119):93-98. [CrossRef]
- Lorig KR, Holman HR. Self-management education: history, definition, outcomes, and mechanisms. Ann Behav Med 2003 Aug;26(1):1-7. [CrossRef]
- Michie S, Carey RN, Johnston M, Rothman AJ, de Bruin M, Kelly MP, et al. From theory-inspired to theory-based interventions: a protocol for developing and testing a methodology for linking behaviour change techniques to theoretical mechanisms of action. Ann Behav Med 2018 May 18;52(6):501-512 [FREE Full text] [CrossRef] [Medline]
- Webb TL, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res 2010 Feb 17;12(1):e4 [FREE Full text] [CrossRef] [Medline]
- Crabtree A, Chamberlain A, Grinter RE, Jones M, Rodden T, Rogers Y. Introduction to the special issue of “The Turn to The Wild”. ACM Trans Comput-Hum Interact 2013 Jul;20(3):1-4. [CrossRef]
- Rogers Y, Marshall P. Research in the wild. Synth Lect Hum-Cent Inform 2017 Apr 04;10(3):i-97. [CrossRef]
- Snoek F, Welch G. Problem Areas in Diabetes (PAID) questionnaire in DAWN (Diabetes Attitudes, Wishes and Needs) study. Novo Nordisk. 2006. URL: https://www.huskyhealthct.org/providers/provider_postings/diabetes/PAID_problem_areas_in_diabetes_questionnaire.pdf [accessed 2021-12-27]
- Sturt J, Hearnshaw H, Wakelin M. Validity and reliability of the DMSES UK: a measure of self-efficacy for type 2 diabetes self-management. Primary Health Care 2010 Jun 22;11(04):374-381. [CrossRef]
- What approvals and decisions do I need? NHS Health Research Authority. 2016. URL: http://www.hra.nhs.uk/resources/before-you-apply/research-requiring-nhs-rd-review-but-not-ethical-review/ [accessed 2021-12-27]
- Pope C, Mays N. Qualitative Research in Health Care, Third Edition. Hoboken, New Jersey, United States: Blackwell Publishing Ltd; 2006.
- Bandura A. Social cognitive theory of self-regulation. Organ Behav Hum Dec Proc 1991 Dec;50(2):248-287. [CrossRef]
- Alkhaldi G, Hamilton FL, Lau R, Webster R, Michie S, Murray E. The effectiveness of prompts to promote engagement with digital interventions: a systematic review. J Med Internet Res 2016 Jan 08;18(1):e6 [FREE Full text] [CrossRef] [Medline]
- Harden RM. What is a spiral curriculum? Med Teach 1999 Jul 03;21(2):141-143. [CrossRef] [Medline]
- Adult Literacy 2017. National Literacy Trust. URL: https://literacytrust.org.uk/parents-and-families/adult-literacy/ [accessed 2021-12-27]
- Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med 2017 Jun 13;7(2):254-267 [FREE Full text] [CrossRef] [Medline]
- Jordan K. Massive open online course completion rates revisited: assessment, length and attrition. Int Rev of Res Open Distrib Learn 2015 Jun 19;16(3). [CrossRef]
- Poduval S, Ross J, Pal K, Newhouse N, Hamilton F, Murray E. Use of the TIDieR checklist to describe an online structured education programme for type 2 diabetes. Digit Health 2020 Nov 30;6:2055207620975647 [FREE Full text] [CrossRef] [Medline]
- Speight JC, Bradley C. The ADKnowl: identifying knowledge deficits in diabetes care. Diabet Med 2001 Aug;18(8):626-633. [CrossRef] [Medline]
- Zghebi SS, Steinke DT, Carr MJ, Rutter MK, Emsley RA, Ashcroft DM. Examining trends in type 2 diabetes incidence, prevalence and mortality in the UK between 2004 and 2014. Diabetes Obes Metab 2017 Nov 05;19(11):1537-1545. [CrossRef] [Medline]
- Sharma M, Nazareth I, Petersen I. Trends in incidence, prevalence and prescribing in type 2 diabetes mellitus between 2000 and 2013 in primary care: a retrospective cohort study. BMJ Open 2016 Jan 13;6(1):e010210 [FREE Full text] [CrossRef] [Medline]
- Feasibility and pilot studies. National Institute for Health Research. 2015. URL: https://www.nihr.ac.uk/funding-and-support/documents/funding-for-research-studies/research-programmes/PGfAR/CCF-PGfAR-Feasibility-and-Pilot-studies.pdf [accessed 2021-12-27]
- Levati S, Campbell P, Frost R, Dougall N, Wells M, Donaldson C, et al. Optimisation of complex health interventions prior to a randomised controlled trial: a scoping review of strategies used. Pilot Feasibility Stud 2016 Mar 15;2(1):17 [FREE Full text] [CrossRef] [Medline]
- Mohr DC, Schueller SM, Riley WT, Brown CH, Cuijpers P, Duan N, et al. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res 2015 Jul 08;17(7):e166 [FREE Full text] [CrossRef] [Medline]
- Greenwood DA, Gee PM, Fatkin KJ, Peeples M. A systematic review of reviews evaluating technology-enabled diabetes self-management education and support. J Diabetes Sci Technol 2017 Sep 31;11(5):1015-1027 [FREE Full text] [CrossRef] [Medline]
- Poduval S, Ahmed S, Marston L, Hamilton F, Murray E. Crossing the digital divide in online self-management support: analysis of usage data from help-diabetes. JMIR Diabetes 2018 Dec 06;3(4):e10925 [FREE Full text] [CrossRef] [Medline]
|BCT: behavior change technique|
|DHI: digital health intervention|
|GP: general practitioner|
|HCI: human-computer interaction|
|HDSO: HeLP-Diabetes: Starting Out|
|HRA: Health Research Authority|
|MRC: Medical Research Council|
|NHS: National Health Service|
|QISMET: Quality Institute for Self-Management Education|
|QOF: Quality and Outcomes Framework|
|REC: Research Ethics Committee|
|T2DM: type 2 diabetes mellitus|
Edited by A Kushniruk; submitted 25.06.21; peer-reviewed by D Lungu, E Børøsund; comments to author 26.08.21; revised version received 24.09.21; accepted 15.10.21; published 14.01.22Copyright
©Shoba Poduval, Jamie Ross, Kingshuk Pal, Nikki Newhouse, Fiona Hamilton, Elizabeth Murray. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 14.01.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.