Participatory Research as One Piece of the Puzzle: A Systematic Review of Consumer Involvement in Design of Technology-Based Youth Mental Health and Well-Being Interventions

Introduction

Technology and Youth Mental Health

More than a quarter of young Australians aged 16-24 years old will experience a mental illness in a 12-month period, with anxiety, substance abuse, and mood disorders the most common [1]. Alarmingly, 3 quarters of first episode mental illness occurs before the age of 25 years [2], and it has been reported that only 30% of these younger people are accessing the professional help that would benefit them [1,3]. With that in mind, technology-based mental health resources and interventions, part of Australia’s e-mental health strategy [4], may offer an opportunity to engage the other 70%. The potential of technology, therefore, to increase youth engagement with formal mental health services, particularly in rural and remote contexts where service options can be limited, is yet to be fully realized.

Technology-based mental health care interventions are often cited as methods for providing greater access to and engagement with services [5-7]. A recent review, however, identified only 2 studies that investigated the use of technology to increase engagement with clinical youth mental health services, and a further 3 explored the role of technology as an adjunct to face-to-face therapy [8]. This review detailed promising results and possibilities for the role of technology in creating and augmenting developmentally appropriate and responsive youth mental health services. However, the research included lacked rigor and the dearth of studies highlight the need for more research and development in the field that is guided by an evidence base [8].

Technology-based health interventions commonly suffer from limited uptake and/or adherence [9-13], which may be dependent on methodological issues such as design, particularly how human factors are incorporated [6,12,14]. For example, failing to obtain an in-depth insight into intended consumer behavior and their environments, which is crucial for good design [15]. Guidelines for technology-based mental health design increasingly emphasize the need for formal incorporation of consumer participation into intervention design [6,16-19]. Therefore, engaging young people and their support communities at all stages of development is likely to be crucial in enhancing uptake and adherence of technology-based interventions, particularly those from rural, remote, and disadvantaged communities [20,21].

Participatory Research

There is a rich history of participatory research with children and young people in the social sciences [22-25]. Participatory research is conducted in partnership with the individuals or community of interest and not on them, and in this way differs from traditional research. It purports to increase research relevance and usability through improved context appreciation. Other reported benefits of participatory research include greater stakeholder buy-in and improved efficacy and sustainability of research products (or outcomes) [26-29]. When considering the reported average 17-year gap between publication and translation of findings in health care, it is not surprising that participatory methodologies have gained prominence in the field over the last 20 years [5,28,30,31].

Within mental health design research, common participatory methodologies include community-based participatory research (CBPR), participatory action research (PAR), participatory design (PD), and user-centered design (UCD). PAR aims to develop an egalitarian partnership with a chosen community or group to generate positive, self-identified individual-, group-, and community-level change. While the research goals and associated theories of change may vary, PAR and CBPR are different terms for 1 research methodology underpinned by the same core principles. As such, the terms are used interchangeably in the literature depending on the country of origin [32,33]. PD—borne out of British, North American, and Scandinavian traditions—employs iterative design cycles in which knowledge production and research output(s) are shared by researchers and end-users [34]. Unlike PD, UCD is controlled by the design and research professionals, and participation takes on a strictly consultative role; the project is led, and decisions are made, by “experts” [35]. At the other end of the participatory continuum sits consumer-led research (ie, research initiated and/or controlled by consumers), which has recently taken on new life in the context of social media.

Most research has focused on consumer participation in service delivery, with the literature around participation in intervention design via research projects still developing [36]. It is also less common for the intervention development process to be reported [36]. Boote, Telford, and Cooper [37] argue that consumer involvement in research can be rationalized in 2 ways: (1) empowerment—defined as consumer involvement linked to greater autonomy in decision-making for disempowered/marginalized groups; and (2) consumerism—defined as consumer involvement linked to creating outcomes (eg, products, services or interventions) that generate satisfaction and value-for-money, with consumer input directed at improving efficiency, economy, and effectiveness. Each has different implications for the chosen methodology and role of the consumer.

The Current Review

Given the potential for technology to increase engagement with mental health services, the current review explored the question: “How have participatory methodologies been employed to develop technology-based youth mental health and well-being interventions?”

Youth participation in the development and delivery of mental health services designed to benefit them has received attention and resourcing for some time [38]. On- and offline service-wide youth participation models are well documented and demonstrate a recognition that young people are best placed to judge what works for them given their developmental-specific experience of mental illness [38]. Online services such as Eheadspace [39], beyondblue [40], and ReachOut.com [41] provide examples of youth participation best practice. This review, however, focuses on participatory development of technology-based interventions by research groups, which may include collaboration with services or other health organizations, as compared to youth participation in an existing service. Project teams involved in production and design of technology-based mental health interventions are interdisciplinary and diverse, and their outputs and findings are distributed across multiple channels and fields depending on the discipline focus of the authors. These factors make a review of this kind a complex undertaking. This review has chosen to focus on work titled, indexed, and stored in databases with a mental health focus and, as such, will not have accessed the body of literature that exists in humanities and social sciences databases (particularly around child, youth and consumer rights and youth participation) that are reflective of multiple stakeholder contributions.

Projects that involved consumers in the design and development of interventions spanning the breadth of the mental health intervention spectrum were included to maximize learning opportunities and to gain a broad understanding of participatory processes in this emerging field of research. The aim was to synthesize previous literature and make practical recommendations for mental health technology designers who wish to employ participatory research methods in a youth context. The major concepts under investigation were: (1) the nature of consumer involvement and the participatory process in intervention development; (2) the nature and outcomes of the design process; and (3) the relationship between participatory research and the implementation of research.

By “technology-based” we refer to information and communications technology-based (ICT-based) digital interventions such as health promotion/prevention Web sites, community-focused health promotion/prevention technologies, treatment-focused Web sites/programs/therapies, and other mental health apps, games, and products. The interventions may act as standalone entities or as an adjunct to existing face-to-face treatment or programs. For inclusion in this review, developers need to have adequately defined and documented (ie, via a project report, journal article, conference paper, or thesis) a participatory development/design project.

Methods

Search Strategy

A systematic search strategy was used to identify published and unpublished studies that described participatory research mental health projects. Database search strategies employed both resource-specific subject headings (where available) and keywords describing 2 broad concepts—participatory research and mental health/illness (the emphasis on illness terms reflected the focus on treatment-focused interventions). Keywords were often combined using proximity operators in order to increase search sensitivity (generated by SO, RD, SL, and NB). Comprehensive literature searches were undertaken in the following 15 databases: OvidSP Medline (1946-), PubMed, PsycINFO (1806-), CINAHL, Scopus, Web of Science, Informit (health, social sciences, and science and engineering subsets), arXiv.org, ACM Digital Library, and IEEE Xplore Digital Library. Database searches were limited to studies published in English. The time period for searches was database inception to June 2014. Full search strategies for the OvidSP Medline and PsycINFO databases are provided as Appendix 1.

To identify unpublished studies, 3 simplified versions of the search strategy were used in the Google Advanced search engine and results were restricted to PDF documents. Only the first 100 results for each search variant were reviewed for relevance (ie, total n=300). Web sites of relevant conferences were also checked for additional unpublished papers, including: Participatory Design Conference; Special Interest Group on Computer-Human Interaction; and the Computer-Human Interaction Special Interest Group of the Human Factors and Ergonomics Society of Australia. Reference lists of relevant citations were checked and email contact was made with authors to source additional relevant documentation and current information on the intervention. All searches were conducted in June 2014. EndNote X6 (Thomson Reuters) was used to manage all database citations. A first screening of titles/abstracts by a research assistant (MW) eliminated clearly irrelevant citations/documents based on research method and age group. The remaining citations were screened by a second reviewer (SO). Full text articles were sourced when a decision on relevance could not be made by title or abstract alone.

Inclusion and Exclusion Criteria

All research papers that involved projects judged as having a primary focus on youth mental health and well-being were included in the review, irrespective of whether the mental health focus was related to an existing physical condition. This decision ensured that learnings from the development of interventions spanning the breadth of the health intervention spectrum would inform development of treatment-focused interventions. Specific criteria are outlined below.

Inclusion criteria:

• Mental health or well-being focus (defined in consultation with a multidisciplinary team comprised of clinical mental health, technology and consumer perspectives, and informed by the DSM-V definition of mental disorder) [42]
• English language
• Development and/or design of ICT- or digital technology-based intervention
• Youth-based intervention (or include a youth element)
• Inclusion of participatory research processes or elements thereof

Exclusion criteria:

• Commentaries, opinion pieces, or editorials
• Photovoice studies (judged as a distinct research methodology that does not involve design or development of a technology-based intervention)

Data Collection and Analyses

A multidimensional framework analysis, adapted from research conducted by Oliver et al [43] and Lorenc et al [44], was employed to categorize research. This involved an iterative approach of familiarization with the literature and gradual development of the conceptual framework based on the broad research question. Concepts were drawn from the literature around participatory research and technology-based health intervention design. The outcome criteria were populated by criteria drawn from previous participatory research evaluation and the information needs of the study [28,37,45,46]. Due to the exploratory nature of the review, all levels of evidence were considered. Refer to Textbox 1 for definitions of concepts used and their relationship to the areas of investigation. Each study was evaluated by 2 members of the research team using the definitions in Textbox 1. Discrepancies were discussed and consensus reached. A third member of the team was consulted if required.

Results

Study Selection

In total, 14,021 citations and Web documents were identified through database searches and open Web searching. Once duplicate citations were removed, 6210 items remained for preliminary assessment of relevance. After title, abstract, and full paper screening, 17 studies were chosen for inclusion in this systematic review (Figure 1 and Table 1). Of these, 1 study reached proposal stage [48], and 1 was designed but not developed [49].

‎View this figure

Characteristics of the Included Studies

Of the 17 projects included in the review, included treatment-focused interventions [7,36,52-55,62]. The remaining 10 were preventive interventions [48-51,56-61]. UCD [7,53-55], PD [50,52,61,62], and PAR [48,51,59,60] were the most common methodologies used (4 projects each). PD provided the sub-framework for an iterative design process in a further 4 projects [49,53,55,56]. UCD or PD methodologies tended to scaffold development of treatment-focused interventions. Three projects were based in the US and Australia, respectively, and 2 each in Ireland, Sweden, England, and Canada. The final 3 were based in Denmark, Norway, and Scotland. The age range of youth involved was 10-26 years old; 5 studies did not report age, 9 did not report gender. Besides age, no other socio-demographic variables were reported.

Nature of Consumer Involvement and the Participatory Process

Most projects (11 of the 17) involved young people (and other relevant stakeholders) for principally consumerist purposes [7,48,49,51-55,57,58,62]; that is, to create usable, effective, and efficient interventions. A further 2 reported elements of both empowerment and consumerism [36,50]. No projects actively involved youth consumers in the project planning stage, with project aims and goals unreflective of their input.

Overall, consumers were involved in a combination of 3 main stages of research: (1) Needs analysis/design specification; (2) Intervention design/prototyping and development; and (3) Usability and pilot testing. Two projects involved consumers in all 3 stages [52,61]. Projects commonly included consumers, who were most often youth and mental health clinicians (rarely family or caregivers), in the needs analysis/design specification stage [7,49,50,52,54,56,58,61,62]. Some projects entered this stage with a predetermined intervention in mind [49,50,52,54,61], while others operated with a looser set of intended outcomes [7,56,58,62]. Four projects involved consumers in the intervention design/prototyping and development stage [36,52,58,61]. In other projects, consumer involvement involved consulting to refine an existing intervention [51,57] or solely usability and pilot testing [53,55]. The community-based projects of Schmidt [59] and Stewart et al [60] developed community health education tools. They involved consumers at all stages of the project besides initial project planning.

Youth participation was variable, both across and within projects. Overall, 70% of projects reported predominantly consultative consumer involvement [7,49-57,62] and the remaining projects were collaborative in nature [36,58-61]. The projects, therefore, sat in the middle of Biggs’ modes of participation [46]. Youth involvement was consultative in 6 of 7 treatment-focused projects [7,52-55,62], and 4 of these projects involved mental health clinicians as part of the research team [7,54,55,62]. Projects that developed treatment-focused interventions generally involved the most limited forms of consumer input. The highest level of youth participation was evident in the prevention-intervention projects, particularly Lakey, Stewart et al, Valaitis et al, and Schmidt [58-61].

Families, caregivers, and intended implementation-site representatives were under-represented in the projects. Of the 16 carried out, 7 projects clearly identified the intended implementation site and included representatives in the design phase [7,49,51,57,58,61,62]. The Stewart et al [60] and Schmidt [59] projects developed community-education focused interventions with local community representatives; however, it was unclear how widely their products were intended for distribution and thus the specific implementation site(s).

Overall, it was difficult to gain insight into consumers’ views on their participation in the projects (process evaluation) and their outputs (evaluation of the intervention). Three projects involved consumer evaluation of their experience of research [59-61]. These evaluations suggested a general trend toward perceived legitimacy and accountability of the research process and its outputs, but they also served to highlight the different expectations regarding process and outcomes between project/research leads and consumers. Other projects reported informal and anecdotal consumer support for the research process [51,55-57]. In some cases, pilot and small-scale clinical evaluation data were reported [7,51,52,54,57,61,62].

In line with the consumerist rationale for most projects, deliberate capacity building and learning for consumers was limited; only 5 projects involved significant opportunities for this [51,58-61]. These involved development of preventive interventions.

Consumer involvement was seen as crucial to intervention design and development in most projects; emergent knowledge was evident in all project outputs and each made explicit reference to value of consumer involvement in intervention development. Projects reported clear dichotomies around designer/researcher assumptions of effective and acceptable interventions and those of the intended consumer. These differences were present in intervention premise and content [50], and mode of delivery and characteristics/components [52,56,62]. Projects reported compromises between the perspectives, which were evident in the designs. Consumer consultations in the needs analysis/design specification stage were used to underpin and inform intervention design [7,49,50,52,54,56,58,61,62]. Consumers also played a role in tailoring and contextualizing interventions [7,53-55].

Eleven of the 15 completed projects reported challenges with consumer recruitment, capacity, commitment, and reliability [7,36,50-52,54,55,58,59,61,62]. Cited reasons included lack of access to the target consumer group, consumer personal circumstances and/or condition-related factors, and the busy lives of youth. All projects aiming to develop treatment-focused mental health interventions found recruitment and ongoing participation of intended youth consumers difficult to achieve; however, youth consumer attrition during intervention design and development was not specific to development of treatment-focused interventions [50,59,61].

Nature and Outcomes of the Design Process

Three projects used heuristic guidelines to support intervention design [7,54,62]. Monshat et al [57] was guided by constructs of the Technology Acceptance Model (TAM) [64]. Overall, 4 projects reported use of technology frameworks or theory to guide intervention development [7,54,57,62]. Valaitis et al [61] used logic models to support major project decisions, including those specifically related to intervention design, such as the prototyping process, as well as techniques from scenario-based design [63,65]. Ekberg et al [49] employed design rationales and design space analysis, which detail reasons for and justification of design decisions, to guide development of their intervention [66]. Eight of 17 studies utilized PD methodology or principles to guide intervention development [49,50,52,53,55,56,61,62]. Nine projects mentioned the broad theories (including psychological, health, education, group, empowerment, and cultural) on which the intervention or project were based [6,36,48,51,54-56,60,61] (the details of 2 were found in project reports provided by the authors, not in the published articles [51,61]).

A structured design process, with activities able to scaffold consumer input through the design stages, was seen to be effective in a third of completed projects [49,52,55,58,61]. Use of scenario-based design—which included techniques such as storyboarding, personas [63,65], think-aloud techniques [67,68], and varied methods for capturing user experience and knowledge—was seen to assist the design process. Inspiration/idea progression and prototyping was facilitated by appropriate planning and resourcing with respect to design activities and the space in which they were conducted.

Project flexibility and responsiveness, including the ability to adapt to changing resources, priorities, work styles/preferences, output standards, and deadlines, was often built into design and was a common thread throughout projects that reported high levels of consumer involvement and influence [36,51,59,61]. Projects led by nontechnical researchers also reported the need for integration of technical expertise at all stages of intervention design and development [36,49,61]. A professional appearance of the final intervention product was also seen as important by youth consumers in a number of projects [49,50,62].

In addition, balancing consumer requirements with what was possible technically, ethically, and practically (ie, time and resource, both financial and human, restrictions) was highlighted in 3 projects [49,50,56]. Of particular concern were social and consumer self-authoring components of interventions, privacy, confidentiality, clinical risk, and authenticity of information. Formal outcome data was available for 2 projects [50,55].

Relationship Between Participatory Research and Implementation

While leadership was not always clearly defined, most projects were researcher-led. Interdisciplinary project teams were common, including researchers or professionals with various combinations of mental health and technology domain expertise. Often, however, 1 discipline had overall responsibility for the project.

Five projects [36,51,58,59,61] reported existing relationships with outside champions who were linked to implementation sites or organizations capable of progressing the project beyond the intervention development stage. In 2 projects, Hallett et al [51] and Valaitis et al [61], project and governance plans were designed such that implementation of the intervention was integrated and a further 4 studies reported established links with intended intervention sites [55,57,58,62]. Stewart et al [60] and Schmidt [59] integrated community-based dissemination of outputs into their project plans. Many projects were, however, exploratory and involved development of technology-based interventions with a limited evidence base.

With the information available at the time of writing, 5 projects had extended beyond the intervention design, development, and pilot stage [50,51,55,61,62]. It is unclear the extent to which outputs from the 2 community-based projects [59,60] were used in a health promotion or prevention capacity beyond the life of the project.

Eleven projects utilized existing relationships and networks to assist with recruitment of target consumers [36,49-51,53,55-59,62]. The benefits of accessing consumers through existing networks was often noted; in particular, this made a significant difference in recruiting consumers with lived experience of mental illness for studies developing treatment-focused mental health interventions [36,53,55,62].

Discussion

Nature of Consumer Involvement and the Participatory Process

A strong history of youth participation in mental health research and service development exists, rooted in the empowerment of young people to address service quality and access issues [38]. In contrast, the projects included in this review generally involved consumers for consumerist intentions and in a consultative capacity. This represents a departure from the traditional empowerment and emancipatory rationales for participatory research demonstrated in a minority of projects in this review [36,50,56,59-61]. These increasingly consumerist underpinnings have implications for why and how consumers are asked to participate in research and the degree of mutual benefit that is possible, desired, and ethical.

Eight of the 17 projects explicitly reported using PD methodology or methods to guide intervention development, and others used PD-related design techniques such as user journeys, personas, and workshops. PD originated in the 1970s from a Scandinavian tradition of empowering workers to exercise control over the role of technology in their workplace [69]. Increasingly, however, the application of PD as a methodology or collection of techniques/methods has moved into design underpinned by consumerist principles that emphasize usability, effectiveness, and acceptability of the product [5,19]. This shift was embodied in several projects in this review [49,52,53,55,58,62]. Participatory methodologies with consumerist underpinnings tend to seek information and understanding through consultation and, thus, support a more passive role of the consumer in the research.

In attempting to assess perceived accountability and the legitimacy of the research process and outputs in the studies reviewed, it became clear that researchers are not in the practice of evaluating and reporting on the consumers’ participation experience. This is not only a missed opportunity for consumers to collect data in order to reflect on and learn from their experience of research, but it represents an invaluable source of data from which other projects wishing to conduct participatory intervention design and development could benefit. Email correspondence with 1 author of the studies reviewed revealed that the intervention did not progress any further from the design stage due to possible consumer dissatisfaction with the design, despite the intervention being designed and developed in collaboration with them. This highlights the need for formal assessment of consumer perceptions of accountability and legitimacy of the intervention. Existing literature notes the value derived by researchers and consumers in building in evaluation/reflection cycles, particularly for promoting the dialogue, critical reflection, and trust that are crucial components of high-quality participatory research [23].

While it has been reported that participatory research can enhance recruitment rates [70,71], this review highlights the consumer access, recruitment, and participation challenges faced by projects aiming to develop mental health and well-being interventions, particularly those with a treatment focus that target involvement of consumers with lived experience of mental illness. Those individuals who identify as struggling with mental illness still face stigma and privacy concerns, which restrict use of common recruitment methods such as advertising [62]. Even projects that reported collaboration with mental health services or access to those with lived experience of mental illness noted ongoing participation difficulties with maintaining consumer participation throughout the intervention design and development process [36,62].

Collaborating with existing groups of young people such as schools and youth groups [49,53,55,58-60] or organizations with a strong track record of engagement and outreach with the target consumers [51,56,57] represented a recruitment starting point for multiple projects. However, they too still reported struggling with ongoing participation difficulties. These recruitment concerns are not surprising considering the move into more consumerist-based projects that tend to be less integrated into communities than traditional participatory research.

Personal capacity, reliability, and attrition of consumers, particularly in the treatment-focused intervention development projects, must also be considered [36,50,52,55,61,62]. Todays’ young people contend with a myriad of demands on their time, and projects included in this review experienced this in the form of participant nonattendance, unreliability, and dropout. This effect may be amplified when the youth consumer is currently living with a mental illness. Consumers may also face financial or transport [62] barriers in attending planned project activities that may be related to their age and/or health status. Broadly speaking, participatory research that involves consumers, particularly those who are members of minority or vulnerable populations, carries with it particular ethical considerations that require careful and sensitive negotiation and practical restrictions [72-75]. This is best exemplified in the Løventoft et al project [52], which reported moving from egalitarian principles of PD to a designer-led user-centered approach due to challenges with consumer engagement, retention, and capacity.

The projects with the most extensive youth consumer participation were those in which young people were involved in design and development of health prevention interventions, as exemplified in Stewart et al [60], Valaitis et al [61], Lakey [58], and Schmidt [59]. This nonclinical consumer group is far easier to access and does not have the same privacy, stigma, and personal capacity concerns facing the clinical youth consumers.

Despite this, many studies reported successful participatory research with youth consumers from a range of backgrounds. Participation is greatly assisted by links to existing consumer groups. Integration into the community of interest, via sustained partnerships between academic and nonacademic partners, is a hallmark of participatory research and has previously been shown to enhance recruitment capacity [70,71]. Beyond this, future research projects would be well advised to plan for attrition; both with respect to an ongoing recruitment source and development of materials that can be provided to consumers for seamless integration into the project whenever they choose to engage or reengage. As borne out in this review, participation can and will fluctuate throughout the project and must be planned for and communicated to consumers [59].

Flexibility and open-mindedness, embodied by a willingness to work with a non-static group of consumers and to renegotiate the time, length, style, and content of planned interactions, was repeatedly noted by the projects included in this review [36,51,55,61]. Owens et al [36] in particular highlights the flexibility required by a project when working in an egalitarian manner with consumers. Their intervention became more complex than planned and required extra time and resources to create. Increased cost in terms of necessary resources, time and expertise associated with participatory research [29], along with the need for flexibility in terms of role division, project structure(s), timeframes, and even communication methods have been noted elsewhere [23].

In working with adolescents with behavioral problems, Mazzone et al [55] recommend small groups and many short activities with simple tasks and objectives. They also endorse building in praise and a sense of ownership when working with all youth consumers (see also Dold et al [73]). A structured design process that scaffolds consumers throughout was also found to be effective [49,52,55,58,61]. Given the probable lack of technical and design knowledge of the average consumer (via techniques like storyboarding, think-aloud techniques, and scenario-based design), scaffolding the design process appears to be an important consideration for researchers.

Planning for and understanding consumer expectations of participation in research, along with their self-perceptions as mental health consumers, matters [73]. Given the limited data available regarding consumer experience of research, building reflection and evaluation into research plans should be a focus for future research projects. Ideally, projects wishing to collaborate with youth mental health consumers require committed, youth-supportive research leadership and a process that is well-resourced and supported. Previous research suggests that projects that are age and developmentally appropriate and incorporate meaningful, individualized, empowering, and capacity-building elements improve consumer output and buy-in [59,73,76], which has obvious implications for improving the current recruitment and participation issues.

Recognizing that issues of power and agency are embedded in participatory research with young people, it is important to achieve best practice [23]. When researchers adopt the mind-set that “young people are creative agents who bring about change” [23], participatory research can represent an important opportunity for young people to be recognized and contribute meaningfully.

Nature and Outcomes of the Design Process

Most studies indicated that consumer participation was integral to good intervention design and development [7,36,49-51,53-56,58,60-62]. Accessing consumers’ implicit domain knowledge was the cornerstone of producing relevant, accessible, and usable interventions and output, which is consistent with prior reviews of participatory research [71,77].

Consumer involvement was associated with flexibility, responsiveness, human-centeredness, and adaptability in design. For example, in their online adaptation of peer-based health promotion for adult men who have sex with men and same sex attracted young people, Hallett et al [51] engaged peer volunteers to develop and pilot the intervention. This allowed the project to be responsive and to adapt the intervention and its evaluation as needed. The peer volunteers provided important information regarding online etiquette and technical proficiency, and during piloting facilitated access to clients and development of rapport and credibility through use of shared language and cultural understandings.

Consumer collaboration significantly altered Owen et al’s [36] text-based self-harm prevention intervention from the original design brief. Researchers originally planned for a replication study in which generic texts were sent at predetermined, high-risk times; the co-design process resulted in a more flexible and human-centered design involving client self-authored texts accessible on demand. Authors noted that the final form and function of the intervention would not have been possible without consumer input.

Successful outcomes require researchers to balance consumer requirements against those of other stakeholders, such as funders and implementation sites, while managing time, resourcing, and ethical considerations. This difficult task requires careful negotiation along with clear and ongoing communication [36,49,50,55,56].

This is best exemplified by analysis of (1) an exit focus group with youth consumers; and (2) youth consumer-designer/researcher email conversations throughout the Elf et al [50] project. Analysis revealed that, as the project progressed, the mind-set of the researcher/designers changed from exploration of ideas with consumers to concrete production of output. This shift in priorities was attributed to increasing pressure around resources (eg, human, financial, time), and delivering technical components on time became the priority over implementing consumer ideas/suggestions.

Theory to Support Intervention Design

Consistent with prior literature, limited application of theory to guide technology development was evident [17]. As a result, researchers are not maximizing the potential uptake, efficacy, and impact of their interventions. Three projects [7,54,62] used heuristic guidelines to support technology-based intervention design and development. The guidelines emphasize design for outcomes, with mental health professionals, within a UCD framework [6,18]. Consideration of clinical validity, therapist and client usability, along with intervention acceptability, access, engagement, adaptability, and sustainability are also highlighted. Monshat et al [57] was guided by constructs from TAM [64]. Beyond this, theory or models with the ability to explain consumer interaction with the technology were absent.

While the literature is still developing, the behavioral intervention technology model [17] is an example of a model to guide the conceptual and technical architecture of behavior-changing eHealth and mHealth interventions—where eHealth is defined as “internet or other electronic media to disseminate health related information or services” [78] and mHealth as “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, PDAs, and other wireless devices” [79]. The model guides researchers through development of clinical and usage aims, choice of technical elements and characteristics, and development of the intended workflow associated with the intervention. It assists in translating intervention aims into intervention elements and characteristics [17].

eHealth participatory design best practice advocates for intended users as co-designers and partners in all phases of research, along with intervention evaluation criteria that balances youth relevance, meaning, and engagement with existing evidence [19]. This type of theoretical integration is sorely needed in a field constrained by issues with uptake, adherence, and engagement [9-13,17]. Furthermore, persuasive features that “reinforce, change, or shape attitudes or behaviors or both without using coercion or deception” [80] and consumer motivation have had limited application in participatory technology-based mental health intervention design and, therefore, represents a focus of inquiry for future projects [10,14,81,82].

Planning for uptake and established connections with intervention sites were common to projects that successfully implemented their interventions or secured future funding [51,55,58,61,62]. Few projects reported evidence of inclusion of representatives from intended implementation sites in design and development of their interventions, even when accounting for the exploratory nature of some of the projects. A narrow definition of consumer may have led to limited representation of intervention site stakeholders in the intervention design phase.

Researchers need to be designing with an implementation site in mind and integrating influential system and organization level representatives into the process. In the case of treatment-focused interventions, mental health teams exist within larger systems that play an important role in acceptance and adoption of new interventions. Intimate knowledge of, and a strong working relationship with, the implementation sites of interest must be a priority of designer-researchers. Wolbling et al [83] argues that “ground-breaking ideas that arise within an existing organization that are not consistent with their values, routines, and overall strategy will be more difficult, if not impossible, to implement.” This assertion has clear implications for a research team wishing to implement new interventions from the outside. Organizational factors such as workplace ICT culture and policy and availability of resources have shown to be facilitators of uptake of ICT in health care [8]. Whilst Coyle et al [6] and Doherty et al [18] account for individual therapist considerations in their heuristic guidelines, they fail to account for organizational and system level factors that can impact on intervention uptake and impact.

Designing with target consumers is crucial. The most commonly reported barriers to uptake of ICT in health care are design and technology concerns including lack of clinical relevance or impracticality; in addition lack of clinician time and perceived ICT skills are frequently reported barriers. On the flip side, facilitators of ICT uptake include system usefulness and functionality, clinical relevance and ease of use [8,84]. This research indicates a clear role for application of theory to guide design and systematic consideration of human factors.

Limitations

A limitation of this review was the broad inclusion criteria. This is particularly evident with respect to the Schmidt [59] project, which developed community health education outputs to explore youth conceptions of the relationship between the built environment and well-being. Whether these outputs can be categorized as interventions is debatable given the limited detail reported on the project. Despite the fact that youth participation was identifiable in the Owens et al [36] paper, it did not have an exclusive youth focus. It was chosen for inclusion due to the nature of the project and its value in contributing to the aims of the review. In addition, the screening process may have benefited from involvement of a second reviewer to double screen. Evaluation of consumer representation was deemed too complex and broad to explore fully within this review beyond the description provided in the results table (Table 1). Finally, while every reasonable effort was made to find all relevant citations, the broad terminology used to describe the research in question may have resulted in some studies being overlooked, particularly where participatory processes may have been described in the methods sections of papers and not noted in the keywords, title, or abstract. Furthermore, the broad research field means the publication of some studies may not have been amenable to the titles, search terms, and databases that were used to construct this study and answer the research question. Moreover, participatory approaches are used in service settings but not always evaluated with the findings published and as such this work was not represented in the review. This review highlights the need for more research, evaluation, and publication on the use and outcomes of participatory approaches in the design and delivery of technology-based youth mental health services and interventions. The Young and Well Cooperative Research Centre (CRC) [85] is an initiative that prioritizes this connection and creates the required space for the corresponding evidence base to be built.

Given the nascent stage of this field of research and the corresponding exploratory aims of this review, the broad nature of the search terms and studies included facilitated a wide-ranging description and analysis of participatory design and development of technology-based youth mental health and well-being interventions. This ensured that insights and learnings from the breadth of the mental health intervention spectrum were incorporated. The heterogeneous nature of the projects included, however, prevented the number of specific comparisons that could be made between similar projects and intervention types. We also wish to acknowledge that analysis and results of this review attempted to define and summarize a diverse and often ill-defined research field, and in doing so may have inadvertently oversimplified the practical application of participatory intervention design. Finally, in a rapidly evolving field, the search cutoff date meant that highly relevant recent projects found in conference abstracts were not included in the review.

Conclusions

The current review found limited evidence that consumer consultations lead to routine uptake of interventions in practice; that is, consumer participation does not act as a default implementation or uptake strategy. Overall, strategies aimed at increasing uptake of technology in health care practice are not well understood or reported. A consumerist rationale, which prioritizes acceptability and usability of the intervention, has characterized most projects in this field. It was clear that consumer involvement shaped intervention design in ways that were reported as beneficial by the designers/researchers. While consumer consultations were associated with flexibility, responsiveness, human-centeredness, and adaptability in design, it was not possible to determine the impact of this on intervention effectiveness due to lack of outcome data. The implications for why and how consumers are asked to participate in this field of research and the degree of mutual benefit that is possible, desired, and ethical requires rigorous examination. Participatory intervention design projects are advised to develop flexible and well-resourced project plans, which integrate theory and implementation within the design and make space for reflection, evaluation, and publication of consumer experience of research.