This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on http://humanfactors.jmir.org, as well as this copyright and license information must be included.
Recruitment processes for clinical trials of digital interventions for psychosis are seldom described in detail in the literature. Although trial staff have expertise in describing barriers to and facilitators of recruitment, a specific focus on understanding recruitment from the point of view of trial staff is rare, and because trial staff are responsible for meeting recruitment targets, a lack of research on their point of view is a key limitation.
The primary aim of this study was to understand recruitment from the point of view of trial staff and discover what they consider important.
We applied pluralistic ethnographic methods, including analysis of trial documents, observation, and focus groups, and explored the recruitment processes of the EMPOWER (Early Signs Monitoring to Prevent Relapse in Psychosis and Promote Well-being, Engagement, and Recovery) feasibility trial, which is a digital app–based intervention for people diagnosed with schizophrenia.
Recruitment barriers were categorized into 2 main themes: service characteristics (lack of time available for mental health staff to support recruitment, staff turnover, patient turnover [within Australia only], management styles of community mental health teams, and physical environment) and clinician expectations (filtering effects and resistance to research participation). Trial staff negotiated these barriers through strategies such as emotional labor (trial staff managing feelings and expressions to successfully recruit participants) and trying to build relationships with clinical staff working within community mental health teams.
Researchers in clinical trials for digital psychosis interventions face numerous recruitment barriers and do their best to work flexibly and to negotiate these barriers and meet recruitment targets. The recruitment process appeared to be enhanced by trial staff supporting each other throughout the recruitment stage of the trial.
To better understand how interventions could be developed, evaluated, and implemented in routine care, it is important to fully understand which aspects of the implementation of randomized control trials (RCTs) are most challenging [
Trial staff responsible for recruiting participants must implement something novel (in this case, the recruitment process for a new intervention) within a health care system that comes with existing norms, knowledge, and social practices. Trial recruitment involves interacting with diverse groups [
Use of and interest in digital interventions is high in people diagnosed with schizophrenia [
Within trials of digital interventions, it is recommended that the recruitment of end users should be described in sufficient detail to enable readers who wish to contextualize or replicate the work [
Trial staff are responsible for meeting recruitment targets, which requires interacting with potential participants. This places them in a unique position to comment on the overall recruitment process and provides a narrative on (1) what happened during trial recruitment and (2) to enable researchers to make informed comment on why. Identifying barriers to recruitment has been identified as a strength of qualitative research within clinical trials [
This qualitative study within a trial (SWAT) [
EMPOWER [
In cluster trials, outcomes are usually measured at the level of the individual; however, trial procedures (such as recruitment) are applied by the research team at the level of the cluster (in this case, adult community mental health teams) [
In line with the EMPOWER process evaluation protocol [
The primary focus of the analysis was on achieving the a priori study aims (understanding the context of recruitment during the feasibility trial stage to refine recruitment in a full trial). Particular attention was paid to the reporting of barriers and facilitators to recruitment because this helps understand the context of recruitment. We now describe the 2 methods of the study in line with the key aim.
Ethnography refers to both the process and outcome of research that produces rich descriptions and interpretations of a social system from the point of view of its key social actors, including their behaviors, roles, and methods of interaction [
SA was based within the main office of EMPOWER for the full duration of trial recruitment and was able to observe trial staff both within meetings and within their daily office-based tasks during the recruitment process. Although ethnography commonly involves a researcher directly observing social processes, the examination of administrative data and study documents is important within process evaluation research [
To triangulate findings from the observation-based ethnography, focus groups were held with members of trial staff who were involved in the recruitment process. The use of qualitative methods [
SA (who was based in the UK office for the EMPOWER study) was present at the majority of weekly team meetings in the United Kingdom that were held during the recruitment process and had access to the minutes of meetings from this time. All members of the EMPOWER team who were based in Glasgow attended these meetings, with the focus of discussion being on general trial business. Recruitment procedures for both the United Kingdom and Australia were discussed in these meetings. Beyond formal meetings, SA was able to observe the work of the trial staff within the office and was privy to their discussions and reflections on the matter for the duration of trial recruitment. SA recorded reflective notes during the recruitment process from ethnographic observations at both formal meetings and more informal
Both focus groups were facilitated by SA (independent of the research team). One focus group was facilitated in person in Glasgow, United Kingdom, and another was facilitated remotely with the Australian team in Melbourne, who participated remotely via a secure telephone interface. Verbal informed consent was obtained before the start of each focus group. Each focus group followed a schedule of questions designed to explore barriers and facilitators to recruitment in some depth. A semistructured interview schedule was developed for broad exploration of the recruitment process from the perspective of trial staff (schedule available in prepublished protocol [
All participants in this SWAT (through observation or focus group participation or both) were employed in the EMPOWER trial and were involved in trial recruitment (either directly or indirectly). EMPOWER was a feasibility study; therefore, the numbers reflect the relatively small pool of trial staff, which is highlighted in
Description of participants’ characteristics.
Location | Focus group attendees | Roles |
United Kingdom | 6 (out of a possible 7) | Researcher, Chief Investigator, and Trial Manager |
Australia | 3 (out of a possible 5) | Principal Investigator, Researchers, and Trial Manager |
SA is a PhD student working on a process evaluation for the EMPOWER cRCT [
All data, including ethnographic observations and focus group transcripts, were analyzed thematically by SA using thematic analysis, a qualitative method used to identify, analyze, and report patterns constructed within text data [
Following thematic analyses of ethnographic observations and focus groups, it seemed that there were several key recruitment barriers encountered by the research team during recruitment to the trial. Beyond simply listing recruitment issues, trial staff discussed how these issues were addressed and what work was done to best negotiate these issues. To frame these discussions as distinct from merely reporting key issues, the concept of
Thematic map of recruitment themes.
The key barriers described by trial staff into trial recruitment broadly fell into 2 main themes: service characteristics (lack of time available to mental health staff to support recruitment, staff turnover, patient turnover [within Australia only], management styles of community mental health teams, and physical environment) and clinician expectations (filtering effect and resistance to research participation).
Research trial staff frequently spoke about mental health staff not having much time to engage in the recruitment process. The research team was highly aware of the broader social context of low staff capacity in the face of high numbers of patient referrals in routine care with limited staff to meet demand. Trial staff at both sites made empathetic references to being aware of mental health staff working within a context of immense pressure with a lack of resources and support. During the analysis by SA, it was constructed that the trial staff in EMPOWER felt it was inevitable that structural barriers that lead to mental health staff not having much spare time would inevitably be a barrier to trial recruitment:
I don’t think you can relate how busy they are. And much pressure they’re under. Some of the numbers we heard about in terms of new referrals into teams were quite staggering.
Forty. Forty referrals a week, yeah. And there doesn’t seem to be any sort of throughput to accommodate that additional pressure being moved around.
Closely linked to a lack of staff time was high staff turnover, which appeared to be systemic across both trial sites. Meeting notes and focus group data from both the United Kingdom and Australia indicated that high clinical staff turnover was a challenge to recruitment. Practically, this led to issues such as new clinical staff not being aware of the study because they were not employed when staff teams were initially told about it. Clinical staff changing jobs or taking leaves as they were unwell also appeared to be systemic issues within mental health services and was a macrolevel recruitment challenge. In the following example, a member of the EMPOWER team reflects on the impact of high staff turnover:
What we’re seeing is the key workers [mental health staff] are very fluid, there’s loads of movement, there’s massive changes as to who your key worker is, there’s lots of staff turnover.
A related subtheme (which was exclusive to Australia) was patient turnover because patients are discharged back to general practice (as evidenced in the quote below where participant alludes to “it’s not only a high turnover of consumers [patients]”) following the end of an acute episode of psychosis, unlike in the United Kingdom where clinical support is generally more long term for people diagnosed with schizophrenia. This was a particular barrier to recruitment because if patients were no longer in the service, they simply could not be recruited. However, this issue intersected with high clinical staff turnover, resulting in a complex barrier to recruitment into the study because the high clinical staff turnover within mental health services blocked the ability of trial staff to build relationships with clinical staff to build trust in the team and the project:
I think it's also worth noting that in public mental health services it's not only a high turnover of consumers [patients] but there's also a pretty high turnover of staff in some places, so you would have some clinicians that hadn’t heard of it or you know were quite new around that time and that kind of translates to recruiting consumers as well in terms of the discharges and the change in people being part of the service.
In both the United Kingdom and Australia, there were discussions about differences in management styles between the different mental health teams. In the first example, a trial team member explicitly stated that although participant numbers between sites may not have appeared too different, this obscured the challenges of having to adapt to different leadership styles across mental health teams. This was viewed as a key determinant of recruitment success:
I think at the big picture level the rate of recruitment wasn't particularly different and you know, [other named research assistants] might be able to say a bit more about the style of how it happens etc., there are certainly very different personality styles of managers so in terms of us managing the managers, we had to take into account that there are very different people who had a very different styles.
However, as pointed out in the UK site, it was not always the case that managers were those who were
Leadership’s hugely important in this. And always underestimated how much influence it has in any field, but this one no less. That the messages and the values and the attitudes that are being shared by the person who’s pulling the strings is really, really important. And that person who’s pulling the strings isn’t necessarily always the person who is supposed to be pulling the strings.
A further important recruitment challenge stemmed from the layout of the physical premises of mental health services themselves. Although this may be unique to a particular center, the impact upon recruitment was considered by trial staff to be large. For example, 2 researchers recalled the impact of the physical layout of premises, which hindered their ability to develop relationships with staff and acted as a significant block to successful social interactions:
The physical environment’s really problematic there [named recruitment site] as well, because they’re all in small, separate offices, so it doesn’t really feel like a team. So individual and...
There’s nowhere to circulate and to talk to the nurses.
There’s nowhere to chat amongst yourself, just to build the rapport with nurses. It was like, everyone’s all huddled away in separate offices.
As seen in the data from both the team meeting notes and focus groups, the research team was concerned that mental health staff sometimes acted as gatekeepers for some service users. This
Even when you approached them with eligible participants, they [staff] were maybe more likely to discount them straight away. Just say “no, they’re not suitable,” or “I don’t think they want to take part.”
Research staff working on EMPOWER theorized that mental health staffs’ resistance to research participation emerged because mental health staff believed that they were expected to participate in clinical research as part of their role as mental health clinicians. There were some concerns that if mental health staff felt that their participation in the project was mandatory, this may have limited their motivation and commitment, resulting in resistance to participation. In the following example, a member of the EMPOWER trial reflects on an encounter with a clinician who stated that they had to become involved because of expectations from management. This appeared to be linked with hierarchical relationships within mental health services. Therefore, clinical staff participating within research appeared to be a role expectation for clinical staff:
I remember one staff member talking about whether he agreed to be involved and he said “oh, do I really have a choice?” kind of saying “well, we've heard about it from, you know, management” and I got the sense he was communicating there was an expectation to get involved but that was just one thing I picked up about that kind of involvement. Yeah.
Trial staff used several trial work strategies to facilitate recruitment in face of barriers, including flexibility in approach to barriers, persistence, and emotional labor (trial staff managing feelings and expressions to successfully recruit participants), in addition to building relationships (using preexisting relationships with clinicians and using supportive research team relationships).
Regardless of how barriers to recruitment were negotiated, something that stood out in both the minutes and the focus groups was the need for trial staff to be flexible in their approaches. Discussions around the benefits of the flexible approach were common throughout both the Australian and UK focus groups. In the following example, a team member from Australia highlights that being flexible (and not rigid) in their approach to recruitment enabled staff to work through problems as they occurred:
I think that one of the real strengths in our research team has been how flexible and adaptive we’ve been when these challenges have come up, everyone involved in the process has been really thinking about ways to problem solve these things and coming up with suggestions.
One example trial staff provided, which illustrates taking a flexible approach, was in their discussions with clinical staff surrounding the trial protocol. Within a feasibility study, information about the recruitment process is a key outcome. Therefore, when encountering potential staff
...and our primary method of trying to get around that was to blame a third party to blame the protocol which says we needed to screen everyone and invite everyone rather than, you know directly, it feeling [sic] more like a direct challenge to the judgement of the key clinicians.
The researcher noted in their reflective memo that flexibility appeared a key process that emerged from the very beginning of recruitment when trial staff were working to build relationships and engage with the staff. Trial staff did not appear to rigidly stick to one recruitment approach:
When looking through minutes from the start of the trial. I am struck by how apparent flexibility was from the early stages of recruitment. For example, working around the availability of clinical staff as much as was possible. Furthermore, it feels important to note that because clinical staff are so busy that being flexible appeared essential in moving recruitment forward. However, in later stages flexibility involved clinical trial staff.
Within EMPOWER,
It depended quite a lot on the key workers that were involved within teams. How open they were to the study, and how much they followed through on things they said they were going to do. So, a lot of the time was spent chasing up key workers who said they would do something, and then didn’t.
Although the need to be persistent in chasing up mental health staff and trying out different recruitment strategies was apparent from both the minutes of meetings and the focus groups, the focus groups foregrounded an important role for the emotional aspects of recruitment within a clinical trial. In the following example, it is clear that simply being persistent is not enough and that it is important for it not to be obvious that the research team experienced frustration. Indeed, the need to portray constant positivity to get the work done appeared to be considered key in successfully recruiting participants. Therefore, there appeared to be an important role for
Persistence. Always smiling. Always the utmost professionalism.
Sometimes it’s fake. [shared laughter].
To the best of my knowledge, no trial staff used the term emotional labour to describe the maintaining professionalism during interactions with mental health staff, carers and patients. However, when reflecting on my observations of the research process, emotional labour appeared a highly relevant interactional framework for understanding the actual work underpinning trial staff describing the competency of staying polite and professional even when faced with potentially stressful challenges. Emotional labour seemed especially pertinent because trial staff are trying to invoke positive feelings within clinical research staff to build trust in both the project and the research team themselves.
Trial work appeared to be sustained and facilitated by relationship building. When trial staff described the work that they performed throughout the recruitment process, at all stages, the work appeared to be underpinned by trial staffs’ ability to successfully build and use relationships. In the absence of the ability to tap into existing relationships, trial staff had to be able to quickly build working relationships with clinical staff to facilitate the recruitment process. Reflecting on the overall emergent process, trial staff centered on the importance of building relationships with clinical staff in both the United Kingdom and Australia. One key change that came from this was trial staff becoming trusted to make direct approaches to patients instead of always having to go through mental health staff:
I think the reason that it became more possible was um that the services got used to the research team and got confident in the research team, or at least management did, so I think there’s something about us building the relationship that enabled us to move into a different way of doing it.
By appraising the minutes of team meetings, it is clear that trial staff initially had to go almost entirely through mental health staff. However, if a good relationship was built, this was perceived as helpful for recruitment because the staff were generally more engaged with the team:
Within two months, trial work moved on to the establishment of relationships between mental health staff and the research team. In this stage, the EMPOWER staff became trusted to make direct approaches. Linked to the process of building relationships over time with mental health staff, in both Glasgow and Melbourne, a clinical team member [Research Nurse and Peer Support Worker, respectively] became involved in trial recruitment. Both teams reflected upon this positively because both of these clinical team members brought their pre-existing relationships with clinical staff. While the earlier stages of recruitment may have seemed slow, it appears productive in terms of carrying out trial work that built relationships and trust with clinical staff, ultimately moving trial recruitment forward.
Although building relationships underpinned all aspects of trial work, preexisting relationships were described as helpful in establishing clinician trust. The
I think the real turning point where [peer support worker who participated in recruitment process] was speaking to somebody perhaps because she has that more casual kind of pre-existing relationship with some of these people where they were explicitly saying “I’m a bit sick of this EMPOWER stuff” and that’s when you know, that sent out the message we need to pump the brakes hard in terms of how much we are asking clinicians to do here.
Relationships appeared to serve important internal functions within the EMPOWER team. Across both the United Kingdom and Australia, trial staff made reference to the importance of having team members who understood the challenges associated with clinical trial recruitment. Furthermore, the importance of having space to be open about difficulties encountered, so that discussions were focused on how best to move forward, was described:
Because I think at times it is quite demotivating. And particularly if you’ve got that third [unanswered] phone call and think “please just answer the phone.” I think we [trial recruitment staff] do try and support each other through those times.
From the meeting minutes, being part of the UK meetings while recruitment was on-going and appraising themes constructed during the focus groups, it seemed as though having a space within the trial team to discuss and share frustrations that were inevitable from negotiating the various recruitment barriers. From my observations of actual meetings and continued within the focus groups, there appeared to be lots of in-jokes within the teams about the recruitment process including challenging aspects. For trial staff, this appeared to provide camaraderie and support.
To summarize, relationship building internally within the team appeared to be just as important in facilitating the recruitment process as building external relationships with mental health staff. Trial staff were there for each other throughout recruitment challenges and provided a supportive space for each other to discuss problems.
This study explored recruitment from the point of view of trial staff working on a digital intervention for psychosis. Detailed descriptions of the recruitment process are rarely reported within RCTs of digital interventions for psychosis, which minimizes the opportunity for sharing learning on how best to overcome recruitment barriers. By examining the recruitment process in EMPOWER using ethnography supplemented with focus groups, we now present such a detailed description. In doing so, we demonstrate not only the kind of recruitment barriers encountered by trial staff but also what strategies trial staff use to overcome them. Recruitment barriers appeared to span macro (structure and systems, eg, lack of staff time), meso (roles, eg, staff leadership), and micro (idiosyncratic, eg, the physical layout of community mental health premises) levels. The findings from this qualitative study suggest that simply reporting the number of participants recruited (n=86) clouds a highly complex social process underpinning trial recruitment. Taken together, the findings from this study can start to theorize the recruitment barriers and facilitators within the recruitment process for the EMPOWER trial.
Although it has been recommended that research exploring recruitment barriers should go beyond reporting a lack of staff time [
To negotiate complex recruitment barriers, trial staff put significant amounts of work in to engaging mental health staff during the recruitment process. Trial work is multifactorial and comprises emotional labor and social and professional competencies. Initially, in performing trial work, staff in EMPOWER reported the importance of persistence, being flexible in trying different approaches, and always being professional in their interactions with staff. Previous research on clinical trial staff has suggested that emotional labor is a key part of trial work when staff are working to meet recruitment targets [
Clinicians’ exclusion of people independent of trial protocol criteria is noted to be a key challenge in mental health intervention recruitment [
Mental health staff have perceptions of what is required from them professionally, and these perceptions seemed to cause tension and role conflict during the recruitment process. For example, clinical staff may not feel that they have the autonomy to decline participation because participating in research is a role expectation for clinical staff. Previous oncology research has indicated that nurses involved in conducting research describe a role conflict, where duty of care to the patient can sit uncomfortably with
Persistence and flexibility of approach were important in negotiating everything from macrolevel barriers, such as a lack of staff time, to more microlevel issues, such as community mental health center managers with different styles. One key element of the flexible approach to recruitment that emerged during the EMPOWER trial was a peer support worker (a person with their own experiences of psychosis employed to support people in their use of the intervention) advising how to approach recruitment challenges. A review concluded that patient involvement in clinical research may be associated with increased recruitment (but not retention) in clinical trials [
The research team reported that conveying to staff that highlighting the importance of gathering data on rates of participant refusal was helpful in negotiating filtering behavior by clinical staff. Future research could explore this observed phenomenon further, perhaps using relevant behavioral change theories as a theoretical framework [
EMPOWER was a feasibility study, which means there were a limited number of trial staff to observe and speak to. Beyond the small sample, the findings from this study should be considered in light of several key limitations. Ethnography is an opportunistic methodology [
Facilitators addressing ongoing
Exploring recruitment from the perspective of trial staff provides rich insights into barriers and facilitators to recruitment within clinical trials of digital intervention. For example, rather than people diagnosed with schizophrenia being a
community mental health services
Consolidated Standards of Reporting Trials
cluster randomized control trial
Early Signs Monitoring to Prevent Relapse in Psychosis and Promote Well-being, Engagement, and Recovery
nonadoption, abandonment, scale-up, spread, and sustainability
randomized control trial
study within a trial
The authors are grateful to all the service users, carers and mental health staff, and CMHS who gave their time and resources to contribute to the development of the EMPOWER study during the consultation phase and to the c-RCT and the process evaluation. This was critical in developing the process evaluation.
None declared.