This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on http://humanfactors.jmir.org, as well as this copyright and license information must be included.
A significant number of chronic pain patients experience a decline in therapeutic effects after rehabilitation. As face-to-face contacts with health care professionals are not always feasible after treatment, new, innovative, fully automated relapse-prevention programs are highly needed.
In this study an online, automated relapse-prevention program based on acceptance and commitment therapy (ACT)—both as a website and as a mobile app—was developed and evaluated. At each step of the development, end users (ie, chronic pain patients) were consulted in order to fully address their needs.
In a step-by-step process, a contextual inquiry, requirement specification, and design were executed with chronic pain patients by conducting, respectively, a focus group (n=10), interviews with rapid prototyping (n=28), and a user- and expert-based usability evaluation (n=14). Furthermore, a pilot evaluation was conducted with 14 chronic pain or fatigue patients who had received the online relapse-prevention program following a multidisciplinary ACT treatment. They were interviewed about their usage and the usefulness of the program in supporting them to maintain changed behaviors and prevent relapses in avoidance and pain control behaviors.
The three stages provided information about the expected needs of end users, comments about the usefulness of the proposed features, and feedback about the design and usability of the program. This resulted in a fully operational, online relapse-prevention program. Results from the pilot evaluation showed that 9 patients used the online program after treatment, 5 of whom indicated that the program supported them after treatment. Of all the patients, 4 of them indicated that the program did not support them because they wanted more social interaction with other users.
This study showed that an innovative, automated, online program that is user friendly can be developed by involving the end users in each step. The program was evaluated positively by some participants. The evaluation showed that the online relapse-prevention program has the potential to support chronic pain patients in maintaining their changed behaviors and preventing relapses in avoidance and pain control behaviors.
Nederlands Trial Register (NTR) Number: NTR4177; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=4177 (Archived by WebCite at http://www.webcitation.org/6Ur6EFD1D).
Multidisciplinary rehabilitation programs based on cognitive behavioral therapy (CBT) or acceptance and commitment therapy (ACT) for chronic pain patients have shown positive effects on the interference of pain in daily life and on physical and mental functioning [
In this study we describe the development of a new, online relapse-prevention program—in the form of a website and/or mobile app—based on ACT. The main focus of ACT is enhancing psychological flexibility which includes the processes of acceptance and value-based behavior [
However, despite the promising results of eHealth interventions for chronic pain, the effect sizes are often less than expected. One reason for this might lie in the low adherence rates of eHealth interventions, for instance, participants are not following the intervention as intended. For example, in the study of Moessner et al it was shown that a Web-based aftercare intervention following multidisciplinary therapy for chronic back pain had positive effects on pain intensity, but 70% of the patients did not adhere to the intervention [
The first aim of this study is to develop an innovative, fully automated, online relapse-prevention program that enables rehabilitation centers to implement this program. For the development process, a user-centered design was used that might overcome the problems described above. A user-centered design has been shown to have positive effects, especially on user-satisfaction levels and on addressing user needs [
In the contextual inquiry, intended users are asked for information about their needs for a technology. Contextual inquiry also examines how the technology may fit into the day-to-day life of the intended users [
Respondents were asked to discuss what they believed would help them to prevent relapses and whether they would use a website and/or a mobile app as a relapse-prevention program. They were also asked whether they wanted to have guidance by email or SMS text message. This focus group session was audiotaped with the permission of the respondents. The audiotapes were transcribed and were analyzed qualitatively by the researcher by summarizing common themes.
In the requirement specification activity the expected needs are translated into requirements of the technology [
1. Values and actions: Users could add or change various life values and corresponding actions.
2. Diary: The diary is for monitoring value-based living and included one question that asked whether the user had lived according to his/her values on a rating scale from 1 to 10. The answers to this question were presented in a chart including positive smileys if a score of 6 or more was achieved.
3. Exercises: Users could add various ACT exercises or search in a database that contains all the exercises included in the treatment.
4. Tips: Users could add their own tips, share these tips with other users, or see shared tips from other users.
5. Coach: Various options of guidance with SMS text message or email were shown to the participants and they could all choose which text message they would like to receive: (1) a reminder SMS text message to use the diary or to use the program itself, (2) a motivational SMS text message written by a health care professional, (3) a self-composed motivational SMS text message which could be sent at a later date, or (4) a motivational SMS text message tailored to particular scores that are obtained from the diary—this tailors the content more to the individual situation.
To obtain the requirements, semistructured interviews combined with the prototypes of the eHealth technology were conducted with chronic pain patients [
Example of a website prototype (Values and actions).
Example of a mobile app prototype (Exercises).
A fully operational program, both as a website and as a mobile app, was developed. For examples of these, see
In this phase, the quality of the design was examined by evaluating its usability. Consequently, a user-based and expert-based evaluation method using a scenario-based think-aloud protocol was performed [
Screenshot of a page of the website (Values and actions).
Screenshot of the mobile app homepage.
Based on the results of the usability evaluation, the system quality was improved and a manual was developed to give the users more information, for example, whom the tips were shared with. A pilot user evaluation was conducted with patients who received the online program after a multidisciplinary pain treatment for 2 months. This study was approved by an independent Dutch medical ethics committee (Medical Research Ethics Committee Twente, no. P13-07) and recorded in the Dutch primary trial registry for clinical trials (Nederlands Trial Register, NTR4177).
Inclusion criteria were chronic pain or fatigue patients who were following an inpatient 8- to 12-week, group pain treatment which started in March or April 2013 at the Pain Department of the Roessingh Rehabilitation Centre. Chronic pain and chronic fatigue patients receive the same multidisciplinary treatment in the rehabilitation center. Patients needed to have access to the Internet through a computer and/or a mobile phone at home. The information letter described that it was recommended to use the online program for 5 to 10 minutes daily. The researcher had access to the actual accessed data use of the program, including how many times the participants were logged in and which features they had used, so self-report data use could be compared to the actual usage. In the fourth week of their treatment, the researcher explained the study and the patients received a letter with information about the study and a letter requesting their informed consent. In the sixth week of the treatment, patients who agreed to participate handed over the signed informed consent, were given access to the online ACT relapse-prevention program, and received a short introduction to the program. Following a period of 2 months after the group treatment, all of the participants were invited for a telephonic, structured interview conducted by the researcher (MF). The interviews were audiotaped with the permission of the participants and took, on average, 20 minutes to complete. The interview scheme started with a general question on the helpfulness of the online ACT relapse-prevention program in supporting them with the actions they intended to carry out after treatment. Participants were asked how many times they had used the program and whether they intended to continue using the program. The interview continued with an open discussion to identify which parts of the intervention they found to be most useful. Furthermore, they were also asked whether they would recommend this program to other patients. The interview ended with the question “Do you have any recommendations for improving the program?”
In total, there were 5 groups that started with the pain treatment with a total of 27 patients. All of them were given a letter with information about the program and a form requesting their informed consent. The informed consent form was signed by 17 patients and they subsequently received the online program. The mean age of the participants was 38 years and most of them were female (14/17, 82%). All 17 patients were invited for the interview and 14 of them were interviewed by the researcher. The reasons for not taking part in the interview were medical (1/17, 6%), feeling very good (1/17, 6%), and unknown (1/17, 6%).
All participants indicated that there is a need for an aftercare program and they would all like to have contact with their health professionals after treatment, for instance, through a telephone call or individual face-to-face contact. Most respondents would use an eHealth program, such as a website or mobile app. In this online program 8 participants out of 10 (80%) found it useful to register their values and actions and to read ACT-based exercises. Of the 10 participants, 4 of them (40%) thought it would be useful to make a start with completing these exercises and registering their values and actions during treatment. All participants would like to have contact with other patients to share tips and to give each other pep talks. Of the 10 participants, 2 of them (20%) proposed a forum for facilitating this contact with other patients. All participants would like to receive reminders by SMS text messaging or email from a health care professional to help them adhere to value-based behavior after treatment. Of the 10 participants, 8 of them (80%) indicated that reminders could be preformulated and sent automatically, preferably at a higher frequency rate and directly after the end of the treatment. Of the 10 participants, 3 of them (30%) also thought that it would be helpful to add their own motivational texts during treatment in the online program that can be sent after treatment. Additionally, 2 participants out of 10 (20%) indicated that the text messages should be tailored to the individual patient’s personal situation.
Almost all of the participants indicated that they would use the online aftercare program (27/28, 96%). Some stated that they would like to use a website (12/28, 43%), a combination of a website and a mobile app (9/28, 32%), or only a mobile app (7/28, 25%). In general, participants found the design of the five features to be clear and convenient.
User evaluation of the features of the online relapse-prevention program (n=28).
Features | Usefulness, n (%) | Use, n (%) | Moment of use | Expected needs |
Values and actions | 28 (100%) | 26 (93%) | During and after treatment. | Database with some examples of actions. |
Diary | 28 (100%) | 25 (89%) | After treatment. | Changing the name “Diary”. |
Exercises | 28 (100%) | 27 (96%) | During and after treatment. | Database with some examples of exercises. |
Tips | 28 (100%) | 27 (96%) | During and after treatment. | Tips from a health care professional. |
Coach | 28 (100%) | 28 (100%) | After treatment. | Preference for receiving an SMS text message instead of an email. |
All comments are represented in
Number and positivity or negativity of comments yielded from the user- and expert-based methods.
Property of program | Number of comments by usersa | Number of comments by expertsb | Total | ||||
|
+c | +/-d | -e | + | +/- | - |
|
System | 45 | 5 | 22 | 52 | 10 | 65 | 199 |
Content | 2 | 3 | 8 | 0 | 4 | 19 | 36 |
Service | 15 | 0 | 3 | 11 | 2 | 10 | 41 |
aThere were 5 users.
bThere were 9 experts.
cPositive comment.
dNeutral comment.
eNegative comment.
Results of the pilot evaluation.
Participants (n=14) | n (%) | ||
|
5 (36) | ||
|
Preference for use during treatment | 4 (29) | |
|
Recommend to other patients | 5 (36) | |
|
Use in future | 3 (21) | |
|
|
|
|
|
|
Motivational messages | 2 (14) |
|
5 (36) | ||
|
Preference for use during treatment | 3 (21) | |
|
Recommend to other patients | 5 (36) | |
|
Use in future | 5 (36) | |
|
|
|
|
|
|
(Mindfulness) exercises | 4 (29) |
|
|
Motivational messages | 3 (21) |
|
|
Tips | 3 (21) |
|
4 (29) | ||
|
Preference for use during treatment | 3 (21) | |
|
Recommend to other patients | 2 (14) | |
|
Use in future | 4 (29) | |
|
|
|
|
|
|
Motivational messages | 3 (21) |
|
|
Tips | 1 (7) |
aThe online program was supportive in maintaining behavioral changes and preventing relapses in avoidance and pain control behaviors.
bThe online program was not supportive in maintaining behavioral changes and preventing relapses in avoidance and pain control behaviors.
Reasons for nonuse were feeling well at that particular moment (3/14, 21%), a medical reason (1/14, 7%), and having other priorities (1/14, 7%). Of the 5 nonusers, 2 of them (40%) received motivational messages and found them very pleasant and supporting. Of the 5 nonusers, 3 of them (60%) thought they would use the program in the near future, especially in the case of a relapse (2/5, 40%). No suggestions for improvements were given.
Out of the 14 participants, 5 of them (36%) indicated that the online program supported them in their efforts to perform their intended actions. Out of these 5 participants, 2 of them (40%) used the online program regularly (eg, at least once a week) and the other 3 participants (60%) used it 3 to 5 times in total. The online program functioned as a summary of the treatment and reminded them of what they had learned during treatment. The (mindfulness) exercises, the motivational messages, and the tips were evaluated as being “useful” and “very pleasant” as they functioned as reminders about the treatment. One participant recommended changing the question “How are you?” to more personal actions.
Of the 14 participants, 4 of them (29%) indicated that the program did not support them after treatment. Of these 4 participants, 3 of them (75%) used the program 1 to 3 times, and 1 of them (25%) used the program at least once a week. All 4 participants out of 14 (29%) indicated that they missed the interaction with other patients in the program, for example, using a forum to share experiences with other users. Participants were satisfied with the motivational messages and the exercises because they served as a reminder for the treatment. They will continue to use the program, particularly the motivational messages (3/4, 75%), as well as the tips and the values and actions (1/4, 25%).
In this study, a new, innovative, online relapse-prevention program for chronic pain patients based on acceptance and commitment therapy was developed and evaluated. The first aim of the study was to develop an automated, user-friendly, online relapse-prevention program that fulfills the needs of chronic pain patients. A contextual inquiry by focus group discussion, requirement specification by interviews with rapid prototyping, and a user- and expert-based usability evaluation of the fully operational program successively provided the input for the next step in the development process. The chronic pain patients reviewed the program from their points of view and context, both on design and content for ensuring that the eHealth program was usable and acceptable. Accordingly, our user-centered developmental process resulted in a program with a simple design, large icons, and few layers of information and text. A pilot evaluation with chronic pain or fatigued patients who had received the online relapse-prevention program following a multidisciplinary ACT treatment (n=14) showed that this development process was satisfactory. Two-thirds of the participants (9/14, 64%) used the program in the 2 months after treatment. They did not have new suggestions on the usability of the program and nonusers (5/14, 36%) stated that their nonuse was not caused by complexity or inadequate usability of the program. This probably improves the uptake of the online relapse-prevention program, as many problems with adherence to eHealth programs are due to complexity and inadequate usability [
Furthermore, the user-centered development process resulted in information on the needs concerning the content of the program. The resulting program consists of the essential building blocks of a relapse-prevention program [
The second aim of the study was to evaluate whether the online program supported patients after treatment in maintaining their changed behaviors and preventing relapses in avoidance and pain-control behaviors. The results of the pilot evaluation showed that, most of all, the motivational messages and the exercises were evaluated as very useful and pleasant. Results from the pilot evaluation showed that 9 patients out of 14 (64%) used the online program after treatment, 5 (5/9, 56%) of whom indicated that the program supported them after treatment. In general, the adherence rates and the evaluation of the program were disappointing. Although it was recommended to use the program daily, the 9 participants out of 14 (64%) that actually used the program only used it once a week or 1 to 5 times in total. Furthermore, 5 out of 14 (36%) participants never used the program. Of these 5 nonusers, 3 (60%) of them indicated that they will use the program in the near future, but we do not know whether these participants really intended to use the program or whether they were providing socially desirable answers. Our results corroborate with earlier results on low adherence rates in online programs for chronic pain patients [
A possible explanation for the low adherence in this study is that most patients had an initial preference with using the program during treatment, particularly with regard to registration of values and values-based actions. For further use, we recommend integrating the program into the multidisciplinary pain treatment program and to train health care professionals to inform and motivate patients to use the program during treatment. Furthermore, the program is not yet successful in providing sufficient social support. The 4 participants out of 14 (29%) who stated that the program was not helpful in the period after treatment indicated that they missed interacting with other participants. Therefore, it is advisable to implement a function where users can react to the shared tips.
Limitations of this study included the small samples of users involved in each step and the recruitment of all participants from the same rehabilitation center. Although generalization has to be made with care, the number of participants concurs with the recommended numbers by other studies [
Relapse prevention is still a topic that is hardly examined in the area of chronic pain [
acceptance and commitment therapy
cognitive behavioral therapy
information and communications technology
short message service
The authors would like to thank the Ministry of Health, Welfare and Sport of the Netherlands for funding this project.
None declared.