Hand and upper limb injuries are some of the most common injuries in orthopedic settings [1,2], and approximately 11% to 20% of emergency department visits are because of hand and upper limb injuries [3,4]. In addition to pathophysiology, psychosocial factors can predict disability in individuals with hand and upper limb injuries [5,6]. These injuries have been shown to impact employment, body image [7], relationships [8], and functional abilities [9-11] negatively.

Most studies conducted in hand therapy have focused on maximizing physical recovery and adjustments with regard to medical or occupational therapy procedures [12-15]. Interventions such as joint protection [16], exercise therapy [17], mobilization [18], and modalities [19] in hand therapy have well-established benefits for pain and function. However, they do not directly target psychosocial factors that contribute to patient morbidity [20]. Several studies have established the mediating effect of psychological distress on hand and upper limb pain and disability [21-24] based on the far-reaching impact of psychosocial problems on pain and disability, and patient expectations after hand and upper limb injuries, a greater understanding of how to facilitate psychosocial adjustments is warranted [2,25]. Psychological interventions such as cognitive behavioral therapy (CBT) interventions have been shown to yield long-term [26] improvements in pain, daily function, quality of life, and overall mental health compared with active treatments alone for several musculoskeletal (MSK) problems [27,28] including knee pain [29], low back pain, [30,31] fibromyalgia, [32] preoperative spine [33] and post total joint surgery [34]. CBT is also cost-effective [35] and cost neutral when considering the overall health care sector and labor market perspective [33], with reduced health care utilization at the 5-year follow-up [26]. CBT has also been shown to be effective in improving adherence to exercise [36]. CBT techniques such as graded activity can be integrated into traditional physiotherapy [37,38].

In hand therapy, CBT may be efficient treatment to improve pain and distress by increasing adjustment to hand injury in relation to illness perception and coping strategies [2,39-41]. Unfortunately, traditional CBT is resource intensive and not feasible to implement in busy hand therapy practices because of prolonged face-to-face encounters and associated cost implications [42]. Web-based CBT is a potential emerging tool with modern interactive and communicative technologies for use in rural and urban areas, across languages and cultures, and on a global scale [43]. Web-based CBT has been shown to be effective for reducing catastrophization and improving the attitudes of patients with MSK conditions to exercise therapy [44]. Current evidence supports the feasible and efficacious delivery of web-based CBT using nontraditional health professionals such as physiotherapists (PTs) and occupational therapists (OTs) [45], with reduced time commitment and treatment costs, and positive self-reported changes in the PTs’ attitudes, confidence, and practice [46-48]. Therapist competence and therapeutic alliance are crucial factors influencing CBT [49]. Therapist competency can be developed online [50], and therapeutic alliance required for CBT to be effective does not diminish with the web-based delivery of CBT [51].

Hagemen et al [52] reported that almost 50% of outpatients presenting to hand surgery clinics investigated their symptoms online, which increases the potential to deliver evidence-based pain management and coping skills for HULI online. Further studies on the use of psychosocial interventions in HULI have the potential to convince payers to fund psychotherapy treatments, generate enthusiasm to include psychosocial treatments in educational curriculums, and advance incorporation of evidence-based psychosocial treatments in hand therapy recommendations for psychosocial problems [53]. In view of the evidence showing evidence-based CBT can be delivered via the internet and feasible to implement during wait times for hand therapy or in home-settings and reduce the costs associated with training providers and fewer hospital visits. From the foregoing, online evidence-based CBT is feasible to implement during wait times for hand therapy, is easy to use in home settings, and reduces costs associated with training providers and leads to fewer hospital visits. To meet the needs of patients with hand and upper limb injuries at risk of prolonged pain and disability because of psychosocial treatments, we decided to develop an intervention that incorporates evidence from CBT in orthopedic practice.

Hand Therapy Online Coping Skills (HOCOS) is an evidence-based and theory-based psychosocial coping skills program based on CBT principles [37] and the Technology Acceptance Model [54]. HOCOS was developed by Folarin Babatunde (PT) during his doctoral studies at McMaster University In collaboration with a team of PTs (JM and LM), OT (MS), hand therapists (JM and MS) and an orthopedic surgeon (RG). HOCOS involves five ‘hand therapist-guided’ modules. It is a multi-component, interactive online-based program consisting of hand and upper-limb specific information covering pain education and training in coping skills (activity-rest cycling, pleasant activity scheduling, problem-solving, identifying and challenging negative thoughts, relaxation response and their applications) to daily life for adults with hand and upper limb injuries (Table 1). Asynchronous learning was facilitated using PowerPoint (Microsoft Office) presentations, audio files, workbooks, and downloadable PDF (Adobe) and Word (Microsoft Office) files. Links to evidence-based external educational resources were included to reinforce learning. The program was designed to supplement traditional hand therapy with coping skills training. The design and development of HOCOS were guided by the 5 steps of the Analysis, Design, Development, Implementation, and Evaluation from the ADDIE model [55,56]. The structure and specific ingredients of HOCOS were based on the following recommendations suggested by Bennell et al [57]: (1) impact of psychosocial factors on pain and disability in hand injuries, (2) evidence base for the effects of CBT on MSK conditions, (3) the importance of incorporating the management of hand injuries into a biopsychosocial framework, and (4) practical issues related to the delivery of the intervention.

This paper aimed to provide a brief overview of the web-based system and to report on its usability from the perspectives of information and communications technology (ICT) experts and clinicians practicing in the field of hand therapy. Usability testing is a critical step in the development of online interventions and involves obtaining feedback to understand what is positive or negative about a system and identify existing gaps in content or functionality using iterative cycles of prototype alteration [58].

A mixed methods usability testing approach with semistructured interviews, observations, and questionnaires was undertaken, with iterative cycles to determine the usability of HOCOS and to further refine the prototype [59,60]. Participants were recruited using snowball sampling by asking key informants to suggest another participant who they believe is suitable for the study and introducing that person to the researcher [61]. This paper reported on step 3 of the ADDIE process (Figure 1). ADDIE is commonly used in instructional development as a systematic way to achieve the desired results [62].

We recruited ICT experts online through the Weebly support (Weebly) portal to participate as heuristic evaluators in phase 1 of usability testing. In phase 2, PTs and OTs based in Ontario, Canada, and practicing in the clinical area of hand therapy were invited to participate in the study to enhance the development of HOCOS. Clinicians were messaged directly using contact details available to the public on the Canadian Society of Hand Therapy (CSHT) website. Interested participants contacted the research team directly by telephone or email and were provided with a letter of information and signed consent forms before data collection. Log-in access to the password-protected HOCOS website was provided free of charge.

One of the researchers, FB, facilitated data collection by conducting interviews, taking notes, and observing participants’ behavior. Appointments were made to meet with participants at the study site or at a desired destination within 2 hours of the study site. A brief description of the study was provided to each participant, with emphasis that the evaluation was about the content and functionality of the website. An explanation of cognitive interviews and information about privacy, and protection of the data collected were also provided. Before the interviews, demographic data, including age, gender, educational level, practice area, and use of technology, were collected. All participants were identified by pseudonyms to ensure anonymity [63]. According to Nielsen [56], 5 users are adequate to identify most usability problems. Current evidence shows that 80% of usability problems can be identified with 4 to 9 participants and 95% with 9 participants [64], thus we proposed a convenience sample of 12 participants for usability testing and to account for attrition. The usability testing protocol was approved by the Western University Health Sciences Research Ethics Board (no. 108064). Guided by steps 1 and 2 (Figure 1), we revised the prototype and developed a preliminary version of HOCOS that was tested by ICT experts (n=4) and therapists (n=12). No incentive was provided to participate in this study. Parking costs were covered for participants involved in face-to-face cognitive interviews.

All interviews were audiotaped and transcribed verbatim in an anonymized format. The usability testing and interview data were analyzed together using triangulation [76]. Content analysis [77] of transcripts from the thinking aloud sessions, field notes, and feedback interviews was coded using predetermined codes related to usability issues (navigation, content, layout, learnability, errors, and satisfaction) after each iterative cycle. The interviews from the first cycle were analyzed and used to make minor modifications to the website before evaluation in the second cycle of testing. Very few modifications to the prototype were required after the second cycle of testing. To calculate the SUS score, the score contributions from each item are summed. Each item’s score contribution ranges from 0 to 4. For items 1, 3, 5, 7, and 9, the score contribution is the scale position minus 1. For items 2, 4, 6, 8, and 10, the contribution is 5 minus the scale position. Quantitative data from SUS (10 questions, each scored from 0 to 4 points) were transformed by multiplying by 2.5 to convert scores to a 0 to 100 range and categorized using adjective ratings [74]. The descriptive analysis (means and SD) of the quantitative data was conducted using Stata 13 software for Microsoft Office.

We enrolled four ICT experts to act as evaluators during heuristics evaluation, which meets the optimal requirement for detecting all usability problems [78]. During user testing, 26 clinicians agreed to participate in this study (14 for needs assessment, and 12 for usability testing). A total of 69% (18/26) of participants were female (Table 2). Most participants (17/26, 65%) had a background in occupational therapy, had at least 16 years of experience in hand therapy (10/26, 38%), and practiced in outpatient rehabilitation facility (10/26, 38%). Most therapists were very comfortable using a computer/tablet or internet. See Table 2 for participants’ characteristics.

The heuristic evaluation of HOCOS against the HLO checklist identified violations in 15 of the 35 criteria with violations seen across all domains (Multimedia Appendix 2). Domain 1 showed violations in 2 of the 7 criteria. There were 4 violations in the 13 criteria for domain 2. Most of the violations were represented in domain 3, with 6 of the 10 violations reported. Violations included (1) the home page image not representing the context of the website, (2) lack of a search function, and (3) links that are difficult to differentiate from the surrounding text or other graphic elements. Corrections were made and included adding a welcome image on the home page, adding a search function, and creating a box around link icons. Domain 4 revealed 3 violations in the 5 criteria because of heavy reliance on text-based information, lack of quizzes or forms, and lack of social media sharing options. We included more pictures and reduced the words per page, creating a separate link for the form. We decided against adding a social media link because of privacy concerns and the sensitive nature of psychosocial issues. Evaluation of HOCOS using Monkman heuristics identified violations in 5 of the 11 criteria (Multimedia Appendix 3) including lack of options for tailoring information to the user, poor use of plain language including medical jargon and Gunning Fog readability index greater than 8 [79,80], information in multiple languages, few succinct summaries versus more detailed information, need for scrolling to find important information, and poor communication of risks. The remaining violations were managed by adding activities that could be personalized, editing the content for therapists and patients using the Gunning Fog index, creating a summary of key points in the slides, adding an icon to important information, and adding a disclaimer to express inherent risks and benefits of the program.

This included findings from the user task performance and cognitive interviews (thinking aloud) components of usability testing of the HOCOS.

The overall objectives of this paper were to provide an overview of HOCOS and report the findings from usability testing with ICT experts and clinicians practicing hand therapy. HOCOS is designed to help patients with hand and upper limb injuries learn how to better manage psychosocial issues. The uniqueness of HOCOS is an interface design that offers learning opportunities to both clinicians and patients. Overall, therapists were pleased with the objective and content of HOCOS and found it a useful resource for meeting patients’ needs in hand therapy.

Formal usability testing is a key process required to ensure the relevance of content and make the website easy to use, learn, efficient, and acceptable to users [75]. Usability testing uncovered several violations during heuristic testing with ICT experts. Furthermore, user performance errors and areas for enhancing user satisfaction were also identified by therapists during user testing. Therapists reported some positive features of the website including being simple, user-friendly, and engaging and having a functional design that was accessible on several browsers following usability testing. Several changes were made to the online portal that corrected the errors uncovered and improved overall user satisfaction. Although HOCOS was initially designed for all modules to be completed together, feedback from clinicians highlighted the benefits of having modules as stand-alone options to reduce potential participant burden. Hand therapists expressed confidence that patients could execute the activities in the workbook, especially with the DIY guide. Testing also demonstrated that the primary condition of the patients determined the modules that therapists chose to introduce and apply in clinical practice. This process was enhanced by providing a guide on how to use the features, when to introduce the modules, and how the website may fit within the broader tool kits used in hand therapy.

This study contributes to the dearth of literature on the usability testing of web-based portals developed for managing psychosocial factors in orthopedic hand and upper extremity services. Chad-Friedman et al [84] reported the use of an online interface designed to deliver a brief 60-second mindfulness exercise for hand and upper limb pain with improvements in state anxiety, pain intensity, distress, anxiety, depression, and anger after watching the video. In another study by Westernberg et al [42], a free online mindfulness-based video exercise was targeted at individuals with upper extremity conditions and psychosocial problems. Study findings reported improvement in momentary pain, anxiety, depression, and anger in patients with low levels of pain and psychologic distress. Similarly, Vranceanu et al [85] described the Toolkit for Optimal Recovery (TOR), a 4-session, live video, and manualized program informed by the fear-avoidance model to prevent chronic pain in at-risk adults with orthopedic injuries. TOR combines relaxation response with CBT, Acceptance and Commitment therapy skills. HOCOS provides a larger platform offering multiple options to therapists and patients using concepts from pain education science, relaxation response and behavior change techniques. On the basis of postcognitive interview feedback, therapists involved in this study preferred an online program that teaches patients how to change maladaptive cognitions and not simply accept such thoughts for long term effects. To close existing gaps in the literature, HOCOS was designed using CBT principles which teaches patients how to challenge automatic thoughts by holding them up to disproving evidence and then change them into different thoughts [86,87]. CBT begins by identifying a primary treatment goal and continuous striving to meet those goals [88]. HOCOS also provides modules that can be targeted at psychosocial problems associated with acute to subacute and chronic hand and upper limb conditions with therapist guidance. This is important because the untargeted use of psychological interventions in hand therapy and when self-directed by patients has been shown to demonstrate no benefits [89].

Dissemination of evidence-based therapies remains poor in routine practice [90]. Although allied health care professionals (HCPs) are aware of the benefits of incorporating psychological interventions within their practice, they feel insufficiently trained to optimize their use of such interventions [91]. Barriers to practicing the evidence-based therapies include a lack of access to resources that contain such evidence [92,93] and limited usable formats of the evidence [94]. Training hand therapists to manage the physical and psychological sequelae of hand and upper limb conditions using HOCOS would increase their knowledge of psychosocial interventions and build their capacity and confidence to deliver it in clinical practice.

Our study should be viewed with consideration of certain limitations encountered. The study was conducted among hand therapists in Ontario, and most participants were comfortable using the computer and the internet, which limit the generalizability of the study results. This may not be representative of the end users, such as patients seen in most hand therapy clinics. In recruiting participants for this study, we chose snowball sampling, a form of convenience sampling. This increases the risk of compiling a nonrepresentative sample. We planned to create an online platform that is user friendly for a significant portion of patients with hand and upper limb injuries who are mostly elderly [95], low skilled [96], and with less education [97]. These groups of individuals tend to be less computer literate, and to this end, we did our best to incorporate recommendations to ensure accessibility and ease of use in the web design and simplify the user experience [66]. This included a larger font size, white space around texts, and a simple color scheme to enhance readability.

The presence of one of the researchers (FB) during the usability testing sessions may have affected the behavior of end users conducting the testing. The participants may have felt reluctant to be critical despite encouragement to highlight both weak and strong features of the website. Furthermore, we were unable to test the HOCOS website in the context of the patient-user’s experience to gain a comprehensive view of the system’s functioning in a clinical setting because of financial and time constraints. This needs to be addressed in future research by examining the effectiveness of HOCOS in a randomized controlled trial to determine if the present system design can contribute to improved outcomes in practice.

User testing of an online intervention should include the ultimate end users, including patients, to allow for the examination of factors related to participants (age, gender, and education), disease (severity and duration of symptoms), and experience (access to and comfort with using the internet and computers) [75,98]. On the basis of ergonomic quality and safety principles, it has been recommended that prototypes of eHealth interventions should be fully inspected and walked through by HCPs before exposure to potentially vulnerable user groups such as individuals with significant psychosocial problems after a hand injury [99]. Financial and time constraints were significant barriers to testing HOCOS in patients with hand injuries. The next phase of the project is to evaluate the impact of HOCOS training on the actual implementation of the program on patients. We plan to carry out further testing in a proof-of-concept study to establish if individuals with hand and upper limb conditions and psychosocial problems are willing and able to complete the HOCOS program, complete the activities correctly, and adhere to the program principles.

This study provides initial support for the usability of HOCOS. Ensuring that therapists were involved in the design and development process of HOCOS enhanced the user-centeredness and user-friendliness of the website. Usability testing during the formative stage of eHealth intervention development is necessary to ensure that online interventions are effective and acceptable to potential users. HOCOS has the potential to increase access and acceptability of coping skills training programs for many individuals with hand and upper limb injuries who are not able to receive hospital- or clinic-based treatment psychotherapy. We plan to conduct a pilot study to determine the feasibility of the website for adults with hand and upper limb injuries and further refine the tool for a fully powered randomized controlled trial. If effective in improving outcomes, this program could be used as a template to develop more interventions targeting the psychosocial challenges confronting individuals with hand and upper limb injuries.