Acquired brain injury (ABI) is defined as temporary or permanent damage to the brain that occurs after birth from a traumatic brain injury (TBI) or non-TBI [1]. ABI is a leading cause of death and disability in children [2]. After sustaining an ABI, outcomes vary based on several individual and injury-related factors, such as personality, preinjury strengths and needs, and location and severity of the injury [1-3]. Mild (ie, concussion), moderate, and severe brain injuries can lead to a variety of lasting cognitive, physical, and psychosocial outcomes [4-6] that affect students’ school performance [7,8].

Globally, educators report feeling underprepared and unaware of how to support students with ABI in the classroom [8-10]. School-aged children with ABI often experience challenges returning to school due in part to a lack of educator support and awareness of ABI [1]. Separate from an educators’ teaching approach, having ABI knowledge has been shown to influence academic and social domains for students [11]. Moreover, students with ABI report greater life satisfaction when their teachers are understanding of their needs and provide encouragement [11]. Therefore, a supportive school environment can facilitate successful school reintegration for children with ABI [12].

In Canada, children with ABI are a “silent voice” in the education system [13]. Outside of exploring additional education or training specific to developmental disorders, educators do not receive adequate instruction related to ABI during preservice training or as practicing professionals [14]. For example, in Ontario, Canada’s most populated province, the Education Act (1980) [15] separates students with special education needs into 5 broad categories: intellectual, behavioral, communication, physical, and multiple. ABI is not a separate category, and unfortunately, the evolving nature of ABI may make it difficult to fit students’ areas of need into a distinct category [1]. For example, the diverse range of ABI symptoms includes a combination of cognitive, physical, psychosocial, and communication concerns. Hence, ABI is a unique exceptionality due to its wide, significant, and individualized impact across many domains of functioning [14]. Identification within a category equips educators with additional knowledge and awareness of strategies to support students within the Ontario education system. In 2018, the passing of Bill 193, also known as Rowan’s Law [16], mandated requirements to enhance concussion safety in Ontario. The act was created to raise awareness about concussion and improve concussion safety within amateur competitive sport by mandating sport organizations to (1) have athletes review concussion awareness and education resources approved by the Minister of Tourism, Culture and Sport; (2) develop a concussion code of conduct and have athletes review the code; and (3) establish a removal-from-sport and a return-to-sport protocol [16]. Many Ontario school boards responded to Rowan’s Law by implementing yearly concussion training for educators; however, evidence suggests that this training is brief and focuses on signs and symptoms, rather than addressing potential long-term impacts and how to support deficits [17]. Furthermore, concussion is only 1 condition under the diverse umbrella of ABI. Therefore, a gap in training related to mild, moderate, and severe ABI remains. Recently, Stevens and colleagues [10] confirmed that Ontario educators lack the knowledge and confidence to support students with ABI in the classroom. Ontario educators also reported the need for a course to improve their knowledge and awareness of pediatric ABI. Researchers at the Holland Bloorview Kids Rehabilitation Hospital (HBKRH) in Toronto, Ontario, responded to this need by creating an online professional development module called Teach-ABI.

A mixed method prospective study design was used. Elementary educators were recruited including preservice teachers, classroom teachers, special education teachers, principals, vice principals, registered early childhood educators, and educational assistants. The authors acknowledge that the word educator is an umbrella term that encompasses teachers and other education-related professionals, the same way that the school environment is a term that includes the classroom and other aspects of school such as the playground. For the remainder of this paper, we will be using these terms interchangeably, in a similar fashion as our participants, to best reflect participant data.

Individuals were eligible to participate in the study if they met the following criteria: (1) currently enrolled in a teacher’s college program that will provide certification as an elementary school teacher with the Ontario College of Teachers; or (2) currently registered with the Ontario College of Teachers as an elementary school teacher (primary/junior or junior/intermediate teaching qualifications); or (3) currently working in an Ontario public elementary school as an educational assistant or registered early childhood educator. None of the interested participants met the following exclusion criteria: (1) non-English speaking; or (2) had cognitive, physical, or visual impairments that would require accommodations to use Teach-ABI. Community sampling through research flyers, social media, and the HBKRH website was used.

Consent was obtained from all participants prior to commencing the study in compliance with the research ethics procedures (REB approval number 2020-0294-1588-2).

Each participant attended a virtual private meeting over Zoom (Zoom Video Communications, Inc.), an online videoconferencing platform that has been utilized and found to be effective for facilitating qualitative data collection [26]. All questionnaires were hosted on REDCap (Vanderbilt University), a web-based platform for creating and managing surveys and survey data [27]. The usability of the Teach-ABI module was evaluated through qualitative analysis of think-aloud data and semistructured interviews, direct observation, user success rate during task completion, and the System Usability Scale (SUS) [28]. The think-aloud method originated from cognitive interviewing and invites participants to verbally share what they are thinking, feeling, and doing as they complete a task [29]. Topic probes were adapted from past health information usability studies [30]. Participants were instructed to comment on the module’s content, images, features, ease of interface use, aspects they liked or disliked, and suggestions for improvement.

While participants reviewed the module, they shared their screen in the Zoom meeting with the research team member. This allowed the researcher to capture the usability of the module through direct observation and evaluating task completion, which have been used to investigate the usability of online resources [30,31].

Field notes were taken as participants navigated the module and referenced content, images, and features. Notes were also included when participant verbalizations were vague (eg, “I really like this part”) or when any areas of difficulty or confusion arose. These notes were combined with the think-aloud data to examine the research questions.

The researcher observed participants’ completion of 11 tasks related to Teach-ABI (see Table 1 for the task list) and rated their level of success based on 1 of 3 outcomes: completed with ease, completed with help, and did not complete.

The chosen tasks were characteristic of actions that must be completed to successfully engage with the module. Participants were instructed to navigate the module as they normally would, which involved minimal to no interference from the researcher. Thus, participants were not asked to complete the specific tasks, rather, the researcher observed their completion of the tasks without any direction.

After reviewing the module, participants completed the SUS and a semistructured exit interview. The SUS is a validated 10-item questionnaire that provides a quick assessment of a system or tool’s perceived usability [32-34]. While the questionnaire was modified to suit this study (see Multimedia Appendix 1), research has demonstrated that minor linguistic changes do not impact the validity of the scale [32]. A semistructured interview guide was used to learn more about participants’ experiences completing Teach-ABI. Topics were consistent with previous usability studies and guidelines [35-37] and included overall impression, liked and disliked aspects of the module, navigation experience and feature usability, and suggestions for improvement. The exit interview was audio-recorded.

The average score on each of the 11 tasks was examined, in addition to participants’ individual scores on each task. A task was flagged as a usability problem if less than 70% of participants were able to complete it independently [30,38]. For this study, the usability goal was that more than half of the 11 tasks (ie, 6/11 tasks) would be completed independently by more than 70% (ie, n≥6/8) of participants. The ability to complete tasks needed to navigate Teach-ABI is a suitable way to determine module usability [39].

The SUS was scored using the steps outlined by Brooke [28]. Raw scores were converted to a total score out of 100. Scores were interpreted in relation to norm-referenced data, with an average score of 68 representing above average usability [33], and using a curved grading scale developed by Sauro and Lewis [40,41], which pairs scores out of 100 with a letter grade ranging from F (low) to A+ (high).

Two members of the research team (LS and HA-H) analyzed the qualitative data using content analysis [42,43]. Usability sessions and exit interviews were transcribed verbatim and reviewed multiple times for accuracy. Initial codes were generated from LS’s familiarization with the data and applied to the first transcript during line-by-line open coding. The initial list of codes was flexible and changed as the first transcript was coded. HA-H then coded the first transcript using the flexible list of codes. LS and HA-H discussed the codes and collaborated to clarify the existing codes and to create additional codes. These initial codes were used to code each transcript independently, and LS and HA-H met regularly to check for agreement related to the assigned codes and to create a final codebook. An explanation of each code was provided to ensure that they were applied consistently. Each code was also linked to one of the research questions to ensure the study focus remained central [44]. The codebook was flexible, as new codes were added throughout the coding process. Before a code was added, both researchers agreed on its inclusion and subsequent definition. The transcripts and codes were then organized in NVivo (QSR International), which was used to recode the transcripts based on the updated coding list. NVivo also provided structure and accessibility to the codes and meaning units within each code and allowed the data to be easily explored and reviewed to generate meaning and establish categories and subcategories [45]. The study reached data saturation as new or valuable information was not expected with additional interviews [46,47]. This is evident by the comprehensive information gathered when developing the categories and their relationships, as no new codes were identified following transcripts’ reviews.

A total of 8 participants were enrolled in this study. The participants identified as female and were primarily early career practicing teachers employed by public school boards. Participants varied in their self-assessment of ABI knowledge. See Table 2 for participant information.

The average scores on each of the 11 tasks revealed that more than 85% (ie, 7/8) of participants independently completed 10 out of 11 tasks and 100% (8/8) of participants independently completed 7 out of 11 tasks. A usability problem occurred with downloading a tip sheet, with 5 participants able to independently download and open the tip sheet, 2 needing assistance, and 1 unable to complete this task. Overall, the study usability goal was met, as all participants completed more than 50% (>6/11) of tasks independently. See Table 3 for task performance scores.

The average overall score on the SUS was 86.25 (range 65-100), surpassing the above average score for system usability (ie, 68) [33]. Using the curved grading scale [34,40,41], 86.25 translated into a score of A+.

This study describes the development of Teach-ABI and outcomes of usability testing. The primary aim of this work was to assess ease of use of the Teach-ABI interface, determine end user satisfaction with the module design, and consider content modifications to optimize usability. The positive results expressed by the teacher participants regarding navigation of and satisfaction with the module maintain that Teach-ABI is a highly usable, professional development resource.

Use and navigation of Teach-ABI were assessed through data triangulation across multiple sources: participant observation and task performance, think-aloud data, the SUS, and exit interview data. Results demonstrated usability and ease of navigation of the Teach-ABI module. Participants completed 7 out of the 11 selected tasks at a rate of 100%. The average score on the SUS was 86.25, which is almost 20 points greater than scores awarded to systems with average usability [34]. Interview data also showed that participants described the module as easy to use and follow. Participant’s positive feedback on their experiences navigating the module contribute to an understanding of the module as user-friendly and highly learnable. Although a small number of participants encountered minor difficulties while navigating the module (eg, downloading the tip sheet, understanding pop-up arrow function), this can be remedied through minimal changes to the system.

In terms of Teach-ABI satisfaction, participants believed the module content was valuable, informative, and relevant to their profession. Module design, including the elements of appearance, organization, and delivery, was also rated positively. For example, participants appreciated the engaging features of the module (eg, case study, personal videos). Research highlights the importance of a module’s appearance and delivery, as it may influence users’ appeal and their ability to engage with the content [49]. Teacher participants also valued the self-reflection activities in Teach-ABI. Their reflections indicated that the module structure (eg, case study, knowledge checks) promoted a high level of user engagement and encouraged them to connect the topics to their practice.

The high level of satisfaction with the content and features of Teach-ABI speaks to the value of using an iKT approach combined with process models to inform product development [18-20]. Furthermore, the actions of reflecting on information, connecting it to previous experiences, and challenging existing knowledge link to higher-order thinking skills. This is significant, as eliciting higher-order thinking skills is associated with greater long-term recall and application of information [50]. In addition, educators were actively engaged with the module as they reflected on the content and made new connections to their personal teaching experience. Research has shown that reflection activities foster ongoing improvement in education practice and help to situate one’s context in their learning experience [51,52]. The inclusion of activities that end users can relate to through their teaching practice is a valuable design feature of online resources [51,53] and was the approach taken with the development of Teach-ABI.

Access to online professional development opportunities, such as Teach-ABI, does not ensure that educators have the knowledge needed to support students with ABI in the classroom. However, it is important to acknowledge that teacher participants identified the potential for Teach-ABI to improve their knowledge and understanding of ABI. This finding is promising and consistent with previous research on ABI training and its association with increased educator knowledge, fewer ABI misconceptions, and higher levels of confidence related to teaching students with ABI [54,55]. Furthermore, preliminary research has examined the effect of an online TBI training module on educators’ knowledge and confidence related to supporting students with TBI. The results indicated that the online module significantly improved educators’ knowledge and confidence related to supporting students with TBI and this improvement was maintained at the 30-day follow-up [9,56]. Guided by an iKT approach [18], next steps involve understanding facilitators and barriers to implementing Teach-ABI; supports needed to foster implementation; and the impact of the module on shifts in knowledge, confidence, and teaching practices. In addition, considering how educators with different backgrounds (eg, previous work with students with ABI, familiarity with e-learning modules, and level of ABI knowledge) experience and benefit from Teach-ABI is an important future direction.

The use of qualitative methods to examine usability was valuable, as it helped to explain the users’ response to content and features and situated their ratings of the module and suggestions for improving Teach-ABI. Besides, the sample size of the study is consistent with suggestions in the field of usability [57] and is suitable for reaching data saturation in qualitative interviewing [58]. Previous research studies examining training programs related to TBI have predominantly utilized quantitative methods, such as closed-ended surveys to examine usability [9,56], which provide a simple picture of usability. Qualitative methods helped achieve the program’s broader goal of creating a tool that is valuable and usable to Ontario educators and understanding participants’ experiences navigating Teach-ABI. In addition, the virtual data collection session resulted in an experience closely related to the real-world use of Teach-ABI by the study population. Instead of accessing Teach-ABI from a device provided by the researcher, participants accessed it using their own device and completed the module from a location of their choice, highlighting the ecological functionality of the module.

There are some limitations related to the study sample. For example, all 8 participants were practicing (n=7) or preservice (n=1) classroom teachers. Originally, the study aimed to recruit Ontario educators broadly, including practicing and preservice classroom teachers, principals, educational assistants, and early childhood educators, to extend the generalizability of the results. All 8 participants identified as female. Although there are a significantly greater number of Ontario elementary educators that identify as female, as many as 4 times more female elementary school teachers than male teachers [59], the sample was not representative of the teacher population. Furthermore, usability research suggests that males evaluate e-learning systems differently than females [60]; therefore, future research should aim to capture male educators’ perspectives on Teach-ABI. In addition, the sample consisted of mostly early career classroom teachers. It would be important to capture the perspective of teachers with more than 5 years of experience to understand any differences in their experience completing and navigating virtual modules in comparison to educators in the beginning stages of their careers. Lastly, the sample characteristics were limited due to self-selection bias. Participation in the study was voluntary and participants were recruited through sharing the research flyer and information on the social media and website of a research hospital. Future research should target a broader group of educators using a wider variety of recruitment methods.

This study demonstrated strong usability and satisfaction with Teach-ABI, an innovative and novel online professional development module. Validated measures of usability combined with qualitative methodology revealed educators’ high level of satisfaction with the design, content, and navigation of Teach-ABI. Educators engaged with the module as active participants in knowledge construction, as they reflected, questioned, and connected content to their experiences and knowledge. This study highlights the importance of usability testing in the build of online professional development modules. Furthermore, the comprehensive approach to testing the usability of Teach-ABI may be applied in future studies evaluating online modules.