JMIR Publications

JMIR Human Factors

Making health care interventions and technologies usable, safe, and effective


Journal Description

JMIR Human Factors (JHF) is a new spin-off journal of JMIR, a leading open access eHealth journal (Impact Factor 2015: 4.532). 
JMIR Human Factors is a multidisciplinary journal with contributions from medical researchers, engineers, and social scientists.
JMIR Human Factors focuses on understanding how the behaviour and thinking of humans can influence and shape the design of health care interventions and technologies, and how the design can be evaluated and improved to make health care interventions and technologies usable, safe, and effective.
JHF aspires to lead health care towards a culture of testing and safety. All articles are professionally copyedited and typeset, ready for indexing in PubMed/PubMed Central. Possible contributions include usability studies and heuristic evaluations, studies concerning ergonomics and error prevention, design studies for medical devices and healthcare systems/workflows, enhancing teamwork through Human Factors based teamwork training, measuring non-technical skills in staff like leadership, communication, situational awareness and teamwork, and healthcare policies and procedures to reduce errors and increase safety. Reviews, viewpoint papers and tutorials are as welcome as original research.

Editorial Board members are currently being recruited, please contact us if you are interested ( at


Recent Articles:

  • Understandability of Patient Journey Models. Copyright held by UOIT & article authors.

    An Evaluation of Understandability of Patient Journey Models in Mental Health


    Background: There is a significant trend toward implementing health information technology to reduce administrative costs and improve patient care. Unfortunately, little awareness exists of the challenges of integrating information systems with existing clinical practice. The systematic integration of clinical processes with information system and health information technology can benefit the patients, staff, and the delivery of care. Objectives: This paper presents a comparison of the degree of understandability of patient journey models. In particular, the authors demonstrate the value of a relatively new patient journey modeling technique called the Patient Journey Modeling Architecture (PaJMa) when compared with traditional manufacturing based process modeling tools. The paper also presents results from a small pilot case study that compared the usability of 5 modeling approaches in a mental health care environment. Method: Five business process modeling techniques were used to represent a selected patient journey. A mix of both qualitative and quantitative methods was used to evaluate these models. Techniques included a focus group and survey to measure usability of the various models. Results: The preliminary evaluation of the usability of the 5 modeling techniques has shown increased staff understanding of the representation of their processes and activities when presented with the models. Improved individual role identification throughout the models was also observed. The extended version of the PaJMa methodology provided the most clarity of information flows for clinicians. Conclusions: The extended version of PaJMa provided a significant improvement in the ease of interpretation for clinicians and increased the engagement with the modeling process. The use of color and its effectiveness in distinguishing the representation of roles was a key feature of the framework not present in other modeling approaches. Future research should focus on extending the pilot case study to a more diversified group of clinicians and health care support workers.

  • Image Source: The authors, licensed under Creative Commons Attribution 2.0.

    Do Gender-Specific and High-Resolution Three Dimensional Body Charts Facilitate the Communication of Pain for Women? A Quantitative and Qualitative Study


    Background: Chronic pain is more prevalent among women; however, the majority of standardized pain drawings are often collected using male-like androgynous body representations. Objective: The purpose of this study was to assess whether gender-specific and high-resolution three-dimensional (3D) body charts facilitate the communication of pain for women. Methods: Using mixed-methods and a cross-over design, female patients with chronic pain were asked to provide detailed drawings of their current pain on masculine and feminine two-dimensional (2D) body schemas (N=41, Part I) or on female 2D and 3D high-resolution body schemas (N=41, Part II) on a computer tablet. The consistency of the drawings between body charts were assessed by intraclass correlation coefficient (ICC) and Bland-Altman plots. Semistructured interviews and a preference questionnaire were then used to obtain qualitative and quantitative responses of the drawing experience. Results: The consistency between body charts were high (Part I: ICC=0.980, Part II: ICC=0.994). The preference ratio for the masculine to feminine body schemas were 6:35 and 18:23 for the 2D to 3D female body charts. Patients reported that the 3D body chart enabled a more accurate expression of their pain due to the detailed contours of the musculature and bone structure, however, patients also reported the 3D body chart was too human and believed that skin-like appearance limited ‘deep pain’ expressions. Conclusions: Providing gender-specific body charts may facilitate the communication of pain and the level of detail (2D vs 3D body charts) should be used according to patients’ needs.

  • Electronic Health Records. Image sourced and copyright owned by the authors Anne Press et al.

    Usability Testing of a National Substance Use Screening Tool Embedded in Electronic Health Records


    Background: Screening, brief intervention, and referral to treatment (SBIRT) is currently being implemented into health systems nationally via paper and electronic methods. Objective: The purpose of this study was to evaluate the integration of an electronic SBIRT tool into an existing paper-based SBIRT clinical workflow in a patient-centered medical home. Methods: Usability testing was conducted in an academic ambulatory clinic. Two rounds of usability testing were done with medical office assistants (MOAs) using a paper and electronic version of the SBIRT tool, with two and four participants, respectively. Qualitative and quantitative data was analyzed to determine the impact of both tools on clinical workflow. A second round of usability testing was done with the revised electronic version and compared with the first version. Results: Personal workflow barriers cited in the first round of testing were that the electronic health record (EHR) tool was disruptive to patient’s visits. In Round 2 of testing, MOAs reported favoring the electronic version due to improved layout and the inclusion of an alert system embedded in the EHR. For example, using the system usability scale (SUS), MOAs reported a grade “1” for the statement, “I would like to use this system frequently” during the first round of testing but a “5” during the second round of analysis. Conclusions: The importance of testing usability of various mediums of tools used in health care screening is highlighted by the findings of this study. In the first round of testing, the electronic tool was reported as less user friendly, being difficult to navigate, and time consuming. Many issues faced in the first generation of the tool were improved in the second generation after usability was evaluated. This study demonstrates how usability testing of an electronic SBRIT tool can help to identify challenges that can impact clinical workflow. However, a limitation of this study was the small sample size of MOAs that participated. The results may have been biased to Northwell Health workers’ perceptions of the SBIRT tool and their specific clinical workflow.

  • NIH. Image created, sourced and copyright owned by authors Pamela A Williams at al.

    Usability Testing and Adaptation of the Pediatric Cardiovascular Risk Reduction Clinical Decision Support Tool


    Background: Cardiovascular disease (CVD) is 1 of the leading causes of death, years of life lost, and disability-adjusted years of life lost worldwide. CVD prevention for children and teens is needed, as CVD risk factors and behaviors beginning in youth contribute to CVD development. In 2012, the National Heart, Lung, and Blood Institute released their "Integrated Guidelines for Cardiovascular Health and Risk Reduction in Children and Adolescents" for clinicians, describing CVD risk factors they should address with patients at primary care preventative visits. However, uptake of new guidelines is slow. Clinical decision support (CDS) tools can improve guideline uptake. In this paper, we describe our process of testing and adapting a CDS tool to help clinicians evaluate patient risk, recommend behaviors to prevent development of risk, and complete complex calculations to determine appropriate interventions as recommended by the guidelines, using a user-centered design approach. Objective: The objective of the study was to assess the usability of a pediatric CVD risk factor tool by clinicians. Methods: The tool was tested using one-on-one in-person testing and a "think aloud" approach with 5 clinicians and by using the tool in clinical practice along with formal usability metrics with 14 pediatricians. Thematic analysis of the data from the in-person testing and clinical practice testing identified suggestions for change in 3 major areas: user experience, content refinement, and technical deployment. Descriptive statistical techniques were employed to summarize users' overall experience with the tool. Results: Data from testers showed that general reactions toward the CDS tool were positive. Clinical practice testers suggested revisions to make the application more user-friendly, especially for clinicians using the application on the iPhone, and called for refining recommendations to be more succinct and better tailored to the patient. Tester feedback was incorporated into the design when feasible, including streamlining data entry during clinical visits, reducing the volume of results displayed, and highlighting critical results. Conclusions: This study found support for the usability of our pediatric CVD risk factor tool. Insights shared about this tool may be applicable for designing other mHealth applications and CDS tools. The usability of decision support tools in clinical practice depends critically on receiving (ie, through an accessible device) and adapting the tool to meet the needs of clinicians in the practice setting.

  • Source:; CC BY-2.5.

    How Regrouping Alerts in Computerized Physician Order Entry Layout Influences Physicians’ Prescription Behavior: Results of a Crossover Randomized Trial


    Background: As demonstrated in several publications, low positive predictive value alerts in computerized physician order entry (CPOE) induce fatigue and may interrupt physicians unnecessarily during prescription of medication. Although it is difficult to increase the consideration of medical alerts by physician through an improvement of their predictive value, another approach consists to act on the way they are presented. The interruption management model inspired us to propose an alternative alert display strategy of regrouping the alerts in the screen layout, as a possible solution for reducing the interruption in physicians’ workflow. Objective: In this study, we compared 2 CPOE designs based on a particular alert presentation strategy: one design involved regrouping the alerts in a single place on the screen, and in the other, the alerts were located next to the triggering information. Our objective was to evaluate experimentally whether the new design led to fewer interruptions in workflow and if it affected alert handling. Methods: The 2 CPOE designs were compared in a controlled crossover randomized trial. All interactions with the system and eye movements were stored for quantitative analysis. Results: The study involved a group of 22 users consisting of physicians and medical students who solved medical scenarios containing prescription tasks. Scenario completion time was shorter when the alerts were regrouped (mean 117.29 seconds, SD 36.68) than when disseminated on the screen (mean 145.58 seconds, SD 75.07; P=.045). Eye tracking revealed that physicians fixated longer on alerts in the classic design (mean 119.71 seconds, SD 76.77) than in the centralized alert design (mean 70.58 seconds, SD 33.53; P=.001). Visual switches between prescription and alert areas, indicating interruption, were reduced with centralized alerts (mean 41.29, SD 21.26) compared with the classic design (mean 57.81, SD 35.97; P=.04). Prescription behavior (ie, prescription changes after alerting), however, did not change significantly between the 2 strategies of display. The After-Scenario Questionnaire (ASQ) that was filled out after each scenario showed that overall satisfaction was significantly rated lower when alerts were regrouped (mean 4.37, SD 1.23) than when displayed next to the triggering information (mean 5.32, SD 0.94; P=.02). Conclusions: Centralization of alerts in a table might be a way to motivate physicians to manage alerts more actively, in a meaningful way, rather than just being interrupted by them. Our study could not provide clear recommendations yet, but provides objective data through a cognitive psychological approach. Future tests should work on standardized scenarios that would enable to not only measure physicians’ behavior (visual fixations and handling of alerts) but also validate those actions using clinical criteria.

  • Older adult using computer. Source: CC0 Public Domain.

    Effects of Information Visualization on Older Adults’ Decision-Making Performance in a Medicare Plan Selection Task: A Comparative Usability Study


    Background: Technology gains have improved tools for evaluating complex tasks by providing environmental supports (ES) that increase ease of use and improve performance outcomes through the use of information visualizations (info-vis). Complex info-vis emphasize the need to understand individual differences in abilities of target users, the key cognitive abilities needed to execute a decision task, and the graphical elements that can serve as the most effective ES. Older adults may be one such target user group that would benefit from increased ES to mitigate specific declines in cognitive abilities. For example, choosing a prescription drug plan is a necessary and complex task that can impact quality of life if the wrong choice is made. The decision to enroll in one plan over another can involve comparing over 15 plans across many categories. Within this context, the large amount of complex information and reduced working memory capacity puts older adults’ decision making at a disadvantage. An intentionally designed ES, such as an info-vis that reduces working memory demand, may assist older adults in making the most effective decision among many options. Objective: The objective of this study is to examine whether the use of an info-vis can lower working memory demands and positively affect complex decision-making performance of older adults in the context of choosing a Medicare prescription drug plan. Methods: Participants performed a computerized decision-making task in the context of finding the best health care plan. Data included quantitative decision-making performance indicators and surveys examining previous history with purchasing insurance. Participants used a colored info-vis ES or a table (no ES) to perform the decision task. Task difficulty was manipulated by increasing the number of selection criteria used to make an accurate decision. A repeated measures analysis was performed to examine differences between the two table designs. Results: Twenty-three older adults between the ages of 66 and 80 completed the study. There was a main effect for accuracy such that older adults made more accurate decisions in the color info-vis condition than the table condition. In the low difficulty condition, participants were more successful at choosing the correct answer when the question was about the gap coverage attribute in the info-vis condition. Participants also made significantly faster decisions in the info-vis condition than in the table condition. Conclusions: Reducing the working memory demand of the task through the use of an ES can improve decision accuracy, especially when selection criteria is only focused on a single attribute of the insurance plan.

  • Diabetes MAP homepage. Source:, Copyright: Author (CYO).

    The Usability of Diabetes MAP: A Web-delivered Intervention for Improving Medication Adherence


    Background: Web-delivered interventions are a feasible approach to health promotion. However, if a website is poorly designed, difficult to navigate, and has technical bugs, it will not be used as intended. Usability testing prior to evaluating a website’s benefits can identify barriers to user engagement and maximize future use. Objective: We developed a Web-delivered intervention called Diabetes Medication Adherence Promotion (Diabetes MAP) and used a mixed-methods approach to test its usability prior to evaluating its efficacy on medication adherence and glycemic control in a randomized controlled trial. Methods: We recruited English-speaking adults with type 2 diabetes mellitus (T2DM) from an academic medical center who were prescribed diabetes medications. A trained research assistant administered a baseline survey, collected medical record information, and instructed participants on how to access Diabetes MAP. Participants were asked to use the site independently for 2 weeks and to provide survey and/or focus group feedback on their experience. We analyzed survey data descriptively and qualitative data thematically to identify participants’ favorable and unfavorable experiences, characterize usability concerns, and solicit recommendations for improving Diabetes MAP. Results: Enrolled participants (N=32) were an average of 51.7 ± 11.8 years old, 66% (21/32) female, 60% (19/32) non-Hispanic White, 88% (28/32) had more than 12 years of education, half had household incomes over $50,000, and 78% (25/32) were privately insured. Average duration of diagnosed diabetes was 7.8 ± 6.3 years, average A1c was 7.4 ± 2.0, and 38% (12/32) were prescribed insulin. Of enrolled participants, 91% (29/32) provided survey and/or focus group feedback about Diabetes MAP. On the survey, participants agreed website information was clear and easy to understand, but in focus groups they reported navigational challenges and difficulty overcoming user errors (eg, entering data in an unspecified format). Participants also reported difficulty accessing the site and, once accessed, using all of its features. Participants recommended improving the site’s user interface to facilitate quick, efficient access to all features and content. Conclusions: Adults with T2DM rated the Diabetes MAP website favorably on surveys, but focus groups gave more in-depth feedback on the user experience (eg, difficulty accessing the site, maximizing all of the site’s features and content, and recovering from errors). Appropriate usability testing methods ensure Web-delivered interventions work as intended and any benefits are not diminished by usability challenges.

  • Senior Asian man using a tablet. Image Source:; copyright: Angela et al, Licensed under ISTOCK CONTENT LICENSE AGREEMENT.

    The McMaster Optimal Aging Portal: Usability Evaluation of a Unique Evidence-Based Health Information Website


    Background: Increasingly, older adults and their informal caregivers are using the Internet to search for health-related information. There is a proliferation of health information online, but the quality of this information varies, often based on exaggerated or dramatic findings, and not easily comprehended by consumers. The McMaster Optimal Aging Portal (Portal) was developed to provide Internet users with high-quality evidence about aging and address some of these current limitations of health information posted online. The Portal includes content for health professionals coming from three best-in-class resources (MacPLUS, Health Evidence, and Health Systems Evidence) and four types of content specifically prepared for the general public (Evidence Summaries, Web Resource Ratings, Blog Posts, and Twitter messages). Objective: Our objectives were to share the findings of the usability evaluation of the Portal with particular focus on the content features for the general public and to inform designers of health information websites and online resources for older adults about key usability themes. Methods: Data analysis included task performance during usability testing and qualitative content analyses of both the usability sessions and interviews to identify core themes. Results: A total of 37 participants took part in 33 usability testing sessions and 21 focused interviews. Qualitative analysis revealed common themes regarding the Portal’s strengths and challenges to usability. The strengths of the website were related to credibility, applicability, browsing function, design, and accessibility. The usability challenges included reluctance to register, process of registering, searching, terminology, and technical features. Conclusions: The study reinforced the importance of including end users during the development of this unique, dynamic, evidence-based health information website. The feedback was applied to iteratively improve website usability. Our findings can be applied by designers of health-related websites.

  • High-fidelity prototype version of Loop with Patient and Team and Team Only toggle from message compose box.

    In the Loop: The Organization of Team-Based Communication in a Patient-Centered Clinical Collaboration System


    Background: We describe the development and evaluation of a secure Web-based system for the purpose of collaborative care called Loop. Loop assembles the team of care with the patient as an integral member of the team in a secure space. Objective: The objectives of this paper are to present the iterative design of the separate views for health care providers (HCPs) within each patient’s secure space and examine patients’, caregivers’, and HCPs’ perspectives on this separate view for HCP-only communication. Methods: The overall research program includes cycles of ethnography, prototyping, usability testing, and pilot testing. This paper describes the usability testing phase that directly informed development. A descriptive qualitative approach was used to analyze participant perspectives that emerged during usability testing. Results: During usability testing, we sampled 89 participants from three user groups: 23 patients, 19 caregivers, and 47 HCPs. Almost all perspectives from the three user groups supported the need for an HCP-only communication view. In an earlier prototype, the visual presentation caused confusion among HCPs when reading and composing messages about whether a message was visible to the patient. Usability testing guided us to design a more deliberate distinction between posting in the Patient and Team view and the Health Care Provider Only view at the time of composing a message, which once posted is distinguished by an icon. Conclusions: The team made a decision to incorporate an HCP-only communication view based on findings during earlier phases of work. During usability testing we tested the separate communication views, and all groups supported this partition. We spent considerable effort designing the partition; however, preliminary findings from the next phase of evaluation, pilot testing, show that the Patient and Team communication is predominantly being used. This demonstrates the importance of a subsequent phase of the clinical trial of Loop to validate the concept and design.

  • Photo by NEC Corporation of America with Creative Commons license.

Source URL:

    Implementation of a Computerized Screening Inventory: Improved Usability Through Iterative Testing and Modification


    Background: The administration of health screeners in a hospital setting has traditionally required (1) clinicians to ask questions and log answers, which can be time consuming and susceptible to error, or (2) patients to complete paper-and-pencil surveys, which require third-party entry of information into the electronic health record and can be vulnerable to error and misinterpretation. A highly promising method that avoids these limitations and bypasses third-party interpretation is direct entry via a computerized inventory. Objective: To (1) computerize medical and behavioral health screening for use in general medical settings, (2) optimize patient acceptability and feasibility through iterative usability testing and modification cycles, and (3) examine how age relates to usability. Methods: A computerized version of 15 screeners, including behavioral health screeners recommended by a National Institutes of Health Office of Behavioral and Social Sciences Research collaborative workgroup, was subjected to systematic usability testing and iterative modification. Consecutive adult, English-speaking patients seeking treatment in an urban emergency department were enrolled. Acceptability was defined as (1) the percentage of eligible patients who agreed to take the assessment (initiation rate) and (2) average satisfaction with the assessment (satisfaction rate). Feasibility was defined as the percentage of the screening items completed by those who initiated the assessment (completion rate). Chi-square tests, analyses of variance, and Pearson correlations were used to detect whether improvements in initiation, satisfaction, and completion rates were seen over time and to examine the relation between age and outcomes. Results: Of 2157 eligible patients approached, 1280 agreed to complete the screening (initiation rate=59.34%). Statistically significant increases were observed over time in satisfaction (F3,1061=3.35, P=.019) and completion rates (F3,1276=25.44, P<.001). Younger age was associated with greater initiation (initiated, mean [SD], 46.6 [18.7] years; declined: 53.0 [19.5] years, t2,155=−7.6, P<.001), higher completion (r=−.20, P<.001), and stronger satisfaction (r=−.23, P<.001). Conclusions: In a rapid-paced emergency department with a heterogeneous patient population, 59.34% (1280/2157) of all eligible patients initiated the computerized screener with a completion rate reaching over 90%. Usability testing revealed several critical principles for maximizing usability of the computerized medical and behavioral health screeners used in this study. Further work is needed to identify usability issues pertaining to other screeners, racially and ethnically diverse patient groups, and different health care settings.

  • Usability labs at Toronto General Hospital showing a complete set up of a simulated operating room (including a patient simulator).

    Challenges and Paradoxes of Human Factors in Health Technology Design


    Usability testing allows human factors professionals to identify and mitigate issues with the design and use of medical technology. The test results, however, can be paradoxical and therefore be misinterpreted, limiting their usefulness. The paradoxical findings can lead to products that are not aligned with the needs and constraints of their users. We herein report on our observations of the paradox of expertise, the paradox of preference versus performance, and the paradox of choice. Each paradox explored is in the perspective of the design of medical technology, the issues that need to be considered in the interpretation of the test results, as well as suggestions on how to avoid the pitfalls in the design of medical technology. Because these paradoxes can influence product design at various stages of product development, it is important to be aware of the effects to interpret the findings properly.

  • Photo courtesy of / stockimages. Image ID: 100102403.

    Evaluating the Usability and Perceived Impact of an Electronic Medical Record Toolkit for Atrial Fibrillation Management in Primary Care: A Mixed-Methods...


    Background: Atrial fibrillation (AF) is a common and preventable cause of stroke. Barriers to reducing stroke risk through appropriate prescribing have been identified at the system, provider, and patient levels. To ensure a multifaceted initiative to address these barriers is effective, it is essential to incorporate user-centered design to ensure all intervention components are optimized for users. Objective: To test the usability of an electronic medical record (EMR) toolkit for AF in primary care with the goal of further refining the intervention to meet the needs of primary care clinicians. Methods: An EMR-based toolkit for AF was created and optimized through usability testing and iterative redesign incorporating a human factors approach. A mixed-methods pilot study consisting of observations, semi-structured interviews, and surveys was conducted to examine usability and perceived impact on patient care and workflow. Results: A total of 14 clinicians (13 family physicians and 1 nurse practitioner) participated in the study. Nine iterations of the toolkit were created in response to feedback from clinicians and the research team; interface-related changes were made, additional AF-related resources were added, and functionality issues were fixed to make the toolkit more effective. After improvements were made, clinicians expressed that the toolkit improved accessibility to AF-related information and resources, served as a reminder for guideline-concordant AF management, and was easy to use. Most clinicians intended to continue using the toolkit for patient care. With respect to impact on care, clinicians believed the toolkit increased the thoroughness of their assessments for patients with AF and improved the quality of AF-related care received by their patients. Conclusions: The positive feedback surrounding the EMR toolkit for AF and its perceived impact on patient care can be attributed to the adoption of a user-centered design that merged clinically important information about AF management with user needs. This study demonstrates the utility of a human factors approach to piloting and refining an intervention prior to wide-scale implementation.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)