JMIR Publications

JMIR Human Factors

Making health care interventions and technologies usable, safe, and effective


Journal Description

JMIR Human Factors (JHF) is a new spin-off journal of JMIR, a leading open access eHealth journal (Impact Factor 2014: 3.4). 
JMIR Human Factors is a multidisciplinary journal with contributions from medical researchers, engineers, and social scientists.
JMIR Human Factors focuses on understanding how the behaviour and thinking of humans can influence and shape the design of health care interventions and technologies, and how the design can be evaluated and improved to make health care interventions and technologies usable, safe, and effective.
JHF aspires to lead health care towards a culture of testing and safety. All articles are professionally copyedited and typeset, ready for indexing in PubMed/PubMed Central. Possible contributions include usability studies and heuristic evaluations, studies concerning ergonomics and error prevention, design studies for medical devices and healthcare systems/workflows, enhancing teamwork through Human Factors based teamwork training, measuring non-technical skills in staff like leadership, communication, situational awareness and teamwork, and healthcare policies and procedures to reduce errors and increase safety. Reviews, viewpoint papers and tutorials are as welcome as original research.

Editorial Board members are currently being recruited, please contact us if you are interested ( at


Recent Articles:

  • Sociotechnical human factors. Created by Lori Wozney (author) who holds the copyright.

    Sociotechnical Human Factors Involved in Remote Online Usability Testing of Two eHealth Interventions


    Background: Research in the fields of human performance technology and human computer interaction are challenging the traditional macro focus of usability testing arguing for methods that help test moderators assess “use in context” (ie, cognitive skills, usability understood over time) and in authentic “real world” settings. Human factors in these complex test scenarios may impact on the quality of usability results being derived yet there is a lack of research detailing moderator experiences in these test environments. Most comparative research has focused on the impact of the physical environment on results, and rarely on how the sociotechnical elements of the test environment affect moderator and test user performance. Improving our understanding of moderator roles and experiences with conducting “real world” usability testing can lead to improved techniques and strategies Objective: To understand moderator experiences of using Web-conferencing software to conduct remote usability testing of 2 eHealth interventions. Methods: An exploratory case study approach was used to study 4 moderators’ experiences using Blackboard Collaborate for remote testing sessions of 2 different eHealth interventions. Data collection involved audio-recording iterative cycles of test sessions, collecting summary notes taken by moderators, and conducting 2 90-minute focus groups via teleconference. A direct content analysis with an inductive coding approach was used to explore personal accounts, assess the credibility of data interpretation, and generate consensus on the thematic structure of the results. Results: Following the convergence of data from the various sources, 3 major themes were identified: (1) moderators experienced and adapted to unpredictable changes in cognitive load during testing; (2) moderators experienced challenges in creating and sustaining social presence and untangling dialogue; and (3) moderators experienced diverse technical demands, but were able to collaboratively troubleshoot with test users. Conclusions: Results highlight important human-computer interactions and human factor qualities that impact usability testing processes. Moderators need an advanced skill and knowledge set to address the social interaction aspects of Web-based usability testing and technical aspects of conferencing software during test sessions. Findings from moderator-focused studies can inform the design of remote testing platforms and real-time usability evaluation processes that place less cognitive burden on moderators and test users.

  • Design Thinking.

Page URL:
File URL:
Attribution: By Saad Alzarooni (Own work) [CC BY-SA 3.0 (], via Wikimedia Commons.

    Mental Health Technologies: Designing With Consumers


    Despite growing interest in the promise of e-mental and well-being interventions, little supporting literature exists to guide their design and the evaluation of their effectiveness. Both participatory design (PD) and design thinking (DT) have emerged as approaches that hold significant potential for supporting design in this space. Each approach is difficult to definitively circumscribe, and as such has been enacted as a process, a mind-set, specific practices/techniques, or a combination thereof. At its core, however, PD is a design research tradition that emphasizes egalitarian partnerships with end users. In contrast, DT is in the process of becoming a management concept tied to innovation with strong roots in business and education. From a health researcher viewpoint, while PD can be reduced to a number of replicable stages that involve particular methods, techniques, and outputs, projects often take vastly different forms and effective PD projects and practice have traditionally required technology-specific (eg, computer science) and domain-specific (eg, an application domain, such as patient support services) knowledge. In contrast, DT offers a practical off-the-shelf toolkit of approaches that at face value have more potential to have a quick impact and be successfully applied by novice practitioners (and those looking to include a more human-centered focus in their work). Via 2 case studies we explore the continuum of similarities and differences between PD and DT in order to provide an initial recommendation for what health researchers might reasonably expect from each in terms of process and outcome in the design of e-mental health interventions. We suggest that the sensibilities that DT shares with PD (ie, deep engagement and collaboration with end users and an inclusive and multidisciplinary practice) are precisely the aspects of DT that must be emphasized in any application to mental health provision and that any technology development process must prioritize empathy and understanding over innovation for the successful uptake of technology in this space.


Copyright: Australian National University.

    University Students’ Views on the Perceived Benefits and Drawbacks of Seeking Help for Mental Health Problems on the Internet: A Qualitative Study


    Background: University students experience high levels of mental health problems yet very few seek professional help. Web-based mental health interventions may be useful for the university student population. However, there are few published qualitative studies that have examined the perceived benefits and drawbacks of seeking help for mental health problems on the Internet from the perspective of university students. Objective: To investigate the attitudes of university students on mental health help-seeking on the Internet. Methods: A total of 19 university students aged 19-24 years participated in 1 of 4 focus groups to examine their views toward help-seeking for mental health problems on the Internet. Results: Perceived concerns about Web-based help-seeking included privacy and confidentiality, difficulty communicating on the Internet, and the quality of Web-based resources. Potential benefits included anonymity/avoidance of stigma, and accessibility. Participants reported mixed views regarding the ability of people with similar mental health issues to interact on the Internet. Conclusions: These factors should be considered in the development of Web-based mental health resources to increase acceptability and engagement from university students.

  • Elder Tree participant using the website at her kitchen table.  Copyright Center for Health Enhancement Systems Studies, UW-Madison.

    Using the NIATx Model to Implement User-Centered Design of Technology for Older Adults


    What models can effectively guide the creation of eHealth and mHealth technologies? This paper describes the use of the NIATx model as a framework for the user-centered design of a new technology for older adults. The NIATx model is a simple framework of process improvement based on the following principles derived from an analysis of decades of research from various industries about why some projects fail and others succeed: (1) Understand and involve the customer; (2) fix key problems; (3) pick an influential change leader; (4) get ideas from outside the field; (5) use rapid-cycle testing. This paper describes the use of these principles in technology development, the strengths and challenges of using this approach in this context, and lessons learned from the process. Overall, the NIATx model enabled us to produce a user-focused technology that the anecdotal evidence available so far suggests is engaging and useful to older adults. The first and fourth principles were especially important in developing the technology; the fourth proved the most challenging to use.

  • This is the Image of the Bedside Cardiac Monitor Used at the Study Site. (Image taken by authors who hold the copyright).

    Changes in Default Alarm Settings and Standard In-Service are Insufficient to Improve Alarm Fatigue in an Intensive Care Unit: A Pilot Project


    Background: Clinical alarm systems safety is a national concern, specifically in intensive care units (ICUs) where alarm rates are known to be the highest. Interventional projects that examined the effect of changing default alarm settings on overall alarm rate and on clinicians’ attitudes and practices toward clinical alarms and alarm fatigue are scarce. Objective: To examine if (1) a change in default alarm settings of the cardiac monitors and (2) in-service nursing education on cardiac monitor use in an ICU would result in reducing alarm rate and in improving nurses’ attitudes and practices toward clinical alarms. Methods: This quality improvement project took place in a 20-bed transplant/cardiac ICU with a total of 39 nurses. We implemented a unit-wide change of default alarm settings involving 17 parameters of the cardiac monitors. All nurses received an in-service education on monitor use. Alarm data were collected from the audit log of the cardiac monitors 10 weeks before and 10 weeks after the change in monitors’ parameters. Nurses’ attitudes and practices toward clinical alarms were measured using the Healthcare Technology Foundation National Clinical Alarms Survey, pre- and postintervention. Results: Alarm rate was 87.86 alarms/patient day (a total of 64,500 alarms) at the preintervention period compared to 59.18 alarms/patient day (49,319 alarms) postintervention (P=.01). At baseline, Arterial Blood Pressure (ABP), Pair Premature Ventricular Contractions (PVCs), and Peripheral Capillary Oxygen Saturation (SpO2) alarms were the highest. ABP and SpO2 alarms remained among the top three at the postproject period. Out of the 39 ICU nurses, 24 (62%) provided complete pre- and postproject survey questionnaires. Compared to the preintervention survey, no remarkable changes in the postproject period were reported in nurses’ attitudes. Themes in the narrative data were related to poor usability of cardiac monitors and the frequent alarms. The data showed great variation among nurses in terms of changing alarm parameters and frequency of replacing patients' electrodes. Despite the in-service, 50% (12/24) of the nurses specified their need for more training on cardiac monitors in the postproject period. Conclusions: Changing default alarm settings and standard in-service education on cardiac monitor use are insufficient to improve alarm systems safety. Alarm management in ICUs is very complex, involving alarm management practices by clinicians, availability of unit policies and procedures, unit layout, complexity and usability of monitoring devices, and adequacy of training on system use. The complexity of the newer monitoring systems requires urgent usability testing and multidimensional interventions to improve alarm systems safety and to attain the Joint Commission National Patient Safety Goal on alarm systems safety in critical care units.

  • Thumbnail.

    Assessing the Usability of Six Data Entry Mobile Interfaces for Caregivers: A Randomized Trial


    Background: There is an increased demand in hospitals for tools, such as dedicated mobile device apps, that enable the recording of clinical information in an electronic format at the patient’s bedside. Although the human-machine interface design on mobile devices strongly influences the accuracy and effectiveness of data recording, there is still a lack of evidence as to which interface design offers the best guarantee for ease of use and quality of recording. Therefore, interfaces need to be assessed both for usability and reliability because recording errors can seriously impact the overall level of quality of the data and affect the care provided. Objective: In this randomized crossover trial, we formally compared 6 handheld device interfaces for both speed of data entry and accuracy of recorded information. Three types of numerical data commonly recorded at the patient’s bedside were used to evaluate the interfaces. Methods: In total, 150 health care professionals from the University Hospitals of Geneva volunteered to record a series of randomly generated data on each of the 6 interfaces provided on a smartphone. The interfaces were presented in a randomized order as part of fully automated data entry scenarios. During the data entry process, accuracy and effectiveness were automatically recorded by the software. Results: Various types of errors occurred, which ranged from 0.7% for the most reliable design to 18.5% for the least reliable one. The length of time needed for data recording ranged from 2.81 sec to 14.68 sec, depending on the interface. The numeric keyboard interface delivered the best performance for pulse data entry with a mean time of 3.08 sec (SD 0.06) and an accuracy of 99.3%. Conclusions: Our study highlights the critical impact the choice of an interface can have on the quality of recorded data. Selecting an interface should be driven less by the needs of specific end-user groups or the necessity to facilitate the developer’s task (eg, by opting for default solutions provided by commercial platforms) than by the level of speed and accuracy an interface can provide for recording information. An important effort must be made to properly validate mobile device interfaces intended for use in the clinical setting. In this regard, our study identified the numeric keyboard, among the proposed designs, as the most accurate interface for entering specific numerical values. This is an important step toward providing clearer guidelines on which interface to choose for the appropriate use of handheld device interfaces in the health care setting.

  • Screenshot of the Wells tool in the ED EHR. Photo by Press et al. through Allscripts Sunrise Clinical Manager.

    Usability Testing of a Complex Clinical Decision Support Tool in the Emergency Department: Lessons Learned


    Background: As the electronic health record (EHR) becomes the preferred documentation tool across medical practices, health care organizations are pushing for clinical decision support systems (CDSS) to help bring clinical decision support (CDS) tools to the forefront of patient-physician interactions. A CDSS is integrated into the EHR and allows physicians to easily utilize CDS tools. However, often CDSS are integrated into the EHR without an initial phase of usability testing, resulting in poor adoption rates. Usability testing is important because it evaluates a CDSS by testing it on actual users. This paper outlines the usability phase of a study, which will test the impact of integration of the Wells CDSS for pulmonary embolism (PE) diagnosis into a large urban emergency department, where workflow is often chaotic and high stakes decisions are frequently made. We hypothesize that conducting usability testing prior to integration of the Wells score into an emergency room EHR will result in increased adoption rates by physicians. Objective: The objective of the study was to conduct usability testing for the integration of the Wells clinical prediction rule into a tertiary care center’s emergency department EHR. Methods: We conducted usability testing of a CDS tool in the emergency department EHR. The CDS tool consisted of the Wells rule for PE in the form of a calculator and was triggered off computed tomography (CT) orders or patients’ chief complaint. The study was conducted at a tertiary hospital in Queens, New York. There were seven residents that were recruited and participated in two phases of usability testing. The usability testing employed a “think aloud” method and “near-live” clinical simulation, where care providers interacted with standardized patients enacting a clinical scenario. Both phases were audiotaped, video-taped, and had screen-capture software activated for onscreen recordings. Results: Phase I: Data from the “think-aloud” phase of the study showed an overall positive outlook on the Wells tool in assessing a patient for a PE diagnosis. Subjects described the tool as “well-organized” and “better than clinical judgment”. Changes were made to improve tool placement into the EHR to make it optimal for decision-making, auto-populating boxes, and minimizing click fatigue. Phase II: After incorporating the changes noted in Phase 1, the participants noted tool improvements. There was less toggling between screens, they had all the clinical information required to complete the tool, and were able to complete the patient visit efficiently. However, an optimal location for triggering the tool remained controversial. Conclusions: This study successfully combined “think-aloud” protocol analysis with “near-live” clinical simulations in a usability evaluation of a CDS tool that will be implemented into the emergency room environment. Both methods proved useful in the assessment of the CDS tool and allowed us to refine tool usability and workflow.

  • (cc) Davies et al. CC-BY-SA 2.0, please cite as (

    Development and Acceptability of a Co-Produced Online Intervention to Prevent Alcohol Misuse in Adolescents: A Think Aloud Study


    Background: The prototype willingness model (PWM) may offer an appropriate basis for explaining and preventing adolescent alcohol misuse. An intervention was developed using a co-production approach, and consisted of an online quiz featuring 10 questions linked to the PWM. Objective: This study sought to determine the acceptability and relevance of the intervention content to young people, to incorporate their feedback into a final version. Methods: A qualitative think aloud study with follow-up semistructured interviews was undertaken with 16 young people aged 11-15 (50%). Transcripts were analyzed using thematic analysis. Results: The following 3 main themes relating the acceptability of the intervention were identified: “challenging expectations of alcohol education”; “motivations for drinking or not drinking,” and “the inevitability of drinking.” Participants found the intervention appealing because it was counter to their expectations. The content appeared to reflect their experiences of social pressure and drinking encounters. There was evidence that a focus on drinker/nondrinker prototypes was too narrow and that because adolescents perceived drinking as inevitable, it would be challenging to enact any plans to resist pressure to drink. Conclusions: An online intervention based on the PWM has the potential to engage and interest adolescents. A wide range of alcohol prototypes should be targeted and a focus on short-term harms should ensure that the intervention is credible to young people.

  • CC-BY 3.0 (

    Participatory Research as One Piece of the Puzzle: A Systematic Review of Consumer Involvement in Design of Technology-Based Youth Mental Health and...


    Background: Despite the potential of technology-based mental health interventions for young people, limited uptake and/or adherence is a significant challenge. It is thought that involving young people in the development and delivery of services designed for them leads to better engagement. Further research is required to understand the role of participatory approaches in design of technology-based mental health and well-being interventions for youth. Objective: To investigate consumer involvement processes and associated outcomes from studies using participatory methods in development of technology-based mental health and well-being interventions for youth. Methods: Fifteen electronic databases, using both resource-specific subject headings and text words, were searched describing 2 broad concepts-participatory research and mental health/illness. Grey literature was accessed via Google Advanced search, and relevant conference Web sites and reference lists were also searched. A first screening of titles/abstracts eliminated irrelevant citations and documents. The remaining citations were screened by a second reviewer. Full text articles were double screened. All projects employing participatory research processes in development and/or design of (ICT/digital) technology-based youth mental health and well-being interventions were included. No date restrictions were applied; English language only. Data on consumer involvement, research and design process, and outcomes were extracted via framework analysis. Results: A total of 6210 studies were reviewed, 38 full articles retrieved, and 17 included in this study. It was found that consumer participation was predominantly consultative and consumerist in nature and involved design specification and intervention development, and usability/pilot testing. Sustainable participation was difficult to achieve. Projects reported clear dichotomies around designer/researcher and consumer assumptions of effective and acceptable interventions. It was not possible to determine the impact of participatory research on intervention effectiveness due to lack of outcome data. Planning for or having pre-existing implementation sites assisted implementation. The review also revealed a lack of theory-based design and process evaluation. Conclusions: Consumer consultations helped shape intervention design. However, with little evidence of outcomes and a lack of implementation following piloting, the value of participatory research remains unclear.

  • ASSESS MS being used to record a finger-nose test while the health professional monitors.

    Usability and Acceptability of ASSESS MS: Assessment of Motor Dysfunction in Multiple Sclerosis Using Depth-Sensing Computer Vision


    Background: Sensor-based recordings of human movements are becoming increasingly important for the assessment of motor symptoms in neurological disorders beyond rehabilitative purposes. ASSESS MS is a movement recording and analysis system being developed to automate the classification of motor dysfunction in patients with multiple sclerosis (MS) using depth-sensing computer vision. It aims to provide a more consistent and finer-grained measurement of motor dysfunction than currently possible. Objective: To test the usability and acceptability of ASSESS MS with health professionals and patients with MS. Methods: A prospective, mixed-methods study was carried out at 3 centers. After a 1-hour training session, a convenience sample of 12 health professionals (6 neurologists and 6 nurses) used ASSESS MS to capture recordings of standardized movements performed by 51 volunteer patients. Metrics for effectiveness, efficiency, and acceptability were defined and used to analyze data captured by ASSESS MS, video recordings of each examination, feedback questionnaires, and follow-up interviews. Results: All health professionals were able to complete recordings using ASSESS MS, achieving high levels of standardization on 3 of 4 metrics (movement performance, lateral positioning, and clear camera view but not distance positioning). Results were unaffected by patients’ level of physical or cognitive disability. ASSESS MS was perceived as easy to use by both patients and health professionals with high scores on the Likert-scale questions and positive interview commentary. ASSESS MS was highly acceptable to patients on all dimensions considered, including attitudes to future use, interaction (with health professionals), and overall perceptions of ASSESS MS. Health professionals also accepted ASSESS MS, but with greater ambivalence arising from the need to alter patient interaction styles. There was little variation in results across participating centers, and no differences between neurologists and nurses. Conclusions: In typical clinical settings, ASSESS MS is usable and acceptable to both patients and health professionals, generating data of a quality suitable for clinical analysis. An iterative design process appears to have been successful in accounting for factors that permit ASSESS MS to be used by a range of health professionals in new settings with minimal training. The study shows the potential of shifting ubiquitous sensing technologies from research into the clinic through a design approach that gives appropriate attention to the clinic environment.

  • An example of a person using a home health care device. Copyright 2015 by Philip Kortum.

    Evaluation of Home Health Care Devices: Remote Usability Assessment


    Background: An increasing amount of health care is now performed in a home setting, away from the hospital. While there is growing anecdotal evidence about the difficulty patients and caregivers have using increasingly complex health care devices in the home, there has been little systematic scientific study to quantify the global nature of home health care device usability in the field. Research has tended to focus on a handful of devices, making it difficult to gain a broad view of the usability of home-care devices in general. Objective: The objective of this paper is to describe a remote usability assessment method using the System Usability Scale (SUS), and to report on the usability of a broad range of health care devices using this metric. Methods: A total of 271 participants selected and rated up to 10 home health care devices of their choice using the SUS, which scores usability from 0 (unusable) to 100 (highly usable). Participants rated a total of 455 devices in their own home without an experimenter present. Results: Usability scores ranged from 98 (oxygen masks) to 59 (home hormone test kits). An analysis conducted on devices that had at least 10 ratings showed that the effect of device on SUS scores was significant (P<.001), and that the usability of these devices was on the low end when compared with other commonly used items in the home, such as microwave ovens and telephones. Conclusions: A large database of usability scores for home health care devices collected using this remote methodology would be beneficial for physicians, patients, and their caregivers.

  • eHealth Literacy is important to address in design processes with the diversity of users and equipment. Copyright: iStock, purchased according to Standard License Agreement.

    Enhancing the Effectiveness of Consumer-Focused Health Information Technology Systems Through eHealth Literacy: A Framework for Understanding Users' Needs


    Background: eHealth systems and applications are increasingly focused on supporting consumers to directly engage with and use health care services. Involving end users in the design of these systems is critical to ensure a generation of usable and effective eHealth products and systems. Often the end users engaged for these participatory design processes are not actual representatives of the general population, and developers may have limited understanding about how well they might represent the full range of intended users of the eHealth products. As a consequence, resulting information technology (IT) designs may not accommodate the needs, skills, cognitive capacities, and/or contexts of use of the intended broader population of health consumers. This may result in challenges for consumers who use the health IT systems, and could lead to limitations in adoption if the diversity of user attributes has not been adequately considered by health IT designers. Objective: The objective of this paper is to propose how users’ needs and competences can be taken into account when designing new information and communications technology solutions in health care by expanding the user-task-context matrix model with the domains of a new concept of eHealth literacy. Methods: This approach expands an existing method for supporting health IT system development, which advocates use of a three-dimensional user-task-context matrix to comprehensively identify the users of health IT systems, and what their needs and requirements are under differing contexts of use. The extension of this model involved including knowledge about users’ competences within the seven domains of eHealth literacy, which had been identified based on systematic engagement with computer scientists, academics, health professionals, and patients recruited from various patient organizations and primary care. A concept map was constructed based on a structured brainstorm procedure, card sorting, and computational analysis. Results: The new eHealth literacy concept (based on 7 domains) was incorporated as a key factor in expanding the user-task-context matrix to describe and qualify user requirements and understanding related to eHealth literacy. This resulted in an expanded framework and a five-step process, which can support health IT designers in understanding and more accurately addressing end-users’ needs, capabilities, and contexts to improve effectiveness and broader applicability of consumer-focused health IT systems. It is anticipated that the framework will also be useful for policy makers involved in the planning, procuring, and funding of eHealth infrastructure, applications, and services. Conclusions: Developing effective eHealth products requires complete understanding of the end-users’ needs from multiple perspectives. In this paper, we have proposed and detailed a framework for modeling users’ needs for designing eHealth systems that merges prior work in development of a user-task-context matrix with the emerging area of eHealth literacy. This framework is intended to be used to guide design of eHealth technologies and to make requirements explicitly related to eHealth literacy, enabling a generation of well-targeted, fit-for-purpose, equitable, and effective products and systems.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)