JMIR Publications

JMIR Human Factors

Making health care interventions and technologies usable, safe, and effective

Advertisement

Journal Description

JMIR Human Factors (JHF) is a new spin-off journal of JMIR, a leading open access eHealth journal (Impact Factor 2014: 3.4). 
JMIR Human Factors is a multidisciplinary journal with contributions from medical researchers, engineers, and social scientists.
JMIR Human Factors focuses on understanding how the behaviour and thinking of humans can influence and shape the design of health care interventions and technologies, and how the design can be evaluated and improved to make health care interventions and technologies usable, safe, and effective.
JHF aspires to lead health care towards a culture of testing and safety. All articles are professionally copyedited and typeset, ready for indexing in PubMed/PubMed Central. Possible contributions include usability studies and heuristic evaluations, studies concerning ergonomics and error prevention, design studies for medical devices and healthcare systems/workflows, enhancing teamwork through Human Factors based teamwork training, measuring non-technical skills in staff like leadership, communication, situational awareness and teamwork, and healthcare policies and procedures to reduce errors and increase safety. Reviews, viewpoint papers and tutorials are as welcome as original research.

Editorial Board members are currently being recruited, please contact us if you are interested (jmir.editorial.office at gmail.com).

 
 
 
 

Recent Articles:

  • Diabetes MAP homepage. Source: https://diabetesmap.com, Copyright: Author (CYO).

    The Usability of Diabetes MAP: A Web-delivered Intervention for Improving Medication Adherence

    Abstract:

    Background: Web-delivered interventions are a feasible approach to health promotion. However, if a website is poorly designed, difficult to navigate, and has technical bugs, it will not be used as intended. Usability testing prior to evaluating a website’s benefits can identify barriers to user engagement and maximize future use. Objective: We developed a Web-delivered intervention called Diabetes Medication Adherence Promotion (Diabetes MAP) and used a mixed-methods approach to test its usability prior to evaluating its efficacy on medication adherence and glycemic control in a randomized controlled trial. Methods: We recruited English-speaking adults with type 2 diabetes mellitus (T2DM) from an academic medical center who were prescribed diabetes medications. A trained research assistant administered a baseline survey, collected medical record information, and instructed participants on how to access Diabetes MAP. Participants were asked to use the site independently for 2 weeks and to provide survey and/or focus group feedback on their experience. We analyzed survey data descriptively and qualitative data thematically to identify participants’ favorable and unfavorable experiences, characterize usability concerns, and solicit recommendations for improving Diabetes MAP. Results: Enrolled participants (N=32) were an average of 51.7 ± 11.8 years old, 66% (21/32) female, 60% (19/32) non-Hispanic White, 88% (28/32) had more than 12 years of education, half had household incomes over $50,000, and 78% (25/32) were privately insured. Average duration of diagnosed diabetes was 7.8 ± 6.3 years, average A1c was 7.4 ± 2.0, and 38% (12/32) were prescribed insulin. Of enrolled participants, 91% (29/32) provided survey and/or focus group feedback about Diabetes MAP. On the survey, participants agreed website information was clear and easy to understand, but in focus groups they reported navigational challenges and difficulty overcoming user errors (eg, entering data in an unspecified format). Participants also reported difficulty accessing the site and, once accessed, using all of its features. Participants recommended improving the site’s user interface to facilitate quick, efficient access to all features and content. Conclusions: Adults with T2DM rated the Diabetes MAP website favorably on surveys, but focus groups gave more in-depth feedback on the user experience (eg, difficulty accessing the site, maximizing all of the site’s features and content, and recovering from errors). Appropriate usability testing methods ensure Web-delivered interventions work as intended and any benefits are not diminished by usability challenges.

  • Senior Asian man using a tablet. Image Source: http://www.istockphoto.com/photo/senior-asian-man-using-a-tablet-gm155147058-18577056; copyright: Angela et al, Licensed under ISTOCK CONTENT LICENSE AGREEMENT.

    The McMaster Optimal Aging Portal: Usability Evaluation of a Unique Evidence-Based Health Information Website

    Abstract:

    Background: Increasingly, older adults and their informal caregivers are using the Internet to search for health-related information. There is a proliferation of health information online, but the quality of this information varies, often based on exaggerated or dramatic findings, and not easily comprehended by consumers. The McMaster Optimal Aging Portal (Portal) was developed to provide Internet users with high-quality evidence about aging and address some of these current limitations of health information posted online. The Portal includes content for health professionals coming from three best-in-class resources (MacPLUS, Health Evidence, and Health Systems Evidence) and four types of content specifically prepared for the general public (Evidence Summaries, Web Resource Ratings, Blog Posts, and Twitter messages). Objective: Our objectives were to share the findings of the usability evaluation of the Portal with particular focus on the content features for the general public and to inform designers of health information websites and online resources for older adults about key usability themes. Methods: Data analysis included task performance during usability testing and qualitative content analyses of both the usability sessions and interviews to identify core themes. Results: A total of 37 participants took part in 33 usability testing sessions and 21 focused interviews. Qualitative analysis revealed common themes regarding the Portal’s strengths and challenges to usability. The strengths of the website were related to credibility, applicability, browsing function, design, and accessibility. The usability challenges included reluctance to register, process of registering, searching, terminology, and technical features. Conclusions: The study reinforced the importance of including end users during the development of this unique, dynamic, evidence-based health information website. The feedback was applied to iteratively improve website usability. Our findings can be applied by designers of health-related websites.

  • High-fidelity prototype version of Loop with Patient and Team and Team Only toggle from message compose box.

    In the Loop: The Organization of Team-Based Communication in a Patient-Centered Clinical Collaboration System

    Abstract:

    Background: We describe the development and evaluation of a secure Web-based system for the purpose of collaborative care called Loop. Loop assembles the team of care with the patient as an integral member of the team in a secure space. Objective: The objectives of this paper are to present the iterative design of the separate views for health care providers (HCPs) within each patient’s secure space and examine patients’, caregivers’, and HCPs’ perspectives on this separate view for HCP-only communication. Methods: The overall research program includes cycles of ethnography, prototyping, usability testing, and pilot testing. This paper describes the usability testing phase that directly informed development. A descriptive qualitative approach was used to analyze participant perspectives that emerged during usability testing. Results: During usability testing, we sampled 89 participants from three user groups: 23 patients, 19 caregivers, and 47 HCPs. Almost all perspectives from the three user groups supported the need for an HCP-only communication view. In an earlier prototype, the visual presentation caused confusion among HCPs when reading and composing messages about whether a message was visible to the patient. Usability testing guided us to design a more deliberate distinction between posting in the Patient and Team view and the Health Care Provider Only view at the time of composing a message, which once posted is distinguished by an icon. Conclusions: The team made a decision to incorporate an HCP-only communication view based on findings during earlier phases of work. During usability testing we tested the separate communication views, and all groups supported this partition. We spent considerable effort designing the partition; however, preliminary findings from the next phase of evaluation, pilot testing, show that the Patient and Team communication is predominantly being used. This demonstrates the importance of a subsequent phase of the clinical trial of Loop to validate the concept and design.

  • Photo by NEC Corporation of America with Creative Commons license.

Source URL: https://www.flickr.com/photos/neccorp/14445634744/in/photolist-CEWYsM-cJxAEY-o1vDW1-cJorC1-8yiJ8h-cJMKHy-xZgk6a-nJ7P9M-9M2QU4-8cSJ53-BSt9Kx-BxBYp7-nJ8QAF-8pP5hY-oWLwF8-frDEFa-bz5dkX-aJ2zH4-85z1hy-eShZ4G-9pMhdh-eSjtuu-eS7UZv-eSjr2L-eLSmog-nVgbaB-qZ7ADt-wo2boW-qdbbML-9sKExt-eM4J91-eS6yvF-nAZfjd-nTtJLP/.

    Implementation of a Computerized Screening Inventory: Improved Usability Through Iterative Testing and Modification

    Abstract:

    Background: The administration of health screeners in a hospital setting has traditionally required (1) clinicians to ask questions and log answers, which can be time consuming and susceptible to error, or (2) patients to complete paper-and-pencil surveys, which require third-party entry of information into the electronic health record and can be vulnerable to error and misinterpretation. A highly promising method that avoids these limitations and bypasses third-party interpretation is direct entry via a computerized inventory. Objective: To (1) computerize medical and behavioral health screening for use in general medical settings, (2) optimize patient acceptability and feasibility through iterative usability testing and modification cycles, and (3) examine how age relates to usability. Methods: A computerized version of 15 screeners, including behavioral health screeners recommended by a National Institutes of Health Office of Behavioral and Social Sciences Research collaborative workgroup, was subjected to systematic usability testing and iterative modification. Consecutive adult, English-speaking patients seeking treatment in an urban emergency department were enrolled. Acceptability was defined as (1) the percentage of eligible patients who agreed to take the assessment (initiation rate) and (2) average satisfaction with the assessment (satisfaction rate). Feasibility was defined as the percentage of the screening items completed by those who initiated the assessment (completion rate). Chi-square tests, analyses of variance, and Pearson correlations were used to detect whether improvements in initiation, satisfaction, and completion rates were seen over time and to examine the relation between age and outcomes. Results: Of 2157 eligible patients approached, 1280 agreed to complete the screening (initiation rate=59.34%). Statistically significant increases were observed over time in satisfaction (F3,1061=3.35, P=.019) and completion rates (F3,1276=25.44, P<.001). Younger age was associated with greater initiation (initiated, mean [SD], 46.6 [18.7] years; declined: 53.0 [19.5] years, t2,155=−7.6, P<.001), higher completion (r=−.20, P<.001), and stronger satisfaction (r=−.23, P<.001). Conclusions: In a rapid-paced emergency department with a heterogeneous patient population, 59.34% (1280/2157) of all eligible patients initiated the computerized screener with a completion rate reaching over 90%. Usability testing revealed several critical principles for maximizing usability of the computerized medical and behavioral health screeners used in this study. Further work is needed to identify usability issues pertaining to other screeners, racially and ethnically diverse patient groups, and different health care settings.

  • Usability labs at Toronto General Hospital showing a complete set up of a simulated operating room (including a patient simulator).

    Challenges and Paradoxes of Human Factors in Health Technology Design

    Abstract:

    Usability testing allows human factors professionals to identify and mitigate issues with the design and use of medical technology. The test results, however, can be paradoxical and therefore be misinterpreted, limiting their usefulness. The paradoxical findings can lead to products that are not aligned with the needs and constraints of their users. We herein report on our observations of the paradox of expertise, the paradox of preference versus performance, and the paradox of choice. Each paradox explored is in the perspective of the design of medical technology, the issues that need to be considered in the interpretation of the test results, as well as suggestions on how to avoid the pitfalls in the design of medical technology. Because these paradoxes can influence product design at various stages of product development, it is important to be aware of the effects to interpret the findings properly.

  • Photo courtesy of FreeDigitalPhotos.net / stockimages. Image ID: 100102403. http://www.freedigitalphotos.net/images/Computing_g368-Senior_Doctor_Using_Computer_p102403.html.

    Evaluating the Usability and Perceived Impact of an Electronic Medical Record Toolkit for Atrial Fibrillation Management in Primary Care: A Mixed-Methods...

    Abstract:

    Background: Atrial fibrillation (AF) is a common and preventable cause of stroke. Barriers to reducing stroke risk through appropriate prescribing have been identified at the system, provider, and patient levels. To ensure a multifaceted initiative to address these barriers is effective, it is essential to incorporate user-centered design to ensure all intervention components are optimized for users. Objective: To test the usability of an electronic medical record (EMR) toolkit for AF in primary care with the goal of further refining the intervention to meet the needs of primary care clinicians. Methods: An EMR-based toolkit for AF was created and optimized through usability testing and iterative redesign incorporating a human factors approach. A mixed-methods pilot study consisting of observations, semi-structured interviews, and surveys was conducted to examine usability and perceived impact on patient care and workflow. Results: A total of 14 clinicians (13 family physicians and 1 nurse practitioner) participated in the study. Nine iterations of the toolkit were created in response to feedback from clinicians and the research team; interface-related changes were made, additional AF-related resources were added, and functionality issues were fixed to make the toolkit more effective. After improvements were made, clinicians expressed that the toolkit improved accessibility to AF-related information and resources, served as a reminder for guideline-concordant AF management, and was easy to use. Most clinicians intended to continue using the toolkit for patient care. With respect to impact on care, clinicians believed the toolkit increased the thoroughness of their assessments for patients with AF and improved the quality of AF-related care received by their patients. Conclusions: The positive feedback surrounding the EMR toolkit for AF and its perceived impact on patient care can be attributed to the adoption of a user-centered design that merged clinically important information about AF management with user needs. This study demonstrates the utility of a human factors approach to piloting and refining an intervention prior to wide-scale implementation.

  • Woman looking at laptop. (Website link: http://www.istockphoto.com/photo/business-woman-looking-at-laptop-gm503422576-82510999).

Copyright: grinvalds.

    How Does Learnability of Primary Care Resident Physicians Increase After Seven Months of Using an Electronic Health Record? A Longitudinal Study

    Abstract:

    Background: Electronic health records (EHRs) with poor usability present steep learning curves for new resident physicians, who are already overwhelmed in learning a new specialty. This may lead to error-prone use of EHRs in medical practice by new resident physicians. Objective: The study goal was to determine learnability gaps between expert and novice primary care resident physician groups by comparing performance measures when using EHRs. Methods: We compared performance measures after two rounds of learnability tests (November 12, 2013 to December 19, 2013; February 12, 2014 to April 22, 2014). In Rounds 1 and 2, 10 novice and 6 expert physicians, and 8 novice and 4 expert physicians participated, respectively. Laboratory-based learnability tests using video analyses were conducted to analyze learnability gaps between novice and expert physicians. Physicians completed 19 tasks, using a think-aloud strategy, based on an artificial but typical patient visit note. We used quantitative performance measures (percent task success, time-on-task, mouse activities), a system usability scale (SUS), and qualitative narrative feedback during the participant debriefing session. Results: There was a 6-percentage-point increase in novice physicians’ task success rate (Round 1: 92%, 95% CI 87-99; Round 2: 98%, 95% CI 95-100) and a 7-percentage-point increase in expert physicians’ task success rate (Round 1: 90%, 95% CI 83-97; Round 2: 97%, 95% CI 93-100); a 10% decrease in novice physicians’ time-on-task (Round 1: 44s, 95% CI 32-62; Round 2: 40s, 95% CI 27-59) and 21% decrease in expert physicians’ time-on-task (Round 1: 39s, 95% CI 29-51; Round 2: 31s, 95% CI 22-42); a 20% decrease in novice physicians mouse clicks (Round 1: 8 clicks, 95% CI 6-13; Round 2: 7 clicks, 95% CI 4-12) and 39% decrease in expert physicians’ mouse clicks (Round 1: 8 clicks, 95% CI 5-11; Round 2: 3 clicks, 95% CI 1-10); a 14% increase in novice mouse movements (Round 1: 9247 pixels, 95% CI 6404-13,353; Round 2: 7991 pixels, 95% CI 5350-11,936) and 14% decrease in expert physicians’ mouse movements (Round 1: 7325 pixels, 95% CI 5237-10,247; Round 2: 6329 pixels, 95% CI 4299-9317). The SUS measure of overall usability demonstrated only minimal change in the novice group (Round 1: 69, high marginal; Round 2: 68, high marginal) and no change in the expert group (74; high marginal for both rounds). Conclusions: This study found differences in novice and expert physicians’ performance, demonstrating that physicians’ proficiency increased with EHR experience. Our study may serve as a guideline to improve current EHR training programs. Future directions include identifying usability issues faced by physicians when using EHRs, through a more granular task analysis to recognize subtle usability issues that would otherwise be overlooked.

  • The homepage of the Social POD app.

Image copyright: Sarah (author).

    A Mixed-Methods Approach to the Development, Refinement, and Pilot Testing of Social Networks for Improving Healthy Behaviors

    Abstract:

    Background: Mobile health (mHealth) has shown promise as a way to deliver weight loss interventions, including connecting users for social support. Objective: To develop, refine, and pilot test the Social Pounds Off Digitally (POD) Android app for personalized health monitoring and interaction. Methods: Adults who were overweight and obese with Android smartphones (BMI 25-49.9 kg/m2; N=9) were recruited for a 2-month weight loss pilot intervention and iterative usability testing of the Social POD app. The app prompted participants via notification to track daily weight, diet, and physical activity behaviors. Participants received the content of the behavioral weight loss intervention via podcast. In order to re-engage infrequent users (did not use the app within the previous 48 hours), the app prompted frequent users to select 1 of 3 messages to send to infrequent users targeting the behavioral theory constructs social support, self-efficacy, or negative outcome expectations. Body weight, dietary intake (2 24-hr recalls), and reported calories expended during physical activity were assessed at baseline and 2 months. All participants attended 1 of 2 focus groups to provide feedback on use of the app. Results: Participants lost a mean of 0.94 kg (SD 2.22, P=.24) and consumed significantly fewer kcals postintervention (1570 kcal/day, SD 508) as compared to baseline (2384 kcal/day, SD 993, P=.01). Participants reported expending a mean of 171 kcal/day (SD 153) during intentional physical activity following the intervention as compared to 138 kcal/day (SD 139) at baseline, yet this was not a statistically significant difference (P=.57). There was not a statistically significant correlation found between total app entries and percent weight loss over the course of the intervention (r=.49, P=.19). Mean number of app entries was 77.2 (SD 73.8) per person with a range of 0 to 219. Messages targeting social support were selected most often (32/68, 47%), followed by self-efficacy (28/68, 41%), and negative outcome expectations (8/68, 12%). Themes from the focus groups included functionality issues, revisions to the messaging system, and the addition of a point system with rewards for achieving goals. Conclusions: The Social POD app provides an innovative way to re-engage infrequent users by encouraging frequent users to provide social support. Although more time is needed for development, this mHealth intervention can be disseminated broadly for many years and to many individuals without the need for additional intensive in-person hours.

  • Sociotechnical human factors. Created by Lori Wozney (author) who holds the copyright.

    Sociotechnical Human Factors Involved in Remote Online Usability Testing of Two eHealth Interventions

    Abstract:

    Background: Research in the fields of human performance technology and human computer interaction are challenging the traditional macro focus of usability testing arguing for methods that help test moderators assess “use in context” (ie, cognitive skills, usability understood over time) and in authentic “real world” settings. Human factors in these complex test scenarios may impact on the quality of usability results being derived yet there is a lack of research detailing moderator experiences in these test environments. Most comparative research has focused on the impact of the physical environment on results, and rarely on how the sociotechnical elements of the test environment affect moderator and test user performance. Improving our understanding of moderator roles and experiences with conducting “real world” usability testing can lead to improved techniques and strategies Objective: To understand moderator experiences of using Web-conferencing software to conduct remote usability testing of 2 eHealth interventions. Methods: An exploratory case study approach was used to study 4 moderators’ experiences using Blackboard Collaborate for remote testing sessions of 2 different eHealth interventions. Data collection involved audio-recording iterative cycles of test sessions, collecting summary notes taken by moderators, and conducting 2 90-minute focus groups via teleconference. A direct content analysis with an inductive coding approach was used to explore personal accounts, assess the credibility of data interpretation, and generate consensus on the thematic structure of the results. Results: Following the convergence of data from the various sources, 3 major themes were identified: (1) moderators experienced and adapted to unpredictable changes in cognitive load during testing; (2) moderators experienced challenges in creating and sustaining social presence and untangling dialogue; and (3) moderators experienced diverse technical demands, but were able to collaboratively troubleshoot with test users. Conclusions: Results highlight important human-computer interactions and human factor qualities that impact usability testing processes. Moderators need an advanced skill and knowledge set to address the social interaction aspects of Web-based usability testing and technical aspects of conferencing software during test sessions. Findings from moderator-focused studies can inform the design of remote testing platforms and real-time usability evaluation processes that place less cognitive burden on moderators and test users.

  • Design Thinking.

Page URL: https://commons.wikimedia.org/wiki/File%3AXhibit_015.JPG
File URL: https://upload.wikimedia.org/wikipedia/commons/6/61/Xhibit_015.JPG
Attribution: By Saad Alzarooni (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons.

    Mental Health Technologies: Designing With Consumers

    Abstract:

    Despite growing interest in the promise of e-mental and well-being interventions, little supporting literature exists to guide their design and the evaluation of their effectiveness. Both participatory design (PD) and design thinking (DT) have emerged as approaches that hold significant potential for supporting design in this space. Each approach is difficult to definitively circumscribe, and as such has been enacted as a process, a mind-set, specific practices/techniques, or a combination thereof. At its core, however, PD is a design research tradition that emphasizes egalitarian partnerships with end users. In contrast, DT is in the process of becoming a management concept tied to innovation with strong roots in business and education. From a health researcher viewpoint, while PD can be reduced to a number of replicable stages that involve particular methods, techniques, and outputs, projects often take vastly different forms and effective PD projects and practice have traditionally required technology-specific (eg, computer science) and domain-specific (eg, an application domain, such as patient support services) knowledge. In contrast, DT offers a practical off-the-shelf toolkit of approaches that at face value have more potential to have a quick impact and be successfully applied by novice practitioners (and those looking to include a more human-centered focus in their work). Via 2 case studies we explore the continuum of similarities and differences between PD and DT in order to provide an initial recommendation for what health researchers might reasonably expect from each in terms of process and outcome in the design of e-mental health interventions. We suggest that the sensibilities that DT shares with PD (ie, deep engagement and collaboration with end users and an inclusive and multidisciplinary practice) are precisely the aspects of DT that must be emphasized in any application to mental health provision and that any technology development process must prioritize empathy and understanding over innovation for the successful uptake of technology in this space.

  • https://www.flickr.com/photos/anu-image-library/19456191996/in/album-72157655144457668/

Copyright: Australian National University.

    University Students’ Views on the Perceived Benefits and Drawbacks of Seeking Help for Mental Health Problems on the Internet: A Qualitative Study

    Abstract:

    Background: University students experience high levels of mental health problems yet very few seek professional help. Web-based mental health interventions may be useful for the university student population. However, there are few published qualitative studies that have examined the perceived benefits and drawbacks of seeking help for mental health problems on the Internet from the perspective of university students. Objective: To investigate the attitudes of university students on mental health help-seeking on the Internet. Methods: A total of 19 university students aged 19-24 years participated in 1 of 4 focus groups to examine their views toward help-seeking for mental health problems on the Internet. Results: Perceived concerns about Web-based help-seeking included privacy and confidentiality, difficulty communicating on the Internet, and the quality of Web-based resources. Potential benefits included anonymity/avoidance of stigma, and accessibility. Participants reported mixed views regarding the ability of people with similar mental health issues to interact on the Internet. Conclusions: These factors should be considered in the development of Web-based mental health resources to increase acceptability and engagement from university students.

  • Elder Tree participant using the website at her kitchen table.  Copyright Center for Health Enhancement Systems Studies, UW-Madison.

    Using the NIATx Model to Implement User-Centered Design of Technology for Older Adults

    Abstract:

    What models can effectively guide the creation of eHealth and mHealth technologies? This paper describes the use of the NIATx model as a framework for the user-centered design of a new technology for older adults. The NIATx model is a simple framework of process improvement based on the following principles derived from an analysis of decades of research from various industries about why some projects fail and others succeed: (1) Understand and involve the customer; (2) fix key problems; (3) pick an influential change leader; (4) get ideas from outside the field; (5) use rapid-cycle testing. This paper describes the use of these principles in technology development, the strengths and challenges of using this approach in this context, and lessons learned from the process. Overall, the NIATx model enabled us to produce a user-focused technology that the anecdotal evidence available so far suggests is engaging and useful to older adults. The first and fourth principles were especially important in developing the technology; the fourth proved the most challenging to use.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Advertisement