A Novel Digital Pill System for Medication Adherence Measurement and Reporting: Usability Validation Study

Introduction

Background

Medication nonadherence is a problem that continues to plague the health care system. Data have shown that patients do not report their own adherence accurately [1] and that health care providers are generally poor judges of their patients’ adherence [2]. In clinical practice, it is known that up to 50% of patients do not take their medications as prescribed [3], even in serious disease states or conditions where the consequences can be severe, such as diabetes, heart failure, hyperlipidemia, hypertension, and organ transplantation [4,5].

In addition, medications that have been shown to improve the quality of life, prevent tumor progression, and prolong survival are often not taken as prescribed by patients with cancer [6]. A systematic review of publications on oral anticancer medications from 2003 to 2015 showed that medication adherence rates varied widely from 46% to 100% [7]. In interviews with patients with breast cancer, de Mendoza et al [8] found that 78.9% of the patients failed to report medication discontinuation immediately and 57.9% overreported medication adherence.

Nonadherence is multifactorial. The common reasons for nonadherence are confusion (about complex drug regimens), a lack of commitment to the treatment plan, fear of adverse events, cost of drugs, forgetfulness, lack of symptoms, illness factors such as depression or psychosis, and miscommunication or lack of trust between the patient and the health care team [9-11].

Clinical trials too are often impaired by suboptimal adherence and flawed in the way they track medication adherence [12-14]. For example, in a systematic review that captured adherence data from 95 clinical trials involving 16,907 participants, there was an immediate 4% drop-off of the enrolled participants because of noninitiation of therapy. By day 100, 20% of the participants had stopped taking the medication. A further 12% displayed imperfect adherence on a daily basis [14]. Adherence errors can result in suboptimal dosing and inaccurate assessments of efficacy, safety, and tolerability, thus delaying the drug development process and potentially adding millions of dollars in additional costs [9,15].

The need for objective and reliable ways to confirm medication use has driven the development of various tracking methods, including patient self-reports, adherence-reporting mobile apps, pill counts, pharmacy prescription refill rates, electronic pill dispensers, and other solutions—but none have been optimized, and many are not reliable [16]. Digital pill systems, in contrast, have demonstrated a high rate of accuracy, with a study showing a 99.4% adherence rate across 2824 digital pill ingestions that were tracked [17].

The ID-Cap System (etectRx, Inc) is a digital pill system and ingestible event marker (Code of Federal Regulations 21 §880.6305) that enables adherence measurement through an embedded ingestible sensor. The biocompatible sensor, upon coming into contact with gastrointestinal fluid, communicates a digital signal through radio frequency after ingestion and dissolution of the pharmaceutical-grade capsule shell that encapsulates it. A reader worn by the patient detects the radio frequency signal and forwards ingestion data to the patient app and clinician dashboard. Information about the ingestion event is then wirelessly transferred to a Health Insurance Portability and Accountability Act–compliant cloud-based server [18] for secure sharing with authorized users. The sensor is naturally and safely eliminated from the body.

The Food and Drug Administration granted 510(k) clearance to the ID-Cap System in December 2019. The regulatory review of this medical device and its related software encompassed the results of human factors validation testing among patient users, clinician users, and system administrators, including the results of the study reported herein. The components of the ID-Cap System are shown in Figures 1-5. The patient user testing was conducted with the following system components: the ID-Capsule, the ID-Cap Reader, and the ID-Cap Patient App.

‎View this figure

‎View this figure

‎View this figure

‎View this figure

‎View this figure

Objective

In this paper, we describe the human factors validation study involving patient users, the intent of which is to evaluate the ID-Cap System for usability, ensuring that patient users will be able to operate the system as intended in a simulated real-world use environment. Our key questions were as follows:

1. Are patient users able to perform critical tasks effectively and safely, using an interface representative of the final device design, in conditions representing the actual conditions of use?
2. Can they do so without errors and without difficulties that could cause harm?

This validation study excluded clinical safety and effectiveness elements, which have been assessed and documented in separate evaluations and pivotal clinical trials supporting the use of the device. The researchers hypothesized a priori that the patient users would successfully demonstrate their ability to safely and effectively use the ID-Cap System for its intended use.

Methods

Overview

To conduct the human factors validation test of the ID-Cap System, we used assessment testing. This type of test provides users with realistic tasks to perform, using a working prototype of the device but without requiring any clinical use [19].

The training and testing took place at the office facilities of Tensentric, Inc, in Boulder, Colorado. The test was conducted in either conference rooms or dedicated research rooms set up to represent a typical home-use environment. An ID-Cap System was provided, including supplemental test equipment (eg, laptops for training videos) and product labeling.

Participants

The research team had a recruitment target of up to 18 patient users to ensure that the goal of a minimum of 15 participants from the intended user population would be met; 15 test participants per user group represents the most stringent sample size guidance from regulatory bodies for human factors validation testing [20]. The participants were recruited by an independent third-party recruiting firm that had no knowledge of etectRx’s involvement at the time of recruitment. None of the participants were employed by, or affiliated with, etectRx, nor had they participated in a preceding formative usability or validation test of the ID-Cap System. Each participant signed a nondisclosure agreement and an informed consent form documenting their agreement for participation in the test session and video recording. Each participant received an honorarium for participation in the study, which was distributed after the completion of the test session.

Our goal was to obtain a sample of test participants who represent the intended patient users of the ID-Cap System. The recruited participants took medication by mouth on a regular basis, were able to understand and follow directions, and communicated clearly. If a caregiver assisted the participant in taking medication or in day-to-day activities, the caregiver also participated in the test. Participants with conditions that affected their ability to make health decisions or follow their physician’s instructions and those who did not use a smartphone were excluded.

This was an all-comers study that was, by design, both inclusive and diverse. Diversity within the sample size was promoted within the recruitment screener by quantified maximums or minimums on specific populations as well as instructions to recruit a variety of participants, including a mix of handedness, sexes, races, educational levels, and disease states. The demographics that were recorded after scheduling included visual acuity, age, handedness, sex, medical conditions, how the participant currently ensures that they are taking their medication, color blindness, visual impairments, and any visual corrections.

Training

The participants received a 40-minute training session for the ID-Cap System to orient them to the basic features, functions, and nomenclature of the ingestible sensor, wearable reader, and patient app. They were shown a series of 5 short training videos containing key information and demonstrating the correct use of the device. They were then asked to demonstrate the steps for completing a successful ingestion using the reader and patient app before using the system independently during the test session. The training was equivalent to what is expected to be delivered to actual users, and the content, format, and method of delivery of training were comparable to the training that actual users would receive. The training materials and device instructions for use were designed to support a self-guided supplemental training program for the patient user.

The training videos covered the following topics:

• Overview of the ID-Cap System
• Setup of charger and reader for the first time
• Routine use of the system without use of the patient app
• Routine use of the system with use of the patient app
• App navigation and functionality

Training was conducted at least one hour before the patient’s test session to simulate a typical level of training decay. After completing the training, the participants were sent away from the test environment for at least one hour with no materials. Only after the waiting period of an hour were the participants able to begin executing the test scenarios. This 1-hour gap, which was added to be more indicative of real-world gaps that exist between training and first-time use, represents the recommended time frame when evaluating potential use-related risks related to training decay [19].

Testing

After the training session and training decay period, the participants initiated a guided 60-minute test session. Time variation was expected in completion of the test session based on factors such as operator skill, training retention, and experience with similar devices and smartphone apps. Actual task times were documented through the video recording of each session and referenced as needed.

During the recorded test session, the participants were observed as they completed the use tasks and monitored by at least one facilitator and either a second facilitator or an observer seated in the same room as the participants. Printed instructions for use were provided to the participants and placed where they would be freely available for reference. However, the facilitator did not direct the participants to use them; this enabled insight into how patients might (or might not) refer to them during actual use.

During the test session, the participants were asked to complete 12 patient-use scenarios using the ID-Cap System, which included device use tasks and knowledge assessment tasks. They verbally read the use-scenario description that was provided on a printed note card and started the task. The participants were instructed by the facilitator to use the system as independently and naturally as possible to reflect actual use behavior. They were not informed how to complete the task or when to expect error conditions nor were they given any other information that would bias their realistic interaction with the system.

The research team observed the participants as they attempted to carry out the task scenarios and recorded any difficulties or participant comments, which were revisited with the participant during a postscenario interview. A data logger observed and logged participant behavior, user comments, and system activity. The participant was asked to simulate the ingestion of the ID-Capsule and the medication that it was intended to track during the testing. No actual ingestion events occurred during the test session.

After each use scenario, the research facilitators conducted a postscenario interview with each participant to analyze any use-related problems. The participants were prompted to provide subjective and candid assessments of any use issues experienced during the test, their probable causes, and impact.

After all task scenarios were attempted, the participants completed a postsession debriefing interview with the facilitator, where neutrally worded, open-ended questions were posed to them regarding their experience. The participants were asked to provide feedback regarding the use, safety, and usability of the ID-Cap System, as well as the clarity and effectiveness of user resources containing instructions for its use.

Critical Tasks and Use Scenarios

This test protocol was designed to validate the critical use tasks associated with the ID-Cap System for patient users. The tasks were selected and prioritized using a risk-based analysis to cover critical tasks, safety-related tasks, frequent tasks, tasks that must be performed correctly for the device to work as intended, and key device labeling (Table 1).

The tasks that tested safety mitigations were given the highest priority; for example, tasks that have the potential to alter decisions about ID-Capsule ingestion or ingestions of medications that are taken coincident with, or co-ingested with, the ID-Capsule were determined to be the most important to evaluate. Next on the priority scale were tasks that enable proper operation of the system, followed by tasks that occur infrequently or are provided as a convenience to the user.

Data Analysis and Reporting

The cross-functional research team members performed a risk-based review of the human factors validation test results, including the participants’ subjective assessments of any use errors, close calls, or operational difficulties that occurred during the test. Final pass or fail determination was made based on the risk-based review of the test results and if further design modifications were required to mitigate use-related errors.

The final report summarized the test results, which included evaluation of the actual versus expected task outcomes, subjective assessments, any specific use-related problems, and recommendations for resolution.

Results

Demographics and Baseline Characteristics

A total of 17 participants met the recruitment criteria and were enrolled in the validation test, fulfilling the minimum target of 15 participants from the intended user population of patient users. The participants ranged in age from 27 to 74 years (mean 51 years, SD 13.8 years), and they were heterogeneous in other demographic factors as well, such as education level, handedness, and sex (Table 2). Of the 17 participants, 7 (41%) were women, and nearly one-quarter of the participants reported high school as the highest level of education attained. All participants reported taking prescription medications by mouth on a regular basis and using a smartphone. This is consistent with the expected user population for the ID-Cap System.

Test Session Results

In this validation study, the participants successfully completed 97.5% (196/201) of the total patient use scenarios with the ID-Cap System. Of the 12 use scenarios, 9 (75%) were successfully completed without any failures or use errors among the 17 test participants.

The task scenarios included first-time use tasks and repeat-use tasks, as well as tasks that were only completed on a single occasion, as appropriate, to represent actual use of the system. Of note, we found that when use errors did occur, they occurred only in the first instance and were not repeated. The use errors, close calls, and patterns of use difficulties identified in the testing are reported in Table 3.

The results of the human factors validation test were reviewed by a cross-functional team that conducted a risk-based review of each use-related finding. No new use-related risks were identified in the validation test. It was determined that no modifications of the device or software were required to improve safety or usability for the intended patient users, uses, and use environments of the ID-Cap System when operated as indicated and in a manner consistent with its labeling.

Participant Feedback

The participants grasped the potential value of the system in helping them track and report medication adherence and could see the benefits to people who take medications regularly (including themselves). Many commented on the simplicity and ease of use of the system and liked the training and user resources that were provided, especially the training videos and quick start guides.

The participants, in general, conceptually understood that they should always take their medication as prescribed, supporting the intended role of the system as an adjunct tracking system that does not replace or change physician instructions. Of the 16 participants who answered the postsession interview questions (1 participant did not complete the postsession interview), 16 (100%) stated that they felt that they could use the system effectively and safely to record and track ingestion events and 16 (100%) reported that they believed that the system is safe to use as is (Textbox 1).

Discussion

Principal Findings

The patient’s voice has become an important one in health care. Patients are increasingly involved in decisions about their health and medical treatment, and they have become sophisticated health care technology users who understand the value of digital platforms and are eager to use them.

Certainly, the value of the ID-Cap System and other remotely deployed digital health solutions depends on the willingness of patients to engage with them and the ability of patient users to effectively, safely, and conveniently incorporate them into everyday life with minimal training and oversight. The US Food and Drug Administration requires human factors validation testing of digital pill systems and many other medical devices to ensure safety and effectiveness for a device’s users, uses, and intended use environments [20].

The results of this human factors validation study show that a representative group of patient users successfully completed the critical and safety-related use tasks necessary for optimal use of the system independently, after receiving training that was followed by a period of training decay. Although the participants were representative of a diverse group of potential patient users who regularly take oral prescription medications, the sample size in this study was limited. The participants ranged in age from 27 to 74 years (mean 51 years, SD 13.8 years), and they were heterogeneous in other demographic factors as well, such as education level (3/17, 18% reported high school graduate as highest education level), handedness (3/17, 18% were left-handed), and sex (7/17, 41% were women). This study was neither designed nor powered to evaluate differences in usability based on demographic factors, medical history, or medication use. It is important to note that nearly every screened participant, regardless of age or education level, agreed to participate in this study after receiving information about the digital pill system and successfully completed the use tasks, showing that patient users of all types adapted well to a novel digital pill system. Use errors, when they did occur, occurred only in the first instance for tasks that are repeated with use, indicating that the participants learned to use the ID-Cap System rapidly—a positive prognosticator for real-world use. Most patients expressed satisfaction with the ID-Cap System and responded favorably to questions about the ease of use and the perceived value of the system.

Prior iterations of digital pill systems used a patch-based reader that adhered to the patient’s skin. Clinical evaluations of the patch-based form factor indicated significant limitations from the patient user’s perspective with respect to tolerability and usability [17,21]. The ID-Cap System that was tested uses a reader on a lanyard. The patient users found this reader to be easy to use and acceptable in its current form. A wrist-worn reader that may be worn like a watch or may be attached or integrated into the user’s existing watch or smartwatch is currently being evaluated. Patients are not only adjusting to new therapeutic regimens but also working to develop new medication-taking behaviors and to adopt support tools and programs that will help them to be successful. Readers that are unobtrusive and can be easily incorporated into daily life will be most readily accepted [18].

The limitations of our study include the fact that this was a simulated use of the ID-Cap System and did not include the actual taking of medication for adherence tracking. The participants reported taking prescription medications by mouth on a regular basis. However, they were not asked to use the digital pill system in this study to actually track and record their own medication use in the same way that patient users would use it in the real world.

In most clinical applications, patient users would use the system chronically over extended periods of time. This validation test was limited to only a few simulated ingestion events and did not evaluate use over time. Certainly, there may be specific patient populations, medical conditions, or treatment-related effects that would affect the usability of the ID-Cap System. This study evaluated the general operation of the device across a diverse group of patient users with different health conditions and varied medication history. Use-related risks should be assessed when the device is applied to specific clinical situations and patient populations to ensure continued safe and effective use from a human factors perspective. Digital pill systems may be incorrectly used or misused by patients in clinical trials or in clinical practice; however, these aspects of use and failure modes were not specifically explored in this study. Risk analyses have been conducted by the device manufacturer to examine and mitigate risks to patients and device performance associated with device use and misuse.

The availability of a call center or additional supporting resources during the use of the device may assist patients, as would engagement with their health care provider. These resources were not evaluated in this test protocol. Patient support programs and data-driven interventions offered by the care team members, research personnel, or device manufacturer may be beneficial for the use of the ID-Cap System in clinical practice and clinical research. Continued efforts to educate clinicians and patients alike regarding the value of the information provided by the system and proper use of the system will further enhance its adoption. In addition, integration of the ID-Cap System into existing care models, electronic health records, and clinical data management systems is being explored.

There is great potential for digital pill systems such as the ID-Cap System to contribute to the efficiency and ultimate success of clinical trial programs. For example, digital pills may be used to assess the likelihood of adherent behavior among prospective trial participants. Once a trial has started, digital pills can identify patient nonadherence and changes in patterns of use early, enabling rapid intervention and course correction. Dose-finding studies for self-administered oral medications are another ideal application for digital pills because their results and outcomes are dependent on (1) human behavior as it relates to medication taking and (2) the quality of adherence measurement to optimize drug exposure and dosing decisions. The robustness of the adherence data collected with digital pills provides an added level of reassurance that the efficacy and safety results of pivotal drug development trials eventually reported are accurate. The inconsistency of medication adherence reporting within clinical trials has led to the creation of guidelines for researchers and trial sponsors on the inclusion and implementation of adherence measures in study protocols [12-14,22].

Conclusions

The extent of medication adherence, both in clinical use and in clinical trials, is a controllable factor important for therapeutic success and drug development. The pursuit of a solution to the widespread problem of medication nonadherence has led to digital pill systems, which have shown strong performance and a high rate of accuracy. Currently, >15 years of experience and safety data support the use of digital pills with >140,000 ingestions recorded in >1000 patients who have used these devices safely and effectively [23]. In this human factors validation study, the patient users demonstrated the ability to rapidly learn how to use the ID-Cap System and to safely and effectively use the system as intended. The patient users concluded that the device was easy to use and had the potential to be a useful tool for helping to manage their medications. As health care continues to evolve toward remote care delivery and digital health solutions become ubiquitous, systems such as the ID-Cap System that are easy to use, accepted by patients, and valuable in achieving health outcomes will be indispensable.