The study’s design used a multi-modal case study approach, using quantitative (survey) and qualitative (interviews and focus groups) to characterize the phenomenon and context of integrating RPM into the practice setting [13-18]. This approach encourages multiple sources of evidence to understand the clinician’s perspectives about the use and function of RPM in real-world clinical practice and its influence on patient care workflows.

Initially, survey methods assessed clinicians’ responses to the overall usability, functionality, and value of RPM. Second, clinicians participated in semistructured interviews and focus group sessions to elicit feedback and explanations about items of concern found in the survey responses [13,14,17]. The interview sessions aimed to delve deeper into the survey concerns and examine the use of the RPM application from a clinical perspective.

This study focused on clinician end users, specifically nurses involved in clinical office practices that integrated the initial organization’s RPM initiative [3,9-11] from October 15, 2018, to July 10, 2019, at MSK in the Northeastern United States. The clinicians in this study were the first group of end users involved in caring for oncology patients using RPM. The RPM program was rolled out sequentially, beginning with the oncology services that care for patients with the highest symptom burden (Table 1). Clinicians from physician office practice settings, who had experience in caring for at least 5 patients enrolled in RPM, were invited to participate in the usability survey. This criterion ensured that the clinicians had ample experience with using the RPM program. Additionally, a nested sample of clinicians who completed the survey were recruited to take part in semistructured interviews to provide further information about the challenges of the RPM identified from the survey [19-22].

This research is part of an ongoing organizational program of RPM initiatives. This research obtained ethical approval from the MSK Institutional Review Board (X20-086) as exempt research and follows the ethical principles and guidelines of the Belmont Report. All responders to surveys completed web-based informed consent to participate. Participants in focus groups and interviews provided informed verbal consent.

A total of 35 clinicians participated in this study from both survey and interview sessions. Of the 98 clinicians who had experiences with at least 5 patients enrolled in RPM and received the study’s survey, 30 responded (31% response rate). A nested sample of 5 clinicians (17% response rate) from this original group of 30 clinicians who completed the survey, also agreed to take part in 4 sessions (3 interviews; 1 focus group of 2 clinicians). Demographic data were not collected with the survey to maintain the anonymity of the clinician responders. However, the majority of clinicians who responded to the survey and interview sessions represented the Gastrointestinal, Thoracic, and Breast oncology services known to have the highest symptom burden. Of the clinicians who took part in the interview sessions, 3 clinicians had 5 or more years of organizational work experience, while 2 clinicians had 2 years or less of experience.

Overall, the clinicians reported that the RPM was easy to use and learn and that symptoms communicated through the RPM program were confirmed during follow-up phone calls. Suggested areas for improvement included communication and practice changes related to symptom management and the clinicians’ workload burden. Findings highlighting survey items with the highest frequency and proportion of survey responses are presented (Table 3).

The RPM was easy for clinicians to use but was influenced by a shift in basic assumptions with patients now instructed to report symptoms through portal messages rather than calling the medical offices (14/26, 54%).

The proportion of responses was fairly divided between agreement and disagreement. While 14 (52%) clinicians agreed that the RPM had usable functions and capabilities, 14 (52%) clinicians disagreed that the RPM was suitable for their patient care needs.

A total of 16 (59%) clinicians reported a disagreement with the notion that RPM improved their ability to care for their patients and, 12 (46%) clinicians reported that the RPM was useful in their practice.

Despite similar proportions of agreement and disagreement overall, a higher proportion of clinicians (13/26, 50%) disagreed with feeling comfortable communicating with patients about symptoms through portal messages.

Clinicians reported that devices were appropriately used for 12 (44%) patients and that high-risk symptoms were confirmed during follow-up phone calls to 20 (74%) patients. The responses were fairly split between agreement (10/27, 37%) and disagreement (12/27, 44%) on how well the devices provided severity levels. Additionally, 10 (38%) patients agreed that RPM helped them manage their patients.

A total of 21 (81%) clinicians reported that the RPM did not decrease their workload and 15 (56%) clinicians reported it did not improve job satisfaction. In addition, 14 (52%) of the clinicians were not satisfied with RPM, and 13 (50%) clinicians would not recommend the RPM to colleagues. The proportion of clinicians who neither agreed nor disagreed was similar to those who reported disagreement and was further explored in the interview sessions.

The interviews and focus group sessions consistently provided similar information about the RPM thus achieving thematic saturation [30]. All clinicians reported a limited understanding of the RPM during its implementation and suggested that ongoing educational modules and supporting technical support would enhance the RPM program for the clinician end users. Three major themes emerged from the interviews: clinician understanding and usability constraints, patient constraints, and suggestions for improvement. Subthemes were further explored (Table 4).

This study presented the perceptions and experiences of oncology clinicians when caring for their patients using RPM after hospital discharge. Initially, clinician survey responses highlighted concerns about their understanding, perceptions, and challenges when using RPM as well as its influence on their clinical workflows. These topics were further explored in interview sessions and revealed 3 themes: clinician understanding and usability constraints, patient constraints, and suggestions for improvement. Most clinicians found the RPM easy to use and learn, allowing them to confirm the patient’s reported symptoms during follow-up phone calls. They supported RPM’s value for both clinicians and patients as a care delivery method in oncology practice, aligning with findings from other organizational studies using ePROs [11,31-36], as well as the challenges and limitations of addressing symptom alerts after the patient transitions from an inpatient to the home setting [7,35,36].

This research further emphasized multiple challenges faced by clinicians during the initial RPM roll-out. A major concern was the practice change in clinical workflows whereby patients reported symptoms through electronic portals rather than contacting the medical offices by phone. Despite this change, clinicians felt their comfort level would improve with more experience, and as RPMs aligned with their workflows [7]. An opportunity to improve clinician confidence includes continuing educational tutorials from the initial RPM orientation and more technical and clinical support for both patients and clinicians. These efforts would facilitate aligning clinician workflows with clinical practice, thus leading to reduced stress and improved job satisfaction when caring for oncology patients using RPM [7].

Another challenge was patient constraints and their lack of understanding of RPM, which clinicians suggested might contribute to their uncertainty in digital symptom management reporting. Some patients completed surveys late in the clinic hours making it difficult to address symptoms. In many instances, clinicians reported that family members completed the surveys instead of the patients which raised concerns about the integrity of the symptom alerts. Further research could enhance the understanding of logistical limitations involving patients and caregivers with completing postdischarge symptom assessments at home. A similar study recommended that concerns revealed by patients and caregivers postdischarge could be included in future clinical outcomes [37]. Additionally, the timeframe for enrolling patients in RPM was another constraint identified by clinicians. Although older patients were comfortable using the RPM [38,39], enrollment at discharge was considered suboptimal. They suggested educating patients in a relaxed environment or office practices before hospitalizations would benefit patients and improve the enrollment process.

A major opportunity for RPM improvement involves addressing repetitive alerts from the daily ePRO assessments, also identified in other ePRO studies [23]. Clinicians verify all patient-reported symptom alerts. When prior symptoms are stable, these alerts could be averted to reduce their workflow burden [40,41]. To enhance effective communication with the patients, clinicians recommended adding context about symptom changes relative to the prior day’s ePRO assessment to improve the integrity of alerts.

The perspectives of all clinicians involved in this study expressed a need for additional resources to improve the effectiveness of RPM and their ability to clinically support it. The RPM was initially implemented using a core team of designated clinicians for addressing patient alerts. Studies of RPM cited the inclusion of core teams for this purpose which may be a preferred approach for clinicians in other studies [42]. However, clinicians in this research preferred a hybrid RPM program with primary care clinicians in the clinical practices addressing alerts on weekdays and centralized after-hours and on weekends. Clinicians proposed that leaders provide consistent status updates and education about RPM functionality through tutorial videos in an accessible location. Additional technological support could improve data collection and relieve clinicians of the burden of providing technical support for patients at home [41,42], which is not the best use of their skills [43].

The consistent reports from survey responses and interviewed clinicians strengthen the study’s findings and provide pragmatic recommendations for workflow redesign and enhancing access to care for oncology patients postdischarge [1,2,7,31], As a result of this study, selected modifications were implemented into practice to improve the program’s experience for both clinicians and patients. These results provide important foundational work for future pragmatic trials and implementation science to enhance the usability and value of RPM in oncology practice.

The sample was limited to only clinicians who had experiences with at least 5 patients enrolled in the RPM, but typical for case studies [13-18]. They expressed strong positive or negative experiences with the RPM which may have created a selection bias. The study’s design was multi-modal which could have included both interviewer and report biases. Clinicians were also employees of the organization and therefore specific demographics were not collected to ensure their anonymity. This study was conducted at one comprehensive cancer center, limiting the generalizability of results. However, the perspectives of clinician end users from oncology services with known symptom burdens can apply to similar RPM implementation initiatives in other organizations worldwide.

The National Cancer Institute calls for studies about cancer-related interventions, which also include program effectiveness from the perspectives of both patients and clinician end users [44]. This study contributes to the National Cancer Institute’s initiatives, demonstrating that RPM is a valuable method for communicating with clinical providers and managing patient symptoms during transitions of care from inpatient to the home setting.