This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.
During the COVID-19 pandemic, patient portals and their message platforms allowed remote access to health care. Utilization patterns in patient messaging during the COVID-19 crisis have not been studied thoroughly. In this work, we propose characterizing patients and their use of asynchronous virtual care for COVID-19 via a retrospective analysis of patient portal messages.
This study aimed to perform a retrospective analysis of portal messages to probe asynchronous patient responses to the COVID-19 crisis.
We collected over 2 million patient-generated messages (PGMs) at Mayo Clinic during February 1 to August 31, 2020. We analyzed descriptive statistics on PGMs related to COVID-19 and incorporated patients’ sociodemographic factors into the analysis. We analyzed the PGMs on COVID-19 in terms of COVID-19–related care (eg, COVID-19 symptom self-assessment and COVID-19 tests and results) and other health issues (eg, appointment cancellation, anxiety, and depression).
The majority of PGMs on COVID-19 pertained to COVID-19 symptom self-assessment (42.50%) and COVID-19 tests and results (30.84%). The PGMs related to COVID-19 symptom self-assessment and COVID-19 test results had dynamic patterns and peaks similar to the newly confirmed cases in the United States and in Minnesota. The trend of PGMs related to COVID-19 care plans paralleled trends in newly hospitalized cases and deaths. After an initial peak in March, the PGMs on issues such as appointment cancellations and anxiety regarding COVID-19 displayed a declining trend. The majority of message senders were 30-64 years old, married, female, White, or urban residents. This majority was an even higher proportion among patients who sent portal messages on COVID-19.
During the COVID-19 pandemic, patients increased portal messaging utilization to address health care issues about COVID-19 (in particular, symptom self-assessment and tests and results). Trends in message usage closely followed national trends in new cases and hospitalizations. There is a wide disparity for minority and rural populations in the use of PGMs for addressing the COVID-19 crisis.
The COVID-19 pandemic accelerated the adoption of digital and virtual patient care technology as sustainable and scalable parts of health systems. This includes the use of video, audio, and even Health Insurance Portability and Accountability Act–secure portals as a means for patients to remain connected with their providers [
Millions of nonurgent and non–COVID-19 medical encounters were postponed or cancelled by patients and health systems to reduce the risk of COVID-19 infection during in-person visits and prevent virus spread [
In the early stages of the global response, the public health strategy involved isolation for those infected or at risk, reducing social contact to slow the spread, and masking and hand washing to reduce infection risk. However, this unintentionally led to increased feelings of loneliness, reduced access social support, and worsening stress, anxiety, and depressive symptoms [
Studies on the use of telehealth and patient portal technologies have recently increased during the COVID-19 pandemic [
Mayo Clinic is a large multispecialty academic medical center focused on integrated patient care, education, and research. Mayo Clinic has three main medical sites in Minnesota, Florida, and Arizona and Mayo Clinic Health System (MCHS). MCHS is as a network of community-based medical services and consisted of more than 40 hospitals and clinics in Minnesota, Iowa, and Wisconsin in 2021. Mayo Clinic’s patient portal (Patient Online Services) has been operational since 2010 [
We analyzed the distribution of unique patients by age, gender, marriage, ethnicity, race, language, and residence. We excluded anonymous patients who sent messages for COVID-19 symptom self-assessment in the demographic analysis. We conducted a subanalysis comparing three different cohorts within our sample: patients who sent messages related to COVID-19 only (COVID-19 message senders), any patients who sent messages related to any topic (general message senders), and all patients who were active on the portal regardless of whether they composed messages (general patients). Statistical analysis involved chi-square goodness-of-fit tests.
We calculated the daily numbers of total PGMs related to COVID-19 between February 1 and August 31, 2020. The daily numbers would exhibit a week periodicity (typically with a maximum on Monday and a minimum on weekends). Because of this, we calculated their weekly smoothing averages (WSAs). The WSAs displayed a reduction around the holidays (Memorial Day on May 25, 2020, and Independence Day on July 4, 2020); hence, holidays were excluded from the analysis. The daily numbers and WSAs of the PGMs on COVID-19 can approximate the overall utilization of portal messages by the patients for addressing the COVID-19 crisis over time.
We analyzed the PGMs used for assessing COVID-19 symptoms and discussing COVID-19 care plans to understand the message utilization for COVID-19 diagnosis and treatment. We filtered the PGMs on COVID-19 symptom assessment by searching the relevant phrase, “COVID-19 (Coronavirus) Symptom Assessment,” as well as relevant keywords such as “test” and “result” for diagnostic tests and results and “care plan,” “monitoring,” and “interactive care” for care plans (see Table S1 in the
In addition, we examined other health care issues caused by COVID-19 reported in the portal messages to understand the impacts of the COVID-19 pandemic on health services and patients. We calculated the number of PGMs explicitly mentioning the phrase “due to COVID-19” and its synonyms to examine patient-reported health care issues caused by the COVID-19 pandemic. We also computed the number of PGMs on COVID-19, which discussed rescheduling or cancelling appointments, mental health, and suicidal ideation using relevant keywords (eg, “cancel” and “reschedule” for appointments, “anxiety” and “depressed” for mental health, and “suicide” for suicidal ideation) and their synonyms to quantify the impact of the COVID-19 pandemic on health services and patients (see Table S1 in the
We recruited 2 medical students for annotating the binary code for each studied topic in portal messages: whether a portal message is linked to COVID-19, COVID-19–related care, or other health care issues due to COVID-19. We randomly sampled 1800 portal messages for annotation, and the results are shown in Table S2 in
No patients were exposed to any intervention. We used the data from the Mayo Clinic Unified Data Platform for analysis. The study was approved by the Mayo Clinic institutional review board (19-002211).
We found that both COVID-19 and general message senders had a significantly different distribution compared to all patients active on the portal (
Demographic distribution of patients: COVID-19 message senders, general message senders, and general patients.
Patient demographics | COVID-19 message senders (N=102,470), % | General message senders (N=384,922), % | General patients (N=1,055,319), % | |
|
||||
|
<18 | 7.58 | 8.25 | 10.67 |
|
18-29 | 9.80 | 9.75 | 10.22 |
|
30-39 | 13.14 | 11.81 | 10.36 |
|
40-49 | 13.84 | 12.94 | 11.45 |
|
50-64 | 29.11 | 28.05 | 24.83 |
|
≥65 | 26.53 | 29.20 | 32.47 |
|
||||
|
Female | 60.97 | 58.54 | 54.37 |
|
Male | 39.03 | 41.46 | 45.63 |
|
||||
|
Married or has a life partner | 63.87 | 63.22 | 56.06 |
|
Not married or legally separated | 36.13 | 36.78 | 43.94 |
|
||||
|
Non–Hispanic or Latino | 95.84 | 95.82 | 95.28 |
|
Hispanic or Latino | 4.16 | 4.18 | 4.72 |
|
||||
|
White | 92.73 | 92.40 | 90.84 |
|
Asian | 2.44 | 2.42 | 2.28 |
|
Black or African American | 2.11 | 2.44 | 3.40 |
|
American Indian or Alaska Native | 0.37 | 0.37 | 0.41 |
|
Native Hawaiian or Pacific Islander | 0.09 | 0.10 | 0.12 |
|
Other | 2.26 | 2.28 | 2.94 |
|
||||
|
English | 99.09 | 99.06 | 97.56 |
|
Arabic | 0.12 | 0.14 | 0.25 |
|
Spanish | 0.34 | 0.37 | 1.03 |
|
Other | 0.45 | 0.44 | 1.16 |
|
||||
|
Urban | 74.77 | 70.41 | 61.02 |
|
Rural | 25.23 | 29.59 | 38.98 |
More than half (>55%) of patients were in the age groups of 50-64 years and ≥65 years. The proportion of patients in the age ranges of 30-39 years, 40-49 years, and 50-59 years was observed to have increased when looking at general portal users to general message senders to COVID-19–specific message senders. Meanwhile, the proportion of message senders in the age groups of <18 years and ≥65 years was the lowest in the COVID-19 message sender cohort.
More than half (>54%) of the patients were female and were married or had a life partner. The proportion of female patients (61% vs 54%) and married patients (64% vs 56%) also was highest in the COVID-19 message sender cohort when compared to the entire active portal user cohort. More than 90% of patients were of non–Hispanic or Latino ethnicity, White race, and spoke English. This proportion was also highest in the COVID-19 message sender cohort and lowest in the general portal cohort.
At least 61% of all patients assessed lived in the urban area. The percentage of urban patients increased to 70% among general message senders and 75% among COVID-19 message senders.
We illustrated the daily numbers and WSAs of PGMs on COVID-19 in
Daily numbers and weekly smoothing averages (WSAs) of patient-generated messages (PGMs) related to COVID-19.
Patient-generated messages (PGMs) related to COVID-19–related care and other health care issues caused by COVID-19.
Category | PGMs on COVID-19 (N=360,523), n (%) | |
|
||
|
Self-checker | 153,224 (42.50) |
|
e-Visit | 4619 (1.28) |
|
Tests and results | 111,183 (30.84) |
|
Care plan | 3844 (1.07) |
|
||
|
General issues | 13,333 (3.70) |
|
Postponement | 26,924 (7.47) |
|
Cancellation | 19,000 (5.27) |
|
Anxiety | 21,413 (5.94) |
|
Depression | 3673 (1.02) |
|
Suicidal ideation | 288 (0.08) |
Similar to the total PGMs on COVID-19 in
Daily numbers and weekly smoothing averages (WSAs) of patient-generated messages (PGMs) regarding COVID-19–related care (diagnosis and treatment): (A) COVID-19 symptom assessment via self-checker, (B) COVID-19 symptom assessment by providers via e-visits, (C) discussions regarding COVID-19 tests and results, and (D) care plans.
Daily numbers and weekly smoothing averages (WSAs) of patient-generated messages (PGMs) regarding COVID-19–related other health care issues: (A) general issues due to COVID-19, (B) postponement, (C) cancellation, (D) anxiety, (E) depression, and (F) suicidal ideation.
The COVID-19 pandemic and subsequent public health mitigation strategies, including stay-at-home orders and business restrictions, substantially impacted delivery of health care services. As the COVID-19 pandemic progressed in the United States, and specifically in Minnesota, newly confirmed cases had two peaks during February 1 and August 31, 2020, owing to the initial outbreak and late termination of stay-at-home orders. We observed similar dynamic patterns in PGMs on COVID-19, particularly, COVID-19 diagnosis and treatment, suggesting that patients actively used the portal messaging for addressing their concerns regarding the COVID-19 crisis [
Our findings also indicated that patients used the portal to report feelings of anxiety and depression about their existing medical conditions and potential contagious risks due to COVID-19 and seek support from their providers. Similar mental health concerns increased in the general population, according to a study of Twitter data, which showed an increased in tweets expressing mental health concerns due to infection risk and isolation strategies in the early stages of the pandemic [
After analyzing PGMs related to COVID-19 and unique patient senders from the Epic Clarity system (see Figure S2 in
Telehealth, including the use of patient portals, is transforming the delivery of health care [
There are several limitations to our study. First, the patient portal messages were collected at Mayo Clinic, a multispecialty academic medical center. The collected data might not be representative of different clinical settings or patient populations in other areas of the country. Second, keyword searching was carried out to identify patient portal messages associated with COVID-19, COVID-19–related care, and other health care issues due to COVID-19. Although the keyword sets cover a large number of relevant keywords, synonyms, and morphological variations, they may not be totally comprehensive; hence, bias could exist in our results. We are developing robust detection algorithms based on state-of-art deep learning techniques to accurately identify interesting health topics in portal messages during the pandemic. Finally, we investigated patient portal messages in the early stage of the COVID-19 pandemic. The analysis of patient portal messages in the following time or later stages of the pandemic is beyond the scope of this study but represents an important area for further exploration.
During the COVID-19 pandemic, patient portal utilization increased to address questions and concerns about the COVID-19 pandemic, revolving mainly around symptom self-assessment, tests, and results. The increased usage statistics for COVID-19 indicates the patient portal was a valuable web-based platform for patients to remotely discuss COVID-19 diagnosis and treatment as well as seek support for other health care issues impacted by the pandemic. The volume of PGMs on COVID-19–related care fluctuated as the pandemic developed. After initial increase in March, the PGMs regarding other health care issues such as appointment cancellations and anxiety about disease progression exhibited a declining trend. We observed differences in patient demographics between general portal users, general message senders, and COVID-19–specific message senders, mainly that the majority demographic took on a larger proportion of COVID-19 messages. There is still great potential to increase PGM engagement for minority populations and rural communities with regard to the COVID-19 pandemic. Time-series analysis of portal messages could offer us a timely surveillance of COVID-19 and its impacts on patients to improve patient-centered care related to the COVID-19 crisis.
Table S1 Keywords used to filter relevant portal messages; Table S2. Performance of Keyword searching methods; Table S3 Chi-squared goodness-of-fit test between patient role groups; Figure S1 Newly confirmed COVID-19 cases, hospitalized cases, and deaths in the US and Minnesota; Figure S2 Daily numbers and weekly smoothing averages (WSAs) of patient-generated messages (PGMs), unique patient senders, and messages per patient from the Epic Clarity database.
Mayo Clinic Health System
patient-generated message
weekly smoothing average
This research work was supported by the National Center for Advancing Translational Science of the National Institutes of Health (award U01TR002062). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
None declared.