Published on in Vol 8, No 3 (2021): Jul-Sep

Preprints (earlier versions) of this paper are available at, first published .
Factors Affecting Portal Usage Among Chronically Ill Patients During the COVID-19 Pandemic in the Netherlands: Cross-sectional Study

Factors Affecting Portal Usage Among Chronically Ill Patients During the COVID-19 Pandemic in the Netherlands: Cross-sectional Study

Factors Affecting Portal Usage Among Chronically Ill Patients During the COVID-19 Pandemic in the Netherlands: Cross-sectional Study

Original Paper

1Rotterdam School of Management, Erasmus University, Rotterdam, Netherlands

2Erasmus School of Health Policy & Management, Erasmus University, Rotterdam, Netherlands

Corresponding Author:

Qingxia Kong, BSc, MSc, PhD

Rotterdam School of Management

Erasmus University

Burgemeester Oudlaan 50

Rotterdam, 3062 PA


Phone: 31 10 408 2277

Fax:31 10 408 9014


Background: The COVID-19 pandemic has impacted the capacity of the regular health care system, which is reflected in limited access to nonurgent care for patients who are chronically ill in the Dutch health care system. Nevertheless, many of them still depend on health care assistance to manage their illnesses. Patient portals are used to provide continued health care (remotely) and offer self-management tools during COVID-19 and potentially after. However, little is known about the factors influencing portal use and users’ satisfaction among patients who are chronically ill during the COVID-19 pandemic.

Objective: This study aims to examine predictors of patient portal use among patients who are chronically ill, the willingness to recommend the portal to others, and the likelihood of future use among portal nonusers.

Methods: An online self-administered questionnaire was distributed among patients who are chronically ill via social media in May 2020. The questionnaire consisted of four parts: (1) demographics including age and hours of daily internet use; (2) physical health status including COVID-19 infection, perceived level of control, and hospital visits; (3) mental health status including depression and life satisfaction; and (4) portal use including response waiting time and awareness. Descriptive, correlation, univariate, and multivariate analyses were conducted to identify factors that affect portal use, users’ willingness to recommend, and nonusers’ likelihood of future portal use.

Results: A total of 652 patients responded, and 461 valid questionnaires were included. Among the 461 patients, 67% (n=307) were identified as patient portal users. Of the nonusers, 55% (85/154) reported not being aware of the existence of a patient portal at their hospital. Significant predictors of portal use include level of control (P=.04), hospital visit time (P=.03), depression scale (P=.03), and status of life satisfaction (P=.02). Among portal users, waiting time to get a response via the portal (P<.001) and maximum acceptable waiting time (P<.001) were the strongest predictors for willingness to recommend the portal; among nonusers, the model predicted that those who were not aware of patient portals (P<.001) and were willing to wait moderately long (P<.001) were most likely to use the portal in the future.

Conclusions: This study provides insights into factors that influence portal use and willingness to recommend, based on which health care providers can improve the adoption of patient portals and their services. It suggests that health care providers should leverage efficient operations management to improve responsiveness and reduce waiting time to enhance user satisfaction and willingness to recommend use. Health care organizations need to increase portal awareness among nonusers and train their patients to increase both use and longer adoption of patient portals. Factors including depression and life satisfaction can influence portal use; therefore, future studies on determinants of portal use and nonuse in this specific population are needed.

JMIR Hum Factors 2021;8(3):e26003



On January 30, 2020, COVID-19 was officially declared a pandemic by the World Health Organization [1]. As a result of the virus outbreak, the Netherlands, along with other countries, announced a lockdown. This lockdown, called an intelligent lockdown, entailed that people were encouraged (not forced) to stay inside as much as possible, social gatherings with more than three people were prohibited, and many (nonessential) businesses were temporarily closed [2]. The primary purpose of this intelligent lockdown was to prevent peak loads of patients requiring intensive care [3]. The pandemic’s consequences were a massive burden on the Dutch health care system, particularly in the initial period of the outbreak (March 2020). Intensive care units were struggling with allocating their capacity, causing patients to be distributed over various hospitals throughout the Netherlands [4]. Meanwhile, regular health care was disrupted due to the COVID-19 outbreak. To alleviate the pressure of health care professionals and to prevent them and nonurgent patients from infecting each other [5], nonurgent patients’ appointments were canceled, postponed, or moved online [6,7]. Several experts and health care professionals subsequently proposed eHealth as a solution for the continuation of care for patients who are chronically ill [8,9].

According to a study supported by the Dutch government, approximately 5.3 million Dutch patients have one or more chronic illnesses. This number is expected to rise to 7 million by 2030 [10]. In Europe, about 70% to 80% of the total health care budget is spent on treating and preventing chronic disease [11], which indicates that chronic illness is a common issue with an enormous financial burden. Two critical elements of chronic care are frequent contact with their care providers and self-management (eg, adapting to their condition and learning to deal with their disease) [12]. Therefore, some still rely on regular nonurgent health care and need assistance during the COVID-19 pandemic to keep their illness under control. Limited access to care, in addition to the fear of contracting the virus, getting sick, or even passing away, could potentially lead to diminished (perceived) physical and mental health outcomes for this group of patients [13,14]. Indeed, previous studies have shown that people with chronic diseases are more prone to anxiety and depression than those without [15,16]. Although care for patients with COVID-19 requires the most attention during this crisis, it is crucial to continue to provide patients who are chronically ill the care they need, including offering self-management tools, monitoring, controlling, and disease treatment. It will ultimately reduce the risk of emergency care and hospital admission, and prevent long-term complications in these patients [17].

During the COVID-19 pandemic, eHealth has been suggested as a valuable solution to provide care to patients who are chronically ill, enabling self-management of chronic conditions and providing care remotely and safely [18,19]. The Dutch government has compiled a subsidy program (VIPP) to accelerate the implementation of eHealth solutions in specialized medical care organizations throughout the country [20]. By 2019, 60 out of 73 Dutch hospitals offered an eHealth solution [21]. The solution is essentially a platform called a patient portal. In these patient portals, patients are, among other things, able to investigate their electronic health records, directly message their health care practitioner, and view their laboratory results along with personal details. Each hospital was allowed to decide on the functionalities implemented in its patient portal. Despite the VIPP implementation, several reports provide evidence on the lack of patient engagement, reflected in a large portion of nonusers [22].

Therefore, it is crucial to understand which factors influence portal use for patients who are chronically ill, users’ satisfaction, and nonusers’ likelihood of future use to promote the adoption of patient portals and retain current users. A retrospective cohort study among the adult patient population found that those who are younger, are White, have commercial insurance, and have higher annual income are more likely to be portal users [23]. Another cross-sectional survey also found that age and income are significant predictors of portal adoption [24]. A cross-sectional survey among adult patients of a university hospital revealed that being chronically ill and having higher eHealth literacy were the best predictors for portal use [22]. However, it remains unknown which factors influence portal use among the patient group of interest—patients who are chronically ill—and, in particular, whether and how perceived physical and mental health conditions play a role during the COVID-19 pandemic. Besides, several papers published after the outbreak of COVID-19 studied patient satisfaction on patient portals or telehealth [25]. However, those studies are mainly descriptive (ie, they survey how many patients are satisfied with their experience rather than predicting or investigating the causal relationship). This study contributes to understanding which factors predicted portal use, portal users’ satisfaction, and portal nonusers’ likelihood of future adoption among patients who are chronically ill during the COVID-19 pandemic.

The research questions of this study are what factors affect patient portal use among patients who are chronically ill during the COVID-19 pandemic in the Netherlands and what factors affect portal users’ willingness to recommend and nonusers’ likelihood of using patient portals during the COVID-19 pandemic in the Netherlands?

Study Design and Procedure

A cross-sectional study was designed using an online self-administered questionnaire (the survey is available upon request). The survey was written in English and then translated into Dutch and verified by a person proficient in Dutch. The questionnaire was distributed throughout several Facebook groups aimed at (peer) support and providing information for patients who are chronically ill in May 2020.

After displaying the introduction of the questionnaire, informed consent was obtained electronically, before actual enrollment. It was explicitly stated that participation was voluntary, and participants could withdraw at any time without any consequences. Moreover, complete anonymity of the response was ensured.

During the period of data collection, the number of Dutch people who tested positive for COVID-19 exceeded 40,000, over 11,000 people were hospitalized, and almost 6000 deaths related to COVID-19 had been reported in a population of 17 million inhabitants [26]. When distributing the questionnaire, the national intelligent lockdown had been active for approximately 1.5 months. As compensation for the time spent on the survey, online gift codes were distributed through a raffle. To ensure good quality responses, some survey items were programmed to be restricted in range so that incorrect inputs were not allowed.

This study followed the Institutional Review Board (IRB) for the Protection of Human Subjects Guidelines. All procedures in this study were approved by the IRB (2020/04/24-61392qko) prior to its initiation.


Since the study focused on patient portals as implemented by Dutch hospitals, the targeted population for our study was patients who are chronically ill and residing in the Netherlands. Inclusion criteria were as follows: patients aged between 18 and 65 years, having at least one chronic illness, and having spent more than 2 minutes completing the questionnaire. Questionnaires that were not completed were removed from the final data set.


Demographic Characteristics

Demographic variables included participant’s gender (male, female, or other), age, highest educational level completed, main occupation, yearly income, chronic illness or illnesses, hours of daily internet use, and portal use (yes or no).

Physical Health Status and Hospital Visits

Physical health status was assessed using four categories: (1) COVID-19 status, (2) level of control over chronic illness, (3) lifestyle and exercise, and (4) perceived health. COVID-19 status was assessed by inquiring about the prevalence of any COVID-19 symptoms over the last 2 weeks (yes, no, or unsure), COVID-19 testing (yes or no), and COVID-19 infection (yes, no, or unsure). Level of control over chronic illness was assessed using a single 5-point item, asking people to rate their current level of control over their chronic illness (totally in control to not at all in control). Lifestyle was assessed using a common measure of lifestyle and activity [27,28]. Exercise was measured by the frequency of exercise in the last 2 weeks. Perceived health was measured using the Self-Rated Health measure, a widely used, single-item measure of self-perceived health status [29]. The item consisted of one question (“In general, would you say your health is:”) with five answer options between 1 (excellent) and 5 (poor) [30]. Finally, patients’ frequency of hospital visits and their durations were also measured.

Mental Health

Mental health was assessed using questions about both depression severity and life satisfaction. Depression severity was measured using the Patient Health Questionnaire 9 assessment scale, which is generally used to aid clinicians in diagnosing, monitoring, and treating depressive symptoms and their severity [31]. Patients score nine different items on a scale of 0 (not at all) to 3 (nearly every day). The scores are then summed up to achieve a final score, which can be assessed by the clinician or researcher over a few cut-off categories. The categories are 0 to 4, 5 to 9, 10 to 14, 15 to 19, and 20 to 27, in sequence of increasing depression severity [32]. General well-being was assessed by examining participants’ satisfaction with life, using the Satisfaction with Life Scale [33]. The tool allows participants to self-report their opinions regarding the satisfaction they experience with their own lives. The scale contains five items, and participants report their answers over a 7-point Likert scale (1, strongly disagree, to 7, strongly agree). After the assessment, scores are summed up to arrive at a final score. The outcomes are categorized as 5 to 9 (extremely dissatisfied), 10 to 14 (dissatisfied), 15 to 19 (slightly dissatisfied), 20 (neutral), 21 to 25 (slightly satisfied), 26 to 30 (satisfied), and 31 to 35 (extremely satisfied).

Patient Portal Use

Participants were identified as portal users if they selected yes to the question “Have you ever used a patient portal?” Among portal users, their portal use was assessed by frequency of use, time duration of use, and waiting time. The time of use measures the average duration each time a patient uses a portal, and frequency of use indicates how often a patient uses a portal. They are two dimensions of patient engagement with the portal. Furthermore, the waiting time from sending a request until receiving a reply was recorded. For patient portal users (ie, people who have used a portal before), their usual and maximum acceptable waiting time was asked. In contrast, for nonusers, only the maximum acceptable waiting time was recorded. Lastly, portal users’ willingness to recommend the portal to others—a strong indicator of customer loyalty and predictor for growth [34]—was measured by a single question: “Would you recommend the patient portal to others?” Portal nonusers were asked to input a percentage value (from 0 to 100) to answer the question “What is your likelihood of using such a portal?” to measure their likelihood of future portal use.

Data Analysis

Descriptive analysis was performed to gain insight into the patient population, portal users, and nonusers regarding their demographics, physical health status, hospital visits, and mental health status. Additionally, chi-square tests (for categorical variables) and Welch t tests (for numerical variables) were performed to compare the characteristics between portal users and nonusers. Next, univariate analyses were performed for the dependent variable (portal use) to detect its possible predictors. Variables with P<.20 in the univariate regression were consequently included in the multiple regression analysis after considering the correlation between variables (using statistical analysis and expert opinion). A stepwise backward elimination was then applied to reduce the number of independent variables and obtain the final multiple regression model. This approach allowed thorough exploration and testing of possible predictors to arrive at a final model [35]. Similar procedures were applied to the other two dependent variables (willingness to recommend among portal users and likelihood of use among portal nonusers) to get the final multiple regression models. All analyses were performed using RStudio (version 4.0.2; RStudio, PBC).

Descriptive Analysis

A total of 652 respondents started the questionnaire, whereby 461 respondents completed it successfully. Only completed questionnaires were used in the final data set for analysis. Of all the participants, 307 (66.6%) reported to have used a patient portal, and 154 (33.4%) reported that they had not used a patient portal until the moment the survey was conducted.

Demographics of all participants, portal users, and nonusers are displayed in Table 1. From our sample (N=461), 94 (20.4%) were male, 365 (79.2%) were female, and 2 (0.4%) individuals identified as other. The mean age of the sample was 42.9 (SD 13.0) years. The number of participants that reported having a single chronic disease was 302 (65.5%), and 159 (34.5%) reported having multiple chronic diseases. Significant differences were noted in the mean age (P=.008) and main occupations (P=.03) between portal users and nonusers.

Table 2 displays the physical health status, hospital visits, and mental health status of all participants, portal users, and nonusers. The majority of the 461 patients reported to have their illness “a little bit” (n=113, 24.5%) to “moderately in control” (n=229, 49.7%), 66 (14.3%) reported to have total control, and 46 (10.0%) reported to have no control over their illness at all. Furthermore, few patients (n=30, 6.5%) reported that their perceived health was very good or excellent, while the majority reported good (n=127, 27.6%), fair (n=177, 38.4%), or poor (n=127, 27.6%) perceived health. Moreover, the majority of respondents reported spending 0.5 hours to 1 hour (n=151, 32.8%) and 1 hour to 2 hours (n=166, 36.0%) each time they visit a hospital (including travel time); 70 (15.2%) respondents spent less than half an hour, while only 19 (4.1%) spent more than 3 hours. Furthermore, only 18 (3.9%) participants reported having or having had COVID-19, 130 (28.2%) were uncertain, and 313 (67.9%) reported that they never had COVID-19. About mental health, most participants reported having no (n=172, 37.3%) to mild forms of (n=158, 34.3%) depression, while only 9.8% (n=45) reported having moderately severe or severe depression. Relative to life satisfaction, 146 (31.7%) and 117 (25.4%) of the participants were satisfied and slightly satisfied with their lives, respectively. Moreover, 27 (5.9%) and 28 (6.1%) were extremely satisfied and extremely dissatisfied with their lives, respectively. Among the measured characteristics, level of control (P=.005), average time of hospital visits (P=.04), depression (P=.02), and life satisfaction (P=.005) were significantly different between portal users and nonusers.

Table 1. Demographics of all the participants, portal nonusers, and users during the COVID-19 pandemic in the Dutch population of patients who are chronically ill.
DemographicsTotal (N=461)Nonusers (n=154)Users (n=307)P value
Gender, n (%).60

Female365 (79.2)122 (79.2)243 (79.2)

Male94 (20.4)32 (20.8)62 (20.2)

Other2 (0.4)0 (0.0)2 (0.7)
Age (years), mean (SD)42.9 (13.0)45.1 (12.5)41.8 (13.1).008
Highest education, n (%).06

Primary school13 (2.8)8 (5.2)5 (1.6)

Secondary/high school82 (17.8)29 (18.8)53 (17.3)

MBOa,b completed201 (43.6)73 (47.4)128 (41.7)

HBOc,d or university degree155 (33.6)41 (26.6)114 (37.1)

Graduate degree10 (2.2)3 (1.9)7 (2.3)
Main occupation, n (%).03

Self-employed45 (9.8)11 (7.1)34 (11.1)

Employee242 (52.5)73 (47.4)169 (55.0)

Student35 (7.6)9 (5.8)26 (8.5)

Unemployed117 (25.4)50 (32.5)67 (21.8)

Retired22 (4.8)11 (7.1)11 (3.6)
Yearly income (€e), n (%).29

0-20,000171 (37.1)61 (39.6)110 (35.8)

20,001-30,000103 (22.3)38 (24.7)65 (21.2)

30,001-40,000120 (26.0)39 (25.3)81 (26.4)

≥40,00167 (14.5)16 (10.4)51 (16.6)
Chronic illness, n (%).52

Single chronic illness302 (65.5)104 (33.9)198 (64.5)

Multiple chronic illnesses159 (34.5)50 (16.3)109 (35.5)
Daily internet use (hours), mean (SD)5.7 (4.3)5.4 (4.4)5.8 (4.3).60

aMBO: Middelbaar beroepsonderwijs.

bEnglish translation: secondary vocational education. It is oriented toward vocational training and is equivalent to a junior college education.

cHBO: Hoger beroepsonderwijs.

dEnglish translation: higher professional education. It is oriented toward higher learning and professional training, and is the equivalent to a college education in the United States.

eA currency exchange rate of €1=US $1.18 is applicable.

Table 2. Physical health status, hospital visits, and mental health status of all the participants, portal nonusers, and users during the COVID-19 pandemic in the Dutch population of patients who are chronically ill.
VariablesTotal (N=461), n (%)Nonusers (n=154), n (%)Users (n=307), n (%)P value
COVID-19 infection.72

Yes18 (3.9)101 (65.6)212 (69.1)

Not sure130 (28.2)46 (29.9)84 (27.4)

No313 (67.9)7 (4.5)11 (3.6)
Level of control.005

Totally66 (14.3)28 (18.2)38 (12.4)

Moderately229 (49.7)63 (40.9)166 (54.1)

A little bit113 (24.5)41 (26.6)72 (23.5)

Not at all46 (10.0)16 (10.4)30 (9.8)

I don’t know7 (1.5)6 (3.9)1 (0.3)
Perceived health (SRHa).31

Excellent4 (0.9)2 (1.3)2 (0.7)

Very good26 (5.6)8 (5.2)18 (5.9)

Good127 (27.5)36 (23.4)91 (29.6)

Fair177 (38.4)57 (37.0)120 (39.1)

Poor127 (27.5)51 (33.1)76 (24.8)
Average time of hospital visit (hours).04

<0.570 (15.2)32 (20.8)38 12.4)

0.5-1151 (32.8)42 (27.3)109 (35.5)

1-2166 (36.0)56 (36.4)110 (35.8)

2-355 (11.9)21 (13.6)34 (11.1)

>319 (4.1)3 (1.9)16 (5.2)
Depression (PHQ-9b).02

None172 (37.3)56 (36.4)116 (37.8)

Mild158 (34.3)49 (31.8)109 (35.5)

Moderate86 (18.7)25 (16.2)61 (19.9)

Moderately severe33 (7.2)15 (9.7)18 (5.9)

Severe12 (2.6)9 (5.8)3 (1.0)
Life satisfaction.005

Extremely satisfied27 (5.9)6 (3.9)21 (6.8)

Satisfied146 (31.7)44 (28.6)102 (33.2)

Slightly satisfied117 (25.4)28 (18.2)89 (29.0)

Neutral21 (4.6)10 (6.5)11 (3.6)

Slightly dissatisfied70 (15.2)27 (17.5)43 (14.0)

Dissatisfied52 (11.3)24 (15.6)28 (9.1)

Extremely dissatisfied28 (6.1)15 (9.7)13 (4.2)

aSRH: Self-Rated Health.

bPHQ-9: Patient Health Questionnaire 9.

Table 3 reports the frequency of portal use before and after the lockdown. An increase in the frequency of portal use has been observed after the lockdown as compared to before, whereby the relative difference was 500%, 221.1%, and 8.3% in daily, weekly, and monthly use, respectively. After the lockdown, 67 (21.8%) reported daily to weekly use, and 106 (34.5%) have used the patient portal monthly. Among all the portal users, the most common use times were “5 minutes or less” (n=121, 39.4%) and “5-10 minute” (n=124, 40.4%), while 62 (20.2%) of them reported using the portals for more than 10 minutes.

In relation to the maximum acceptable waiting time, nonusers reported a lower maximum acceptable waiting time than users. Among the users, 78 (28.4%) reported a longer actual waiting time than they deem acceptable. Finally, among portal users, 257 (83.7%) would likely recommend portals to others, and among nonusers, the average likelihood of future use (ranging from 0% to 100%) was 53.6% (SD 33.3%).

Table 3. Descriptive of patient portal use before and after lockdown (n=307).a
Portal useFrequency of use before lockdown, nFrequency of use after initiation of lockdown, nRelative difference (%)
(Almost) daily16500
Less than monthly231134–42.0

aRelative comparison between periods translated according to relative frequency of use (period before the intelligent lockdown had a much larger timespan than the period after initiation of the intelligent lockdown and thus included portal nonusers).

Multiple Regression Analysis

To investigate which combinations of the different predictors could best explain the variance in portal use versus nonuse and portal users’ willingness to recommend and portal nonusers’ likelihood of future use, three separate regression models were constructed after performing univariate regression analysis and considering possible correlations. In the first analysis (model 1), a logistic regression was performed to investigate the association between portal use and the included variables after the first steps, which were age, hospital visit time, level of control, depression, and life satisfaction. In the second analysis (model 2), a logistic regression was performed to study the relationship between portal users’ willingness to recommend and the variables average number of hours spent on the internet daily, the frequency of portal use after the COVID-19 lockdown in March 2020, waiting time for portal response, and maximum acceptable time to wait. In the third analysis (model 3), a multiple regression analysis was conducted between portal nonusers’ likelihood of use and age, income, maximum acceptable waiting time, and their awareness of patient portals’ existence as candidate variables. The results of the regression analysis are displayed in Table 4.

Regression results of model 1 showed that shorter hospital visit times (“less than half an hour”) predict less portal use (β=–.725; P=.03) compared to longer visit times. Compared to “totally under control,” moderate level of control predicts a higher chance (β=.629; P=.04) of portal use. Two mental health conditions were shown to significantly affect participants’ portal use. Participants with severe depression (β=–1.652; P=.03) and life dissatisfaction or extreme life dissatisfaction (β=–.844; P=.02) were found to be less likely to use patient portals. Furthermore, age demonstrates a small yet nonsignificant impact on portal use, whereby older age negatively affects portal use (β=–.015; P=.08).

Among portal users, the logistic regression results from model 2 showed that actual waiting time and maximum acceptable waiting time were the strongest predictors of users’ willingness to recommend. Participants whose average waiting time was between 1 to 2 days (β=–2.081; P<.001) or greater than 2 days (β=–1.784; P<.001) were less likely to recommend the portal system to others, compared to those who received responses via portal systems within 24 hours. Participants who reported a moderate maximum waiting time (1-2 days) were more likely to recommend portal systems (β=2.292; P<.001).

For portal nonusers (model 3), awareness of the portal existence was the strongest predictor besides maximum acceptable waiting time. Among nonusers, 85 (55.2%) reported being unaware of the existence of a patient portal at their hospital. Participants that were unaware of the existence of portal systems were 25.9% (P<.001) more likely to use portal systems, compared to those that already knew of their existence before the time of the survey. Participants who had a moderate maximum acceptable waiting time (12-24 hours) were 21.2% (P<.001) more likely to use portal systems in the future. Furthermore, middle income class participants (€30,001 [US $35,440.20] to €40,000 [US $47,252.00]) were 15.3% (P=.01) more likely to use portal systems compared to low income class participants (<€20,001 [US $23,627.20]), and older-aged participants also showed a slightly lower likelihood (β=–.003; P=.10) of use.

Table 4. Results of the multiple regression model, indicating the significant predictors.
VariablesModel 1 portal use
(all participants)
Model 2 recommendation (portal users)Model 3 likelihood of using (portal nonusers)

EstimatesP valueEstimatesP valueEstimatesP value
Income (€b; reference: 0-20,000)



Daily internet hoursN/AN/A0.093.07N/AN/A
Hospital visit time (hours; reference: 0.5-1)




Level of control (reference: totally)


Little bit0.328.36N/AN/AN/AN/A

Not at all0.823.08N/AN/AN/AN/A

I don’t know–1.825.11N/AN/AN/AN/A
Depression scale (reference: mild)



Moderately severe–0.321.45N/AN/AN/AN/A

Life satisfaction scale (reference: satisfied or more)

Slightly satisfied0.116.69N/AN/AN/AN/A


Slightly dissatisfied–0.589.08N/AN/AN/AN/A

Dissatisfied or less–0.844.02N/AN/AN/AN/A
Portal use COVID-19 (reference: daily)

Weekly or moreN/AN/A1.269.11N/AN/A

3-5 timesN/AN/A2.050.06.N/AN/A

1-2 timesN/AN/A0.124.74N/AN/A
Waiting time (reference: less than 24 hours)

1-2 daysN/AN/A–2.081<.001N/AN/A

>2 daysN/AN/A–1.784<.001N/AN/A

Never triedN/AN/A–0.911.16N/AN/A

No possibilityN/AN/A–0.681.35N/AN/A
Maximum acceptable waiting time (reference: <12 hours)

12-24 hoursN/AN/A1.187.0060.212<.001

1-2 daysN/AN/A2.292<.0010.192.006

>2 daysN/AN/A1.502.030.181.05
Awareness (reference: yes)N/AN/AN/AN/A0.259<.001
Participants, n461N/A307N/A154N/A
Akaike information criterion570.87N/A246.96N/AN/AN/A
Adjusted R2N/AN/AN/AN/A0.2904N/A

aN/A: not applicable.

bA currency exchange rate of €1=US $1.18 is applicable.

Main Findings and Comparison With Other Studies

Although the societal and health impacts of the COVID-19 pandemic have been present for nearly a year, there is no evidence on factors that affect patient portal adoption among patients who are chronically ill during the COVID-19 pandemic. Moreover, little research has been done on what influences users’ willingness to recommend and nonusers’ likelihood of using patient portals during the COVID-19 pandemic. Our findings portray some interesting insights for portal service providers and health care professionals.

In the participant population under study, we found that almost 67% (307/461) of participants were portal users, which is much higher than for general patient populations reported [36-38]. For example, Griffin et al [36] found in their study that 83.4% of patients were nonusers of the UNC Chart patient portal among a general patient population. It could be attributed to the difference in the study population and the impact of the COVID-19 pandemic. Ancker et al [39] found that patients with chronic illness were more likely to use a patient portal. Table 3 shows that both the number of portal users and frequency of use have increased significantly after the lockdown initiation in spring 2020. We found that participants whose level of control was moderate had a higher likelihood of using portal systems than participants with total control. This may be attributed to participants’ perception whereby they deem a portal as unnecessary when their health is well managed and under control.

Besides, we found that participants with shorter visit times to a hospital have a reduced likelihood of portal use compared to those with longer visit times. As reported in many other studies [40-42], savings on travel time and cost are among the major benefits of eHealth. This result suggests that the convenience of physical visits most likely reduces remote visits using patient portals. Furthermore, participants with severe depression and lower life satisfaction tend to use patient portals less. Mental health problems likely deter patients from using portal systems. This result coincides with the observation that patients with chronic anxiety and depression are less likely to be intense eHealth users [37]. Future studies should focus on determinates of portal use and nonuse in this specific population.

Our results show that older age may negatively affect portal use. It is in line with a recent study in the older population on the intention to use medical applications. Feelings of having control, service availability, perceived ease of use and usefulness, and attitude toward the medical application affect the intention to use in older adults, which may be attributed to anxiety triggered by technology use, lack of privacy, or trust [43]. Another study also argues that this is probably because older people often lack the infrastructure, knowledge, and skills needed to use eHealth programs [44]. Future studies are required to investigate determinants of portal use and nonuse in the older adult population.

Willingness to recommend patient portal systems was also investigated. No less than 83.7% reported willingness to recommend the portal to family and friends, which suggests that most users were satisfied and loyal with their hospital’s patient portal system [34]. The average waiting time to receive a response was a strong predictor for users’ positive experience using portal systems. Approximately 29% of patients reported receiving responses within 2 hours of a request, which is considered rapid. Numerous studies in the appointment scheduling area have shown the importance of managing waiting time in health care management [45-47]. Marketing research has shown that waiting time is a crucial determinant of customer satisfaction and loyalty [48]. Nonusers seem to expect faster response rates from patient portals than users. Palawatta [49] demonstrated that if nonusers perceive the response rate is longer than their perceived acceptable waiting time, they will feel less satisfied and, therefore, less inclined to try the portal system. Users, if they experience disconfirmation in waiting time and maximum acceptable waiting time, are likely to be less satisfied and therefore less likely to remain committed to using the portal system. These are essential insights for health care practitioners and managers to leverage operational efficiencies such as appointment scheduling and resource allocation.

Among the nonuser group, the majority (85/154, 55%) reported not being aware of a patient portal system at their hospital. Awareness of portal systems was found to be the largest predictor for future use in our study. It seems that many patients do not use portals partly due to unawareness of their existence. This result is in line with Griffin et al [36], who found that patients often did not use patient portals simply because they were unaware of their existence. This result suggests that enhancing the awareness of portal systems is the first step for health care organizations to take to increase portal use.

Limitations and Future Research

There are some limitations bound to this study. First, the survey is cross-sectional, making it impossible to make causal claims, limiting the study to predictions only. Furthermore, the study relies on self-reported data on portal use. This is because when the study was implemented, we did not have access to the actual use data, such as log data of portal users. Besides, this survey focused on both portal users and nonusers to study factors that influence portal use and future use of nonusers. This, for example, cannot be replicated by merely approaching the actual users. However, it would be more insightful to use real use data (eg, log data retrieved from the portal) to establish the length and frequency of use. We suggest this as a future study when access to portal data is possible.

Second, the study invited participants via social media (Facebook peer support group) to complete the self-administered questionnaire. On one hand, sampling from Facebook support groups has apparent benefits, such as convenience and its focus on the targeted population. On the other hand, it also has a few known biases [50,51]. For example, Facebook excludes people who have a lower eHealth literacy, one important predictor of portal use among adult patients [22]. Besides, not everybody uses Facebook, especially older people. Although this problem is partly compensated by focusing on the age group 18 to 65 years, our results might overestimate the proportion of portal users among the total population. This partly explains why the ratio of portal users is higher than reported in many other studies. Little is known about the characteristics of people who do not use technology and why they do not use the portal. We suggest that future studies should focus on older people and people with less eHealth literacy.

Moreover, more females than males participated in this study. According to Smith [52], females are more likely to respond to (online) surveys than males. The authors proposed different reasons that could be grounded to this observation, including behavioral differences between males and females in relation to the internet or inherent internal feelings. Another study [53] found similar results (70% female response).

Finally, it is important to see which functionalities users use and the respective frequency to understand the perceived value of these functionalities to patients. This will potentially improve the frequency of use and tailoring portal systems according to the needs of patient. Future research could build on our results, aimed at further investigation of the use dimension of patient portals.


Individuals that have spent less time on physical hospital visits, whose health is moderately under control or with severe depression or lower life satisfaction are less likely to use patient portal systems. Among users, short waiting time was the most important predictor for satisfaction of portal use, and among nonusers, awareness was the most important predictor of future portal system use. These findings provide insights for health care providers on how to promote patient portal use and improve user satisfaction.


We would like to express our gratitude to those who have helped to establish this study. First, we would like to thank all of the participants for their contribution. Second, we would like to thank the Erasmus Research Institute of Management for funding support. Furthermore, we would like to thank Dr Stephen Zhang for providing professional help on the survey design.

Conflicts of Interest

None declared.


  1. Ho CS, Chee CY, Ho RC. Mental health strategies to combat the psychological impact of COVID-19 beyond paranoia and panic. Ann Acad Med Singap 2020 Mar 16;49(3):155-160. [Medline]
  2. Coronavirus COVID-19. Government of the Netherlands. 2020.   URL: [accessed 2020-05-20]
  3. Holdert M. Is onze intelligente lockdown wel zo intelligent? NOS.   URL: [accessed 2020-05-20]
  4. Intensive care steeds voller, opnieuw coronapatiënten uit Brabant verplaatst. Hart van Nederland.   URL: [accessed 2020-04-15]
  5. Ng K, Poon BH, Kiat Puar TH, Shan Quah JL, Loh WJ, Wong YJ, et al. COVID-19 and the risk to health care workers: a case report. Ann Intern Med 2020 Jun 02;172(11):766-767 [FREE Full text] [CrossRef] [Medline]
  6. The Lancet. COVID-19: protecting health-care workers. Lancet 2020 Mar 21;395(10228):922 [FREE Full text] [CrossRef] [Medline]
  7. COVID-19: virtual health care services ensuring continuity of care. Canadian Psychological Association.   URL: [accessed 2020-07-12]
  8. Hollander JE, Carr BG. Virtually Perfect? Telemedicine for Covid-19. N Engl J Med 2020 Apr 30;382(18):1679-1681. [CrossRef] [Medline]
  9. Portnoy J, Waller M, Elliott T. Telemedicine in the era of COVID-19. J Allergy Clin Immunol Pract 2020 May;8(5):1489-1491 [FREE Full text] [CrossRef] [Medline]
  10. Hoeymans N, Harbers MM, Hilderink HBM. [Living longer, with more disease and less disability; trends in public health 2000-2030]. Ned Tijdschr Geneeskd 2014;158:A7819. [Medline]
  11. Chronic Diseases Summit (3-4 April 2014). European Commission. 2014.   URL: [accessed 2020-11-13]
  12. Agarwal R, Anderson C, Zarate J, Ward C. If we offer it, will they accept? Factors affecting patient use intentions of personal health records and secure messaging. J Med Internet Res 2013 Feb 26;15(2):e43 [FREE Full text] [CrossRef] [Medline]
  13. Hajure M, Tariku M, Mohammedhussein M, Dule A. Depression, anxiety and associated factors among chronic medical patients amid COVID-19 pandemic in Mettu Karl Referral Hospital, Mettu, Ethiopia, 2020. Neuropsychiatr Dis Treat 2020;16:2511-2518. [CrossRef] [Medline]
  14. Salari N, Hosseinian-Far A, Jalali R, Vaisi-Raygani A, Rasoulpoor S, Mohammadi M, et al. Prevalence of stress, anxiety, depression among the general population during the COVID-19 pandemic: a systematic review and meta-analysis. Global Health 2020 Jul 06;16(1):57 [FREE Full text] [CrossRef] [Medline]
  15. Roberts RE, Kaplan GA, Shema SJ, Strawbridge WJ. Are the obese at greater risk for depression? Am J Epidemiol 2000 Jul 15;152(2):163-170. [CrossRef] [Medline]
  16. Surís JC, Parera N, Puig C. Chronic illness and emotional distress in adolescence. J Adolesc Health 1996 Aug;19(2):153-156. [CrossRef] [Medline]
  17. Liu N, Huang R, Baldacchino T, Sud A, Sud K, Khadra M, et al. Telehealth for noncritical patients with chronic diseases during the COVID-19 pandemic. J Med Internet Res 2020 Aug 07;22(8):e19493 [FREE Full text] [CrossRef] [Medline]
  18. Al-Ruzzieh M, Ayaad O, Qaddumi B. The role of e-health in improving control and management of COVID 19 outbreak: current perspectives. Int J Adolesc Med Health 2020 Aug 31:1. [CrossRef] [Medline]
  19. Alromaihi D, Alamuddin N, George S. Sustainable diabetes care services during COVID-19 pandemic. Diabetes Res Clin Pract 2020 Aug;166:108298 [FREE Full text] [CrossRef] [Medline]
  20. VIPP is het Versnellingsprogramma informatie-uitwisseling patiënt en professional. VIPP. 2019.   URL: [accessed 2020-11-15]
  21. Patiëntportaal actief bij 60 van 73 ziekenhuizen. ICT&health.   URL: [accessed 2021-02-07]
  22. Hoogenbosch B, Postma J, de Man-van Ginkel JM, Tiemessen NA, van Delden JJ, van Os-Medendorp H. Use and the users of a patient portal: cross-sectional study. J Med Internet Res 2018 Sep 17;20(9):e262 [FREE Full text] [CrossRef] [Medline]
  23. Huang J, Chen Y, Landis JR, Mahoney KB. Difference between users and nonusers of a patient portal in health behaviors and outcomes: retrospective cohort study. J Med Internet Res 2019 Oct 07;21(10):e13146 [FREE Full text] [CrossRef] [Medline]
  24. Emani S, Peters E, Desai S, Karson AS, Lipsitz SR, LaRocca R, et al. Who adopts a patient portal?: an application of the diffusion of innovation model. J Innov Health Inform 2018 Oct 25;25(3):149-157. [CrossRef] [Medline]
  25. Andrews E, Berghofer K, Long J, Prescott A, Caboral-Stevens M. Satisfaction with the use of telehealth during COVID-19: an integrative review. Int J Nurs Stud Adv 2020 Nov;2:100008 [FREE Full text] [CrossRef] [Medline]
  26. Actuele informatie over COVID-19. Rijksinstituut voor Volksgezondheid en Milieu. 2020.   URL: [accessed 2020-05-27]
  27. Al-Hazzaa HM, Al-Sobayel HI, Abahussain NA, Qahwaji DM, Al-Haifi AR, Desoki TE, et al. Joint associations of activity energy expenditure and sedentary behaviors with adolescent’s obesity and dietary habits. Med Sci Sports Exerc 2014;46:518. [CrossRef]
  28. Farajian P, Panagiotakos DB, Risvas G, Malisova O, Zampelas A. Hierarchical analysis of dietary, lifestyle and family environment risk factors for childhood obesity: the GRECO study. Eur J Clin Nutr 2014 Oct;68(10):1107-1112. [CrossRef] [Medline]
  29. Meng Q, Xie Z, Zhang T. A single-item self-rated health measure correlates with objective health status in the elderly: a survey in suburban beijing. Front Public Health 2014;2:27. [CrossRef] [Medline]
  30. Suchman EA, Streib GF, Phillips BS. An analysis of the validity of health questionnaires. Soc Forces 1958 Mar 01;36(3):223-232. [CrossRef]
  31. Huang FY, Chung H, Kroenke K, Delucchi KL, Spitzer RL. Using the Patient Health Questionnaire-9 to measure depression among racially and ethnically diverse primary care patients. J Gen Intern Med 2006 Jun;21(6):547-552 [FREE Full text] [CrossRef] [Medline]
  32. Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
  33. Diener E, Emmons RA, Larsen RJ, Griffin S. The Satisfaction With Life Scale. J Pers Assess 1985 Feb;49(1):71-75. [CrossRef] [Medline]
  34. Reichheld FF. The one number you need to grow. Harv Bus Rev 2003 Dec;81(12):46-54, 124. [Medline]
  35. Askari M, Eslami S, Scheffer A, Medlock S, de Rooij SE, van der Velde N, et al. Different risk-increasing drugs in recurrent versus single fallers: are recurrent fallers a distinct population? Drugs Aging 2013 Oct;30(10):845-851. [CrossRef] [Medline]
  36. Griffin A, Skinner A, Thornhill J, Weinberger M. Patient portals: Who uses them? What features do they use? And do they reduce hospital readmissions? Appl Clin Inform 2016;7(2):489-501 [FREE Full text] [CrossRef] [Medline]
  37. Lupiáñez-Villanueva F, Anastasiadou D, Codagnone C, Nuño-Solinís R, Garcia-Zapirain Soto MB. Electronic health use in the European Union and the effect of multimorbidity: cross-sectional survey. J Med Internet Res 2018 May 03;20(5):e165 [FREE Full text] [CrossRef] [Medline]
  38. Wynn R, Oyeyemi SO, Budrionis A, Marco-Ruiz L, Yigzaw KY, Bellika JG. Electronic health use in a representative sample of 18,497 respondents in Norway (The Seventh Tromsø Study - Part 1): population-based questionnaire study. JMIR Med Inform 2020 Mar 05;8(3):e13106 [FREE Full text] [CrossRef] [Medline]
  39. Ancker JS, Barrón Y, Rockoff ML, Hauser D, Pichardo M, Szerencsy A, et al. Use of an electronic patient portal among disadvantaged populations. J Gen Intern Med 2011 Oct;26(10):1117-1123 [FREE Full text] [CrossRef] [Medline]
  40. Powell RE, Henstenburg JM, Cooper G, Hollander JE, Rising KL. Patient perceptions of telehealth primary care video visits. Ann Fam Med 2017 May;15(3):225-229 [FREE Full text] [CrossRef] [Medline]
  41. Smrke A, Younger E, Wilson R, Husson O, Farag S, Merry E, et al. Telemedicine during the COVID-19 pandemic: impact on care for rare cancers. JCO Glob Oncol 2020 Jul;6:1046-1051 [FREE Full text] [CrossRef] [Medline]
  42. Soegaard Ballester JM, Scott MF, Owei L, Neylan C, Hanson CW, Morris JB. Patient preference for time-saving telehealth postoperative visits after routine surgery in an urban setting. Surgery 2018 Apr;163(4):672-679. [CrossRef] [Medline]
  43. Askari M, Klaver NS, van Gestel TJ, van de Klundert J. Intention to use medical apps among older adults in the Netherlands: cross-sectional study. J Med Internet Res 2020 Sep 04;22(9):e18080 [FREE Full text] [CrossRef] [Medline]
  44. Price-Haywood EG, Harden-Barrios J, Ulep R, Luo Q. eHealth literacy: patient engagement in identifying strategies to encourage use of patient portals among older adults. Popul Health Manag 2017 Dec;20(6):486-494. [CrossRef] [Medline]
  45. Gupta D, Denton B. Appointment scheduling in health care: Challenges and opportunities. IIE Trans 2008 Jul 21;40(9):800-819. [CrossRef]
  46. Kong Q, Lee C, Teo C, Zheng Z. Scheduling arrivals to a stochastic service delivery system using copositive cones. Operations Res 2013 Jun;61(3):711-726. [CrossRef]
  47. Kong Q, Li S, Liu N, Teo C, Yan Z. Appointment scheduling under time-dependent patient no-show behavior. Manage Sci 2020 Aug;66(8):3480-3500. [CrossRef]
  48. Bielen F, Demoulin N. Waiting time influence on the satisfaction‐loyalty relationship in services. Managing Serv Quality 2007 Mar 27;17(2):174-193. [CrossRef]
  49. Palawatta T. Waiting times and defining customer satisfaction. Vidyodaya J Manage 2015 Dec 30;1(1):1. [CrossRef]
  50. Jeske D, Shultz K. Using social media content for screening in recruitment and selection: pros and cons. Work Employment Soc 2015 Nov 20;30(3):535-546. [CrossRef]
  51. Gruzd A, Jacobson J, Mai P, Dubois E. The State of Social Media in Canada 2017. SSRN J 2018:1. [CrossRef]
  52. Smith G. Does gender influence online survey participation?: A record-linkage analysis of university faculty online survey response behavior. ERIC Doc Reprod Serv No ED 501717 2008:1.
  53. Sax LJ, Gilmartin SK, Lee JJ, Hagedorn LS. Using web surveys to reach community college students: an analysis of response rates and response bias. Community Coll J Res Pract 2010 Jul 08;32(9):712-729. [CrossRef]

IRB: Institutional Review Board

Edited by A Kushniruk; submitted 24.11.20; peer-reviewed by J van Delden, J Jones; comments to author 29.12.20; revised version received 01.02.21; accepted 03.05.21; published 19.07.21


©Qingxia Kong, Danique Riedewald, Marjan Askari. Originally published in JMIR Human Factors (, 19.07.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.