Modeling Trust in COVID-19 Contact-Tracing Apps Using the Human-Computer Trust Scale: Online Survey Study

Introduction

Background

The COVID-19 pandemic has changed how we view technology as a resource to stop the spread of disease. To address the need to control the spread of the virus, many governments and public health authorities worldwide have launched several technological initiatives, including the development of artificial intelligence (AI) contact-tracing mobile apps (CTAs). As a result, by the end of 2020, there were more than 50 CTAs available in both Google Play and iOS App Store [1,2]. According to Nguyen et al [3], security and privacy are crucial in designing AI-based CTA technologies. If users perceive CTAs as a threat to their privacy, this might affect their predisposition to use the app, ultimately affecting its adoption rate and tool effectiveness. This evidence has led to an increased discourse for design systems toward focusing on ensuring that CTAs are secure and private. Previous studies have recommended several criteria such as ensuring a low level of complexity of the security feature so that it is easy to use and understandable for the general population [4,5], visibility and interaction from the user, and unambiguous and clear messages to follow while designing security measures [6-8]. Similar arguments were put forth in Europe’s stated goals to ensure ethical and responsible technological development. Although COVID-19 CTAs in Europe followed the General Data Protection Regulation and ISO/IEC 27001 [9] regulations, and were also designed in consideration of current AI principles to regulate technology use (ie, Ethical guidelines for Trustworthy AI [10]), this was not sufficient to ensure the trustworthiness from citizens. This lack of trustworthiness exists despite widely available information on how these technologies were built with transparent and ethical principles in mind. Moreover, despite government initiatives to push through their adoption, the download rates and actual usage rates of these apps remained low [2,6,11-13]. One reason for this low adoption might be that security and privacy in computer science are still mainly approached from a technical perspective [14]. Privacy attributes in technology can be more profound and complex than technical qualities. Privacy is defined as a person’s control over the information that is manipulated and communicated to others [6,15-18].

Privacy also includes interpersonal characteristics such as the perception of privacy, system honesty or benevolence communication, and shared control to minimize associated risk and uncertainty. For instance, despite appropriate regulations and principles being considered when designing Estonia’s COVID-19 CTA (HOIA), the adoption of HOIA by citizens did not increase. The critical reasons for the low adoption of HOIA included lack of effectiveness (10%) and concerns of security and privacy (19%) according to a survey initiated by The Ministry of Social Affairs, surveying 92% of Estonian residents [13,19]. Thus, all efforts made in designing AI-based transparent and ethically responsible CTAs that can prevent data misuse and ensure the development of responsible trustworthy AI interactions were unsuccessful.

We believe that it is essential to find new ways to ensure incorporating trust values in the design of such apps that could lead to building more technological, socially responsible societies. One should expect trust to be increasingly in demand as a means of enduring the complexity of a future that technology will generate. The quality and depth of technology use are also significantly affected by users’ trust in the technology. Trust is defined according to the ability to determine who to trust, and represents the willingness of a party to be vulnerable to the actions of another party based on the expectation that the other will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other party [20-22].

Research Gaps

Prior research confirms that technology acceptance and adoption are affected by the level of trust users have in the technology [11,20-23]. However, evidence shows that designing trustworthy technologies is complex and needs to be better understood. Like privacy, trust is an interpersonal quality that is present in many moments of our daily lives, and is thus often considered unconsciously. Whether being conscious or unconscious of its existence, trust represents an important key of the relationships encountered in daily life, including interactions between humans and machines. Establishing a trustful relationship implies peoples’ permission to share knowledge, delegation, and cooperative actions [11,22,24,25]. Thus, in addition to the current research challenge for ensuring that all ethical, privacy, and technical security requirements are considered [5,7,9], we argue that trust might be the reason why users do not feel comfortable using CTAs that depend on citizens’ data to function properly. If this is indeed the case, besides existing design regulations and principles, designers will also need mechanisms to analyze individuals’ perceived trustworthiness in AI apps. In this way, designers and other stakeholders can gain a deeper understanding of how individuals perceive the benefits of AI, and assess their predisposition to cooperate and be more willing to use the technologies. Thus, it is important to gauge the extent to which such AI data–driven technologies are perceived as trustworthy (ie, the gains of using CTAs are higher than the possible losses).

There are three main rationales for the above argument. First, with the current culture of increased introduction and use of complex systems in our daily activities, researchers need to focus more on conceiving responsible human-computer interactions. Second, current paradigms supporting ethical and responsible design practices are insufficient to ensure technology trustworthiness. Third, a new human-machine interaction mechanism is needed to effectively evaluate users’ trust perceptions in technology (eg, assess users’ experience toward incorporated trust values). Namely, we propose new human-centered design frameworks and mechanisms to guide the design and technology evaluation process. Overall, in the past decade, human-computer interaction has contributed significantly toward improving the quality of living with technology. Consequently, regular individuals are getting more involved, engaged, and dependent on technology to achieve their goals. It is true that we no longer live without technology. Despite this, the above arguments indicate that we are entering a new era that depends on data to thrive. This symbiotic dependence of humans in systems abilities and of systems dependence in our data to provide meaningful information has increased the complexity of the technology provided. Consequently, we have become more reliant on trust to survive in these complex symbiotic relationships. This is clearly shown in how digital CTAs were affected by these symbiotic relationships. Most of these apps are collecting highly sensitive data from individuals, including where they have been and with whom they have been in contact.

Methods

Study Aims and Design

This study builds on the prior work of Gulati et al [20] and Sousa et al [22], and is guided by one central research question: Can the Human-Computer Trust Scale (HCTS) be used to assess an individual’s perception of trust in health technologies? The main goal of this study was to propose a novel design evaluation mechanism to incorporate trust values in health care technologies, and make health care interventions and technologies more trustworthy and accepted. Namely, we used partial least-squares structural equation modeling (PLS-SEM) to empirically ascertain which attributes of the proposed scale (HCTS) hold in health care contexts and can be used as lenses to evaluate individuals’ trust predisposition to interact. The study was divided into two main stages: (1) adaptation and translation of the scale, and (2) measurement and validation of the questionnaire (HCTS).

Theoretical Model

The adopted theoretical model, the HCTS [20], illustrates the multidimensional nature of trust, taking into account several attributes of trust, as shown in Figure 1. This model was validated with statistical modeling techniques. The proposed attributes of the model were gathered from a systematic multidisciplinary literature review, combined with (1) a word elicitation study to capture a rich set of multidisciplinary notions encapsulating trust; (2) participatory design sessions and exploratory interviews with users to further identify antecedents of trust; (3) the unification of technology acceptance models [22]; and (4) separate studies to ensure statistical certainty of the scale proposed: trust in Siri, trust in the Estonian electronic voting system, trust in futuristic scenarios, and trust in human-robot interaction [20,26]. The final scale to measure trust consists of three main attributes: risk perception, competency, and benevolence. In line with the above findings and with the awareness that trust assessment is context- and culture-dependent, we assessed the validity of the scale to measure citizens’ trust attitudes in CTAs. To achieve our goal, we developed four sets of assumptions that might affect or predict a user’s trust when interacting with the HOIA app. The four hypotheses (H1-H4) established in regard to our main research question are outlined in Textbox 1.

‎View this figure

Study Procedure

Questionnaire

We used a semistructured questionnaire to collect data. Before distributing the questionnaire, we adapted the original scale to the context and translated the content from English into Estonian. The translation and adaptation of the instrument followed the guidelines of the adaptation, translation, and validation process [27]. The survey was designed based on the HCTS in the Estonian language and was administered during April 2021. The objective of this study was to build on prior works and empirically assess HCTS to ascertain which attributes of the model hold true in health user–technology interactions.

The survey was created using both Lime Survey and Google Forms. During the pilot study, the feedback from the respondents was that the visual design of the Google Forms is less confusing; therefore, it was decided to adopt Google Forms as the final survey format.

Stimuli

To ensure that all participants understood the technical artefact in question and their perceptions of trust regarding similar experiences, we provided the official video that explains HOIA to the users as a stimulus, following the concept of technology probe and design fiction, also known as a vignette-based study in psychology.

Recruitment

The survey was carried out among the Estonian population, which was distributed online, mainly through Facebook and other social network groups available to the authors. A convenience sample was used in data collection because this enables reaching members of the population who are easily accessible, available, and willing to participate [28].

Ethical Considerations

This study complies with the basic ethical principles for the responsible conduct of research involving human subjects. Informed consent was requested from all participants, and authorization was obtained from the authors of the scale [20] to carry out the contextual adaptation and validation of the scale. The study was approved by the Tallinn University Ethics committee on July 9th, 2021 (study name: “Survey on the dynamic trust relationships between technology, society and culture"; approval number: Taotlus nr 6-5.1/17).

Results

Participant Characteristics

A total of 78 responses were obtained and used for data analyses; very few responses were excluded as all respondents fully completed the survey. The three excluded cases included answers leaning in majority toward neutral options. Data collected included the following information: demographics, usage patterns of mobile apps and HOIA, trust in HOIA (including risk perception, benevolence, competence, and general trust), and opinions about HOIA’s existing and additional functionalities. Among the 78 respondents, 73% (n=57) were women and only 27% (n=21) were men. Almost half of the respondents (36/78, 47%) were between the ages of 31-42 years and approximately one third (25/78, 32%) were 43-55 years old.

HOIA Usage Patterns

Among the 78 respondents, 61 had downloaded the HOIA CTA. Among them, the 47 women showed the highest rate of downloads compared with the 14 male respondents. Younger respondents (aged 18-30 years) had a higher number of downloads (88%), but they also represented the smallest sample. Slightly more than half of the participants (56%) admitted that they do not feel confident in how to use HOIA; this perception was more prominent among men (n=13). Twenty participants admitted that they had never opened the app, despite 61 claiming to use mobile apps daily.

Among the 17 respondents who had not downloaded the HOIA app, the majority were men. The main reasons claimed by participants for not downloading HOIA included: do not understand how it works, and concerns about the privacy and security of their data. When asked what additional features they expect from the CTA, some mentioned the need to understand the benefits of using it actively. When asked about their most common activities on their mobile devices, 76 participants stated that they are used for communication, 66 stated social networking, 60 stated entertainment purposes, and 40 indicated uses related to health and well-being.

Assessment of the Scale

The HCTS under investigation includes five constructs: risk perception, competency, benevolence, reciprocity, and trust [20,22,26] (see Figure 1). Following the recommendation of Hair et al [29], the minimum sample size needed to effectively perform a PLS-SEM for our study was calculated to be 40 (ie, 10 times the maximum number of arrowheads pointing at a latent variable in a PLS path model). This method was selected because measuring trust in technology is complex, including four constructs and model relationships in this case. The measures used in the study were adapted from Gulati et al [20]. Their work models trust in technology with different studies, including trust in Siri using design fiction (future scenarios), the Estonian electronic voting service, and trust in human-robot interactions [24]. Gulati et al [20] measured risk perception using the concept of willingness and motivation developed through two independent studies [6,24]. This study added two additional items created through Schoorman et al’s [21] conceptualizations of trust. Gulati et al [20] measured competency and reciprocity based on the methodology of Mcknight et al [30], and measured benevolence based on adaptation of the prior work of Harwood and Garry [31] and McKnight et al [30]. The survey used a 7-point Likert scale to collect data, where 1 indicates strongly disagree and 7 indicates strongly agree. All of the items were positively worded except for the risk perception scale, which was adapted as a negatively worded statement and reversed before analyzing the data. The HCTS measures are summarized in Textbox 2.

Data Analysis

We analyzed a total of 78 answers. All scales for analyzing data in our study were positively worded, except perceived risk, which was negatively worded. The first steps in the analyses involved assessing the reliability and validity of the HCTS to measure trust in HOIA. In this phase, we calculated if the items have good measurements of the latent construct [29,32]. We discarded risk perception item 6 and competency item 4 because the loadings were below 0.5, and kept all loadings above their respective thresholds (>0.5). Table 1 and Figure 2 demonstrate all items used in the analysis and their loadings.

We further verified if the average variance extracted (AVE) was higher than 0.5; as shown in Table 1, all AVE values were >0.5, demonstrating that the items have good convergent reliability [12,32]. Similarly, the composite reliability of all indicators was above >0.7, showing adequate internal consistency. The Dillon-Goldstein ρ statistic, according to Hair et al [29], is similar to Cronbach α but allows the indicator variables to have varying outer loadings, and should be higher than 0.7 (or >0.6 in exploratory research). These values were above 0.7 for all items (Table 1), further demonstrating that the model is acceptable and has satisfactory internal consistency.

The discriminant validity and cross-loading values obtained using the Fornell-Lacker criterion (Table 2) indicated that the validity of each construct is higher for itself than for each corresponding construct [32].

‎View this figure

Trust Toward HOIA

In addition, we assessed the coefficient of determination (R²) values, which represent the combined effect of exogenous latent variables on the endogenous latent variable, and is interpreted in the same way as in a conventional regression analysis procedure [29]. In this study, the R² value was 0.806 and the adjusted R² was 0.795. According to Hair et al [29], R² values of 0.75, 0.50, or 0.25 are considered substantial, moderate, or weak, respectively. In line with this interpretation, both the R² and adjusted R² values of this study indicate a substantial effect. Thus, approximately 83% of the changes in technology trust can be explained by the statistically significant exogenous variables in the HCTS. Accordingly, we conclude that the statistically significant attributes significantly predict user trust in COVID-19 CTAs, namely HOIA. Keeping in mind all of the empirical values obtained thus far, it is safe to say that our model passes the criteria for both measurement and structural model evaluation, and the final scale exhibits good validity, reliability, and predictive power.

Discussion

Principal Findings

To contribute toward our central research question (can the HCTS be used to assess an individual’s perception of trust in health technologies?), we empirically assessed the suitability of the HCTS to assess an individual’s perception of trust in health technologies, with the broader goal of understanding which attributes of the HCTS hold true in health technologies. As shown in Table 3, all but one of our four hypotheses were supported, based on statistically significant effects.

For instance, H1 (risk perception mediates trust), H2 (competency mediates trust), and H4 (reciprocity mediates trust) were statistically significant, which is in line with the work of Gulati et al [20]. However, we also found that H3 (benevolence mediates trust) was nonsignificant (P=.52). To understand these results, it is important to consider how these constructs were operationalized. H1 and H2 were operationalized based on Gulati et al’s [20] and Schoorman et al’s [21] conceptualizations of trust, whereas H3 and H4 were operationalized based on Gulati et al [20].

Limitations

Our study is not without its limitations, which can guide future research. First, culture influences trust. Second, the proposed scale (HCTS) demonstrated that trust is a dynamic construct that evolves in context and is culturally dependent. Third, the additional suggested items based on Schoorman et al’s [21] conceptualizations need further reassessment, as the results are more in line with those of Gulati et al [20], but also indicate no significant correlation between the Estonian citizens’ perception of HOIA as a benevolent trait.

Conclusion

In conclusion, the results of this study indicate that the degree of trust toward the Estonian CTA (HOIA) is significantly correlated with the extent to which users perceive the system as competent, reciprocal, and risky. This study used PLS-SEM to identify statistically significant factors for assessing individuals’ perception of trust in human-technology interactions for health. This work contributes toward establishing a final version of the scale derived from the HCTS consisting of 13 items that can be used to measure user trust levels, including competence, reciprocity, and perceived risk. Moreover, these results should not only be limited to HOIA but can also be implemented to measure trust in other CTAs.