Published on in Vol 9, No 4 (2022): Oct-Dec

This is a member publication of National University of Singapore

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/39312, first published .
Reinterpretation of Health Information in the Context of an Emerging Infectious Disease: A Digital Focus Group Study

Reinterpretation of Health Information in the Context of an Emerging Infectious Disease: A Digital Focus Group Study

Reinterpretation of Health Information in the Context of an Emerging Infectious Disease: A Digital Focus Group Study

Original Paper

1University of North Carolina Project-China, Guangzhou, China

2Saw Swee Hock School of Public Health, National University of Singapore and National University Health System, Singapore, Singapore

3Dermatology Hospital of Southern Medical University, Guangzhou, China

4Research for Impact, Singapore, Singapore

Corresponding Author:

Rayner Kay Jin Tan, PhD

University of North Carolina Project-China

2 Lujing Road

Guangzhou, 510095

China

Phone: 65 91878576

Email: rayner.tan@nus.edu.sg


Background: Misinformation related to the COVID-19 pandemic has accelerated global public concern and panic. The glut of information, or “infodemic,” has caused concern for authorities due to its negative impacts on COVID-19 prevention and control, spurring calls for a greater scholarly focus on health literacy during the pandemic. Nevertheless, few studies have sought to qualitatively examine how individuals interpreted and assimilated health information at the initial wave of COVID-19 restrictions.

Objective: We developed this qualitative study adopting chat-based focus group discussions to investigate how individuals interpreted COVID-19 health information during the first wave of COVID-19 restrictions.

Methods: We conducted a qualitative study in Singapore to investigate how individuals perceive and interpret information that they receive on COVID-19. Data were generated through online focus group discussions conducted on the mobile messaging smartphone app WhatsApp. From March 28 to April 13, 2020, we held eight WhatsApp-based focus groups (N=60) with participants stratified by age groups, namely 21-30 years, 31-40 years, 41-50 years, and 51 years and above. Data were thematically analyzed.

Results: A total of four types of COVID-19 health information were generated from the thematic analysis, labeled as formal health information, informal health information, suspicious health information, and fake health information, respectively. How participants interpreted these categories of information depended largely on the perceived trustworthiness of the information source as well as the perceived veracity of information. Both factors were instrumental in determining individuals’ perceptions, and their subsequent treatment and assimilation of COVID-19–related information.

Conclusions: Both perceived trustworthiness of the information source and perceived veracity of information were instrumental concepts in determining one’s perception, and thus subsequent treatment and assimilation of such information for one’s knowledge of COVID-19 or the onward propagation to their social networks. These findings have implications for how policymakers and health authorities communicate with the public and deal with fake health information in the context of COVID-19.

JMIR Hum Factors 2022;9(4):e39312

doi:10.2196/39312

Keywords



Background

The COVID-19 pandemic was declared a public health emergency of international concern by the World Health Organization (WHO) in January 2020 [1] and was subsequently declared a pandemic in March 2020 [2]. As of April 2022, globally, over 500 million people have been infected with the virus that causes COVID-19 and more than 6 million people have died from the disease [3]. Besides its impact on morbidity and mortality, COVID-19 has deeply impacted economies, health systems, and social lives globally [4].

The rapid spread of COVID-19 and its variants Delta and Omicron led to an urgent need for reliable, evidence-based, trustworthy, and updated health information that could help individuals to inform decision-making around COVID-19 prevention and management. Early dissemination of COVID-19 prevention guidelines by global and national authorities was accompanied by a concomitant rise of misinformation or fake news [5], termed an “infodemic” by the WHO Director-General Tedros Adhanom Ghebreyesus. The infodemic has caused concern for authorities due to its negative impacts on COVID-19 prevention and control [6], spurring calls for a greater scholarly focus on health literacy during the pandemic [7-9].

The infodemic has resulted in challenges in efficiently and trustworthily conveying reliable, rigorous COVID-19 information to the public. For example, a nationally representative online survey in Germany found that 47.8% of participants had trouble assessing if media information on COVID-19 could be trusted [10]. The infodemic has simultaneously facilitated the spread of conspiracy theories on COVID-19, including its origins and vaccines [11-13]. Studies also highlight how those with low levels of health literacy and those residing in low- to middle-income countries are at the greatest risk of succumbing to false or misleading pandemic-related information [10,14].

Past studies have attempted to determine the relationship between general literacy and COVID-19–related health literacy [15-17] or nuance the nature and types of COVID-19 information [18-20]. However, there has been minimal research to explore the factors that affect how individuals negotiate, construe, and interpret information in an infodemic. An understanding of such human factors, specifically how individuals interpret health information during emerging infectious disease contexts, is important for the widespread assimilation and integration of health information technologies in our ongoing management of the current pandemic and future pandemics.

Use of Mobile Chat Apps for Focus Group Discussions

The use of focus group discussions (FGDs) for social science and health research, especially in reference to its methodological rigor for qualitative research, has been widely researched [21,22]. Such inquiries have extended to online FGDs, with their origins extending to the late 1990s with the use of emails and message boards [23], and subsequently with virtual discussion rooms and teleconferencing software [24,25].

Studies have found that online FGDs have important benefits relative to in-person groups. While responses were typically less detail-rich, researchers found that they were more immediate. Past studies also argued that online FGDs lack contextual clues that may inform perceived power differences between participants, which may facilitate the sharing of sensitive information and disagreements among participants [26,27]. Logistically, online groups can be more inclusive than traditional in-person FGDs, reducing physical access–related barriers to participation and allowing for participants to share diverse types of media throughout the course of the FGDs [28].

We also recognize the drawbacks of online FGDs compared to in-person groups. Online FGDs tend to generate lower word counts, shorter responses, and provide less detail or richness [25,26,29]. They also tend to have fewer group interactions and lower responsiveness to facilitators’ questions and probes [25,27,29].

Recent scholarship has highlighted the usefulness of mobile chat platforms such as WhatsApp in eliciting and generating qualitative data. Chen and Neo’s [30] study comparing indicators of data depth and breadth between WhatsApp-based and in-person FGDs in Singapore found that while data richness and detail in the WhatsApp FGDs did not match that of in-person groups, younger and more digitally savvy participants generated well-elaborated responses and were interactive within the chat groups. Findings from Colom’s [31] study in Western Kenya employing digital ethnographic methods, including the use of WhatsApp FGDs, also showed that the use of WhatsApp provided a high level of ecological validity [31].

The COVID-19 Pandemic in Singapore

The first case of COVID-19 was reported in Singapore on January 23, 2020, prompting national authorities to implement a series of movement control measures to curb disease spread. One of the first steps taken involved the closure of entertainment establishments in late February 2020 following the change in Singapore’s Disease Outbreak Response System Condition (DORSCON) color code from yellow to orange. This signaled an official recognition of COVID-19’s severity and infectiousness, and a recognition that the disease had arrived in Singapore. A “circuit-breaker” period was implemented from April 7 to June 1, 2020, characterized by strict movement control measures aimed at curbing the growing incidence of COVID-19 cases in the community, and a “break the circuit” of transmission [32] period, including the closure of all nonessential services and a mask mandate. Since June 2020, Singapore has gradually eased these measures in phases; phase one took place between June 2 and June 18, 2020, which involved the progressive resumption of select businesses and activities.

Study Rationale and Objective

Against the backdrop of information-related concern and policy responses in the early stages of the COVID-19 pandemic in Singapore, we developed this study to investigate how individuals interpreted COVID-19 health information during the first wave of COVID-19 restrictions.


Participants and Data Generation

We conducted a series of eight FGDs on WhatsApp between March 28 and April 13, 2020. Further details on how the FGDs were conducted have been published elsewhere [33]. We chose WhatsApp as our means of data collection because it is the most widely used mobile chat app in Singapore [34] and to protect the health and well-being of participants during the pandemic. Participants were recruited via an online flyer distributed on social media platforms, including Facebook, Twitter, and Instagram. Eligibility criteria were aged 21 years or older and being a Singapore citizen or permanent resident at the point of recruitment. Assuming varying levels of technological savvy and their impacts on group dynamics, participants were purposively recruited by strata according to the following age categories: 21-30 years, 31-40 years, 41-50 years, and 51 years and above. Within each group, we ensured a mix of participants of varying ethnicity, gender, and educational attainment. A summary of participant demographics for the study can be found in Table 1.

Table 1. Summary of participant demographics.
VariablesParticipants, n
Age (years)

21-3016

31-4016

41-5016

≥5116
Gender identity

Female27

Male20

Another gender1
Race

Chinese42

Malay2

Indian2

Another race2
Formal education attainment

Secondary and below1

Preuniversity13

University34

Design of FGDs

We conducted two FGDs per age strata, totaling eight FGDs with 6-8 participants per group and 60 participants overall. Each FGD was led by a main facilitator with two observers present. FGDs were conducted over 5 consecutive days, with new discussion topics introduced daily. Topics covered included knowledge and perceptions of COVID-19 and attitudes toward varying information sources.

We conducted both synchronous (ie, all participants were required to be online at a specific time for a specific duration) and asynchronous (ie, participants could reply at their convenience over the course of the day) sessions. Synchronous discussions of approximately 2 hours each were held on the first and last days of the study; asynchronous discussions were held on the second, third, and fourth days. For synchronous sessions, we prepared a list of questions that we had asked participants consecutively during the stipulated timing of 2 hours and participants were expected to stay online to participate at that time. For days that involved asynchronous participation, participants were told that questions would be posed from 9 AM all the way up until 6 PM that day, and that they could choose to answer at any point.

All chat transcripts, including media files such as photos, videos, and memes, were directly downloaded from the researchers’ WhatsApp mobile apps and stored in a secure, password-protected location accessible only to research team members. Participants were reimbursed SG $50 for their time (approximately US $35).

Ethics Considerations

Ethics approval was obtained from the Saw Swee Hock School of Public Health Department Ethics Review Committee (SSHSPH-014). All participants provided documented informed consent before participating and completed a demographic questionnaire to indicate their interest to participate.

Data Analysis

Data were analyzed by the lead author, adopting Braun and Clarke’s [35,36] six steps of reflexive thematic analysis. Both semantic and latent codes and themes were derived from the data without a pre-existing framework. Following the first two stages of familiarization and coding procedures in classic thematic analysis, the lead author noted clear patterns in how participants discussed and privileged varying forms of health information in the COVID-19 context. The lead author discussed these themes with coauthors to ensure that codes and developing themes were adequately fleshed out and authentic to the data generated [37]. Next, the lead author continued to code according to thematic analysis procedures, drawing links between various constructions and interpretations of health information and the factors underpinning them. At this stage, a typology of interpretation was developed by grouping types along two broad axes based on the perceived trustworthiness of an information source and perceived veracity of health information. All data were organized and analyzed using NVivo 11 software (QSR International Pty Ltd).

Data Quality and Trustworthiness

We took several steps to improve the quality and trustworthiness of the data generated in this study. We worked to minimize biases around social desirability or discomfort with sharing by reiterating discussion ground rules on confidentiality, safety, respect, and voluntary participation on a daily basis. To reassure participants of our authenticity, we ensured that our WhatsApp profile pictures used a standardized template featuring a clear photo of our faces and our institutional affiliations. We also created separate “field notes” WhatsApp chat groups for the research team, which we used to reflect on group dynamics and themes in real time, which helped inform probes and prompts for more information and deepen our understanding of participants’ interpretations of health information.


Overview

Based on our analysis of responses in our FGDs, we generated four types of health information, including formal health information, informal health information, suspicious information, and fake health information. These are summarized in Figure 1.

Figure 1. Summary of four types of health information interpreted by participants.
View this figure

Formal Health Information

Participants highlighted how they determined what was formal COVID-19 information through an understanding that such information was both trustworthy and veracious. Trustworthiness was premised largely on familiarity and past knowledge of an information source. Veracity was described by participants as being closely aligned with facts from “official” information sources. Participants described that formal health information around COVID-19 was directly relevant to how they would respond to the ongoing pandemic.

Participants described how they depended on trustworthy information sources such as local news channels and the national newspaper, The Straits Times, for formal COVID-19 information. Participants mentioned that the information source was considered trustworthy based on an understanding of its situation within a national regulatory information framework. This trustworthiness of an information source for formal COVID-19 information was reflected in participants’ beliefs that while some forms of reporting by the same news outlet may lack transparency due to censorship guidelines in Singapore, they could depend on said information source to report based on public interest. Participants also perceived such information to be accurate based on their understanding that it would be fact-checked. One participant described this as follows:

ST [The Straits Times] is a local news and it is under the supervision of IDA [Information Development Authority of Singapore], I know journalists check their facts and they are reviewed by an editor before it is published. […] The only thing is that there might be some info that is not shared for some reason (eg, not to alarm the public)

Other participants displayed slightly more critical attitudes toward the same information source but were able to distinguish between its position as a trustworthy information source for COVID-19 health information and its political intent. Similarly, they viewed the information source to be “factual and accurate” despite their preconceived notions of its role as state-controlled media:

Because it’s ST [The Straits Times], I trust that the article is factual and accurate, but sometimes I may be skeptical about the intent of ST articles. I guess there’s this general perception that ST is state-controlled media, so their articles may be biased towards the government or certain positions or promoting messages that are favorable to the government. But not that there’s anything very wrong about that. I will still read ST articles regardless. […] Yes, I read ST articles quite regularly, especially for local news like updates on the COVID-19 situation and related COVID-19 measures in Singapore.

Additionally, participants described how in the absence of such an understanding, they could fact-check on their own accord and rely on such information’s congruence with other information sources to determine veracity. Participants illustrated this in response to the facilitator’s prompt for participants’ sources of knowledge on the symptoms of COVID-19:

I first heard about it through credible news articles - but I’ve also googled to see what MOH [Ministry of Health] had to say. and I’ve checked the government resource (Singapore COVID-19 Symptom Checker [38])
[Participant 1]
From news media, but lately, just to prevent my drowning from all the news sites, I just follow CNA [Channel News Asia] and the government’s information.
[Participant 2]

Informal Health Information

Participants described informal health information as originating from trustworthy sources, but may contain information that was not perceived to be veracious. Participants discussed how trustworthy information sources, including health information from traditional Chinese medicine practices, religious healing, or other informal sources of health information, served as important information that supplemented formal sources of health information. Participants reported relying on such forms of information as supplementary means of protecting oneself, rather than treating it as formal COVID-19 information.

Participants in the FGDs for older participants offered insight into how some information sources of health information were perceived to be trustworthy even if they did not provide veracious COVID-19 health information. When provided an image prompt depicting the use of onions, turmeric, and ginger to ward off COVID-19, participants discussed how health information from sources such as traditional Chinese medicine could be viewed as trustworthy, even though it was not perceived as being aligned with veracious information on COVID-19:

Yes, I heard onion is good even before this COVID-19. This has been circulating for a while already. I don’t believe it works this way but I can understand why it is being circulated. They are known to have anti-inflammatory properties when eaten so people might skew this info.
[Participant 1]
I believe they [Chinese people] know better than me. This is not supported by scientific research. […] Being elderly, more familiar with traditional Chinese medicine (TCM), but especially Chinese.
[Participant 2]

Other participants cited how friends served as trustworthy sources of information, even though they could not ascertain the veracity of such health information in the context of COVID-19 prevention. When prompted with the same picture from above, a participant from a separate FGD highlighted the following: “I heard onions do kill germs, and friends do practice it. But how effective, I don’t know.”

Other participants also discussed the use of a “silver ion spray”; one participant shared information about and promoted the product, citing its antiviral properties, even though such information was not specific to COVID-19:

Never seen or heard. But heard of silver ion spray. Please watch this video with a high-resolution camera showing how the virus can be spread. Don’t bother about the Japanese language. The images are enough to understand. [Video attachment of silver ion spray product]. It’s an alcohol-free antivirus spray that can kill up to 99.9% [of viruses]. It’s safe to be sprayed into the eye as well.

Suspicious Health Information

Participants described suspicious health information as typically originating from less trustworthy information sources, even though it might be consistent with known, veracious COVID-19–related information. Such information was typically not viewed as a favorable information source that participants would assimilate or share with others. One participant pointed out how despite factual reporting of COVID-19 information from certain sources, they would “take it with a pinch of salt” due to perceived political biases of such publications:

We even discount [South China Morning Post] as credible. We need to analyze whether certain posts are politically motivated. News outlets like Fox News may be official and credible but I will take it with a pinch of salt.

When asked to discuss and compare three articles by various news outlets on the same topic, participants highlighted that their levels of trust in that same piece of health information varied depending on the source of the news article. One participant highlighted how it was difficult to measure trustworthiness among news articles and preferred to stick to government bodies as an authority for formal COVID-19 information. Participants also pointed out that a media outlet’s trustworthiness would also depend on the subject matter, and that they would trust varying sources depending on whether they were reporting on certain topics or geographies. A discussion by several participants highlights this nuance:

I’ll trust ST [The Straits Times] more because I've read articles from BBC [British Broadcasting Corporation] and SCMP [South China Morning Post], especially that latter, that have a very biased view against Singapore. Since ST is more towards the local context, as a Singaporean I’ll trust it more.
[Participant 1]
Depending on the locations that are being written about, I would trust different sources. For example, if I wish to find out about local updates, I would be inclined to read Straits Times instead.
[Participant 2]

Fake Health Information

Participants described how fake health information usually came from untrustworthy sources and could be viewed as not being veracious through two mechanisms. First, it could be determined as a function of trustworthiness, in that information was not trustworthy owing to its historical and consistent lack of fact-checking. Therefore, information would be viewed as inaccurate regardless of its content. Second, information from such sources did not align or comport with veracious COVID-19 information propagated by official sources. In general, participants disregarded such forms of health information.

Broadly, participants across most FGDs described how forwarded WhatsApp messages from relatives were regarded as untrustworthy, and therefore contained unverified information on COVID-19. Participants from one FGD forwarded a text message to the group from a relative on the use of food to protect oneself from COVID-19, and subsequently commented on its trustworthiness:

I didn’t take it seriously because there was no trustworthy source cited.
[Participant 1]
Recently I couldn’t find vinegar in the supermarket. I asked my mummy’s circle [a motherhood-centered social group] why vinegar is OOS [out of stock]. According to them, some elderly believe drinking vinegar can protect themselves from COVID-19. Btw [by the way], I just need it for my gyoza [Japanese dumpling].
[Participant 2]

By contrast, participants in a separate FGD focused on COVID-19 prevention measures in their discussion on determining fake health information. One participant had forwarded a video that was circulating on WhatsApp of a police officer chatting with a civilian on the sidewalk next to a police van and issuing a fine. Another participant responded that it was untrustworthy by virtue of how it was shared (ie, through WhatsApp). However, while participants viewed forwarded messages as an untrustworthy source, they also described how they would discern if such information aligned with their knowledge of accurate and formal COVID-19 information. Specifically, the original poster approached this as suspicious health information instead and called this a “half-truth” due to the lack of trustworthiness of the source, although aligning with their understanding around current COVID-19 movement control measures:

[This video] Definitely, sent through WA [WhatsApp] groups.
[Participant 1]
I saw that too. I feel like it’s some sort of a half-truth, around people being issued notices [for flouting safe distancing measures]. I don’t know if the actual fines are legit [legitimate].
[Participant 2]
I think the fines are for people who do not observe safe distancing. The authorities encourage people to wear masks now even if well but it is not mandatory. So yeah, such forwarded message creates confusion.
[Participant 1]

Principal Findings

Our study found that participants interpreted four types of health information, namely formal health information, informal health information, suspicious health information, and fake health information. These forms of health information were determined as a function of the perceived trustworthiness of a given information source and the perceived veracity of information. We discuss the implications of these findings in the context of existing studies on health literacy and approaches to dealing with the COVID-19 infodemic.

We found that participants were able to distinguish between formal and informal sources of health information. Determining if health information belonged to these respective categories would involve receiving information from trustworthy sources of information and assessing if it is veracious or not, which may be a given for certain sources of formal information or requires further fact-checking. For study participants, supplementary health information sources played an important role in providing additional perceived protections from COVID-19, although participants recognized that such information was not official or they were unaware of the mechanisms through which it would positively impact COVID-19 prevention efforts. This was especially salient for the older age groups, who discussed how certain forms of trustworthy sources, including alternative healing, traditional Chinese medicine, or frameworks of cultural beliefs, provided resources to supplement their COVID-19 prevention behaviors. Studies in other settings have found that complementary, traditional, or alternative forms of medicines have played a strong role in the alleged treatment and prevention of COVID-19 [39-41], despite the lack of rigorous randomized controlled trials to underpin such evidence [42,43].

The trustworthiness of a source played a key role in determining if participants would assimilate or share certain forms of information. In general, both suspicious and fake health information led to participants having reservations around assimilating and sharing such information, even if they knew that the information had some “truth” in it due to existing knowledge of formal and accurate COVID-19 information. Past studies show that trust in a particular source plays a strong role in eventual health information–seeking behaviors [44,45]. In turn, those with higher levels of health literacy tended to trust information from health care professionals and were less likely to do so for information from social media, celebrities, and friends [46]. This comports with our finding that older adults, who have been found in multiple settings to have limited health literacy [47-49], may also rely on complementary or alternative sources of health information to inform their COVID-19 prevention efforts. Several scholars have also reviewed the antecedents of trust for online health information [50,51], while others have proposed sociodemographic correlates of trust for varying sources of offline health information [52,53].

Overall, our findings indicate that individuals may label the same piece of health information differently depending on their perceptions of a source’s trustworthiness and veracity of health information. This finding aligns with research that points out a shift toward a “posttruth” era, one that situates truthful information as at risk of being undermined by “alternative facts” or misinformation [54]. The findings of this study are also consistent with past scholarship that posits how the dynamics of belief formation and the definition of truth are being contested in contemporary societies [55], which may impact health and media literacy in society.

Strengths and Limitations

We identified three key study strengths. First, we generated a dynamic framework to understand how people approach or interpret health information in the context of an infodemic. The typology provides opportunities for policymakers to propose interventions and communication strategies that may enable effective and rapid communication in a pandemic, while acknowledging the varied ways in which individuals interpret information. Second, WhatsApp proved to be an appropriate platform for discussions on health literacy in times of an infodemic, as participants were able to share and forward information and media that helped initiate dynamic discussions on their approaches to determining the trustworthiness and/or veracity of such information. Third, WhatsApp-based FGDs allowed us to continue with the study despite the ongoing movement control restrictions that were progressively implemented midway through the study, thus allowing us to retain the integrity of our proposed methods across all groups while keeping our participants safe.

We are also mindful of several limitations. First, the use of WhatsApp FGDs did not allow facilitators to make sense of nonverbal cues such as the use of body language, which might have diminished rapport between participants and researchers. Furthermore, the lack of tone that would normally be present in verbal communication meant that some meanings could have been misconstrued by the facilitators. Nevertheless, our team members were able to analyze emojis, images, or stickers to gain more context in lieu of such cues. These provided additional context for tone when analyzing participants’ responses:

This is the most prolonged WhatsApp conversation I’ve been a part of […]
[Participant 1]
Thank you for the great insights! Have a great week ahead […]
[Participant 2]
Not that I remember
[Participant 3]

Furthermore, having experience within the team in employing this method, steps were taken to mitigate such methodological shortcomings, as articulated in the Methods section [33].

Conclusion

We conclude with two specific recommendations for health authorities and policymakers to enhance effective communications during a health infodemic such as during the COVID-19 pandemic. These recommendations aim to intervene on levels of perceived trust in health information sources and the perceived veracity of health information. In the context of the trustworthiness of health information sources, we recommend policies that shape norms and build trust in select sources of information to combat a glut of health information. The nature of trust and trustworthiness of sources is complex, and draws on participants’ past experiences and literacy around a particular source’s attributes and authority. Scholarly work on the antecedents of trust for online health information might serve as useful starting points for interventions aimed at promoting trust in key sources of information. Such efforts should be implemented well before times of crises and infodemics.

Second, we recommend acknowledging nuances between formal, evidence-based information and information on alternative or complementary medicine or treatments, and implementing an information framework that distinguishes between, yet supports both. Our results suggest that the recognition of supplementary health information may not necessarily be harmful, given that such information may be used to complement formal health information in times of a pandemic. However, this presupposes a strong understanding of what comprises formal and veracious health information and the information architecture that supports such interpretations. Upon establishing key sources of veracious information, individuals may be better equipped to distinguish what must be done to protect themselves in a pandemic, relative to what can be done as a means of supplementing formal COVID-19 prevention measures.

Acknowledgments

The publication of this study was generously funded by the Saw Swee Hock School of Public Health, National University of Singapore and the National University Health System. We thank all participants for their contributions to, and our colleagues for their inputs and support for the study.

Conflicts of Interest

None declared.

  1. Coronavirus disease (COVID-19) pandemic. World Health Organization.   URL: https://www.who.int/emergencies/diseases/novel-coronavirus-2019 [accessed 2022-06-16]
  2. Cucinotta D, Vanelli M. WHO declares COVID-19 a pandemic. Acta Biomed 2020 Mar 19;91(1):157-160 [FREE Full text] [CrossRef] [Medline]
  3. Weekly epidemiological update on COVID-19 - 20 April 2022. World Health Organization. 2022.   URL: https://www.who.int/publications/m/item/weekly-epidemiological-update-on-covid-19---20-april-2022 [accessed 2022-04-26]
  4. Nicola M, Alsafi Z, Sohrabi C, Kerwan A, Al-Jabir A, Iosifidis C, et al. The socio-economic implications of the coronavirus pandemic (COVID-19): a review. Int J Surg 2020 Jun;78:185-193 [FREE Full text] [CrossRef] [Medline]
  5. Zarocostas J. How to fight an infodemic. Lancet 2020 Feb 29;395(10225):676 [FREE Full text] [CrossRef] [Medline]
  6. Munich Security Conference. World Health Organization. 2020.   URL: https://www.who.int/director-general/speeches/detail/munich-security-conference [accessed 2021-02-22]
  7. Paakkari L, Okan O. COVID-19: health literacy is an underestimated problem. Lancet Public Health 2020 May;5(5):e249-e250 [FREE Full text] [CrossRef] [Medline]
  8. Spring H. Health literacy and COVID-19. Health Info Libr J 2020 Sep 16;37(3):171-172 [FREE Full text] [CrossRef] [Medline]
  9. Abel T, McQueen D. Critical health literacy and the COVID-19 crisis. Health Promot Int 2020 Dec 01;35(6):1612-1613 [FREE Full text] [CrossRef] [Medline]
  10. Okan O, Bollweg TM, Berens E, Hurrelmann K, Bauer U, Schaeffer D. Coronavirus-related health literacy: a cross-sectional study in adults during the COVID-19 infodemic in Germany. Int J Environ Res Public Health 2020 Jul 30;17(15):5503 [FREE Full text] [CrossRef] [Medline]
  11. Earnshaw V, Eaton LA, Kalichman SC, Brousseau NM, Hill EC, Fox AB. COVID-19 conspiracy beliefs, health behaviors, and policy support. Transl Behav Med 2020 Oct 08;10(4):850-856 [FREE Full text] [CrossRef] [Medline]
  12. Naeem SB, Bhatti R, Khan A. An exploration of how fake news is taking over social media and putting public health at risk. Health Info Libr J 2021 Jun 12;38(2):143-149 [FREE Full text] [CrossRef] [Medline]
  13. Moscadelli A, Albora G, Biamonte MA, Giorgetti D, Innocenzio M, Paoli S, et al. Fake news and Covid-19 in Italy: results of a quantitative observational study. Int J Environ Res Public Health 2020 Aug 12;17(16):5850 [FREE Full text] [CrossRef] [Medline]
  14. Dash S, Parray AA, De Freitas L, Mithu MIH, Rahman MM, Ramasamy A, et al. Combating the COVID-19 infodemic: a three-level approach for low and middle-income countries. BMJ Glob Health 2021 Jan 29;6(1):e004671 [FREE Full text] [CrossRef] [Medline]
  15. Seng JJB, Yeam CT, Huang CW, Tan NC, Low LL. Pandemic related health literacy – a systematic review of literature in COVID-19, SARS and MERS pandemics. medRxiv. 2020 May 11.   URL: https://www.medrxiv.org/content/10.1101/2020.05.07.20094227v1 [accessed 2022-11-07]
  16. Riiser K, Helseth S, Haraldstad K, Torbjørnsen A, Richardsen KR. Adolescents' health literacy, health protective measures, and health-related quality of life during the Covid-19 pandemic. PLoS One 2020 Aug 28;15(8):e0238161 [FREE Full text] [CrossRef] [Medline]
  17. Nguyen HT, Do BN, Pham KM, Kim GB, Dam HT, Nguyen TT, et al. Fear of COVID-19 Scale-Associations of its scores with health literacy and health-related behaviors among medical students. Int J Environ Res Public Health 2020 Jun 11;17(11):4164 [FREE Full text] [CrossRef] [Medline]
  18. Ashrafi-Rizi H, Kazempour Z. Information typology in coronavirus (COVID-19) crisis; a commentary. Arch Acad Emerg Med 2020;8(1):e19 [FREE Full text] [Medline]
  19. Hernández-García I, Giménez-Júlvez T. Assessment of health information about COVID-19 prevention on the internet: infodemiological study. JMIR Public Health Surveill 2020 Apr 01;6(2):e18717 [FREE Full text] [CrossRef] [Medline]
  20. Ferrara E. What types of COVID-19 conspiracies are populated by Twitter bots? First Monday 2020 May 19;25(6):10633 [FREE Full text] [CrossRef]
  21. Kitzinger J. Qualitative research. Introducing focus groups. BMJ 1995 Jul 29;311(7000):299-302 [FREE Full text] [CrossRef] [Medline]
  22. Oates C. The use of focus groups in social science research. In: Burton D, editor. Research training for social scientists. Thousand Oaks, CA: Sage Publishing; 2000:186-195.
  23. Stewart K, Williams M. Researching online populations: the use of online focus groups for social research. Qual Res 2016 Aug 17;5(4):395-416. [CrossRef]
  24. Tuttas CA. Lessons learned using Web conference technology for online focus group interviews. Qual Health Res 2015 Jan 05;25(1):122-133. [CrossRef] [Medline]
  25. Brüggen E, Willems P. A critical comparison of offline focus groups, online focus groups and E-Delphi. Int J Market Res 2018 Apr 25;51(3):1-15. [CrossRef]
  26. Schneider SJ, Kerwin J, Frechtling J, Vivari BA. Characteristics of the discussion in online and face-to-face focus groups. Soc Sci Comput Rev 2016 Aug 18;20(1):31-42. [CrossRef]
  27. Woodyatt CR, Finneran CA, Stephenson R. In-person versus online focus group discussions: a comparative analysis of data quality. Qual Health Res 2016 May 01;26(6):741-749. [CrossRef] [Medline]
  28. Lijadi AA, van Schalkwyk GJ. Online Facebook focus group research of hard-to-reach participants. Int J Qual Meth 2015 Dec 11;14(5):160940691562138. [CrossRef]
  29. Abrams KM, Wang Z, Song YJ, Galindo-Gonzalez S. Data richness trade-offs between face-to-face, online audiovisual, and online text-only focus groups. Soc Sci Comput Rev 2014 Jun 03;33(1):80-96. [CrossRef]
  30. Chen J, Neo P. Texting the waters: an assessment of focus groups conducted via the WhatsApp smartphone messaging application. Method Innov 2019 Nov 04;12(3):205979911988427. [CrossRef]
  31. Colom A. Using WhatsApp for focus group discussions: ecological validity, inclusion and deliberation. Qual Res 2021 Jan 21;22(3):452-467. [CrossRef]
  32. Yip W, Ge L, Ho AHY, Heng BH, Tan WS. Building community resilience beyond COVID-19: The Singapore way. Lancet Reg Health West Pac 2021 Feb;7:100091 [FREE Full text] [CrossRef] [Medline]
  33. Neo PH, Lim JM, Tan RK, Ong SE. Using WhatsApp Focus Group Discussions for Qualitative Data Collection: Exploring Knowledge, Attitudes, and Perceptions of COVID-19 in Singapore. Int J Qual Methods 2022 Apr 10;21:16094069221090355 [FREE Full text] [CrossRef] [Medline]
  34. Digital 2021 Singapore. We Are Social. 2021.   URL: https://wearesocial.com/sg/blog/2021/01/digital-2021-singapore/ [accessed 2022-11-07]
  35. Braun V, Clarke V. Successful qualitative research: a practical guide for beginners. London: Sage Publishing; 2013.
  36. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
  37. Morrow SL. Quality and trustworthiness in qualitative research in counseling psychology. J Counsel Psychol 2005 Apr;52(2):250-260. [CrossRef]
  38. Singapore COVID-19 Symptom Checker.   URL: https://www.sgcovidcheck.gov.sg/ [accessed 2022-11-07]
  39. Li Q, Wang H, Li X, Zheng Y, Wei Y, Zhang P, et al. The role played by traditional Chinese medicine in preventing and treating COVID-19 in China. Front Med 2020 Oct 10;14(5):681-688 [FREE Full text] [CrossRef] [Medline]
  40. Charan J, Bhardwaj P, Dutta S, Kaur R, Bist SK, Detha MD, et al. Use of complementary and alternative medicine (CAM) and home remedies by COVID-19 patients: a telephonic survey. Indian J Clin Biochem 2021 Jan 31;36(1):108-111 [FREE Full text] [CrossRef] [Medline]
  41. Weeks J. Call to Action: announcing the traditional, complementary and integrative health and medicine COVID-19 support registry. J Altern Complement Med 2020 Apr 01;26(4):256-258. [CrossRef] [Medline]
  42. Shankar A, Dubey A, Saini D, Prasad CP. Role of complementary and alternative medicine in prevention and treatment of COVID-19: an overhyped hope. Chin J Integr Med 2020 Aug 05;26(8):565-567 [FREE Full text] [CrossRef] [Medline]
  43. Ganguly S, Bakhshi S. Traditional and complementary medicine during COVID-19 pandemic. Phytother Res 2020 Dec 12;34(12):3083-3084 [FREE Full text] [CrossRef] [Medline]
  44. Yang Q, Chen Y, Wendorf Muhamad J. Social Support, Trust in Health Information, and Health Information-Seeking Behaviors (HISBs): a study using the 2012 Annenberg National Health Communication Survey (ANHCS). Health Commun 2017 Sep 02;32(9):1142-1150. [CrossRef] [Medline]
  45. Hou J, Shim M. The role of provider-patient communication and trust in online sources in internet use for health-related activities. J Health Commun 2010 Dec 10;15(Sup 3):186-199. [CrossRef] [Medline]
  46. Chen X, Hay JL, Waters EA, Kiviniemi MT, Biddle C, Schofield E, et al. Health literacy and use and trust in health information. J Health Commun 2018 Aug 30;23(8):724-734 [FREE Full text] [CrossRef] [Medline]
  47. Berens E, Vogt D, Messer M, Hurrelmann K, Schaeffer D. Health literacy among different age groups in Germany: results of a cross-sectional survey. BMC Public Health 2016 Nov 09;16(1):1151 [FREE Full text] [CrossRef] [Medline]
  48. von Wagner C, Knight K, Steptoe A, Wardle J. Functional health literacy and health-promoting behaviour in a national sample of British adults. J Epidemiol Community Health 2007 Dec 01;61(12):1086-1090 [FREE Full text] [CrossRef] [Medline]
  49. Sørensen K, Pelikan JM, Röthlin F, Ganahl K, Slonska Z, Doyle G, HLS-EU Consortium. Health literacy in Europe: comparative results of the European health literacy survey (HLS-EU). Eur J Public Health 2015 Dec 05;25(6):1053-1058 [FREE Full text] [CrossRef] [Medline]
  50. Sbaffi L, Rowley J. Trust and credibility in web-based health information: a review and agenda for future research. J Med Internet Res 2017 Jun 19;19(6):e218 [FREE Full text] [CrossRef] [Medline]
  51. Kim Y. Trust in health information websites: a systematic literature review on the antecedents of trust. Health Informatics J 2016 Jun 16;22(2):355-369 [FREE Full text] [CrossRef] [Medline]
  52. Peterson E, Chou WYS, Kelley DE, Hesse B. Trust in national health information sources in the United States: comparing predictors and levels of trust across three health domains. Transl Behav Med 2020 Oct 08;10(4):978-988 [FREE Full text] [CrossRef] [Medline]
  53. Jackson DN, Peterson EB, Blake KD, Coa K, Chou WS. Americans' trust in health information sources: trends and sociodemographic predictors. Am J Health Promot 2019 Nov 23;33(8):1187-1193. [CrossRef] [Medline]
  54. McIntyre L. Post-Truth. Cambridge, MA: MIT Press; 2018.
  55. Waisbord S. Truth is what happens to news. Journal Stud 2018 Jul 06;19(13):1866-1878. [CrossRef]


DORSCON: Disease Outbreak Response System Condition
FGD: focus group discussion
WHO: World Health Organization


Edited by A Kushniruk; submitted 05.05.22; peer-reviewed by M Popovic, SF Tsao; comments to author 17.08.22; revised version received 05.09.22; accepted 10.09.22; published 22.11.22

Copyright

©Rayner Kay Jin Tan, Jane Mingjie Lim, Pearlyn Hui Min Neo, Suan Ee Ong. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 22.11.2022.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.