A cross-sectional study was conducted.

Participants were recruited between May 2023 and April 2024 via convenience sampling. Recruitment posters with QR codes were disseminated on social networks specific to health care professionals. In addition, health care professionals attending local academic conferences were invited to participate. The inclusion criteria for participants were (1) employment within a health care organization and (2) consent to participate. The exclusion criteria were (1) individuals who were retired or had less than 1 year of work experience; (2) health care students; (3) individuals who completed the questionnaire in less than 150 seconds, as this indicated random clicking; or (4) individuals who displayed erratic response patterns.

According to the thumb rule, the sample size should be 5 to 10 times the number of items. With an anticipated 10% rate of invalid responses, a minimum of 231 samples was necessary. Finally, a total of 398 cases were included in the study. To assess the test-retest reliability of the instrument, 43 participants who provided contact information completed the questionnaire again after a 2-week interval.

The DigiHealthCom instrument comprises 5 domains, comprising a total of 42 items—competence in human-centered remote counseling (16 items), digital solutions as part of work (9 items), competence in ICT (5 items), competence in using and evaluating digital solutions (8 items), and ethical competence related to digital solutions (4 items). Each item was rated on a 4-point Likert scale (1=completely disagree, 2=partially disagree, 3=partially agree, and 4=completely agree). For each domain, a mean value of ≤2.49 indicated low competence, 2.50‐3.49 indicated intermediate competence, and ≥3.50 indicated high competence [20]. DigiHealthCom has been validated among health care professionals across tertiary, primary, and private health care settings (n=817), demonstrating satisfactory internal consistency (Cronbach α=0.80‐0.97) and content validity (item content validity index [I-CVI]: 0.77‐1.00; S-CVI/Ave: 0.94) [19].

Data were collected using the Wen Juan Xing (a Chinese web questionnaire platform) [22]. Respondents accessed the questionnaire by scanning a QR code or clicking a link, with 1 response allowed per IP address to prevent duplicates. The questionnaire platform performed completeness checks before submission. The Chinese version of DigiHealthCom was presented alongside a sociodemographic questionnaire. The questionnaire consisted of 2 pages and 73 items. Sex, age, education, service area, professional license, work experience, type of organization, and frequency of patient work were collected. Participants who voluntarily chose to provide contact information participated in a retest after a 2-week interval. An introduction outlined the study’s purpose and provided questionnaire instructions.

Item analysis was used to evaluate item discrimination, item correlation, and item homogeneity. For item discrimination, respondents were classified into high-score (top 27%) and low-score (bottom 27%) groups based on their total scores. An independent t test determined whether each item could significantly distinguish between these groups. Items with a critical ratio (|t|) less than 3.0 were considered for exclusion [23]. In addition, item homogeneity and item correlation were tested using Cronbach α if the item was deleted and corrected item-total correlation coefficient. Items with a corrected item-total correlation coefficient below 0.40 were considered for exclusion [23]. Cronbach α and Intraclass correlation coefficient (ICC) were used to assess the internal consistency and test-retest reliability of the instrument. A Cronbach α≥0.70 indicated good internal consistency, while ICC>0.70 indicated good time stability [23]. The I-CVI and the Scale CVI/Average (S-CVI/Ave) were used to evaluate content validity of the instrument. I-CVI≥0.78 and S-CVI/Ave≥0.90 indicate satisfactory content validity [24].

Construct validity was evaluated using confirmatory factor analysis (CFA) with a robust chi-square to df ratio (χ²/df) less than 3, Tucker-Lewis Index (TLI), Incremental Fit Index (IFI), and Comparative Fit Index (CFI) greater than 0.90, and root-mean-square error of approximation (RMSEA) and standardized root-mean-square residual (SRMR) less than 0.08, indicating an acceptable data-model fit [25]. The average variance extracted (AVE) and composite reliability (CR) were used to assess convergent validity, with AVE greater than 0.50 and CR greater than 0.70 indicating good convergent validity [26]. Discriminant validity was performed using the heterotrait-monotrait ratio (HTMT), with a correlation matrix value <0.90 considered good [26].

Furthermore, participants were divided into 2 groups based on geographic location for the sensitivity analysis. The DigiHealthCom scores were compared to evaluate potential selection bias. The absence of a significant difference in digital health competence between the groups indicates that selection bias is unlikely in the study sample. Statistical analysis was conducted using IBM SPSS (version 27.0), IBM AMOS (version 29.0), and Smart PLS (Version 4.1.0.0; GmbH Corp). A P value of less than .05 was considered statistically significant.

Ethical approval was obtained from the Ethical Committee at Nanfang Hospital, Southern Medical University, China (NFEC-2023‐165). This research adhered to the principals of the Declaration of Helsinki. All participants provided informed consent and voluntarily completed the web questionnaire. Participants had the option to skip questions, review, and delete their responses. Participants had the right to withdraw from the survey at any time. Those who completed it received a random monetary reward ranging from CNY 2 to 5 (US $0.28 to 0.69). Participants’ rights and researcher’s contact information were provided on the first page of the web survey. Minimal sociodemographic was collected to maintain ethical standards. Participant information was kept confidential and anonymous. The CHERRIES (Checklist for Reporting Results of Internet E-Surveys; Checklist 1) was used to enhance the transparency of the study [27].

Initially, 431 participants were recruited for this study. Furthermore, 33 participants were subsequently excluded due to not meeting the inclusion criteria (n=7), having less than 1 year work experience (n=4), completing the questionnaire in under 150 seconds (n=2), and exhibiting erratic response patterns (n=20). Finally, 398 eligible health care professionals were included. Figure 2 illustrates the participant enrollment flowchart. The participants enrolled from 36 cities across 15 provinces, as identified by IP addresses. Among the participants, 249 (62.6%) were recruited from Guangzhou, a megacity in Guangdong province that contains 6125 registered medical facilities. As shown in Table 1, 364 (91.5%) participants were female, 386 (97%) worked in health care services, and 357 (89.7%) were nurses. The participants had an average of 13.7 (SD 9.4) years of work experience, with 281 (70.6%) working directly with patients for at least 5 days per week. Participant characteristics are summarized in Table 1.

Item analysis showed a significant difference between high-score and low-score groups. The critical ratio values for all items were above 3.0 (range 16.05‐23.77, P<.001; Table 2), indicating excellent item discrimination. All corrected item-total correlation coefficients exceeded 0.4 (Table 2). The Cronbach α if the item deleted (range 0.91‐0.96) were acceptable (Table 2).

As shown in Table 3, internal consistency (Cronbach α=0.94‐0.98) and test-retest reliability were good (ICC=0.90; 95% CI 0.81‐0.95).

To test content validity, 11 experts evaluated the relevance of the items. The S-CVI/Ave for the instrument was 0.97, and the I-CVI ranged from 0.82 to 1.00 (Table 2), indicating satisfactory content validity.

As illustrated in Figure 3, the results of CFA supported the 5-factors structure (digital solutions as part of work; ethical competence related to digital solutions; ICT competence; human-centered remote counselling competence; competence in using and evaluating digital solutions). The indices, including χ²/df (3.10), CFI (0.91), TLI (0.90), IFI (0.91), RMSEA (0.07), and SRMR (0.05), indicated an acceptable model fit. The factor loadings of items ranged from 0.69 to 0.93 (Figure 3). The AVE values for each dimension exceeded 0.50 (range: 0.60‐0.79), with the CR values >0.70 (range: 0.94‐0.95), indicating excellent convergent validity. The HTMT values within the matrix were less than 0.90 (range: 0.72‐0.89), indicating good divergent validity. These results collectively demonstrated satisfactory construct validity of the instrument.

Participants were divided into 2 groups based on geographic location: Southern China and Northern and Western China. The DigiHealthCom scores were compared to evaluate potential selection bias. As illustrated in Table S3 in Multimedia Appendix 2, no significant difference was found in DigiHealthCom scores between the 2 groups (P>.05), indicating that selection bias is unlikely in the study sample.

In this study, we translated and culturally adapted the DigiHealthCom instrument into Chinese and assessed its reliability and validity among Chinese health care professionals. The Chinese version of DigiHealthCom demonstrated satisfactory internal consistency, test-retest reliability, content validity, and construct validity. To our knowledge, this study represents the first effort to validate a comprehensive Chinese tool for measuring digital health competence among health care professionals.

In this study, each item of the Chinese version of DigiHealthCom was translated and back-translated strictly following the dual direct-to-back translation model [21] to ensure alignment in semantic, conceptual, and content with the original English version. And then, we conducted expert review involving 11 experts in nursing and IT to examine content validity of the Chinese version. High content validity indices imply that the instrument provides a broad enough range of content to allow conclusions about the targeted construct. We also conducted cognitive interviewing and a pilot study to ensure its clarity and comprehensibility [28]. Issues related to semantic validation and understanding identified in the initial Chinese version were addressed after receiving feedback from the expert review and cognitive interviewing, leading to improvements such as clarifying complex terms and vague definitions. Item analysis revealed good differentiation and high correlations between the items in this study. Furthermore, this instrument underwent rigorous testing for internal consistency, test-retest reliability, and construct validity, exhibiting excellent internal consistency, time stability, and construct validity, resonating with the original study [19]. The types of reliability and validity assessed in this study represent essential psychometric properties for a measurement instrument.

Discrepancies between existing instruments largely stem from differing conceptual frameworks. The DHLI measures 7 individual skills: operation, navigation, information searching, evaluating reliability, determining relevance, adding self-generated content, and protecting privacy [16]. The eHLQ comprises 7 dimensions, addressing users’ attributes, their interaction with technologies, and their experience with systems [17]. Our CFA findings confirm a 5-factor structure, consistent with the exploratory factor analysis findings of the initial study [19]. Beyond competence in using and evaluating digital solutions and ICTs, the DigiHealthCom instrument addresses additional essential domains that were previously unaddressed, that is acceptance of digital solutions, human-centered remote counseling, digital interaction skills with patients and interprofessional teams, and ethical competence related to digital solutions [19]. Notably, acceptance of digital solutions significantly influences health care professionals’ adoption of digital solutions [13,14]. Competence in person-centered remote consultations is crucial for fostering patient engagement and improving accessibility to equitable digital health services [8]. Furthermore, proficient digital interaction skills facilitate effective coordination in digital settings, thereby supporting decision-making and optimizing treatment plans [8]. As digital solutions become more prevalent, attention to data privacy and information security has intensified [29,30]. Ethical competence related to digital solutions ensures appropriate management of patient information, fostering trust in digital health technologies and enhancing the safety of digital health services [20,31]. These domains provide a unified theoretical framework for comprehensively measuring digital health competencies among various professionals [8,9,19], which are essential for delivering high-quality digital health services to meet future demands.

The Finnish health care system is renowned for its universality, robust public funding, and centralized digital infrastructure [32]. Regional variations exist in Finland; for example, northern districts have fewer professionals specializing in remote counseling and the integration of digital solutions compared with southern areas [20,33]. Strategies have been implemented to enhance health care professionals’ digital health competence and improve education related to health care digitalization, including technology to support client engagement, digital services integrated into nursing work, and considerations of safety and ethics in the digital environment [20,34]. In contrast, China experiences regional disparities in digital health infrastructure and access to digital health services, especially in rural areas, despite the rapid growth of digital health services [35]. In addition, system compatibility across hospitals remains an unsolved issue [35]. These factors may influence the development of digital health competence among health care professionals. A cross-sectional study conducted in a central Chinese province reported that 49.9% (1690/3386) of clinical nurses demonstrate low telehealth readiness [36]. Consequently, evaluating, addressing, and reducing regional disparities in health care professionals’ digital health competence is vital for promoting equity in health care service provision. Validating a comprehensive digital health competence measurement instrument for health care professionals is imperative to address challenges posed by diverse health care environments, such as China’s. The Chinese version of DigiHealthCom is now ready for application in a wide range of settings and various health care professionals. Researchers, educators, health care providers, and policy makers can use it to evaluate digital health competence among diverse health care professionals, and develop digital health training curricula and policies based on the assessment.

This study acknowledges several limitations. First, the primary participants were nurses. Future research should involve a broader spectrum of health care professionals. Second, although recruited from diverse health care institutions, the participants mainly came from southern China, which may introduce selection bias due to regional disparities in digital health infrastructure. To address this, participants were divided into 2 groups (Southern China versus Northern and Western China) for sensitivity analysis, and their DigiHealthCom scores were compared. We did not find significant differences in digital health competence between the 2 groups, indicating that selection bias is unlikely in the study sample. Future studies could benefit from employing a stratified sampling method. Third, the absence of a standardized method for assessing digital health competence among health care professionals hindered the evaluation of criterion validity. Assessments of digital health competence predominantly rely on subjective self-reports, which might be susceptible to response bias. However, relying exclusively on performance-based assessments or other objective measures to evaluate competence also presents challenges. Therefore, using a more holistic combination of methods could offer a viable solution. Future research should consider larger sample sizes and multicenter external evaluations to address these limitations.

In conclusion, we have successfully translated, culturally adapted, and validated DigiHealthCom for Chinese health care professionals. Our findings demonstrate that the Chinese version of DigiHealthCom is a reliable and valid instrument for assessing digital health competence among these health care professionals.