Related works focusing on the relationship between visual expertise, ECG interpretation [3,8-11], and other clinical fields [4-7] requiring medical images interpretation were taken into consideration before creating the following hypotheses. The eye-tracking study by Davies et al [10] especially inspired our second and third hypothesis as the authors noted that experienced interpreters adopt a duel processing model of ECG interpretation. Additionally, the study by Wu et al [3] also emphasized this nuance among different categories of medical practitioners. The following are our three hypotheses:

The conducted study uses eye-tracking technology to quantify and understand differences in human visual behavior during ECG interpretation. With different clinical roles in the health care sector, recruited participants were tasked with interpreting 10 ECGs with different types of cardiac abnormalities. During their interpretation, their eye movements were recorded using an eye tracker and the collected eye movements data was analyzed quantitatively. Participants were also tasked with selecting their final diagnosis for each ECG from among four available choices or writing down a diagnosis other than those proposed. The choices for each ECG are available in Multimedia Appendix 1.

The experiment used a counterbalanced within-subjects design with the following one independent variable: the expertise level of the cardiology practitioner. This can be quantified as a categorical variable based on the number of years of ECG interpretation experience, as described in Table 1. Medical practitioners may also be placed in one of the clinical categories/roles highlighted in Table 1.

Two measured dependent eye-tracking variables were expected to change when the independent variable changed. These two variables are measured according to our definition of grid-based areas of interest (AOIs). A sample grid-based AOI applied to the normal sinus rhythm ECG can be referred to in Multimedia Appendix 2. Our explanation behind our choice for the grid-based AOIs can be found in our previous work [14]. The following are the two dependent variables:‬‬‬‬

The experiment also had one control variable. The time given for each participant to look at each ECG was limited to 30 seconds. This time limit allowed for all categories of participants to be held to the same standards in terms of the amount of time given for them to analyze each case. This time limit was chosen by consulting the cardiology consultant and professor involved in designing this experiment. The time is also supported by studies investigating the choice of this parameter within different categories of medical practitioners such as medical students and consultants. The time allowed for scanning an ECG was found to have no statistically significant effect on the result of the diagnosis [20]. It was also found that there is a negative correlation between the duration spent looking at an ECG and the accuracy of the final interpretation provided [21].

Table 1 summarizes the demographics of the participants included in this study. A total of 63 participants with varying ECG interpretation expertise were recruited from a university campus and a cardiac hospital. Participants were recruited based on their medical category represented by their job title/role in clinical practice. The mean age was 28 (SD 4) years. In addition, participants were asked to provide an approximation of their years of work experience in ECG interpretation. The medical categories are defined as follows:

The ECG stimuli were acquired from the collection belonging to the cardiology consultant involved in designing the experiment. Since the study is motivated by quantifying visual behavior across different expertise levels of different health care practitioners, we selected ECGs commonly encountered by all those categories in their day-to-day medical practice [22]. The ECGs sampled are defined in Multimedia Appendix 3. We limited our selection to 10 representative ECG cases.

A Tobii Pro X2-60 eye tracker and iMotions version 8.1 software [23] were used to track eye movements with a frequency of 60 Hz (±1 Hz). In addition, keyboard presses and mouse input were recorded to register the participants’ responses showing their final diagnosis for each ECG. The study was conducted on a 25-inch diagonal laptop monitor with a resolution of 1366 by 768 pixels.

This study received institutional review board approval from the ethical board of both the Qatar Biomedical Research Institute at Hamad bin Khalifa University [24] under the research protocol number QBRI-IRB-2020-01-009 and the Hamad Medical Corporation under the research protocol number MRC-02-20-714. Approvals were granted before the start of the experiment. Institutional review board approval guarantees that all study methods were conducted following the guidelines and recommendations of international regulatory agencies [24].

Figure 1 summarizes the accuracy of the participants’ answers across the 10 showcased ECGs. Consultants are the most accurate interpreters, with an accuracy percentage of 97.8%. Fellows are the second most accurate, with an overall accuracy of 87%, followed by technicians with an accuracy of 70%. Nurses were the least accurate of those with working experience, with an accuracy of 63%. Finally, medical students were overall the least accurate category with an accuracy of 52.2%. A chi-square test was conducted since the interpretation response data is dichotomous. The obtained P value was .02, which shows that there is a statistically significant difference in interpretation accuracy between the five categories. To calculate the effect size of the chi-square independence test, we used the Cramér V, providing a value of 0.36 that indicates a weak association between the categories. Although some ECGs were easy to interpret correctly (eg, the normal sinus rhythm), other ECGs, including the ventricular paced rhythm and the left bundle branch block, were harder to interpret correctly.

Table 2 summarizes the median and SD from the IQR for the total fixations count per participant for all leads in the ECG. The fixations data follows a nonnormal, left-skewed distribution. Total fixation counts were calculated for the 10 ECGs for 300 seconds. Table 2 also includes the median and SD from the IQR for the fixation count per lead across all 10 ECGs. Consultants are the category with the lowest number of fixations, while medical students have the highest number of fixations. Applying the Kruskal-Wallis test to the total fixations count data resulted in a P value of .03, which showed there is a statistically significant difference in participants’ fixation count.

Table 3 summarizes the median total fixations count per participant and median fixation count per lead per participant, while Table 4 summarizes the mean and SD of the ECG lead revisitations for each category of participants. The ECG lead revisitations data follows a normal distribution. On average, technicians are the category of participants with the highest number of revisitations for each lead with an average of 3.61 revisitations, while consultants are the category that revisits a lead the least with an average of 2.01 revisitations. However, based on the SD results, variation among participants in the same category was the highest among nurses and the lowest among technicians. A one-way analysis of variance test was applied to the data, showing an F value of 30.56, which is larger than the critical F value (2.36). We measured the effect size using the Eta squared formula and the result is η²=0.36.

The results indicate that the interpreter’s expertise, revealed by the number of years of work experience in ECG interpretation, is the primary influence for both the accuracy of ECG interpretation and the acquired visual expertise strategies. Through the analysis of the three hypotheses, three main findings were confirmed.

First, the accuracy of ECG interpretation correlates with the expertise level of the participant. The results confirm the first hypothesis by indicating that consultants are the category with the most accurate interpretations, while medical students are the category with the least accurate interpretations. In between these two extremes are nurses, technicians, and fellows.

Second, as expertise for participants increases, participants’ fixations count on ECG signal waveform abnormalities decreases. This finding translates into participants fixating on the overall ECG for less time while not compromising the accuracy of the interpretation. This finding confirms the second hypothesis.

Third, the results for testing hypothesis 3 confirm that medical practitioners observe and focus on certain ECG leads and waveform abnormalities according to their role in the health care sector and their expertise level. Figures 2 and 3 show sample aggregate heat maps for the differences in fixation distribution across the left bundle branch block and the complete heart block ECGs between the studied categories. The results from the third assessment aiming to confirm the third hypothesis indicate that both consultants and fellows target their fixations on specific leads to identify the correct ECG diagnosis. This finding was also confirmed by looking at the heat maps for different categories across different ECGs other than the ones in Figures 2 and 3. However, nurses and technicians thoroughly interpreted the ECGs systematically by scanning through all 12 leads and primarily looking at abnormalities in the ST segment and wide QRS complex. This finding explains the high number of fixations per lead measured by the fixations count. This behavior may be because nurses are usually not extensively trained to interpret the ECG the way cardiologists do for a diagnosis but to ensure that signs of imminent heart attacks are not missed. The heat maps of medical students indicate that they randomly fixate on the waveform abnormalities that they first perceive and then continue to transition from one lead to another until the 30-second time limit is over. This finding justifies medical students having the highest number of fixations per ECG case. Heat maps for all the categories of interpreters and all the ECGs can be found in Multimedia Appendix 4.