This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.
Visual expertise refers to advanced visual skills demonstrated when performing domain-specific visual tasks. Prior research has emphasized the fact that medical experts rely on such perceptual pattern-recognition skills when interpreting medical images, particularly in the field of electrocardiogram (ECG) interpretation. Analyzing and modeling cardiology practitioners’ visual behavior across different levels of expertise in the health care sector is crucial. Namely, understanding such acquirable visual skills may help train less experienced clinicians to interpret ECGs accurately.
This study aims to quantify and analyze through the use of eye-tracking technology differences in the visual behavior and methodological practices for different expertise levels of cardiology practitioners such as medical students, cardiology nurses, technicians, fellows, and consultants when interpreting several types of ECGs.
A total of 63 participants with different levels of clinical expertise took part in an eye-tracking study that consisted of interpreting 10 ECGs with different cardiac abnormalities. A counterbalanced within-subjects design was used with one independent variable consisting of the expertise level of the cardiology practitioners and two dependent variables of eye-tracking metrics (fixations count and fixation revisitations). The eye movements data revealed by specific visual behaviors were analyzed according to the accuracy of interpretation and the frequency with which interpreters visited different parts/leads on a standard 12-lead ECG. In addition, the median and SD in the IQR for the fixations count and the mean and SD for the ECG lead revisitations were calculated.
Accuracy of interpretation ranged between 98% among consultants, 87% among fellows, 70% among technicians, 63% among nurses, and finally 52% among medical students. The results of the eye fixations count, and eye fixation revisitations indicate that the less experienced cardiology practitioners need to interpret several ECG leads more carefully before making any decision. However, more experienced cardiology practitioners rely on their skills to recognize the visual signal patterns of different cardiac abnormalities, providing an accurate ECG interpretation.
The results show that visual expertise for ECG interpretation is linked to the practitioner’s role within the health care system and the number of years of practical experience interpreting ECGs. Cardiology practitioners focus on different ECG leads and different waveform abnormalities according to their role in the health care sector and their expertise levels.
Visual expertise refers to advanced visual skills demonstrated when executing domain-specific visual tasks [
The importance of conducting this study stems from the fact that the ECG is one of the most used medical tests in modern medicine, reaching over 300 million ECGs done annually in the United States alone [
This study extends the results of our initial work [
Related works focusing on the relationship between visual expertise, ECG interpretation [
There exists a significant quantifiable difference in the accuracy of the interpretation of each expertise level category of participants as they gain more years of experience.
There is a significant correlation between the number of years of participant’s experience, depicted by their cardiology practitioner roles, and their fixations’ behavior around specific areas of the ECG, demonstrated by the fixations count.
There is a significant correlation between the number of years of participant’s experience, depicted by their cardiology practitioner roles, and their eye movement transition frequency between different parts/leads on the standard 12-lead ECG, demonstrated by fixation revisitations.
The conducted study uses eye-tracking technology to quantify and understand differences in human visual behavior during ECG interpretation. With different clinical roles in the health care sector, recruited participants were tasked with interpreting 10 ECGs with different types of cardiac abnormalities. During their interpretation, their eye movements were recorded using an eye tracker and the collected eye movements data was analyzed quantitatively. Participants were also tasked with selecting their final diagnosis for each ECG from among four available choices or writing down a diagnosis other than those proposed. The choices for each ECG are available in
The experiment used a counterbalanced within-subjects design with the following one independent variable: the expertise level of the cardiology practitioner. This can be quantified as a categorical variable based on the number of years of ECG interpretation experience, as described in
Two measured dependent eye-tracking variables were expected to change when the independent variable changed. These two variables are measured according to our definition of grid-based areas of interest (AOIs). A sample grid-based AOI applied to the normal sinus rhythm ECG can be referred to in
The average fixations count for each ECG lead for each category of participant
The average fixation revisitations for each ECG lead for each category of participant
The experiment also had one control variable. The time given for each participant to look at each ECG was limited to 30 seconds. This time limit allowed for all categories of participants to be held to the same standards in terms of the amount of time given for them to analyze each case. This time limit was chosen by consulting the cardiology consultant and professor involved in designing this experiment. The time is also supported by studies investigating the choice of this parameter within different categories of medical practitioners such as medical students and consultants. The time allowed for scanning an ECG was found to have no statistically significant effect on the result of the diagnosis [
Corresponding variables to each hypothesis.
Hypothesis | Independent variable | Dependent variable |
Hypothesis 1 | Years of experience | Accuracy of interpretation |
Hypothesis 2 | Years of experience | Fixations count |
Hypothesis 3 | Years of experience | Fixation revisitations |
Junior medical students: those in a preclinical curriculum
Senior medical students: those in a clinical curriculum
Nurses: nurses either serving in the catheterization laboratory or the cardiac care unit
Technicians: cardiovascular technologists working in a cardiac catheterization laboratory
Fellows: physicians undergoing postgraduate training in cardiology
Cardiology consultants: board-certified independent cardiology practitioners
The ECG stimuli were acquired from the collection belonging to the cardiology consultant involved in designing the experiment. Since the study is motivated by quantifying visual behavior across different expertise levels of different health care practitioners, we selected ECGs commonly encountered by all those categories in their day-to-day medical practice [
A Tobii Pro X2-60 eye tracker and iMotions version 8.1 software [
This study received institutional review board approval from the ethical board of both the Qatar Biomedical Research Institute at Hamad bin Khalifa University [
The three hypotheses regarding participants’ visual behavior toward ECG interpretation were tested as follows.
To test the first hypothesis, interpretations were assessed for participants’ accuracy by determining if they chose the correct ECG diagnosis from among the four offered choices. Analyzing the participants’ accuracy of interpretation scores using the Cramér V statistical test contributed toward constructing a clear understanding of how much the interpreters understood the ECG signals and its waveform abnormalities presented to them throughout the 10 ECG cases.
To test the second hypothesis, interpretations were assessed for the frequency with which the participants fixated on ECG images. This assessment was done by comparing eye movement behavior for the five categories. Eye movement was quantified using a median fixations count for each participant. A prior study showed that the average duration for one fixation ranged from 150 to 300 milliseconds [
To test the third hypothesis, interpretations were assessed for the frequency with which the interpreters revisit different areas, or leads, in the ECG. This was done by comparing each participant’s average ECG lead revisitation among the five categories. A
Accuracy of the participants’ answers across the 10 showcased ECGs. ECG: electrocardiogram.
Demographics for the participants included in the eye-tracking study.
Feature and demographics | Participants, n | |
|
||
|
Medical students (junior year) | 9 |
|
Medical students (senior year) | 10 |
|
Fellows | 11 |
|
Technicians | 10 |
|
Nurses | 14 |
|
Consultants | 9 |
|
||
|
20-23 | 10 |
|
23-25 | 9 |
|
26-30 | 21 |
|
30-35 | 11 |
|
35-45 | 12 |
|
||
|
Male | 51 |
|
Female | 12 |
|
||
|
0 years | 10 |
|
1 year | 9 |
|
2-5 years | 15 |
|
5-10 years | 22 |
|
≥15 years | 7 |
Median total fixations count per participant and median fixation count per lead per participant.
Category | Fixations count per participant (for all ECGsa) | Fixation count per lead for each ECG | |||
|
Median | SD from the IQR | Median | SD from the IQR | |
Medical students | 2829 | 1411 | 9.93 | 5.01 | |
Technicians | 2535 | 301 | 10.83 | 1.27 | |
Nurses | 2444 | 1031 | 9.49 | 3.96 | |
Fellows | 2135 | 579 | 9.12 | 2.52 | |
Consultants | 1385 | 794 | 6.57 | 3.95 |
aECG: electrocardiogram.
Average electrocardiogram (ECG) lead revisitation per participant for every category.
Category | ECG lead revisitation per participant | |
|
Mean (μ) | SD (σ) |
Technician | 3.61 | 0.06 |
Nurse | 3.25 | 1.60 |
Medical students | 2.90 | 0.85 |
Fellow | 2.55 | 0.67 |
Consultant | 2.01 | 0.98 |
The results indicate that the interpreter’s expertise, revealed by the number of years of work experience in ECG interpretation, is the primary influence for both the accuracy of ECG interpretation and the acquired visual expertise strategies. Through the analysis of the three hypotheses, three main findings were confirmed.
First, the accuracy of ECG interpretation correlates with the expertise level of the participant. The results confirm the first hypothesis by indicating that consultants are the category with the most accurate interpretations, while medical students are the category with the least accurate interpretations. In between these two extremes are nurses, technicians, and fellows.
Second, as expertise for participants increases, participants’ fixations count on ECG signal waveform abnormalities decreases. This finding translates into participants fixating on the overall ECG for less time while not compromising the accuracy of the interpretation. This finding confirms the second hypothesis.
Third, the results for testing hypothesis 3 confirm that medical practitioners observe and focus on certain ECG leads and waveform abnormalities according to their role in the health care sector and their expertise level.
Sample aggregate heat maps showing differences in fixation distribution across the left bundle branch block ECG between the studied categories. ECG: electrocardiogram.
Sample aggregate heat maps showing differences in fixation distribution across the complete heart block ECG between the studied categories. ECG: electrocardiogram.
This paper presents a quantitative analysis of the ECG interpretation visual behavior of different health care practitioners. These health care practitioners belonged to five different categories: medical students, nurses, technicians, fellows, and consultants. Eye-tracking data for these categories were collected while they each interpreted a total of 10 ECGs. Specific eye-tracking metrics such as fixations count and fixation revisitations were quantitatively analyzed for each lead in the standard 12-lead ECG and across all ECGs. This analysis was done to meet the objective of quantifying, using eye tracking, medical practitioners’ visual expertise strategies in ECG interpretation as they advance in their medical careers. The main findings relate to how accurate each medical category is in ECG interpretation when considering their eye movements and visual behavior. The accuracy of the final ECG diagnosis was also associated with the expertise level of participants. Moreover, the increased level of participant expertise made experienced practitioners require less time to fixate on ECG abnormalities and decreased fixation counts, leading to correct diagnoses. Lastly, medical practitioners focus on certain ECG leads and specific waveform abnormalities according to their role in the health care sector and their expertise level.
Since eye-tracking data is idiosyncratic to every interpreter, a sample size of approximately 60 participants from different categories may not be representative enough. Sample size determination depends on what the designers of the study aim to represent. Recruited sample size may therefore vary according to the targeted population, CIs, and interpreters’ confidence level in their responses. Based on these uncontrollable factors, recruiting more participants and increasing the number of medical practitioner categories are necessary. We addressed this by recruiting a diverse and reasonable number of health care practitioners, but including larger numbers of participants in future work would contribute to a better understanding of visual expertise in ECG interpretation and understanding how different health care practitioners with different roles and expertise levels interpret ECGs. The richness of the study’s collected eye movement data has the potential to be further analyzed using machine learning algorithms to deeply reveal differences in visual behavior among the different categories of medical practitioners. We also plan on experimenting with more subtle examples of ECG diagnoses such as nonspecific/incomplete abnormalities and see how the experts deal with conflicting or vague data.
Multiple choice questions for the electrocardiogram eye-tracking experiment.
Sample grid-based areas of interest applied to the normal sinus rhythm electrocardiogram.
Definition of electrocardiogram samples used in the eye-tracking experiment.
Heat maps for all the categories of interpreters and all the electrocardiograms.
area of interest
electrocardiogram
None declared.