Frontline Learning Research Vol. 5 No. 3 Special
issue (2017) 43 - 54
ISSN 2295-3159
aBeth Israel Deaconess Medical
Center, Boston, MA, USA; Harvard Medical School, Boston, MA, USA
bLSU Health Sciences Center, New Orleans, LA, USA
Article received 7 May / revised 23 March / accepted 24 March / available online 14 July
Eye-tracking is the measurement of eye motions and point of gaze of a viewer. Advances in this technology have been essential to our understanding of many forms of visual learning, including the development of visual expertise. In recent years, these studies have been extended to the medical professions, where eye-tracking technology has helped us to understand acquired visual expertise, as well as the importance of visual training in various medical specialties. Medical decision-making involves a complex interplay between knowledge and sensory information, and the study of eye-movements can reveal the mechanisms involved in acquiring the visual component of these skills. Eye-tracking studies have even been extended to develop computational models of procedures for “expert” skill assessment, and to eliminate potential sources of error in image-based diagnostics. This review will examine the current eye-tracking frontier for the study of visual expertise, with specific application to medical professions.
Keywords: eye-tracking; visual expertise; digital imaging
Eye-movements and their meaning have long been the subject of scientific study, and recent advances in technology have allowed for the study of eye gaze – or eye-tracking – during the acquisition of complex forms of visual expertise. In this review of eye-tracking methodology, we will examine the way in which the science of acquired visual expertise, as well as the development of eye-tracking technology, have allowed us to better understand visual training in medicine. Training and expertise in medicine involves not only acquisition of knowledge, but also the integration of sensory information during the process of diagnosis and disease management. In many instances, that sensory information is visual, and the study of eye-movements can reveal not only the cognitive processes behind medical expertise, but also the mechanisms involved in acquiring these skills. Furthermore, studies of expert gaze patterns can help us to understand common perceptual pitfalls, and to develop technologies that may assist in training medical professionals and eliminate sources of error.
Eye movements represent one of the most frequent sensorimotor activities in humans. Large scanning movements, or saccades, typically occur 3-4 times per second (Holmqvist, 2011). The most frequently reported eye gaze metric, however, represents the relative “pause” between saccades, known as fixations. Individual fixations generally last approximately 200-300 milliseconds, with the fovea of the eye remaining relatively still along a point of gaze (Holmqvist, 2011). In the early 1970s, eye-tracking techniques were advanced by using video-based techniques, wherein recorded features of reflections of light from the eye could be systematically tracked. One option was to scan for the lack of reflectance from the pupil (“dark-pupil” tracking), although low contrast between the pupil and dark-brown irises led to suboptimal results. If the eye is lit from the front, the light will bounce off the back of the lens and appear very bright (“bright-pupil” tracking). This bright circle can then be more reliably detected than the dark-pupil technique (Merchant, Morrissette, & Porterfield, 1974; Rayner, 1978).
In addition to the ability to capture the motion of the eyes, a second and important component of eye-tracking for the study of vision is the ability to capture the visual stimulus in such a way that gaze can be determined. Most early eye-tracking studies involved significant restraint of the head for this purpose. An important innovation in this area was the development of eye-tracking systems that measured multiple features of the eye in order to infer position relative to a visual stimulus (Holmqvist, 2011). Optical properties of the eye, such as corneal reflection and pupil location, vary differently under conditions of head versus eye movement, and their recordings can be used to solve for the actual gaze point of the viewer (for additional discussion, (Wolfe, Evans, Drew, Aizenman, & Josephs, 2015)). By utilizing non-visible light, such as near-infrared light sources, such eye-trackers can be made less noticeable, and easier to use in a variety of lighting conditions. Such systems are particularly useful in the study of natural learning environments, and allow results to be generalized to real world situations.
The balance between obtaining a high-precision record of an observer's point-of-regard and allowing natural head- and body-movements is where much of the technological advancement in eye-tracking has arisen over the past twenty years, in addition to "scene cameras" that allow eye-tracking data to be superimposed on the naturalistic point-of-view of the participant during movement (Browatzki, Bulthoff, & Chuang, 2014; Huette, Winter, Matlock, & Spivey, 2012; Johnson, Liu, Thomas, & Spencer, 2007). Computer embedded and table-mounted remote optical eye-trackers have also been developed that allow some natural head movement while sitting in front of a computer screen for two-dimensional stimulus presentation. Such advances have placed fewer constraints on experimental subjects, such that modern researchers are able to record the eye movements of freely moving subjects carrying out everyday tasks. These features, combined with improvement in user interface, have allowed eye-tracking to be used in the study of visual training, and the application of visual training within the field of medicine. It is also possible that future advances in this technology could allow eye-tracking devices to assist visual processes in medical education and decision-making.
Eye-tracking provides a potential means for understanding the nature and acquisition of visual expertise as it relates to medical knowledge. As a method of assessing gaze patterns during task execution, eye-tracking has been used to understand which parts of an image are important to medical decision-making, and also the process which experts adopt to analyze these images. To date, the majority of the published eye-tracking studies involving medical expertise are naturally in the fields requiring extensive visual training, such as radiology (Drew, Evans, Vo, Jacobson, & Wolfe, 2013; Kundel, Nodine, Krupinski, & Mello-Thoms, 2008; G. Tourassi, Voisin, Paquit, & Krupinski, 2013; G. D. Tourassi, Mazurowski, Harrawood, & Krupinski, 2010; Wolfe et al., 2015). These studies often utilize static images, which are relatively easy to employ in eye-tracking experimental designs and analyses, but which don’t always translate to the visual environments of other clinical specialties. In recent years, additional studies have appeared in relation to fields such as dermatology, surgery, and anatomic pathology (Ahmidi, Ishii, Fichtinger, Gallia, & Hager, 2012; Bombari, Mora, Schaefer, Mast, & Lehr, 2012; Brunye et al., 2014; Fox, Law, & Faulkner-Jones, 2017; E. Krupinski, Chao, Hofmann-Wellenhof, Morrison, & Curiel-Lewandrowski, 2014; E. A. Krupinski, Graham, & Weinstein, 2013; E. A. Krupinski et al., 2006; Law, Atkins, Lomax, & Wilson, 2003; Tiersma, Peters, Mooij, & Fleuren, 2003). The ability to use dynamic stimuli in eye-tracking research (Ahmidi, Ishii, et al., 2012; Drew, Vo, Olwal, et al., 2013; Fox et al., 2017; Mallett et al., 2014), as well as the use of wearable and increasingly portable devices has expanded the opportunities in which eye-tracking can be utilized to understand the gain of visual expertise in clinical settings. In the study of medical education, most experimental designs involve two or three participants groups, each at a different level of medical training (Ahmidi, Ishii, et al., 2012; E. A. Krupinski et al., 2013; Mallett et al., 2014; Phillips et al., 2013). Analyses vary widely, however, with some reporting qualitative information, such as regions of an image with the greatest or least attention, and others using quantitative analyses of fixations in relation to defined areas of interest (AOIs) (Ahmidi, Ishii, et al., 2012; Brown et al., 2014; Fox et al., 2017; E. A. Krupinski et al., 2013; Mallett et al., 2014; Phillips et al., 2013). A recent, and notable study of the use of eye-tracking in medical education involved the use of expert eye-tracking data to train observational techniques (H. Jarodska et al., 2012), and will be further discussed in subsequent sections. Overall, eye-tracking studies across medical specialties have suggested that more experienced physicians require fewer fixations, and less time spent on areas of interest, while performing at a higher rate of accuracy than novices. The visual patterns identified by eye-tracking experiments, however, depend in part on the type of visual expertise acquired, and the integration of medical knowledge in that process. In order to understand how eye-tracking methodology may be applied to the study of visual expertise in medicine, we must first understand the differences between visual tasks in a variety of clinical scenarios.
Radiology is the most extensively studied field of medicine in relation to visual expertise, and it is not surprising that a significant literature involving eye-tracking has arisen in relation to this specialty (Drew, Evans, et al., 2013; Kundel et al., 2008; G. Tourassi et al., 2013; G. D. Tourassi et al., 2010; Wolfe et al., 2015). The visual expertise learned by radiologists is an example of a search-related task, in which the radiologist identifies visual “targets” in an image containing both expected and “distractor” elements (Wen et al., 2016; Wolfe et al., 2015). Screening tests, both in radiology and pathology, are generally performed using a visual search model (Stewart et al., 2007; Wolfe, 1995). In this visual task, search is required because everything in the visual field cannot be identified and processed simultaneously. Object recognition is limited to one, or a small number of objects at one time (Wolfe, 2012b; Wolfe et al., 2015). Attention may appear random, but is often guided by multiple cognitive mechanisms.
At the most basic level, exploratory visual gaze is directed towards items with “bottom-up” salience (Braun, 1994; Wolfe & Horowitz, 2004). These items do not depend on the purpose of the search, and therefore are not the result of acquired visual expertise. By contrast, gaze patterns of expert radiologists during search tasks may be less affected by bottom-up salience when these elements are known distracters from a visual target (Wolfe et al., 2015). Bottom-up salience of a visual item is determined by basic features, such as color, size, contrast, movement and luminosity (Braun, 1994; Wolfe & Horowitz, 2004; Wolfe, Horowitz, Kenner, Hyle, & Vasan, 2004). Wolfe et al. (2015) classifies these attributes as “pre-attentive,” because they do not require a specific goal or form of expert attention to bias patterns of gaze.
Through medical training, including knowledge acquisition and an understanding of the visual properties of targets, “top-down,” or user-guided visual search develops (Draschkow, Wolfe, & Vo, 2014; Wolfe, 1994). At this level of visual processing, the radiologist guides attention toward a mental representation of the features and potential location of a target (Wen et al., 2016). Top-down processing is also employed at the point of diagnosis, when the visual item is matched to both a mental depiction of the target, and general medical knowledge (Drew, Evans, et al., 2013; Drew, Vo, Olwal, et al., 2013). Wolfe et al. (1994) summarize this process in the “Guided Search Model,” in which bottom-up attention to distracters is at least partially suppressed by the top-down effects of visual expertise. The expert’s knowledge provides scene guidance to relevant parts of the image, which combines with these effects to create a mental representation of the likely location of targets (Wen et al., 2016). This course of attentional gaze may be modulated by systematic training algorithms for visual inspection (for example, a trained pattern of attention to each potential diagnostic target within a chest radiograph), as well as clinical information (Draschkow et al., 2014; Drew, Evans, et al., 2013; Wolfe et al., 2015; Wolfe et al., 2007). Eye-tracking has been utilized to better understand the development of a “priority map,” within a radiological image, and how this priority map may evolve during the diagnostic process. It is proposed that the priority map must change as the radiologist’s eyes move about the image as the salience of items will change with their distance from the present fixation (Wen et al., 2016). Interestingly, the development of visual expertise with training in radiology seems to indicate a move towards efficiency, and away from repeated attention to “non-priority” regions of the diagnostic image (G. Tourassi et al., 2013; Wolfe et al., 2015). Search for multiple visual targets, held in memory, is known as “hybrid search” (Wolfe, 2012a). Attention is not guided as effectively in this form of search, which may play a role in the training process of visual expertise in medicine (Eckstein, 2011). This effect can be highlighted as it relates to simulation studies in medical education. In a study of senior nursing students administering medications in a clinical simulation setting, 40% administered a contraindicated medication to a patient with a known allergy (B. Amster et al., 2015). Eye-tracking data in this educational study was used to determine whether students administered a contraindicated medication due to a knowledge deficit, or because the information required was not visualized. In this case, the necessary information was visualized by all students, and the deficit in knowledge related to pharmacology could be corrected.
In addition, acquired knowledge through experience in the field, as well as risk-aversion in certain types of medical cases, may lead to altered salience depending upon “value.” The learned value of visual stimuli significantly affects attentional priority (Laurent, Hall, Anderson, & Yantis, 2015; Sali, Anderson, & Yantis, 2014). In basic eye-tracking experiments involving manipulation of object low level properties, participants quickly learned to search for objects of a property that would produce a valuable reward (Laurent et al., 2015; Sali et al., 2014). Thus, if a radiologist is “rewarded,” for example, for finding cancerous lesions as opposed to incidental pathology, these objects may affect salience maps.
In contrast to the study of errors due to a knowledge deficit (Amster et al., 2015), eye-tracking has also been utilized to identify the errors which can occur during visual search guided by expertise. In a version of the “invisible gorilla” study Drew et al. utilized an eye-tracker to follow the eye movements of radiologists as they searched for lung nodules in a serial stack of CT images. On the last case, a gorilla was inserted into the lung, and the majority of radiologists failed to notice this (Drew, Vo, & Wolfe, 2013). Eye-tracking was able to demonstrate that this was not because the radiologists were negligent – they did, in fact, attend to the region of the gorilla – but their expert attention to expected visual targets, acquired through medical training, led to an “inattentional blindness” to the gorilla (Drew, Evans, et al., 2013; Drew, Vo, & Wolfe, 2013). This particular study illustrates the way in which eye-tracking can be used to study the visual process of “missing” an obvious abnormality as compared to the presence of disease.
Wolfe and Van Wert (2010) have also discussed the importance of prevalence effects in designing appropriate eye-tracking experiments for understanding medical expertise. Screening tasks such as screening for breast cancer on a mammogram, or cervical cancer on a Pap smear, represent an important class of search with low prevalence of visual targets within the population screened (Wolfe & Van Wert, 2010). Studies of vigilance and low-prevalence search tasks have shown that rare events are missed more often than common ones (Wolfe et al., 2007; Wolfe & Van Wert, 2010). Wolfe and Van Wert (2010) explain that when targets are rare, observers are more likely to reject an ambiguous target. By contrast, participants are much more likely to label an ambiguous item as a target when prevalence is high. This effect is attributed to an unconscious decision rule that is changing (Wolfe & Van Wert, 2010). Observers also become faster to declare themselves to be finished with an image under low-prevalence conditions, but forcing them to slow down does not make observers less likely to reject low-prevalence targets (Wolfe, 2012b). Importantly, Evans et al. (2013) found the same effect to be true for both experts and novices. In an experiment involving manipulation of prevalence in the setting of mammography, radiologists had a false-negative rate of 12 % in the setting of high prevalence, and 30% at low prevalence (Evans, Birdwell, & Wolfe, 2013). Similar results were found with cytologists reading cervical cancer screening slides (Evans, Tambouret, Evered, Wilbur, & Wolfe, 2011). In all cases, rare targets were missed more often. This has important implications for the generalizability of eye-tracking experiments related to visual expertise, as it suggests that prevalence in the experimental setting may affect visual performance in a way that is not seen in clinical practice. In some cases, prevalence in an experimental setting may actually offer the opportunity for increased training (Jarodska et al., 2012), and the subsequent improvement of visual patterns. While some studies have demonstrated an acquisition of a pattern of “expert” gaze with focused training on gaze patterns (Jardoska et al., 2012; E.A. Henneman et al., 2014), it would be highly informative to examine the effects of such training over a prolonged period of time to ascertain whether this represents a permanent shift due to the use of eye-tracking as an educational tool. Furthermore, the patterns of vision seen in the search-related expertise employed for medical screening may differ significantly from the examination of images from which a pathologic diagnosis is expected.
We have discussed several studies in which eye-tracking has elucidated search-related features of visual expertise, however some forms of visual expertise acquired in medicine involve a visual categorization or “gestault” assessment. Dermatologists, for example, are often required to diagnose a clearly visible skin lesion or rash from clinical appearance. Similarly, pathologists are frequently asked to render a diagnosis from a distinct image of a lesion or cell type. While search-related expertise is certainly employed in both of these fields, a significant proportion of visual expertise is devoted to recognition and identification, rather than locating the target. Kundel (2008) noted that visual search was not necessary for all radiologic images, and he incorporated the idea of holistic visual processing in radiologic diagnosis into eye-tracking experiments. The ability to interpret complex visual information in a short period of time is common to all humans, most notably in the form of face perception (Bukach, Gauthier, & Tarr, 2006; Rossion, Collins, Goffaux, & Curran, 2007). It is likely that this technique is employed by medical experts who have significant experience with the visual characteristics of a diagnostic entity. Experts in these fields may describe recognizing an image like one does an acquaintance, which implies visual processing that may be similar to face processing – a visual task long studied with eye-tracking methodology (Gauthier & Nelson, 2001; Pascalis et al., 2005; Vanderwert et al., 2015; Wagner, Hirsch, Vogel-Farley, Redcay, & Nelson, 2013).
The eye-tracking methods used to study holistic visual processing during medical training differ from those employed for search-related expertise. One study of holistic processing asked if mammographers could look at a bilateral mammogram for less than a second and determine if the woman should be called back (Evans, Georgian-Smith, Tambouret, Birdwell, & Wolfe, 2013). The technique of a brief exposure is often useful in identifying holistic processing, as well as the most important areas of interest for the rapid interpretation of images by experts (Evans, Georgian-Smith, et al., 2013; E. Krupinski et al., 2014; Kundel et al., 2008). With increasing medical expertise, several studies have also shown specific changes in eye movement patterns (Drew, Vo, Olwal, et al., 2013; Fox et al., 2017; E. A. Krupinski et al., 2013). Characteristically, trainees make more eye movements when evaluating an image than do experts, and those eye movements cover more of the area of the image. This development of visual efficiency is also seen in the development of human face processing from infancy to adulthood, suggesting another link between these two forms of holistic visual processing (Fox et al., 2017; Gauthier & Nelson, 2001; Pascalis et al., 2005). We have noted in preliminary work that while this form of visual efficiency seems to be naturally acquired over time, more efficient patterns can develop as a result of visual tools designed to enhance the educational process (Fox et al., 2017). This is similar to the finding that training for efficiency of gaze, as well as accuracy in the assessment of infant seizures, could be enhanced through the use of image visualizations based upon expert eye-tracking data (Jarodska et al., 2012).
In recent years, the importance of visual as well as tactile expertise in medical procedures has been recognized, and several eye-tracking studies have examined the type of visual expertise acquired in the training of medical procedural skills (Ahmidi, Ishii, et al., 2012; Law et al., 2003). One such study compared the eye movements utilized by expert and novice surgeons performing a laparoscopic procedure in a computer-based simulator (Law et al., 2003). The results of this study showed that novices needed more visual feedback of the tool position to complete the task than did experts. In addition, the experts tended to maintain eye gaze on the surgical target while manipulating surgical instruments, whereas novices were more varied in eye-hand coordination, and often tracked the surgical tool rather than the surgical target (Law et al., 2003). The development of robotic and laparoscopic surgical instruments has also allowed for the acquisition of kinematic data related to common procedures, and skill level can be evaluated with these measures in conjunction with eye-tracking data (Ahmidi, Ishii, et al., 2012). Data from robotic surgical systems show that Hidden Markov Models (HMM) can facilitate the recognition of surgical skill level (Ahmidi, Hager, Ishii, Gallia, & Ishii, 2012; Ahmidi, Ishii, et al., 2012; Ahmidi et al., 2015). Methods for the assessment of surgical skill utilizing eye-tracking as a quantitative measure have also been developed (Ahmidi, Ishii, et al., 2012). In the experiment by Ahmidi et al. (2012), sinus procedures were performed by experts and novices on cadavers with the use of an endoscope and a visualization screen. A 50Hz remote eye-tracker was utilized in this case, with alignment to video data collected through the endoscope. The results of HMM generated from both kinematic and eye-tracking data reveal that eye-gaze does contain expertise-related structures, and the addition of this data to kinematic information improves models of skill expertise by 13.2% for expert and 5.3% for novice levels (Ahmidi, Ishii, et al., 2012). Models combining both measures can reportedly quantify a surgeon’s skill level on a specific procedure with an accuracy of 82.5% (Ahmidi, Ishii, et al., 2012).
In the field of medical learning, several studies involving all levels of medical professionals have used individual eye-tracking data for both training, and debriefing after simulations (Jarodska et al., 2012; Henneman et al., 2014). As a debriefing strategy for medical simulations, eye-tracking can offer useful information about errors that cannot be readily observed or verbalized. Further, allowing medical trainees to observe expert scanpaths derived from controlled eye-tracking has been proven to be more effective than verbal didactics describing the methods of visual assessment. The scanpaths of trainees in these procedural settings can sometimes be used to assess best practices in medicine, as well as assure competency. This was performed as a follow-up to the study of errors in medication administration (Marquard et al, 2011; Amster, 2015), to analyze patterns of gaze most associated with identification errors among nurses. Nurses who recognized errors were more likely to focus on one piece of identification information at one time, comparing medication labels and the corresponding patient information on an ID badge in sequence, as opposed to reading either the badge or medication bottle in full before changing the point of fixation. Eye-tracking has also been a useful technique for the study of real world disruptions in the medical training environment. For example, in simulated emergency room settings, medical professionals who visually engaged an interruption during a visual task were more likely to commit an error (Marquard et al., 2011). Taken together, these studies of gaze patterns in procedural settings suggest a role for eye-tracking as both a training and assessment tool in medical education, and the necessity of a realistic clinical environment for the application of trained visual patterns of gaze. With the continuing trend towards quantitative assessment of procedural skill in medical training, these early studies suggest a role for eye-tracking methodology in the study and evaluation of visual training as one component of technical proficiency.
Several of the studies mentioned have necessitated the use of visual stimuli that require manipulation by the viewer, such as three-dimensional images and dynamic video recordings. Several recent studies provide evidence that radiologists examining CT images in three-dimensions developed visual patterns that involved maintaining little movement in one dimension while scanning across the plane of the other two dimensions (Drew, Vo, Olwal, et al., 2013; Wen et al., 2016). Two distinct techniques within this pattern could be clustered, but no significant superiority of one technique over another was demonstrated. Recent advances in software for analyzing dynamic scenes or areas of interest allow for greater opportunity to investigate expertise with these types of medical images in the clinical environment (Holmqvist, 2011; Mallett et al., 2014; Phillips et al., 2013). In the field of pathology, for example, advances in digital imaging have allowed for the development of three-dimensional models (Jeong et al., 2010; Ward, Rosen, Law, Rosen, & Faulkner-Jones, 2015), as well as whole-slide images, which more closely replicate the experience of microscopy (Fallon, Wilbur, & Prasad, 2010; Fox et al., 2017). While some educational tasks involve the use of static images, the diagnostic process of anatomic pathology almost always involves movement of the slide image, as well as adjustment of magnification. While several studies of eye-tracking have focused on specific features of digital pathology images (Bombari et al., 2012; Fox et al., 2017; E. Krupinski et al., 2014; E. A. Krupinski et al., 2013; E. A. Krupinski et al., 2006; Tiersma et al., 2003), we now have the tools available to examine the effects of expertise upon the dynamic diagnostic process. The authors have examined pathology trainees at the early and late stages of training, and found support for the use of specific dynamic digital platforms in the acquisition of “expert” patterns of gaze (Fox et al., 2017). As mentioned previously, it may be possible to use specific viewing platforms to teach a pattern of evaluation that replicates that of medical experts (Fox et al., 2017; Jarodska et al., 2012). This could potentially involve the observation of expert gaze during a procedure or visual diagnostic process, the direction of trainee gaze through focused visualization guided by this data (Jardoska et al., 2012), or the development of training viewers designed to improve gaze direction and efficiency.
While there are many important questions related to the study of visual expertise in medicine, a major constraint on these studies is always participant number and available time. Medical professionals are a limited participant resource, and they often do not have the time to participate in experimental settings. Furthermore, there is a limit to how one can manipulate clinical practice for research purposes, and the generalizability of experimental findings to real-world workflow. For this reason, most studies involving eye-tracking are designed to utilize only a small number of participants (often 5-15 per group), as well as fewer visual exemplars to allow for time efficiency. The invention of increasingly portable eye-trackers that can be integrated with a variety of cameras or image interfaces allows for greater accessibility, and perhaps studies which can occur within real clinical settings. In a study of operating room technicians utilizing circulation machines, eye-tracking revealed that expert technicians visually fixated upon a larger number of critical sources of information during the operative procedure as compared to novices (Y. Tomizawa et al., 2012). With knowledge of these differences, it may be possible to incorporate eye-tracking as a component of self-assessment in early medical practice. A key feature in this form of training would be the instruction of optimal gaze – either in a didactic, or simulation environment – followed by evaluation in applicable real-world scenarios. The collection of expert gaze patterns in the clinical environment, as well as the assessment of trainees in real applications, will allow for greater understanding of the generalizability of the results derived from research settings. Furthermore, the feedback of scanpath and fixation data to individual medical trainees may prove useful as not only a research, but an educational tool. In two separate studies, comparisons of standard verbal debriefing of medical trainees following a simulated patient encounter, debriefing involving videos of their scanpaths, and debriefing involving both verbal and scanpath information, revealed that addition of eye-tracking data most significantly improves subsequent performance (Jarodska et al., 2012; Henneman et al., 2014). It is likely that the increasing ease of use of eye-tracking tools will allow investigators to recruit a greater number of “expert” participants, and thereby provide accurate and generalizable data to the medical community.
Visual expertise is an important component of medical learning, and eye-tracking is one method by which we can better understand this skill and its acquisition in relation to the clinical workflow. Studies to date have divided visual expertise among categories of search, image categorization, and procedural skill, with significant overlap of these tasks between fields of medicine. With the continued development of cheaper, faster, and more ergonomic eye-tracking devices, it is likely that we will have many opportunities to study the increasing number of professional tasks requiring visual expertise, and to utilize these results for improved medical training and quality of care. The use of eye-tracking in the training of visual medical expertise, as well as self-evaluation, has the potential to impact overall competency. Visualization of the eye movements of expert clinicians may provide insights into a diagnostic process, or the means to avoid medical errors. These tools can enhance the traditional processes of learning through lectures, simulations, or observational sessions. Specifically, the process of observing the scanpath or directed gaze of a medical expert during a visual task can improve trainee performance, while eye-tracking data from trainees can provide a method of feedback and self-assessment to students. Finally, eye-tracking research may allow us to understand the complex patterns of gaze that underlie diagnostic reasoning, and provide further insight into additional learning methods that improve upon clinical expertise.
We would like to give special thanks to Dr. Charles Nelson, and Dr. Jeremy Wolfe for their expertise and guidance in writing this manuscript. In addition, we would like to give thanks to NIH support for Dr. Beverly Faulkner-Jones (NIH: NIBIB SBIR Grant 2R44EB013518-02A1) and to Dr. Sharon Fox (NIH: NIBIB 3R25NS070682-04S1).
Ahmidi, N., Hager, G. D., Ishii, L., Gallia, G. L., & Ishii,
M. (2012). Robotic path planning for surgeon skill evaluation in
minimally-invasive sinus surgery. Med Image Comput Comput Assist
Interv, 15(Pt 1), 471-478. doi: 10.1007/978-3-642-33415-3_58
Ahmidi, N., Ishii, M., Fichtinger, G., Gallia, G. L., & Hager,
G. D. (2012). An objective and automated method for assessing
surgical skill in endoscopic sinus surgery using eye-tracking and
tool-motion data. Int Forum Allergy Rhinol, 2(6), 507-515.
doi:10.1002/alr.21053
Ahmidi, N., Poddar, P., Jones, J. D., Vedula, S. S., Ishii, L.,
Hager, G. D., & Ishii, M. (2015). Automated objective surgical
skill assessment in the operating room from unstructured tool
motion in septoplasty. Int J Comput Assist Radiol Surg, 10(6),
981-991. doi:10.1007/s11548-015-1194-1
Amster B., Marquard J., Henneman E., Fisher D. (2015). Using an
eye tracker during medication administration to identify gaps in
nursing students' contextual knowledge: an observational study.
Nurse Educ, 40:83–86. doi: 10.1097/NNE.0000000000000097
Bombari, D., Mora, B., Schaefer, S. C., Mast, F. W., & Lehr,
H. A. (2012). What was I thinking? Eye-tracking experiments
underscore the bias that architecture exerts on nuclear grading in
prostate cancer. PLoS One, 7(5), e38023.
doi:10.1371/journal.pone.0038023
Braun, J. (1994). Visual search among items of different salience:
removal of visual attention mimics a lesion in extrastriate area
V4. J Neurosci, 14(2), 554-567.
Browatzki, B., Bulthoff, H. H., & Chuang, L. L. (2014). A
comparison of geometric- and regression-based mobile
gaze-tracking. Front Hum Neurosci, 8, 200.
doi:10.3389/fnhum.2014.00200
Brown, P. J., Marquard, J. L., Amster, B., Romoser, M., Friderici,
J., Goff, S., & Fisher, D. (2014). What do physicians read
(and ignore) in electronic progress notes? Appl Clin Inform, 5(2),
430-444. doi:10.4338/ACI-2014-01-RA-0003
Brunye, T. T., Carney, P. A., Allison, K. H., Shapiro, L. G.,
Weaver, D. L., & Elmore, J. G. (2014). Eye movements as an
index of pathologist visual expertise: a pilot study. PLoS One,
9(8), e103447. doi:10.1371/journal.pone.0103447
Bukach, C. M., Gauthier, I., & Tarr, M. J. (2006). Beyond
faces and modularity: the power of an expertise framework. Trends
Cogn Sci, 10(4), 159-166. doi:10.1016/j.tics.2006.02.004
Crane, H. D., & Steele, C. M. (1985). Generation-V
dual-Purkinje-image eyetracker. Appl Opt, 24(4), 527.
Dodge, R., & Cline, T. S. (1901). The angle velociy of eye
movements. Psychological Review, 8(2), 145-157.
doi:http://dx.doi.org/10.1037/h0076100
Draschkow, D., Wolfe, J. M., & Vo, M. L. (2014). Seek and you
shall remember: scene semantics interact with visual search to
build better memories. J Vis, 14(8), 10. doi:10.1167/14.8.10
Drew, T., Evans, K., Vo, M. L., Jacobson, F. L., & Wolfe, J.
M. (2013). Informatics in radiology: what can you see in a single
glance and how might this guide visual search in medical images?
Radiographics, 33(1), 263-274. doi:10.1148/rg.331125023
Drew, T., Vo, M. L., Olwal, A., Jacobson, F., Seltzer, S. E.,
& Wolfe, J. M. (2013). Scanners and drillers: characterizing
expert visual search through volumetric images. J Vis, 13(10).
doi:10.1167/13.10.3
Drew, T., Vo, M. L., & Wolfe, J. M. (2013). The invisible
gorilla strikes again: sustained inattentional blindness in expert
observers. Psychol Sci, 24(9), 1848-1853.
doi:10.1177/0956797613479386
Eckstein, M. P. (2011). Visual search: a retrospective. J Vis,
11(5). doi:10.1167/11.5.14
Evans, K. K., Birdwell, R. L., & Wolfe, J. M. (2013). If you
don't find it often, you often don't find it: why some cancers are
missed in breast cancer screening. PLoS One, 8(5), e64366.
doi:10.1371/journal.pone.0064366
Evans, K. K., Georgian-Smith, D., Tambouret, R., Birdwell, R. L.,
& Wolfe, J. M. (2013). The gist of the abnormal: above-chance
medical decision making in the blink of an eye. Psychon Bull Rev,
20(6), 1170-1175. doi:10.3758/s13423-013-0459-3
Evans, K. K., Tambouret, R. H., Evered, A., Wilbur, D. C., &
Wolfe, J. M. (2011). Prevalence of abnormalities influences
cytologists' error rates in screening for cervical cancer. Arch
Pathol Lab Med, 135(12), 1557-1560. doi:10.5858/arpa.2010-0739-OA
Fallon, M. A., Wilbur, D. C., & Prasad, M. (2010). Ovarian
frozen section diagnosis: use of whole-slide imaging shows
excellent correlation between virtual slide and original
interpretations in a large series of cases. Arch Pathol Lab Med,
134(7), 1020-1023. doi:10.1043/2009-0320-OA.1
Fox, S. E., Law, C. C., & Faulkner-Jones, B. E. (2017).
Quantitative gaze assessment of a dual ”side by side” viewer
versus a single whole slide image viewer for pathology education.
Manuscript submitted for publication.
Gauthier, I., & Nelson, C. A. (2001). The development of face
expertise. Curr Opin Neurobiol, 11(2), 219-224.
Henneman, E.A., Cunningham H., Fisher D.L., et al. (2014) Eye
tracking as a debriefing mechanism in the simulated setting
improves patient safety practices. Dimens Crit Care Nurs,
33:129–135. doi: 10.1097/DCC.0000000000000041.
Holmqvist, K. (2011). Eye tracking : a comprehensive guide to
methods and measures. Oxford ; New York: Oxford University Press.
Huette, S., Winter, B., Matlock, T., & Spivey, M. (2012).
Processing motion implied in language: eye-movement differences
during aspect comprehension. Cogn Process, 13 Suppl 1, S193-197.
doi:10.1007/s10339-012-0476-6
Jarodzka, H., Balslev, T., Holmqvist, K., Nyström, M., Scheiter,
K., Gerjets, P., & Eika, B. (2012). Conveying clinical
reasoning based on visual observation via eye-movement modelling
examples. Instructional Science, 40(5), 813-827. doi:
10.1007/s11251-012-9218-5
Jeong, W. K., Schneider, J., Turney, S. G., Faulkner-Jones, B. E.,
Meyer, D., Westermann, R., . . . Pfister, H. (2010). Interactive
histology of large-scale biomedical image stacks. IEEE Trans Vis
Comput Graph, 16(6), 1386-1395. doi:10.1109/TVCG.2010.168
Johnson, J. S., Liu, L., Thomas, G., & Spencer, J. P. (2007).
Calibration algorithm for eyetracking with unrestricted head
movement. Behav Res Methods, 39(1), 123-132. doi:
10.3758/BF03192850
Krupinski, E., Chao, J., Hofmann-Wellenhof, R., Morrison, L.,
& Curiel-Lewandrowski. (2014). Understanding Visual Search
Patterns of Dermatologists Assessing Pigmented Skin Lesions Before
and After Online Training. J Digit Imaging, 27, 779-785.
doi:10.1007/s10278-014-9712-1
Krupinski, E. A., Graham, A. R., & Weinstein, R. S. (2013).
Characterizing the development of visual search expertise in
pathology residents viewing whole slide images. Hum Pathol, 44(3),
357-364. doi:10.1016/j.humpath.2012.05.024
Krupinski, E. A., Tillack, A. A., Richter, L., Henderson, J. T.,
Bhattacharyya, A. K., Scott, K. M., . . . Weinstein, R. S. (2006).
Eye-movement study and human performance using telepathology
virtual slides: implications for medical education and differences
with experience. Hum Pathol, 37(12), 1543-1556.
doi:10.1016/j.humpath.2006.08.024
Kundel, H. L., Nodine, C. F., Krupinski, E. A., & Mello-Thoms,
C. (2008). Using gaze-tracking data and mixture distribution
analysis to support a holistic model for the detection of cancers
on mammograms. Acad Radiol, 15(7), 881-886.
doi:10.1016/j.acra.2008.01.023
Laurent, P. A., Hall, M. G., Anderson, B. A., & Yantis, S.
(2015). Valuable Orientations Capture Attention. Vis cogn,
23(1-2), 133-146. doi:10.1080/13506285.2014.965242
Law, B., Atkins, M. S., Lomax, A. J., & Wilson, J. G. (2003).
Eye trackers in a virtual laparoscopic training environment. Stud
Health Technol Inform, 94, 184-186.
Mallett, S., Phillips, P., Fanshawe, T. R., Helbren, E., Boone,
D., Gale, A., . . . Halligan, S. (2014). Tracking eye gaze during
interpretation of endoluminal three-dimensional CT colonography:
visual perception of experienced and inexperienced readers.
Radiology, 273(3), 783-792. doi:10.1148/radiol.14132896
Marquard J.L., Henneman P.L., He Z., Jo J., Fisher D.L., Henneman
E.A. (2011). Nurses' behaviors and visual scanning patterns may
reduce patient identification errors. J Exp Psychol Appl,
17:247–256. doi: http://dx.doi.org/10.1037/a0025261
Merchant, J., Morrissette, R., & Porterfield, J. L. (1974).
Remote measurement of eye direction allowing subject motion over
one cubic foot of space. IEEE Trans Biomed Eng, 21(4), 309-317.
doi:10.1109/TBME.1974.324318
Pascalis, O., Scott, L. S., Kelly, D. J., Shannon, R. W.,
Nicholson, E., Coleman, M., & Nelson, C. A. (2005). Plasticity
of face processing in infancy. Proc Natl Acad Sci U S A, 102(14),
5297-5300. doi:10.1073/pnas.0406627102
Phillips, P., Boone, D., Mallett, S., Taylor, S. A., Altman, D.
G., Manning, D., . . . Halligan, S. (2013). Method for tracking
eye gaze during interpretation of endoluminal 3D CT colonography:
technical description and proposed metrics for analysis.
Radiology, 267(3), 924-931. doi:10.1148/radiol.12120062
Rayner, K. (1978). Eye movements in reading and information
processing. Psychol Bull, 85(3), 618-660.
Rossion, B., Collins, D., Goffaux, V., & Curran, T. (2007).
Long-term expertise with artificial objects increases visual
competition with early face categorization processes. J Cogn
Neurosci, 19(3), 543-555. doi:10.1162/jocn.2007.19.3.543
Sali, A. W., Anderson, B. A., & Yantis, S. (2014). The role of
reward prediction in the control of attention. J Exp Psychol Hum
Percept Perform, 40(4), 1654-1664. doi:10.1037/a0037267
Stewart, J., 3rd, Miyazaki, K., Bevans-Wilkins, K., Ye, C.,
Kurtycz, D. F., & Selvaggi, S. M. (2007). Virtual microscopy
for cytology proficiency testing: are we there yet? Cancer,
111(4), 203-209. doi:10.1002/cncr.22766
Tiersma, E. S., Peters, A. A., Mooij, H. A., & Fleuren, G. J.
(2003). Visualising scanning patterns of pathologists in the
grading of cervical intraepithelial neoplasia. J Clin Pathol,
56(9), 677-680. doi: http://dx.doi.org/10.1136/jcp.56.9.677
Tomizawa Y., Aoki H., Suzuki S., Matayoshi T., Yozu R. (2012).
Eye-tracking analysis of skilled performance in clinical
extracorporeal circulation. J Artif Organs, 15:146–157. doi:
10.1007/s10047-012-0630-z
Tourassi, G., Voisin, S., Paquit, V., & Krupinski, E. (2013).
Investigating the link between radiologists' gaze, diagnostic
decision, and image content. J Am Med Inform Assoc, 20(6),
1067-1075. doi:10.1136/amiajnl-2012-001503
Tourassi, G. D., Mazurowski, M. A., Harrawood, B. P., &
Krupinski, E. A. (2010). Exploring the potential of
context-sensitive CADe in screening mammography. Med Phys, 37(11),
5728-5736. doi:10.1118/1.3501882
Vanderwert, R. E., Westerlund, A., Montoya, L., McCormick, S. A.,
Miguel, H. O., & Nelson, C. A. (2015). Looking to the eyes
influences the processing of emotion on face-sensitive
event-related potentials in 7-month-old infants. Dev Neurobiol,
75(10), 1154-1163. doi:10.1002/dneu.22204
Wagner, J. B., Hirsch, S. B., Vogel-Farley, V. K., Redcay, E.,
& Nelson, C. A. (2013). Eye-tracking, autonomic, and
electrophysiological correlates of emotional face processing in
adolescents with autism spectrum disorder. J Autism Dev Disord,
43(1), 188-199. doi:10.1007/s10803-012-1565-1
Ward, A., Rosen, D. M., Law, C. C., Rosen, S., &
Faulkner-Jones, B. E. (2015). Oxalate nephropathy: a
three-dimensional view. Kidney Int, 88(4), 919.
doi:10.1038/ki.2015.31
Wen, G., Aizenman, A., Drew, T., Wolfe, J. M., Haygood, T. M.,
& Markey, M. K. (2016). Computational assessment of visual
search strategies in volumetric medical images. J Med Imaging
(Bellingham), 3(1), 015501. doi:10.1117/1.JMI.3.1.015501
Wolfe, J. M. (1994). Guided Search 2.0 A revised model of visual
search. Psychon Bull Rev, 1(2), 202-238. doi:10.3758/BF03200774
Wolfe, J. M. (1995). The pertinence of research on visual search
to radiologic practice. Acad Radiol, 2(1), 74-78.
Wolfe, J. M. (2012a). Saved by a log: how do humans perform hybrid
visual and memory search? Psychol Sci, 23(7), 698-703.
doi:10.1177/0956797612443968
Wolfe, J. M. (2012b). When do I quit? The search termination
problem in visual search. Nebr Symp Motiv, 59, 183-208. doi:
10.1007/978-1-4614-4794-8_8
Wolfe, J. M., Evans, K. K., Drew, T., Aizenman, A., & Josephs,
E. (2015). How Do Radiologists Use the Human Search Engine? Radiat
Prot Dosimetry. doi:10.1093/rpd/ncv501
Wolfe, J. M., & Horowitz, T. S. (2004). What attributes guide
the deployment of visual attention and how do they do it? Nat Rev
Neurosci, 5(6), 495-501. doi:10.1038/nrn1411
Wolfe, J. M., Horowitz, T. S., Kenner, N., Hyle, M., & Vasan,
N. (2004). How fast can you change your mind? The speed of
top-down guidance in visual search. Vision Res, 44(12), 1411-1426.
doi:10.1016/j.visres.2003.11.024
Wolfe, J. M., Horowitz, T. S., Van Wert, M. J., Kenner, N. M.,
Place, S. S., & Kibbi, N. (2007). Low target prevalence is a
stubborn source of errors in visual search tasks. J Exp Psychol
Gen, 136(4), 623-638. doi:10.1037/0096-3445.136.4.623
Wolfe, J. M., & Van Wert, M. J. (2010). Varying target
prevalence reveals two dissociable decision criteria in visual
search. Curr Biol, 20(2), 121-124. doi:10.1016/j.cub.2009.11.066