Unboxing the black box of visual expertise in medicine

Main Article Content

Halszka Jarodzka
Henny P.A. Boshuizen


Visual expertise in medicine has been a subject of research since many decades. Interestingly, it has been investigated from two little related fields, namely the field that focused mainly on the visual search aspects whilst ignoring higher-level cognitive processes involved in medical expertise, and the field that mainly focused on these higher-level cognitive processes largely ignoring the relevant visual aspects. Consequently, both research lines have traditionally used different methodologies. Recently, this gap is being increasingly closed and this special issue presents methods to investigate visual expertise in medicine from both research lines, namely those investigating vision (eye tracking, pupillometry, flash preview moving window paradigm), verbalisations, brain activity, and performance measures (ROC analysis, gesture coding, expert performance approach). We discuss the benefits and drawbacks of each method and suggest directions for future research that could help to unbox the black box of visual expertise in medicine.

Article Details

How to Cite
Jarodzka, H., & Boshuizen, H. P. (2017). Unboxing the black box of visual expertise in medicine. Frontline Learning Research, 5(3), 167–183. https://doi.org/10.14786/flr.v5i3.332


Anstis, S. M., Mayhew, J. W., & Morley, T. (1969). The perception of where a face or television portrait is looking. American Journal of Psychology, 82, 474-489.

Argyle, M., & Cook, M. (1976). Gaze and mutual gaze. Oxford, England: Cambridge University Press.

Balslev, T., Jarodzka, H., Holmqvist, K., De Grave, W. S., Muijtjens, A., Eika, B., . . . Scherpbier, A. (2012). Visual expertise in paediatric neurology. European Journal of Paediatric Neurology, 16 (2), 161-166. doi: 10.1016/j.ejpn.2011.07.004

Bertram, R., Helle, L., Kaakinen, J. K., & Svedström, E. (2013). The effect of expertise on eye movement behaviour in medical image perception. PLoS ONE, 8(6), 1-15. doi: 10.1371/journal.pone.0066169

Boshuizen, H. P. A., & Schmidt, H. G. (1992). On the role of biomedical knowledge in clinical reasoning by experts, intermediates, and novices. Cognitive Science, 16(2), 153-184. doi: 10.1207/s15516709cog1602_1

Boshuizen, H. P. A., & Schmidt, H. G. (2008a). The development of clinical reasoning expertise. In J. Higgs, M. Jones, S. Loftus & N. Christensen (Eds.), Clinical reasoning in the health professions (pp. 113-122). Oxford: Butterworth Heinemann / Elsevier.

Boshuizen, H. P. A., & Schmidt, H. G. (2008b). The development of clinical reasoning expertise: Implications for teaching. In J. Higgs, M. Jones, S. Loftus & N. Christensen (Eds.), Clinical reasoning in the health professions (Vol. 3rd rev. ed. ed., pp. 131-121). Oxford, UK: Butterworth-Heinemann/Elsevier.

Brooks, L. R., LeBlanc, V. R., & Norman, G. R. (2000). On the difficulty of noticing obvious features in patient appearance. Psychological Science, 11, 112-117.

Brünken, R., Steinbacher, S., Plass, J., & Leutner, D. (2002). Assessment of cognitive load in multimedia learning using dual-task methodology. Experimental Psychology, 49, 109-119. doi: 10.1027//1618-3169.49.2.109

Chi, M. T. H. (2006). Two approaches to the study of experts' characteristics. In K. A. Ericsson, N. Charness, R. R. Hoffman & P. Feltovich (Eds.), The Cambridge handbook of expertise and expert performance (pp. 21-30). Cambridge: Cambridge University Press.

Davson, H. (2012). Physiology of the Eye: Elsevier.

Ericsson, K. A. (2006). Protocol analysis and expert thought: Concurrent verbalizations of thinking during experts’ performance on representative tasks. In K. A. Ericsson, N.

Charness, P. J. Feltovich & R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance (pp. 223-241). Cambridge: Cambridge University Press.

Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional performance: Evidence of maximal adaption to task constraints. Annual Reviews in Psychology, 47(1), 273-305. doi: 10.1146/annurev.psych.47.1.273

Ericsson, K. A., & Smith, J. (1991). Prospects and limits in the empirical study of expertise. In K. A. Ericsson & J. Smith (Eds.), Towards a general theory of expertise: Prospects and limits (pp. 1-38). Cambridge, MA: Cambridge University Press.

Feltovich, P. J., Johnson, P. E., Moller, J. H., & Swanson, D. B. (1984). LCS: The role and development of medical knowledge in diagnostic expertise. In W. J. Clancey & E. H. Shortliffe (Eds.), Readings in medical artificial inteligence: the first decade (pp. 275-319). Reading, MA: Addison-Wesley Publishing Company.

Fiorio, M., Cesari, P., Bresciani, M. C., & Tinazzi, M. (2010). Expertise with pathological actions modulates a viewer’s motor system. Neuroscience, 167, 691-699. doi: 10.1016/j.neuroscience.2010.02.010

Foulsham, T., Dewhurst, R., Nyström, M., Jarodzka, H., Johansson, R., Underwood, G., & Holmqvist, K. (2012). Comparing scanpaths during scene encoding and recognition: A multidimensional approach. Journal of Eye Movement Research, 5(5), 1-12.

Fox, S. E., & Faulkner-Jones, B. E. (2017). Eye-tracking in the study of visual expertise: methodology and approaches in medicine. Frontline Learning Research, 5(3), 43 - 54. doi: 10.14786/flr.v5i3.258

Fox, S. E., Law, C. C., & Faulkner-Jones, B. E. (2016). Quantitative gaze assessment of a dual ”side by side” viewer versus a single whole slide image viewer for pathology education. Journal of Pathology Informatics.

Gegenfurtner, A., Kok, E., Van Geel, K., De Bruin, A. B. H., Jarodzka, H., Szulewski, A., & V., V. M. J. J. G. (2016). The challenges of studying visual expertise in medical image diagnosis. Medical Education, 51(1), 97-104. doi: 10.1111/medu.13205

Gegenfurtner, A., Kok, E., Van Geel, K., De Bruin, A. B. H., & Sorger, B. (2017). Neural correlates of visual perceptual expertise: Evidence from cognitive neuroscience using functional neuroimaging. Frontline Learning Research, 5(3), 14 - 30. doi: 10.14786/flr.v5i3.259

Gegenfurtner, A., Lehtinen, E., & Säljö, R. (2011). Expertise Differences in the Comprehension of Visualizations: a Meta-Analysis of Eye-Tracking Research in Professional Domains. Educational Psychology Review, 23(4), 523-552. doi: 10.1007/s10648-011-9174-7

Gegenfurtner, A., & Van Merriënboer, J. (2017). Methodologies for studying visual expertise. Frontline Learning Research, 5(3), 1-13. doi: 10.14786/flr.v5i3.361

Gilhooly, K. J., McGeorge, P., Hunter, J., Rawles, J. M., Kirby, I. K., Green, C., & Wynn, V. (1997). Biomedical knowledge in diagnostic thinking: The case of electrocardiogram (ECG) interpretation. European Journal of Cognitive Psychology, 9(2), 199-223. doi: 10.1080/713752555

Helle, L. (2017). Prospects and pitfalls in combining eye-tracking data and verbal reports. Frontline Learning Research, 5(3), 81 - 93. doi: 10.14786/flr.v5i3.254

Hess, E. H., & Polt, J. M. (1960). Pupil size as related to interest value of visual stimuli. Science, 132(3423), 349-350. doi: 10.1126/science.132.3423.349

Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & Van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures. Oxford: Oxford University Press.

Hruska, P., Hecker, K. G., Coderre, S., McLaughlin, K., Cortese, F., Doig, C., . . . Krigolson, O. (2016). Hemispheric activation differences in novice and expert clinicians during clinical decision making. Advances in Health Science Education, 21(5), 921–933. doi: 10.1007/s10459-017-9787-9

Huettel, S., Song, A., & McCarthy, G. (2008). Functional Magnetic Resonance Imaging. Sunderland, MA: Sinauer Associates.

Ivarsson, J. (2017). Visual expertise as embodied practice. Frontline Learning Research, 5(3), 123 - 138. doi: 10.14786/flr.v5i3.253

Jaarsma, T., Boshuizen, H. P. A., Jarodzka, H., Nap, M., Verboon, P., & Van Merriënboer, J. J. G. (2016). Tracks to a medical diagnosis: expertise differences in visual problem solving. Applied Cognitive Psychology, 30(3), 314-322. doi: 10.1002/acp.3201

Jaarsma, T., Jarodzka, H., Nap, M., Van Merriënboer, J. J. G., & Boshuizen, H. P. A. (2015). Expertise in clinical pathology: bridging the gap. Advances in Health Sciences Education, 20(4), 1089-1106. doi: 10.1007/s10459-015-9589-x

Jaarsma, T., Jarodzka, H., Nap, M., Van Merriёnboer, J. J. G., & Boshuizen, H. P. A. (2014). Expertise differences under the microscope: processing histopathological slides. Medical Education, 48(3), 292-300. doi: 10.1111/medu.12385

Jarodzka, H., Balslev, T., Holmqvist, K., Nyström, M., Scheiter, K., Gerjets, P., & Eika, B. (2012). Conveying clinical reasoning based on visual observation via eye-movement modelling examples. Instructional Science, 40(5), 813-827. doi: 10.1007/s11251-012-9218-5

Jarodzka, H., Boshuizen, H. P. A., & Kirschner, P. A. (2012). Cognitive skills in medical diagnosis and intervention. In P. Lanzer (Ed.), Catheter-based Cardiovascular Interventions; Knowledge-based Approach (pp. 69-86). Berlin, Germany: Springer.

Johansson, R., & Johansson, M. (2014). Look here: Eye movements play a funtional role in memory retrieval. Psychological Science, 25, 236-242.

Kicken, W., Brand-Gruwel, S., Van Merriënboer, J. J. G., & Slot, W. (2009). The effects of portfolio-based advice on the development of self-directed learning skills in secondary vocational education. Educational Technology, Research and Development, 57, 439-460. doi: 10.1007/s11423-009-9111-3

Kok, E. M., De Bruin, A. B. H., Robben, S., G. F., & Van Merriënboer, J. (2012). Looking in the same manner but seeing it differently: Bottom-up and expertise effects in radiology. Applied Cognitive Psychology, 26(6), 854-862. doi: DOI: 10.1002/acp.2886

Kok, E. M., & Jarodzka, H. (2016). Before your very eyes: The value and limitations of eye tracking in medical education. Medical Education. doi: 10.1111/medu.13066

Kok, E. M., & Jarodzka, H. (2017). Beyond your very eyes: eye movements are necessary, not sufficient. Medical Education. doi: 10.1111/medu.13384

Krupinski, E. (2017). Receiver Operating Characteristic (ROC) analysis. Frontline Learning Research, 5(3), 31 - 42. doi: 10.14786/flr.v5i3.250

Kundel, H. L., & Nodine, C. (1975). Interpreting chest radiographs without visual search. Radiology, 116, 527-532.

Kundel, H. L., Nodine, C. F., Conant, E. F., & Weinstein, S. P. (2007). Holistic Component of Image Perception in Mammogram Interpretation: Gaze-tracking Study1. Radiology, 242(2), 396-402.

Kundel, H. L., Nodine, C. F., & Toto, L. (1991). Searching for lung nodules. The guidance of visual scanning. Investigative radiology, 26(9), 777.

Laeng, B., Eidet, L. M., Sulutvedt, U., & Panksepp, J. (2016). Music chills: The eye pupil as a mirror to music’s soul. Consciousness and Cognition, 44, 161-178. doi: 10.1016/j.concog.2016.07.009

LeBlanc, V. R., Brooks, L. R., & Norman, G. R. (2002). Believing is seeing: The influence of a diagnostic hypothesis on the interpretation of clinical features. Academic Medicine, 77(10), 67-69.

LeBlanc, V. R., Dore, K., Norman, G. R., & Brooks, L. R. (2004). Limiting the playing field: does restricting the number of possible diagnoses reduce errors due to diagnosis-specific feature identification? Medical Education, 38, 17 - 24.

LeBlanc, V. R., Norman, G. R., & Brooks, L. R. (2001). Effect of a diagnostic suggestion on diagnostic accuracy and identification of clinical features. Academic Medicine, 76(10), 18-20.

Lesgold, A., Rubinson, H., Feltovich, P., Glaser, R., Klopfer, D., & Wang, Y. (1988). Expertise in a complex skill: Diagnosing X-ray pictures. In M. T. H. Chi, R. Glaser & M. Farr (Eds.), The nature of expertise (pp. 311-342). Hillsdale, NJ: Erlbaum.

Litchfield, D., & Donovan, T. (2017). The flash-preview moving window paradigm: Unpacking visual expertise one glimpse at a time. Frontline Learning Research, 5(3), 66 - 80. doi: 10.14786/flr.v5i3.269

Loewenfeld, I. (1999). The pupil: Anatomy, physiology, and clinical applications. Oxford: Butterworth-Heinemann.
Mansfield, E., Farroni, T., & Johnson, M. (2003). Does gaze perception facilitate overt orienting? Visual Cognition, 10(1), 7-14.

McConkie, G. W., & Rayner, K. (1975). The span of the effective stimulus during a fixation in reading. Perception & Psychophysics, 17, 578-586.

Melo, M., Scarpin, D. J., Amaro, E., Passos, R. B., Sato, J. R., & Friston, K. J. (2011). How doctors generate diagnostic hypotheses: A study of radiological diagnosis with functional magnetic resonance imaging. PLoS ONE, 6, e28752. doi: 10.1371/journal.pone.0028752

Niedermeyer, E., & Da Silva, F. (2005). Electroencephalography. Lippincott Williams & Wilkins.

Nodine, C. F., & Mello-Thoms, C. (2010). The role of expertise in radiologic image perception. In E. Samei & E. Krupinski (Eds.), Medical image perception and techniques (pp. 139-156). Cambridge: Cambridge University Press.

Norman, G. R., Coblentz, C. L., Brooks, L. R., & Babcook, C. J. (1992). Expertise in visual diagnosis: a review of the literature. Academic Medicine, 67(10), S78.

Norman, G. R., Young, M., & Brooks, L. (2007). Non-analytical models of clinical reasoning: the role of experience. Medical Education, 41, 1140-1145.

Nourbakhsh, N., Wang, Y., Chen, F., & Calvo, R. A. (2012). Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks. Paper presented at the Proceedings of the 24th Australian Computer-Human Interaction Conference.

Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84, 429-434. doi: 10.1037//0022-0663.84.4.429

Reingold, E. M., & Sheridan, H. (2011). Eye movements and visual expertise in chess and medicine. In S. P. Liversedge, I. D. Gilchrist & S. Everling (Eds.), Oxford handbook of eye movements (pp. 523-550). Oxford: Oxford University Press.

Ribas, L. M., Rocha, F. T., Siqueira Ortega, N. R., Freitas de Rocha, A., & Massad, E. (2013). Brain activity and medical diagnosis: an EEG study. BMC Neuroscience, 14, 109. doi: 10.1186/1471-2202-14-109

Savin-Williams, R. C., Cash, B. M., McCormack, M., & Rieger, G. (2016). Gay, mostly gay, or bisexual leaning gay? An exploratory study distinguishing gay sexual orientations among young men. Archives of Sexual Behavior, 1-8. doi: 10.1007/s10508-016-0848-6

Szulewski, A., Kelton, D., & Howes, D. (2017). Pupillometry as a tool to study expertise in medicine. Frontline Learning Research, 5(3), 55 - 65. doi: 10.14786/flr.v5i3.256

Tatler, B. W., Hirose, Y., Finnegan, S. K., Pievilainen, R., Kirtley, C., & Kennedy, A. (2013). Priorities for selection and representation in natural tasks. Phil. Trans. R. Soc. B, 368(1628), 20130066.

Van de Wiel, M. (2017). Examining expertise using interviews and verbal protocols. Frontline Learning Research, 5(3), 94 - 122. doi: 10.14786/flr.v5i3.257

Van der Gijp, A., Ravesloot, C., Jarodzka, H., Van der Schaaf, M., Van der Schaaf, I., Van Schaik, J., & Ten Cate, T. J. (2016). How visual search relates to visual diagnostic performance: a narrative systematic review of eye-tracking research in radiology. Advances in Health Sciences Education, 1-23. doi: 10.1007/s10459-016-9698-1

Van Gog, T., Jarodzka, H., Scheiter, K., Gerjets, P., & Paas, F. (2009). Attention guidance during example study via the model‘s eye movements. Computers in Human Behavior, 25(3), 785-791. doi: 10.1016/j.chb.2009.02.007

Vanderhasselt, M.-A., Remue, J., Ng, K. K., & De Raedt, R. (2014). The interplay between the anticipation and subsequent online processing of emotional stimuli as measured by pupillary dilatation: the role of cognitive reappraisal. Frontiers in psychology, 5, 207. doi: 10.3389/fpsyg.2014.00207

Wade, N. J., & Tatler, B. (2005). 'The Moving Tablet of the Eye': The Origins of Modern Eye Movement Research. Oxford: Oxford University Press.

Wang, C.-A., McInnis, H., Brien, D. C., Pari, G., & Munoz, D. P. (2016). Disruption of pupil size modulation correlates with voluntary motor preparation deficits in Parkinson's disease. Neuropsychologia, 80, 176-184. doi: 10.1016/j.neuropsychologia.2015.11.019

Williamsa, A. M., Fawvera, B., & Hodges, N. J. (2017). Using the ‘expert performance approach’ as a framework for improving understanding of expert learning. Frontline Learning Research, 5(3), 139 - 154. doi: 10.14786/flr.v5i3.267