Three scenarios of ranking inconsistencies involving search tasks

24 March 2016 Three scenarios of ranking inconsistencies involving search tasks

Xin He, Frank W. Samuelson, Rongping Zeng, Berkman Sahiner

Proceedings Volume 9787, Medical Imaging 2016: Image Perception, Observer Performance, and Technology Assessment; 97870U (2016) https://doi.org/10.1117/12.2217617
Event: SPIE Medical Imaging, 2016, San Diego, California, United States

Abstract

Our previous work on assessment of digital breast tomosynthesis (DBT) image quality revealed inconsistencies in ranking the reconstruction algorithms’ performances for a location-known-exactly (LKE) detection and a location-unknown searching task. Such results made us wonder that ranking inconsistencies may not be rare phenomena at all. In this work, we conducted a small literature review that involved three publications (He, Samuelson, Zeng and Sahiner SPIE 2016; Park, Kupinski, Clarkson and Barrett, IPMI 2003 and JOSA 2005). These publications compared the LKE and search performance for a variety of observers using the AUC value as the performance criterion (human observers, CHOs for detection, scanning CHOs for search, and the Markov Chain Monte Carlo ideal observer for detection and search). We categorized the experimental findings into three types of ranking inconsistencies: 1) Ranking inconsistencies in LKE and search tasks; 2) human/ideal observer ranking inconsistencies; and 3) LKE/search ranking inconsistencies in the presence of signal variability. The empirical evidence presented in this work suggested that ranking inconsistencies for imaging systems existed, but these inconsistencies often do not draw enough attention in the literature.

Citation Download Citation

Xin He, Frank W. Samuelson, Rongping Zeng, and Berkman Sahiner "Three scenarios of ranking inconsistencies involving search tasks", Proc. SPIE 9787, Medical Imaging 2016: Image Perception, Observer Performance, and Technology Assessment, 97870U (24 March 2016); https://doi.org/10.1117/12.2217617

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available