Abstract
Signal detection theory offers several indexes of sensitivity (d’,A z, andA’) that are appropriate for two-choice discrimination when data consist of one hit rate and one false alarm rate per condition. These measures require simplifying assumptions about how target and lure evidence is distributed. We examine three statistical properties of these indexes: accuracy (good agreement between the parameter and the sampling distribution mean), precision (small variance of the sampling distribution), and robustness (small influence of violated assumptions on accuracy). We draw several conclusions from the results. First, a variety of parameters (sample size, degree of discriminability, and magnitude of hits and false alarms) influence statistical bias in these indexes. Comparing conditions that differ in these parameters entails discrepancies that can be reduced by increasing N. Second, unequal variance of the evidence distributions produces significant bias that cannot be reduced by increasing N—a serious drawback to the use of these sensitivity indexes when variance is unknown. Finally, their relative statistical performances suggest thatA z is preferable toA’.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Donaldson, W. (1993). Accuracy ofd’ andA’ as estimates of sensitivity.Bulletin of the Psychonomic Society,31, 271–274.
Donaldson, W. (1996). The role of decision processes in remembering and knowing.Memory & Cognition,24, 523–533.
Dunn, J. C. (2004). Remember-know: A matter of confidence.Psychological Review,111, 524–542.
Gourevitch, V., &Galanter, E. (1967). A significance test for one parameter isosensitivity functions.Psychometrika,32, 25–33.
Green, D. M. (1964). General prediction relating yes-no and forced-choice results.Journal of the Acoustical Society of America,36, 1042 (Abstract).
Green, D. M., &Swets, J. A. (1966).Signal detection theory and psychophysics. New York: Wiley.
Hautus, M. J. (1995). Corrections for extreme proportions and their biasing effects on estimated values of d’.Behavior Research Methods, Instruments, & Computers,27, 46–51.
Kadlec, H. (1999). Statistical properties of d’ and β estimates of signal detection theory.Psychological Methods,4, 22–43.
Macmillan, N. A., &Creelman, C. D. (1996). Triangles in ROC space: History and theory of “nonparametric” measures of sensitivity and response bias.Psychonomic Bulletin & Review,3, 164–170.
Macmillan, N. A., &Creelman, C. D. (2005).Detection theory: A user’s guide (2nd ed.). Mahwah, NJ: Erlbaum.
Macmillan, N. A., &Kaplan, H. L. (1985). Detection theory analysis of group data: Estimating sensitivity from average hit and false-alarm rates.Psychological Bulletin,98, 185–199.
Macmillan, N. A., Rotello, C. M., &Miller, J. O. (2004). The sampling distributions of Gaussian ROC statistics.Perception & Psychophysics,66, 406–421.
Macmillan, N. A., Rotello, C. M., &Verde, M. F. (2005). On the importance of models in interpreting remember-know experiments: Comments on Gardiner et al.’s (2002) meta-analysis.Memory,13, 607–621.
McDermott, K. B., &Watson, J. M. (2001). The rise and fall of false recall: The impact of presentation duration.Journal of Memory & Language,45, 160–176.
Miller, J. (1996). The sampling distribution of d’.Perception & Psychophysics,58, 65–72.
Niewiadomski, M. W., &Hockley, W. E. (2001). Interrupting recognition memory: Tests of familiarity-based accounts of the revelation effect.Memory & Cognition,29, 1130–1138.
Pastore, R. E., Crawley, E. J., Berens, M. S., &Skelly, M. A. (2003). “Nonparametric” A’ and other modern misconceptions about signal detection theory.Psychonomic Bulletin & Review,10, 556–569.
Pollack, I., &Hsieh, R. (1969). Sampling variability of the area under the ROC-curve and of d?(e).Psychological Bulletin,71, 161–173.
Pollack, I., &Norman, D. A. (1964). A non-parametric analysis of recognition experiments.Psychonomic Science,1, 125–126.
Ratcliff, R., Sheu, C.-F., &Gronlund, S. D. (1992). Testing global memory models using ROC curves.Psychological Review,99, 518–535.
Simpson, A. J., &Fitter, M. J. (1973). What is the best index of detect-ability?Psychological Bulletin,80, 481–488.
Smith, W. D. (1995). Clarification of sensitivity measure A’.Journal of Mathematical Psychology,39, 82–89.
Swets, J. A. (1986). Form of empirical ROCs in discrimination and diagnostic tasks: Implications for theory and measurement of performance.Psychological Bulletin,99, 181–198.
Verde, M. F., &Rotello, C. M. (2003). Does familiarity change in the revelation effect?Journal of Experimental Psychology: Learning, Memory, & Cognition,29, 739–746.
Whittlesea, B. W. A. (2002). False memory and the discrepancy-attribution hypothesis: The prototype-familiarity illusion.Journal of Experimental Psychology: General,131, 96–115.
Zhang, J., &Mueller, S. T. (2005). A note on ROC analysis and nonparametric estimate of sensitivity.Psychometrika,70, 1–10.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported by National Institutes of Health Research Grant MH60274-02 to C.M.R. and N.A.M.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Verde, M.F., Macmillan, N.A. & Rotello, C.M. Measures of sensitivity based on a single hit rate and false alarm rate: The accuracy, precision, and robustness of′,A z, andA’. Perception & Psychophysics 68, 643–654 (2006). https://doi.org/10.3758/BF03208765
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03208765