Abstract
We argue that to best comprehend many data sets, plotting judiciously selected sample statistics with associated confidence intervals can usefully supplement, or even replace, standard hypothesis-testing procedures. We note that most social science statistics textbooks limit discussion of confidence intervals to their use in between-subject designs. Our central purpose in this article is to describe how to compute an analogous confidence interval that can be used in within-subject designs. This confidence interval rests on the reasoning that because between-subject variance typically plays no role in statistical analyses of within-subject designs, it can legitimately be ignored; hence, an appropriate confidence interval can be based on the standard within-subject error term—that is, on the variability due to the subject × condition interaction. Computation of such a confidence interval is simple and is embodied in Equation 2 on p. 482 of this article. This confidence interval has two useful properties. First, it is based on the same error term as is the corresponding analysis of variance, and hence leads to comparable conclusions. Second, it is related by a known factor (√2) to a confidence interval of the difference between sample means; accordingly, it can be used to infer the faith one can put in some pattern of sample means as a reflection of the underlying pattern of population means. These two properties correspond to analogous properties of the more widely used between-subject confidence interval.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Anderson, V., &McLean, R. A. (1974).Design of experiments: A realistic approach. New York: Marcel Dekkar.
Bakan, D. (1966). The test of significance in psychological research.Psychological Bulletin,66, 423–437.
Bayes, T. (1763). An essay towards solving a problem in the doctrine of chances.Philosophical Transactions of the Royal Society,53, 370–418.
Berger, J. O., &Berry, D. A. (1988). Statistical analysis and the illusion of objectivity.American Scientist,76, 159–165.
Boik, R. J. (1981). A priori tests in repeated measures designs: Effects of nonsphericity.Psychometrika,46, 241–255.
Box, G. E. P. (1954). Some theorems on quadratic forms applied in the study of analysis of variance problems: II. Effect of inequality of variance and of correlation between errors in the two-way classification.Annals of Mathematical Statistics,25, 484–498.
Box, G. E. P. (1986). An apology for ecumenism in statistics. In G. E. P. Box, T. Leonard, & C.-F. Wu (Eds.),Scientific inference, data analysis, and robustness (pp. 51–84). New York: Academic Press.
Box, G. E. P., &Tiao, G. C. (1973).Bayesian inference in statistical analysis. Reading, MA: Addison-Wesley.
Camilli, G. (1990). The test of homogeneity for 2 × 2 contingency tables: A review of and some personal opinions on the controversy.Psychological Bulletin,108, 135–145.
Cohen, J. (1990). Things I have learned (so far).American Psychologist,45, 1304–1312.
Fisher, R. A. (1925).Statistical methods for research workers. Edinburgh: Oliver & Boyd.
Fisher, R. A. (1935). The logic of inductive inference.Journal of the Royal Statistical Society,98, 39–54.
Fisher, R. A. (1947).The design of experiments. New York: Hafner Press.
Fisher, R. A. (1955). Statistical methods and scientific induction.Journal of the Royal Statistical Society, Series B,17, 69–78.
Gigerenzer, G., Swijtink, Z., Porter, T., Daston, L., Beatty, J., &Krüger, L. (1989).The empire of chance. Cambridge: Cambridge University Press.
Greenhouse, S. W., &Geisser, S. (1959). On methods in the analysis of profile data.Psychometrika,24, 95–112.
Hays, W. (1973).Statistics for the social sciences (2nd ed.). New York: Holt.
Hertzog, C., &Rovine, M. (1985). Repeated-measures analysis of variance in developmental research: Selected issues.Child Development,56, 787–809.
Huynh, H., &Feldt, L. S. (1970). Conditions under which mean square ratios in repeated measures designs have exactF distributions.Journal of the American Statistical Association,65, 1582–1589.
Huynh, H., &Feldt, L. S. (1976). Estimation of the Box correction for degrees of freedom from sample data in the randomized block and split plot designs.Journal of Educational Statistics,1, 69–82.
Lehmann, E. L. (1993). The Fisher, Neyman-Pearson theories of testing hypotheses: One theory or two?Journal of the American Statistical Association,88, 1242–1249.
Lewis, C. (1993). Bayesian methods for the analysis of variance. In G. Kerens & C. Lewis (Eds.),A handbook for data analysis in the behavioral sciences: Statistical issues (pp. 233–258). Hillsdale, NJ: Erlbaum.
Loftus, G. R. (1991). On the tyranny of hypothesis testing in the social sciences.Contemporary Psychology,36, 102–105.
Loftus, G. R. (1993a). Editorial Comment.Memory & Cognition,21, 1–3.
Loftus, G. R. (1993b, November).On the overreliance of significance testing in the social sciences. Paper presented at the annual meeting of the Psychonomic Society, Washington, DC.
Loftus, G. R. (1993c). Visual data representation and hypothesis testing in the microcomputer age.Behavior Research Methods, Instrumentation, & Computers,25, 250–256.
Loftus, G. R., &Loftus, E. F. (1988).Essence of statistics (2nd ed.). New York: Random House.
Neyman, J. (1957). “Inductive behavior” as a basic concept of philosophy of science.Review of the International Statistical Institute,25, 7–22.
Neyman, J., &Pearson, E. S. (1928). On the use and interpretation of certain test criteria for purposes of statistical inference.Biometrika,20A, 175–240, 263–294.
Neyman, J., &Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses.Philosophical Transactions of the Royal Society of London, Series A,231, 289–337.
O’Brien, R. G., &Kaiser, M. K. (1985). MANOVA method for analyzing repeated measures designs: An extensive primer.Psychological Bulletin,97, 316–333.
Sternberg, S. (1966). High-speed scanning in human memory.Science,153, 652–654.
Tufte, E. R. (1983).The visual display of quantitative information. Cheshire, CT: Graphics Press.
Tufte, E. R. (1990).Envisioning information. Cheshire, CT: Graphics Press.
Tukey, J. W. (1974). The future of data analysis.Annals of Mathematical Statistics,33, 1–67.
Tukey, J. W. (1977).Exploratory data analysis. Reading, MA: Addison-Wesley.
Wainer, H., &Thissen, D. (1993). Graphical data analysis. In G. Kerens & C. Lewis (Eds.),A handbook for data analysis in the behavioral sciences: Statistical issues (pp. 391–458). Hillsdale, NJ: Erlbaum.
Winer, B. J. (1971).Statistical principles in experimental design (2nd ed.). New York: McGraw-Hill.
Winkler, R. L. (1993). Bayesian statistics: An overview. In G. Kerens & C. Lewis (Eds.),A handbook for data analysis in the behavioral sciences: Statistical issues (pp. 201–232). Hillsdale, NJ: Erlbaum.
Author information
Authors and Affiliations
Corresponding author
Additional information
The writing of this manuscript was supported by NIMH Grant MH41637 to G. R. Loftus and Canadian NSERC Grant OGP0007910 to M. E. J. Masson. We thank Jim Colton, Steve Edgell, Rich Gonzalez, David Lane, Jeff Miller, Rich Schweickert, Saul Sternberg, George Wolford, and two anonymous reviewers for very useful comments on earlier drafts of the manuscript.
Rights and permissions
About this article
Cite this article
Loftus, G.R., Masson, M.E.J. Using confidence intervals in within-subject designs. Psychon Bull Rev 1, 476–490 (1994). https://doi.org/10.3758/BF03210951
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03210951