+
Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Reply
  • Published:

Confidence and precision increase with high statistical power

Nature Reviews Neuroscience volume 14, page 585 (2013)Cite this article

Subjects

Access through your institution
Buy or subscribe

We are delighted that our Analysis article (Power failure: why small sample size undermines the reliability of neuroscience. Nature Rev. Neurosci. 14, 365–376 (2013))1 has stimulated debate about the issues arising from low statistical power in neuroscience studies. Here, we take the opportunity to respond to some important points made by Quinlan (Misuse of power: in defence of small-scale science. Nature Rev. Neurosci. http://dx.doi.org/10.1038/nrn3475-c1 (2013))2, Ashton (Experimental power comes from powerful theories — the real problem in null hypothesis testing. Nature Rev. Neurosci. http://dx.doi.org/10.1038/nrn3475-c2 (2013))3 and Bacchetti (Small sample size is not the real problem. Nature Rev. Neurosci. http://dx.doi.org/10.1038/nrn3475-c3 (2013))4, and clarify our position on certain key issues raised in our original article1.

Quinlan2 draws a comparison between our observations and those made by Friston in his recent article5. However, we note that subsequent commentaries by Ingre6 and by Lindquist and colleagues7 echo many of the concerns that we raised and that Friston8 agreed that he was “convinced by most of their observations”. The main concern — originally raised by Friston5 and reiterated by Quinlan2 — is that high-powered studies will generate formally statistically significant differences for a 'trivial' effect. There are three important rejoinders to this concern. First, in studies with low statistical power, an observed effect will necessarily be large if it is to pass a pre-specified p-value threshold (typically 0.05), but this does not mean that the true effect will be large, or even exist at all. Second, the concern only applies if a p-value threshold (that is, typically 0.05) is used to either reject or (implicitly) accept the null hypothesis. As Ingre notes6, larger studies actually protect against inferences from trivial effect sizes by allowing a better estimation of the magnitude of the true effect. A shift in emphasis away from significance testing towards the use of effect sizes and confidence intervals would therefore improve matters — this is a point we return to below. Third, the true effect size is not known in advance; what is considered 'trivial' can only be determined when the effect size is known. For example, candidate gene studies are frequently quite small N studies, with the implicit assumption that common genetic variants will be associated with reasonably large effects. Over a decade of relatively fruitless research showed that this assumption is not correct; the magnitude of these effects turns out to be much smaller9, as has been borne out by more recent genome-wide association studies (GWASs), which are explicitly designed to be able to detect very small effects. Have these smaller effects turned out to be trivial? Not at all — in many cases, the genes that have been identified using well-powered GWAS methods have led to renewed interest in the role of the pathways implicated and have thereby generated genuinely new insights into disease mechanisms10. Moreover, learning that the effect sizes of common genetic variants on complex traits are very small has provided important fundamental insights into the genetic architecture of these traits11. Therefore, high power provides greater precision in the estimation of the actual effect size so that researchers can assess their importance or triviality with confidence.

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Button, K. S. et al. Power failure: why small sample size undermines the reliability of neuroscience. Nature Rev. Neurosci. 14, 365–376 (2013).

    Article  CAS  Google Scholar 

  2. Quinlan, P. T. Misuse of power: in defence of small-scale science. Nature Rev. Neurosci. http://dx.doi.org/10.1038/nrn3475-c1 (2013).

  3. Ashton, J. C. Experimental power comes from powerful theories — the real problem in null hypothesis testing. Nature Rev. Neurosci. http://dx.doi.org/10.1038/nrn3475-c2 (2013).

  4. Bacchetti, P. Small sample size is not the real problem. Nature Rev. Neurosci. http://dx.doi.org/10.1038/nrn3475-c3 (2013).

  5. Friston, K. Ten ironic rules for non-statistical reviewers. Neuroimage 61, 1300–1310 (2012).

    Article  Google Scholar 

  6. Ingre, M. Why small low-powered studies are worse than large high-powered studies and how to protect against “trivial” findings in research: Comment on Friston (2012). Neuroimage http://dx.doi.org/10.1016/j.neuroimage.2013.03.030 (2013).

  7. Lindquist, M. A., Caffo, B. & Crainiceanu, C. Ironing out the statistical wrinkles in “ten ironic rules”. Neuroimage http://dx.doi.org/10.1016/j.neuroimage.2013.02.056 (2013).

  8. Friston, K. Sample size and the fallacies of classical inference. Neuroimage http://dx.doi.org/10.1016/j.neuroimage.2013.02.057 (2013).

  9. Flint, J. & Munafò, M. R. Candidate and non-candidate genes in behavior genetics. Curr. Opin. Neurobiol. 23, 57–61 (2013).

    Article  CAS  Google Scholar 

  10. Fowler, C. D., Lu, Q., Johnson, P. M., Marks, M. J. & Kenny, P. J. Habenular α5 nicotinic receptor subunit signalling controls nicotine intake. Nature 471, 597–601 (2011).

    Article  CAS  Google Scholar 

  11. Munafò, M. R. & Flint, J. Dissecting the genetic architecture of human personality. Trends Cogn. Sci. 15, 395–400 (2011).

    PubMed  Google Scholar 

  12. Tressoldi, P. E., Giofre, D., Sella, F. & Cumming, G. High impact = high statistical standards? Not necessarily so. PLoS ONE 8, e56180 (2013).

    Article  CAS  Google Scholar 

  13. Nosek, B. A., Spies, J. R. & Motyl, M. Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability. Persp. Psychol. Sci. 7, 615–631 (2012).

    Article  Google Scholar 

  14. Inthout, J., Ioannidis, J. P. & Borm, G. F. Obtaining evidence by a single well-powered trial or several modestly powered trials. Stat. Methods Med. Res. http://dx.doi.org/10.1177/0962280212461098 (2012).

  15. Ioannidis, J. P., Hozo, I. & Djulbegovic, B. Optimal type I and type II error pairs when the available sample size is fixed. J. Clin. Epidemiol. http://dx.doi.org/10.1016/j.jclinepi.2013.03.002 (2013).

Download references

Author information

Authors and Affiliations

  1. Katherine S. Button, Claire Mokrysz and Marcus R. Munafò are at the School of Experimental Psychology, University of Bristol, BS8 1TU, UK.,

    Katherine S. Button, Claire Mokrysz & Marcus R. Munafò

  2. Katherine S. Button is also at the School of Social and Community Medicine, University of Bristol, BS8 2BN, UK.,

    Katherine S. Button

  3. John P. A. Ioannidis is at Stanford University School of Medicine, California 94305, USA.,

    John P. A. Ioannidis

  4. Brian A. Nosek is at the Department of Psychology, University of Virginia and the Center for Open Science, Charlottesville, Virginia 22903, USA.,

    Brian A. Nosek

  5. Jonathan Flint is at the Wellcome Trust Centre for Human Genetics, University of Oxford, OX 7BN, UK.,

    Jonathan Flint

  6. Emma S. J. Robinson is at the School of Physiology and Pharmacology, University of Bristol, BS8 1TD, UK.,

    Emma S. J. Robinson

Authors
  1. Katherine S. Button
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. John P. A. Ioannidis
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Claire Mokrysz
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Brian A. Nosek
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Jonathan Flint
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Emma S. J. Robinson
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Marcus R. Munafò
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Marcus R. Munafò.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Button, K., Ioannidis, J., Mokrysz, C. et al. Confidence and precision increase with high statistical power. Nat Rev Neurosci 14, 585 (2013). https://doi.org/10.1038/nrn3475-c4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nrn3475-c4

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing
点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载