Do studies of statistical power have an effect on the power of studies?
Peter Sedlmeier, Gerd Gigerenzer
Abstract
Peter Sedlmeier, Gerd Gigerenzer
Abstract
The long-term impact of studies of statistical power is investigated using J. Cohen's (1962) pioneering work as an example. We argue that the impact is nil; the power of studies in the same journal that Cohen reviewed (now the Journal of Abnormal Psychology) has not increased over the past 24 years. In 1960 the median power (i.e., the probability that a significant result will be obtained if there is a true effect) was.46 for a medium size effect, whereas in 1984 it was only.37. The decline of power is a result of alpha-adjusted procedures. Low power seems to go unnoticed: only 2 out of 64 experi-ments mentioned power, and it was never estimated. Nonsignificance was generally interpreted as confirmation of the null hypothesis (if this was the research hypothesis), although the median power was as low as.25 in these cases. We discuss reasons for the ongoing neglect of power. Since J. Cohen's (1962) classical study on the statistical power of the studies published in the 1960 volume of the Journal of Abnormal and Social Psychology, a number of power analyses have been performed. These studies exhorted researchers to pay attention to the power of their tests rather than to focus exclu-sively on the level of significance. Historically, the concept of
OpenAlex reports 761 citations for this work. Citation counts describe recorded attention and do not establish research quality.
A contribution statement is not available in the OpenAlex record.
Method details are not available in the OpenAlex metadata.
Findings are not separately available in the OpenAlex metadata.
Limitations are not available in the OpenAlex metadata.
Application details are not available in the OpenAlex metadata.
The long-term impact of studies of statistical power is investigated using J. Cohen's (1962) pioneering work as an example. We argue that the impact is nil; the power of studies in the same journal that Cohen reviewed (now the Journal of Abnormal Psychology) has not increased over the past 24 years. In 1960 the median power (i.e., the probability that a significant result will be obtained if there is a true effect) was.46 for a medium size effect, whereas in 1984 it was only.37. The decline of power is a result of alpha-adjusted procedures. Low power seems to go unnoticed: only 2 out of 64 experi-ments mentioned power, and it was never estimated. Nonsignificance was generally interpreted as confirmation of the null hypothesis (if this was the research hypothesis), although the median power was as low as.25 in these cases. We discuss reasons for the ongoing neglect of power. Since J. Cohen's (1962) classical study on the statistical power of the studies published in the 1960 volume of the Journal of Abnormal and Social Psychology, a number of power analyses have been performed. These studies exhorted researchers to pay attention to the power of their tests rather than to focus exclu-sively on the level of significance. Historically, the concept of
Key concepts: Statistical power, Power (physics), Statistical analysis, Psychology, Statistics, Econometrics, Mathematics, Physics