As a scholar-practitioner, it is important for you to understand that just because a hypothesis test indicates a relationship exists between an intervention and an outcome, there is a difference betwe
AMERICAN STATISTICAL ASSOCIATION RELEASES STATEMENT ON STATISTICAL SIGNIFICANCE AND P-VALUES Provides Principles to Improve the Conduct and Inte rpretation of Quantitative Science March 7, 2016 The American Statistical Association (ASA) has rele ased a “Statement on Statistical Significance and P-Values” with six principles underlying the proper use and interpretation of the p-value [ http://amstat.tandfonline.com/doi/abs/10.1080/00031 305.2016.1154108#.Vt2XIOaE2MN ]. The ASA releases this guidance on p-values to improve the conduct and interpretation o f quantitative science and inform the growing emphasis on reproduc ibility of science research. The statement also notes that the increased quantification of sci entific research and a proliferation of large, complex data sets has expanded the scope for statis tics and the importance of appropriately chosen techniques, properly conducted analyses, and correct interpretation. Good statistical practice is an essential component of good scientific practice, the statement observes, and such practice “emphasizes principles of good study design and conduct, a variety of numerical and graphical summaries of data, under standing of the phenomenon under study, interpretation of results in context, complete repo rting and proper logical and quantitative understanding of what data summaries mean.” “The p-value was never intended to be a substitute for sc ientific reasoning,” said Ron Wasserstein, the ASA’s executive director. “Well-re asoned statistical arguments contain much more than the value of a single number and whether that number exceeds an arbitrary threshold. The ASA statement is intended to steer research into a ‘post p<0.05 era.’” “Over time it appears the p-value has become a gatekeeper for whether work is publishable, at least in some fields,” said Jessica Utts, ASA presi dent. “This apparent editorial bias leads to the ‘file-drawer effect,’ in which research with statis tically significant outcomes are much more likely to get published, while other work that migh t well be just as important scientifically is never seen in print. It also leads to practices ca lled by such names as ‘p-hacking’ and ‘data dredging’ that emphasize the search for small p-values over other statistical and scientific reasoning.” The statement’s six principles, many of which address misconceptions and misuse of the p- value, are the following:1. P-values can indicate how incompatible the data are with a specified statistical model. 2.
P-values do not measure the probability that the st udied hypothesis is true, or the probability that the data were produced by random c hance alone.
3.
Scientific conclusions and business or policy decis ions should not be based only on whether a p-value passes a specific threshold. 4.
Proper inference requires full reporting and transp arency. 5.
A p-value, or statistical significance, does not me asure the size of an effect or the importance of a result. 6.
By itself, a p-value does not provide a good measur e of evidence regarding a model or hypothesis.
The statement has short paragraphs elaborating on e ach principle.
In light of misuses of and misconceptions concernin g p-values, the statement notes that statisticians often supplement or even replace p-values with other approaches. These include methods “that emphasize estimation over testing suc h as confidence, credibility, or prediction intervals; Bayesian methods; alternative measures o f evidence such as likelihood ratios or Bayes factors; and other approaches such as decisio n-theoretic modeling and false discovery rates.” “The contents of the ASA statement and the reasonin g behind it are not new—statisticians and other scientists have been writing on the topic for decades,” Utts said. “But this is the first time that the community of statisticians, as represented by the ASA Board of Directors, has issued a statement to address these issues.” “The issues involved in statistical inference are d ifficult because inference itself is challenging,” Wasserstein said. He noted that more than a dozen d iscussion papers are being published in the ASA journal The American Statistician with the statement to provide more perspective on this broad and complex topic. “What we hope will fo llow is a broad discussion across the scientific community that leads to a more nuanced a pproach to interpreting, communicating, and using the results of statistical methods in res earch.” About the American Statistical Association The ASA is the world’s largest community of statist icians and the oldest continuously operating professional science society in the United States. Its members serve in industry, government and academia in more than 90 countries, advancing r esearch and promoting sound statistical practice to inform public policy and improve human welfare. For additional information, please visit the ASA website at www.amstat.org . For more information: Ron Wasserstein (703) 302-1859 [email protected]
