Statistical Significance and Meaningfulness

As a scholar-practitioner, it is important for you to understand that just because a hypothesis test indicates a relationship exists between an intervention and an outcome, there is a difference between groups, or there is a correlation between two constructs, it does not always provide a default measure for its importance. Although relationships are significant, they can be very minute relationships, very small differences, or very weak correlations. In the end, we need to ask whether the relationships or differences observed are large enough that we should make some practical change in policy or practice.

For this Discussion, you will explore statistical significance and meaningfulness.

To prepare for this Discussion:

• Review the Learning Resources related to hypothesis testing, meaningfulness, and statistical significance (please see attachment)
• Review Magnusson’s web blog found in the Learning Resources to further your visualization and understanding of statistical power and significance testing (rpsychologist.com/index.html)
• Review the American Statistical Association’s press release and consider the misconceptions and misuse of p-values (www.amstat.org/asa/files/pdfs/P-ValueStatement.pdf)
• Consider the scenario:
  • A research paper claims a meaningful contribution to the literature based on finding statistically significant relationships between predictor and response variables. In the footnotes, you see the following statement, “given this research was exploratory in nature, traditional levels of significance to reject the null hypotheses were relaxed to the .10 level.”

What is your response to the scenario in which you critically evaluate this footnote. As a reader/reviewer, what response would you provide to the authors about this footnote?