Respond to Discussion Question

As we learn more about statistics, it is important to understand the limitations of statistical significance and to interpret the results within the context of the research and its pragmatic, ‘real world’ application.  Just because something statistically appears to be true, or have a connection, or predict the next President, it does not necessarily mean that it is more probable.

According to David Lane, a low probability value casts doubt on the null hypothesis. How low must the probability value be in order to conclude that the null hypothesis is false? Although there is clearly no right or wrong answer to this question, it is conventional to conclude the null hypothesis is false if the probability value is less than 0.05. More conservative researchers conclude the null hypothesis is false only if the probability value is less than 0.01. When a researcher concludes that the null hypothesis is false, the researcher is said to have rejected the null hypothesis. 

When the null hypothesis is rejected, the effect is said to be statistically significant.  This research paper scenario that we analyzed claims to have a meaningful contribution to the literature even though the research was exploratory in nature, and traditional levels of significance to reject the null hypotheses were relaxed to the .10 level.  That level is entirely way too high for the null hypothesis to be false, and thus significant, therefore it is not that significant. It turns out that when the procedures for hypothesis testing were developed, something was ‘significant’ if it signified something. Thus, finding that an effect is statistically significant signifies that the effect is real and not due to chance.  Higher probabilities provide less evidence that the null hypothesis is false.

The null hypothesis is essentially the ‘devil’s advocate’ position.  That is, it assumes that whatever you are trying to prove did not happen, therefore the object is to reject it.  This research scenario is not significant.  This research was exploratory in nature, and exploratory research “intends merely to explore the research questions and does not intend to offer final and conclusive solutions to existing problems” (http://research-methodology.net).  Exploratory research  helps determine the best research design, data-collection method and selection of subjects. It should draw definitive conclusions only with extreme caution, hence this scenario needs to be extremely careful to make any claims considering it is exploratory in nature and the traditional levels of significance to reject the null hypothesis were relaxed to the .10 level.

Lane, David. Introduction to Statistics. Rice University; University of Houston.

http://research-methodology.net