Discussion: Validity in Quantitative Research Designs
Top of Form
Essentials of Evidence Based Practice
INITIAL POST
Introduction
Elder and Paul defined inferences as “conclusions you come to. It's what the mind does in figuring something out” (2009, as cited by University of Louisville, n.d., para 1). In any research study, conclusions drawn from the results should infer “approximate truth” (Polit and Beck, 2017b, p.217). This is known as validity. The concerns associated with internal validity, the effects of strengthening this type in relation to another in quantitative investigations, and the importance of sound conclusions in research are the topics of this paper.
Article of Focus
Yuan, Chou, Hwu, Chang, Hsu, and Kuo, (2009), conducted a quasi experimental study entitled “An Intervention Program to Promote Health-Related Physical Fitness in Nurses”. The researchers’ background work addressed the incidence of musculoskeletal disorders in nurses and physical fitness measures to improve health. The study involved ninety nurses from five different units in a Taiwanese hospital who volunteered as part of the three month project. Groups were divided into an experimental group in which members were instructed to use a stair- stepper for 20- 30 minutes, at least 3 times weekly following which fitness measurements were taken, and a control group which did not receive the intervention, but like the experimental group, received fitness measurements prior to the study. Results showed experimental group statistics, in which logistic regression was used to modify for certain confounding variables, were significant for improvement in body mass index, grasp strength, flexibility, durability of abdominal muscles and cardiopulmonary function, over the control group, despite the opposite being true before the study.
Internal Validity Concerns, Remedies, and Results
According to Polit and Beck (2017b), quasi-experimental studies are very at risk for internal validity threats (p. 226). In this study, selection bias was noted due to lack of randomization to ensure group similarities. For instance, all participants were voluntary, groups contained those who were married and unmarried, some had lower pre-intervention scores than others and some had families. As well, all members worked at the same hospital, and the author’s surmised contamination of results could have taken place due to one member competing with another (Yuan, Chou, Hwu, Chang, Hsu, and Kuo, 2009, p. 1409).
Homogeneity is one method that can be used to strengthen internal validity in light of this weakness. This would entail grouping individuals according to similarities in confounding variables such as in marital status, pre intervention fitness scores, or other areas. As well, stratification/blocking, or matching could be used. However, enhancements of this nature would weaken statistical conclusion validity due to decreasing the sample size (e.g. all married, all unmarried, age group, etc.), and external validity in terms of generalization. In other words, inferences that the study results would hold true with other people, settings or conditions, could not be made (Polit and Beck, 2017b, p.229).
During the study by Yuan, Chou, Hwu, Chang, Hsu, and Kuo (2009), three nurses quit their jobs and one nurse fell ill and was not able to continue. This left 41 participants in the control group that finished. This variable would affect internal validity by increasing the differences between groups more so than if the participants had stayed. Statistical precision could be used to correct this problem by isolating or removing the effect of the loss on the outcome, thereby improving internal as well as statistical and construct variability. Despite these improvements, external validity is weakened since those who left the study may have been different from those who stayed.
The authors reported no external environment events that may have caused historical threats. However, the same instruments and methods of measurement were used before and after testing. The study does not report if the same individuals performed the measurements each time, but if that was the case, testing/instrumentation threat may have been a factor since each person may have improved (or worsened) in the ability to perform the measurements. It is worth noting that a stair-stepper alone was used for the intervention, yet improvements were noted in grasp strength and flexibility. One might wonder if additional exercises were executed by the intervention group. Finally, before and after testing was completed three months apart and thus did not lend to temporal ambiguity.
Importance of Validity Consideration in Research
Confounding variables affect understanding of the cause and effect relationship between the independent and dependent variable, skewing the accuracy and soundness of conclusions drawn from a study (Polit and Beck, 2017a, p. 723). As much as possible, researchers should carefully plan in advance, to the extent possible, what associated issues may interfere with the quality of results and employ methods to control such concerns. Failure to do so may result in wasted time, money, as well as a loss of the study’s trustworthiness.
However, even in the best situations, there are tradeoffs and priorities in study validity. Polit and Beck (2017b) illustrated this point in that one might go to extremes, for example, to ensure attrition biases were minimized and strict adherence was maintained to enhance internal validity. Other controls could be employed to strengthen construct or statistical validity, as well. However, such measures would weaken eternal validity (p. 231).
Conclusion
Each nurse researcher should be mindful of how the potential confounders can be affected by study design, method of implementation, calculation of statistics, and reporting of results, therefore impacting truthfulness in a study. Through thoughtful consideration, one can reach decisions about which aspects of validity threats are most concerning and make the tradeoffs to achieve study goals while maintaining credibility (Polit and Beck, 2017b, p. 232). A good working mindset to employ might be that research evidence does not prove, but only supports a hypothesis, or recommends areas for further research.
References
Polit, D. F., & Beck, C. T. (2017a). Glossary. In Polit, D. F., & Beck, C. T. Nursing research: Generating and assessing evidence for nursing practice (pp. 719-748). (10th ed.). Philadelphia, PA: Wolters Kluwer
Polit, D. F., & Beck, C. T. (2017b). Rigor and validity in quantitative research. In Polit, D. F., & Beck, C. T. Nursing research: Generating and assessing evidence for nursing practice (pp. 217-233). (10th ed.). Philadelphia, PA: Wolters Kluwer.
University of Louisville. (n.d.). Critical thinking and academic research: Inferences. Retrieved from http://library.louisville.edu/c.php?g=158762&p=1039883
Yuan, S., Chou, M., Hwu, L., Chang, Y., Hsu, W., & Kuo, H. (2009). An intervention program to promote health-related physical fitness in nurses. Journal of Clinical Nursing, 18(10), 1,404–1,411. doi:10.000/j1365-2702.200802699x
