This week you have explored three different approaches to t tests. By this point, you know that each test has assumptions about the data and type of research questions it can answer. For this Assignme

Skill Builder: Research Design and Statistical Design

As a student, you have research questions you want to answer. For example, an agriculture student may want to help farmers select an effective fertilizer for their corn crop. Perhaps farmers in an area of the country have traditionally used cow manure, but turkey dung has now become available at a good price. The student may want to examine the two types of fertilizers to see if there is a difference in their effectiveness and whether farmers will be willing to change from cow manure to turkey dung. To address this research question, the student will need to think about both research design and statistical analysis.

Research Design

The research design for a study is the overall plan for how a researcher will collect data. Research design focuses on obtaining the right data to answer a research question. For example, there are many different ways to design a research study that examines the effectiveness of different types of fertilizer.  Will the agricultural student survey farmers who have used each type of fertilizer and get their reports on how their corn crops have been growing? Or will the student raise corn and use one type of fertilizer on some corn crops and another type on a second set of corn crops and then compare how the two sets of corn crops are growing? In deciding on a research design, the researcher will want to think about which of these designs will give the farmers the best answer about whether the fertilizers differ in effectiveness.

During the research process, the researcher will ideally measure each variable of interest in a study to determine cause and effect. One of the key elements for determining cause and effect in a research study is control. Good research design requires the researcher to be aware of factors besides the independent variable that may have an effect on the dependent variable. These factors are considered threats to validity. The researcher may try to control these factors in some manner in order to make a better judgement about whether the independent variable is, in fact, creating change in the dependent variable. For example, weather and moisture can also affect how corn crops grow, so the researcher needs to control for those factors. Realistically, some factors are more important than others, and the researcher needs to have an understanding of the factors most likely to be in play.

Another key element for determining cause and effect in a research study is random assignment . The researcher investigating cow manure and turkey dung as fertilizers will probably want to test the fertilizers on essentially identical plots of ground. Researchers use random assignment to help them ensure that experimental units (e.g., plots) do not differ in some systematic way at the start of the study. For example, the student would not want the plots receiving cow dung to receive greater sunlight, on average, than the plots receiving turkey dung. It will be important, then, to randomly assign the plots to receive either cow manure or turkey dung.

There are three broad categories of research design covered in this skill builder: experimental, quasi-experimental, and correlational.

Statistical Analysis

After measuring each variable of interest in their study (e.g., fertilizer type and crop yield), researchers use the data they have collected to conduct statistical analyses to answer their research questions.  Statistical analyses involve the use of probabilistic models to analyze the data. Statistical analysis, broadly stated, is evaluating models to determine whether variables are associated with one another. Presuming the researcher has made valid measurements, the results of the statistical model can then be used to make inferences back to the real world. For example, the student will somehow measure the effectiveness of each type of fertilizer, perhaps by focusing on crop yield, and will use those measurements in a statistical model to answer their research question of interest.

In the ideal case, the statistical analysis will be useful in convincing a critic that only the experimental manipulation or chance (a type I error) can explain an experimental effect that is statistically significant.  

There are several types of statistical analyses, including Pearson’s correlation, ANOVA, and regression. In this skill builder, we will briefly discuss two types of statistical analyses, the t-test for independent groups and the matched pairs t-test.

Learn by Doing

Hints, displayed below

You are a researcher trying to answer the question of whether people who engage in regular, lifelong exercise live longer lives. Ideally, which of the following factors should the student be concerned about controlling when developing a research design if the student wants to ensure that the independent variable (regular, lifelong exercise), and not other factors, is affecting how long people live?

Learn by Doing

Hints, displayed below

The researcher investigating cow manure and turkey dung as fertilizers has 20 essentially identical plots of ground available. The plots are adjacent to one another and can be shown to be similar in terms of 15 important criteria used to evaluate soil. The researcher plans to plant corn on all 20 plots and then measure the yield per acre at the end of the growing season. Ten of the plots will receive cow manure as the fertilizer and 10 will receive turkey dung. The rate of fertilizer use in pounds per acre will be the same for all plots. In fact, except for the fertilizer, the student plans to treat all 20 plots the same when it comes to growing corn.

Indicate whether each of the following procedures is useful or not useful in determining whether the two fertilizers differ in their effectiveness. An effective procedure will help convince a critic that an effect exists while an ineffective procedure gives the critic a reason to doubt.