The goal of this assignment is to explore "researcher degrees of freedom", and how they can lead to false positives. Yes, you will be p-hacking! Like bad scientists everywhere, you are convinced you y
Assignment 3: Statistics gone wrong (15 marks)
December 7, 2023
| p-hacked study report | |||
| Novice (0) | Developing (1) | Accomplished/ Exemplary (2) | |
| Description of the hypothesis (2 marks) | No introductory section | Hypothesis present but not clear; lack of justification for additional analyses or variables | Brief justification for hypothesis and any subsequent analyses (i.e., justification for inclusion of covariates or interactions) |
| Novice (0) | Developing (0.5) | Accomplished/ Exemplary (1) | |
| Participants (p-hacked) (1 mark) | No description of participants | Some description but missing statistics | Clear statement of participants and basic socio-demographic description with statistics |
| Novice (0) | Developing (1) | Accomplished/ Exemplary (2) | |
| Methods (p-hacked) (2 marks) | No description of methods | Some mention of methods but omission of some elements or procedures or conditions that are reported in the results. Only include those in the p-hacked report! | Clear description of materials and procedures BUT ONLY THOSE THAT ARE REPORTED IN THE P-HACKED RESULTS. |
| Novice (0) | Developing (1/2) | Accomplished/ Exemplary (3) | |
| Results (p-hacked) (3 marks) | Too unclear to follow; analyses not done, not reported | Reporting of final analysis but missing statistics, confusion about meaning of statistic, unclear or inconsistent statements. | Clear and concise report of the final (p-hacked) analysis in APA format and clear link to the hypothesis. Graph is referred to in the write up. Printout of results from analysis program included in appendix |
| Novice (0) | Developing (0.5) | Accomplished/ Exemplary (1) | |
| Appendix: Figure (1 mark) | No figure included | Figure included but some errors in labeling or representations | Figure uses axes and titles correctly and uses the correct variables; represents the data in the results |
| Appendix: what I tried....(1 mark) | No explanation of tests or manipulations | Some explanation of the tests or manipulations tried but unclear; tests or variables not identified clearly | Clear brief description of what manipulations were attempted, what tests were done, clearly enough that the reader realizes your brilliance! |
| Accurate study report | |||
| Accurate participants and methods (1 mark) | Not included or more than two key elements missing | Omission of some elements of the full study, unclear description, confusion between the p-hacked and accurate report | Clear report of exactly who was in the full sample and their characteristics. Clear report of the full list of variables that were collected and all conditions. Clear description of any transformations done to participants |
| Novice (0) | Developing (1) | Accomplished/ Exemplary (2) | |
| Accurate results (2 marks) | Not included or more than 2 key elements missing | Simple test is reported but is not the most straightforward test of the hypothesis as it is proposed in the introduction | Clear, simple report of a direct test of the hypothesis. |
| APA style (2 marks) | More than 4 different types of errors | Mostly correct but mistakes in a couple of areas | Accurate use of APA style for reporting statistics and for structure of report |
| Grade out of 15
/15 | Comments/ Feedback: | ||
