In a talk presented to Auburn University investigators earlier this month, CCTS Codirector of Biostatistics, Epidemiology, and Research Design (BERD) and UAB Dept. of Biostatistics Chair Dr. David Redden reviewed steps investigators can take to avoid cognitive fallacies in research that undermine reproducibility.
Early in 2016, the NIH implemented new requirements for rigor and transparency in grant applications. The new instructions and concordant review criteria, which are intended to enhance the reproducibility of federally funded research, emphasize four key areas: (1) scientific premise (2) authentication of key biological and/or chemical resources (3) consideration of relevant biological variables and (4) rigorous experimental design for robust and unbiased results. Dr. Redden’s talk focused in particular on rigorous study design and data management techniques to ensure transparency.
Several key tips are highlighted below:
- Explicitly consider alternative hypotheses—then test them out head-to-head.
- Publicly declare a data collection and analysis plan before starting the study.
- Provide all assumptions required to replicate the power calculations within a grant (clinical relevant effect size, assumed variance, Type 1 error rate, sample size per group, and power).
- Develop a data management plan, including the need for archiving and version control.
- Seek an independent blinded analysis by a methodologist if you have conducted your own analyses.
- Take advantage of statistical software packages that support production of a “reproducible document” along with analyses.
The best tip he could provide, Redden said, is “to work with a methodologist early and often in the grant writing process to address NIH requirements for rigor and transparency.”
To request a BERD consult or a CCTS panel to review a grant application, email