On December 17, 2015, the NIH published guide notice NOT-OD-16-034 to notify the community of upcoming requirements for formal instruction in rigorous experimental design and transparency to enhance reproducibility. This notice applies to institutional training grants (D43, T15, T32/TL1, T34, T35, T36, T37, T90/R90, and U2R), institutional career development awards (K12/KL2), and individual fellowships (F05, F30, F32, F37, F38, and FI2).

These new instructions and revised review criteria will focus on four areas deemed important for enhancing rigor and transparency (more information here):
1. Scientific premise of the proposed research
2. Authentication of key biological and/or chemical resources
3. Consideration of relevant biological variables
4. Rigorous experimental design for robust and unbiased results

FAQs

NIH Examples

UAB Activities:

Currently, UAB has two NIH-funded programs focused on rigor and reproducibility – related training; information below was either adapted from materials posted in the NIH RePorter or the grant application.

1. Dr. David Allison (PD)  1R25GM116167-01

BEYOND TEXTBOOK, YET SIMPLE, STATISTICAL TOOLS FOR REPRODUCIBLE ANIMAL RESEARCH

The capacity of laboratory animal research to be translated to interventions for human health is dependent on research reproducibility, replicability, and generalizability (RRG). It is often the case that challenges with RRG arise from differences in scientific protocols, unknown or uncontrolled confounding factors, incomplete communication of scientific protocols, and unintentional misapplication of statistical principles, rather than the oft-discussed research misconduct. Many of these challenges with RRG involve practical and theoretical aspects of statistics and study design that go beyond what is taught in basic statistics courses or textbooks.

With this grant, Dr. Allison’s team is creating a set of animated vignettes to educate early career, laboratory animal investigators (namely, graduate students, postdoctoral fellows, and beginning investigators) about RRG issues. Specifically, modules on variability, power, randomization, hypothesis testing, and inferences that address common, and yet not routine-textbook needs of laboratory researchers are being developed.

Each module will be composed of several short, animated vignettes that will 1) discuss a statistical or design principle that is important for RRG; 2) foster dialogue and collaboration between statistical and laboratory animal scientists; and 3) model diversity of statistical and laboratory animal scientist with respect to expertise and demographic characteristics. Each vignette will also be evaluated for content validity, face validity, and educational value by consulting with statistical experts, experienced lab animal researchers, and early career investigators, respectively. Finally, vignettes will be reinforced with additional online instructional content, including tutorial literature and self-assessment quizzes.

Dr. Allison and colleagues will widely disseminate the instructional materials at UAB and beyond via an openly available web portal.

2. Dr. Brad Yoder (PD)           3T32GM008111-28S1

MASTERING THE ART OF REPRODUCIBLE SCIENCE

The problem of non-reproducible outcomes in the scientific literature is rapidly eroding the credibility of biomedical research. The goal of the Mastering the Art of Reproducible Science course is to advance the visibility and awareness of this critical issue and to equip students to better recognize and eliminate sources of irreproducible data. The course will explore the fundamental causes and consequence of data irreproducibility, discuss best-practice procedures to minimize data irreproducibility, and discuss the responsibility of the scientific community to confront the irreproducibility crisis. The course is structured around four month long modules using a team based learning strategy.  

The four modules will cover:

Module 1 Reproducibility issues related to research tools and reagents.

Module 2 Reproducibility issues related to approaches, pressure, bias, data analysis, and presentation.

Module 3 Reproducibility issues relate to preclinical testing and failures in clinical studies.

Module 4 Reproducibility issues involving gender, race, age and health disparities.

Each module will consist of four one-week long blocks that will involve independent reading and assessment, literature based research, and team preparation for in class presentations that will facilitate active debate on major issues of reproducibly of scientific findings. A final class will consist of a panel forum consisting of researchers, journal editors, peer reviewers, clinicians, academic and industry scientists who will discuss their roles in addressing the reproducibility crisis in scientific and clinical research.