Scientists and researchers are trained to do the work, not to document and standardize it.
The problem is the same in bench, computational and clinical science. Anyone whose process runs through steps only they know, at the bench, at a screen or in a clinic, has work that nobody else can repeat exactly and nothing can automate. Methods is for scientists and researchers who want that process documented and standardized: the judgment calls, the spreadsheet routines, the "you can tell when" steps that never get written down. The evidence below is why it matters, and why funders now require it.
Principal investigators and teams
Three people do the same step three ways. Nobody wrote the rule, so nobody can say which way is right, and nothing can be handed off or scaled.
Computational and desk-based researchers
The analysis lives in a spreadsheet routine, a script you patched, a threshold you set by eye. It runs the same way only when you run it.
Bench scientists
The protocol in your hands is more complete than the one in the notebook. A new student learns it by standing next to you, and something is lost in the teaching.
Most of the reproducibility problem is a documentation problem.
Experiments from 53 high-impact cancer papers that could be attempted without asking the original authors for clarification. A third of authors did not help or never replied; 68% shared no data at all. Only 26% of the planned experiments could be completed.
Of 1,576 surveyed researchers had tried and failed to reproduce another scientist's experiment. More than half had failed to reproduce their own. 52% said there is a significant crisis.
Estimated US spend on preclinical research that cannot be reproduced.
The methods section was never the whole method
A methods section is written from memory, after the fact, under a word limit, for a reader who is assumed to know the field. The steps that make the experiment work — how long "until it looks right" is, what a loose pellet looks like, which lot of antibody was the good one — are exactly the ones that get left out, because to the author they are obvious.
The knowledge lives in one person
When that person leaves the lab, the protocol leaves with them. When they train a replacement, something is lost in the teaching. The reproducibility literature measures the cost of that loss at the scale of a field; every lab feels it at the scale of a failed run.
The analysis is the least documented part, and it is not only a wet-lab problem
The methods section describes the bench in detail and the analysis in a sentence: "expression was normalized to GAPDH and plotted." The export, the spreadsheet, the replicate that was dropped, the threshold set by eye, the software version — none of it is written anywhere, and it is exactly where two people analyzing the same data get different answers. Many scientists run these steps themselves without knowing an automated tool already exists, because no one has ever asked them to name the step.
What a complete protocol has to state
- Materials with identifiers: supplier, catalog number, lot, storage, preparation, what "fresh" means.
- Quantities with tolerances: volumes, concentrations, temperatures, times, speeds, and the range that still works.
- Order and timing: what must happen immediately, what can wait, where the pauses are and how long they can be.
- Judgment points with rules: what you are looking at, the decision rule, what a wrong call looks like, how you recover.
- Controls and checks: what runs alongside, what result would stop the run.
- The analysis: what is done to the data, step by step and by which software, with settings and thresholds.
Funders are turning this list into a condition of the money. See who it is for.