Science is in the midst of a data crisis.
Last year there were more than 1.2 million new papers published in the biomedical sciences alone – bringing the total number of peer-reviewed biomedical papers to over 26 million.
However the average scientist reads only about 250 papers a year.
Meanwhile the quality of the scientific literature has been in decline.
The twin challenges of too much quantity and too little quality are rooted in the finite neurological capacity of the human mind.
Scientists are deriving hypotheses from a smaller and smaller fraction of our collective knowledge and consequently more and more asking the wrong questions or asking ones that have already been answered.
Also human creativity seems to depend increasingly on the stochasticity of previous experiences – particular life events that allow a researcher to notice something others do not.
Although chance has always been a factor in scientific discovery – it is currently playing a much larger role than it should.