Too many US authors of the most innovative and influential papers in the life sciences do not receive NIH funding, conclude Joshua M. Nicholson and John P. A. Ioannidis in a recent issue of Nature. The authors concluded that three out of five of a carefully selected sample of highly cited principal investigators eligible for NIH support did not in fact receive it.
Not perhaps surprisingly, their claim turned out to be highly controversial (see, for example, rebuttal by Santangelo and Lipman). Nevertheless, the question posed by Nicholson and Ioannidis is an extremely important one, and deserves further analysis.
Getting funding for research represents a key stage in the research cycle (See earlier blog) and one that is becoming more and more prescriptive in terms of determining what research is actually to be carried out and how. For example, the NIH receives many more grant applications proposing outstanding scientific projects than its budget can support. The overall success rate for grants during fiscal year 2011 was 18% – an historic low (See chart on right). This highly selective vetting of research intentions is becoming the norm.
Most HHMI grants are awarded through competitions that have a formal invitation and review process. Because HHMI awards are intended to achieve specific objectives through clearly established programs, it does not encourage and rarely funds unsolicited grant proposals. Does this process impact on the quality of the science subsequently published?
We took a look at the citation data generated by articles published in Nature during 2009 and asked whether the levels of citation achieved two years later in 2011 for work supported by several top flight funding agencies (NSF, NIH, HHMI and Wellcome) differed all from other work published in 2009. The box plot shown here summarizes these results.
The shaded boxes enclose the 25th and 75th percentiles and the ends of the whiskers indicate the 5th and 95th percentiles. The lines within the boxes represent the individual medians and the red line indicates the overall median for the 2011 citation sample. Each of the citation distributions is highly skewed, but the median for the four funding agencies represent a consistent trend to exceed the norm.
Biology is likely to be a confounding factor here – biomedical papers tend the be cited more than non-biomedical ones – but the approach can be further refined by using additional metrics, such as how sustained paper citations are over time – an indication of whether a paper has been of consistent value over time.