If you want to hear a scientist groan, ask about the grant applications he or she is writing. You’re likely to hear a rant about the tight federal science budget or the belief that government grants more often fund incremental research than bold ideas. Dr. John Ioannidis of Stanford University decided to apply scientific analysis to examine how science is funded, and discover what trends lie beneath anecdotal experience.

In an analysis and commentary published Wednesday in the journal Nature, Ioannidis and Joshua M. Nicholson of Virginia Tech found that the majority of researchers who led the most influential studies—papers from the past decade that received more than 1,000 citations by other scientists—did not have current funding from the National Institutes of Health, the predominant funder of biomedical research in the United States. In fact, three out of every five authors were not currently supported by NIH money.

That could be for a variety of reasons: Some scientists may have been graduate students when they did the influential work and may be setting up their laboratories, for example. Scientists may have left the field or moved to a company. The authors didn’t look at whether they had NIH funding when they did the pivotal work. But the finding that so many leading-edge scientists aren’t funded by the NIH raises the question, Ioannidis argues, whether the NIH encourages conformity, or even mediocrity.

Advertisement - Continue Reading Below

Ioannidis isn’t afraid to be provocative. Over the years, he has examined bias in research and found that many results ultimately fail to hold up. In an interview, Ioannidis said that the situation may lead researchers to hold back their best ideas because they know they won’t get funded.

“If I feel I have a really good idea, I will not apply for [funding for] it," Ioannidis said. “If I feel that it’s my best idea and something I really love and that I want to do, it’s a waste of time.”

Instead, good ideas are something Ioannidis says he pursues on his own time.

For another perspective, I spoke with a scientist who is soon going to have to start thinking deeply about how life sciences research is funded: Erin K. O’Shea, a Harvard professor of molecular and cellular biology who in January will begin a new job as chief scientific officer of the Howard Hughes Medical Institute. Howard Hughes takes a different approach to choosing scientists to support, focused not on the particulars of their research plans, but based on each individual’s track record.

Q: Do you think there is too much conformity in how grants are awarded?

A: I do. I actually do. ... Consider the positive first: it works well for people who have established a track record, even as a graduate student or post-doc, and who are continuing on an expansion of the problem or something very closely related to that upon which they previously worked. That is the kind of thing that the review panels like to see. They like to see someone who they recognize, associated with the field, continuing on a logical plan—a well-laid out, well-considered plan.

It works badly if someone tries to do something that they haven’t done before. ... When you try to step out of the field in which you work, it is very, very difficult to get money, and I speak with firsthand experience. For 20 years I had NIH grants. I do not now and it’s in part because I changed fields.

If you don’t continue along the kind of track and want to do something new that involves different methods or different approaches or a different problem or a different system, it’s very hard to get money from the NIH to to that. It’s not impossible, but it’s very difficult.

Q: Has the problem gotten better or worse?

A: My sense is this kind of conformity problem has gotten worse over the years as money gets more and more restricted and science gets bigger and bigger. ... The review panels are comprised of NIH-funded people ... very few of whom have made these high-impact discoveries. ... People have an actual tendency to select for a type that is their own.

It’s not that NIH doesn’t want the high-impact people on the review panel. It is a vicious cycle: once people get discouraged by the system ... serving on a review panel is an extraordinary amount of work and it’s extraordinarily important, and I think in the past NIH was more successful at attracting the top group of scientists for service into these panels.

Q: I’ve heard scientists say that the grants that do get funded are incremental. What do you think?

A: I absolutely think that’s right, and it’s a consequence of the way grants are judged. A large part of the judgment is whether the panel members believe the work is likely to succeed ... where one can lay out a neatly described set of experiments and possible outcomes and a sort of flow chart of what one would do. Given this result, one would do this next. Many things in science don’t lend themselves to that type of flow chart, and in fact, some of the most interesting things do not, and the flow chart looks like a hairball. They don’t like that kind of proposal.

To be fair, [the NIH] has created new models for funding, based in part on [Howard Hughes Medical Institute], of funding the investigator. The NIH has several programs in that vein where they’re funding the person, in large part.

I think that changing more of the funding to a model where you’re judging the investigator and you’re giving people money more like Hughes does, based on past performance and less based on a detailed plan [makes sense]. Not all of it should be that way, but more of it should, I think.