I was on the beach in Florida with a business school prof friend of mine a couple of weeks ago and he blew my mind. I like big ideas about how to change things for the better.
He’s always up to something interesting. When I asked him, “what are you working on?” he surprised me. “Fixing the NIH grant system,” he told me. What!?! He gets paid as a consultant to jump in to an area he may not be familiar with, do some research, and then be the answer man. Someone from the NIH was in a executive managment course he teaches and described the grant “problem.” I love this guy because he approaches complex problems like such a boy scout, with an earnest optimism that any situation can be improved. Whether it be how to get our 6th grade basketball team to play better offense, or how to fix the national debt, he’ll say, “here’s what you do..” and roll out a Four Point Plan. It’s funny…at the same time, I can’t help but play the devil’s advocate and bring up all the complexities and possible roadblocks. But this is an important topic, so let me step out of the way and tell you what he said …
His primary recommendation was to crowd-source solutions to revamping grant review. This is his entry for what he proposes to be an open competition for proposals on how to fix the grant review process. His basic idea is simple, but would be revolutionary. He proposes that projects should be categorized and evaluated in four separate research categories, based on how the work relates to existing paradigms, and that NIH funds should be evenly split amongst these bins. This concept of research diversification is taken from the business world, and it’s designed to speed progress and limit risk by spreading support for knowledge generation across classes of research. A key element of this plan is that competition for funds be within these groups so we fund the best of each type. Based on research he did with NIH scientists, the categories he proposes, are:
1. Pre-hypothesis or exploratory studies. These are experiments and projects that may not claim to have a paradigm or hypothesis but are designed to gather new information.
2. Replication studies. We had a discussion about this one because I don’t think anyone in science really sets out to do work replicating the work of someone else. My friend told me that respondents to his questionnaires told him otherwise. In any event, in a way, this is the most intriguing category to me because it addresses the crisis of reproducibility in science while at the same time giving validity to scientists who are technically adept and hard working but who may not want to compete in other areas. no one I know in science would admit that that We had a discussion about this
3. Paradigm extending studies. This is where most NIH-funded work currently fits. Many bemoan this “safe” way of funding research, and one great strength of this system is to limit the amount of resources that go into “incremental” work.
4. Research that may develop a new paradigm. This is the type of work that gets the most press and that everyone vocally supports. Fresh thinking, new ideas, etc. It’s an intriguing and common-sense ideal to compartmentalize this type of work, to recognise that paradigm breaking is high risk and that not all resources should be put in this basket.
A final important point of this scheme is that if it is adopted across institutes (?!) it would facilitate consolidation of bureaucracy. By imposing a constant framework like this one, it would allow cross-institute comparisons. And lastly, he proposes to grade the success of grantees with a new objective measure of knowledge generation. The belief being that papers and patents are not a sufficient index of how well the money was spent. How should we measure how much useful knowledge is generated by a grant? That’s the big question, and he says let’s open that up to crowd-sourcing.
What do I think? I have mixed feelings about shaking up the grant system just at the point when I think I’ve learned the game. But, it’s clear grantsmanship is a game, and that we owe it to eveyone involved to make it more transparently about funding the best research. The more I thought about this system, the more I can see its benefits. The primary road blocks I see to implementing this system are lack of political will, and getting study sections organized to deal with this restructuring. Study section culture is rigid and I’m not sure whether study sections would subclassify the grants they get or whether new study sections would be needed. I’m really curious about what my colleagues think of this scheme.