Grant Applications With a Result-Based Orientation
Outcomes research is focused on results that matter.1–4 An outcomes research approach to research grants would ask what we have accomplished with the resources that were invested. In a previous Editor’s Perspective, I questioned how we might assess the value of our clinical research and determine whether it was worth its cost.5 The focus was on research that is intended to influence clinical practice, healthcare delivery, or healthcare policy. I asked whether funders would pay as much (or even more) for the end result of the research as they actually did when the project was initially funded. In essence, are the research products worth what they cost?
The question is important because the resources invested in research have opportunity costs. If the end results of the research are, on average, worth less than the research cost, then we are not optimally deploying those resources. We cannot expect every project to yield benefits beyond costs because that could be achieved only if low-risk projects were funded. However, if the value does not exceed the investment during the course of a large number of investments, there is a problem.
In a thought experiment, I had suggested that we could assess the value of research by paying for it after completion. In this scenario, researchers would find investors willing to pay the highest price for the completed products of the research. The market would dictate the price for the research and reflect its value. This whimsical approach was intended to provoke reflection on the true value of our research products. In the real world, we will certainly continue to have funders seeking to support research, and funding that is awarded before the project begins.
Therefore, the challenge we face is to find ways to identify research proposals that are likely to produce products with an ultimate value that exceeds their cost. In the current approach, reviewers generally judge grants by their significance, investigators, innovation, approach, and environment. The value of the research is best described in the sections of the grant related to significance. The significance criteria evaluate whether the project addresses an important problem in the field and how it will improve scientific knowledge, technical capability, and clinical practice. The grant application for the National Institutes of Health requires applicants to explain how the successful completion of the aims will change the concepts, methods, technologies, treatments, services, or preventive interventions that drive the relevant field.6
The current approach has face validity, but the yield is hard to determine. We do know that approximately half of funded clinical trials are not published within 3 years.7 Even those funded by the National Institutes of Health are commonly delayed in their publication or are never published.8 Although there are many reasons for nonpublication, this rate still suggests a problem.
In the spirit of quality improvement, it seems worth some experimentation with new models of grant application. New models would be configured to make it easier for applicants to convey, and for reviewers to appreciate, the value of the research, and thus improve the likelihood of making better award decisions.
I would like to see an application that helps reviewers envision the final scientific product. The first question for any funder is whether the research will produce an outstanding scientific product. If the science is not strong, the application will not fulfill this requirement. At that point, reviewers need to consider whether the product is worth the resources that are being requested.
A simple proposal for a new approach would be an application that consists primarily of a sample manuscript, written in the format of a top journal, which presents the applicant’s vision of the completed study, including a high-level description of the anticipated main findings. This approach focuses the application on the end product and prompts applicants to think about the future. If the study is intended to produce multiple major publications, then the applicants might be allowed to submit >1 anticipated manuscript. For grants that are funding a clinical trial, a single major publication might be most appropriate.
To best assess the grant, reviewers need detailed information about the methods, approach, innovation, and feasibility (Figure). Thus, an additional component would be a supplemental appendix. Already a requirement of many journals, the appendix would contain the research protocol and details about other technical and operational aspects of the project. Importantly, the research should be able to be replicated from the information in the supplemental appendix.
Research has many purposes, but the ultimate test of its value is based on what has changed as a result of the work. With this in mind, another part of the application would be a plan for dissemination and application that describes how the science would be translated into action and what changes would accrue. This piece would provide reviewers with additional information to help them determine the importance of the science and the likelihood that change will occur as a result of the findings.
The application would also contain a media packet, with items including a press release, which might accompany the funded project. This component, written in easily understood language, would be particularly helpful for laypersons who may be on the review panel and would provide a best-case scenario of what the lay public might hear about the project.
The final part of the application, already required by funders, would address the budget by including a traditional justification of the financials and a bottom-line amount of funds requested.
The central thesis of this approach is that reviewers will be able to make better decisions if they are given material that represents what will likely be produced by the research. Such an application could also streamline the process for applicants and could serve as a tool to help them focus on the end product and impact on the proposed work. It is unknown whether this approach could improve the yield on our clinical research investments, but it seems sensible to arrange the process such that we begin with the end in mind. Funders should be asking what will be produced and whether it ultimately has the capacity to improve the lives of patients and the health of the public. They need to place their reviewers in the best position to answer that question during the initial review—and to do so with clarity and brevity. An outcomes research approach to grant applications has the capacity to ensure that more projects that will make a difference are funded by improving our ability to assess what we are purchasing with available resources.
Sources of Funding
Dr Krumholz is supported by grant U01 HL105270-04 (Center for Cardiovascular Outcomes Research at Yale University) from the National Heart, Lung, and Blood Institute.
Dr Krumholz is the recipient of a research grant from Medtronic, Inc, through Yale University, for the purpose of disseminating Medtronic’s trials, and is chair of a cardiac scientific advisory board for United Health.
The opinions expressed in this article are not necessarily those of the American Heart Association.
- © 2013 American Heart Association, Inc.
- Krumholz HM
- Krumholz HM
- Krumholz HM
- Krumholz HM
- 6.↵Department of Health and Human Services. Research project grant (parent R01). http://grantsnihgov/grants/guide/pa-files/PA-11-260html. July 27, 2011. Accessed August 21, 2013.