OMB's Peer Review Proposal Improved But Still Flawed

After receiving strong opposition for its peer review proposal from scientists, environmentalists, and public interest groups, the Office of Management and Budget (OMB) released a massively revised version of the guidance and is seeking public comment on the new version. While many of the changes are significant improvements over OMB's initial policy, the new proposal fails to address some of the most fundamental complaints. OMB's goal is to establish government-wide minimum requirements for when and how federal agencies use scientific peer review. The initial proposal, published September 15, 2003, drew sharp and consistent criticism from various sectors for being overly strict and prescriptive for an activity, peer review, that is highly varied and often tailored to the specific needs of each case. This new OMB proposal, released on April 15, is an attempt to address the sharpest criticism leveled at the original proposal. OMB places a high priority on ensuring the "peer review process is transparent by making available to the public a written charge to the peer reviewers, the peer reviewers' report, and the agency's response to the peer reviewers' report." It also tells agencies, "While it will not always be easy for agencies to quantify the benefits and costs of peer review, we encourage agencies to approach peer review from a benefit-cost perspective." The major shift reflected throughout the revised proposal is increased flexibility and control for the individual agencies engaging in scientific peer review of "influential scientific information." For instance, while OMB's revised policy still contains strict requirements for some peer reviews, it reduces the amount of information that would qualify for this stricter review. The new proposal also allows individual agencies to choose, on a case-by-case basis, among a wider range of peer review types. The new proposal has a stricter, more prescriptive approach to peer reviews for "highly influential scientific assessments." OMB will get to decide what is considered "highly influential," including if "the dissemination could have a clear and substantial impact on important public policies (including regulatory actions) or private sector decisions with a potential effect of more than $500 million in any one year or that the dissemination involves precedent setting, novel and complex approaches, or significant interagency interest." In other words, OMB will sit in the driver seat on determining whether an agency must go through a more rigorous peer review process than generally prescribed. OMB also removed restrictions that would have made scientists employed, associated or funded by federal agencies ineligible for selection as peer reviewers for influential information. "[W]hen a scientist is awarded a government research grant through an investigator-initiated, peer-reviewed competition, there generally should be no question as to that scientist's ability to offer independent scientific advice to the agency on other projects," according to OMB. However, OMB encourages agencies to think twice about a scientist who has a consulting or contractual arrangement with the agency conducting a peer review. Agency employees may be peer reviewers, for the more basic peer reviews, as long as they do not possess a conflict of interest and comply with applicable federal ethics requirements. However, OMB's stricter peer review requirements for "highly influential scientific information" continue to bar federal employees from serving as peer reviewers. The selection of peer reviewers was perceived as a highly controversial provision in the original proposal, raising concerns that it would tilt peer review in favor of business interests. In general, OMB "encourages agencies to consider using the panel selection criteria employed by the NAS [National Academy of Sciences]." Additionally, the new proposal permits agencies to decide on the level of public involvement called for in each peer review. While OMB gives agencies enormous leeway, the proposal warns agencies to "avoid open-ended comment periods, which may delay completion of peer reviews and complicate the completion of the final work product." In another change from the original proposal, OMB allows agencies to automatically exempt "time-sensitive medical, health, and safety determinations" from the peer review requirements. In the original proposal agencies had to request a waiver from OMB for such important health information. The change likely stemmed from the scientific community's strong objections to a political office inserting itself within critical health decisions. The new proposal does not address the concerns raised by many that a government-wide peer review requirement is unnecessary and that OMB lacks the legal authority to establish such requirements. The new proposal contains a section that attempts to establish the need for the proposed government-wide peer review requirements. OMB claims "various authorities have argued that peer review practices at federal agencies need to be strengthened." However, OMB still fails to make a case that a fundamental or overarching problem exists throughout the federal agencies' peer review. Improvements in peer review policies and implementation can almost certainly be accomplished. However, considering that generally agencies were implementing peer reviews effectively it seems likely those improvements would best be achieved by incremental and targeted policies. OMB continues to assert that its legal authority is implied in a series of laws and executive orders including the Information Quality Act, which amends the Paperwork Reduction Act, and Executive Order 12866, Regulatory Planning and Review. Yet, it seems more reasonable that OMB should seek specific and explicit approval for such a precedent setting proposal with government wide impacts rather than cobbling together implied authority from a variety of sources. In fact, Congress has considered government-wide peer review requirements in the past -- either as part of larger regulatory reform measures or as stand-alone proposals -- and failed to enact such legislation. The current OMB action would appear to circumvent legislative prerogatives. An additional complaint raised against the original proposal was that OMB proposed placing itself in an oversight role for federal peer review, a role that the office has never played and is poorly qualified for assuming. The proposal is unclear on the degree of oversight that OMB wishes to assume. The new proposal no longer contains certain oversight provisions outlined in the original, including agencies getting OMB's approval of peer review plans. An interesting new provision proposes that OMB and the Office of Science and Technology Policy (OSTP) establish an interagency panel to foster learning about peer review techniques. However, OMB continues to place itself in a clear oversight role including allowing OMB to approve alternative peer review plans and exemptions from the strictest peer review requirements. Many of the changes reflect concerns raised in the nearly 200 public comments that OMB received and made public. However, it is unclear how many of the changes were called for by federal agencies. In addition to the public comment process OMB collected feedback on its initial peer review proposal from federal agencies in an interagency review process. However, these comments have not been made public. Considering that many of these agencies have significant experience performing peer review, their assessment of OMB's proposal could be extremely informative to the public and should therefore be disclosed. OMB is accepting public comments on the new proposal for 30 days, until May 15. Several public interest groups plan to request a 60 day extension of the public comment period in order to allow fuller consideration of the changes made.
back to Blog