On May 3, a group of open government experts, including OMB Watch, released a review of federal agencies’ initial Open Government Plans that were published on April 7. Overall, the independent audit organized by OpenTheGovernment.org found that agencies did good work, but much remains to be done.

Under the Obama administration’s Dec. 8, 2009, Open Government Directive (OGD), all agencies were required to produce Open Government Plans within four months. Agencies met the deadline but with inconsistent levels of success. While many agencies went beyond the requirements of the OGD for certain aspects of the plans, others failed to address basic requirements in the directive.

OpenTheGovernment.org identified key differences between plans that excelled and those that underperformed. The coalition cited each plan's level of specificity, ease of accessing information, identification of key audiences, and the quality and sustainability of flagship initiatives as the critical scoring areas that most often made the difference between strong and weak plans. However, the report also noted that many of the deficiencies in these areas can be easily fixed.

The April 7 plans were graded based on the specific requirements set forth in the OGD. The requirements were judged on a 0 to 2 scale, with 0 assigned for requirements that were unaddressed, 1 assigned for partial progress on a requirement, and 2 assigned for satisfactorily meeting the requirement. Bonus points were awarded for exceeding the requirements. The total score possible, excluding bonus points, was either 58 or 60 depending on whether an agency has original classification authority. This is because the OGD had special declassification requirements for those agencies, increasing their total possible points. Agency plans that were awarded bonus points may have exceeded the maximum score of 58 or 60.

Overall, most agencies scored at 70 percent of total points or higher. Fewer than half of all agencies received 80 percent or higher. The top three agencies, which scored above 100 percent, were the National Aeronautic and Space Administration (NASA), the Department of Housing and Urban Development (HUD), and the U.S. Environmental Protection Agency (EPA). It should be noted that no agency achieved 100 percent compliance with the OGD criteria, as can be seen in the agencies' basic scores (scores that did not include any bonus points). Those agencies that scored over 100 percent overcame minor point deductions by earning bonus points.

The report separated the plans’ scores into three groups. The strongest plans included eight agencies that had the most detailed, deadline-specific, and innovative plans. The middle set was the largest, composed of agencies that made strong efforts on the plans but still needed improvements on several requirements. Five agencies made up the weakest set, with plans that were significantly lacking in several components.

The five lowest scores, in order from lowest to highest, went to the Department of Justice (DOJ), the Department of Energy, the Office of Management and Budget (OMB), the Department of Defense, and the Department of the Treasury. Of particular disappointment to many of the evaluators was the poor performance by OMB and DOJ. Given that OMB has responsibility for overseeing portions of the OGD and DOJ has long overseen federal implementation of the Freedom of Information Act (FOIA), evaluators expected these agencies to seize this opportunity to lead by example.

For instance, OMB could have taken this opportunity to make its new contractor accountability database – the Federal Award Performance and Integrity Information System (FAPIIS) – accessible to the public. DOJ’s ranking at the bottom of the stack was also disappointing given Attorney General Eric Holder’s guidance to federal agencies in 2009, which stated his strong support for President Obama’s commitment to open government.

The open government community and the administration both recognize that the Open Government Plans are evolving, "living documents." Kate Beddingfield, a spokesperson for the White House, commented on the OpenTheGovernment.org evaluation, stating, “We also agree that much remains to be done on this unprecedented effort to make government more transparent, and we look forward to continuing to work together with open government advocates and the public on the evolution and implementation of these plans.”

The Department of Transportation has already produced a new version of its plan. The department refers to its plan as “a living document” that will change and improve over time. This second plan was not scored by the audit, which was restricted to only reviewing the initial plans, but it addresses many of the areas for which the initial plan was found to be deficient. The White House Office of Science and Technology Policy has also announced its intention to develop another version of its plan, demonstrating significant interest in the effort coming from federal agencies.

Agencies also conducted self-assessments of their own plans, the results of which are summarized on the White House’s Open Government Dashboard. Comparing OpenTheGovernment.org’s independent evaluation to the self-assessments reveals different perspectives on what the agencies have achieved so far. The White House provided a similar list of 30 specific criteria on a checklist and graded plans on a three-tiered scale: green for fully satisfying the requirement; yellow for partial progress on the requirement; and red for failing to meet the requirement. The dashboard summarizes performance on the criteria in five categories: formulating the plans; transparency; participation; collaboration; and flagship initiative, along with an overall plan score derived from the scores in the five main categories.

The White House assessment shows that three agencies scored green in all five main categories, as well as for the overall plan: the Department of Health and Human Services (HHS), the Department of Transportation (DOT), and NASA. In OpenTheGovernment.org’s independent audit, both NASA and DOT scored very high and were ranked 1st and 6th, respectively.

However, HHS ranked 20th in the independent audit, which placed it near the bottom of middle group. Although HHS was applauded by evaluators for its specific commitments to identifying and publishing high-value data sets in 2010, it did not score well in all areas. HHS was found lacking in demonstrating the sustainability of its initiative and failed to identify specific timelines for the reduction of its Freedom of Information Act request backlog. The most evident reason for the discrepancy between the White House and independent assessments is that the White House gave credit for compliance even if an agency included an aspirational reference to the requirement without concrete steps for meeting its goals. This only merited one point in the independent audit.

In some respects, the independent audit is the embodiment of the OGD in that it has established a new type of collaborative interaction between the public and federal agencies, aimed at improving government openness. Many of the federal agencies have reached out to the independent evaluators to better understand and respond to the assessments. Building on this, the openness community plans to revisit agency plans in June to see what progress, if any, agencies have made on satisfying all of the OGD requirements.

Currently, the open government community is also developing standards for what information each federal agency should, at a minimum, disclose. This "floor" on government openness is important because it can ensure consistency between agencies, which can enable the public to obtain certain information across the government, regardless of which agency website is visited. The floor criteria will focus on providing basic information and actions designed to achieve agency accountability and promote informed public participation. Once these standards are completed, the openness community will begin assessing whether agencies are meeting them.

The evaluators view the agency plans and the audit as the beginning of a process to make government more transparent, participatory, and collaborative. Future audits will eventually transition from focusing on planning to actual progress on taking action to accomplish their specified goals. As agencies move forward in coming months, their efforts to act on their plans will garner increased attention from the openness community.

Photo in teaser by flickr user seagers, used under a Creative Commons license.