On Nov. 19, the Government Accountability Office (GAO) released a report that details the first round of Recovery Act recipient reports. The GAO report focuses on data quality issues, which have garnered attention following widespread news stories about bad data in the Recovery Act reports. While the GAO report itself is informative, its recommendations, which call for improved guidance from the Office of Management and Budget (OMB), are particularly important. The recommendations echo earlier comments from transparency groups, which have long warned of potential data quality problems, especially concerning the job estimation data.

The GAO report is narrowly focused and seeks to examine "the jobs created or retained as reported by recipients" within the 17 jurisdictions (16 states and the District of Columbia) the GAO has been studying. The report did not undertake an in-depth audit of the recipient reports, in that it did not contact most of the recipients who reported in the first cycle; instead, the GAO looked for obvious errors or inconsistencies in the data. Using this method, the GAO found there were errors in a significant number of reports.

The GAO uncovered a wide variety of problems, many involving the job creation estimates. It found 3,978 reports (out of 56,986 studied) that "showed no dollar amount received or expended but included more than 50,000 jobs created or retained," and "9,247 reports that showed no jobs but included expended amounts approaching $1 billion." At the same time, the GAO found 261 reports where the "job creation narrative" field contained words such as "zero," "none," or "N/A," but showed jobs created or saved in the "number of jobs saved" field. These reports accounted for about a tenth of the jobs reported created or saved nationwide as listed on Recovery.gov.

However, the GAO found these major mistakes were relatively rare, since the erroneous reports only constituted a small fraction of the number of overall reports. The GAO hypothesized that most of the errors had one of two root causes: many were simple keystroke errors, and the rest were likely due to confusion among the agencies and recipients. The GAO is primarily concerned with the second point, as it indicates confusion over the guidance from OMB and a lack of clear communication between OMB, federal agencies, and Recovery Act recipients.

Expanding on this point, the GAO report specifically criticizes OMB's handling of job creation estimates. GAO's communications with recipients revealed that many were confused by OMB's guidance on how to calculate these estimates. The problem, it seems, is that OMB did not use a standard job creation definition. The current OMB guidance leaves it up to recipients to decide what constitutes a full-time job (the so-called "full-time equivalent," or FTE). To correctly estimate their FTEs, recipients had to take the number of hours worked on Recovery Act projects and divide it by the number of hours in a typical full-time schedule. The resulting figure is the number of jobs created or saved by the project (the FTE). Some recipients, however, were unsure how to use this formula or what it meant. For instance, some recipients simply entered in the number of actual people hired or retained, regardless of how many hours they worked. Such mistakes account for the errors described above, where recipients claimed that they created or saved jobs, despite having received no Recovery Act funding yet. Another common problem consisted of recipients simply entering the number of hours worked, leading to a drastic overestimate of jobs.

The lack of a standard FTE definition leads to other problems as well, according to the GAO report. It also makes it impossible to compare jobs across projects or awards, and especially across the country. For instance, if one highway contractor considered an FTE to be three months of a 40-hour work week, then a three-month job would result in one FTE. However, if another contractor considered an FTE to be a year's worth of 40-hour work weeks, then the same three-month job would only equal 0.25 FTEs. In other words, similar jobs, for similar amounts of money, can yield apparently vastly different job creation numbers. And since recipients only report the final FTE determination, and not the standard by which they arrived at the number, it is impossible to tell how individual recipients arrived at their job creation estimates, or what the estimates actually mean.

The lack of a standard FTE is not a minor problem, as the GAO found many recipients used different FTE standards. For instance, four Pennsylvania transit agencies all used different FTE measures, as did two California institutions of higher education. This fact has profound implications for the 640,329 jobs figure posted on Recovery.gov. As the GAO notes, "the current OMB guidance … creates a situation where, because there is no standard starting or ending point, an FTE provides an estimate for the life of the project. Without normalizing the FTE, aggregate numbers should not be considered."

It is unsurprising, then, that the GAO's main recommendation concerns the standardization of the FTE. First and foremost, the GAO recommends OMB should "clarify the definition and standardize the period of measurement for FTEs and work with federal agencies to align this guidance with OMB's guidance and across agencies." A clear standard for estimating FTEs would help prevent many of the problems the GAO found in the recipient reports, while also allowing comparison across states and projects.

Second, GAO recommends that OMB clarify how recipients should report information for jobs saved. Under current guidance, it is not clear how recipients should report funding used to continue to pay existing staff. GAO recommends that the guidance be changed to clearly show that recipients should simply report “hours worked and paid for with Recovery Act funds,” essentially removing the distinction between “created” and “saved.” Such a change would stop recipients from engaging in strange hypothetical situations to decide if employees would have been fired without the funding, further reducing recipient confusion, as well as helping to convey the actual impact of the Recovery Act.

Such ideas are by no means new. OMB Watch has written extensively on the issues and has repeatedly made recommendations similar to the GAO's. However, this is the first time a federal agency has made the recommendations, giving them additional weight.

OMB Watch has also noted other problems with the jobs data. For example, Recovery Act grantees are to report on jobs created or saved by their subrecipients or themselves. However, contractors are not yet required to report information about their subrecipients. Also, job information is limited to the prime recipient and one tier below that entity; it does not always reach the ultimate recipient of the funds. GAO did not address these other types of problems in its report.

In response to the GAO report, OMB said it "generally accepts the report's recommendations." While this statement does not necessarily portend significant change from the agency, it might show OMB understands that the first reporting cycle could have been better if the agency had improved its own guidance. It remains to be seen if the report will affect the next reporting period, which ends in December.