Economists Certain of Statistical Uncertainty

PITTSBURGH – On the designated days of each month, regular as clockwork, the U.S. bureaus of Labor Statistics and of Economic Analysis release the rate of unemployment, worker productivity, inflation, economic growth and the balance of trade with foreign nations.

In addition, the Congressional Budget Office provides U.S. senators and representatives its analyses of the economic impact of the bills they introduce should they become law or fail to be enacted.

For example, should the U.S. Supreme Court decide the Affordable Care Act is unconstitutional, that would increase federal deficits by $353 billion over 10 years, the CBO said June 19. After taking other factors into account, however, the office said the deficits would increase only $137 billion and increase the number of uninsured Americans by 19 million.

The statistics these bureaus issue, reported to a tenth of a percent for the rates of unemployment, inflation, economic growth or contraction, and to one decimal point in the balance of trade, are taken at face value by the public and many members of Congress. They accept the numbers as precise measurements, crediting them with an undeserved accuracy, says Charles Manski, a professor of economics at Northwestern University.

The economics profession and those in the agencies that generate these statistics know better, Manski said.

The upshot is, “Officials use these statistics as if they were gospel,” he told those who attended the second plenary session of this year’s Policy Summit on Housing, Human Capital and Inequality in Pittsburgh.

The Federal Reserve Banks of Cleveland, Philadelphia and Richmond sponsored the policy summit held June 18 and 19. Sharing the stage with Manski were Erica Groshen, commissioner of labor statistics in the Bureau of Labor Statistics, and Jeffrey Kling, associate director for economic analysis in the CBO.

The topic under discussion, “How Does [Statistical] Measurement Contribute to Uncertainty in Public Policy and What Can Be Done about It?” Manski admitted, “is very dry but very important” because it’s naïve to base public policy on data whose accuracy is suspect.

“There’s uncertainty in the statistics we use to set policy,” Manski said. “You know there’s an error but you don’t know the degree of error.”

That’s one reason why the bureaus often issue subsequent reports that seek to clarify and account for data then unavailable.

Sometimes it takes five to 10 years to uncover new and clarifying information, which means policymakers have been basing their assumptions on incomplete or misleading information.

For example, the Bureau of Economic Analysis reported June 3 that the trade deficit “was $40.9 billion in April, down $9.7 billion in March, revised.” As the report explained, “Exports and imports of goods and services for all months through March 2015 shown in this release reflect the incorporation of annual revisions to the goods and services series.”

The problem goes beyond sampling and non-sampling error, Manski said. “Sampling error is well understood. The big problem is non-sampling error. It’s not measured at all by government agencies.”

The agencies, such as the Congressional Budget Office, have “a well-deserved reputation for fairness,” Manski said, but they should still report the uncertainty contained in their numbers including the confidence levels and margins of error.

These can be significant because between 7% and 9% of those companies to whom the Bureau of the Census sends income surveys “refuse to respond,” he said. So the statisticians at Census “make it up,” the economist continued. “The polite word is ‘impute.’ They assign a weight to the missing data” when they make their calculations.

“How good are they [the imputations]?” Manski asked rhetorically. “The Census Bureau admits it doesn’t know the accuracy of its imputing.”

As for the CBO projection of the larger deficits should Obamacare be declared unconstitutional, “The CBO doesn’t do a plus or minus” to the $137 billion it reported, he noted.

While he didn’t offer a prescription as to just how federal agencies should add shibboleths to their reported numbers, he advised agencies to “face up to uncertainness” and shed their attitude of “incredible certitude.”

Groshen, the commissioner of labor statistics, agreed with Manski’s critique and said better communications between state agencies that collect data and send it to her bureau would reduce the scope of the problem. That said, “All measurement has limitations,” she noted.

Quoting the 19th century British scientist Lord Kelvin, “If you cannot measure it, you cannot improve it,” Groshen said that good measurements result in those who set policy making better decisions that have better outcomes.

Bad measures are the result of data that are “incorrect, biased, irrelevant, too late or inaccessible. … Without good data,” she said, “you make what is known technically as ‘lousy decisions.’ ”

The Bureau of Labor Statistics relies upon the states so it can measure labor market activity, that
is, the rate of employment and jobs in demand; working conditions, that is, the number of hours employees work per week; and price changes in the economy, which include how much workers are paid.

With better communications with the state agencies that provide the data to the federal government in a more uniform manner and more timely fashion, Groshen said the BLS and other agencies could better locate the standard errors and understand how to correct for them, identify non-sampling errors, and understand how they revise their figures as they learn new data.

With only 235 employees, the Congressional Budget Office provides between 5,000 and 6,000 estimates per year on how the passage or repeal of laws would affect the U.S. economy, Kling said.

“We are working on how to quantify uncertainty in several different ways but it’s complicated,” the CBO’s associate director for economic analysis said.

His office develops simulation models, Kling said, but “It’s rare that the legislators ask about [the methodology].”

Instead, they’re interested mostly in numbers that support their positions or refute their opponents’ arguments, he said.

Even so, the legislators are pragmatic, Kling continued. They want to know, “Will [a piece of legislation] save money? Will it work?”

Of particular concern of late are policies Congress is trying to adopt that would reduce the deficit. As a result, the CBO is working with the budget committees in both chambers to set spending targets that would promote economic growth while spending less, he said.

Because of the numerous and frequent requests congressmen and senators send the CBO, often late in the process, “We do these things often and in not a lot of time,” Kling said. A congressman might instruct the office, “Look at the effect on interest rates” should a law be passed.

“We provide a range” of what’s likely to result, not definite projections.

That leads to “a picture that gets muddy,” he allowed, and often leads advocates and opponents
“to pick the end that supports their position,
say on a new Medicare policy on who benefits and who pays.”

Important to keep in mind, Manski concluded, is that policymakers should be willing to revise and adapt the programs implemented as conditions change and more data become available.

“It’s important to remember,” he counseled, “that when we set policy it’s not once and for all.”

Manski urged experimenting with more than one policy that would address issues involving labor and education.

With the latter, for example, more than one policy on how to test student teachers could result in a better metric that would lead to widespread adoption.

Copyright 2024 The Business Journal, Youngstown, Ohio.