The Smell Test for Bad Data

Source: Katherine Barrett & Richard Greene, Governing, Smart Management, May 7, 2015

There are many ways numbers can be misleading. Here are a few. ….

…. Another lesson: Whenever anyone is comparing annual data from one state to another or one city to another, the information should be based on precisely the same time period. Consider financial reports for pension plans. Fifteen of the states base them on a calendar year, while the rest use the fiscal year to calculate annual investment results. When policymakers compare data from two states, the results can be very misleading. For example, Oregon’s pension investment results for 2008 showed a 27 percent loss, whereas California’s showed only a 4.9 percent drop. But this says little about actual pension performance because the 2008 investment results for Oregon represent the full blow of the devastating stock market drop in the fall, whereas California’s results only took the state through June 30th and the fall losses showed up in its 2009 report. …..