CRITIQUE
Page 579
Assess the Constituent Data. What is included? Omitted? What are the data based on?
What assumptions are being made? Different retirement calculators give widely different
estimates of how much savings is needed for retirement because of factors they include or omit
(such as entertainment) and assumptions they make (such as inflation rate or healthiness of
annuities and mutual funds) in the calculations.
Two reputable sources can give different figures because they take their data from different
places. Suppose you wanted to know employment figures. The Labor Department’s monthly
estimate of nonfarm payroll jobs is the most popular, but some economists like Automatic Data
Processing’s monthly estimate, which is based on the roughly 20 million paychecks it processes
for clients. Both survey approximately 400,000 workplaces, but the Labor Department selects
employers to mirror the U.S. economy, while ADP’s sample is skewed, with too many
construction firms and too few of the largest employers. On the other hand, the government has
trouble counting jobs at businesses that are opening and closing, and some employers do not
return the survey. (Both organizations do attempt to adjust their numbers to compensate
accurately.) 7
Check the Currency of the Data. Population figures should be from the 2010 census, not
the 2000 one. Technology figures in particular need to be current. Do remember, however, that
some large data sets are one to two years behind in being analyzed. Such is the case for some
government figures, also. If you are doing a report in 2014 that requires national education data
from the Department of Education, for instance, 2013 data may not even be fully collected. And
even the 2012 data may not be fully analyzed, so indeed the 2011 data may be the most current
available.
Hard to Quantify Sports Participation
How many people participate In sports, and which sports do they choose?
Governments and equipment makers want to know, but the data are fuzzy. Multiple
questions contribute to the lack of clarity.
What is a sport? One survey includes bird-watching.
Who should be counted? Do young children count?
How often do you have to participate in a sport to be counted? Is once a year enough?
How was the count made? Because younger and more active people tend to have only cell phones, a survey
made through landlines probably won’t be accurate.
In case you are curious, the National Sporting Goods Association survey says hiking is the most popular
participation sport in the United States, with over 40 million people.
Adapted from Carl Bialik, “Sports Results that Leave Final Score Unclear,” Wall Street Journal , June 9, 2012, A2.
Choosing the Best Data
Sometimes even good sources and authorities can differ on the numbers they offer, or on the
PRINTED BY: SHERIFAT EGBERONGBE . Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher's prior permission. Violators will be prosecuted. design and execution flaws plus sampling error in the Hopkins report. 8 You will be best able to
judge the quality of data if you know how it was collected.
In their books, The Tipping Point and Freakonomics , Malcolm Gladwell and Steven D. Levitt
and Stephen J. Dubner reach different conclusions about the data on dropping crime rates for
New York City. Gladwell attributes the drop to the crackdown by the new police chief on even
minor crimes such as graffiti and public drunkenness. Levitt and Dubner first explain why the
cause was not a crackdown on crime (the years don’t match well; other cities also experienced
the drop) and attribute it to the legalization of abortion (at the time of the crime drop the first
wave of children born after Roe v. Wade was hitting late teen years and thus prime crime time;
that group was short on the category most likely to become criminal: unwanted children). They
also provide corroborating evidence from other countries. 9
Another factor to consider when choosing data is that conditions change over time. Multiple
studies have shown that people who cohabit with their future spouse are more likely to divorce
than those who do not cohabit. Now, that association seems to be weakening for younger
people. 10 As another example, the number of patents held by a company frequently was used as a
measure of that company’s innovation, but with patents increasingly being used as defensive
strategies for even the smallest design changes, patent citations may be a more accurate measure.
A high number of patent citations may show that the company’s patent truly represents
innovation.
Nicholas Felton displays a page from The Feltron Annual Report , a catalog of his life through custom charts
and graph
PRINTED BY: SHERIFAT EGBERONGBE . Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher's prior permission. Violators will be prosecuted.