CRITIQUE

Page 579

Assess the Constituent Data. What is included? Omitted? What are the data based on?

What assumptions are being made? Different retirement calculators give widely different

estimates of how much savings is needed for retirement because of factors they include or omit

(such as entertainment) and assumptions they make (such as inflation rate or healthiness of

annuities and mutual funds) in the calculations.

Two reputable sources can give different figures because they take their data from different

places. Suppose you wanted to know employment figures. The Labor Department’s monthly

estimate of nonfarm payroll jobs is the most popular, but some economists like Automatic Data

Processing’s monthly estimate, which is based on the roughly 20 million paychecks it processes

for clients. Both survey approximately 400,000 workplaces, but the Labor Department selects

employers to mirror the U.S. economy, while ADP’s sample is skewed, with too many

construction firms and too few of the largest employers. On the other hand, the government has

trouble counting jobs at businesses that are opening and closing, and some employers do not

return the survey. (Both organizations do attempt to adjust their numbers to compensate

accurately.) 7

Check the Currency of the Data. Population figures should be from the 2010 census, not

the 2000 one. Technology figures in particular need to be current. Do remember, however, that

some large data sets are one to two years behind in being analyzed. Such is the case for some

government figures, also. If you are doing a report in 2014 that requires national education data

from the Department of Education, for instance, 2013 data may not even be fully collected. And

even the 2012 data may not be fully analyzed, so indeed the 2011 data may be the most current

available.

Hard to Quantify Sports Participation

How many people participate In sports, and which sports do they choose?

Governments and equipment makers want to know, but the data are fuzzy. Multiple

questions contribute to the lack of clarity.

What is a sport? One survey includes bird-watching.

Who should be counted? Do young children count?

How often do you have to participate in a sport to be counted? Is once a year enough?

How was the count made? Because younger and more active people tend to have only cell phones, a survey

made through landlines probably won’t be accurate.

In case you are curious, the National Sporting Goods Association survey says hiking is the most popular

participation sport in the United States, with over 40 million people.

Adapted from Carl Bialik, “Sports Results that Leave Final Score Unclear,” Wall Street Journal , June 9, 2012, A2.

Choosing the Best Data

Sometimes even good sources and authorities can differ on the numbers they offer, or on the

PRINTED BY: SHERIFAT EGBERONGBE . Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher's prior permission. Violators will be prosecuted. design and execution flaws plus sampling error in the Hopkins report. 8 You will be best able to

judge the quality of data if you know how it was collected.

In their books, The Tipping Point and Freakonomics , Malcolm Gladwell and Steven D. Levitt

and Stephen J. Dubner reach different conclusions about the data on dropping crime rates for

New York City. Gladwell attributes the drop to the crackdown by the new police chief on even

minor crimes such as graffiti and public drunkenness. Levitt and Dubner first explain why the

cause was not a crackdown on crime (the years don’t match well; other cities also experienced

the drop) and attribute it to the legalization of abortion (at the time of the crime drop the first

wave of children born after Roe v. Wade was hitting late teen years and thus prime crime time;

that group was short on the category most likely to become criminal: unwanted children). They

also provide corroborating evidence from other countries. 9

Another factor to consider when choosing data is that conditions change over time. Multiple

studies have shown that people who cohabit with their future spouse are more likely to divorce

than those who do not cohabit. Now, that association seems to be weakening for younger

people. 10 As another example, the number of patents held by a company frequently was used as a

measure of that company’s innovation, but with patents increasingly being used as defensive

strategies for even the smallest design changes, patent citations may be a more accurate measure.

A high number of patent citations may show that the company’s patent truly represents

innovation.

Nicholas Felton displays a page from The Feltron Annual Report , a catalog of his life through custom charts

and graph

PRINTED BY: SHERIFAT EGBERONGBE . Printing is for personal, private use only. No part of this book may be reproduced or transmitted without publisher's prior permission. Violators will be prosecuted.