big data Archives - The Pursuit of Happiness

Don’t bother answering questions by the next pollster who calls to do a survey. You’re probably going to lie to him. Because “everybody lies.” And there’s no point in taking a survey if you’re going to lie. Besides, Google’s already got you on the truth meter.

That’s one of the main discussion points in the new book, “Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are.”

The blurb on the book says, “By the end of an average day in the early 21st century, human beings searching the Internet will amass 8 trillion gigabytes (GB) of data.” Every day, 8 trillion GB. What does that even amount to? Who knows, but it’s a lot. The average computer has about 4 GB of memory. A flash memory card in a camera may store 16 GB. We’re talking 8 trillion GB – daily.

So what are people searching? Pretty much everything, according to “Everybody Lies” author Seth Stephens-Davidowitz. And the data these searches reveal can be one useful tool for putting the human psyche under the microscope.

“People are honest on Google. They tell Google what they might not tell to anybody else. They’ll confess things to Google that they wouldn’t tell friends, family members, surveyors, or even themselves,” Stephens-Davidowitz said Tuesday in remarks about his book.

Take, for instance, some of the common confessional-style searches that Google gets: “I hate my boss,” “I’m happy,” “I’m sad,” or even “I’m drunk.”

Some of the searches can become rather morose and depressing. For instance, after the San Bernardino attack in 2015, in which 14 people were killed and another 22 seriously injured, top Google searches that soon followed included “Muslim terrorists” and “kill Muslims.” Stephens-Davidowitz says certainly it lacks context to try to guess what people were trying to express in the search, but it also provides guidance.

Here’s one way the data were used. Shortly after the attack, President Obama delivered a speech to try and calm fears about Muslims in America. But his grandiose sermonizing about opening America’s hearts backfired. Even during the speech, people got angrier. But at one point, Obama said that we have to remember that Muslim-Americans are our friends and neighbors, they are sports heroes, and members of the military who are willing to die to defend this country.

Immediately, while the speech was still being given, Google searches for “Muslim athletes” spiked. The increase was so notable that when Obama gave a speech a couple weeks later on the same topic, he skipped the lecturing and focused on the contributions of Muslim-Americans.

Stephens-Davidowitz argues that while Obama’s sermon didn’t tell anybody anything that they didn’t know, the line about sports heroes provoked curiosity, provided potentially new information, and redirected attention. This may not indicate that there’s a science to calming fears after a terror attack, but it does show the power of the data to change how people act and react.

Stephens-Davidowitz says part of the reason why data searches are more useful than old-fashioned survey questions is because people tend to lie in surveys to make them look good. It’s called social desirability bias. It happened during the elections of 2008.

During that time, most Americans surveyed said Obama’s being black didn’t matter. Yet during the election, there was a spike in racist term searches. And graphing that data revealed that racist term searches were geographically divided between East and West. While correlation is not causation, where the racist term searches spiked, Obama lost about 4 percentage points of the vote over the previous Democratic candidate (John Kerry) in Democratic strongholds. He also generated a 1-2 percentage point increase in the number of African-Americans who voted.

The book, “Everybody Lies,” isn’t entirely about politics. It talks about a variety of topics like the stock market, crime, sports, and of course, sex, a hugely commercial enterprise on the Internet. In one example about the truth of big data, Stephens-Davidowitz notes that American women said in recent polling that they had sex (hetero and homosexual sex) once a week and used condoms about 20 percent of the time. Extrapolating the numbers, that would mean about 1.6 billion condoms were used that year. But asking men the same question (about hetero and homosexual sex) resulted in just 1.1 billion condoms allegedly used that year.

So who’s telling the truth, men or women? Neither. According to sales reports, just 600 million condoms were sold during the year in question.

Stephens-Davidowitz conjectures that people have an incentive to tell the truth to Google in a search, more so than to a pollster asking a survey, because they need information. For instance, an increase in the search volume for voting places in an area in the weeks leading up to an election is more likely to reveal whether turnout is going to be high in that location than whether a pollster finds that 80 percent of the people say they will vote.

But is Internet search a digital truth serum? Is it the best way to get real answers? Yes and no.

It depends on how available other high quality data are. For instance, Google flu, which attempted to determine how sick the population was during flu season based on searches about symptoms, was not as accurate as flu modeling currently used by government agencies like the Centers for Disease Control and Prevention.

Furthermore, what people search doesn’t explain why people search. Likewise, Google doesn’t identify who’s searching so we don’t know if the search is a representative sample of the population. There’s no way of knowing what an absolute level of response would generate. For that, we need lots of different types of data.

But Internet searches may be useful in measuring the human psyche more so than in predicting futures. Big data can be helpful in looking at information that does not require very precise numbers. Predicting an election within 5 percentage points isn’t helpful. But it probably is not a big deal to be off by 10 percent when counting the number of condoms used in a year.

As for topics like child abuse, Stephens-Davidowitz says that he’s not actually sure how to use the data to help governments and protective agencies develop programs to identify and address abuse, but that it’s certainly information that would be helpful in filling a gap in reporting. And like any pollster worth his salt will tell you, being able to ask the right question is one vital way of getting to an accurate answer.

Watch the remarks by Stephens-Davidowitz.

“I see no advantage in these new clocks. They run no faster than the ones made 100 years ago.”
― Henry Ford

Henry Ford is credited with making cars better than those who came before him, but he also found a way to make them cheaper. So perhaps you can appreciate how maddening it must have been for Ford to look at the rising cost of goods that didn’t perform any better than their predecessors.

Same is true for social policy. While Ford revolutionized the production lines for cars, America’s homeless policy could benefit from a big dose of innovation. But where do we find the intellectual muscle?

The new book entitled “A Safety Net That Works” brings together some big thinkers on upward mobility, antipoverty programs, and government assistance. Among them is Kevin Corinth, a research fellow in economic policy studies at the American Enterprise Institute, who argues that innovation, on both a small and a large scale, is a key component needed to fix the homelessness crisis facing too many Americans.

Homelessness in America remains a real and daunting problem despite reports of a decline in the number of homeless. While the number of homeless counted since 2007 has fallen, Corinth explains that the changing methodologies used to count the number of homeless may better explain the drop than an actual reduction in the number of people needing shelter. Meanwhile, Corinth reports, “A number of major cities have reportedly seen recent spikes in the numbers sleeping on the street, leading several to declare a homelessness state of emergency.”

Rather than double down on plans to end homelessness with old solutions, we should invest in innovative ideas that push progress forward, while ensuring that resources are prioritized to the people who need them most.”

That seems a simple ask … and a logical start. Knowing who needs help and then tailoring assistance programs to their needs seems like a much less complicated task if we know the population we’re dealing with and the variables in their situations. Without that, current housing assistance programs are throwing possible solutions at the wall to see what sticks.

To start, Corinth divides the homeless population into single adults versus families. He notes that “while 43 percent of homeless single adults are found on the street, only 10 percent of homeless families are found in unsheltered locations.” Disability, mental illness and addiction also play a critical role in identifying homeless individuals.

Better homelessness policy starts with making a fundamental distinction — homeless families are different than homeless single adults, and they require wholly different policy responses. Homeless families generally live in private rooms in shelters. They most often need temporary housing assistance to get back on their feet. Homeless single adults generally sleep on the street or in congregate shelters, and they often suffer from severe mental illness or substance abuse problems. They are more likely to require longer-term, service-rich interventions.”

After identifying who needs help and how we improve upon their current sheltering is just one step. Creating new ways to help people through better prioritization of resources, improved outreach, and increased quality of services, comes next.

How? One way would be to incentivize service providers – program managers who serve the homeless – by holding them responsible for achieving specific goals.

Service providers should be offered substantial flexibility in their service models, but they should be held accountable for their performance in helping their clients achieve desired outcomes.”

One way to do this could be to innovate new ways of tracking people, including where they sleep from night to night, if they are gaining and maintaining employment, and how their physical and mental health is affecting these variables.

Data mining, Corinth says, is critical to this kind of tracking, and as easy as using something as commonplace as smartphones:

Homelessness policy could be reoriented around smartphones and big data. Homeless individuals could be given free smartphones and full service plans in return for providing daily information on their sleeping locations, health status, and other outcomes. Research could be revolutionized with access to detailed, longitudinal data on an otherwise hidden population.”

He certainly does think outside the box. And why not? With more than $4 billion a year spent on programs, greater accountability would certainly help measure success.

Instead of continuing to spend, spend, spend on programs that aren’t meeting goals, we need big thinkers like Corinth to be backed by leaders who control purse strings. We need them to collaborate, to innovate, to invent, and to implement new ways to tackle old problems.

There is no one-size-fits-all to helping those who are homeless. It’s an extremely challenging and complex issue. And while it’s easy to point a finger at the failures to help keep individuals and families safely sheltered, we can look once again to America’s great innovator, Henry Ford, to remind us that it’s not enough to see the problem.

“Don’t find fault, find a remedy.”

Everybody Lies: Except in a Google Search

How Innovation Can Defeat Homelessness