At the planet’s most prestigious school, during an event designed to make sense of the most consequential upset in a year full of them, the world’s most prominent statistician admitted that his research and reporting, which millions of readers had turned to for reason during an unreasonable time, may have been open to misinterpretation.
“It’s not intuitive, for people, to have numbers convey uncertainty,” pioneering data
Throughout the exhausting 2016 presidential race, millions followed the mathematical election forecasting popularized by Silver, whose previous precision had made him the darling of the political data world (and the sports world before that). Silver’s polling models correctly predicted the electoral outcomes of forty-nine states in the 2008 election, and in 2012 he called them all. So when his FiveThirtyEight formulas consistently showed Hillary Clinton as about a 3–1 favorite to defeat Donald Trump for most of the political season, onlookers took that to mean America’s first female presidency was all but a lock.
With two weeks to go, Silver had Clinton at 86 percent likely to win, while his previous employers at The New York Times had the Democrat’s odds pegged at 93 percent. Her lead persisted in both until election night, when the Times had Clinton’s odds at 85 percent and FiveThirtyEight measured them to be 71 percent. When Donald Trump clinched his shocking Electoral College victory on November 8, those who had come to believe that the odds were impossibly against
Had the experts made a
Neither, to hear Silver tell it. The mistake that nonquantitative thinkers made, he argues, was presuming fate from a tool meant only to calculate likelihood. They misread the numbers they were looking at. As his Times counterpart Amanda Cox put it in describing her own election night model, “It’s really
Held March 31 at the Center for Government and International Studies in Cambridge, Massachusetts, the Political Analytics Conference is the brainchild of Harvard University academics Ryan Enos and Kirk Goldsberry. Enos is a fellow and professor of government at the school, specializing in political psychology. Goldsberry is a former visiting scholar whose expertise in visual communication and spatial reasoning led him to a career in sports analytics. He began as a sports journalist, presenting innovative new ways of understanding basketball to readers at ESPN’s Grantland, before ascending to his current role as the vice president of strategic research for the San Antonio Spurs.
Sponsored by Harvard’s Institute for Quantitative Social Science and the Center for American Political Studies, the conference is modeled after MIT’s Sloan Sports Analytics Conference. What South by Southwest is to multimedia and Comic-Con is to Hollywood entertainment, Sloan is to the sports intelligentsia. Now in its eleventh year, the conference picks up where Michael Lewis’s Moneyball left off, using principles derived primarily from the economics world to push forward more sophisticated means of understanding and driving success in sports. Goldsberry says it’s time that deeper understanding finds its way beyond athletics and into the realm of politics and political discourse.
“In a weird way, sports is ahead of politics in using analytical reasoning to uncover, to explore, and to confirm ideas,” he says. “We want to provide a forum on an annual basis that brings together quantitative thinking and political discourse.”
“I’ve been looking for something like this for the last couple of years,” says Travis Bunner, a data scientist with polling firm EMC Research who has attended both installments of the conference. “I respect what these guys are doing because there is nothing like this.”
Less unique was the topic of inquiry the conference pursued, which was the same as had been featured at innumerable symposiums since November, not to mention countless dinner tables discussions and barroom debates. Attendees sought to answer one overriding question: How had a fear-mongering reality TV star and purported real estate titan been elected the nation’s forty-fifth president, and why hadn’t any of them seen it coming?
In the days since November, people have grasped for all kinds of explanations. By the readings of some, the election signaled that the nation’s gender biases were more intractable than its racial ones. Others saw race and racial identity as the deciding factors. Political operatives suggested a tactical failure. Some historians called it the inevitable result of ongoing demographic trends. The Harvard conference saw experts testify to all of these as causes of our national blindsiding, including a set of qualitative perspectives on how blind one had to be in order to feel surprised.
The powers and dangers of microtargeting, the lines Google and Facebook draw between their user data and voter registration data, the role of forecasting, race, the way we disseminate political information overall—each was cited as a data point during the day’s expansive postmortem.
In hindsight, according to experts, it was the forecasting models that had given Clinton hopefuls what turned out to be a false sense of security. After years of advance polling bearing out in eventual elections, Silver suggested that the masses overlearned their lessons, saying, “People took 2012 to be the iron law of how politics works, and I think came to some sophomoric conclusions because of that.”
Perhaps the group guiltiest of misunderstanding, misinterpreting, or misrepresenting the data was the national media—some of whom were on hand for confession.
Looking back to the 2012 election, when Mitt Romney’s camp insisted all the way up to the finish line that polling trends would prove to be incorrect, MSNBC television journalist Steve Kornacki conceded that his on-air performance during this latest election night was shaped by a sense of certainty brought on by the leading indicators. “We all came in with a certain expectation [of] where things were likely to go … and the exit polls suggested that was probably where it was going to go,” he told the crowd of about seventy people.
In an industry that puts a huge premium on being first, it was suggested that part of what made reporters treat a Clinton victory as inevitable was a fear of eventual embarrassment.
“I think the iconic moment from 2012 that hung over this entire election in the eyes of the media was Karl Rove, on the set of Fox News, saying, ‘It’s not over! They called it, but it’s not over,’” Kornacki said.
For MTV News political columnist Ana Marie Cox, the election’s biggest surprise would be Trump’s ability to win over college-educated white women, despite his numerous public statements demeaning women and allegations of sexual misconduct.
“It’s one thing to know intellectually that something can happen,” Cox said. “It’s another thing to be able to internalize it and believe it’s a real thing.”
Post-election, this perception gap has frequently been attributed to the idea of self-reinforcing “information bubbles,” largely shaped by interactions with social media. Robert D’Onofrio, Facebook director of data communications, was at the conference to explain that while the company is intimately aware of the issue, there are limits to its willingness to do something about it.
“There’s a behavioral component to it that we can certainly influence, but we can’t change behavior,” D’Onofrio said. “Everyone has their own personal experience on Facebook. So we’re trying to take a product approach to it, letting the user control as much as possible about their experience.”
According to D’Onofrio, 128 million unique Americans discussed the election at some point on Facebook. Internal research shows that the most frequent topics of these conversations were crime and jobs. But as eager as Facebook is to celebrate the volume of the discussions it hosts, the social media giant is reluctant to publicly divulge much data regarding the sentiments expressed by its users, especially around issues as culturally divisive as this last presidential election. And while Facebook could remain nonpartisan about the sentiments of individuals, others have been tasked with delving right into them.
Trump’s success has primarily been attributed to the emergent loyalty of the supposed “white working class.” According to pollster Matt Barreto, who has conducted extensive research into the minds of the electorate both before and after November’s contest through his firm Latino Decisions, the unifying principle held by Trump’s voter base was not rooted in financial concerns but rather in cultural ones.
Even after controlling for partisanship and ideology, “the most powerful explanatory variables for Trump,” Baretta said, “are the variables related to your views towards blacks, your views towards immigrants, and your views towards Muslims. In some cases, depending on how you build the model, the economic variables are flat. They are not more likely to predict a Trump or Hillary vote at all.”
While its various factions might not have ultimately been aligned by economic outlook, one demographic made its influence known as a bloc in November. “One of the ways to understand this election is to say that the white working class started voting like a minority group,” said veteran RealClearPolitics analyst Sean Trende, citing Trump’s 70 percent margin with such voters.
Before last year’s national election, Wisconsin public opinion pollster Kathy Cramer had seen firsthand her own governor, Scott Walker, build a similar coalition around racial and economic resentment in small towns. “They’ve been telling me since 2007 that they are not getting their fair share of attention and respect and resources,” she said. In their minds, Cramer explained, the money they deserve is going to “city” folks, allocated by politicians they see as unsympathetic to their needs.
“The racial anxiety and the economic anxiety are inseparable, and maybe that’s always been the case in the United States,” Cramer said. “Our conversations about redistribution are about race whether we are willing to admit it or not.”
All these swirling, complicating factors were explored during the conference’s competitive “hackathon” component, which saw teams comprising Harvard students, data science and visualization pros, and politicos coming together over a shared set of election data to try to extrapolate new and deeper insights. The winning team attempted to quantify the very anxieties Cramer and Baretta described.
For Matt Lackey, a data expert with experience on Barack Obama’s 2008 campaign who now serves as vice president of research and development for
“Nonresponse bias is a real problem,” Lackey said. “If we lose our ability to accurately measure what people think, we lose our ability to govern.”
As for solving the challenges of a potentially ungovernable democracy, Americans will need to reconcile their discomfort with the differences between big numbers and sure things.
“People tend to underestimate the likelihood of relatively low-probability events,” said New York Times Upshot reporter Nate Cohn. He continued: “Weather forecasters treat this problem way differently. … If you have a five percent chance that you’ll be hit by a hurricane tomorrow, you’re
Washington Post political scientist John Sides warned against over-interpreting a single election. “Had Clinton gotten about a percentage point more of the popular vote, she would have won the Electoral College, and we would have had a conversation about data and analytics that’s the exact polar opposite of the one we’re having now,” he pointed out.
Even if they didn’t call a perfect game, analysts in this realm played a key role in the 2016 presidential election. As in professional sports, the influence of data cannot simply be measured by whether the ultimate victors opted for analytically driven strategies. If the day had a persistent theme, it was that the low probability of a Trump victory and a misunderstanding of what that meant were perhaps the deciding factors that shaped the narrative around the campaign, which could have in turn dramatically affected the electorate.
At Harvard, Sides couldn’t help but paraphrase another old, venerated institution. “The real lesson from this election is that data and analytics are the worst way to run a campaign, except for all the other ways,” he said with resignation.
The truism held up to the scrutiny of a room full of data scientists, by any calculation or measurement. But if the Political Analytics Conference taught attendees anything, it was to be a little careful of looking backward when trying to think forward.
This article was produced in collaboration with the Boston Institute for Nonprofit Journalism.