The Demise Of Nate Silver's Infallibility And Data-Driven Journalism

With last month's UK election  once again having left national politics totally confused and close to chaose, with both main political parties are in disarray and the minor parties have once again blown their chances of any significant breakthrough by virtue of sheer ineptitude,  the big data boys with their graphs, charts and tables, are once again running around like headless chickens because most opinion polls pointed in very different directions yet none came close to predicting the result.

 

In 2012 some little statistics nerd named Nate Silver was elevated to the pantheon of Technological Gods, when he correctly predicted, having modelled the outcome of the vote on his meta - analysis of opinion polls, that Barack Obama would win a second term as US President. That was the election in which Obama's opponent Mitt Romney famously threw the fat lady off the stage long before she had even done her warm up scales, when he conceded a key state with less than twenty five percent of the votes counted.

 

It was the comedian Groucho Marx who said "The show ain't over til the fat lady sings" in a 1930s comedy. More recently somebody else said, "It ain’t over till it’s over." With the advent of Big Data and reliance upon data-driven modeling of climates, national economies, political elections, economic performance and sports events we might just as well rephrase it as, “It’s over before it begins.” Which is exactly what one TV pundit did after that Romney concession. When his colleague commented that it seemed a little early to concede, the man, whose name I don't know and have no interest in rediscovering, said something like, "Come on, you know how the science works." But did we?

 

Obviously not, given what happened in Britain's in / out referendum on continued EU membership and the US election of Donald Trump to the presidency in 2016. On the day before the election the divine wisdom of Nate Silver's digital models gave both the Keep Britain In the EU campaign and Hillary Clinton better than 80% chances of winning.

 

Another spectacular Nate Silver cock up in 2016 related to American Football's top prize Super Bowl, Nate Silver’s FiveThirtyEight.com website predicted the New England Patriots to win. Going into the half-time with Atlanta Falcons leading 28-3, the site gave the Patriots a less than 1 percent chance of winning. FiveThirtyEight tweeted: “That Patriots drive took another 5:07 off the clock and actually dropped their win probability from 1.1% to 0.5%.”

 

Of course we all know what happened next. In yet another brilliant statistical upset that exposed Nate Silver's infallible mathematical modelling as no better than guesswork, in the second half the Patriots turned mathematical modelling on its head. They scored 25 unanswered points to win, giving America an timely lesson in how self - belief, determination, and the ability of human beings to make mockery of statistics.

 

It was a lesson for those who tell us we must believe in science and mathematics that sometimes the whole is greater than the sum of the parts, a reminder of the value of commitment, fortitude and preparedness to risk everything; values that were once an integral part of the American mythos, and drove the European pioneers who founded the American nation, but that increasingly become unfashionable and been sidelined in exchange for the perceived infallibility of data-driven analysis and computer modelling.

 

Since information (data) replaced wisdom and judgement and became, for the media at least, the highest level of human intellect, as demonstrated by Nate Silver’s elevation to divine status and FiveThirtyEight’s place in pop culture, a cultural shift has taken place with regard to the use of statistics. No longer were the words of nineteenth century statesman George Canning, "I can make statistics tell me anything except the truth," quoted at those who relied on Big Data, polling, and more mathematical models. On TV shows and social media feeds the pronouncements of nerds with tablet computers were treated as if they were inscribed on tablets of stone. Data-driven analysis, whether accurate or not,(and it was not accurate as often as it was) provided a quick and unchallengable way to excoriate opponents in debate who relied on old fashioned things like empirical evidence. “Silver gives Hilary a x++ percent chance to win the election” became the Trump card in any debate on the US election. Nate Silver gives the Remain campaign x+ chance of winning the Brexit referendum was offered to counter any mention of the anti - E.U. mood in the old English industrial areas of the north and midlands, in South Wales and central Scotland.

 

And in both cases the fat lady was singing at the top of her range, but only those with the ability to see how the mathematical models were bring manipulated to produce politically expedient answers could see the abyss opening up under the predictions of the big data boys.

 

We had reached a point where pundits were willing to assign data modelling more value than observable reality. The irrationality and risk-taking inherent in human decision-making were brushed aside, this was the age of nerds. In politics, sport and weather forecasting, statistical models were held up as unassailable predictors.

 

And in all cases, in 2016, they were wrong. Trump and Brexit, New England Patriots in the US Super Bowl and Leicester City in the English Premier League soccer, and as for climate change predictions, well snow should have been unknown to European and American children by now, the northern ice cap should have disappeared and we were promised 50 million refugees, displaced by rapidly rising sea levels would be wandering the world looking for a new home (that last one was particularly stupid because if all the predictions had come true those people could have settled in the sub - tropical paradises of Siberia, Alaska and northern Canada.)

 

For his part, no matter how certain Silver's fans might have been of the infallibility of his model, the man himself, more often than not, would hedge when asked a direct question. Like those climate scientists whose opinions trigger predictions of sure and certain destruction of life on earth by half past two tomorrow if we do not abandon coal and oil NOW!, Nate's own predictions are fenced in qualifiers, possible, probably, is likely to etc. In October 2016, under a headline that read “Clinton Probably Finished Off Trump Last Night,” Silver commented: “I’m not sure I need to tell you this, but Hillary Clinton is probably going to be the next president. It’s just a question of what ‘probably’ means.” He then spent the bulk of the article presenting the statistical case to back up his claim that Clinton would win, but in the end equivocated again, acknowledging that he could be mistaken, that other signs pointed to the possibility of another outcome.

 

Earlier in the election cycle, when results of the Republican primary, the Michigan Democratic primary, and the general election proved him very wrong, Silver’s excuses involved not moving the goalposts so much as moving the entire effing stadium. He claimed that certain unprecedented events had influenced voters (Wikileaks publishing documents from the DNC 'hack'? The Democrat candidate being a known liar and abuser of office for personal gain?) which skewed the initial models. Even after the Super Bowl, in an attempt to make light of the situation, he tweeted: “At least the Falcons won the popular vote.” To which a user responded, “Nate, you don’t get to make election jokes.”

 

Silver also acknowledged that in any prediction based on mathematical modelling of data, subjective best guesses and assumptions must necessarily be factored into the algorithm. When unprecedented things happen those guesses and assumptions cannot easily be factored out. By saying this Silver admits that statistical analysis of data might work for plotting trends but cannot predict outcomes when people are involved and instinct, emotion and individual circumstances come into play. This was best summed up in an article by David Morris, on Silver’s failure to predict Trump’s victory in the Republican primary: “Unlikely events like the Trump nomination are, by their very nature, impossible to predict.”

 

The models are not ancient oracles, they do not predict the future. At best they are informed guesses on the outcome of events conformed to the established patters of similar events in the past. The trouble with trusting the Oracle, however, is that when events occur, though there may be a superficial similarity, there at many different factors at play.

 

Nate Silver’s infallibility is therefore not the issue. Everyone get things wrong. Silver made his money and his name as a poker player, not one of those pallid, intense characters who spent their lives in smoky, poorly lit back rooms, that we see in movies, but a player against computers in online games. This is interesting because against a computer it's possible to play the percentages and win. In those smoke filled, dimply lit back rooms, emotions, stresses. personal qualities and other things that cannot be factored into a computer program are at work. And then there are the bluffs and the psychology.

 

Yet despite his being fabulously wrong many times, and despite his admissions of fallibility, people still cling to his as the ultimate argument from authority. "Nate Silver says," you will hear as confusion reigns ahead of the UK election, with polls showing leads for the conservatives ranging between one and twelve percent (i.e. a hung parliament to a super - majority). This signals a problem with the scientific, control freak culture leaders such as Barack Obama, Tony Blair, David Cameron and Angela Merkel have been eager to impose. Science demands simple answers and seeks to provide such answers to complex and often unquantifiable questions. When those questions relate to a globalist, scientific views of how the world should be, we should be doubly diligent in questioning 'the science'.

 

It’s not Silver. He’s just a poster - boy for the Church of Sciencology Cult. The irony is that mathematics is not really a science, just as Julius Caesar was proclaimed a god by the mob, so mathematics has been proclaimed a science by an academic mob. In fact it is, in the truest sense of the word, an art; an artifice, something created by humans. Nature does no mathematics, a couple of years ago in a BBC Science documentary on British TV two mathematicians were burbling ecstatically about how the invention of 'zero' (by some ancient Assyrian, Babylonian, Egyptian or Hindu philosopher or had changed our understanding of the universe. Well it may have changed the way we do calculations but it does not change the fact that there is no zero in nature. There may be an absence of something (breathable air for example) but there can never be zero nitrogen, oxygen, carbon dioxide atoms to nature, that would only be significant to a human trying to measure what was (or wasn't) present.Jason Rhode, in Paste Magazine, wrote a withering critique of Silver,  opening with a quotation from Federalist 55: “Nothing can be more fallacious than to found our political calculations on arithmetical principles.” And yet today many seem to believe that Silver is The God Mathematics made flesh, an avatar of a desire for certainty in the unpredictable and unscientific world of human interactions. As novelist Terry Pratchett wrote in "The Thief Of Time," in any battle between order and chaos, chaos will win because it is better organised.

Invoking FiveThirtyEight seemed, before Brexit and Trump, to bestow upon those who quoted the Silverian doctrines an air of both intellectual superiority and mathematical indifference. “Nate Silver predicts…” is akin to saying “Shut up idiot, what do you know? If you don't know 'the science works' you are not entitled to an opinion. The numbers can’t lie, because science!” But that appeal to Silver is really an appeal to the desire of insecure people for a totally stable and predictable future. So how will it play out in Britain? Here are the headline figures for the most recent polls:

Key: Con - Conservative; Lab = Labour; LDem = Liberal Democrat; UKIP = United Kingdom Independence Party

<p>June 5:
ICM / Guardian. CON 45%(nc), LAB 34%(nc), LDEM 8%(-1), UKIP 5%(nc). Fieldwork was Friday to Sunday, so will have been largely before the terrorist attack in London Bridge.
June 4:
Opinium / Observer. CON 43%(-2), LAB 37%(+2), LDEM 6%(-1), UKIP 5%(nc). Fieldwork was on Tuesday and Wednesday and changes are from a week ago.
Com Res / Sunday Mirror. CON 47%(+1), LAB 35(+1), LDEM 8%(nc), UKIP 4%(nc). Fieldwork was Wednesday to Friday and changes are from a week ago.
ICM / Sun. CON 45%(nc), LAB 34%(+1), LDEM 9%(+1), UKIP 5%(nc). Changes are from the ICM poll for the Guardian at the start of the week.
You Gov / Sunday Times. CON 42%, LAB 38%, LDEM 9%, UKIP 4%. Fieldwork was Thursday to Friday
Orb / Sunday Telegraph. CON 45%(+1), LAB 36%(-2), LDEM 8%(+1), UKIP 4%(-1). Fieldwork was Wed-Thurs with changes from a week ago.
Survation / Mail on Sunday. CON 40%(-6), LAB 39%(+5), LDEM 8%(nc), UKIP 5%(+2). Fieldwork was conducted wholly this morning (after London Bridge incident).

June 2:
IPSOS Mori / Evening standard. CON 45%(-4), LAB 40%(+6), LDEM 7%(nc). Changes are from their previous poll in mid-May

Source: UK Polling report

 

Discounting the one percent Conservative lead in the Survation / Mail on Sunday poll as it was an instant raction type of thing, and also because no sample details were made available, we see widely differing results but all pointing to a win for the conservatives. In the past however, while a meta - analysis or poll of polls has been reasonably accurate in predicting overall percentages of the vote for different parties, those percentages cannot project the number of seats won in Parliament, because of the way our system works. Similarly, in the USA in 2016, while Hillary Clinton won the popular vote by virtue of huge majorities in states where her opponent had not campaigned because he knew he could not win, trump campaigned hardest in states hardest hit by globalism, and won the electoral college.

 

If it had been a poker game, Hillary would have been playing against computers, Trump would have been shuffling cards in a dimply lit, smoke filled back room.

 

Ultimately, an unwavering faith in Big Data has become a sort of religious fundamentalism, an attempt, using only belief and awe of the superior being, to impose order on chaos, an almost fundamentalist approach that borders on statistical fanaticism. Unfortunately, just as mathematics is not a science, neither is statistics, which is more akin to the dark arts that the documented and demonstrable proofs of the natural sciences. It’s an attempt to overlook how little we know and how much we imagine we can control.

 

With a large segment of the population and an even larger segment of academics, media pundits and politicians eager to reduce human interaction to data points, which enables them to imagine they could control human nature, we run the risk of becoming an increasingly technocratic society where people value safety over stimulation, ease over innovation and convenience over adventure. Ironically, if some Christian preacher talks about The Creation, Nate Silvers fans will scream in unison about the undeniable fact of the evolutionary process, thereby only affirming that they don't have a clue how survival of the fittest works. (Hint, it doesn't mean those who work out at the most expensive gym and health club will lead the way to the future.)

Comments

Ian Thorpe Added Jun 6, 2017 - 11:23am
This addendum is off topic so I decided not to put it in the main body of the article. Yesterday I was in a thread where one commenter complained about the habit some readers have of demanding links to sources very time we offer an opinion. It was a perfectly valid point and the same thing annoys me.
Can I remind readers however that links in internet text are not merely to identify our sources, the web relies on links, without links we would get very little traffic. So my links in here, mostly to my own original material, are really a cynical self promotion exercise, the easiest way to kick start traffic to my article.
Dino Manalis Added Jun 6, 2017 - 1:46pm
Both Britain's major parties have to turn Right because of constant terrorism, as well as economic and immigration concerns as well.
Ari Silverstein Added Jun 6, 2017 - 3:24pm
I wasn’t aware Mitt Romney conceded a key state too early. What state was that?  I believe Romney lost by a wide enough margin that any state conceded too early would not have changed the result of the election.
 
As far as Big Data and elections are concerned, it’s safe to say Trump made them all look like fourth graders. In fact, I’m not aware of a single prominent pollster that predicted he would win.  In doing that, Trump proved nothing is over before it begins when it comes to elections.  He also proved that one doesn’t need belong to a third party to win as a third party candidate.   
John G Added Jun 7, 2017 - 3:12am
Her's hoping that Jeremey Corbyn's Labour defeats your evil bitch's Tory scum.
Maureen Foster Added Jun 7, 2017 - 5:45am
Before reading this article I had no idea who Nate Silver is and after reading this article I see that it probably doesn’t matter that I don’t know who he is.  I say this because it  seems he's always wrong.  That begs the question, why write an article about such an insignificant person?
 
For whatever it’s worth, because pollsters are often wrong, is not a reason to ignore data and use it to your political advantage.    
Barath Nagarajan Added Jun 7, 2017 - 10:01am
I suppose it's only in the last 30 years or so that people have been able to utilize computers to analyze large amounts of data. The problem is that even with computers there is too much information for anyone to fully understand. A computer can spit out reams of data, but someone has to make sense of it all to understand what it all means. If one even takes fundamental constants in mathematics people haven't fully understood and evaluated the meaning of those numbers, though it is simple enough to use mathamamatical constants and equations to find answers. In fact most computation is now done by computers anyway.
Advance in theory has to go along with the data. In fact there is even more data to make sense of because of computers. This of course isn't bad, but the reality is that we only know a small portion of what there is to know.
Statistical Polling is a relatively simple science, but sometimes it is wrong. Nevertheless, politics isn't about science, that's political science, it's about winning elections.
The statistical model for example predicted that the Marciano would have beat Ali in a famous computer trial fight, though I would trust a fighter or trainer who had seen both fight to be more accurate than a statistical model. No one can really say who would have won with certainty. As someone who plays and watches a sport seriously I can tell that I can predict outcomes and even under certain circumstances occasionally effect outcomes better than a statistical model can.
If you take permutations of a small set of numbers one gets thousands of permutations. We can only learn a small portion of what
data means. Thousands of research papers might be written on any topic in academics. It is unusual for even an expert in any field to have understood them all.
  The central principle in the study of Science and philosophy remains human limitation.
Mircea Negres Added Jun 8, 2017 - 10:55am
Everything is infallible until it fails. At the same time, using mathematics to predict human behavior is dumb because for the most part people do a lot of unpredictable things, many of which can't be programmed or calculated mathematically. This is because programming and mathematics are rational sciences, while a lot of human beings are irrational if not outright insane and insanely stupid. Nice one, Ian. I'd never heard of Nate Silver until now and learned something new today. Thanks for a different kind of post.
Autumn Cote Added Jun 9, 2017 - 5:33pm
For whatever it's worth, if you had engaged your commenters, I would have promoted this article and you would have received a lot more comments.
Ian Thorpe Added Jun 13, 2017 - 12:27pm
Autumn, I have considered your comment and on reflection have concluded that my time was far better spent sitting on the balcony of a hotel with a wonderful view over Galway Bay, very limited Wi-Fi access and, with a little planning, a complete absence of news coverage of the UK election. Or driving on roads with little traffic and enjoying some wonderful coastal scenery.
All things are relative of course, and that might be somebody else's idea of hell.
Ian Thorpe Added Jun 13, 2017 - 12:37pm
To everyone who commented, thanks for visiting and appy polly loggies for not responding. The decision to go away was a complete impulse, we were halfway down a bottle of St. Emilion when we learned our favourite TV show had been cancelled to accommodate a news bulletin extended for election coverage. And in unision Teri and I said, "Oh no, I can't take any more of this shite."
So next morning we packed a bag, drove to the airport and took off. That's the beauty of being retired of course.
Ian Thorpe Added Jun 13, 2017 - 12:40pm
John G, just because my wife is named Teresa you should not assume she is the Prime Minister. Jumping to wild conclusions will only make you look ridiculous.
Ian Thorpe Added Jun 13, 2017 - 3:44pm
As a response to having my wrist slapped (ever so gently) by Autumn, I posted a little update at the bottom of the article so you can see what a wonderful location I spent the past week in. Much better than hanging around in Writer Beat comment threads.
The Burghal Hidage Added Jul 18, 2017 - 11:06pm
It was a good piece Ian. Nothing to apologize for. I am semi retired and live alone. I have time for this, but I have other projects too. When the weather is good I am more apt to spend my time outdoors. The worst day fishing is still better than your best days at work!

Recent Articles by Writers Ian Thorpe follows.