Nate Silver: “Think Probabilistically”

Famed statistician and FiveThirtyEight founder says the signal to noise ratio can be addressed.

For years, a revolutionary idea has been brewing. Can we eliminate the analyst as big data systems become smarter? Do you need expertise or just faster computers and more data?

In a presentation at Teradata Universe in Prague, Nate Silver, author of The Signal and The Noise, founder of FiveThirtyEight, and famed statistician who correctly predicted the outcomes of all 50 states in the 2012 U.S. presidential election, says this idea is probably flawed, not to mention a bit dangerous.

Nate Silver, Founder of https://t.co/Cg4e23BQUi and Author Why Some Predictions Fail – But Some Don’t #tduniv pic.twitter.com/6Nal425o2e

— Darryl McDonald (@darryl_mcdonald) April 7, 2014

Finance is a very data driven field but the models are often built on flawed assumptions, he argues. This ranges from the viability of sub-prime loans, or that the increase in housing prices are sustainable. The challenge is, prediction often includes wishful thinking. “It’s uncanny the number of cases where something deemed impossible happens or something believed to be certain never comes to fruition.”

As the saying goes, garbage in and garbage out. “Maybe the data wasn’t that bad,” Silver says, “but put up against bad assumptions we’re given garbage as a result.”

The Bias

Countering poor assumptions can be challenging. To see the whole picture we need to aggregate all sources of information and avoid cherry picking the ones we like.

Finding trusted sources is also difficult. Silver gives examples of CNBC (liberal viewership) and FOX (conservative viewership) polls leading up to the recent US presidential election, which showed unsurprising and overwhelming favor for Obama and Romney for the respective networks. The evening prior to the election where it was largely concluded Obama was the winner, Drudge Report, a conservative news site, populated the homepage with pro-Romney predictions. Silver explains, “News stations are offering facts and information they like, not the ones likely to occur.”

Suggestions: 3 Steps

Instead of looking for data to justify your gut, do a gut check at the end. Common sense may override instructions and experience may be helpful to separate real signals from false ones.

For those looking for certainty, consider these 3 rules of thumb:

1. Think probabilistically - While there is a big market for certainty, sometimes being less confident means you’ve wisely accounted for the risks in your environment. “As you go out in time and space uncertainty increases,” says Silver.

He offers an example of an area that experienced a particularly rainy season. The weather station publicaly offered a flood probability of 49 feet. The levees were built for 51 feet of flooding so the people in the region gave a sigh of relief and carried on. “What they failed to say was that there was a margin of error of 9 feet. That’s a 40 percent probability of flooding.” The waters did in fact hit 53 feet causing great damage to the region. Had the station been more willing to share their uncertainty, and the citizens more prepared, some of the damage could have been mitigated.

2) Know where you’re coming from: “Objectivity is something that none of us has a monopoly on ourselves, in this shared world, this empirical world, perceptions are often skewed and biased by beliefs, but it’s helpful to know what our strengths and weaknesses are,” he says. “You are often more vulnerable in your blind spots… knowing what those are is important.”

As an example, a firm might have strong data, talented engineers and analysts, but if they have incentive to tell the boss something they’d like to hear rather than the truth – a political response rather than a correct one – the system is flawed. “With more data, we have the ability to cherry pick. We have biases, trying to overcome them is the name of the game.”

3) Trial and Error (Try, and Err) Consider the 80/20 learning curve, based on the Pareto principal that 80% of profits come from 20% of customers. In this case, increased efforts see diminished returns on skill proficiency. Significant effort is needed for marginal competitive advantage. It’s an uphill battle but given the very competitive and efficient market today we must take advantage of data sets to make continuous improvement.

The 80-20 rule

Google, for example, runs experiments on search projects and makes thousands of tweaks to their codes to test for better results. Silver closed with words from a Piet Hein poem titled The Road to Wisdom, “Err and err and err again, but less and less and less.”

Becca Lipman is Senior Editor for Wall Street & Technology. She writes in-depth news articles with a focus on big data and compliance in the capital markets. She regularly meets with information technology leaders and innovators and writes about cloud computing, datacenters, ... View Full Bio