The Signal and the Noise

The Signal and the Noise

Nate Silver's book has been a lot of fun to read. There is too much good information in it for me to attempt a comprehensive summary, so instead, here are a few points that I found especially interesting:

  • There is a common perception that houses are good investments, because they increase in value over time. But US home prices have been pretty constant over the long run, according to the Case-Shiller index.

  • Weather predictions like "40% chance of rain" are obtained by running a deterministic simulation multiple times, with slightly different initial parameters. It's not a stochastic simulation, like I thought. Increasing computational power has made weather and hurricane forecasts much better over time, but still, the weather is such a chaotic system that forecasts 8-9 days into the future are no better than historical climate averages.

  • The efficient-market hypothesis explains why, for practical purposes, you probably can't beat the stock market as an individual investor. But if you are a professional trader, there are cases when it's rational to buy, even when the chance of an upcoming crash is high.

  • The author made some $400K in the "online poker bubble" of 2003-2006. But making money in poker relies on having lots of "fish" in the game -- bad players who lose money.

  • Statistical significance as measured by p-values is a broken way to determine the truth of a hypothesis. Here is a cool demonstration of p-hacking -- slicing and dicing your data in order to get p < 0.05 so that you can get published.

  • Most political and economic forecasts are garbage, but prediction markets offer an interesting alternative. Making bets based on your beliefs (in the probabilistic sense of "belief") is the essence of being Bayesian.

  • Earthquakes and terrorist attacks obey a power law distribution. Each individual event is mostly unpredictable, but we can make statements about the frequency of events of a given magnitude. The plots in the book suggest that a 9/11-scale terrorist attack (killing 3,000 people) would happen once every 40 years, and that larger-scale attacks are a clear possibility.