The Signal and the Noise by Nate Silver

34 results back to index

Outnumbered: From Facebook and Google to Fake News and Filter-Bubbles – the Algorithms That Control Our Lives by David Sumpter

affirmative action, algorithmic bias, AlphaGo, Bernie Sanders, Brexit referendum, Cambridge Analytica, classic study, cognitive load, Computing Machinery and Intelligence, correlation does not imply causation, crowdsourcing, data science, DeepMind, Demis Hassabis, disinformation, don't be evil, Donald Trump, Elon Musk, fake news, Filter Bubble, Geoffrey Hinton, Google Glasses, illegal immigration, James Webb Space Telescope, Jeff Bezos, job automation, Kenneth Arrow, Loebner Prize, Mark Zuckerberg, meta-analysis, Minecraft, Nate Silver, natural language processing, Nelson Mandela, Nick Bostrom, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, p-value, post-truth, power law, prediction markets, random walk, Ray Kurzweil, Robert Mercer, selection bias, self-driving car, Silicon Valley, Skype, Snapchat, social contagion, speech recognition, statistical model, Stephen Hawking, Steve Bannon, Steven Pinker, TED Talk, The Signal and the Noise by Nate Silver, traveling salesman, Turing test

He had accumulated financial resources (FiveThirtyEight is owned by ESPN) that had allowed him to build sophisticated models based on large quantities of reliable data. From reading his book, The Signal and the Noise, I could see that he was an intelligent and level-headed individual, who had thought deeply about how predictions work. He knew the maths and he understood the relationship between data and the real world. If anyone could create a good model of an election, it was Nate Silver. When it came to analysing our behaviour, the algorithms we have looked at up to now were, at best, on a par with humans. Julia Dressel’s Mechanical Turkers were able to predict probability of reoffending to a level similar to that of a state-of-the-art algorithm, but using a lot less data; personality models based on likes were still a long way from ‘knowing us’ as individuals and Spotify was struggling to come up with music recommendations as good as those of our friends.

…

Candid here, here Fair Housing Act (US) here fairness here fake news here, here, here feedback loops here MacronLeaks here post-truth world here, here, here false negatives here, here false positives here, here, here, here Fark here Feedly here Feller, Avi here Fergus, Rob here Ferrara, Emilio here filter bubbles here, here, here FiveThirtyEight here, here, here, here Flipboard here Flynn, Michael here football here, here robot players here, here Fortunato, Santo here, here Fowler, James here Franks, Nigel here Frostbite here Future of Life Institute here, here Gates, Bill here Gelade, Garry here gender bias here, here, here GloVe (global vectors for word representation) here Genter, Katie here Gentzkow, Matthew here, here Geoengineering Watch here, here Glance, Natalie here GloVe (global vectors for word representation) here Go here, here, here, here Goel, Sharad here Google here, here, here, here, here, here, here, here, here, here, here, here, here, here, here, here artificial intelligence (AI) here, here, here black hats here, here, here DeepMind here, here, here, here, here, here, here, here ‘Don’t be evil’ here Google autocomplete here, here Google News here Google Scholar here, here, here, here Google Search here Google+ here personalised adverts here, here, here, here SharedCount here Gore, Al here Grammatas, Angela here, here Guardian here, here, here, here, here, here, here, here, here, here, here, here, here, here Guardian US here, here h-index here, here Häggström, Olle here, here, here Here Be Dragons here Hassabis, Demis here, here, here Hawking, Stephen here, here, here He, Kaiming here Her here Higginson, Andrew here Hinton, Geoffrey here HotUKDeal here Huckfeldt, Bob here, here, here, here Huffington Post here, here, here Independent here Instagram here Internet here, here, here, here Internet service providers (ISPs) here Intrade here Ishiguro, Kazuo Never Let Me Go here iTunes here, here James Webb Sapce Telescope here Jie, Ke here job matching here Johansson, Joakim here, here Journal of Spatial Science here Kaminski, Juliane here Kasparov, Garry here, here Keith, David here Kerry, John here Keuschnigg, Marc here Kleinberg, Jon here Kluemper, Donald here Kogan, Alex here, here, here Kosinski, Michal here, here, here, here, here, here, here Kramer, Adam here, here Krizhevsky, Alex here Kulsrestha, Juhi here Kurzweil, Ray here Labour Party here, here Momentum here Lake, Brenden here language here Laue, Tim here Le Comber, Steve here Le Cun, Yan here Le Pen, Marine here Le, Quoc here Lerman, Kristina here, here, here Levin, Simon here Libratus here LinkedIn here, here, here, here literature here logic gates here Luntz, Frank here Machine Bias here Macron, Emmanuel here Major League Soccer (MLS) here, here Mandela effect here, here Mandela, Nelson here Martin, Erik here matchmaking here mathematics here, here assessing bias here mathematical models here, here, here power laws here Matrix, The here May, Lord Robert here McDonald, Glenn here, here Mechanical Turk here, here, here, here, here Medium here Mercer, Robert here Microsoft here, here, here, here, here, here Mikolov, Tomas here, here Minecraft here Mosseri, Adam here, here, here Mrsic-Flogel, Thomas here Ms Pac-Man here, here, here Munafò, Marcus here Musk, Elon here, here, here myPersonality project here National Health Service (NHS) here, here National Women’s Soccer League (NWSL) here, here Nature here, here, here Natusch, Waffles Pi here Netflix here neural networks here, here convolutional neural networks here limitations here recurrent neural networks here New York Times here, here, here, here, here, here, here, here The Upshot here, here news aggregators here Nix, Alexander here, here, here, here Noiszy here Northpointe here, here, here, here O’Neil, Cathy here Weapons of Math Destruction here Obama, Barack here, here Observer here online data collection here, here gender bias here preventing here principal component analysis (PCA) here online help services here OpenWorm here Overwatch here, here Pasquale, Frank The Black Box Society here, here Paul, Jake here, here, here, here Pennington, Jeffrey here personality analysis here Big Five here, here, here, here PewDiePie here Pierson, Emma here Pittsburgh Post-Gazette here political blogs here political discussions here, here, here PolitiFact here polls here, here, here, here Popular Mechanics here post-truth world here, here, here power laws here Pratt, Stephen here, here PredictIt here, here, here, here, here, here Prince here principal component analysis (PCA) here categorising personalities here COMPAS algorithm here probability distributions here ProPublica here, here, here, here, here, here Pundit here Q*bert 214, here Qualtrics here racial bias here, here, here, here, here GloVe (global vectors for word representation) here randomness here Reddit here, here, here, here, here regression models here, here Republican Party here, here, here, here, here RiceGum here, here Richardson, Kathleen here Road Runner here Robotank here, here robots here, here, here, here, here, here Russian interference here, here, here Salganik, Matthew here, here Sanders, Bernie here Scholz, Monika here Science here SCL here, here search histories here Silver, David here Silver, Nate here, here, here The Signal and the Noise here Silverman, Craig here Simonyan, Karen here singularity hypothesis here Skeem, Jen here Sky Sports here slime moulds (Physarum polycephulum) here, here, here Snapchat here Snopes here social feedback here Space Invaders here, here, here, here Spotify here, here, here, here, here, here, here Stack Exchange here StarCraft here statistics here, here, here, here, here regression models here, here Stillwell, David here, here Sullivan, Andrew here, here Sumpter, David Soccermatics here, here, here, here, here, here, here Sun, The here superforecasters here, here superintelligence here, here Szorkovszky, Alex here, here, here, here, here, here Taleb, Nassim here, here, here Tegmark, Max here, here, here, here Telegraph here, here, here, here Tesla here, here, here, here Tetlock, Philip here, here Texas, Virgil here, here, here The Gateway here TIDAL here Times, The here, here Tinder here, here, here Tolstoy, Leo here, here, here Anna Karenina here trolls here true positives here, here Trump, Donald here, here, here, here, here, here election campaign here, here, here, here, here, here election outcome here, here, here Twitter here, here TUI here, here Turing, Alan here Twitter here, here, here, here, here, here, here, here, here, here, here, here, here, here MacronLeaks here Tyson, Gareth here van Seijen, Harm here, here Vinyals, Oriol here vloggers here voter analysis here, here, here Wall Street Journal here Ward, Ashley here Washington Post here, here, here, here Watts, Duncan here, here Which?

…

When the victory failed to materialise, the New York Times published an article (headlined: ‘How data failed us in calling an election’) that proclaimed the number crunchers had had a rough night.3 It listed supposed problems in both their own model (the newspaper’s Upshot model had given Clinton a 91 per cent chance of winning) and the approach taken by Nate Silver and FiveThirtyEight. The newspaper was blaming statisticians for its own inability to account for uncertainty. For Silver, this was just one example of how the media finds it very difficult to write sensible articles based on probabilistic reasoning.4 What struck me, looking at how FiveThirtyEight had evolved over the past 10 years, was that the site provides a powerful case study of the limits of mathematical models. Nate Silver had been propelled to a position of authority. He had accumulated financial resources (FiveThirtyEight is owned by ESPN) that had allowed him to build sophisticated models based on large quantities of reliable data.

The Signal and the Noise: Why So Many Predictions Fail-But Some Don't by Nate Silver

airport security, Alan Greenspan, Alvin Toffler, An Inconvenient Truth, availability heuristic, Bayesian statistics, Bear Stearns, behavioural economics, Benoit Mandelbrot, Berlin Wall, Bernie Madoff, big-box store, Black Monday: stock market crash in 1987, Black Swan, Boeing 747, book value, Broken windows theory, business cycle, buy and hold, Carmen Reinhart, Charles Babbage, classic study, Claude Shannon: information theory, Climategate, Climatic Research Unit, cognitive dissonance, collapse of Lehman Brothers, collateralized debt obligation, complexity theory, computer age, correlation does not imply causation, Credit Default Swap, credit default swaps / collateralized debt obligations, cuban missile crisis, Daniel Kahneman / Amos Tversky, disinformation, diversification, Donald Trump, Edmond Halley, Edward Lorenz: Chaos theory, en.wikipedia.org, equity premium, Eugene Fama: efficient market hypothesis, everywhere but in the productivity statistics, fear of failure, Fellow of the Royal Society, Ford Model T, Freestyle chess, fudge factor, Future Shock, George Akerlof, global pandemic, Goodhart's law, haute cuisine, Henri Poincaré, high batting average, housing crisis, income per capita, index fund, information asymmetry, Intergovernmental Panel on Climate Change (IPCC), Internet Archive, invention of the printing press, invisible hand, Isaac Newton, James Watt: steam engine, Japanese asset price bubble, John Bogle, John Nash: game theory, John von Neumann, Kenneth Rogoff, knowledge economy, Laplace demon, locking in a profit, Loma Prieta earthquake, market bubble, Mikhail Gorbachev, Moneyball by Michael Lewis explains big data, Monroe Doctrine, mortgage debt, Nate Silver, negative equity, new economy, Norbert Wiener, Oklahoma City bombing, PageRank, pattern recognition, pets.com, Phillips curve, Pierre-Simon Laplace, Plato's cave, power law, prediction markets, Productivity paradox, proprietary trading, public intellectual, random walk, Richard Thaler, Robert Shiller, Robert Solow, Rodney Brooks, Ronald Reagan, Saturday Night Live, savings glut, security theater, short selling, SimCity, Skype, statistical model, Steven Pinker, The Great Moderation, The Market for Lemons, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, Timothy McVeigh, too big to fail, transaction costs, transfer pricing, University of East Anglia, Watson beat the top human players on Jeopardy!, Wayback Machine, wikimedia commons

Tim Parker, University of Oxford Figure 7-1: From “1918 Influenza: The Mother of All Pandemics” by Jeffery Taubenberger and David Morens, Emerging Infectious Disease Journal, vol. 12, no. 1, January 2006, Centers for Disease Control and Prevention Figures 9-2, 9-3A, 9-3C, 9-4, 9-5, 9-6 and 9-7: By Cburnett, Wikimedia Commons Figure 12-2: Courtesy of Dr. J. Scott Armstrong, The Wharton School, University of Pennsylvania LIBRARY OF CONGRESS CATALOGING IN PUBLICATION DATA Silver, Nate. The signal and the noise : why most predictions fail but some don’t / Nate Silver. p. cm. Includes bibliographical references and index. ISBN 978-1-101-59595-4 1. Forecasting. 2. Forecasting—Methodology. 3. Forecasting—History. 4. Bayesian statistical decision theory. 5. Knowledge, Theory of. I. Title. CB158.S54 2012 519.5'42—dc23 2012027308 While the author has made every effort to provide accurate telephone numbers, Internet addresses, and other contact information at the time of publication, neither the publisher nor the author assumes any responsibility for errors, or for changes that occur after publication.

…

“Election Results: House Big Board,” New York Times, November 2, 2010. http://elections.nytimes.com/2010/results/house/big-board. 26. Nate Silver, “A Warning on the Accuracy of Primary Polls,” FiveThirtyEight, New York Times, March 1, 2012. http://fivethirtyeight.blogs.nytimes.com/2012/03/01/a-warning-on-the-accuracy-of-primary-polls/. 27. Nate Silver, “Bill Buckner Strikes Again,” FiveThirtyEight, New York Times; September 29, 2011. http://fivethirtyeight.blogs.nytimes.com/2011/09/29/bill-buckner-strikes-again/. 28. Otherwise, you should have assigned the congressman a 100 percent chance of victory instead. 29. Matthew Dickinson, “Nate Silver Is Not a Political Scientist,” in Presidential Power: A Nonpartisan Analysis of Presidential Power, Blogs Dot Middlebury, November 1, 2010. http://blogs.middlebury.edu/presidentialpower/2010/11/01/nate-silver-is-not-a-political-scientist/. 30.

…

But relatively little of it is useful. We perceive it selectively, subjectively, and without much self-regard for the distortions that this causes. We think we want information when we really want knowledge. The signal is the truth. The noise is what distracts us from the truth. This is a book about the signal and the noise. 1 A CATASTROPHIC FAILURE OF PREDICTION It was October 23, 2008. The stock market was in free fall, having plummeted almost 30 percent over the previous five weeks. Once-esteemed companies like Lehman Brothers had gone bankrupt. Credit markets had all but ceased to function.

Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are by Seth Stephens-Davidowitz

affirmative action, AltaVista, Amazon Mechanical Turk, Asian financial crisis, Bernie Sanders, big data - Walmart - Pop Tarts, Black Lives Matter, Cass Sunstein, computer vision, content marketing, correlation does not imply causation, crowdsourcing, Daniel Kahneman / Amos Tversky, data science, desegregation, Donald Trump, Edward Glaeser, Filter Bubble, game design, happiness index / gross national happiness, income inequality, Jeff Bezos, Jeff Seder, John Snow's cholera map, longitudinal study, Mark Zuckerberg, Nate Silver, Nick Bostrom, peer-to-peer lending, Peter Thiel, price discrimination, quantitative hedge fund, Ronald Reagan, Rosa Parks, sentiment analysis, Silicon Valley, statistical model, Steve Jobs, Steven Levy, Steven Pinker, TaskRabbit, The Signal and the Noise by Nate Silver, working poor

Many teams began using increasingly intricate statistics—rather than relying on old-fashioned human scouts—to make decisions. It was easy to measure offense and pitching but not fielding, so some organizations ended up underestimating the importance of defense. In fact, in his book The Signal and the Noise, Nate Silver estimates that the Oakland A’s, a data-driven organization profiled in Moneyball, were giving up eight to ten wins per year in the mid-nineties because of their lousy defense. The solution is not always more Big Data. A special sauce is often necessary to help Big Data work best: the judgment of humans and small surveys, what we might call small data.

…

Dahl, “Family Violence and Football: The Effect of Unexpected Emotional Cues on Violent Behavior,” Quarterly Journal of Economics 126, no. 1 (2011). 197 Here’s how Bill Simmons: Bill Simmons, “It’s Hard to Say Goodbye to David Ortiz,” ESPN.com, June 2, 2009, http://www.espn.com/espnmag/story?id=4223584. 198 how can we predict how a baseball player will perform in the future: This is discussed in Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (New York: Penguin, 2012). 199 “beefy sluggers” indeed do, on average, peak early: Ryan Campbell, “How Will Prince Fielder Age?” October 28, 2011, http://www.fangraphs.com/blogs/how-will-prince-fielder-age/. 199 Ortiz’s doppelgangers’: This data was kindly provided to me by Rob McQuown of Baseball Prospectus. 204 Kohane asks: I interviewed Isaac Kohane by phone on June 15, 2015. 205 James Heywood is an entrepreneur: I interviewed James Heywood by phone on August 17, 2015.

…

See also bias; discrimination; race/racism self-employed people, and taxes, 178–80 sentiment analysis, 87–92, 247–48 sex as addiction, 219 and benefits of digital truth serum, 158–59, 161 and childhood experiences, 50–52 condoms and, 5, 122 and digital revolution, 274, 279 and dimensions of sexuality, 279 during marriage, 5–6 and fetishes, 120 and Freud, 45–52 Google searches about, 5–6, 51–52, 114, 115, 117, 118, 122–24, 126, 127–28 and handling the truth, 158–59, 161 and Harvard Crimson editorial about Zuckerberg, 155 how much, 122–23, 124–25, 127 in India, 19 new information about, 19 oral, 128 and physical appearance, 120, 120n, 125–26, 127 and power of Big Data, 53 pregancy and having, 189 Rolling Stones song about, 278 and sex organs, 123–24 Stephens-Davidowitz’s first New York Times column about, 282 and traditional research methods, 274 truth about, 5–6, 112–28, 114n, 117 and typing errors, 48–50 and women’s genitals, 126–27 See also incest; penis; pornography; rape; vagina Shadow (app), 47 Shakespeare, William, 89–90 Shapiro, Jesse, 74–76, 93–97, 141–44, 235, 273 “Shattered” (Rolling Stones song), 278 shopping habits, predictions about, 71–74 The Signal and the Noise (Silver), 254 Silver, Nate, 10, 12–13, 133, 199, 200, 254, 255 Simmons, Bill, 197–98 Singapore, pregnancy in, 190 Siroker, Dan, 211–12 sleep and digital revolution, 279 Jawbone and, 276–77 and pregnancy, 189 “Slutload,” 58 small data, 255–56 smiles, and pictures as data, 99 Smith, Michael D., 224 Snow, John, 275 Sochi, Russia, gays in, 119 social media bias of data from, 150–53 doppelganger hunting on, 201–3 and wives descriptions of husbands, 160–61, 160–61n See also specific site or topic social science, 272–74, 276, 279 social security, and words as data, 93 socioeconomic background and predicting success in basketball, 34–41 See also pedigrees sociology, 273, 274 Soltas, Evan, 130, 162, 266–67 South Africa, pregnancy in, 189 Southern Poverty Law Center, 137 Spain, pregnancy in, 190 Spartanburg Herald-Journal (South Carolina), and words as data, 96 specialization, extreme, 186 speed, for transmitting data, 56–59 “Spider Solitaire,” 58 Stephens-Davidowitz, Noah, 165–66, 165–66n, 169, 206, 263 Stephens-Davidowitz, Seth ambitions of, 33 lying by, 282n mate choice for, 25–26, 271 motivations of, 2 obsessiveness of, 282, 282n professional background of, 14 and writing conclusions, 271–72, 279, 280–84 Stern, Howard, 157 stock market data for, 55–56 and examples of Big Data searches, 22 Summers-Stephens-Davidowitz attempt to predict the, 245–48, 251–52 Stone, Oliver, 185 Stoneham, James, 266, 269 Storegard, Adam, 99–101 stories categories/types of, 91–92 viral, 22, 92 and zooming in, 205–6 See also specific story Stormfront (website), 7, 14, 18, 137–40 stretch marks, and pregnancy, 188–89 Stuyvesant High School (New York City), 231–37, 238, 240 suburban areas, and origins of notable Americans, 183–84 successful/notable Americans factors that drive, 185–86 zooming in on, 180–86 suffering, and benefits of digital truth serum, 161 suicide, and danger of empowered government, 266, 267–68 Summers, Lawrence and Obama-racism study, 243–44 and predicting the stock market, 245, 246, 251–52 Stephens-Davidowitz’s meeting with, 243–45 Sunstein, Cass, 140 Super Bowl games, advertising during, 221–25, 239 Super Crunchers (Gnau), 264 Supreme Court, and abortion, 147 Surowiecki, James, 203 surveys in-person, 108 internet, 108 and lying, 105–7, 108, 108n and pictures as data, 97 skepticism about, 171 telephone, 108 and truth about sex, 113, 116 and zooming in on hours and minutes, 193 See also specific survey or topic Syrian refugees, 131 Taleb, Nassim, 17 Tartt, Donna, 283 TaskRabbit, 212 taxes cheating on, 22, 178–80, 206 and examples of Big Data searches, 22 and lying, 180 and self-employed people, 178–80 and words as data, 93–95 zooming in on, 172–73, 178–80, 206 teachers, using tests to judge, 253–54 teenagers adopted, 108n as gay, 114, 116 lying by, 108n and origins of political preferences, 169 and truth about sex, 114, 116 See also children television and A/B testing, 222 advertising on, 221–26 Terabyte, 264 terrorism, 18, 129–31 tests/testing of high school students, 231–37, 253–54 and judging teacher, 253–54 and obsessive infatuations with numbers, 253–54 online behavior as supplement to, 278 and small data, 255–56 See also specific test or study Thiel, Peter, 155 Think Progress (website), 130 Thinking, Fast and Slow (Kahneman), 283 Thome, Jim, 200 Tourangeau, Roger, 107, 108 towns, zooming in on, 172–90 Toy Story (movie), 192 Trump, Donald elections of 2012 and, 7 and ignoring what people tell you, 157 and immigration, 184 issues propagated by, 7 and origins of notable Americans, 184 polls about, 1 predictions about, 11–14 and racism, 8, 9, 11, 12, 14, 133, 139 See also elections, 2016 truth benefits of knowing, 158–63 handling the, 158–63 See also digital truth serum; lying; specific topic Tuskegee University, 183 Twentieth Century Fox, 221–22 Twitter, 151–52, 160–61n, 201–3 typing errors by searchers, 48–50 The Unbearable Lightness of Being (Kundera), 233 Uncharted (Aiden and Michel), 78–79 unemployment and child abuse, 145–47 data about, 56–57, 58–59 unintended consequences, 197 United States and Civil War, 79 as united or divided, 78–79 University of California, Berkeley, racism in 2008 election study at, 2 University of Maryland, survey of graduates of, 106–7 urban areas and life expectancy, 177 and origins of notable Americans, 183–84, 186 vagina, smells of, 19, 126–27, 161 Varian, Hal, 57–58, 224 Vikingmaiden88, 136–37, 140–41, 145 violence and real science, 273 zooming in on, 190–97 See also murder voter registration, 106 voter turnout, 9–10, 109–10 voting behavior, and lying, 106, 107, 109–10 Vox, 202 Walmart, 71–72 Washington Post, and words as data, 75, 94 Washington Times, and words as data, 75, 94–95 wealth and life expectancy, 176–77 See also income distribution weather, and predictions about wine, 73–74 Weil, David N., 99–101 Weiner, Anthony, 234n white nationalism, 137–40, 145.

Thinking in Bets by Annie Duke

banking crisis, behavioural economics, Bernie Madoff, Cass Sunstein, cognitive bias, cognitive dissonance, cognitive load, Daniel Kahneman / Amos Tversky, delayed gratification, Demis Hassabis, disinformation, Donald Trump, Dr. Strangelove, en.wikipedia.org, endowment effect, Estimating the Reproducibility of Psychological Science, fake news, Filter Bubble, Herman Kahn, hindsight bias, Jean Tirole, John Nash: game theory, John von Neumann, loss aversion, market design, mutually assured destruction, Nate Silver, p-value, phenotype, prediction markets, Richard Feynman, ride hailing / ride sharing, Stanford marshmallow experiment, Stephen Hawking, Steven Pinker, systematic bias, TED Talk, the scientific method, The Signal and the Noise by Nate Silver, urban planning, Walter Mischel, Yogi Berra, zero-sum game

The Believing Brain: From Ghosts and Gods to Politics and Conspiracies—How We Construct Beliefs and Reinforce Them as Truths. New York: Times Books, 2011. Silver, Nate. “14 Versions of Trump’s Presidency, from #MAGA to Impeachment.” FiveThirtyEight.com, February 3, 2017. http://fivethirtyeight.com/features/14-versions-of-trumps-presidency-from-maga-to-impeachment. Silver, Nate. The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t. New York: Penguin, 2012. Simmons, Joseph, Leif Nelson, and Uri Simonsohn. “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant.” Psychological Science 22, no. 11 (November 2011): 1359–66.

…

Hillary Clinton had been favored going into the election, and her probability of winning, based on an accumulation of the polls, was somewhere between 60% and 70%, according to FiveThirtyEight.com. When Donald Trump won, pollsters got the Pete Carroll treatment, maybe no one more than Nate Silver, founder of FiveThirtyEight.com and a thoughtful analyzer of polling data. (“Nate Silver was wrong.” “The pollsters missed it.” “Just like Brexit, the bookies blew it.” Etc.) The press spun this as a certain win for Clinton, despite the Trump branch of the tree being no mere twig at 30%–40%. By the day after the election, the Clinton branch had been severed, only the Trump branch remained, and how could pollsters and the polling process have been so blind?

…

Mistaking Odds for Wrong When the Underdog Wins,” Huffington Post, September 21, 2016, http://www.huffingtonpost.com/annie-duke/even-dershowitz-mistaking_b_12120592.html. Nate Silver and his website, FiveThirtyEight.com, bore the brunt of the criticism for pollsters and forecasters after the 2016 presidential election. Silver’s site updated, in real time, polling and forecasting data on the election and had (depending on the date) the probability of a Clinton victory at approximately 60%–70%. If you Google (without the quotation marks) “Nate Silver got it wrong election,” 465,000 results come up. Politico’s November 9 headline was “How Did Everyone Get It So Wrong?

Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die by Eric Siegel

Alan Greenspan, Albert Einstein, algorithmic trading, Amazon Mechanical Turk, Apollo 11, Apple's 1984 Super Bowl advert, backtesting, Black Swan, book scanning, bounce rate, business intelligence, business process, butter production in bangladesh, call centre, Charles Lindbergh, commoditize, computer age, conceptual framework, correlation does not imply causation, crowdsourcing, dark matter, data is the new oil, data science, driverless car, en.wikipedia.org, Erik Brynjolfsson, Everything should be made as simple as possible, experimental subject, Google Glasses, happiness index / gross national happiness, information security, job satisfaction, Johann Wolfgang von Goethe, lifelogging, machine readable, Machine translation of "The spirit is willing, but the flesh is weak." to Russian and back, mass immigration, Moneyball by Michael Lewis explains big data, Nate Silver, natural language processing, Netflix Prize, Network effects, Norbert Wiener, personalized medicine, placebo effect, prediction markets, Ray Kurzweil, recommendation engine, risk-adjusted returns, Ronald Coase, Search for Extraterrestrial Intelligence, self-driving car, sentiment analysis, Shai Danziger, software as a service, SpaceShipOne, speech recognition, statistical model, Steven Levy, supply chain finance, text mining, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Davenport, Turing test, Watson beat the top human players on Jeopardy!, X Prize, Yogi Berra, zero-sum game

TIME Magazine, November 07, 2012. http://swampland.time.com/2012/11/07/inside-the-secret-world-of-quants-and-data-crunchers-who-helped-obama-win/. Colbert Nation, www.colbertnation.com. Stephen Colbert interviews Nate Silver, New York Times blogger about his book, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t. http://www.colbertnation.com/the-colbert-report-videos/420765/november-05-2012/nate-silver. Peggy Noonan, “They’ve Lost That Lovin’ Feeling.” Wall Street Journal, July 30, 2011. http://online.wsj.com/article/SB10001424053111904800304576474620336602248.html. Jack Gillum, “Mitt Romney Uses Secretive Data Mining To Identify Wealthy Donors.”

…

A Faulty Oracle Everyone Loves The first step toward predicting the future is admitting you can’t. —Stephen Dubner, Freakonomics Radio, March 30, 2011 The “prediction paradox”: The more humility we have about our ability to make predictions, the more successful we can be in planning for the future. —Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t Half of what we will teach you in medical school will, by the time you are done practicing, be proved wrong. —Dr. Mehmet Oz Your resident “oracle,” PA, tells you which customers are most likely to respond. It earmarks a quarter of the entire list and says, “These folks are three times more likely to respond than average!”

…

However, they kept at it, squeezing every drop of potential out of their brainshare and data, right up until the final weeks before the big match. Confidence without Overconfidence Both experts and laypeople mistake more confident predictions for more accurate ones. But overconfidence is often the reason for failure. If our appreciation of uncertainty improves, our predictions can get better too. —Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. —Bertrand Russell You got to know when to hold ‘em, know when to fold ‘em. —Don Schlitz, “The Gambler” (sung by Kenny Rogers) Jeopardy!

Superforecasting: The Art and Science of Prediction by Philip Tetlock, Dan Gardner

Affordable Care Act / Obamacare, Any sufficiently advanced technology is indistinguishable from magic, availability heuristic, behavioural economics, Black Swan, butterfly effect, buy and hold, cloud computing, cognitive load, cuban missile crisis, Daniel Kahneman / Amos Tversky, data science, desegregation, drone strike, Edward Lorenz: Chaos theory, forward guidance, Freestyle chess, fundamental attribution error, germ theory of disease, hindsight bias, How many piano tuners are there in Chicago?, index fund, Jane Jacobs, Jeff Bezos, Kenneth Arrow, Laplace demon, longitudinal study, Mikhail Gorbachev, Mohammed Bouazizi, Nash equilibrium, Nate Silver, Nelson Mandela, obamacare, operational security, pattern recognition, performance metric, Pierre-Simon Laplace, place-making, placebo effect, precautionary principle, prediction markets, quantitative easing, random walk, randomized controlled trial, Richard Feynman, Richard Thaler, Robert Shiller, Ronald Reagan, Saturday Night Live, scientific worldview, Silicon Valley, Skype, statistical model, stem cell, Steve Ballmer, Steve Jobs, Steven Pinker, tacit knowledge, tail risk, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Watson beat the top human players on Jeopardy!

A mere two years after it was published the Arab Spring turned the Middle East topsy-turvy, but I can’t find it in Friedman’s book, which casts some doubt on his forecasts for the remaining ninety-eight years. Friedman is also the author of the 1991 book The Coming War with Japan—that’s the coming American war with Japan—which has yet to prove its prescience. 7. For islands of professionalism in a sea of malpractice, see the forecasting concepts and tools reviewed in Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t (New York: Penguin Press, 2012); J. Scott Armstrong, ed., Principles of Forecasting: A Handbook for Researchers and Practitioners (Boston: Kluwer, 2001); and Bruce Bueno de Mesquita, The Predictioneer’s Game (New York: Random House, 2009).

…

At best, they have vague hunches. That’s because the forecast-measure-revise procedure operates only within the rarefied confines of high-tech forecasting, such as the work of macroeconomists at central banks or marketing and financial professionals in big companies or opinion poll analysts like Nate Silver.7 More often forecasts are made and then … nothing. Accuracy is seldom determined after the fact and is almost never done with sufficient regularity and rigor that conclusions can be drawn. The reason? Mostly it’s a demand-side problem: The consumers of forecasting—governments, business, and the public—don’t demand evidence of accuracy.

…

A forecaster who followed a mindless rule like, “always assign 100% to hot and sunny” could get a Brier score close to 0, leaving 0.2 in the dust. Here, the right test of skill would be whether a forecaster can do better than mindlessly predicting no change. This is an underappreciated point. For example, after the 2012 presidential election, Nate Silver, Princeton’s Sam Wang, and other poll aggregators were hailed for correctly predicting all fifty state outcomes, but almost no one noted that a crude, across-the-board prediction of “no change”—if a state went Democratic or Republican in 2008, it will do the same in 2012—would have scored forty-eight out of fifty, which suggests that the many excited exclamations of “He called all fifty states!”

Stuffocation by James Wallman

3D printing, Abraham Maslow, Adam Curtis, Airbnb, Alvin Toffler, back-to-the-land, Berlin Wall, big-box store, Black Swan, BRICs, carbon footprint, Cass Sunstein, clean water, collaborative consumption, commoditize, creative destruction, crowdsourcing, David Brooks, Fall of the Berlin Wall, Future Shock, Great Leap Forward, happiness index / gross national happiness, hedonic treadmill, high net worth, income inequality, Intergovernmental Panel on Climate Change (IPCC), James Hargreaves, Joseph Schumpeter, Kitchen Debate, Martin Wolf, mass immigration, McMansion, means of production, Nate Silver, Occupy movement, Paul Samuelson, planned obsolescence, post-industrial society, post-materialism, public intellectual, retail therapy, Richard Florida, Richard Thaler, sharing economy, Silicon Valley, Simon Kuznets, Skype, spinning jenny, Streisand effect, The future is already here, The Signal and the Noise by Nate Silver, The Theory of the Leisure Class by Thorstein Veblen, Thorstein Veblen, Tyler Cowen, Tyler Cowen: Great Stagnation, World Values Survey, Zipcar

Today they are only wrong by half that amount, three degrees. When hurricane forecasters predicted where a hurricane would hit land in the 1980s they were usually out by 350 miles. Today, their predictions are only wrong by 100 miles.” If you’re not ready – yet – to take on all of Nate Silver’s The Signal and the Noise, read Nate Silver, “The Weatherman Is Not a Moron”, New York Times, 7 September 2012. The Farm Where the Corn Did Not Grow Tall For Everett Rogers’s version of his life, read Everett M Rogers, The Fourteenth Paw (Singapore: Asian Media Information and Communication Centre, 2008).

…

Forecasts Are Not Facts, They Are Maps For Nassim Nicholas Taleb’s dismal view of forecasting, read Nassim Nicholas Taleb, The Black Swan (New York: Penguin, 2007). The story of the turkey is borrowed, as Taleb notes, from the philosopher Bertrand Russell. Using the Past to Tell the Future I am indebted to three sources for this section: Peter N Stearns, “Why Study History?”, American Historical Association, 1998; Nate Silver, The Signal and the Noise (New York: Allen Lane, 2012); and Rob Hyndman, “Why are some things easier to forecast than others?”, 18 September 2012, on his blog, Hyndsight (www.robjhyndman.com/hyndsight). “In the 1970s, the high temperature forecasts were wrong, on average, by about six degrees. Today they are only wrong by half that amount, three degrees.

The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World by Pedro Domingos

Albert Einstein, Amazon Mechanical Turk, Arthur Eddington, backpropagation, basic income, Bayesian statistics, Benoit Mandelbrot, bioinformatics, Black Swan, Brownian motion, cellular automata, Charles Babbage, Claude Shannon: information theory, combinatorial explosion, computer vision, constrained optimization, correlation does not imply causation, creative destruction, crowdsourcing, Danny Hillis, data is not the new oil, data is the new oil, data science, deep learning, DeepMind, double helix, Douglas Hofstadter, driverless car, Erik Brynjolfsson, experimental subject, Filter Bubble, future of work, Geoffrey Hinton, global village, Google Glasses, Gödel, Escher, Bach, Hans Moravec, incognito mode, information retrieval, Jeff Hawkins, job automation, John Markoff, John Snow's cholera map, John von Neumann, Joseph Schumpeter, Kevin Kelly, large language model, lone genius, machine translation, mandelbrot fractal, Mark Zuckerberg, Moneyball by Michael Lewis explains big data, Narrative Science, Nate Silver, natural language processing, Netflix Prize, Network effects, Nick Bostrom, NP-complete, off grid, P = NP, PageRank, pattern recognition, phenotype, planetary scale, power law, pre–internet, random walk, Ray Kurzweil, recommendation engine, Richard Feynman, scientific worldview, Second Machine Age, self-driving car, Silicon Valley, social intelligence, speech recognition, Stanford marshmallow experiment, statistical model, Stephen Hawking, Steven Levy, Steven Pinker, superintelligent machines, the long tail, the scientific method, The Signal and the Noise by Nate Silver, theory of mind, Thomas Bayes, transaction costs, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, white flight, yottabyte, zero-sum game

(Science, 2009). Sasha Issenberg’s The Victory Lab (Broadway Books, 2012) dissects the use of data analysis in politics. “How President Obama’s campaign used big data to rally individual votes,” by the same author (MIT Technology Review, 2013), tells the story of its greatest success to date. Nate Silver’s The Signal and the Noise (Penguin Press, 2012) has a chapter on his poll aggregation method. Robot warfare is the theme of P. W. Singer’s Wired for War (Penguin, 2009). Cyber War, by Richard Clarke and Robert Knake (Ecco, 2012), sounds the alarm on cyberwar. My work on combining machine learning with game theory to defeat adversaries, which started as a class project, is described in “Adversarial classification,”* by Nilesh Dalvi et al.

…

I discuss the importance of prior knowledge in machine learning in “Toward knowledge-rich data mining”* (Data Mining and Knowledge Discovery, 2007) and misinterpretations of Occam’s razor in “The role of Occam’s razor in knowledge discovery”* (Data Mining and Knowledge Discovery, 1999). Overfitting is one of the main themes of The Signal and the Noise, by Nate Silver (Penguin Press, 2012), who calls it “the most important scientific problem you’ve never heard of.” “Why most published research findings are false,”* by John Ioannidis (PLoS Medicine, 2005), discusses the problem of mistaking chance findings for true ones in science. Yoav Benjamini and Yosef Hochberg propose a way to combat it in “Controlling the false discovery rate: A practical and powerful approach to multiple testing”* (Journal of the Royal Statistical Society, Series B, 1995).

…

They could see the other side buying ads in particular cable stations in particular towns but couldn’t tell why; their crystal ball was too fuzzy. In the end, Obama won every battleground state save North Carolina and by larger margins than even the most accurate pollsters had predicted. The most accurate pollsters, in turn, were the ones (like Nate Silver) who used the most sophisticated prediction techniques; they were less accurate than the Obama campaign because they had fewer resources. But they were a lot more accurate than the traditional pundits, whose predictions were based on their expertise. You might think the 2012 election was a fluke: most elections are not close enough for machine learning to be the deciding factor.

The Behavioral Investor by Daniel Crosby

affirmative action, Asian financial crisis, asset allocation, availability heuristic, backtesting, bank run, behavioural economics, Black Monday: stock market crash in 1987, Black Swan, book value, buy and hold, cognitive dissonance, colonial rule, compound rate of return, correlation coefficient, correlation does not imply causation, Daniel Kahneman / Amos Tversky, disinformation, diversification, diversified portfolio, Donald Trump, Dunning–Kruger effect, endowment effect, equity risk premium, fake news, feminist movement, Flash crash, haute cuisine, hedonic treadmill, housing crisis, IKEA effect, impact investing, impulse control, index fund, Isaac Newton, Japanese asset price bubble, job automation, longitudinal study, loss aversion, market bubble, market fundamentalism, mental accounting, meta-analysis, Milgram experiment, moral panic, Murray Gell-Mann, Nate Silver, neurotypical, Nick Bostrom, passive investing, pattern recognition, Pepsi Challenge, Ponzi scheme, prediction markets, random walk, Reminiscences of a Stock Operator, Richard Feynman, Richard Thaler, risk tolerance, Robert Shiller, science of happiness, Shai Danziger, short selling, South Sea Bubble, Stanford prison experiment, Stephen Hawking, Steve Jobs, stocks for the long run, sunk-cost fallacy, systems thinking, TED Talk, Thales of Miletus, The Signal and the Noise by Nate Silver, Tragedy of the Commons, trolley problem, tulip mania, Vanguard fund, When a measure becomes a target

It is also what makes them almost impossible to beat. Notes 42 Zweig, Your Money and Your Brain, p. 22. 43 Greg B. Davies, Behavioral Investment Management: An Efficient Alternative to Modern Portfolio Theory (McGraw-Hill, 2012), p. 53. 44 Nate Silver, The Signal and the Noise: Why So Many Predictions Fail – but Some Don’t (Penguin, 2015), p. 185. Chapter 7. Emotion “The world is a tragedy to those who feel, but a comedy to those who think.” — Horace Walpole Emotion: friend or foe? It must be stated from the outset that there is some disagreement within the behavioral finance community about whether emotion is a help or hindrance when making investment decisions.

…

Several experiments suggest that when those presented with an extensive array of options make fewer purchases and are less happy with the purchases they make than those operating from a more limited decisional universe. Another consequence of financial information overload is that it leads to drawing spurious correlations between variables. As Nate Silver reports, the government produces data on 45,000 economic variables each year!44 Pair this reality with the fact that there are relatively few dramatic economic events (e.g., there have been 11 recessions since the end of World War II) and you get what Silver refers to as putting data into a blender and calling the result haute cuisine.

…

Not only did the simple model outperform the psychologists’ intuition head-to-head, but it also bested psychologists who were given access to the model.109 Models have also been shown to outperform human intuition in predicting the outcomes of Supreme Court decisions,110 Presidential elections (Nate Silver), movie preferences, prison recidivism, wine quality, marital satisfaction and military success, to name just a few of the over 45 domains in which they have demonstrated their superiority.111 A meta-analysis performed by William Grove, David Zald, Boyd Lebow, Beth Snitz and Chad Nelson found that models equal or beat expert decision-making a whopping 94.12% of the time, meaning that they are only defeated by human discretion 5.88% of the time.112 Moreover, many of the domains in which algorithms greatly outperformed had human behavior as a central component (as do financial markets).

The Art of Statistics: Learning From Data by David Spiegelhalter

Abraham Wald, algorithmic bias, Anthropocene, Antoine Gombaud: Chevalier de Méré, Bayesian statistics, Brexit referendum, Carmen Reinhart, Charles Babbage, complexity theory, computer vision, confounding variable, correlation coefficient, correlation does not imply causation, dark matter, data science, deep learning, DeepMind, Edmond Halley, Estimating the Reproducibility of Psychological Science, government statistician, Gregor Mendel, Hans Rosling, Higgs boson, Kenneth Rogoff, meta-analysis, Nate Silver, Netflix Prize, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, p-value, placebo effect, probability theory / Blaise Pascal / Pierre de Fermat, publication bias, randomized controlled trial, recommendation engine, replication crisis, self-driving car, seminal paper, sparse data, speech recognition, statistical model, sugar pill, systematic bias, TED Talk, The Design of Experiments, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Malthus, Two Sigma

An observation with a Z-score of 3 corresponds to being 3 standard deviations above the mean, which is a fairly extreme outlier. A Z-score can also be defined in terms of a population mean μ and standard deviation σ, in which case zi = (xi – μ)/σ. Notes INTRODUCTION 1. The Signal and the Noise by Nate Silver (Penguin, 2012) is an excellent introduction to how statistical science can be applied to making predictions in sport and other domains. 2. The Shipman data is discussed in more detail in D. Spiegelhalter and N. Best, ‘Shipman’s Statistical Legacy’, Significance1:1 (2004), 10–12.

…

So in summary, we assume that observation = deterministic model + residual error. This formula can be interpreted as saying that, in the statistical world, what we see and measure around us can be considered as the sum of a systematic mathematical idealized form plus some random contribution that cannot yet be explained. This is the classic idea of the signal and the noise. Do speed cameras reduce accidents? This section contains a simple lesson: just because we act, and something changes, it doesn’t mean we were responsible for the result. Humans seem to find this simple truth difficult to grasp – we are always keen to construct an explanatory narrative, and even keener if we are at its centre.

…

CODE FOR EXAMPLES R code and data for reproducing most of the analyses and Figures are available from https://github.com/dspiegel29/ArtofStatistics. I am grateful for the assistance received in preparing this material. Introduction The numbers have no way of speaking for themselves. We speak for them. We imbue them with meaning. — Nate Silver, The Signal and the Noise1 Why We Need Statistics Harold Shipman was Britain’s most prolific convicted murderer, though he does not fit the archetypal profile of a serial killer. A mild-mannered family doctor working in a suburb of Manchester, between 1975 and 1998 he injected at least 215 of his mostly elderly patients with a massive opiate overdose.

The Art of Statistics: How to Learn From Data by David Spiegelhalter

Abraham Wald, algorithmic bias, Antoine Gombaud: Chevalier de Méré, Bayesian statistics, Brexit referendum, Carmen Reinhart, Charles Babbage, complexity theory, computer vision, confounding variable, correlation coefficient, correlation does not imply causation, dark matter, data science, deep learning, DeepMind, Edmond Halley, Estimating the Reproducibility of Psychological Science, government statistician, Gregor Mendel, Hans Rosling, Higgs boson, Kenneth Rogoff, meta-analysis, Nate Silver, Netflix Prize, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, p-value, placebo effect, probability theory / Blaise Pascal / Pierre de Fermat, publication bias, randomized controlled trial, recommendation engine, replication crisis, self-driving car, seminal paper, sparse data, speech recognition, statistical model, sugar pill, systematic bias, TED Talk, The Design of Experiments, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Malthus, Two Sigma

An observation with a Z-score of 3 corresponds to being 3 standard deviations above the mean, which is a fairly extreme outlier. A Z-score can also be defined in terms of a population mean μ and standard deviation σ, in which case zi = (xi–μ)/σ. Also by David Spiegelhalter The Norm Chronicles Notes INTRODUCTION 1. The Signal and the Noise by Nate Silver (Penguin, 2012) is an excellent introduction to how statistical science can be applied to making predictions in sport and other domains. 2. The Shipman data is discussed in more detail in D. Spiegelhalter and N. Best, ‘Shipman’s Statistical Legacy’, Significance 1:1 (2004), 10–12.

…

So in summary, we assume that observation = deterministic model + residual error. This formula can be interpreted as saying that, in the statistical world, what we see and measure around us can be considered as the sum of a systematic mathematical idealized form plus some random contribution that cannot yet be explained. This is the classic idea of the signal and the noise. Do speed cameras reduce accidents? This section contains a simple lesson: just because we act, and something changes, it doesn’t mean we were responsible for the result. Humans seem to find this simple truth difficult to grasp—we are always keen to construct an explanatory narrative, and even keener if we are at its centre.

…

CODE FOR EXAMPLES R code and data for reproducing most of the analyses and Figures are available from https://github.com/dspiegel29/ArtofStatistics. I am grateful for the assistance received in preparing this material. Introduction The numbers have no way of speaking for themselves. We speak for them. We imbue them with meaning. —Nate Silver, The Signal and the Noise1 Why We Need Statistics Harold Shipman was Britain’s most prolific convicted murderer, though he does not fit the archetypal profile of a serial killer. A mild-mannered family doctor working in a suburb of Manchester, between 1975 and 1998 he injected at least 215 of his mostly elderly patients with a massive opiate overdose.

The Revolt of the Public and the Crisis of Authority in the New Millennium by Martin Gurri

Affordable Care Act / Obamacare, Alan Greenspan, Albert Einstein, anti-communist, Arthur Eddington, Ayatollah Khomeini, bitcoin, Black Monday: stock market crash in 1987, Black Swan, Burning Man, business cycle, citizen journalism, Climategate, Climatic Research Unit, collective bargaining, creative destruction, crowdsourcing, currency manipulation / currency intervention, dark matter, David Graeber, death of newspapers, disinformation, Eddington experiment, en.wikipedia.org, Erik Brynjolfsson, facts on the ground, Francis Fukuyama: the end of history, Frederick Winslow Taylor, full employment, Great Leap Forward, housing crisis, income inequality, Intergovernmental Panel on Climate Change (IPCC), invention of writing, job-hopping, military-industrial complex, Mohammed Bouazizi, Nate Silver, Occupy movement, Port of Oakland, Republic of Letters, Ronald Reagan, scientific management, Skype, Steve Jobs, the scientific method, The Signal and the Noise by Nate Silver, too big to fail, traveling salesman, University of East Anglia, urban renewal, War on Poverty, We are the 99%, WikiLeaks, Yochai Benkler, young professional

[112] Dollar, “Man Who Predicted Earthquake” and Pielke, “Lessons of L’Aquila Lawsuit.” [113] Ibid. [114] “L’Aquila quake: Italy scientists guilty of manslaughter,” BBC, October 27, 2012, http://www.bbc. co.uk/news/world-europe-20025626. [115] An excellent evaluation of the state of the art in the forecasting of earthquakes is found in Nate Silver’s The Signal and the Noise: Why So Many Predictions Fail – But Some Don’t (The Penguin Press, 2012), 142-175. [116] Climategate Emails, 84. [117] Wikipedia Commons. [118] Bob Woodward, Maestro: Greenspan’s Fed and the American Boom (Simon and Schuster, 2000), Kindle location 507. [119] Ibid., Kindle location 602-609, 641

…

Seeing Like a State: How Certain Schemes To Improve the Human Condition Have Failed. Yale University Press, 1998. Shirky, Clay. Cognitive Surplus: Creativity and Generosity in a Connected Age. Penguin Books, 2010. Shirky, Clay. Here Comes Everybody: The Power of Organizing Without Organizations. Penguin Books, 2008. Silver, Nate. The Signal and the Noise: Why So Many Predictions Fail – But Some Don’t. The Penguin Press, 2012. Sorkin, Andrew. Too Big To Fail: The Inside Story of How Wall Street and Washington Fought To Save the Financial System – And Themselves. Penguin Books, 2009. Sreberny, Annabelle, and Khiabany, Gholam. Blogistan: The Internet and Politics in Iran.

…

Here was a bold attempt at prophecy by the new team of experts: in the event, it was wildly over-optimistic. Unemployment peaked at 10.1 percent after the stimulus bill passed, and didn’t touch 8 percent until late 2012 – much worse than the worst-case projections without the stimulus.[137] In human terms, the White House numbers had missed the plight of over 3 million unemployed Americans. Nate Silver offered two reasons for Romer and Bernstein’s disconcerting failure at prediction, and neither of them seemed flattering to the expert class. The first was ignorance of actual economic conditions. The economy in 2009 happened to be in far worse shape than the experts, for all their statistical wizardry, had realized.

Radical Uncertainty: Decision-Making for an Unknowable Future by Mervyn King, John Kay

Airbus A320, Alan Greenspan, Albert Einstein, Albert Michelson, algorithmic trading, anti-fragile, Antoine Gombaud: Chevalier de Méré, Arthur Eddington, autonomous vehicles, availability heuristic, banking crisis, Barry Marshall: ulcers, battle of ideas, Bear Stearns, behavioural economics, Benoit Mandelbrot, bitcoin, Black Swan, Boeing 737 MAX, Bonfire of the Vanities, Brexit referendum, Brownian motion, business cycle, business process, capital asset pricing model, central bank independence, collapse of Lehman Brothers, correlation does not imply causation, credit crunch, cryptocurrency, cuban missile crisis, Daniel Kahneman / Amos Tversky, David Ricardo: comparative advantage, DeepMind, demographic transition, discounted cash flows, disruptive innovation, diversification, diversified portfolio, Donald Trump, Dutch auction, easy for humans, difficult for computers, eat what you kill, Eddington experiment, Edmond Halley, Edward Lloyd's coffeehouse, Edward Thorp, Elon Musk, Ethereum, Eugene Fama: efficient market hypothesis, experimental economics, experimental subject, fear of failure, feminist movement, financial deregulation, George Akerlof, germ theory of disease, Goodhart's law, Hans Rosling, Helicobacter pylori, high-speed rail, Ignaz Semmelweis: hand washing, income per capita, incomplete markets, inflation targeting, information asymmetry, invention of the wheel, invisible hand, Jeff Bezos, Jim Simons, Johannes Kepler, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Snow's cholera map, John von Neumann, Kenneth Arrow, Kōnosuke Matsushita, Linda problem, Long Term Capital Management, loss aversion, Louis Pasteur, mandelbrot fractal, market bubble, market fundamentalism, military-industrial complex, Money creation, Moneyball by Michael Lewis explains big data, Monty Hall problem, Nash equilibrium, Nate Silver, new economy, Nick Leeson, Northern Rock, nudge theory, oil shock, PalmPilot, Paul Samuelson, peak oil, Peter Thiel, Philip Mirowski, Phillips curve, Pierre-Simon Laplace, popular electronics, power law, price mechanism, probability theory / Blaise Pascal / Pierre de Fermat, quantitative trading / quantitative finance, railway mania, RAND corporation, reality distortion field, rent-seeking, Richard Feynman, Richard Thaler, risk tolerance, risk-adjusted returns, Robert Shiller, Robert Solow, Ronald Coase, sealed-bid auction, shareholder value, Silicon Valley, Simon Kuznets, Socratic dialogue, South Sea Bubble, spectrum auction, Steve Ballmer, Steve Jobs, Steve Wozniak, Suez crisis 1956, Tacoma Narrows Bridge, Thales and the olive presses, Thales of Miletus, The Chicago School, the map is not the territory, The Market for Lemons, The Nature of the Firm, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Thomas Bayes, Thomas Davenport, Thomas Malthus, Toyota Production System, transaction costs, ultimatum game, urban planning, value at risk, world market for maybe five computers, World Values Survey, Yom Kippur War, zero-sum game

., Narrative Economics: How Stories Go Viral and Drive Major Economic Events (Princeton: PUP, 2019) Shubber, K., ‘Theranos Founder Charged with “Massive” Securities Fraud’, Financial Times (14 Mar 2018) Shulman, L. B. and Driskell, T. D., ‘Dental Implants: A Historical Perspective’ (1997) in Block, M., Kent, J. and Guerra, L., Implants in Dentistry (Philadelphia: Saunders, 1997) Silver, N., The Signal and the Noise: The Art and Science of Prediction (London: Allen Lane, 2012) Silver, N., ‘When We Say 70 Percent, It Really Means 70 Percent’, FiveThirtyEight (4 Apr 2019) < https://fivethirtyeight.com/features/when-we-say-70-percent-it-really-means-70-percent/ > (accessed 23 Apr 2019) Silver, N., ‘Why FiveThirtyEight Gave Trump a Better Chance Than Almost Anyone Else’, FiveThirtyEight (11 Nov 2016) < https://fivethirtyeight.com/features/why-fivethirtyeight-gave-trump-a-better-chance-than-almost-anyone-else/ > (accessed 23 Apr 2019) Simon, H., Models of Man: Social and Rational (New York: John Wiley and Sons, 1957) Simon, R.

…

And with good reason, as we will now see. The probability of an attack on the Twin Towers ‘We may treat people as if they assigned numerical probabilities to every conceivable event.’ So what was the probability that terrorists would fly passenger planes into the World Trade Center on 11 September 2001? Nate Silver, a well-known political pundit in the United States and a devotee of subjective probabilities and Bayesian reasoning, has attempted to answer that question. According to Silver, ‘most of us would have assigned almost no probability to terrorists crashing planes into buildings in Manhattan when we woke up that morning . . .

…

If you crumple a piece of paper, it makes some shape; but the probability that it would make that particular shape is infinitesimally low. 14 It would be absurd, or at least trivial, to conclude that one has observed a 25 standard deviation occurrence. Crimes are rare and unique events. In chapter 5 we saw Nate Silver fail to make sense of the question ‘What is the probability that an unlikely and unique event would occur?’ when we know that the event has in fact happened. Silver was writing in the context of the attack on the World Trade Center; David Viniar struggled similarly in relation to probabilities in the global financial crisis.

Big Data: A Revolution That Will Transform How We Live, Work, and Think by Viktor Mayer-Schonberger, Kenneth Cukier

23andMe, Affordable Care Act / Obamacare, airport security, Apollo 11, barriers to entry, Berlin Wall, big data - Walmart - Pop Tarts, Black Swan, book scanning, book value, business intelligence, business process, call centre, cloud computing, computer age, correlation does not imply causation, dark matter, data science, double entry bookkeeping, Eratosthenes, Erik Brynjolfsson, game design, hype cycle, IBM and the Holocaust, index card, informal economy, intangible asset, Internet of things, invention of the printing press, Jeff Bezos, Joi Ito, lifelogging, Louis Pasteur, machine readable, machine translation, Marc Benioff, Mark Zuckerberg, Max Levchin, Menlo Park, Moneyball by Michael Lewis explains big data, Nate Silver, natural language processing, Netflix Prize, Network effects, obamacare, optical character recognition, PageRank, paypal mafia, performance metric, Peter Thiel, Plato's cave, post-materialism, random walk, recommendation engine, Salesforce, self-driving car, sentiment analysis, Silicon Valley, Silicon Valley startup, smart grid, smart meter, social graph, sparse data, speech recognition, Steve Jobs, Steven Levy, systematic bias, the scientific method, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, Thomas Davenport, Turing test, vertical integration, Watson beat the top human players on Jeopardy!

A sample of 1,100 observations is sufficient—Earl Babbie, Practice of Social Research (12th ed. 2010), pp. 204–207. [>] The cellphone effect—“Estimating the Cellphone Effect,” September 20, 2008 (http://www.fivethirtyeight.com/2008/09/estimating-cellphone-effect-22-points.html); for more on polling biases and other statistical insights see Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (Penguin, 2012). [>] Steve Jobs’s gene sequencing—Walter Isaacson, Steve Jobs (Simon and Schuster, 2011), pp. 550–551. [>] Google Flu Trends predicting to city level—Dugas et al., “Google Flu Trends.” Etzioni on temporal data—Interview by Cukier, October 2011. [>] John Kunze quotation—Jonathan Rosenthal, “Special Report: International Banking,” The Economist, May 19, 2012, pp. 7–8.

…

Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed. Yale University Press, 1998. Seltzer, William, and Margo Anderson. “The Dark Side of Numbers: The Role of Population Data Systems in Human Rights Abuses.” Social Research 68 (2001) pp. 481–513. Silver, Nate. The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t. Penguin, 2012. Singel, Ryan. “Netflix Spilled Your Brokeback Mountain Secret, Lawsuit Claims.” Wired, December 17, 2009 (http://www.wired.com/threatlevel/2009/12/netflix-privacy-lawsuit/). Smith, Adam. The Wealth of Nations (1776). Reprinted Bantam Classics, 2003.

…

Systematic biases in the way the data is collected can lead to the extrapolated results being very wrong. There are echoes of such problems in election polling using landline phones. The sample is biased against people who only use cell-phones (who are younger and more liberal), as the statistician Nate Silver has pointed out. This has resulted in incorrect election predictions. In the 2008 presidential election between Barack Obama and John McCain, the major polling organizations of Gallup, Pew, and ABC/Washington Post found differences of between one and three percentage points when they polled with and without adjusting for cellphone users—a hefty margin considering the tightness of the race.

Prediction Machines: The Simple Economics of Artificial Intelligence by Ajay Agrawal, Joshua Gans, Avi Goldfarb

Abraham Wald, Ada Lovelace, AI winter, Air France Flight 447, Airbus A320, algorithmic bias, AlphaGo, Amazon Picking Challenge, artificial general intelligence, autonomous vehicles, backpropagation, basic income, Bayesian statistics, Black Swan, blockchain, call centre, Capital in the Twenty-First Century by Thomas Piketty, Captain Sullenberger Hudson, carbon tax, Charles Babbage, classic study, collateralized debt obligation, computer age, creative destruction, Daniel Kahneman / Amos Tversky, data acquisition, data is the new oil, data science, deep learning, DeepMind, deskilling, disruptive innovation, driverless car, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, everywhere but in the productivity statistics, financial engineering, fulfillment center, general purpose technology, Geoffrey Hinton, Google Glasses, high net worth, ImageNet competition, income inequality, information retrieval, inventory management, invisible hand, Jeff Hawkins, job automation, John Markoff, Joseph Schumpeter, Kevin Kelly, Lyft, Minecraft, Mitch Kapor, Moneyball by Michael Lewis explains big data, Nate Silver, new economy, Nick Bostrom, On the Economy of Machinery and Manufactures, OpenAI, paperclip maximiser, pattern recognition, performance metric, profit maximization, QWERTY keyboard, race to the bottom, randomized controlled trial, Ray Kurzweil, ride hailing / ride sharing, Robert Solow, Salesforce, Second Machine Age, self-driving car, shareholder value, Silicon Valley, statistical model, Stephen Hawking, Steve Jobs, Steve Jurvetson, Steven Levy, strong AI, The Future of Employment, the long tail, The Signal and the Noise by Nate Silver, Tim Cook: Apple, trolley problem, Turing test, Uber and Lyft, uber lyft, US Airways Flight 1549, Vernor Vinge, vertical integration, warehouse automation, warehouse robotics, Watson beat the top human players on Jeopardy!, William Langewiesche, Y Combinator, zero-sum game

Kelly Gonsalves, “Google Has More Than 1,000 Artificial Intelligence Projects in the Works,” The Week, October 18, 2016, http://theweek.com/speedreads/654463/google-more-than-1000-artificial-intelligence-projects-works. 5. A rich, entertaining, and ultimately useless debate rages about whether these sabermetric analysts are better or worse than the scouts. As Nate Silver highlights, both the Moneyball types and the scouts have important roles to play. Nate Silver, The Signal and the Noise (New York: Penguin Books, 2015), chapter 3. 6. You may counter and say that surely, in order to improve, the prediction machine needs that past repository of data? This is a subtle issue. Prediction works best when adding new data does not change algorithms too much—that stability is an outcome of good statistical practice.

The Joys of Compounding: The Passionate Pursuit of Lifelong Learning, Revised and Updated by Gautam Baid

Abraham Maslow, activist fund / activist shareholder / activist investor, Airbnb, Alan Greenspan, Albert Einstein, Alvin Toffler, Andrei Shleifer, asset allocation, Atul Gawande, availability heuristic, backtesting, barriers to entry, beat the dealer, Benoit Mandelbrot, Bernie Madoff, bitcoin, Black Swan, book value, business process, buy and hold, Cal Newport, Cass Sunstein, Checklist Manifesto, Clayton Christensen, cognitive dissonance, collapse of Lehman Brothers, commoditize, corporate governance, correlation does not imply causation, creative destruction, cryptocurrency, Daniel Kahneman / Amos Tversky, deep learning, delayed gratification, deliberate practice, discounted cash flows, disintermediation, disruptive innovation, Dissolution of the Soviet Union, diversification, diversified portfolio, dividend-yielding stocks, do what you love, Dunning–Kruger effect, Edward Thorp, Elon Musk, equity risk premium, Everything should be made as simple as possible, fear index, financial independence, financial innovation, fixed income, follow your passion, framing effect, George Santayana, Hans Rosling, hedonic treadmill, Henry Singleton, hindsight bias, Hyman Minsky, index fund, intangible asset, invention of the wheel, invisible hand, Isaac Newton, it is difficult to get a man to understand something, when his salary depends on his not understanding it, Jeff Bezos, John Bogle, Joseph Schumpeter, junk bonds, Kaizen: continuous improvement, Kickstarter, knowledge economy, Lao Tzu, Long Term Capital Management, loss aversion, Louis Pasteur, low interest rates, Mahatma Gandhi, mandelbrot fractal, margin call, Mark Zuckerberg, Market Wizards by Jack D. Schwager, Masayoshi Son, mental accounting, Milgram experiment, moral hazard, Nate Silver, Network effects, Nicholas Carr, offshore financial centre, oil shock, passive income, passive investing, pattern recognition, Peter Thiel, Ponzi scheme, power law, price anchoring, quantitative trading / quantitative finance, Ralph Waldo Emerson, Ray Kurzweil, Reminiscences of a Stock Operator, reserve currency, Richard Feynman, Richard Thaler, risk free rate, risk-adjusted returns, Robert Shiller, Savings and loan crisis, search costs, shareholder value, six sigma, software as a service, software is eating the world, South Sea Bubble, special economic zone, Stanford marshmallow experiment, Steve Jobs, Steven Levy, Steven Pinker, stocks for the long run, subscription business, sunk-cost fallacy, systems thinking, tail risk, Teledyne, the market place, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, time value of money, transaction costs, tulip mania, Upton Sinclair, Walter Mischel, wealth creators, Yogi Berra, zero-sum game

Judge them based on their execution track record and do not get influenced by the use of superlatives and adjectives by other people (which is usually more of a persuasion technique than anything else). 4. Distinguish between the signal and the noise. The signal is the truth. The noise is what distracts us from seeing it. In his book The Signal and the Noise, Nate Silver writes, “There isn’t any more truth in the world than there was before the Internet or the printing press. Most of the data is just noise, as most of the universe is filled with empty space.”23 If we want to increase the signal-to-noise ratio, we need to first cut out the cacophony of all the noise surrounding us.

…

Warren Buffett, Berkshire Hathaway 1987 Annual Letter to Shareholders, February 29, 1988, http://www.berkshirehathaway.com/letters/1987.html. 21. Tetlock and Gardner, Superforecasting. 22. Harvey S. Firestone, Men and Rubber: The Story of Business (Whitefish, MT: Kessinger, 2003). 23. Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t (London: Penguin, 2015). 24. Goodreads.com, accessed December 10, 2019, https://www.goodreads.com/quotes/393102-when-information-is-cheap-attention-becomes-expensive. 25. Ian Cassel, “Paul Lountzis on Differential Insights,” MicroCapClub, October 10, 2016, https://microcapclub.com/2016/10/paul-lountzis-differential-insights/. 26.

…

Outlook Business, January 8, 2018. https://outlookbusiness.com/specials/my-best-pick_2018/bhavin-shah-4052. Siegel, Jeremy. “Valuing Growth Stocks: Revisiting the Nifty Fifty.” American Association of Individual Investors Journal, October 1998. https://www.aaii.com/journal/article/valuing-growth-stocks-revisiting-the-nifty-fifty. Silver, Nate. The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t. London: Penguin, 2015. Simon, Herbert A. “Designing Organizations for an Information Rich World.” In Computers, Communications, and the Public Interest: edited by M. Geenberger. Baltimore, MD: Johns Hopkins Press, 1971. https://digitalcollections.library.cmu.edu/awweb/awarchive?

The Patient Will See You Now: The Future of Medicine Is in Your Hands by Eric Topol

23andMe, 3D printing, Affordable Care Act / Obamacare, Anne Wojcicki, Atul Gawande, augmented reality, Big Tech, bioinformatics, call centre, Clayton Christensen, clean water, cloud computing, commoditize, computer vision, conceptual framework, connected car, correlation does not imply causation, creative destruction, crowdsourcing, dark matter, data acquisition, data science, deep learning, digital divide, disintermediation, disruptive innovation, don't be evil, driverless car, Edward Snowden, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Firefox, gamification, global village, Google Glasses, Google X / Alphabet X, Ignaz Semmelweis: hand washing, information asymmetry, interchangeable parts, Internet of things, Isaac Newton, it's over 9,000, job automation, Julian Assange, Kevin Kelly, license plate recognition, lifelogging, Lyft, Mark Zuckerberg, Marshall McLuhan, meta-analysis, microbiome, Nate Silver, natural language processing, Network effects, Nicholas Carr, obamacare, pattern recognition, personalized medicine, phenotype, placebo effect, quantum cryptography, RAND corporation, randomized controlled trial, Salesforce, Second Machine Age, self-driving car, Silicon Valley, Skype, smart cities, Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia, Snapchat, social graph, speech recognition, stealth mode startup, Steve Jobs, synthetic biology, the scientific method, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, traumatic brain injury, Turing test, Uber for X, uber lyft, Watson beat the top human players on Jeopardy!, WikiLeaks, X Prize

Although Eisenstein stopped short of claiming that the first industrial revolution was an outgrowth of the printing press, many others have claimed this. Marshall McLuhan, in The Gutenberg Galaxy, wrote: “The invention of typography confirmed and extended the new visual stress of applied knowledge, providing the first uniformly repeatable commodity, the first assembly-line, and the first mass-production.”11 More recently, Nate Silver, in The Signal and the Noise, asserted that the industrial revolution of 1775 was sparked by the printing press, whereby the economic growth rate that was stagnant at 0.1 percent per year then grew faster than the growth rate of the population.12 But I prefer to principally assess the Gutenberg transformative effects by the specific attributes that they induced or cultivated instead of as a precursor for subsequent momentous periods in history.

…

Matthew, “The World’s Most Expensive Book Just Sold For Over $14 Million,” Business Insider, November 26, 2013, http://www.businessinsider.com/worlds-most-expensive-book-sells-for-14-million-2013-11. 5. Eisenstein, The Printing Press as an Agent of Change, 152. 6. Ibid., 159. 7. N. Silver, The Signal and the Noise (New York, NY: Penguin, 2012), 2. 8. N. Carr, The Shallows: What the Internet Is Doing to Our Brains (New York, NY: W.W. Norton, 2010), 69. 9. Silver, The Signal and the Noise, 12. 10. Eisenstein, The Printing Press as an Agent of Change, 41. 11. McLuhan, The Gutenberg Galaxy, 124. 12. Silver, The Signal and the Noise, 7. 13. Ibid., 3. 14. Carr, The Shallows: What the Internet Is Doing to Our Brains, 71. 15. “The Book of Jobs: Hope, Hype, and Apple’s iPad,” The Economist, January 30–February 5, 2010. 16.

…

Standage, “Social Networking in the 1600s,” New York Times, June 23, 2013, http://www.nytimes.com/2013/06/23/opinion/sunday/social-networking-in-the-1600s.html?pagewanted=all. 22. Eisenstein, The Printing Press as an Agent of Change, 53. 23. V. Goel, “Our Daily Cup of Facebook,” New York Times, August 13, 2013, http://bits.blogs.nytimes.com/2013/08/13/our-daily-cup-of-facebook/?ref=technology&_r=0&pagewanted=print. 24. N. Silver, The Signal and the Noise, 2. 25. “The March of Protest,” The Economist, June 29, 2013, http://www.economist.com/printedition/2013-06-29. 26. W. Ghonim, Revolution 2.0: The Power of the People Is Greater Than the People in Power (New York: Houghton Mifflin Harcourt, 2012). 27. Eisenstein, The Printing Press as an Agent of Change, 129. 28.

The Myth of Artificial Intelligence: Why Computers Can't Think the Way We Do by Erik J. Larson

AI winter, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, Alignment Problem, AlphaGo, Amazon Mechanical Turk, artificial general intelligence, autonomous vehicles, Big Tech, Black Swan, Bletchley Park, Boeing 737 MAX, business intelligence, Charles Babbage, Claude Shannon: information theory, Computing Machinery and Intelligence, conceptual framework, correlation does not imply causation, data science, deep learning, DeepMind, driverless car, Elon Musk, Ernest Rutherford, Filter Bubble, Geoffrey Hinton, Georg Cantor, Higgs boson, hive mind, ImageNet competition, information retrieval, invention of the printing press, invention of the wheel, Isaac Newton, Jaron Lanier, Jeff Hawkins, John von Neumann, Kevin Kelly, Large Hadron Collider, Law of Accelerating Returns, Lewis Mumford, Loebner Prize, machine readable, machine translation, Nate Silver, natural language processing, Nick Bostrom, Norbert Wiener, PageRank, PalmPilot, paperclip maximiser, pattern recognition, Peter Thiel, public intellectual, Ray Kurzweil, retrograde motion, self-driving car, semantic web, Silicon Valley, social intelligence, speech recognition, statistical model, Stephen Hawking, superintelligent machines, tacit knowledge, technological singularity, TED Talk, The Coming Technological Singularity, the long tail, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, theory of mind, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, Yochai Benkler

Lee Gomes, “Machine-­Learning Maestro Michael Jordan on the Delusions of Big Data and Other Huge Engineering Efforts,” IEEE Spectrum, October 20, 2014, https://­spectrum​.­ieee​.­org​/­robotics​/­a rtificial​-­i ntelligence​ /­machinelearning​-­maestro​-­m ichael​-­jordan​-­on​-­t he​-­delusions​-­of​-­big​-­d ata​ -­a nd​-­other​-­huge​-­engineering​-­efforts. 12. Ibid. 13. Gary Marcus and Ernest Davis, “Eight (No Nine!) Prob­lems with Big Data,” New York Times, April 6, 2014. 14. Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—­But Some D ­ on’t (New York: Penguin Books, 2015). Chapter 17: Neocortical Theories of H ­ uman Intelligence 1. Jeff Hawkins, On Intelligence: How a New Understanding of the Brain ­ ill Lead to the Creation of Truly Intelligent Machines (New York: St.

…

Buzzwords have changed, but the major fad pushing Big Data AI as a panacea threatens pro­g ress in fundamental areas like neuroscience, in spite of bold claims made by enthusiasts like Markram and ­others. The takeaway ­here is that the myth ­really does have practical consequences for our h­ uman f­ utures—in a­ ctual science. OV ER F IT T I NG Statistician Nate Silver has also pointed out the inherent danger of overfitting theories (models) to data, where “overfitting” ­here means spuriously matching a set of data points to a description that contains no genuine explanatory power, ­because the description does not generalize to new, unseen data points in the under­lying distribution in question.

Misbehaving: The Making of Behavioral Economics by Richard H. Thaler

3Com Palm IPO, Alan Greenspan, Albert Einstein, Alvin Roth, Amazon Mechanical Turk, Andrei Shleifer, Apple's 1984 Super Bowl advert, Atul Gawande, behavioural economics, Berlin Wall, Bernie Madoff, Black-Scholes formula, book value, business cycle, capital asset pricing model, Cass Sunstein, Checklist Manifesto, choice architecture, clean water, cognitive dissonance, conceptual framework, constrained optimization, Daniel Kahneman / Amos Tversky, delayed gratification, diversification, diversified portfolio, Edward Glaeser, endowment effect, equity premium, equity risk premium, Eugene Fama: efficient market hypothesis, experimental economics, Fall of the Berlin Wall, George Akerlof, hindsight bias, Home mortgage interest deduction, impulse control, index fund, information asymmetry, invisible hand, Jean Tirole, John Nash: game theory, John von Neumann, Kenneth Arrow, Kickstarter, late fees, law of one price, libertarian paternalism, Long Term Capital Management, loss aversion, low interest rates, market clearing, Mason jar, mental accounting, meta-analysis, money market fund, More Guns, Less Crime, mortgage debt, Myron Scholes, Nash equilibrium, Nate Silver, New Journalism, nudge unit, PalmPilot, Paul Samuelson, payday loans, Ponzi scheme, Post-Keynesian economics, presumed consent, pre–internet, principal–agent problem, prisoner's dilemma, profit maximization, random walk, randomized controlled trial, Richard Thaler, risk free rate, Robert Shiller, Robert Solow, Ronald Coase, Silicon Valley, South Sea Bubble, Stanford marshmallow experiment, statistical model, Steve Jobs, sunk-cost fallacy, Supply of New York City Cabdrivers, systematic bias, technology bubble, The Chicago School, The Myth of the Rational Market, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, Thomas Kuhn: the structure of scientific revolutions, transaction costs, ultimatum game, Vilfredo Pareto, Walter Mischel, zero-sum game

Since Romer wrote his paper, the frequency of going for it on fourth down has marginally gone down, meaning that teams have gotten dumber! (Similarly, there has been no noticeable change in teams’ draft strategy since our paper came out.) Nate Silver, the ex–sports analytics junkie who became famous for his political forecasts and for the excellent book The Signal and the Noise, estimates that bad fourth-down decisions cost a football team an average of half a win per season. The Times analysts estimate it to be closer to two-thirds of a win per year. That may not seem like a lot, but the season is only sixteen games.

…

Chapter 29: Football 277 “Division of labor strongly attenuates”: Stewart (1997). 278 football paper: Massey and Thaler (2013). 280 The winner’s curse: For a review, see my “Anomalies” column on the subject (Thaler, 1988a). 280 The false consensus effect: Ross, Greene, and House (1977). 284 If a team is paying a high draft pick a lot of money: Camerer and Weber (1999). 292 teams don’t go for it: Romer (2006). 292 New York Times used his model: For an example of Brian Burke’s work, see http://www.advancedfootballanalytics.com/. 292 “New York Times 4th Down Bot”: The bot’s recommendations can be found at http://nyt4thdownbot.com/. For a comparison between coaches and the NYT Bot’s performances, see Burk and Quealy (2014). 292 The Signal and the Noise: Silver (2012). 293 Peter Principle: Peter and Hull (1969). Chapter 30: Game Shows 296 They asked me if I would like to join the team: Post et al. (2008). 299 my paper with Eric Johnson: Thaler and Johnson (1990). 300 an experiment to measure the difference between public and private decisions: Baltussen, van den Assem, and van Dolder (2015). 301 more risk averse in front of the crowd: This lines up with findings that investors take more risks online than in front of others.

…

“Comments on Miller and on Kleidon.” Journal of Business 59, no. 4, part 2: S501–5. ———. 2000. Irrational Exuberance. Princeton: Princeton University Press. Shleifer, Andrei, and Robert W. Vishny. 1997. “The Limits of Arbitrage.” Journal of Finance 52, no. 1: 35–55. Silver, Nate. 2012. The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t. New York: Penguin. Simon, Herbert A. 1957. Models of Man, Social and Rational: Mathematical Essays on Rational Human Behavior in a Social Setting. Oxford: Wiley. Sloman, Steven A. 1996. “The Empirical Case for Two Systems of Reasoning.” Psychological Bulletin 119, no. 1: 3.

Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins by Garry Kasparov

3D printing, Ada Lovelace, AI winter, Albert Einstein, AlphaGo, AltaVista, Apple Newton, barriers to entry, Berlin Wall, Bletchley Park, business process, call centre, Charles Babbage, Charles Lindbergh, clean water, computer age, cotton gin, Daniel Kahneman / Amos Tversky, David Brooks, DeepMind, Donald Trump, Douglas Hofstadter, driverless car, Drosophila, Elon Musk, Erik Brynjolfsson, factory automation, Freestyle chess, gamification, Gödel, Escher, Bach, Hans Moravec, job automation, Ken Thompson, Leonard Kleinrock, low earth orbit, machine translation, Max Levchin, Mikhail Gorbachev, move 37, Nate Silver, Nick Bostrom, Norbert Wiener, packet switching, pattern recognition, Ray Kurzweil, Richard Feynman, rising living standards, rolodex, Second Machine Age, self-driving car, Silicon Valley, Silicon Valley startup, Skype, speech recognition, stem cell, Stephen Hawking, Steven Pinker, technological singularity, The Coming Technological Singularity, The Signal and the Noise by Nate Silver, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, zero-sum game

The punch line to his theory was that Deep Blue’s mysterious move wasn’t profound at all; it was a blunder and the result of a yet another bug. Per Campbell and Hsu, the move was “random,” the result of a known bug they had failed to kill before the match began. This tale acquired new life when election analyst Nate Silver used it as the centerpiece for an entire chapter of his 2012 book, The Signal and the Noise. The narrative suggested by Frederic and spread by Campbell was irresistible: Kasparov lost to Deep Blue because of a bug! Writes Silver, “The bug was anything but unfortunate for Deep Blue: it was likely what allowed the computer to beat Kasparov.” TIME, Wired, and other outlets ran with breathless variations on this theme, each story containing more errors about chess and more silly assumptions about my mental state than the last.

…

Two paragraphs after Illescas says IBM had hired Russian speakers to spy on me, he says the team entered this critical line into Deep Blue’s book that morning? An obscure variation that I had only discussed with my team in the privacy of our suite at the Plaza Hotel that week in New York? I’m no Nate Silver, but the odds of winning the lottery are quite attractive in comparison to those of the Deep Blue team entering a specific variation I had never played before in my life into the computer’s book on the very same day it appeared on the board in the final game. And not only preparing the machine for the 4..Nd7 Caro-Kann—even during my brief dalliance with the Caro-Kann as a fifteen-year-old I played the 4..Bf5 line exclusively—but also forcing it to play 8.Nxe6 and doing this despite generally giving Deep Blue “a lot of freedom to play,” in Illescas’s own words.

Warnings by Richard A. Clarke

"Hurricane Katrina" Superdome, active measures, Albert Einstein, algorithmic trading, anti-communist, artificial general intelligence, Asilomar, Asilomar Conference on Recombinant DNA, Bear Stearns, behavioural economics, Bernie Madoff, Black Monday: stock market crash in 1987, carbon tax, cognitive bias, collateralized debt obligation, complexity theory, corporate governance, CRISPR, cuban missile crisis, data acquisition, deep learning, DeepMind, discovery of penicillin, double helix, Elon Musk, failed state, financial thriller, fixed income, Flash crash, forensic accounting, friendly AI, Hacker News, Intergovernmental Panel on Climate Change (IPCC), Internet of things, James Watt: steam engine, Jeff Bezos, John Maynard Keynes: Economic Possibilities for our Grandchildren, knowledge worker, Maui Hawaii, megacity, Mikhail Gorbachev, money market fund, mouse model, Nate Silver, new economy, Nicholas Carr, Nick Bostrom, nuclear winter, OpenAI, pattern recognition, personalized medicine, phenotype, Ponzi scheme, Ray Kurzweil, Recombinant DNA, Richard Feynman, Richard Feynman: Challenger O-ring, risk tolerance, Ronald Reagan, Sam Altman, Search for Extraterrestrial Intelligence, self-driving car, Silicon Valley, smart grid, statistical model, Stephen Hawking, Stuxnet, subprime mortgage crisis, tacit knowledge, technological singularity, The Future of Employment, the scientific method, The Signal and the Noise by Nate Silver, Tunguska event, uranium enrichment, Vernor Vinge, WarGames: Global Thermonuclear War, Watson beat the top human players on Jeopardy!, women in the workforce, Y2K

Dan Gardner’s 2012 Future Babble draws on recent research in psychology, neuroscience, and behavioral economics to detail the biases and other cognitive processes that skew our judgment when we try to make predictions about the future. And building on a successful career in sports and political forecasting, Nate Silver discusses in his book, The Signal and the Noise, how thinking more probabilistically can help us distill more accurate predictions from a sea of raw data. Fundamentally, these books all identify the difficulties inherent in trying to see into the future. Predicting natural phenomena is stymied by the chaotic nature of the universe: natural processes are nonlinear systems driven by feedback loops that are often inherently unpredictable themselves.

…

However, such a response does little to help prepare for future disaster or to ensure that Cassandra’s warnings do not again go unheeded. As it seemed we weren’t the only ones who had noticed Cassandras in our midst, we then wondered if there existed any scholarly research on the topic of predictions. In fact, prediction is something that academics have spent a lot of time studying and considering. The statistician Nate Silver has taken a highly quantitative approach to prediction, one that works for a certain class of event. The jurist Richard Posner examined the phenomenon of catastrophes in the years after 9/11. Psychologists like Dan Ariely and Tsachi Ein-Dor have probed the way our brains work (and don’t) through empirical observation and the study of warnings.

Bulletproof Problem Solving by Charles Conn, Robert McLean

active transport: walking or cycling, Airbnb, Amazon Mechanical Turk, asset allocation, availability heuristic, Bayesian statistics, behavioural economics, Big Tech, Black Swan, blockchain, book value, business logic, business process, call centre, carbon footprint, cloud computing, correlation does not imply causation, Credit Default Swap, crowdsourcing, David Brooks, deep learning, Donald Trump, driverless car, drop ship, Elon Musk, endowment effect, fail fast, fake news, future of work, Garrett Hardin, Hyperloop, Innovator's Dilemma, inventory management, iterative process, loss aversion, megaproject, meta-analysis, Nate Silver, nudge unit, Occam's razor, pattern recognition, pets.com, prediction markets, principal–agent problem, RAND corporation, randomized controlled trial, risk tolerance, Silicon Valley, SimCity, smart contracts, stem cell, sunk-cost fallacy, the rule of 72, the scientific method, The Signal and the Noise by Nate Silver, time value of money, Tragedy of the Commons, transfer pricing, Vilfredo Pareto, walkable city, WikiLeaks

Todd, and the ABC Research Group, Simple Heuristics That Make Us Smart (Oxford University Press, 2000). 3  Report prepared for the United Kingdom's Department of International Development by The Nature Conservancy, WWF, and the University of Manchester, “Improving Hydropower Outcomes through System Scale Planning, An Example from Myanmar,” 2016. 4  Warren Buffett, “My Philanthropic Pledge,” Fortune, June 16, 2010. 5  Our friend Barry Nalebuff of Yale points out that the actual rule is 69.3, but is usually rounded up to 72 because it is easier to do the division in your head. 6  CB Insights, May 25, 2015, www.cbinsights.com. 7  Nate Silver, The Signal and the Noise (Penguin, 2012). 8  Dan Lovallo, Carmina Clarke, and Colin Camerer, “Robust Analogizing and the Outside View: Two Empirical Tests of Case Based Decision Making,” Strategic Management Journal 33, no. 5 (2012): 496–512. 9  “‘Chainsaw Al’ Axed,” CNN Money, June 15, 1998. 10  This problem was suggested by Barry Nalebuff of Yale University. 11  Nicklas Garemo, Stefan Matzinger, and Robert Palter, “Megaprojects: The Good, the Bad, and the Better,” McKinsey Quarterly, July 2015 (quoting Bent Flyvberg, Oxford Saïd Business School). 12  Daniel Kahneman, Dan Lovallo, and Olivier Sibony, “Before You Make that Big Decision,” Harvard Business Review, June 2011. 13  Gerd Gigerenzer, Peter M.

…

The decision was made to pursue action in the courts because the board felt the chances of success were greater than an indifference probability of 10% ($100 million expected value of a successful court action divided by legal costs of $10 million). We will see this example more fully in the next chapter. Much has been made of Bayesian thinking in recent years with books like The Signal and the Noise.7 Bayesian thinking is really about conditional probability, which is the probability of an event given another event took place which also has a probability, called a prior probability. As a simple example, look at the probability of it raining given that it is cloudy (the prior probability), versus the probability of it raining if it is currently sunny.

Cogs and Monsters: What Economics Is, and What It Should Be by Diane Coyle

3D printing, additive manufacturing, Airbnb, Al Roth, Alan Greenspan, algorithmic management, Amazon Web Services, autonomous vehicles, banking crisis, barriers to entry, behavioural economics, Big bang: deregulation of the City of London, biodiversity loss, bitcoin, Black Lives Matter, Boston Dynamics, Bretton Woods, Brexit referendum, business cycle, call centre, Carmen Reinhart, central bank independence, choice architecture, Chuck Templeton: OpenTable:, cloud computing, complexity theory, computer age, conceptual framework, congestion charging, constrained optimization, coronavirus, COVID-19, creative destruction, credit crunch, data science, DeepMind, deglobalization, deindustrialization, Diane Coyle, discounted cash flows, disintermediation, Donald Trump, Edward Glaeser, en.wikipedia.org, endogenous growth, endowment effect, Erik Brynjolfsson, eurozone crisis, everywhere but in the productivity statistics, Evgeny Morozov, experimental subject, financial deregulation, financial innovation, financial intermediation, Flash crash, framing effect, general purpose technology, George Akerlof, global supply chain, Goodhart's law, Google bus, haute cuisine, High speed trading, hockey-stick growth, Ida Tarbell, information asymmetry, intangible asset, Internet of things, invisible hand, Jaron Lanier, Jean Tirole, job automation, Joseph Schumpeter, Kenneth Arrow, Kenneth Rogoff, knowledge economy, knowledge worker, Les Trente Glorieuses, libertarian paternalism, linear programming, lockdown, Long Term Capital Management, loss aversion, low earth orbit, lump of labour, machine readable, market bubble, market design, Menlo Park, millennium bug, Modern Monetary Theory, Mont Pelerin Society, multi-sided market, Myron Scholes, Nash equilibrium, Nate Silver, Network effects, Occupy movement, Pareto efficiency, payday loans, payment for order flow, Phillips curve, post-industrial society, price mechanism, Productivity paradox, quantitative easing, randomized controlled trial, rent control, rent-seeking, ride hailing / ride sharing, road to serfdom, Robert Gordon, Robert Shiller, Robert Solow, Robinhood: mobile stock trading app, Ronald Coase, Ronald Reagan, San Francisco homelessness, savings glut, school vouchers, sharing economy, Silicon Valley, software is eating the world, spectrum auction, statistical model, Steven Pinker, tacit knowledge, The Chicago School, The Future of Employment, The Great Moderation, the map is not the territory, The Rise and Fall of American Growth, the scientific method, The Signal and the Noise by Nate Silver, the strength of weak ties, The Wealth of Nations by Adam Smith, total factor productivity, transaction costs, Uber for X, urban planning, winner-take-all economy, Winter of Discontent, women in the workforce, Y2K

We are typically trying to test hypotheses about a small number of variables in a complex world of millions of variables, with a huge amount of two-way feedback or simultaneity, and using a relatively small amount of data, sometimes of dubious quality. It is difficult in the extreme to establish causality rather than correlation. Neither RCTs nor more realistic assumptions in economic models make a difference to the sheer difficulty of the empirical challenge. Nate Silver writes in his bestseller The Signal and the Noise: The government produces data on literally 45,000 economic indicators each year. Private data providers track as many as four million statistics. The temptation that some economists succumb to is to put all this data into a blender and claim that the resulting gruel is haute cuisine.

…

., 2013, Finance and the Good Society, Princeton, NJ: Princeton University Press. Shiller, Robert J., 2017, ‘Narrative Economics’, American Economic Review, 107 (4), 967–1004, doi: 10.1257/aer.107.4.967. Shiller, R. J., 2019, Narrative Economics: How Stories Go Viral and Drive Major Economic Events, Princeton, NJ: Princeton University Press. Silver, N., 2012, The Signal and the Noise: The Art and Science of Prediction, London: Penguin. Skidelsky, R., 1992, John Maynard Keynes: A Biography, Vol. 2: The Economist as Saviour, 1920–1937, London: Macmillan. Skidelsky, Robert, 2020, What’s Wrong with Economics: A Primer for the Perplexed, New Haven, CT: Yale University Press.

Army of None: Autonomous Weapons and the Future of War by Paul Scharre

"World Economic Forum" Davos, active measures, Air France Flight 447, air gap, algorithmic trading, AlphaGo, Apollo 13, artificial general intelligence, augmented reality, automated trading system, autonomous vehicles, basic income, Black Monday: stock market crash in 1987, brain emulation, Brian Krebs, cognitive bias, computer vision, cuban missile crisis, dark matter, DARPA: Urban Challenge, data science, deep learning, DeepMind, DevOps, Dr. Strangelove, drone strike, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, facts on the ground, fail fast, fault tolerance, Flash crash, Freestyle chess, friendly fire, Herman Kahn, IFF: identification friend or foe, ImageNet competition, information security, Internet of things, Jeff Hawkins, Johann Wolfgang von Goethe, John Markoff, Kevin Kelly, Korean Air Lines Flight 007, Loebner Prize, loose coupling, Mark Zuckerberg, military-industrial complex, moral hazard, move 37, mutually assured destruction, Nate Silver, Nick Bostrom, PalmPilot, paperclip maximiser, pattern recognition, Rodney Brooks, Rubik’s Cube, self-driving car, sensor fusion, South China Sea, speech recognition, Stanislav Petrov, Stephen Hawking, Steve Ballmer, Steve Wozniak, Strategic Defense Initiative, Stuxnet, superintelligent machines, Tesla Model S, The Signal and the Noise by Nate Silver, theory of mind, Turing test, Tyler Cowen, universal basic income, Valery Gerasimov, Wall-E, warehouse robotics, William Langewiesche, Y2K, zero day

Later that evening while pouring over a recreation of the final moves, Kasparov discovered that in 20 moves he would have checkmated Deep Blue. The implication was that Deep Blue made a nonsense move and resigned because it could see 20 moves ahead, a staggering advantage in chess. Nate Silver reports that this bug may have irreparably shaken Kasparov’s confidence. Nate Silver, The Signal and the Noise: Why So Many Predictions Fail (New York: Penguin, 2015), 276–289. 150 recent UNIDIR report on autonomous weapons and risk: UN Institute for Disarmament Research, “Safety, Unintentional Risk and Accidents in the Weaponization of Increasingly Autonomous Technologies,” 2016, http://www.unidir.org/files/publications/pdfs/safety-unintentional-risk-and-accidents-en-668.pdf.

Artificial Unintelligence: How Computers Misunderstand the World by Meredith Broussard

"Susan Fowler" uber, 1960s counterculture, A Declaration of the Independence of Cyberspace, Ada Lovelace, AI winter, Airbnb, algorithmic bias, AlphaGo, Amazon Web Services, autonomous vehicles, availability heuristic, barriers to entry, Bernie Sanders, Big Tech, bitcoin, Buckminster Fuller, Charles Babbage, Chris Urmson, Clayton Christensen, cloud computing, cognitive bias, complexity theory, computer vision, Computing Machinery and Intelligence, crowdsourcing, Danny Hillis, DARPA: Urban Challenge, data science, deep learning, Dennis Ritchie, digital map, disruptive innovation, Donald Trump, Douglas Engelbart, driverless car, easy for humans, difficult for computers, Electric Kool-Aid Acid Test, Elon Musk, fake news, Firefox, gamification, gig economy, global supply chain, Google Glasses, Google X / Alphabet X, Greyball, Hacker Ethic, independent contractor, Jaron Lanier, Jeff Bezos, Jeremy Corbyn, John Perry Barlow, John von Neumann, Joi Ito, Joseph-Marie Jacquard, life extension, Lyft, machine translation, Mark Zuckerberg, mass incarceration, Minecraft, minimum viable product, Mother of all demos, move fast and break things, Nate Silver, natural language processing, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, One Laptop per Child (OLPC), opioid epidemic / opioid crisis, PageRank, Paradox of Choice, payday loans, paypal mafia, performance metric, Peter Thiel, price discrimination, Ray Kurzweil, ride hailing / ride sharing, Ross Ulbricht, Saturday Night Live, school choice, self-driving car, Silicon Valley, Silicon Valley billionaire, speech recognition, statistical model, Steve Jobs, Steven Levy, Stewart Brand, TechCrunch disrupt, Tesla Model S, the High Line, The Signal and the Noise by Nate Silver, theory of mind, traumatic brain injury, Travis Kalanick, trolley problem, Turing test, Uber for X, uber lyft, Watson beat the top human players on Jeopardy!, We are as Gods, Whole Earth Catalog, women in the workforce, work culture , yottabyte

But now it’s also going to be about poring over data and equipping yourself with the tools to analyse it and picking out what’s interesting. And keeping it in perspective, helping people out by really seeing where it all fits together, and what’s going on in the country.”19 By the time Nate Silver launched FiveThirtyEight.com and published his book The Signal and the Noise in 2012, the term data journalism was in widespread use among investigative journalists.20 As computers have evolved, human nature has not. People need to be kept honest. I hope that this book will help you think like a data journalist so that you can challenge false claims about technology and uncover injustice and inequality embedded in today’s computational systems.

…

Anderson, “Towards a Sociology of Computational and Algorithmic Journalism”; Schudson, “Four Approaches to the Sociology of News.” 16. Usher, Interactive Journalism. 17. Royal, “The Journalist as Programmer.” 18. Hamilton, Democracy’s Detectives. 19. Arthur, “Analysing Data Is the Future for Journalists, Says Tim Berners-Lee.” 20. Silver, The Signal and the Noise. II When Computers Don’t Work 5 Why Poor Schools Can’t Win at Standardized Tests Machines, code, and data can all work together to produce amazing, exciting insights. Getting hold of the right numbers can increase revenue, improve decision making, or help you find a mate—or so the thinking goes.

…

New York: HarperCollins, 2016. Silver, David, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, et al. “Mastering the Game of Go with Deep Neural Networks and Tree Search.” Nature 529 (January 28, 2016): 484–489. doi:10.1038/nature16961. Silver, Nate. The Signal and the Noise: Why so Many Predictions Fail—but Some Don’t. New York: Penguin Books, 2015. Singh, Santokh. “Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey.” Traffic Safety Facts Crash Stats. Washington, DC: Bowhead Systems Management, Inc., working under contract with the Mathematical Analysis Division of the National Center for Statistics and Analysis, NHTSA, February 2015. https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812115.

Everydata: The Misinformation Hidden in the Little Data You Consume Every Day by John H. Johnson

Affordable Care Act / Obamacare, autism spectrum disorder, Black Swan, business intelligence, Carmen Reinhart, cognitive bias, correlation does not imply causation, Daniel Kahneman / Amos Tversky, data science, Donald Trump, en.wikipedia.org, Kenneth Rogoff, labor-force participation, lake wobegon effect, Long Term Capital Management, Mercator projection, Mercator projection distort size, especially Greenland and Africa, meta-analysis, Nate Silver, obamacare, p-value, PageRank, pattern recognition, publication bias, QR code, randomized controlled trial, risk-adjusted returns, Ronald Reagan, selection bias, statistical model, The Signal and the Noise by Nate Silver, Thomas Bayes, Tim Cook: Apple, wikimedia commons, Yogi Berra

That said, the USGS does describe the places “most likely to produce earthquakes in the long term.” They call this forecasting, when they estimate the likelihood of a seismic event occurring over a period of time. This brings us to the distinction—or lack thereof—between a prediction and a forecast. As Nate Silver notes in The Signal and the Noise, the terms are used differently by some (most notably seismologists, who study earthquakes) but interchangeably by others. Some would argue that predictions are binary—something will or won’t happen—while forecasts are more probabilistic—there’s an X percent chance that something will happen.

Whiplash: How to Survive Our Faster Future by Joi Ito, Jeff Howe

3D printing, air gap, Albert Michelson, AlphaGo, Amazon Web Services, artificial general intelligence, basic income, Bernie Sanders, Big Tech, bitcoin, Black Lives Matter, Black Swan, Bletchley Park, blockchain, Burning Man, business logic, buy low sell high, Claude Shannon: information theory, cloud computing, commons-based peer production, Computer Numeric Control, conceptual framework, CRISPR, crowdsourcing, cryptocurrency, data acquisition, deep learning, DeepMind, Demis Hassabis, digital rights, disruptive innovation, Donald Trump, double helix, Edward Snowden, Elon Musk, Ferguson, Missouri, fiat currency, financial innovation, Flash crash, Ford Model T, frictionless, game design, Gerolamo Cardano, informal economy, information security, interchangeable parts, Internet Archive, Internet of things, Isaac Newton, Jeff Bezos, John Harrison: Longitude, Joi Ito, Khan Academy, Kickstarter, Mark Zuckerberg, microbiome, move 37, Nate Silver, Network effects, neurotypical, Oculus Rift, off-the-grid, One Laptop per Child (OLPC), PalmPilot, pattern recognition, peer-to-peer, pirate software, power law, pre–internet, prisoner's dilemma, Productivity paradox, quantum cryptography, race to the bottom, RAND corporation, random walk, Ray Kurzweil, Ronald Coase, Ross Ulbricht, Satoshi Nakamoto, self-driving car, SETI@home, side project, Silicon Valley, Silicon Valley startup, Simon Singh, Singularitarianism, Skype, slashdot, smart contracts, Steve Ballmer, Steve Jobs, Steven Levy, Stewart Brand, Stuxnet, supply-chain management, synthetic biology, technological singularity, technoutopianism, TED Talk, The Nature of the Firm, the scientific method, The Signal and the Noise by Nate Silver, the strength of weak ties, There's no reason for any individual to have a computer in his home - Ken Olsen, Thomas Kuhn: the structure of scientific revolutions, Two Sigma, universal basic income, unpaid internship, uranium enrichment, urban planning, warehouse automation, warehouse robotics, Wayback Machine, WikiLeaks, Yochai Benkler

., 13 34 Page is referring to the famous scene in the mockumentary This Is Spinal Tap in which the mentally addled lead guitarist, Nigel Tufnel, tries to explain the significance of an amplifier with the capacity to exceed the conventional 10 on the volume knob. “Well, it’s one louder, isn’t it?” 35 Quoted in Joichi Ito and Jeff Howe, “The Future: An Instruction Manual,” LinkedIn Pulse, October 2, 2012, https://www.linkedin.com/pulse/20121002120301-1391-the-future-an-instruction-manual. 36 Nate Silver, The Signal and the Noise: Why So Many Predictions Fail (New York: Penguin, 2012); Louis Menand, “Everybody’s an Expert,” New Yorker, December 5, 2005, http://www.newyorker.com/magazine/2005/12/05/everybodys-an-expert; Stephen J. Dubner, “The Folly of Prediction,” Freakonomics podcast, September 14, 2011, http://freakonomics.com/2011/09/14/new-freakonomics-radio-podcast-the-folly-of-prediction/. 37 National Council for Science and the Environment, The Climate Solutions Consensus: What We Know and What to Do About It, edited by David Blockstein and Leo Wiegman (Washington, D.C.: Island Press, 2012), 3. 38 Oxford Advanced Learner’s Dictionary, http://www.oxforddictionaries.com/us/definition/learner/medium. 39 The Media Lab’s website includes a comprehensive overview of the Lab’s funding model, current research, and history. http://media.mit.edu/about/about-the-lab. 40 Olivia Vanni.

The Formula: How Algorithms Solve All Our Problems-And Create More by Luke Dormehl

3D printing, algorithmic bias, algorithmic trading, Alvin Toffler, Any sufficiently advanced technology is indistinguishable from magic, augmented reality, big data - Walmart - Pop Tarts, call centre, Cass Sunstein, classic study, Clayton Christensen, commoditize, computer age, death of newspapers, deferred acceptance, disruptive innovation, Edward Lorenz: Chaos theory, Erik Brynjolfsson, Evgeny Morozov, Filter Bubble, Flash crash, Florence Nightingale: pie chart, Ford Model T, Frank Levy and Richard Murnane: The New Division of Labor, fulfillment center, Google Earth, Google Glasses, High speed trading, Internet Archive, Isaac Newton, Jaron Lanier, Jeff Bezos, job automation, John Markoff, Kevin Kelly, Kodak vs Instagram, Lewis Mumford, lifelogging, machine readable, machine translation, Marshall McLuhan, means of production, Nate Silver, natural language processing, Netflix Prize, Panopticon Jeremy Bentham, Paradox of Choice, pattern recognition, price discrimination, recommendation engine, Richard Thaler, Rosa Parks, scientific management, self-driving car, sentiment analysis, Silicon Valley, Silicon Valley startup, Slavoj Žižek, social graph, speech recognition, stable marriage problem, Steve Jobs, Steven Levy, Steven Pinker, Stewart Brand, technological determinism, technological solutionism, TED Talk, the long tail, the scientific method, The Signal and the Noise by Nate Silver, upwardly mobile, Wall-E, Watson beat the top human players on Jeopardy!, Y Combinator

While I was first coming up with formulas at college, trying to mathematically determine whether we should go to the library to get some work done, deep down in the recesses of our dorky ids I think that what we were saying is that life is uncertain and we were trying to make it more certain. I’m not as disturbed by numbers providing answers as I am by the potential that there might not be answers.” What is it about the modern world that makes us demand easy answers? Is it that we are naturally pattern-seeking creatures, as the statistician Nate Silver argues in The Signal and the Noise? Or is there something about the effects of the march of technology that demands the kind of answers only an algorithm can provide? “[The algorithm does] seem to be a key metaphor for what matters now in terms of organizing the world,” acknowledges McKenzie Wark, a media theorist who has written about digital technologies for the last 20 years.

What Algorithms Want: Imagination in the Age of Computing by Ed Finn

Airbnb, Albert Einstein, algorithmic bias, algorithmic management, algorithmic trading, AlphaGo, Amazon Mechanical Turk, Amazon Web Services, bitcoin, blockchain, business logic, Charles Babbage, Chuck Templeton: OpenTable:, Claude Shannon: information theory, commoditize, Computing Machinery and Intelligence, Credit Default Swap, crowdsourcing, cryptocurrency, data science, DeepMind, disruptive innovation, Donald Knuth, Donald Shoup, Douglas Engelbart, Douglas Engelbart, Elon Musk, Evgeny Morozov, factory automation, fiat currency, Filter Bubble, Flash crash, game design, gamification, Google Glasses, Google X / Alphabet X, Hacker Conference 1984, High speed trading, hiring and firing, Ian Bogost, industrial research laboratory, invisible hand, Isaac Newton, iterative process, Jaron Lanier, Jeff Bezos, job automation, John Conway, John Markoff, Just-in-time delivery, Kickstarter, Kiva Systems, late fees, lifelogging, Loebner Prize, lolcat, Lyft, machine readable, Mother of all demos, Nate Silver, natural language processing, Neal Stephenson, Netflix Prize, new economy, Nicholas Carr, Nick Bostrom, Norbert Wiener, PageRank, peer-to-peer, Peter Thiel, power law, Ray Kurzweil, recommendation engine, Republic of Letters, ride hailing / ride sharing, Satoshi Nakamoto, self-driving car, sharing economy, Silicon Valley, Silicon Valley billionaire, Silicon Valley ideology, Silicon Valley startup, SimCity, Skinner box, Snow Crash, social graph, software studies, speech recognition, statistical model, Steve Jobs, Steven Levy, Stewart Brand, supply-chain management, tacit knowledge, TaskRabbit, technological singularity, technological solutionism, technoutopianism, the Cathedral and the Bazaar, The Coming Technological Singularity, the scientific method, The Signal and the Noise by Nate Silver, The Structural Transformation of the Public Sphere, The Wealth of Nations by Adam Smith, transaction costs, traveling salesman, Turing machine, Turing test, Uber and Lyft, Uber for X, uber lyft, urban planning, Vannevar Bush, Vernor Vinge, wage slave

Ipeirotis, “Analyzing the Amazon Mechanical Turk Marketplace,” 21. 50. “Mechanical Turk Concepts.” 51. Riskin, “Machines in the Garden.” 52. Ibid., 27. 53. Zuniga, “Kasparov Tries New Strategy to Thwart Computer Opponent.” 54. Finley, “Did a Computer Bug Help Deep Blue Beat Kasparov? | WIRED.” 55. Silver, The Signal and the Noise, 288. 56. Isaacson, “‘Smarter Than You Think,’ by Clive Thompson.” 57. Ipeirotis, “Analyzing the Amazon Mechanical Turk Marketplace,” 21. 58. Glanz, “Data Centers Waste Vast Amounts of Energy, Belying Industry Image.” 59. Cooper, Ipeirotis, and Suri, “The Computer Is the New Sewing Machine: Benefits and Perils of Crowdsourcing”; “Amazon Mechanical Turk.” 60.

…

Aeon, August 28, 2013. http://aeon.co/magazine/living-together/so-you-want-to-invent-your-own-currency. Sedgewick, Robert. “Computer Science 226: Algorithms and Data Structures,” Fall 2007. http://www.cs.princeton.edu/~rs/AlgsDS07/00overview.pdf. “Shit That Siri Says—Baby Stores.” Accessed May 27, 2014. http://knowyourmeme.com/photos/187708-shit-that-siri-says. Silver, Nate. The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t. New York: Penguin, 2012. Snyder, Blake. “Save the Cat!” Accessed June 10, 2014. http://www.savethecat.com. Sopor, Spencer. “Inside Amazon’s Warehouse.” The Morning Call, September 18, 2011. http://articles.mcall.com/2011-09-18/news/mc-allentown-amazon-complaints-20110917_1_warehouse-workers-heat-stress-brutal-heat.

…

Something of this knowing spectacle remained in Kasparov’s matches with Deep Blue, including his persistent efforts to deploy openings and play styles that would throw off IBM’s algorithms.53 The human engineer was never entirely hidden behind the mechanism, as IBM employees tweaked the system between every game. Perhaps the best example of this relationship was a highly scrutinized move near the end of game one, the truth of which was only revealed in 2012 when IBM researcher Murray Campbell was interviewed by the popular statistician Nate Silver. The move in question had sent Kasparov “into a tizzy” as it seemed to reflect the ambiguity and refinement of a human-level intelligence, and many have suggested it threw off the grandmaster’s concentration for the second game, which he proceeded to lose.54 In fact, as Campbell revealed, the move had been a bug, one the engineers corrected after the first match.55 Kasparov himself had made magic out of the algorithm, inventing a sophisticated cultural explanation for what was in the end a random computational artifact.

Simple Rules: How to Thrive in a Complex World by Donald Sull, Kathleen M. Eisenhardt

Affordable Care Act / Obamacare, Airbnb, Apollo 13, asset allocation, Atul Gawande, barriers to entry, Basel III, behavioural economics, Berlin Wall, carbon footprint, Checklist Manifesto, complexity theory, Craig Reynolds: boids flock, Credit Default Swap, Daniel Kahneman / Amos Tversky, democratizing finance, diversification, drone strike, en.wikipedia.org, European colonialism, Exxon Valdez, facts on the ground, Fall of the Berlin Wall, Glass-Steagall Act, Golden age of television, haute cuisine, invention of the printing press, Isaac Newton, Kickstarter, late fees, Lean Startup, Louis Pasteur, Lyft, machine translation, Moneyball by Michael Lewis explains big data, Nate Silver, Network effects, obamacare, Paul Graham, performance metric, price anchoring, RAND corporation, risk/return, Saturday Night Live, seminal paper, sharing economy, Silicon Valley, Startup school, statistical model, Steve Jobs, TaskRabbit, The Signal and the Noise by Nate Silver, transportation-network company, two-sided market, Wall-E, web application, Y Combinator, Zipcar

., “Health on Impulse: When Low Self-Control Promotes Healthy Food Choices,” Health Psychology 33, no. 2 (2013): 103–9, http://www.medscape.com/medline/abstract/2347758. [>] In contrast, people: Brian Wansink, David R. Rust, and Collin R. Payne, “Mindless Eating and Healthy Heuristics for the Irrational,” American Economic Review: Papers and Proceedings 99, no. 2 (2009): 165–69. [>] Meteorologists make: Nate Silver, The Signal and the Noise (New York: Penguin, 2012), 126–27. [>] Japanese honeybees: Atsushi Ugajin et al., “Detection of Neural Activity in the Brains of Japanese Honeybee Workers During the Formation of a ‘Hot Defensive Bee Ball,’” PLoS One 7, no. 3 (2012), available at the website of the National Center for Biotechnology Information, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3303784/. [>] As an example of: Our account of the bees’ choice of new nest is based on the research of Thomas Seeley, especially Thomas D.

Artificial Whiteness by Yarden Katz

affirmative action, AI winter, algorithmic bias, AlphaGo, Amazon Mechanical Turk, autonomous vehicles, benefit corporation, Black Lives Matter, blue-collar work, Californian Ideology, Cambridge Analytica, cellular automata, Charles Babbage, cloud computing, colonial rule, computer vision, conceptual framework, Danny Hillis, data science, David Graeber, deep learning, DeepMind, desegregation, Donald Trump, Dr. Strangelove, driverless car, Edward Snowden, Elon Musk, Erik Brynjolfsson, European colonialism, fake news, Ferguson, Missouri, general purpose technology, gentrification, Hans Moravec, housing crisis, income inequality, information retrieval, invisible hand, Jeff Bezos, Kevin Kelly, knowledge worker, machine readable, Mark Zuckerberg, mass incarceration, Menlo Park, military-industrial complex, Nate Silver, natural language processing, Nick Bostrom, Norbert Wiener, pattern recognition, phenotype, Philip Mirowski, RAND corporation, recommendation engine, rent control, Rodney Brooks, Ronald Reagan, Salesforce, Seymour Hersh, Shoshana Zuboff, Silicon Valley, Silicon Valley billionaire, Silicon Valley ideology, Skype, speech recognition, statistical model, Stephen Hawking, Stewart Brand, Strategic Defense Initiative, surveillance capitalism, talking drums, telemarketer, The Signal and the Noise by Nate Silver, W. E. B. Du Bois, Whole Earth Catalog, WikiLeaks

See the controversy surrounding Cambridge Analytica as reported, for instance, in “Cambridge Analytica Files,” Guardian, https://www.theguardian.com/news/series/cambridge-analytica-files.     9.   Nick Bilton, “Google Buys A.I. Company for Search, Not Robots,” New York Times, January 28, 2014.   10.   David Ignatius, “A Tech Bridge from the Pentagon to Silicon Valley,” Washington Post, January 25, 2019.   11.   See, for example, Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t (New York: Penguin, 2012), 264.   12.   The Global Artificial Intelligence Conference of 2018 invited participants to “hear from AI pioneers representing: Google, Capital One, Johnson & Johnson, Bloomberg, Facebook, Amazon, McKinsey, SAP, Microsoft, Biogen, Royal Caribbean Cruise Line, Uber, IBM, The Boston Consulting Group, Trulia Teradata, Columbia Sportswear Company etc.”

…

Tech, March 7, 2019. Shubik, Martin. “Bibliography on Simulation, Gaming, Artificial Intelligence and Allied Topics.” Journal of the American Statistical Association 55, no. 292 (1960): 736–51. Silk, Leonard. “Nobel Winner’s Heretical Views.” New York Times, November 9, 1978. Silver, Nate. The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t. New York: Penguin Press, 2012. Simon, Herbert A., and Allen Newell. “Heuristic Problem Solving: The Next Advance in Operations Research.” Operations Research 6, no. 1 (1958): 1–10. Simpson, Audra. Mohawk Interruptus: Political Life Across the Borders of Settler States.

The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies by Erik Brynjolfsson, Andrew McAfee

2013 Report for America's Infrastructure - American Society of Civil Engineers - 19 March 2013, 3D printing, access to a mobile phone, additive manufacturing, Airbnb, Alan Greenspan, Albert Einstein, Amazon Mechanical Turk, Amazon Web Services, American Society of Civil Engineers: Report Card, Any sufficiently advanced technology is indistinguishable from magic, autonomous vehicles, barriers to entry, basic income, Baxter: Rethink Robotics, Boston Dynamics, British Empire, business cycle, business intelligence, business process, call centre, carbon tax, Charles Lindbergh, Chuck Templeton: OpenTable:, clean water, combinatorial explosion, computer age, computer vision, congestion charging, congestion pricing, corporate governance, cotton gin, creative destruction, crowdsourcing, data science, David Ricardo: comparative advantage, digital map, driverless car, employer provided health coverage, en.wikipedia.org, Erik Brynjolfsson, factory automation, Fairchild Semiconductor, falling living standards, Filter Bubble, first square of the chessboard / second half of the chessboard, Frank Levy and Richard Murnane: The New Division of Labor, Freestyle chess, full employment, G4S, game design, general purpose technology, global village, GPS: selective availability, Hans Moravec, happiness index / gross national happiness, illegal immigration, immigration reform, income inequality, income per capita, indoor plumbing, industrial robot, informal economy, intangible asset, inventory management, James Watt: steam engine, Jeff Bezos, Jevons paradox, jimmy wales, job automation, John Markoff, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, Joseph Schumpeter, Kevin Kelly, Khan Academy, Kiva Systems, knowledge worker, Kodak vs Instagram, law of one price, low skilled workers, Lyft, Mahatma Gandhi, manufacturing employment, Marc Andreessen, Mark Zuckerberg, Mars Rover, mass immigration, means of production, Narrative Science, Nate Silver, natural language processing, Network effects, new economy, New Urbanism, Nicholas Carr, Occupy movement, oil shale / tar sands, oil shock, One Laptop per Child (OLPC), pattern recognition, Paul Samuelson, payday loans, post-work, power law, price stability, Productivity paradox, profit maximization, Ralph Nader, Ray Kurzweil, recommendation engine, Report Card for America’s Infrastructure, Robert Gordon, Robert Solow, Rodney Brooks, Ronald Reagan, search costs, Second Machine Age, self-driving car, sharing economy, Silicon Valley, Simon Kuznets, six sigma, Skype, software patent, sovereign wealth fund, speech recognition, statistical model, Steve Jobs, Steven Pinker, Stuxnet, supply-chain management, TaskRabbit, technological singularity, telepresence, The Bell Curve by Richard Herrnstein and Charles Murray, the Cathedral and the Bazaar, the long tail, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, total factor productivity, transaction costs, Tyler Cowen, Tyler Cowen: Great Stagnation, Vernor Vinge, warehouse robotics, Watson beat the top human players on Jeopardy!, winner-take-all economy, Y2K

., “Grading Student Loans,” Liberty Street Economics blog, Federal Reserve Bank of New York, March 5, 2012, http://libertystreeteconomics.newyorkfed.org/2012/03/grading-student-loans.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+LibertyStreetEconomics+(Liberty+Street+Economics). 21. Tim Hornyak, “Towel-folding Robot Won’t Do the Dishes,” CNET, March 31, 2010, http://news.cnet.com/8301-17938_105-10471898-1.html. 22. Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t, 1st ed. (New York: Penguin, 2012). Chapter 13 POLICY RECOMMENDATIONS 1. “Employment Level,” Economic Research—Federal Reserve Bank of St. Louis (U.S. Department of Labor, Bureau of Labor Statistics, August 2, 2013), http://research.stlouisfed.org/fred2/series/LNU02000000. 2.

The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences by Rob Kitchin

Bayesian statistics, business intelligence, business process, cellular automata, Celtic Tiger, cloud computing, collateralized debt obligation, conceptual framework, congestion charging, corporate governance, correlation does not imply causation, crowdsourcing, data science, discrete time, disruptive innovation, George Gilder, Google Earth, hype cycle, Infrastructure as a Service, Internet Archive, Internet of things, invisible hand, knowledge economy, Large Hadron Collider, late capitalism, lifelogging, linked data, longitudinal study, machine readable, Masdar, means of production, Nate Silver, natural language processing, openstreetmap, pattern recognition, platform as a service, recommendation engine, RFID, semantic web, sentiment analysis, SimCity, slashdot, smart cities, Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia, smart grid, smart meter, software as a service, statistical model, supply-chain management, technological solutionism, the scientific method, The Signal and the Noise by Nate Silver, transaction costs

Sicular, S. (2013) ‘Big data is falling into the trough of disillusionment’, Gartner, 22 January, http://blogs.gartner.com/svetlana-sicular/big-data-is-falling-into-the-trough-of-disillusionment/ (last accessed 26 February 2013). Siegel, E. (2013) Predictive Analytics. Wiley, Hoboken, NJ. Sigala, M. (2005) ‘Integrating customer relationship management in hotel operations: managerial and operations implications’, International Journal of Hospitality Management, 24(3): 391–413. Silver, N. (2012) The Signal and the Noise: The Art and Science of Prediction. Penguin, London. Singer, N. (2012a) ‘You for sale: mapping, and sharing, the consumer genome’, New York Times, 17 June, http://www.nytimes.com/2012/06/17/technology/acxiom-the-quiet-giant-of-consumerdatabase-marketing.html (last accessed 11 October 2013).

…

, International Data Privacy Law, online first, http://idpl.oxfordjournals.org/content/early/2013/01/24/idpl.ips036.short (last accessed 15 July 2013). Ruppert, E. (2012) ‘The governmental topologies of database devices’, Theory, Culture Society, 29: 116–36. Ruppert, E. (2013) ‘Rethinking empirical social sciences’, Dialogues in Human Geography, 3(3): 268–73. Salmon, F. (2014) ‘Why the Nate Silvers of the world don’t know everything’, Wired, 7 January, http://www.wired.com/business/2014/01/quants-dont-know-everything/ (last accessed 8 January 2014). Salus, P. (1995) Casting the Net: From Arpanet to Internet and Beyond. Addison Wesley, Reading, MA. Sawyer, S. (2008) ‘Data wealth, data poverty, science and cyberinfrastructure’, Prometheus: Critical Studies in Innovation, 26(4): 355–71.

How to Run a Government: So That Citizens Benefit and Taxpayers Don't Go Crazy by Michael Barber

Affordable Care Act / Obamacare, anti-fragile, Atul Gawande, battle of ideas, Berlin Wall, Black Swan, Checklist Manifesto, collapse of Lehman Brothers, collective bargaining, deep learning, deliberate practice, facts on the ground, failed state, fear of failure, full employment, G4S, illegal immigration, invisible hand, libertarian paternalism, Mark Zuckerberg, Nate Silver, North Sea oil, obamacare, performance metric, Potemkin village, Ronald Reagan, school choice, The Signal and the Noise by Nate Silver, transaction costs, WikiLeaks

And, by the way, if I go down, you’re coming with me.’ In other words, reputations are at risk. But David’s point was right; if you want thousands of public servants to take responsibility for their part in achieving a goal, you need to make it clear that you take your responsibility seriously too. Nate Silver, in his magisterial survey of ‘the art and science of prediction’, The Signal and the Noise, urges us to think probabilistically when we forecast: rather than ‘it’s going to rain tomorrow’, ‘there is a 90 per cent chance of rain tomorrow’. This is, of course, the right way to think analytically when a target is being discussed. He also makes another crucial point – that it’s dangerous to depend purely on the data.

…

Bobst Center for Peace and Justice, Princeton University Schlesinger, R. (2008), White House Ghosts: Presidents and Their Speechwriters, New York, Simon & Schuster Seldon, A. (2005), Blair, London, Simon & Schuster —, Snowden, P. and Collings, D. (2007), Blair Unbound, London, Simon & Schuster Sellar, W. and Yeatman, R. (1998), 1066 and All That, London, Methuen Shlaes, A. (2013), Coolidge, New York, HarperCollins Silver, N. (2012), The Signal and the Noise: The Art and Science of Prediction, London, Penguin Smillie, I. (2009), Freedom from Want: The Remarkable Success Story of BRAC, the Global Grassroots Organisation That’s Winning the Fight Against Poverty, Dhaka, Kumarian Press Smith, J. E. (2012), Eisenhower in War and Peace, New York, Random House State of Victoria (2005), Growing Victoria Together Steinberg, J. (2011), Bismarck: A Life, Oxford, Oxford University Press Stevenson, A. (2013), The Public Sector: Managing the Unmanageable, London, Kogan Page Sugden, J. (2012), Nelson: The Sword of Albion, London, Bodley Head Taleb, N.

Uncharted: How to Map the Future by Margaret Heffernan

"World Economic Forum" Davos, 23andMe, Affordable Care Act / Obamacare, Airbnb, Alan Greenspan, Anne Wojcicki, anti-communist, Atul Gawande, autonomous vehicles, banking crisis, Berlin Wall, Boris Johnson, Brexit referendum, chief data officer, Chris Urmson, clean water, complexity theory, conceptual framework, cosmic microwave background, creative destruction, CRISPR, crowdsourcing, data science, David Attenborough, discovery of penicillin, driverless car, epigenetics, Fall of the Berlin Wall, fear of failure, George Santayana, gig economy, Google Glasses, Greta Thunberg, Higgs boson, index card, Internet of things, Jaron Lanier, job automation, Kickstarter, Large Hadron Collider, late capitalism, lateral thinking, Law of Accelerating Returns, liberation theology, mass immigration, mass incarceration, megaproject, Murray Gell-Mann, Nate Silver, obamacare, oil shale / tar sands, passive investing, pattern recognition, Peter Thiel, prediction markets, RAND corporation, Ray Kurzweil, Rosa Parks, Sam Altman, scientific management, Shoshana Zuboff, Silicon Valley, smart meter, Stephen Hawking, Steve Ballmer, Steve Jobs, surveillance capitalism, TED Talk, The Signal and the Noise by Nate Silver, Tim Cook: Apple, twin studies, University of East Anglia

., 2010 11 Author interview with Robert Skidelsky, 2 August 2018 12 Engelberg, Joseph, Sasseville, Caroline and Williams, Jared, ‘Market Madness? The Case of Mad Money’, Management Science, Vol. 58, Iss. 2, 2012 13 This is one reason why Warren Buffett has advised his wife to invest in tracker funds after this death. The pollster Nate Silver makes the challenging observation that the rare group that does outperform the market is US senators, raising all kinds of questions about inside information. See Silver, Nate, The Signal and the Noise: The Art and Science of Prediction, Penguin Books, London, 2013 14 See The Economist, 17 March 2018, p.39 15 I’m indebted to David Kedmey for pointing this out 16 ‘The Risk of Automation for Jobs in OECD Countries’, www.oecd-ilibrary.org/­social-issues-migration-health/­the-risk-of-automation-for-jobs-in-oecd-countries_5jlz9h56dvq7-en, accessed 14 October 2018 and 27 October 2019 17 ‘5 Things to Know about the Future of Jobs’, www.weforum.org/­agenda/­2018/­09/­future-of-jobs-2018-things-to-know/ 18 Google’s Chris Urmson suggested that the autonomous vehicle ‘has the potential to reduce current Federal spending pressures for roadways, parking and public transit’.

How I Built This: The Unexpected Paths to Success From the World's Most Inspiring Entrepreneurs by Guy Raz

Airbnb, AOL-Time Warner, Apple II, barriers to entry, Bear Stearns, Ben Horowitz, Big Tech, big-box store, Black Monday: stock market crash in 1987, Blitzscaling, business logic, call centre, Clayton Christensen, commoditize, Cornelius Vanderbilt, Credit Default Swap, crowdsourcing, data science, East Village, El Camino Real, Elon Musk, fear of failure, glass ceiling, growth hacking, housing crisis, imposter syndrome, inventory management, It's morning again in America, iterative process, James Dyson, Jeff Bezos, Justin.tv, Kickstarter, low cost airline, Lyft, Marc Andreessen, Mark Zuckerberg, move fast and break things, Nate Silver, Paul Graham, Peter Thiel, pets.com, power law, rolodex, Ronald Reagan, Ruby on Rails, Salesforce, Sam Altman, Sand Hill Road, side hustle, Silicon Valley, software as a service, South of Market, San Francisco, Steve Jobs, Steve Wozniak, subprime mortgage crisis, TED Talk, The Signal and the Noise by Nate Silver, Tony Hsieh, Uber for X, uber lyft, Y Combinator, Zipcar

But it’s doable, particularly when you are able to build buzz among many possible customers while at the same time engineering word of mouth among your ideal customers. 13 Get Attention, Part 2: Engineering Word of Mouth In 2012, amidst an explosion of social networking platforms, smartphones, and big data, the statistician Nate Silver published a book titled The Signal and the Noise, about the difficulty of developing accurate predictions. Up until that point, “signal-to-noise ratio” was a term used primarily by scientists and engineers to describe the power of a desired bit of information in relationship to its surroundings. After the success of Silver’s book, however, the phrase morphed into convenient shorthand for nonscientists to talk about problems of communication and value and relevance.

The Impossible Climb: Alex Honnold, El Capitan, and the Climbing Life by Mark Synnott

blue-collar work, California gold rush, Google Earth, index fund, Nate Silver, Skype, South China Sea, Steve Jobs, technological singularity, The Signal and the Noise by Nate Silver, trade route, Y2K

But he was more chatty than usual, and a question I’d been pondering came to mind. He had read three books in Taghia. Open, The Push (Tommy’s autobiography that he shared with Alex in real time via thumb drive as he was writing it at the gîte), and The Signal and the Noise by Nate Silver. Somehow, he had also found time to watch at least three seasons of Spartacus. The Signal and the Noise is all about statistical probability and why most predictions fail. In the book, Silver explains what he calls the prediction paradox: “The more humility we have about our ability to make predictions, the more successful we can be in planning for the future.”

Enlightenment Now: The Case for Reason, Science, Humanism, and Progress by Steven Pinker

3D printing, Abraham Maslow, access to a mobile phone, affirmative action, Affordable Care Act / Obamacare, agricultural Revolution, Albert Einstein, Alfred Russel Wallace, Alignment Problem, An Inconvenient Truth, anti-communist, Anton Chekhov, Arthur Eddington, artificial general intelligence, availability heuristic, Ayatollah Khomeini, basic income, Berlin Wall, Bernie Sanders, biodiversity loss, Black Swan, Bonfire of the Vanities, Brexit referendum, business cycle, capital controls, Capital in the Twenty-First Century by Thomas Piketty, carbon footprint, carbon tax, Charlie Hebdo massacre, classic study, clean water, clockwork universe, cognitive bias, cognitive dissonance, Columbine, conceptual framework, confounding variable, correlation does not imply causation, creative destruction, CRISPR, crowdsourcing, cuban missile crisis, Daniel Kahneman / Amos Tversky, dark matter, data science, decarbonisation, degrowth, deindustrialization, dematerialisation, demographic transition, Deng Xiaoping, distributed generation, diversified portfolio, Donald Trump, Doomsday Clock, double helix, Eddington experiment, Edward Jenner, effective altruism, Elon Musk, en.wikipedia.org, end world poverty, endogenous growth, energy transition, European colonialism, experimental subject, Exxon Valdez, facts on the ground, fake news, Fall of the Berlin Wall, first-past-the-post, Flynn Effect, food miles, Francis Fukuyama: the end of history, frictionless, frictionless market, Garrett Hardin, germ theory of disease, Gini coefficient, Great Leap Forward, Hacker Conference 1984, Hans Rosling, hedonic treadmill, helicopter parent, Herbert Marcuse, Herman Kahn, Hobbesian trap, humanitarian revolution, Ignaz Semmelweis: hand washing, income inequality, income per capita, Indoor air pollution, Intergovernmental Panel on Climate Change (IPCC), invention of writing, Jaron Lanier, Joan Didion, job automation, Johannes Kepler, John Snow's cholera map, Kevin Kelly, Khan Academy, knowledge economy, l'esprit de l'escalier, Laplace demon, launch on warning, life extension, long peace, longitudinal study, Louis Pasteur, Mahbub ul Haq, Martin Wolf, mass incarceration, meta-analysis, Michael Shellenberger, microaggression, Mikhail Gorbachev, minimum wage unemployment, moral hazard, mutually assured destruction, Naomi Klein, Nate Silver, Nathan Meyer Rothschild: antibiotics, negative emissions, Nelson Mandela, New Journalism, Norman Mailer, nuclear taboo, nuclear winter, obamacare, ocean acidification, Oklahoma City bombing, open economy, opioid epidemic / opioid crisis, paperclip maximiser, Paris climate accords, Paul Graham, peak oil, Peter Singer: altruism, Peter Thiel, post-truth, power law, precautionary principle, precision agriculture, prediction markets, public intellectual, purchasing power parity, radical life extension, Ralph Nader, randomized controlled trial, Ray Kurzweil, rent control, Republic of Letters, Richard Feynman, road to serfdom, Robert Gordon, Rodney Brooks, rolodex, Ronald Reagan, Rory Sutherland, Saturday Night Live, science of happiness, Scientific racism, Second Machine Age, secular stagnation, self-driving car, sharing economy, Silicon Valley, Silicon Valley ideology, Simon Kuznets, Skype, smart grid, Social Justice Warrior, sovereign wealth fund, sparse data, stem cell, Stephen Hawking, Steve Bannon, Steven Pinker, Stewart Brand, Stuxnet, supervolcano, synthetic biology, tech billionaire, technological determinism, technological singularity, Ted Kaczynski, Ted Nordhaus, TED Talk, The Rise and Fall of American Growth, the scientific method, The Signal and the Noise by Nate Silver, The Spirit Level, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Thomas Kuhn: the structure of scientific revolutions, Thomas Malthus, total factor productivity, Tragedy of the Commons, union organizing, universal basic income, University of East Anglia, Unsafe at Any Speed, Upton Sinclair, uranium enrichment, urban renewal, W. E. B. Du Bois, War on Poverty, We wanted flying cars, instead we got 140 characters, women in the workforce, working poor, World Values Survey, Y2K

Washington: US Census Bureau. https://www.census.gov/prod/2013pubs/p70-136.pdf. Siegel, R., Naishadham, D., & Jemal, A. 2012. Cancer statistics, 2012. CA: A Cancer Journal for Clinicians, 62, 10–29. Sikkink, K. 2017. Evidence for hope: Making human rights work in the 21st century. Princeton, NJ: Princeton University Press. Silver, N. 2015. The signal and the noise: Why so many predictions fail—but some don’t. New York: Penguin. Simon, J. 1981. The ultimate resource. Princeton, NJ: Princeton University Press. Singer, P. 1981/2010. The expanding circle: Ethics and sociobiology. Princeton, NJ: Princeton University Press. Singer, P. 2010. The life you can save: How to do your part to end world poverty.

…

In the American election, voters in the two lowest income brackets voted for Clinton 52–42, as did those who identified “the economy” as the most important issue. A majority of voters in the four highest income brackets voted for Trump, and Trump voters singled out “immigration” and “terrorism,” not “the economy,” as the most important issues.34 The twisted metal has turned up more promising clues. An article by the statistician Nate Silver began, “Sometimes statistical analysis is tricky, and sometimes a finding just jumps off the page.” That finding jumped right off the page and into the article’s headline: “Education, Not Income, Predicted Who Would Vote for Trump.”35 Why should education have mattered so much? Two uninteresting explanations are that the highly educated happen to affiliate with a liberal political tribe, and that education may be a better long-term predictor of economic security than current income.

…

“How many additional countries will report cases of the Ebola virus in the next eight months?”) and having them write down numerical probabilities. Tetlock also avoided the common fallacy of praising or ridiculing a single probabilistic prediction after the fact, as when the poll aggregator Nate Silver of FiveThirtyEight came under fire for giving Donald Trump just a 29 percent chance of winning the 2016 election.45 Since we cannot replay the election thousands of times and count up the number of times that Trump won, the question of whether the prediction was confirmed or disconfirmed is meaningless.

Data Science for Business: What You Need to Know About Data Mining and Data-Analytic Thinking by Foster Provost, Tom Fawcett

Albert Einstein, Amazon Mechanical Turk, Apollo 13, big data - Walmart - Pop Tarts, bioinformatics, business process, call centre, chief data officer, Claude Shannon: information theory, computer vision, conceptual framework, correlation does not imply causation, crowdsourcing, data acquisition, data science, David Brooks, en.wikipedia.org, Erik Brynjolfsson, Gini coefficient, Helicobacter pylori, independent contractor, information retrieval, intangible asset, iterative process, Johann Wolfgang von Goethe, Louis Pasteur, Menlo Park, Nate Silver, Netflix Prize, new economy, p-value, pattern recognition, placebo effect, price discrimination, recommendation engine, Ronald Coase, selection bias, Silicon Valley, Skype, SoftBank, speech recognition, Steve Jobs, supply-chain management, systems thinking, Teledyne, text mining, the long tail, The Signal and the Noise by Nate Silver, Thomas Bayes, transaction costs, WikiLeaks

However, beating a random model may be easy (or may seem easy), so demonstrating superiority to it may not be very interesting or informative. A data scientist will often need to implement an alternative model, usually one that is simple but not simplistic, in order to justify continuing the data mining effort. In Nate Silver’s book on prediction, The Signal and the Noise (2012), he mentions the baseline issue with respect to weather forecasting: There are two basic tests that any weather forecast must pass to demonstrate its merit: It must do better than what meteorologists call persistence: the assumption that the weather will be the same tomorrow (and the next day) as it was today.

…

A mathematical theory of communication. Bell System Technical Journal, 27, 379–423. Shearer, C. (2000). The CRISP-DM model: The new blueprint for data mining. Journal of Data Warehousing, 5(4), 13–22. Shmueli, G. (2010). To explain or to predict?. Statistical Science, 25(3), 289–310. Silver, N. (2012). The Signal and the Noise. The Penguin Press HC. Solove, D. (2006). A taxonomy of privacy. University of Pennsylvania Law Review, 154(3), 477-564. Stein, R. M. (2005). The relationship between default prediction and lending profits: Integrating ROC analysis and loan pricing. Journal of Banking and Finance, 29, 1213–1236.

…

., Chemistry Is Not About Test Tubes: Data Science Versus the Work of the Data Scientist–Chemistry Is Not About Test Tubes: Data Science Versus the Work of the Data Scientist term frequency (TF), Term Frequency–Term Frequency defined, Term Frequency in TFIDF, Combining Them: TFIDF inverse document frequency, combining with, Combining Them: TFIDF values for, Example: Jazz Musicians terms in documents, Representation supervised learning, Supervised Versus Unsupervised Methods unsupervised learning, Supervised Versus Unsupervised Methods weights of, Topic Models Terry, Clark, Example: Jazz Musicians test data, model building and, A General Method for Avoiding Overfitting test sets, Holdout Data and Fitting Graphs testing, holdout, From Holdout Evaluation to Cross-Validation text, Representing and Mining Text as unstructured data, Why Text Is Difficult–Why Text Is Difficult data, Representing and Mining Text fields, varying number of words in, Why Text Is Difficult importance of, Why Text Is Important Jazz musicians example, Example: Jazz Musicians–Example: Jazz Musicians relative dirtiness of, Why Text Is Difficult text processing, Representing and Mining Text text representation task, Representation–Combining Them: TFIDF text representation task, Representation–Combining Them: TFIDF bag of words approach to, Bag of Words data preparation, The Data–The Data data preprocessing, Data Preprocessing–Data Preprocessing defining, The Task–The Task inverse document frequency, Measuring Sparseness: Inverse Document Frequency–Measuring Sparseness: Inverse Document Frequency Jazz musicians example, Example: Jazz Musicians–Example: Jazz Musicians location mining as, Applying Our Fundamental Concepts to a New Problem: Mining Mobile Device Data measuring prevalence in, Term Frequency–Term Frequency measuring sparseness in, Measuring Sparseness: Inverse Document Frequency–Measuring Sparseness: Inverse Document Frequency mining news stories example, Example: Mining News Stories to Predict Stock Price Movement–Results n-gram sequence approach to, N-gram Sequences named entity extraction, Named Entity Extraction–Named Entity Extraction results, interpreting, Results–Results stock price movement example, Example: Mining News Stories to Predict Stock Price Movement–Results term frequency, Term Frequency–Term Frequency TFIDF value and, Combining Them: TFIDF topic models for, Topic Models–Topic Models TFIDF scores (TFIDF values), Data preparation applied to locations, Applying Our Fundamental Concepts to a New Problem: Mining Mobile Device Data text representation task and, Combining Them: TFIDF The Big Bang Theory, Example: Evidence Lifts from Facebook “Likes” The Colbert Report, Example: Evidence Lifts from Facebook “Likes” The Daily Show, Example: Evidence Lifts from Facebook “Likes” The Godfather, Example: Evidence Lifts from Facebook “Likes” The New York Times, Example: Hurricane Frances, What Data Can’t Do: Humans in the Loop, Revisited The Onion, Example: Evidence Lifts from Facebook “Likes” The Road (McCarthy), Term Frequency The Signal and the Noise (Silver), Evaluation, Baseline Performance, and Implications for Investments in Data The Sound of Music (film), Data Reduction, Latent Information, and Movie Recommendation The Stoker (film comedy), Term Frequency The Wizard of Oz (film), Data Reduction, Latent Information, and Movie Recommendation Thomson Reuters Text Research Collection (TRC2), Example: Clustering Business News Stories thresholds and classifiers, Ranking Instead of Classifying–Ranking Instead of Classifying and performance curves, Profit Curves time series (data), The Data Tobermory single malt scotch, Understanding the Results of Clustering tokens, Representation tools, analytic, Holdout Data and Fitting Graphs topic layer, Topic Models topic models for text representation, Topic Models–Topic Models trade secrets, Unique Intellectual Property training data, Introduction to Predictive Modeling: From Correlation to Supervised Segmentation, Models, Induction, and Prediction, Overfitting evaluating, Holdout Data and Fitting Graphs, Flaws in the Big Red Proposal limits on, Bias, Variance, and Ensemble Methods using, From Holdout Evaluation to Cross-Validation, Learning Curves, Summary training sets, Holdout Data and Fitting Graphs transfers, over the wall, Deployment tree induction, Introduction to Predictive Modeling: From Correlation to Supervised Segmentation ensemble methods and, Bias, Variance, and Ensemble Methods learning curves for, Learning Curves limiting, Avoiding Overfitting with Tree Induction logistic regression vs., Example: Logistic Regression versus Tree Induction–Example: Logistic Regression versus Tree Induction of supervised segmentation, Supervised Segmentation with Tree-Structured Models–Supervised Segmentation with Tree-Structured Models overfitting and, Overfitting in Tree Induction–Overfitting in Tree Induction, Avoiding Overfitting with Tree Induction–Avoiding Overfitting with Tree Induction problems with, Avoiding Overfitting with Tree Induction Tree of Life (Sugden et al; Pennisi), Hierarchical Clustering tree-structured models classification, Supervised Segmentation with Tree-Structured Models creating, Supervised Segmentation with Tree-Structured Models decision, Supervised Segmentation with Tree-Structured Models for supervised segmentation, Supervised Segmentation with Tree-Structured Models–Supervised Segmentation with Tree-Structured Models goals, Supervised Segmentation with Tree-Structured Models probability estimation, Supervised Segmentation with Tree-Structured Models, Probability Estimation pruning, Avoiding Overfitting with Tree Induction regression, Supervised Segmentation with Tree-Structured Models restricting, Overfitting in Tree Induction tri-grams, N-gram Sequences Tron, Example: Evidence Lifts from Facebook “Likes” true negative rate, Costs and benefits true negatives, Costs and benefits true positive rate, Costs and benefits, ROC Graphs and Curves–ROC Graphs and Curves, Cumulative Response and Lift Curves true positives, Costs and benefits Tullibardine single malt whiskey, Hierarchical Clustering Tumblr, online consumer targeting by, Example: Targeting Online Consumers With Advertisements Twitter, Why Text Is Important Two Dogmas of Empiricism (Quine), What Data Can’t Do: Humans in the Loop, Revisited U UCI Dataset Repository, An Example of Mining a Linear Discriminant from Data–Support Vector Machines, Briefly unconditional independence, conditional vs., Conditional Independence and Naive Bayes unconditional probability of hypothesis and evidence, Bayes’ Rule prior probability based on, Applying Bayes’ Rule to Data Science unique context, of strategic decisions, What Data Can’t Do: Humans in the Loop, Revisited University of California at Irvine, Example: Attribute Selection with Information Gain, Example: Logistic Regression versus Tree Induction University of Montréal, Example: Whiskey Analytics University of Toronto, Privacy, Ethics, and Mining Data About Individuals unstructured data, Why Text Is Difficult unstructured data, text as, Why Text Is Difficult–Why Text Is Difficult unsupervised learning, Supervised Versus Unsupervised Methods unsupervised methods of data mining, supervised vs., Supervised Versus Unsupervised Methods–Supervised Versus Unsupervised Methods unsupervised problems, Stepping Back: Solving a Business Problem Versus Data Exploration unsupervised segmentation, Stepping Back: Solving a Business Problem Versus Data Exploration user-generated content, Why Text Is Important V value (worth), adding, to applications, Decision Analytic Thinking I: What Is a Good Model?

Robot Rules: Regulating Artificial Intelligence by Jacob Turner

"World Economic Forum" Davos, Ada Lovelace, Affordable Care Act / Obamacare, AI winter, algorithmic bias, algorithmic trading, AlphaGo, artificial general intelligence, Asilomar, Asilomar Conference on Recombinant DNA, autonomous vehicles, backpropagation, Basel III, bitcoin, Black Monday: stock market crash in 1987, blockchain, brain emulation, Brexit referendum, Cambridge Analytica, Charles Babbage, Clapham omnibus, cognitive dissonance, Computing Machinery and Intelligence, corporate governance, corporate social responsibility, correlation does not imply causation, crowdsourcing, data science, deep learning, DeepMind, Demis Hassabis, distributed ledger, don't be evil, Donald Trump, driverless car, easy for humans, difficult for computers, effective altruism, Elon Musk, financial exclusion, financial innovation, friendly fire, future of work, hallucination problem, hive mind, Internet of things, iterative process, job automation, John Markoff, John von Neumann, Loebner Prize, machine readable, machine translation, medical malpractice, Nate Silver, natural language processing, Nick Bostrom, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, nudge unit, obamacare, off grid, OpenAI, paperclip maximiser, pattern recognition, Peace of Westphalia, Philippa Foot, race to the bottom, Ray Kurzweil, Recombinant DNA, Rodney Brooks, self-driving car, Silicon Valley, Stanislav Petrov, Stephen Hawking, Steve Wozniak, strong AI, technological singularity, Tesla Model S, The Coming Technological Singularity, The Future of Employment, The Signal and the Noise by Nate Silver, trolley problem, Turing test, Vernor Vinge

See Anna-Louise Taylor, “Why Infanticide Can Benefit Animals”, BBC Nature, 21 March 2012, http://​www.​bbc.​co.​uk/​nature/​18035811, accessed 1 June 2018. 100For proposals along these lines, see Oren Etzioni, “How to Regulate Artificial Intelligence”, The New York Times, 1 September 2017, https://​www.​nytimes.​com/​2017/​09/​01/​opinion/​artificial-intelligence-regulations-rules.​html, accessed 1 June 2018. 101Director of Public Prosecutions, “Suicide: Policy for Prosecutors in Respect of Cases of Encouraging or Assisting Suicide”, February 2010, updated October 2014, https://​www.​cps.​gov.​uk/​legal-guidance/​suicide-policy-prosecutors-respect-cases-encouraging-or-assisting-suicide, accessed 1 June 2018. 102As we explore in later chapters, the theoretical ability for AI to avoid human bias does not obviate the need to ensure that those humans originally programming AI or providing their seed data sets do not accidentally or intentionally imbue AI with human fallibilities or prejudice. 103Luciano Floridi, “A Fallacy that Will Hinder Advances in Artificial Intelligence”, The Financial Times, 1 June 2017, https://​www.​ft.​com/​content/​ee996846-4626-11e7-8d27-59b4dd6296b8, accessed 1 June 2018. See also Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (London: Penguin, 2012), 287–288. 104Philippa Foot, The Problem of Abortion and the Doctrine of the Double Effect in Virtues and Vices (Oxford: Basil Blackwell, 1978) (the article originally appeared in the Oxford Review, Number 5, 1967). 105See Judith Jarvis Thompson, “The Trolley Problem”, Yale Law Journal, Vol. 94, No. 6 (May, 1985), 1395–1415. 106In this book, the terms “self-driving” and “autonomous” when used in relation to vehicles refer to the delegation by humans of certain decision-making functions featuring in driving.

Stress Test: Reflections on Financial Crises by Timothy F. Geithner

Affordable Care Act / Obamacare, Alan Greenspan, asset-backed security, Atul Gawande, bank run, banking crisis, Basel III, Bear Stearns, Bernie Madoff, Bernie Sanders, Black Monday: stock market crash in 1987, break the buck, Buckminster Fuller, Carmen Reinhart, central bank independence, collateralized debt obligation, correlation does not imply causation, creative destruction, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, currency manipulation / currency intervention, currency risk, David Brooks, Doomsday Book, eurozone crisis, fear index, financial engineering, financial innovation, Flash crash, Goldman Sachs: Vampire Squid, Greenspan put, housing crisis, Hyman Minsky, illegal immigration, implied volatility, Kickstarter, London Interbank Offered Rate, Long Term Capital Management, low interest rates, margin call, market fundamentalism, Martin Wolf, McMansion, Mexican peso crisis / tequila crisis, money market fund, moral hazard, mortgage debt, Nate Silver, negative equity, Northern Rock, obamacare, paradox of thrift, pets.com, price stability, profit maximization, proprietary trading, pushing on a string, quantitative easing, race to the bottom, RAND corporation, regulatory arbitrage, reserve currency, Saturday Night Live, Savings and loan crisis, savings glut, selection bias, Sheryl Sandberg, short selling, sovereign wealth fund, stock buybacks, tail risk, The Great Moderation, The Signal and the Noise by Nate Silver, Tobin tax, too big to fail, working poor

Their modest returns were still more than the cost of financing them, and they seemed almost bulletproof. Standard & Poor’s estimated a mere 0.12 percent chance that one of its AAA-rated CDOs would fail to pay out over five years—and super-seniors were considered safer than typical AAAs. But as Nate Silver noted in The Signal and the Noise, his excellent book about why many predictions fail, the actual default rate for AAA-rated tranches of CDOs would be 28 percent, more than two hundred times higher than S&P had predicted. Their perceived safety rested on all kinds of flawed assumptions, starting with the notion that housing prices would never fall simultaneously across the country.

Four Battlegrounds by Paul Scharre

2021 United States Capitol attack, 3D printing, active measures, activist lawyer, AI winter, AlphaGo, amateurs talk tactics, professionals talk logistics, artificial general intelligence, ASML, augmented reality, Automated Insights, autonomous vehicles, barriers to entry, Berlin Wall, Big Tech, bitcoin, Black Lives Matter, Boeing 737 MAX, Boris Johnson, Brexit referendum, business continuity plan, business process, carbon footprint, chief data officer, Citizen Lab, clean water, cloud computing, commoditize, computer vision, coronavirus, COVID-19, crisis actor, crowdsourcing, DALL-E, data is not the new oil, data is the new oil, data science, deep learning, deepfake, DeepMind, Demis Hassabis, Deng Xiaoping, digital map, digital rights, disinformation, Donald Trump, drone strike, dual-use technology, Elon Musk, en.wikipedia.org, endowment effect, fake news, Francis Fukuyama: the end of history, future of journalism, future of work, game design, general purpose technology, Geoffrey Hinton, geopolitical risk, George Floyd, global supply chain, GPT-3, Great Leap Forward, hive mind, hustle culture, ImageNet competition, immigration reform, income per capita, interchangeable parts, Internet Archive, Internet of things, iterative process, Jeff Bezos, job automation, Kevin Kelly, Kevin Roose, large language model, lockdown, Mark Zuckerberg, military-industrial complex, move fast and break things, Nate Silver, natural language processing, new economy, Nick Bostrom, one-China policy, Open Library, OpenAI, PalmPilot, Parler "social media", pattern recognition, phenotype, post-truth, purchasing power parity, QAnon, QR code, race to the bottom, RAND corporation, recommendation engine, reshoring, ride hailing / ride sharing, robotic process automation, Rodney Brooks, Rubik’s Cube, self-driving car, Shoshana Zuboff, side project, Silicon Valley, slashdot, smart cities, smart meter, Snapchat, social software, sorting algorithm, South China Sea, sparse data, speech recognition, Steve Bannon, Steven Levy, Stuxnet, supply-chain attack, surveillance capitalism, systems thinking, tech worker, techlash, telemarketer, The Brussels Effect, The Signal and the Noise by Nate Silver, TikTok, trade route, TSMC

Marine Corps, MCDP 1: Warfighting, 10. 274“I kind of felt powerless”: Christopher Moyer, “How Google’s AlphaGo Beat a Go World Champion,” The Atlantic, March 28, 2016, https://www.theatlantic.com/technology/archive/2016/03/the-invisible-opponent/475611/. 275AI “cannot be defeated”: “(Yonhap Interview) Go Master Lee Says He Quits Unable to Win over AI Go Players,” Yonhap News Agency, November 27, 2019, https://en.yna.co.kr/view/AEN20191127004800315. 275overestimation of Deep Blue’s abilities: Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t (New York: Penguin Books, February 3, 2015), 277–289. 33. BATTLEFIELD SINGULARITY 276Battlefield Singularity: The chapter title comes from the title of Elsa Kania’s report, Battlefield Singularity. Elsa B. Kania, Battlefield Singularity: Artificial Intelligence, Military Revolution, and China’s Future Military Power (Center for a New American Security, November 28, 2017), https://www.cnas.org/publications/reports/battlefield-singularity-artificial-intelligence-military-revolution-and-chinas-future-military-power. 276higher rate of fire: Stephen Biddle, Military Power: Explaining Victory and Defeat in Modern Battle (Princeton, NJ: Princeton University Press, 2004), 29, https://www.amazon.com/Military-Power-Explaining-Victory-Defeat/dp/0691128022. 277Battle of the Somme: Wikipedia, s.v.