Nate Silver

86 results back to index


Deep Work: Rules for Focused Success in a Distracted World by Cal Newport

8-hour work day, Albert Einstein, barriers to entry, business climate, Cal Newport, Capital in the Twenty-First Century by Thomas Piketty, Clayton Christensen, David Brooks, David Heinemeier Hansson, deliberate practice, disruptive innovation, Donald Knuth, Donald Trump, Downton Abbey, en.wikipedia.org, Erik Brynjolfsson, experimental subject, follow your passion, Frank Gehry, informal economy, information retrieval, Internet Archive, Jaron Lanier, knowledge worker, Mark Zuckerberg, Marshall McLuhan, Merlin Mann, Nate Silver, new economy, Nicholas Carr, popular electronics, remote working, Richard Feynman, Ruby on Rails, Silicon Valley, Silicon Valley startup, Snapchat, statistical model, the medium is the message, Watson beat the top human players on Jeopardy!, web application, winner-take-all economy, zero-sum game

Chapter 1 Information about Nate Silver’s election traffic on the New York Times website: Tracy, Marc. “Nate Silver Is a One-Man Traffic Machine for the Times.” New Republic, November 6, 2012. http://www.newrepublic.com/article/109714/nate-silvers-fivethirtyeight-blog-drawing-massive-traffic-new-york-times. Information about Nate Silver’s ESPN/ABC News deal: Allen, Mike. “How ESPN and ABC Landed Nate Silver.” Politico, July 22, 2013. http://www.politico.com/blogs/media/2013/07/how-espn-and-abc-landed-nate-silver-168888.html. Examples of concerns regarding Silver’s methodology: Davis, Sean M. “Is Nate Silver’s Value at Risk?” Daily Caller, November 1, 2012. http://dailycaller.com/2012/11/01/is-nate-silvers-value-at-risk/. Marcus, Gary, and Ernest Davis. “What Nate Silver Gets Wrong.” The New Yorker, January 25, 2013. http://www.newyorker.com/online/blogs/books/2013/01/what-nate-silver-gets-wrong.html.

The American Economic Review 71.5 (December 1981): 845–858. “Hearing a succession of mediocre singers does not add up to a single outstanding performance”: Ibid., 846. The Instagram example and its significance for labor disparities were first brought to my attention by the writing/speaking of Jaron Lanier. How to Become a Winner in the New Economy Details on Nate Silver’s tools: • Hickey, Walter. “How to Become Nate Silver in 9 Simple Steps.” Business Insider, November 14, 2012. http://www.businessinsider.com/how-nate-silver-and-fivethityeight-works-2012-11. • Silver, Nate. “IAmA Blogger for FiveThirtyEight at The New York Times. Ask Me Anything.” Reddit. http://www.reddit.com/r/IAmA/comments/166yeo/iama_blogger_for_fivethirtyeight_at_the_new_york. • “Why Use Stata.” www.stata.com/why-use-stata/. The SQL example I gave was from postgreSQL, an open source database system popular in both industry and (especially) academia.

The High-Skilled Workers Brynjolfsson and McAfee call the group personified by Nate Silver the “high-skilled” workers. Advances such as robotics and voice recognition are automating many low-skilled positions, but as these economists emphasize, “other technologies like data visualization, analytics, high speed communications, and rapid prototyping have augmented the contributions of more abstract and data-driven reasoning, increasing the values of these jobs.” In other words, those with the oracular ability to work with and tease valuable results out of increasingly complex machines will thrive. Tyler Cowen summarizes this reality more bluntly: “The key question will be: are you good at working with intelligent machines or not?” Nate Silver, of course, with his comfort in feeding data into large databases, then siphoning it out into his mysterious Monte Carlo simulations, is the epitome of the high-skilled worker.


pages: 276 words: 81,153

Outnumbered: From Facebook and Google to Fake News and Filter-Bubbles – the Algorithms That Control Our Lives by David Sumpter

affirmative action, Bernie Sanders, correlation does not imply causation, crowdsourcing, don't be evil, Donald Trump, Elon Musk, Filter Bubble, Google Glasses, illegal immigration, Jeff Bezos, job automation, Kenneth Arrow, Loebner Prize, Mark Zuckerberg, meta analysis, meta-analysis, Minecraft, Nate Silver, natural language processing, Nelson Mandela, p-value, prediction markets, random walk, Ray Kurzweil, Robert Mercer, selection bias, self-driving car, Silicon Valley, Skype, Snapchat, speech recognition, statistical model, Stephen Hawking, Steven Pinker, The Signal and the Noise by Nate Silver, traveling salesman, Turing test

There has been a shift from newspapers reporting opinion polls to online political sites, like FiveThirtyEight run by Nate Silver, and The Upshot at the New York Times, making probabilistic predictions of outcomes. As we have seen, algorithms work in terms of probabilities and not in binary outcomes. Poll predictions are no exception. Just as no reasonable algorithm would declare that a person is certain to commit a crime, or go to Portugal for their next summer holiday, neither would an algorithm, even one designed for the Huffington Post, declare that Clinton would win with 100 per cent certainty. One challenge for the creators of these poll-based prediction models is that us humans keep flipping their probabilistic predictions to binary: ‘yes’ or ‘no’, ‘Brexit’ or ‘remain’ and ‘Trump’ or ‘Clinton’. Our lazy minds like certainty. After the 2012 US presidential election, when Nate Silver’s model predicted all US states correctly, he was declared a genius in blogs and across social media.

When the victory failed to materialise, the New York Times published an article (headlined: ‘How data failed us in calling an election’) that proclaimed the number crunchers had had a rough night.3 It listed supposed problems in both their own model (the newspaper’s Upshot model had given Clinton a 91 per cent chance of winning) and the approach taken by Nate Silver and FiveThirtyEight. The newspaper was blaming statisticians for its own inability to account for uncertainty. For Silver, this was just one example of how the media finds it very difficult to write sensible articles based on probabilistic reasoning.4 What struck me, looking at how FiveThirtyEight had evolved over the past 10 years, was that the site provides a powerful case study of the limits of mathematical models. Nate Silver had been propelled to a position of authority. He had accumulated financial resources (FiveThirtyEight is owned by ESPN) that had allowed him to build sophisticated models based on large quantities of reliable data.

He knew the maths and he understood the relationship between data and the real world. If anyone could create a good model of an election, it was Nate Silver. When it came to analysing our behaviour, the algorithms we have looked at up to now were, at best, on a par with humans. Julia Dressel’s Mechanical Turkers were able to predict probability of reoffending to a level similar to that of a state-of-the-art algorithm, but using a lot less data; personality models based on likes were still a long way from ‘knowing us’ as individuals and Spotify was struggling to come up with music recommendations as good as those of our friends. I wondered whether the same limitations applied to the FiveThirtyEight model? Given the resources at Nate Silver’s disposal, FiveThirtyEight was the undisputed heavyweight champion of algorithmic prediction. I wanted to find out whether a human contender could take it on.


pages: 829 words: 186,976

The Signal and the Noise: Why So Many Predictions Fail-But Some Don't by Nate Silver

"Robert Solow", airport security, availability heuristic, Bayesian statistics, Benoit Mandelbrot, Berlin Wall, Bernie Madoff, big-box store, Black Swan, Broken windows theory, business cycle, buy and hold, Carmen Reinhart, Claude Shannon: information theory, Climategate, Climatic Research Unit, cognitive dissonance, collapse of Lehman Brothers, collateralized debt obligation, complexity theory, computer age, correlation does not imply causation, Credit Default Swap, credit default swaps / collateralized debt obligations, cuban missile crisis, Daniel Kahneman / Amos Tversky, diversification, Donald Trump, Edmond Halley, Edward Lorenz: Chaos theory, en.wikipedia.org, equity premium, Eugene Fama: efficient market hypothesis, everywhere but in the productivity statistics, fear of failure, Fellow of the Royal Society, Freestyle chess, fudge factor, George Akerlof, global pandemic, haute cuisine, Henri Poincaré, high batting average, housing crisis, income per capita, index fund, information asymmetry, Intergovernmental Panel on Climate Change (IPCC), Internet Archive, invention of the printing press, invisible hand, Isaac Newton, James Watt: steam engine, John Nash: game theory, John von Neumann, Kenneth Rogoff, knowledge economy, Laplace demon, locking in a profit, Loma Prieta earthquake, market bubble, Mikhail Gorbachev, Moneyball by Michael Lewis explains big data, Monroe Doctrine, mortgage debt, Nate Silver, negative equity, new economy, Norbert Wiener, PageRank, pattern recognition, pets.com, Pierre-Simon Laplace, prediction markets, Productivity paradox, random walk, Richard Thaler, Robert Shiller, Robert Shiller, Rodney Brooks, Ronald Reagan, Saturday Night Live, savings glut, security theater, short selling, Skype, statistical model, Steven Pinker, The Great Moderation, The Market for Lemons, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, too big to fail, transaction costs, transfer pricing, University of East Anglia, Watson beat the top human players on Jeopardy!, wikimedia commons

“Election Results: House Big Board,” New York Times, November 2, 2010. http://elections.nytimes.com/2010/results/house/big-board. 26. Nate Silver, “A Warning on the Accuracy of Primary Polls,” FiveThirtyEight, New York Times, March 1, 2012. http://fivethirtyeight.blogs.nytimes.com/2012/03/01/a-warning-on-the-accuracy-of-primary-polls/. 27. Nate Silver, “Bill Buckner Strikes Again,” FiveThirtyEight, New York Times; September 29, 2011. http://fivethirtyeight.blogs.nytimes.com/2011/09/29/bill-buckner-strikes-again/. 28. Otherwise, you should have assigned the congressman a 100 percent chance of victory instead. 29. Matthew Dickinson, “Nate Silver Is Not a Political Scientist,” in Presidential Power: A Nonpartisan Analysis of Presidential Power, Blogs Dot Middlebury, November 1, 2010. http://blogs.middlebury.edu/presidentialpower/2010/11/01/nate-silver-is-not-a-political-scientist/. 30.

Alan Schwarz, “The Great Debate,” Baseball America, January. 7, 2005. http://www.baseballamerica.com/today/features/050107debate.html. 20. Per interview with Billy Beane. 21. Nate Silver, “What Tim Geithner Can Learn from Baseball,” Esquire, March 11, 2009. http://www.esquire.com/features/data/mlb-player-salaries-0409. 22. As a result of my original agreement in 2003 and a subsequent agreement in 2009, Baseball Prospectus now fully owns and operates PECOTA. Beginning with the 2010 season, the PECOTA forecasts reflect certain changes, improvements, and departures from my original methodology. The methods I describe herein apply to the 2003–2009 version of PECOTA specifically. 23. Nate Silver, “PECOTA Takes on the Field,” Baseball Prospectus, January 16, 2004. http://www.baseballprospectus.com/article.php?articleid=2515. 24. Nate Silver, “Lies, Damned Lies: Projection Reflection,” Baseball Prospectus, October 11, 2006. http://www.baseballprospectus.com/article.php?

Stephen’s Green, Dublin 2, Ireland (a division of Penguin Books Ltd) • Penguin Books Australia Ltd, 250 Camberwell Road, Camberwell, Victoria 3124, Australia (a division of Pearson Australia Group Pty Ltd) • Penguin Books India Pvt Ltd, 11 Community Centre, Panchsheel Park, New Delhi – 110 017, India • Penguin Group (NZ), 67 Apollo Drive, Rosedale, Auckland 0632, New Zealand (a division of Pearson New Zealand Ltd) • Penguin Books (South Africa) (Pty) Ltd, 24 Sturdee Avenue, Rosebank, Johannesburg 2196, South Africa Penguin Books Ltd, Registered Offices: 80 Strand, London WC2R 0RL, England First published in 2012 by The Penguin Press, a member of Penguin Group (USA) Inc. Copyright © Nate Silver, 2012 All rights reserved Illustration credits Figure 4-2: Courtesy of Dr. Tim Parker, University of Oxford Figure 7-1: From “1918 Influenza: The Mother of All Pandemics” by Jeffery Taubenberger and David Morens, Emerging Infectious Disease Journal, vol. 12, no. 1, January 2006, Centers for Disease Control and Prevention Figures 9-2, 9-3A, 9-3C, 9-4, 9-5, 9-6 and 9-7: By Cburnett, Wikimedia Commons Figure 12-2: Courtesy of Dr. J. Scott Armstrong, The Wharton School, University of Pennsylvania LIBRARY OF CONGRESS CATALOGING IN PUBLICATION DATA Silver, Nate. The signal and the noise : why most predictions fail but some don’t / Nate Silver. p. cm. Includes bibliographical references and index.


pages: 288 words: 81,253

Thinking in Bets by Annie Duke

banking crisis, Bernie Madoff, Cass Sunstein, cognitive bias, cognitive dissonance, Daniel Kahneman / Amos Tversky, delayed gratification, Donald Trump, en.wikipedia.org, endowment effect, Estimating the Reproducibility of Psychological Science, Filter Bubble, hindsight bias, Jean Tirole, John Nash: game theory, John von Neumann, loss aversion, market design, mutually assured destruction, Nate Silver, p-value, phenotype, prediction markets, Richard Feynman, ride hailing / ride sharing, Stanford marshmallow experiment, Stephen Hawking, Steven Pinker, the scientific method, The Signal and the Noise by Nate Silver, urban planning, Walter Mischel, Yogi Berra, zero-sum game

Mistaking Odds for Wrong When the Underdog Wins,” Huffington Post, September 21, 2016, http://www.huffingtonpost.com/annie-duke/even-dershowitz-mistaking_b_12120592.html. Nate Silver and his website, FiveThirtyEight.com, bore the brunt of the criticism for pollsters and forecasters after the 2016 presidential election. Silver’s site updated, in real time, polling and forecasting data on the election and had (depending on the date) the probability of a Clinton victory at approximately 60%–70%. If you Google (without the quotation marks) “Nate Silver got it wrong election,” 465,000 results come up. Politico’s November 9 headline was “How Did Everyone Get It So Wrong?,” http://www.politico.com/story/2016/11/how-did-everyone-get-2016-wrong-presidential-election-231036. Gizmodo.com jumped on Silver even before the election, in a November 4 article by Matt Novak titled “Nate Silver’s Very Very Wrong Predictions About Donald Trump Are Terrifying,” http://paleofuture.gizmodo.com/nate-silvers-very-very-wrong-predictions-about-donald-t-1788583912, including the declaration, “Silver has no f**king idea.”

Reaction to the 2016 election provides another strong demonstration of what happens when we lop branches off the tree. Hillary Clinton had been favored going into the election, and her probability of winning, based on an accumulation of the polls, was somewhere between 60% and 70%, according to FiveThirtyEight.com. When Donald Trump won, pollsters got the Pete Carroll treatment, maybe no one more than Nate Silver, founder of FiveThirtyEight.com and a thoughtful analyzer of polling data. (“Nate Silver was wrong.” “The pollsters missed it.” “Just like Brexit, the bookies blew it.” Etc.) The press spun this as a certain win for Clinton, despite the Trump branch of the tree being no mere twig at 30%–40%. By the day after the election, the Clinton branch had been severed, only the Trump branch remained, and how could pollsters and the polling process have been so blind?

When the 24% result happened at the final table of the charity tournament, that didn’t reflect inaccuracy about the probabilities as determined before that single outcome. Long shots hit some of the time. Blaming the oddsmakers or the odds themselves assumes that once something happens, it was bound to have happened and anyone who didn’t see it coming was wrong. The same thing happened after Donald Trump won the presidency. There was a huge outcry about the polls being wrong. Nate Silver, the founder of FiveThirtyEight.com, drew a lot of that criticism. But he never said Clinton was a sure thing. Based on his aggregation and weighting of polling data, he had Trump between 30% and 40% to win (approximately between two-to-one and three-to-two against) in the week before the election. An event predicted to happen 30% to 40% of the time will happen a lot. Being a poker player, I’ve played out more two-to-one shots in my tournament career than I could possibly count.


pages: 337 words: 86,320

Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are by Seth Stephens-Davidowitz

affirmative action, AltaVista, Amazon Mechanical Turk, Asian financial crisis, Bernie Sanders, big data - Walmart - Pop Tarts, Cass Sunstein, computer vision, correlation does not imply causation, crowdsourcing, Daniel Kahneman / Amos Tversky, desegregation, Donald Trump, Edward Glaeser, Filter Bubble, game design, happiness index / gross national happiness, income inequality, Jeff Bezos, John Snow's cholera map, longitudinal study, Mark Zuckerberg, Nate Silver, peer-to-peer lending, Peter Thiel, price discrimination, quantitative hedge fund, Ronald Reagan, Rosa Parks, sentiment analysis, Silicon Valley, statistical model, Steve Jobs, Steven Levy, Steven Pinker, TaskRabbit, The Signal and the Noise by Nate Silver, working poor

Our research suggests that a person is significantly more likely to put the candidate they support first in a search that includes both candidates’ names. In the previous three elections, the candidate who appeared first in more searches received the most votes. More interesting, the order the candidates were searched was predictive of which way a particular state would go. The order in which candidates are searched also seems to contain information that the polls can miss. In the 2012 election between Obama and Republican Mitt Romney, Nate Silver, the virtuoso statistician and journalist, accurately predicted the result in all fifty states. However, we found that in states that listed Romney before Obama in searches most frequently, Romney actually did better than Silver had predicted. In states that most frequently listed Obama before Romney, Obama did better than Silver had predicted. This indicator could contain information that polls miss because voters are either lying to themselves or uncomfortable revealing their true preferences to pollsters.

Eight years later, they elected as president Donald J. Trump, a man who retweeted a false claim that black people are responsible for the majority of murders of white Americans, defended his supporters for roughing up a Black Lives Matters protester at one of his rallies, and hesitated in repudiating support from a former leader of the Ku Klux Klan. The same hidden racism that hurt Barack Obama helped Donald Trump. Early in the primaries, Nate Silver famously claimed that there was virtually no chance that Trump would win. As the primaries progressed and it became increasingly clear that Trump had widespread support, Silver decided to look at the data to see if he could understand what was going on. How could Trump possibly be doing so well? Silver noticed that the areas where Trump performed best made for an odd map. Trump performed well in parts of the Northeast and industrial Midwest, as well as the South.

It explains, as discussed earlier, why Obama’s vote totals in 2008 and 2012 were depressed in many regions. It also correlates with the black-white wage gap, as a team of economists recently reported. The areas that I had found make the most racist searches, in other words, underpay black people. And then there is the phenomenon of Donald Trump’s candidacy. As noted in the introduction, when Nate Silver, the polling guru, looked for the geographic variable that correlated most strongly with support in the 2016 Republican primary for Trump, he found it in the map of racism I had developed. That variable was searches for “nigger(s).” Scholars have recently put together a state-by-state measure of implicit prejudice against black people, which has enabled me to compare the effects of explicit racism, as measured by Google searches, and implicit bias.


pages: 366 words: 76,476

Dataclysm: Who We Are (When We Think No One's Looking) by Christian Rudder

4chan, Affordable Care Act / Obamacare, bitcoin, cloud computing, correlation does not imply causation, crowdsourcing, cuban missile crisis, Donald Trump, Edward Snowden, en.wikipedia.org, Frank Gehry, Howard Zinn, Jaron Lanier, John Markoff, John Snow's cholera map, lifelogging, Mahatma Gandhi, Mikhail Gorbachev, Nate Silver, Nelson Mandela, new economy, obamacare, Occupy movement, p-value, pre–internet, race to the bottom, selection bias, Snapchat, social graph, Solar eclipse in 1919, Steve Jobs, the scientific method

If men from such different environments as Mississippi and Massachusetts are looking for gay porn at equal rates, that’s strong evidence that supposed external forces have little effect on same-sex attraction. The second implication of the state-by-state sameness in the data—that is, what it reveals not so much about gay people but about intolerance—needs a little time to unfurl. In early 2013, when he was still covering politics for the Times, Nate Silver applied his famous poll-modeling technique to same-sex-marriage ballot initiatives across the country. As he had done in the presidential elections, he aggregated data to get a snapshot of public opinion in each state, and then he performed some forward-looking analysis to guess how those attitudes might evolve. Silver estimated that gay marriage will be legal in forty-four states by 2020. An interesting thing about Silver’s work on the question, which was based on political polls, is how it relates to another data source: what people in each state told Gallup about their own sexuality.

On the one hand, this is a testament to the strength of home ties, upbringing, and simple inertia. On the other, it means that for every person picking up and moving to a San Francisco or a New York City to live life fully, there are likely dozens still living in self-negation. If you accept these two independent estimates of 5 percent, arrived at using three of the biggest forces in modern data—Nate Silver, Google, and Facebook, with an assist from that standby of old-school polling, Gallup—you begin to see those self-reported numbers in a different light. When Gallup tells us that, for example, 1.7 percent of North Dakotans are gay, then perhaps something like 3.3 percent of the state is gay and unwilling to acknowledge it. In New York, about 4 percent of the population is openly gay, leaving maybe 1 percent gay and silent.

most typical words for … bisexual women bi female bisexual female me and my husband me and my man my boyfriend is hubby and we are a couple i am bisexual and me and my boyfriend fun couple couple we married couple we are not looking fun with me and do have a boyfriend my bf and female to join girl to join another couple bi woman my boyfriend my i am bi sexual my hubby and join me and my female for my boyfriend and i we are looking to a triad no single send us If I could put this to a beat and get Pitbull to do the middle eight, it would go straight to number one. That said, for all the crassness of sexual-identity-as-business-plan, it’s a hopeful sign when a minority identity is something the mainstream thinks is worth co-opting instead of suppressing. Indeed, for sexuality, we see that things are changing, and quickly. Devising the projections we looked at above, Nate Silver clocked a marked change in American attitudes in the last decade. Acceptance of gay marriage accelerated markedly in 2004—and he determined, “One no longer needs to make optimistic assumptions to conclude that same-sex marriage supporters will probably soon constitute a national majority.” Thus, it all comes back to counting, and the fraction is going our way. Though people have been gay forever, in the late nineteenth century, people began to “self-disclose” their homosexuality as a political act.


pages: 317 words: 100,414

Superforecasting: The Art and Science of Prediction by Philip Tetlock, Dan Gardner

Affordable Care Act / Obamacare, Any sufficiently advanced technology is indistinguishable from magic, availability heuristic, Black Swan, butterfly effect, buy and hold, cloud computing, cuban missile crisis, Daniel Kahneman / Amos Tversky, desegregation, drone strike, Edward Lorenz: Chaos theory, forward guidance, Freestyle chess, fundamental attribution error, germ theory of disease, hindsight bias, index fund, Jane Jacobs, Jeff Bezos, Kenneth Arrow, Laplace demon, longitudinal study, Mikhail Gorbachev, Mohammed Bouazizi, Nash equilibrium, Nate Silver, Nelson Mandela, obamacare, pattern recognition, performance metric, Pierre-Simon Laplace, place-making, placebo effect, prediction markets, quantitative easing, random walk, randomized controlled trial, Richard Feynman, Richard Thaler, Robert Shiller, Robert Shiller, Ronald Reagan, Saturday Night Live, scientific worldview, Silicon Valley, Skype, statistical model, stem cell, Steve Ballmer, Steve Jobs, Steven Pinker, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Watson beat the top human players on Jeopardy!

They have no idea how good their forecasts are in the short, medium, or long term—and no idea how good their forecasts could become. At best, they have vague hunches. That’s because the forecast-measure-revise procedure operates only within the rarefied confines of high-tech forecasting, such as the work of macroeconomists at central banks or marketing and financial professionals in big companies or opinion poll analysts like Nate Silver.7 More often forecasts are made and then … nothing. Accuracy is seldom determined after the fact and is almost never done with sufficient regularity and rigor that conclusions can be drawn. The reason? Mostly it’s a demand-side problem: The consumers of forecasting—governments, business, and the public—don’t demand evidence of accuracy. So there is no measurement. Which means no revision. And without revision, there can be no improvement.

Consider the weather in Phoenix, Arizona. Each June, it gets very hot and sunny. A forecaster who followed a mindless rule like, “always assign 100% to hot and sunny” could get a Brier score close to 0, leaving 0.2 in the dust. Here, the right test of skill would be whether a forecaster can do better than mindlessly predicting no change. This is an underappreciated point. For example, after the 2012 presidential election, Nate Silver, Princeton’s Sam Wang, and other poll aggregators were hailed for correctly predicting all fifty state outcomes, but almost no one noted that a crude, across-the-board prediction of “no change”—if a state went Democratic or Republican in 2008, it will do the same in 2012—would have scored forty-eight out of fifty, which suggests that the many excited exclamations of “He called all fifty states!”

Aggregating the judgments of people who know a little is better, and if there are enough of them, it can produce impressive results, but aggregating the judgments of an equal number of people who know lots about lots of different things is most effective because the collective pool of information becomes much bigger. Aggregations of aggregations can also yield impressive results. A well-conducted opinion survey aggregates a lot of information about voter intentions, but combining surveys—a “poll of polls”—turns many information pools into one big pool. That’s the core of what Nate Silver, Sam Wang, and other statisticians did in the presidential election of 2012. And a poll of polls can be further aggregated with other data sources. PollyVote is a project of an academic consortium that forecasts presidential elections by aggregating diverse sources, including election polls, the judgments of a panel of political experts, and quantitative models developed by political scientists.


pages: 502 words: 107,657

Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die by Eric Siegel

Albert Einstein, algorithmic trading, Amazon Mechanical Turk, Apple's 1984 Super Bowl advert, backtesting, Black Swan, book scanning, bounce rate, business intelligence, business process, butter production in bangladesh, call centre, Charles Lindbergh, commoditize, computer age, conceptual framework, correlation does not imply causation, crowdsourcing, dark matter, data is the new oil, en.wikipedia.org, Erik Brynjolfsson, Everything should be made as simple as possible, experimental subject, Google Glasses, happiness index / gross national happiness, job satisfaction, Johann Wolfgang von Goethe, lifelogging, Machine translation of "The spirit is willing, but the flesh is weak." to Russian and back, mass immigration, Moneyball by Michael Lewis explains big data, Nate Silver, natural language processing, Netflix Prize, Network effects, Norbert Wiener, personalized medicine, placebo effect, prediction markets, Ray Kurzweil, recommendation engine, risk-adjusted returns, Ronald Coase, Search for Extraterrestrial Intelligence, self-driving car, sentiment analysis, Shai Danziger, software as a service, speech recognition, statistical model, Steven Levy, text mining, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Davenport, Turing test, Watson beat the top human players on Jeopardy!, X Prize, Yogi Berra, zero-sum game

Michael Scherer, “Inside the Secret World of the Data Crunchers Who Helped Obama Win.” TIME Magazine, November 07, 2012. http://swampland.time.com/2012/11/07/inside-the-secret-world-of-quants-and-data-crunchers-who-helped-obama-win/. Colbert Nation, www.colbertnation.com. Stephen Colbert interviews Nate Silver, New York Times blogger about his book, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t. http://www.colbertnation.com/the-colbert-report-videos/420765/november-05-2012/nate-silver. Peggy Noonan, “They’ve Lost That Lovin’ Feeling.” Wall Street Journal, July 30, 2011. http://online.wsj.com/article/SB10001424053111904800304576474620336602248.html. Jack Gillum, “Mitt Romney Uses Secretive Data Mining To Identify Wealthy Donors.” Huffington Post, August 24, 2012. www.huffingtonpost.com/2012/08/24/mitt-romney-data-mining_n_1827318.html.

The bad news is that it’s actually more than half; the good news is that PA can learn to do better. A Faulty Oracle Everyone Loves The first step toward predicting the future is admitting you can’t. —Stephen Dubner, Freakonomics Radio, March 30, 2011 The “prediction paradox”: The more humility we have about our ability to make predictions, the more successful we can be in planning for the future. —Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t Half of what we will teach you in medical school will, by the time you are done practicing, be proved wrong. —Dr. Mehmet Oz Your resident “oracle,” PA, tells you which customers are most likely to respond. It earmarks a quarter of the entire list and says, “These folks are three times more likely to respond than average!”

However, they kept at it, squeezing every drop of potential out of their brainshare and data, right up until the final weeks before the big match. Confidence without Overconfidence Both experts and laypeople mistake more confident predictions for more accurate ones. But overconfidence is often the reason for failure. If our appreciation of uncertainty improves, our predictions can get better too. —Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. —Bertrand Russell You got to know when to hold ‘em, know when to fold ‘em. —Don Schlitz, “The Gambler” (sung by Kenny Rogers) Jeopardy! wasn’t built for players with no self-doubt. —Chris Jones, Esquire Magazine Besides answering questions, there’s a second skill each Jeopardy!


pages: 75 words: 22,220

Occupy by Noam Chomsky

corporate governance, corporate personhood, deindustrialization, Howard Zinn, income inequality, invisible hand, Martin Wolf, Nate Silver, Occupy movement, Plutonomy: Buying Luxury, Explaining Global Imbalances, precariat, Ralph Nader, Ronald Reagan, too big to fail, union organizing

—Greg Ruggiero April 27, 2012 * Rich Morin, “Rising Share of Americans See Conflict Between Rich and Poor,” Pew Research Center, January 11, 2012. † OccupyArrests.com posts a running total of how many Occupy protesters have been arrested since September 17, 2011, and when and where in the United States the arrests took place. ‡ Nate Silver, “Why Obama Will Embrace the 99 Percent,” New York Times, published online February 15, 2012. http://www.nytimes.com/2012/02/19/magazine/nate-silver-obama-reelection-chances.html?hp § Dan Barry, “In Fuel Oil Country, Cold That Cuts to the Heart,” New York Times, February 3, 2012. ¶ Charles M. Blow, “Romney, the Rich and the Rest,” New York Times, February 3, 2012, citing Richard Kogan and Paul N. Van de Water, “Romney Budget Proposals Would Require Massive Cuts in Medicare, Medicaid, and Other Nondefense Spending,” Center on Budget and Policy Priorities, revised February 16, 2012. ** Allison Kilkenny, “Report: 26 Arrested at Occupy Foreclosure Auction Blockade January 27, 2012, In These Times. †† Bailey McCann, “Cities, states pass resolutions against corporate personhood,” January 4, 2012, CivSource. http://civsourceonline.com/2012/01/04/cities-states-pass-resolutions-against-corporate-personhood/ ‡‡ Emily Ramshaw and Jay Root, “A New Rick Perry Shows Up to GOP Debate,” The Texas Tribune, October 18, 2011.


pages: 412 words: 96,251

Why We're Polarized by Ezra Klein

affirmative action, Affordable Care Act / Obamacare, barriers to entry, Bernie Sanders, Cass Sunstein, centre right, Climategate, collapse of Lehman Brothers, currency manipulation / currency intervention, David Brooks, demographic transition, desegregation, Donald Trump, ending welfare as we know it, Ferguson, Missouri, illegal immigration, immigration reform, Nate Silver, obamacare, Ralph Nader, Ronald Reagan, Silicon Valley, single-payer health, source of truth

Lamar Smith of Texas,” Vox, January 27, 2017, vox.com/science-and-health/2017/1/27/14395978/donald-trump-lamar-smith. 16 David Hookstead, “This Sexy Model Is Blowing Up the Internet [SLIDESHOW],” Daily Caller, December 16, 2016, dailycaller.com/2016/12/16/this-sexy-model-is-blowing-up-the-internet-slideshow/; David Hookstead, “This UFC Octagon Girl’s Instagram Account Is Sizzling Hot [SLIDESHOW],” Daily Caller, December 24, 2016, dailycaller.com/2016/12/24/this-ufc-octagon-girls-instagram-account-is-sizzling-hot-slideshow/; Kaitlan Collins, “13 Syrian Refugees We’d Take Immediately [PHOTOS],” Dailey Caller, November 18, 2015, dailycaller.com/2015/11/18/13-syrian-refugees-wed-take-immediately-photos/. 17 Jonathan A. Rodden, Why Cities Lose: The Deep Roots of the Urban-Rural Political Divide (New York: Basic Books, 2019). 18 Ezra Klein, “What Nate Silver’s Learned Abouty Forecasting Elections,” Vox, October 23, 2018, vox.com/ezra-klein-show-podcast/2018/10/23/18014156/nate-silver-538-forecasting-2018-2020-ezra-klein-podcast. 19 Osita Nwanevu, “How Much Do Democrats Need to Win By?” Slate, March 27, 2018, slate.com/news-and-politics/2018/03/how-much-do-democrats-need-to-win-by.html. 20 Michael Geruso, Dean Spears, and Ishaana Talesara, “Inversions in US Presidential Elections: 1836–2016,” National Bureau of Economics Research, Working Paper No. 26247 (September 2019), nber.org/papers/w26247. 21 “Morning Consult’s Governor Approval Rankings,” Morning Consult, Q2 2019, morningconsult.com/governor-rankings-q2-19/. 22 Theda Skocpol and Vanessa Williamson, The Tea Party and the Remaking of Republican Conservatism (New York: Oxford University Press, 2012). 23 Publius Decius Mus [Michael Anton], “The Flight 93 Election,” Claremont Review of Books, September 5, 2016, claremont.org/crb/basicpage/the-flight-93-election. 24 Megan Brenan, “Democrats Favor More Moderate Party; GOP, More Conservative,” Gallup, December 12, 2018, news.gallup.com/poll/245462/democrats-favor-moderate-party-gop-conservative.aspx.

I launched Wonkblog, the online policy vertical at the Washington Post, and I was a cofounder and the first editor in chief of the explanatory news organization Vox, which now reaches more than 50 million people each month. When I was launching Vox, I often got asked who our competitors were.I The answer people expected me to give was other politics-heavy news and analysis sites. The Atlantic. Nate Silver’s FiveThirtyEight. The Washington Post. But the truth was that other news sites were less competitors than they were collaborators in a shared effort to engage people in politics. If Silver converted a sports fan into a politics junkie, that person became instantly more likely to read Vox’s political coverage. But if someone wasn’t interested in politics, or was just sufficiently more interested in, for example, gardening tips or rewatching old Friends clips on YouTube, then that person was lost to us.

Republicans use their majorities to pass partisan gerrymandering plans, pro-corporate campaign finance laws, strict voter ID requirements, and anti-union legislation, and Supreme Court decisions further weaken Democrats’ electoral performance. My point here is not that this is unjust, though I believe it is. Instead, I want to focus on the way this system restrains polarization among Democrats and unleashes it among Republicans. To win, Democrats don’t just need to appeal to the voter in the middle. They need to appeal to voters well to the right of the middle. In the Senate, FiveThirtyEight’s Nate Silver estimates the average state is six points more Republican than the average voter.18 So when Democrats compete for the Senate, they are forced to appeal to an electorate that is far more conservative than the country as a whole. Similarly, it’s estimated that Democrats need to win a substantial majority in the House popular vote to take the gavel.19 And the fact that Democrats have lost two of the last five presidential elections due to the electoral college—the only times that’s happened in American history—signals a growing imbalance there also.


pages: 807 words: 154,435

Radical Uncertainty: Decision-Making for an Unknowable Future by Mervyn King, John Kay

"Robert Solow", Airbus A320, Albert Einstein, Albert Michelson, algorithmic trading, Antoine Gombaud: Chevalier de Méré, Arthur Eddington, autonomous vehicles, availability heuristic, banking crisis, Barry Marshall: ulcers, battle of ideas, Benoit Mandelbrot, bitcoin, Black Swan, Bonfire of the Vanities, Brownian motion, business cycle, business process, capital asset pricing model, central bank independence, collapse of Lehman Brothers, correlation does not imply causation, credit crunch, cryptocurrency, cuban missile crisis, Daniel Kahneman / Amos Tversky, David Ricardo: comparative advantage, demographic transition, discounted cash flows, disruptive innovation, diversification, diversified portfolio, Donald Trump, easy for humans, difficult for computers, Edmond Halley, Edward Lloyd's coffeehouse, Edward Thorp, Elon Musk, Ethereum, Eugene Fama: efficient market hypothesis, experimental economics, experimental subject, fear of failure, feminist movement, financial deregulation, George Akerlof, germ theory of disease, Hans Rosling, Ignaz Semmelweis: hand washing, income per capita, incomplete markets, inflation targeting, information asymmetry, invention of the wheel, invisible hand, Jeff Bezos, Johannes Kepler, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Snow's cholera map, John von Neumann, Kenneth Arrow, Long Term Capital Management, loss aversion, Louis Pasteur, mandelbrot fractal, market bubble, market fundamentalism, Moneyball by Michael Lewis explains big data, Nash equilibrium, Nate Silver, new economy, Nick Leeson, Northern Rock, oil shock, Paul Samuelson, peak oil, Peter Thiel, Philip Mirowski, Pierre-Simon Laplace, popular electronics, price mechanism, probability theory / Blaise Pascal / Pierre de Fermat, quantitative trading / quantitative finance, railway mania, RAND corporation, rent-seeking, Richard Feynman, Richard Thaler, risk tolerance, risk-adjusted returns, Robert Shiller, Robert Shiller, Ronald Coase, sealed-bid auction, shareholder value, Silicon Valley, Simon Kuznets, Socratic dialogue, South Sea Bubble, spectrum auction, Steve Ballmer, Steve Jobs, Steve Wozniak, Tacoma Narrows Bridge, Thales and the olive presses, Thales of Miletus, The Chicago School, the map is not the territory, The Market for Lemons, The Nature of the Firm, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Thomas Bayes, Thomas Davenport, Thomas Malthus, Toyota Production System, transaction costs, ultimatum game, urban planning, value at risk, World Values Survey, Yom Kippur War, zero-sum game

LeRoy and Singell argue that ‘to deny the existence of subjective probabilities is to deny that agents are able to choose consistently among lotteries’. 14 But that is exactly what Keynes and Knight did deny. And with good reason, as we will now see. The probability of an attack on the Twin Towers ‘We may treat people as if they assigned numerical probabilities to every conceivable event.’ So what was the probability that terrorists would fly passenger planes into the World Trade Center on 11 September 2001? Nate Silver, a well-known political pundit in the United States and a devotee of subjective probabilities and Bayesian reasoning, has attempted to answer that question. According to Silver, ‘most of us would have assigned almost no probability to terrorists crashing planes into buildings in Manhattan when we woke up that morning . . . For instance, say that before the first plane hit, our estimate of the possibility of a terror attack on tall buildings in Manhattan was just 1 chance in 20,000.’ 15 But what is the question to which this number is the answer?

Once the probabilities associated with every element of a narrative are multiplied together, as the mathematics of probability requires, the probability that the particular sequence of events described in the narrative will occur steadily diminishes. If you crumple a piece of paper, it makes some shape; but the probability that it would make that particular shape is infinitesimally low. 14 It would be absurd, or at least trivial, to conclude that one has observed a 25 standard deviation occurrence. Crimes are rare and unique events. In chapter 5 we saw Nate Silver fail to make sense of the question ‘What is the probability that an unlikely and unique event would occur?’ when we know that the event has in fact happened. Silver was writing in the context of the attack on the World Trade Center; David Viniar struggled similarly in relation to probabilities in the global financial crisis. We can say that the probability that the fair coin which has just fallen heads would have fallen heads is one half, because tossing a coin is the subject of a well-defined and stationary frequency distribution.

Modern pollsters, who often experience low response rates, know that their sample is not in any sense random, and now use sophisticated and complex models to adjust for their failure to achieve randomness. But this confronts the pollsters, and those who want to use their results, with the Viniar problem: the probability derived from the model has to be compounded with the probability that the model is itself true. We can usefully say things like ‘the pollsters are very experienced’, or ‘the model has worked well in the past’ – as we could of Nate Silver before 2016. But these are judgements, not statements about probabilities. It is thus very difficult to justify the attachment of a statistical confidence interval to an opinion poll result. Nor is this the last of the problems. The answer to a question about voting intentions needs to be translated into a prediction of voting behaviour. People give more honest answers to some questions than others; aggregate sales statistics reveal that they are much more reliable at reporting their consumption of milk than their consumption of alcohol. 11 And then there is also the requirement to transform predicted shares of the popular vote into an anticipated electoral outcome.


pages: 265 words: 74,941

The Great Reset: How the Post-Crash Economy Will Change the Way We Live and Work by Richard Florida

banking crisis, big-box store, blue-collar work, business cycle, car-free, carbon footprint, collapse of Lehman Brothers, congestion charging, creative destruction, deskilling, edge city, Edward Glaeser, falling living standards, financial innovation, Ford paid five dollars a day, high net worth, Home mortgage interest deduction, housing crisis, if you build it, they will come, income inequality, indoor plumbing, interchangeable parts, invention of the telephone, Jane Jacobs, Joseph Schumpeter, knowledge economy, low skilled workers, manufacturing employment, McMansion, Menlo Park, Nate Silver, New Economic Geography, new economy, New Urbanism, oil shock, Own Your Own Home, pattern recognition, peak oil, Ponzi scheme, post-industrial society, postindustrial economy, reserve currency, Richard Florida, Robert Shiller, Robert Shiller, secular stagnation, Silicon Valley, Silicon Valley startup, social intelligence, sovereign wealth fund, starchitect, the built environment, The Wealth of Nations by Adam Smith, Thomas L Friedman, total factor productivity, urban decay, urban planning, urban renewal, white flight, young professional, Zipcar

Christopher Leinberger, “Car Free in America/Bottom Line: It’s Cheaper,” New York Times, online symposium, May 12, 2009, retrieved from http://roomfordebate.blogs.nytimes.com/2009/05/12/carless-in-america/?hp 14. Yuri Kageyama, “Car-Free: In Japan, That’s How a Generation Rolls,” Associated Press, January 6, 2009. 15. Rich Morin and Paul Taylor, “Luxury or Necessity? The Public Makes a U-Turn,” Pew Research Center, April 23, 2009, retrieved from http://pewsocialtrends.org/pubs/733/luxury-necessity-recession-era-reevaluations. 16. Nate Silver, “The End of Car Culture,” Esquire, May 6, 2009, retrieved from www.esquire.com/features/data/nate-silver-car-culture-stats-0609. 17. Martin Zimmerman, “Rebel without a Car?” Los Angeles Times, October 8, 2009, retrieved from http://latimesblogs.latimes.com/uptospeed/2009/10/james-dean-.html. 18. Micheline Maynard, “Is Happiness Still That New Car Smell?” 19. Felix Salmon, “Chart of the Day: Necessity,” Reuters (blog), April 28, 2009. 20. John Seabrook, Nobrow: The Culture of Marketing, the Marketing of Culture (New York: Vintage, 2001). 21.

The highest-ranked necessity was a car: 88 percent of people surveyed named it as a necessity.15 That may seem like a lot, but in a society where there are two cars or more for every family and where suburban life literally requires a car to accomplish life’s most basic tasks, 12 percent of people believing that it’s no longer a necessity seems like a substantial number. The statistics guru Nate Silver, writing in Esquire, says there is hard evidence that America’s once-great car culture has peaked. Silver performed a statistical analysis to show how much Americans drive based on gas prices and unemployment. He then graphed the results from 1980 to 2009. “Americans should have driven slightly more in January 2009 than they had a year earlier,” he found, pointing to sharply lower gas prices as the overriding factor.


pages: 249 words: 77,342

The Behavioral Investor by Daniel Crosby

affirmative action, Asian financial crisis, asset allocation, availability heuristic, backtesting, bank run, Black Swan, buy and hold, cognitive dissonance, colonial rule, compound rate of return, correlation coefficient, correlation does not imply causation, Daniel Kahneman / Amos Tversky, diversification, diversified portfolio, Donald Trump, endowment effect, feminist movement, Flash crash, haute cuisine, hedonic treadmill, housing crisis, IKEA effect, impulse control, index fund, Isaac Newton, job automation, longitudinal study, loss aversion, market bubble, market fundamentalism, mental accounting, meta analysis, meta-analysis, Milgram experiment, moral panic, Murray Gell-Mann, Nate Silver, neurotypical, passive investing, pattern recognition, Ponzi scheme, prediction markets, random walk, Richard Feynman, Richard Thaler, risk tolerance, Robert Shiller, Robert Shiller, science of happiness, Shai Danziger, short selling, South Sea Bubble, Stanford prison experiment, Stephen Hawking, Steve Jobs, stocks for the long run, Thales of Miletus, The Signal and the Noise by Nate Silver, tulip mania, Vanguard fund

Research shows that lots of choices lead to both paralysis and dissatisfaction with your eventual choice. Several experiments suggest that when those presented with an extensive array of options make fewer purchases and are less happy with the purchases they make than those operating from a more limited decisional universe. Another consequence of financial information overload is that it leads to drawing spurious correlations between variables. As Nate Silver reports, the government produces data on 45,000 economic variables each year!44 Pair this reality with the fact that there are relatively few dramatic economic events (e.g., there have been 11 recessions since the end of World War II) and you get what Silver refers to as putting data into a blender and calling the result haute cuisine. And then consider the strange case of the correlation between moves in the S&P 500 and Bangladeshi butter production.

Complex dynamic systems paradoxically require simple solutions to avoid overfitting. Noise is what makes markets possible. It is also what makes them almost impossible to beat. Notes 42 Zweig, Your Money and Your Brain, p. 22. 43 Greg B. Davies, Behavioral Investment Management: An Efficient Alternative to Modern Portfolio Theory (McGraw-Hill, 2012), p. 53. 44 Nate Silver, The Signal and the Noise: Why So Many Predictions Fail – but Some Don’t (Penguin, 2015), p. 185. Chapter 7. Emotion “The world is a tragedy to those who feel, but a comedy to those who think.” — Horace Walpole Emotion: friend or foe? It must be stated from the outset that there is some disagreement within the behavioral finance community about whether emotion is a help or hindrance when making investment decisions.

A 1968 study by Lewis Goldberg analyzed the performance of a model-based approach to assessing mental illness versus the clinical judgment of trained doctors. Not only did the simple model outperform the psychologists’ intuition head-to-head, but it also bested psychologists who were given access to the model.109 Models have also been shown to outperform human intuition in predicting the outcomes of Supreme Court decisions,110 Presidential elections (Nate Silver), movie preferences, prison recidivism, wine quality, marital satisfaction and military success, to name just a few of the over 45 domains in which they have demonstrated their superiority.111 A meta-analysis performed by William Grove, David Zald, Boyd Lebow, Beth Snitz and Chad Nelson found that models equal or beat expert decision-making a whopping 94.12% of the time, meaning that they are only defeated by human discretion 5.88% of the time.112 Moreover, many of the domains in which algorithms greatly outperformed had human behavior as a central component (as do financial markets).


pages: 788 words: 223,004

Merchants of Truth: The Business of News and the Fight for Facts by Jill Abramson

23andMe, 4chan, Affordable Care Act / Obamacare, Alexander Shulgin, Apple's 1984 Super Bowl advert, barriers to entry, Bernie Madoff, Bernie Sanders, Charles Lindbergh, Chelsea Manning, citizen journalism, cloud computing, commoditize, corporate governance, creative destruction, crowdsourcing, death of newspapers, digital twin, diversified portfolio, Donald Trump, East Village, Edward Snowden, Ferguson, Missouri, Filter Bubble, future of journalism, glass ceiling, Google Glasses, haute couture, hive mind, income inequality, information asymmetry, invisible hand, Jeff Bezos, Joseph Schumpeter, Khyber Pass, late capitalism, Marc Andreessen, Mark Zuckerberg, move fast and break things, move fast and break things, Nate Silver, new economy, obamacare, Occupy movement, performance metric, Peter Thiel, phenotype, pre–internet, race to the bottom, recommendation engine, Robert Mercer, Ronald Reagan, Saturday Night Live, self-driving car, sentiment analysis, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, skunkworks, Snapchat, social intelligence, social web, Steve Jobs, Steven Levy, technoutopianism, telemarketer, the scientific method, The Wisdom of Crowds, Tim Cook: Apple, too big to fail, WikiLeaks

So it wasn’t surprising that the Post: Tom McCarthy, “Washington Post’s Ezra Klein Leaving Newspaper to Start ‘New Venture,’ ” Guardian, January 21, 2014, https://www.theguardian.com/media/2014/jan/21/washington-posts-ezra-klein-leaving-news-organisation. The Times too lost Silver: John McDuling, “ ‘The Upshot’ Is the New York Times’ Replacement for Nate Silver’s FiveThirtyEight,” Quartz, March 10, 2014, https://qz.com/185922/the-upshot-is-the-new-york-times-replacement-for-nate-silvers-fivethirtyeight/. There were bound to be problems: Trevor Butterworth, “The Latest Sad Fate of an Aggregation Serf,” Awl, April 24, 2012, https://www.theawl.com/2012/04/the-latest-sad-fate-of-an-aggregation-serf/. One story BlogPOST recycled: Patrick B. Pexton, “Elizabeth Flock’s Resignation: The Post Fails a Young Blogger,” Washington Post, April 20, 2012, https://www.washingtonpost.com/opinions/elizabeth-flocks-resignation-the-post-fails-a-young-blogger/2012/04/20/gIQAFACXWT_story.html?

Gretchen Morgenson, a Pulitzer winner and business columnist, thought Sorkin was too cozy with sources such as J.P. Morgan Chase CEO Jamie Dimon. After the financial crisis, she argued, the last thing the Times needed was to go soft on Wall Street. But DealBook, replete with a conference headlined by Sorkin, launched with the publisher’s keen support, despite the objections from Morgenson and others who saw the venture as driven more by financial than journalism goals. Nate Silver, a popular political blogger who created FiveThirtyEight, licensed his site to the Times, and alongside Sorkin would come to represent a new breed of reporter, better known as individuals than belonging to the institution. Both stars negotiated special pay-for-performance packages under a new system that Sulzberger and Robinson had installed. With his job cohosting a CNBC morning cable show, Squawk Box, plus his Times column and DealBook, Sorkin was making a small fortune.

Bell helped power Wonkblog with the best technology, and together they developed a new visual system of explaining complex information and policy details. They would edit the information and present coherent information bullets on individual “cards.” Readers loved them. Many people thought Klein had the best coverage of President Obama’s complex health overhaul. Klein’s fluid writing and original takes quickly made Wonkblog a major attraction on WashingtonPost.com. Like Nate Silver, who had established his FiveThirtyEight blog before he joined the Times, Klein attracted a different audience to the Post. But also like Silver, Klein wanted to be his own brand. He was allowed to hire a staff of eight, while the rest of the staffing at the Post tightened. As his blog became better known, he also appeared in the New Yorker and on CNN. Anyone with an individual brand and proven audience was in high demand as digital news start-ups proliferated, on the left as well as on the right, where Breitbart News was gaining a foothold.


pages: 396 words: 117,149

The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World by Pedro Domingos

Albert Einstein, Amazon Mechanical Turk, Arthur Eddington, basic income, Bayesian statistics, Benoit Mandelbrot, bioinformatics, Black Swan, Brownian motion, cellular automata, Claude Shannon: information theory, combinatorial explosion, computer vision, constrained optimization, correlation does not imply causation, creative destruction, crowdsourcing, Danny Hillis, data is the new oil, double helix, Douglas Hofstadter, Erik Brynjolfsson, experimental subject, Filter Bubble, future of work, global village, Google Glasses, Gödel, Escher, Bach, information retrieval, job automation, John Markoff, John Snow's cholera map, John von Neumann, Joseph Schumpeter, Kevin Kelly, lone genius, mandelbrot fractal, Mark Zuckerberg, Moneyball by Michael Lewis explains big data, Narrative Science, Nate Silver, natural language processing, Netflix Prize, Network effects, NP-complete, off grid, P = NP, PageRank, pattern recognition, phenotype, planetary scale, pre–internet, random walk, Ray Kurzweil, recommendation engine, Richard Feynman, scientific worldview, Second Machine Age, self-driving car, Silicon Valley, social intelligence, speech recognition, Stanford marshmallow experiment, statistical model, Stephen Hawking, Steven Levy, Steven Pinker, superintelligent machines, the scientific method, The Signal and the Noise by Nate Silver, theory of mind, Thomas Bayes, transaction costs, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, white flight, zero-sum game

In politics, as in business and war, there is nothing worse than seeing your opponent make moves that you don’t understand and don’t know what to do about until it’s too late. That’s what happened to the Romney campaign. They could see the other side buying ads in particular cable stations in particular towns but couldn’t tell why; their crystal ball was too fuzzy. In the end, Obama won every battleground state save North Carolina and by larger margins than even the most accurate pollsters had predicted. The most accurate pollsters, in turn, were the ones (like Nate Silver) who used the most sophisticated prediction techniques; they were less accurate than the Obama campaign because they had fewer resources. But they were a lot more accurate than the traditional pundits, whose predictions were based on their expertise. You might think the 2012 election was a fluke: most elections are not close enough for machine learning to be the deciding factor. But machine learning will cause more elections to be close in the future.

Either way, to obtain a learner’s prediction for a given training example, we must first apply it to the original training set excluding that example and use the resulting classifier—otherwise the committee risks being dominated by learners that overfit, since they can predict the correct class just by remembering it. The Netflix Prize winner used metalearning to combine hundreds of different learners. Watson uses it to choose its final answer from the available candidates. Nate Silver combines polls in a similar way to predict election results. This type of metalearning is called stacking and is the brainchild of David Wolpert, whom we met in Chapter 3 as the author of the “no free lunch” theorem. An even simpler metalearner is bagging, invented by the statistician Leo Breiman. Bagging generates random variations of the training set by resampling, applies the same learner to each one, and combines the results by voting.

“Machine learning in drug discovery and development,”* by Niki Wale (Drug Development Research, 2001), gives an overview of just that. Adam, the robot scientist, is described in “The automation of science,” by Ross King et al. (Science, 2009). Sasha Issenberg’s The Victory Lab (Broadway Books, 2012) dissects the use of data analysis in politics. “How President Obama’s campaign used big data to rally individual votes,” by the same author (MIT Technology Review, 2013), tells the story of its greatest success to date. Nate Silver’s The Signal and the Noise (Penguin Press, 2012) has a chapter on his poll aggregation method. Robot warfare is the theme of P. W. Singer’s Wired for War (Penguin, 2009). Cyber War, by Richard Clarke and Robert Knake (Ecco, 2012), sounds the alarm on cyberwar. My work on combining machine learning with game theory to defeat adversaries, which started as a class project, is described in “Adversarial classification,”* by Nilesh Dalvi et al.


pages: 296 words: 82,501

Stuffocation by James Wallman

3D printing, Airbnb, back-to-the-land, Berlin Wall, big-box store, Black Swan, BRICs, carbon footprint, Cass Sunstein, clean water, collaborative consumption, commoditize, creative destruction, crowdsourcing, David Brooks, Fall of the Berlin Wall, happiness index / gross national happiness, hedonic treadmill, high net worth, income inequality, Intergovernmental Panel on Climate Change (IPCC), James Hargreaves, Joseph Schumpeter, Kitchen Debate, Martin Wolf, mass immigration, McMansion, means of production, Nate Silver, Occupy movement, Paul Samuelson, post-industrial society, post-materialism, Richard Florida, Richard Thaler, sharing economy, Silicon Valley, Simon Kuznets, Skype, spinning jenny, The Signal and the Noise by Nate Silver, Thorstein Veblen, Tyler Cowen: Great Stagnation, World Values Survey, Zipcar

Using the Past to Tell the Future I am indebted to three sources for this section: Peter N Stearns, “Why Study History?”, American Historical Association, 1998; Nate Silver, The Signal and the Noise (New York: Allen Lane, 2012); and Rob Hyndman, “Why are some things easier to forecast than others?”, 18 September 2012, on his blog, Hyndsight (www.robjhyndman.com/hyndsight). “In the 1970s, the high temperature forecasts were wrong, on average, by about six degrees. Today they are only wrong by half that amount, three degrees. When hurricane forecasters predicted where a hurricane would hit land in the 1980s they were usually out by 350 miles. Today, their predictions are only wrong by 100 miles.” If you’re not ready – yet – to take on all of Nate Silver’s The Signal and the Noise, read Nate Silver, “The Weatherman Is Not a Moron”, New York Times, 7 September 2012. The Farm Where the Corn Did Not Grow Tall For Everett Rogers’s version of his life, read Everett M Rogers, The Fourteenth Paw (Singapore: Asian Media Information and Communication Centre, 2008).


pages: 296 words: 78,112

Devil's Bargain: Steve Bannon, Donald Trump, and the Storming of the Presidency by Joshua Green

4chan, Affordable Care Act / Obamacare, Ayatollah Khomeini, Bernie Sanders, business climate, centre right, Charles Lindbergh, coherent worldview, collateralized debt obligation, conceptual framework, corporate raider, crony capitalism, currency manipulation / currency intervention, Donald Trump, Fractional reserve banking, Goldman Sachs: Vampire Squid, Gordon Gekko, guest worker program, illegal immigration, immigration reform, liberation theology, low skilled workers, Nate Silver, Nelson Mandela, nuclear winter, obamacare, Peace of Westphalia, Peter Thiel, quantitative hedge fund, Renaissance Technologies, Robert Mercer, Ronald Reagan, Silicon Valley, social intelligence, speech recognition, urban planning

“We’re gonna go buck wild”: Joshua Green, “Trump to Intensify Attacks on Clinton over Husband’s Accusers,” Bloomberg.com, October 12, 2016, www.bloomberg.com/politics/articles/2016-10-12/trump-takes-a-back-to-the-future-focus-on-bill-clinton-s-women. “Is the Presidential Race Tightening?”: Nate Silver, “Election Update: Is the Presidential Race Tightening?,” FiveThirtyEight, New York Times, October 26, 2016, fivethirtyeight.com/features/election-update-is-the-presidential-race-tightening/. “Yes, Donald Trump Has a Path to Victory”: Nate Silver, “Election Update: Yes, Donald Trump Has a Path to Victory,” FiveThirtyEight, New York Times, November 1, 2016, fivethirtyeight.com/features/election-update-yes-donald-trump-has-a-path-to-victory/. Chapter Eleven: “The FBI Has Learned of the Existence . . .” “We’re already seeing the effects”: “Hillary Clinton Speaks to Estimated Crowd of 10,000+,” AZCentral.com, November 2, 2016, www.azcentral.com/story/news/politics/elections/2016/11/02/hillary-clinton-arizona-asu-presidential-campaign-rally/93143352/, “Arizona ain’t an indulgence”: Brian Fallon, Twitter post, October 29, 2016, 9:41 a.m., twitter.com/brianefallon/status/792043601463283713.

Whether it was a result of Trump’s apocalyptic turn, disgust at the Clintons, or simply accuser fatigue—it was likely a combination of all three—the pattern of slippage in the wake of negative news was less pronounced in Trump’s internal surveys in mid-October. Overall, he still trailed. But the data were noisy. In some states (Indiana, New Hampshire, Arizona) his support eroded, but in others (Florida, Ohio, Michigan) it actually improved. When Trump held his own at the third and final debate on October 19, the numbers inched up further. The movement was clear enough that Nate Silver and other statistical mavens began to take note of it. “Is the Presidential Race Tightening?” he asked in the title of an October 26 article. Citing Trump’s rising favorability numbers among Republicans and red-state trend lines, he cautiously concluded that probably it was. By November 1, he had no doubt. “Yes, Donald Trump Has a Path to Victory” read the headline for his column that day, in which he noted that Clinton’s lead in national polls had shrunk from seven points in mid-October down to three or four points.


pages: 442 words: 94,734

The Art of Statistics: Learning From Data by David Spiegelhalter

Antoine Gombaud: Chevalier de Méré, Bayesian statistics, Carmen Reinhart, complexity theory, computer vision, correlation coefficient, correlation does not imply causation, dark matter, Edmond Halley, Estimating the Reproducibility of Psychological Science, Hans Rosling, Kenneth Rogoff, meta analysis, meta-analysis, Nate Silver, Netflix Prize, p-value, placebo effect, probability theory / Blaise Pascal / Pierre de Fermat, publication bias, randomized controlled trial, recommendation engine, replication crisis, self-driving car, speech recognition, statistical model, The Design of Experiments, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Malthus

Finally, much as I would like to find someone else to blame, I am afraid I must acknowledge full responsibility for the inevitable remaining inadequacies of this book. CODE FOR EXAMPLES R code and data for reproducing most of the analyses and Figures are available from https://github.com/dspiegel29/ArtofStatistics. I am grateful for the assistance received in preparing this material. Introduction The numbers have no way of speaking for themselves. We speak for them. We imbue them with meaning. — Nate Silver, The Signal and the Noise1 Why We Need Statistics Harold Shipman was Britain’s most prolific convicted murderer, though he does not fit the archetypal profile of a serial killer. A mild-mannered family doctor working in a suburb of Manchester, between 1975 and 1998 he injected at least 215 of his mostly elderly patients with a massive opiate overdose. He finally made the mistake of forging the will of one of his victims so as to leave him some money: her daughter was a solicitor, suspicions were aroused, and forensic analysis of his computer showed he had been retrospectively changing patient records to make his victims appear sicker than they really were.

Figure 1.1 Horizontal bar-chart of 30–day survival rates for thirteen hospitals. The choice of the start of the horizontal axis, here 86%, can have a crucial effect on the impression given by the graphic. If the axis starts at 0%, all the hospitals will look indistinguishable, whereas if we started at 95% the differences would look misleadingly dramatic. I began this book with a quotation from Nate Silver, the founder of data-based platform FiveThirtyEight and first famous for accurately predicting the 2008 US presidential election, who eloquently expressed the idea that numbers do not speak for themselves – we are responsible for giving them meaning. This implies that communication is a key part of the problem-solving cycle, and I have shown in this section how the message from a set of simple proportions can be influenced by our choices of presentation.

Z-score: a means of standardizing an observation xi in terms of its distance from the sample mean m expressed in terms of sample standard deviations s, so that zi = (xi – m)/s. An observation with a Z-score of 3 corresponds to being 3 standard deviations above the mean, which is a fairly extreme outlier. A Z-score can also be defined in terms of a population mean μ and standard deviation σ, in which case zi = (xi – μ)/σ. Notes INTRODUCTION 1. The Signal and the Noise by Nate Silver (Penguin, 2012) is an excellent introduction to how statistical science can be applied to making predictions in sport and other domains. 2. The Shipman data is discussed in more detail in D. Spiegelhalter and N. Best, ‘Shipman’s Statistical Legacy’, Significance1:1 (2004), 10–12. All documents for the public inquiry are available from http://webarchive.nationalarchives.gov.uk/20090808155110/http://www.the-shipman-inquiry.org.uk/reports.asp. 3.


pages: 379 words: 99,340

The Revolt of the Public and the Crisis of Authority in the New Millennium by Martin Gurri

Affordable Care Act / Obamacare, Albert Einstein, anti-communist, Arthur Eddington, Ayatollah Khomeini, bitcoin, Black Swan, Burning Man, business cycle, citizen journalism, Climategate, Climatic Research Unit, collective bargaining, creative destruction, crowdsourcing, currency manipulation / currency intervention, dark matter, David Graeber, death of newspapers, en.wikipedia.org, Erik Brynjolfsson, facts on the ground, Francis Fukuyama: the end of history, Frederick Winslow Taylor, full employment, housing crisis, income inequality, Intergovernmental Panel on Climate Change (IPCC), invention of writing, job-hopping, Mohammed Bouazizi, Nate Silver, Occupy movement, Port of Oakland, Republic of Letters, Ronald Reagan, Skype, Steve Jobs, the scientific method, The Signal and the Noise by Nate Silver, too big to fail, traveling salesman, University of East Anglia, urban renewal, War on Poverty, We are the 99%, WikiLeaks, young professional

Here was a bold attempt at prophecy by the new team of experts: in the event, it was wildly over-optimistic. Unemployment peaked at 10.1 percent after the stimulus bill passed, and didn’t touch 8 percent until late 2012 – much worse than the worst-case projections without the stimulus.[137] In human terms, the White House numbers had missed the plight of over 3 million unemployed Americans. Nate Silver offered two reasons for Romer and Bernstein’s disconcerting failure at prediction, and neither of them seemed flattering to the expert class. The first was ignorance of actual economic conditions. The economy in 2009 happened to be in far worse shape than the experts, for all their statistical wizardry, had realized. The second reason was overconfidence in tracking the trajectory of unemployment.

[111] John Dollar, “The Man Who Predicted an Earthquake,” The Guardian, April 5, 2010, http://www.theguardian.com/world/2010/apr/05/laquila-earthquake-prediction-giampaolo-giuliani. [112] Dollar, “Man Who Predicted Earthquake” and Pielke, “Lessons of L’Aquila Lawsuit.” [113] Ibid. [114] “L’Aquila quake: Italy scientists guilty of manslaughter,” BBC, October 27, 2012, http://www.bbc. co.uk/news/world-europe-20025626. [115] An excellent evaluation of the state of the art in the forecasting of earthquakes is found in Nate Silver’s The Signal and the Noise: Why So Many Predictions Fail – But Some Don’t (The Penguin Press, 2012), 142-175. [116] Climategate Emails, 84. [117] Wikipedia Commons. [118] Bob Woodward, Maestro: Greenspan’s Fed and the American Boom (Simon and Schuster, 2000), Kindle location 507. [119] Ibid., Kindle location 602-609, 641. [120] Ibid., Kindle location 396. [121] The New York Times and Washington Post, in particular, seemed to cover two mutually hostile Alan Greenspans.

[193] “Public Trust in Government: 1958-2013,” Pew Research Center for the People and the Press, October 18, 2013, http://www.people-press.org/2013/10/18/trust-in-government-interactive/. [194] Ormerod belongs to a group of thinkers who in the last decade have thrown much light on the boundary conditions of what human beings can and can’t know, predict, or ordain: N. N. Taleb, Duncan Watts, James C. Scott, Philip Tetlock, Nate Silver. [195] Ormerod, Why Most Things Fail, Kindle location 66. [196] Ibid., 38. [197] Ibid., 236. [198] Ibid., 67-68. [199] Ibid., 99. [200] I urge the reader to browse the many job postings for “community organizer” on the web, which add up to a familiar portrait. [201] Melissa McEwan, “President Obama at Planned Parenthood,” Shakesville, April 26, 2013, http://www.shakesville.com/2013/04/president-obama-at-planned-parenthood.html


The New Class War: Saving Democracy From the Metropolitan Elite by Michael Lind

affirmative action, anti-communist, basic income, Bernie Sanders, Boris Johnson, Bretton Woods, business cycle, capital controls, Cass Sunstein, central bank independence, centre right, collective bargaining, commoditize, corporate governance, crony capitalism, deindustrialization, Doha Development Round, Donald Trump, Edward Snowden, future of work, global supply chain, guest worker program, Haight Ashbury, illegal immigration, immigration reform, invisible hand, knowledge economy, liberal world order, low skilled workers, low-wage service sector, manufacturing employment, Mark Zuckerberg, mass immigration, means of production, moral panic, Nate Silver, new economy, offshore financial centre, oil shock, open borders, plutocrats, Plutocrats, Ponzi scheme, purchasing power parity, Ralph Nader, regulatory arbitrage, rent-seeking, Richard Florida, Ronald Reagan, Silicon Valley, The Wealth of Nations by Adam Smith, Thorstein Veblen, trade liberalization, union organizing, universal basic income, upwardly mobile, WikiLeaks, Wolfgang Streeck, working poor

* * * — ON BOTH SIDES of the Atlantic, the political divide between the educated overclass and the rest of the country is stark. In the 2016 US presidential election, among counties with a population of fifty thousand or more, Hillary Clinton won forty-eight of the fifty counties that had the highest percentage of voters with at least a four-year bachelor’s degree. Support for her presidential bid “collapsed” (to use the pollster Nate Silver’s term) in the fifty counties with the lowest educational levels. Political differences correlated with education can be found among racial and ethnic minority populations as well.16 The same pattern is evident in Europe. In Britain, for example, the chief trait predicting support for the Leave side in the Brexit referendum in 2016 was lower educational qualifications—a trait that was more important than others, including race and ethnicity.17 Because the possession of a diploma tends to indicate birth into the economic elite, these figures manifest conflict among largely hereditary social classes, not a clash between knowledge and ignorance or intelligence and stupidity.

Quinn, and Kathryn Peltier Campbell, Born to Win, Schooled to Lose: Why Talented Students Don’t Get Equal Chances to Be All They Can Be, Georgetown University Center on Education and the Workforce report, 2019. 14. UNCTAD, “Annex Table 24. The World’s Top 100 Non-Financial MNE’s, Ranked by Foreign Assets, 2016.” 15. George Davis, “Boards Aren’t as Global as Their Businesses,” Harvard Business Review, October 28, 2014. 16. Nate Silver, “Education, Not Income, Predicted Who Would Vote for Trump,” FiveThirtyEight.com, November 22, 2016. 17. Martin Rosenbaum, “Local Voting Figures Shed New Light on EU Referendum,” BBC News, February 6, 2017. 18. Edna Bonacich, “A Theory of Ethnic Antagonism: The Split Labor Market,” American Sociological Review 37, no. 5 (October 1972), pp. 547–59. 19. Gavin Wright, Sharing the Prize: The Economics of the Civil Rights Revolution in the American South (Cambridge, MA: Harvard University Press, 2013).


pages: 230 words: 61,702

The Internet of Us: Knowing More and Understanding Less in the Age of Big Data by Michael P. Lynch

Affordable Care Act / Obamacare, Amazon Mechanical Turk, big data - Walmart - Pop Tarts, bitcoin, Cass Sunstein, Claude Shannon: information theory, crowdsourcing, Edward Snowden, Firefox, Google Glasses, hive mind, income inequality, Internet of things, John von Neumann, meta analysis, meta-analysis, Nate Silver, new economy, Panopticon Jeremy Bentham, patient HM, prediction markets, RFID, sharing economy, Steve Jobs, Steven Levy, the scientific method, The Wisdom of Crowds, Thomas Kuhn: the structure of scientific revolutions, WikiLeaks

Arguably the insiders’ information was better, and their take on it more legally sophisticated, than that of the larger crowd. In this sort of case, the larger crowd is not the one you want to listen to. One might think the same goes for predicting something like the success of medical surgery. Unless the crowd has the same information and training as the relevant experts, it is not clear that they have wisdom to impart. As Leonhardt’s colleague Nate Silver noted during the final run-up to the 2012 election, such markets may contain more or less sophisticated participants, and the more sophisticated the average participant, the more other sophisticated participants tend to trust it. Moreover, when a given market is highly cited in the press, “that opens up the possibility that someone could place a wager on [a candidate] in order to influence the news media’s perceptions about which candidate has the momentum.”10 If so, the market may not be reflecting or mapping voter opinion but helping to determine it.

After all, the interviewer example presumes that the tacit commitment in question is itself a product of the individuals’ beliefs. If so, then perhaps the best we can say is that what we can loosely call the group’s implicit commitment “supervenes” or is a product of the individuals’ commitments. 8. Surowiecki, The Wisdom of Crowds, xii. 9. David Leonhardt, “When the Crowd Isn’t Wise,” New York Times, July 7, 2012. 10. Nate Silver, “The Virtues and Vices of Election Prediction Markets,” New York Times, October 24, 2012. 11. I was helped to see these points in discussions with Sandy Goldberg and Nate Sheff. The example in the text is similar to that in Goldberg, “The Division of Epistemic Labor,” 117. 12. Weinberger, Too Big to Know, 21. 13. Descartes, Meditations, 103. 14. Weinberger, Too Big to Know, 23. 15. Sosa, Reflective Knowledge, chs. 7 and 8.. 16.


pages: 443 words: 116,832

The Hacker and the State: Cyber Attacks and the New Normal of Geopolitics by Ben Buchanan

active measures, Bernie Sanders, bitcoin, blockchain, borderless world, Brian Krebs, British Empire, Cass Sunstein, citizen journalism, credit crunch, cryptocurrency, cuban missile crisis, data acquisition, Donald Trump, drone strike, Edward Snowden, family office, hive mind, Internet Archive, Jacob Appelbaum, John Markoff, John von Neumann, Julian Assange, Kickstarter, kremlinology, MITM: man-in-the-middle, Nate Silver, profit motive, RAND corporation, ransomware, risk tolerance, Robert Hanssen: Double agent, rolodex, Ronald Reagan, Silicon Valley, South China Sea, Steve Jobs, Stuxnet, technoutopianism, undersea cable, uranium enrichment, Vladimir Vetrov: Farewell Dossier, WikiLeaks, zero day

Also see Adam Gabbatt, “Trump Says Russia Helped Elect Him—Then Quickly Backtracks,” Guardian, May 30, 2019. 82. Nate Silver, “How Much Did Russian Interference Affect The 2016 Election?,” FiveThirtyEight, February 16, 2018; Lucam Ahmad Way and Adam Casey, “Russia Has Been Meddling in Foreign Elections for Decades. Has It Made a Difference?,” Washington Post, January 8, 2018; Jane Mayer, “How Russia Helped Swing the Election for Trump,” New Yorker, September 24, 2018. 83. For an analysis of the Comey letter, see Nate Silver, “The Comey Letter Probably Cost Clinton The Election,” FiveThirtyEight, May 3, 2017. Among other possibly serious missteps, Clinton never campaigned in Wisconsin during the general election, a state she narrowly lost. Nate Silver, “Donald Trump Had a Superior Electoral College Strategy,” FiveThirtyEight, February 6, 2017.


pages: 345 words: 75,660

Prediction Machines: The Simple Economics of Artificial Intelligence by Ajay Agrawal, Joshua Gans, Avi Goldfarb

"Robert Solow", Ada Lovelace, AI winter, Air France Flight 447, Airbus A320, artificial general intelligence, autonomous vehicles, basic income, Bayesian statistics, Black Swan, blockchain, call centre, Capital in the Twenty-First Century by Thomas Piketty, Captain Sullenberger Hudson, collateralized debt obligation, computer age, creative destruction, Daniel Kahneman / Amos Tversky, data acquisition, data is the new oil, deskilling, disruptive innovation, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, everywhere but in the productivity statistics, Google Glasses, high net worth, ImageNet competition, income inequality, information retrieval, inventory management, invisible hand, job automation, John Markoff, Joseph Schumpeter, Kevin Kelly, Lyft, Minecraft, Mitch Kapor, Moneyball by Michael Lewis explains big data, Nate Silver, new economy, On the Economy of Machinery and Manufactures, pattern recognition, performance metric, profit maximization, QWERTY keyboard, race to the bottom, randomized controlled trial, Ray Kurzweil, ride hailing / ride sharing, Second Machine Age, self-driving car, shareholder value, Silicon Valley, statistical model, Stephen Hawking, Steve Jobs, Steven Levy, strong AI, The Future of Employment, The Signal and the Noise by Nate Silver, Tim Cook: Apple, Turing test, Uber and Lyft, uber lyft, US Airways Flight 1549, Vernor Vinge, Watson beat the top human players on Jeopardy!, William Langewiesche, Y Combinator, zero-sum game

Kelly Gonsalves, “Google Has More Than 1,000 Artificial Intelligence Projects in the Works,” The Week, October 18, 2016, http://theweek.com/speedreads/654463/google-more-than-1000-artificial-intelligence-projects-works. 5. A rich, entertaining, and ultimately useless debate rages about whether these sabermetric analysts are better or worse than the scouts. As Nate Silver highlights, both the Moneyball types and the scouts have important roles to play. Nate Silver, The Signal and the Noise (New York: Penguin Books, 2015), chapter 3. 6. You may counter and say that surely, in order to improve, the prediction machine needs that past repository of data? This is a subtle issue. Prediction works best when adding new data does not change algorithms too much—that stability is an outcome of good statistical practice.


pages: 340 words: 81,110

How Democracies Die by Steven Levitsky, Daniel Ziblatt

Affordable Care Act / Obamacare, Ayatollah Khomeini, basic income, Berlin Wall, Bernie Sanders, Capital in the Twenty-First Century by Thomas Piketty, Cass Sunstein, centre right, Charles Lindbergh, clean water, David Brooks, Donald Trump, Fall of the Berlin Wall, Gunnar Myrdal, illegal immigration, immigration reform, income inequality, Jeff Bezos, Nate Silver, Norman Mailer, old-boy network, Robert Gordon, Ronald Reagan, single-payer health, The Rise and Fall of American Growth, universal basic income

Whereas Bush, Rubio, Cruz, Christie, Walker, and Kasich all had deep Republican roots, Trump had switched his party registration several times and had even contributed to Hillary Clinton’s campaign for the U.S. Senate. Even after Trump began to surge in the polls, few people took his candidacy seriously. In August 2015, two months after Trump declared his candidacy, Las Vegas bookmakers gave him one-hundred-to-one odds of winning the White House. And in November 2015, as Trump sat high atop the Republican polls, Nate Silver, founder of the FiveThirtyEight blog, whose uncannily accurate predictions in the 2008 and 2012 elections had earned him fame and prestige, wrote an article titled “Dear Media: Stop Freaking Out About Donald Trump’s Poll Numbers.” The article predicted that Trump’s weakness among party insiders would spell his demise. Despite Trump’s seemingly large lead, Silver assured us, his chances of winning the nomination were “considerably less than 20 percent.”

We thank Fernando Bizzarro for his assistance in compiling these data. skip the “invisible primary”: For a detailed explanation of why this was the case, see Cohen, Karol, Noel, and Zaller, The Party Decides. Las Vegas bookmakers: James Ceaser, Andrew Busch, and John Pitney Jr., Defying the Odds: The 2016 Elections and American Politics (Washington, DC: Rowman & Littlefield, 2017), p. 69. “considerably less than 20 percent”: Nate Silver, “Dear Media: Stop Freaking Out About Donald Trump’s Polls,” FiveThirtyEight, November 23, 2015, http://fivethirtyeight.com/​features/​dear-media-stop-freaking-out-about-donald-trumps-polls/. Citizens United ruling: Marty Cohen, David Karol, Hans Noel, and John Zaller, “Party Versus Faction in the Reformed Presidential Nominating System, PS (October 2016), pp. 704–5; Theda Skocpol and Alex Hertel-Fernandez, “The Koch Network and Republican Party Extremism,” Perspectives on Politics 14, no. 3 (2016), pp. 681–99.


pages: 293 words: 81,183

Doing Good Better: How Effective Altruism Can Help You Make a Difference by William MacAskill

barriers to entry, basic income, Black Swan, Branko Milanovic, Cal Newport, Capital in the Twenty-First Century by Thomas Piketty, carbon footprint, clean water, corporate social responsibility, correlation does not imply causation, Daniel Kahneman / Amos Tversky, David Brooks, effective altruism, en.wikipedia.org, end world poverty, experimental subject, follow your passion, food miles, immigration reform, income inequality, index fund, Intergovernmental Panel on Climate Change (IPCC), Isaac Newton, job automation, job satisfaction, Lean Startup, M-Pesa, mass immigration, meta analysis, meta-analysis, microcredit, Nate Silver, Peter Singer: altruism, purchasing power parity, quantitative trading / quantitative finance, randomized controlled trial, self-driving car, Skype, Stanislav Petrov, Steve Jobs, Steve Wozniak, Steven Pinker, The Future of Employment, The Wealth of Nations by Adam Smith, universal basic income, women in the workforce

Just about anything you do with your time would be more productive. Given the concept of expected value, however, Levitt’s reasoning is too quick. We can’t just say that the chance of affecting the outcome by voting is so small as to be negligible. We need to work out how large the benefit would be if we did indeed affect the outcome. Luckily, some statisticians have done the hard work for us, including political pundit extraordinaire Nate Silver, who correctly predicted the winner of the 2012 election in all fifty states and the District of Columbia. Along with Columbia University professor of statistics Andrew Gelman and Berkeley professor of law Aaron Edlin, Silver calculated the odds of an individual vote swaying the outcome of the 2008 presidential election and found that, on average, a voter had approximately a one-in-sixty-million chance of affecting the outcome—small odds to be sure.

Hsee, and Ned Welch, “Risk as Feelings,” Psychological Bulletin 127, no. 2 (March 2001), 267–86. “Nobody in their right mind”: “Your FREAK-quently asked questions, answered,” Freakonomics (blog), January 20, 2011, http://freakonomics.com/2011/01/20/freakonomics-radio-your-freak-quently-asked-questions-answered/. the odds of an individual vote swaying the outcome of the 2008 presidential election: Andrew Gelman, Nate Silver, and Aaron Edlin, “What Is the Probability Your Vote Will Make a Difference?” Economic Inquiry 50, no. 2 (April 2012), 321–6. The authors reached this estimate by using election forecasts to calculate the probability that (1) a given state is necessary for an electoral college to win, and that (2) the election for that state is exactly tied—the two quantities that jointly determine whether a single vote is decisive in a US presidential election.


pages: 304 words: 82,395

Big Data: A Revolution That Will Transform How We Live, Work, and Think by Viktor Mayer-Schonberger, Kenneth Cukier

23andMe, Affordable Care Act / Obamacare, airport security, barriers to entry, Berlin Wall, big data - Walmart - Pop Tarts, Black Swan, book scanning, business intelligence, business process, call centre, cloud computing, computer age, correlation does not imply causation, dark matter, double entry bookkeeping, Eratosthenes, Erik Brynjolfsson, game design, IBM and the Holocaust, index card, informal economy, intangible asset, Internet of things, invention of the printing press, Jeff Bezos, Joi Ito, lifelogging, Louis Pasteur, Mark Zuckerberg, Menlo Park, Moneyball by Michael Lewis explains big data, Nate Silver, natural language processing, Netflix Prize, Network effects, obamacare, optical character recognition, PageRank, paypal mafia, performance metric, Peter Thiel, post-materialism, random walk, recommendation engine, self-driving car, sentiment analysis, Silicon Valley, Silicon Valley startup, smart grid, smart meter, social graph, speech recognition, Steve Jobs, Steven Levy, the scientific method, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, Thomas Davenport, Turing test, Watson beat the top human players on Jeopardy!

Its accuracy depends on ensuring randomness when collecting the sample data, but achieving such randomness is tricky. Systematic biases in the way the data is collected can lead to the extrapolated results being very wrong. There are echoes of such problems in election polling using landline phones. The sample is biased against people who only use cell-phones (who are younger and more liberal), as the statistician Nate Silver has pointed out. This has resulted in incorrect election predictions. In the 2008 presidential election between Barack Obama and John McCain, the major polling organizations of Gallup, Pew, and ABC/Washington Post found differences of between one and three percentage points when they polled with and without adjusting for cellphone users—a hefty margin considering the tightness of the race. Most troublingly, random sampling doesn’t scale easily to include subcategories, as breaking the results down into smaller and smaller subgroups increases the possibility of erroneous predictions.

Neyman’s famous paper is Jerzy Neyman, “On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection,” Journal of the Royal Statistical Society 97, no. 4 (1934), pp. 558–625. A sample of 1,100 observations is sufficient—Earl Babbie, Practice of Social Research (12th ed. 2010), pp. 204–207. [>] The cellphone effect—“Estimating the Cellphone Effect,” September 20, 2008 (http://www.fivethirtyeight.com/2008/09/estimating-cellphone-effect-22-points.html); for more on polling biases and other statistical insights see Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (Penguin, 2012). [>] Steve Jobs’s gene sequencing—Walter Isaacson, Steve Jobs (Simon and Schuster, 2011), pp. 550–551. [>] Google Flu Trends predicting to city level—Dugas et al., “Google Flu Trends.” Etzioni on temporal data—Interview by Cukier, October 2011. [>] John Kunze quotation—Jonathan Rosenthal, “Special Report: International Banking,” The Economist, May 19, 2012, pp. 7–8.


pages: 356 words: 91,157

The New Urban Crisis: How Our Cities Are Increasing Inequality, Deepening Segregation, and Failing the Middle Class?and What We Can Do About It by Richard Florida

affirmative action, Airbnb, basic income, Bernie Sanders, blue-collar work, business climate, Capital in the Twenty-First Century by Thomas Piketty, clean water, Columbine, congestion charging, creative destruction, David Ricardo: comparative advantage, declining real wages, deindustrialization, Donald Trump, East Village, edge city, Edward Glaeser, failed state, Ferguson, Missouri, Gini coefficient, Google bus, high net worth, income inequality, income per capita, industrial cluster, informal economy, Jane Jacobs, jitney, Kitchen Debate, knowledge economy, knowledge worker, land value tax, low skilled workers, Lyft, megacity, Menlo Park, mortgage tax deduction, Nate Silver, New Economic Geography, new economy, New Urbanism, occupational segregation, Paul Graham, plutocrats, Plutocrats, RAND corporation, rent control, rent-seeking, Richard Florida, rising living standards, Ronald Reagan, secular stagnation, self-driving car, Silicon Valley, sovereign wealth fund, superstar cities, the built environment, The Chicago School, The Death and Life of Great American Cities, the High Line, The Rise and Fall of American Growth, The Wealth of Nations by Adam Smith, Thorstein Veblen, trickle-down economics, Uber and Lyft, uber lyft, universal basic income, upwardly mobile, urban decay, urban planning, urban renewal, urban sprawl, white flight, young professional

Such efforts are sorely needed, given the extent to which blacks are underrepresented in the creative class. Ironically—and troublingly—cities and metro areas can be both more diverse and more segregated at the same time. My own analysis shows that segregation is positively associated with two common measures of diversity: the concentration of gay and lesbian people and the share of the population that is foreign-born.35 A 2015 analysis by Nate Silver comparing the overall ethnic and racial diversity of the nation’s one hundred largest cities to the racial and ethnic segregation of their neighborhoods found segregation to be higher in more racially and ethnically diverse cities.36 Ultimately, the one-two punch of race and economic segregation hits hardest at poor African American neighborhoods. Black Americans are much more likely to live in areas of concentrated poverty than their white peers.

There is no statistical association between the black creative class and income inequality based on the Gini coefficient, compared to a correlation of 0.40 for the white creative class. The black creative class is modestly associated with my measure of overall economic segregation (0.20), but this correlation is much more modest than that for the white creative class (0.66). 35. The Overall Segregation Index is positively associated with the concentration of gay and lesbian households (0.42) and the share of adults who are foreign-born (0.38). 36. Nate Silver, “The Most Diverse Cities Are Often the Most Segregated,” FiveThirtyEight, May 1, 2015, http://fivethirtyeight.com/features/the-most-diverse-cities-are-often-the-most-segregated. 37. Paul A. Jargowsky, “Architecture of Segregation: Civil Unrest, the Concentration of Poverty, and Public Policy,” Century Foundation, August 9, 2015, http://apps.tcf.org/architecture-of-segregation. 38. Racially concentrated areas of poverty are defined as census tracts where more than half of the population is non-white and more than 40 percent live below the poverty line.


pages: 327 words: 88,121

The Vanishing Neighbor: The Transformation of American Community by Marc J. Dunkelman

Affordable Care Act / Obamacare, Albert Einstein, assortative mating, Berlin Wall, big-box store, blue-collar work, Bretton Woods, Broken windows theory, business cycle, call centre, clean water, cuban missile crisis, dark matter, David Brooks, delayed gratification, different worldview, double helix, Downton Abbey, Fall of the Berlin Wall, Filter Bubble, Francis Fukuyama: the end of history, George Santayana, Gini coefficient, glass ceiling, global supply chain, global village, helicopter parent, if you build it, they will come, impulse control, income inequality, invention of movable type, Jane Jacobs, Khyber Pass, Louis Pasteur, Marshall McLuhan, McMansion, Nate Silver, obamacare, Occupy movement, Peter Thiel, post-industrial society, Richard Florida, rolodex, Saturday Night Live, Silicon Valley, Skype, social intelligence, Stanford marshmallow experiment, Steve Jobs, telemarketer, The Chicago School, The Death and Life of Great American Cities, the medium is the message, Tyler Cowen: Great Stagnation, urban decay, urban planning, Walter Mischel, War on Poverty, women in the workforce, World Values Survey, zero-sum game

Viewed at the time as an intellectual successor to Riesman’s wide-ranging interest in broad social relations, Bowling Alone managed nevertheless to be a sterling example of the more contemporary academic sensibility. Moreover, all sorts of interesting conclusions have been gleaned since researchers began to apply data-centered tools to softer disciplines. Just a few years ago, for example, Freakonomics applied the tools of statistics to realms traditionally held outside the main of economic studies—and to great effect.14 More recently, new, sophisticated models and data sets have propelled authors like Nate Silver to international fame, predicting both sporting outcomes and, famously, the outcome of the 2012 presidential election.15 But that new emphasis has orphaned some of the questions earlier generations asked. There are, after all, aspects of our lives that elude rigorous data analysis. There’s no way to measure numerically how Americans balance the divergent impulses toward inner-directedness and other-directedness, or whether they’re more or less driven to conform to the expectations of each varied social environment.

Dan Clawson (Amherst: University of Massachusetts Press, 1998), 19–27. 9Riesman, 1961 introduction to The Lonely Crowd, xlii. 10Riesman, The Lonely Crowd, 24–25. 11Charles McGrath, “Big Thinkster,” New York Times Magazine, December 29, 2002. 12“Interview with Malcolm Gladwell,” Charlie Rose, December 3, 2013. 13McGrath, “Big Thinkster.” 14Steven D. Levitt and Stephen J. Dubner, Freakonomics: A Rogue Economist Explores the Hidden Side of Everything (New York: William Morrow, 2005). 15Chris Cillizza, “Best year in Washington: Nate Silver,” Washington Post, December 28, 2012. 16http://espn.go.com/nba/salaries/_/year/2011. 17Walter Isaacson, Steve Jobs (New York: Simon & Schuster, 2011). 18Daniel Kahneman, Thinking Fast and Slow (New York: Farrar, Straus and Giroux, 2011). Chapter 2: The Third Wave 1John B. Judis, “Newt’s not-so-weird gurus,” New Republic, October 9, 1995. 2http://www.alvintoffler.net/?fa=biospartnership. 3Nathan Gardels, “Lunch with the FT: He has seen the future,” Financial Times, August 19, 2006. 4Alvin Toffler, The Third Wave (New York: Bantam Books, 1981), 9–11. 5http://www.presidency.ucsb.edu/ws/?


pages: 336 words: 95,773

The Theft of a Decade: How the Baby Boomers Stole the Millennials' Economic Future by Joseph C. Sternberg

Affordable Care Act / Obamacare, Airbnb, American Legislative Exchange Council, Asian financial crisis, banking crisis, Basel III, Bernie Sanders, blue-collar work, centre right, corporate raider, Detroit bankruptcy, Donald Trump, Edward Glaeser, employer provided health coverage, Erik Brynjolfsson, eurozone crisis, future of work, gig economy, Gordon Gekko, hiring and firing, Home mortgage interest deduction, housing crisis, job satisfaction, job-hopping, labor-force participation, low skilled workers, Lyft, Marc Andreessen, Mark Zuckerberg, minimum wage unemployment, mortgage debt, mortgage tax deduction, Nate Silver, new economy, obamacare, oil shock, payday loans, pension reform, quantitative easing, Richard Florida, Ronald Reagan, Saturday Night Live, Second Machine Age, sharing economy, Silicon Valley, sovereign wealth fund, TaskRabbit, total factor productivity, Tyler Cowen: Great Stagnation, uber lyft, unpaid internship, women in the workforce

And he devised a system for fully repaying federal debts within thirty-four years (later shortened to twenty-four by Congress) while leaving the new government with the flexibility to suspend principal repayment during emergencies—a flexibility Jeffersonians exploited to finance the Louisiana Purchase and again during the War of 1812 (Robert E. Wright, One Nation Under Debt: Hamilton, Jefferson, and the History of What We Owe [New York: McGraw-Hill, 2008]). † Policy wonk Nate Silver draws an interesting connection between this phenomenon and changing public attitudes toward the government over the latter part of the twentieth century: “We may have gone from conceiving of government as an entity that builds roads, dams and airports, provides shared services like schooling, policing and national parks, and wages wars, into the world’s largest insurance broker. Most of us don’t much care for our insurance broker” (Nate Silver, “Health Care Drives Increase in Government Spending,” New York Times, January 17, 2013). ‡ Civilian employment by all levels of government had plunged in the immediate aftermath of World War II, by around 11 percent from the peak in 1943 to the trough in 1947, but then nearly tripled between 1947 and 1980, with some of the biggest single-year jumps occurring in the 1960s (US Bureau of Labor Statistics, All Employees: Government, retrieved from Federal Reserve Bank of St.


pages: 393 words: 91,257

The Coming of Neo-Feudalism: A Warning to the Global Middle Class by Joel Kotkin

Admiral Zheng, Andy Kessler, autonomous vehicles, basic income, Bernie Sanders, call centre, Capital in the Twenty-First Century by Thomas Piketty, carbon footprint, Cass Sunstein, clean water, creative destruction, deindustrialization, demographic transition, don't be evil, Donald Trump, edge city, Elon Musk, European colonialism, financial independence, Francis Fukuyama: the end of history, gig economy, Gini coefficient, Google bus, guest worker program, Hans Rosling, housing crisis, income inequality, informal economy, Jane Jacobs, Jaron Lanier, Jeff Bezos, job automation, job satisfaction, Joseph Schumpeter, land reform, liberal capitalism, life extension, low skilled workers, Lyft, Mark Zuckerberg, market fundamentalism, Martin Wolf, mass immigration, megacity, Nate Silver, new economy, New Urbanism, Occupy movement, Parag Khanna, Peter Thiel, plutocrats, Plutocrats, post-industrial society, post-work, postindustrial economy, postnationalism / post nation state, precariat, profit motive, RAND corporation, Ray Kurzweil, rent control, Richard Florida, road to serfdom, Robert Gordon, Sam Altman, Satyajit Das, sharing economy, Silicon Valley, smart cities, Steve Jobs, Stewart Brand, superstar cities, The Death and Life of Great American Cities, The Future of Employment, The Rise and Fall of American Growth, Thomas L Friedman, too big to fail, trade route, Travis Kalanick, Uber and Lyft, uber lyft, universal basic income, unpaid internship, upwardly mobile, We are the 99%, Wolfgang Streeck, women in the workforce, working-age population, Y Combinator

., 511, 551; Dmitri Volkogonov, Autopsy for Empire: The Seven Leaders Who Built the Soviet Regime (New York: Free Press, 1998), 63. 27 Eric D. Weitz, Weimar Germany: Promise and Tragedy (Princeton: Princeton University Press, 2007), 334. 28 Frederic Spotts, Hitler and the Power of Aesthetics (New York: Overlook Press, 2003), 30, 79–82, 98. 29 Klaus P. Fischer, Nazi Germany: A New History (New York: Continuum, 1996), 17. 30 Nate Silver, “There Really Was a Liberal Media Bubble,” FiveThirtyEight, March 10, 2017, https://fivethirtyeight.com/features/there-really-was-a-liberal-media-bubble/; “Media Bias: Pretty Much All of Journalism Now Leans Left, Study Shows,” Investor’s Business Daily, November 16, 2018, https://www.investors.com/politics/editorials/media-bias-left-study/. 31 Christopher M. Finan, “A Shameful Season for American Journalism,” Wall Street Journal, September 24, 2018, https://www.wsj.com/articles/a-shameful-season-for-american-journalism-1537830679; Justin Ward, “The death of the working class reporter,” Medium, June 25, 2019, https://blog.usejournal.com/the-death-of-the-working-class-reporter-48b467300f4d; Amee LaTour, “Do 97 percent of journalist donations go to Democrats?”

Malcolm Debevoise (New Haven: Yale University Press, 2019), 24. 25 “The far right’s new fascination with the Middle Ages,” Economist, January 2, 2017, https://www.economist.com/blogs/democracyinamerica/2017/01/medieval-memes. 26 Guilluy, Twilight of the Elites, 43. 27 Neil Munro, “Billionaire Steve Case says immigrants will offset middle class job losses,” Daily Caller, December 5, 2013, https://dailycaller.com/2013/12/05/billionaire-steve-case-says-immigrants-will-offset-middle-class-job-losses/. 28 Alex Pfeiffer, “Bill Kristol Says ‘Lazy’ White Working Class Should Be Replaced by ‘New Americans,’” Daily Caller, February 8, 2017, https://dailycaller.com/2017/02/08/bill-kristol-says-lazy-white-working-class-should-be-replaced-by-new-americans/. 29 Geoff Colvin, “Donald Trump’s Immigration Ban Ushers In a New Era of CEO Activism,” Fortune, February 7, 2017, http://fortune.com/2017/02/07/donald-trumps-immigration-ban-ushers-in-a-new-era-of-ceo-activism/. 30 Douglas Murray, The Strange Death of Europe: Immigration, Identity, Islam (London: Bloomsbury, 2017), 99, 226. 31 Giles Kepel, remarks at the Tocqueville Conversations, Château de Tocqueville, Normandy, France, June 7–8, 2018. 32 Goodhart, The Road to Somewhere, 3–4, 100; “The Brexit Index: a who’s who of Remain and Leave supporters,” Populus, https://www.populus.co.uk/insights/2016/05/the-brexit-index-a-whos-who-of-remain-and-leave-supporters/; House of Commons Library, “General Election 2019: Results and Analysis,” Number CBP 8749, 28 January 2020. 33 Peter Foster, “Denmark’s EU referendum is a blow to David Cameron,” Telegraph, December 4, 2015, https://www.telegraph.co.uk/news/worldnews/europe/denmark/12032958/Denmarks-EU-referendum-is-a-blow-to-David-Cameron.html; “Dutch referendum voters overwhelmingly reject closer EU links to Ukraine,” Guardian, April 7, 2016, https://www.theguardian.com/world/2016/apr/06/dutch-voters-reject-closer-eu-links-to-ukraine-in-referendum; Nick Gutteridge, “European Superstate to be unveiled: EU nations ‘to be morphed into one’ post-Brexit,” Express, June 29, 2016, https://www.express.co.uk/news/politics/683739/EU-referendum-German-French-European-superstate-Brexit; Project 28, “Handling the Immigration Crisis,” Századvég Foundation, http://project28.eu/. 34 Matthew Karnitschnig, “Cologne puts Germany’s ‘lying press’ on defensive,” Politico, January 25, 2016, https://www.politico.eu/article/cologne-puts-germany-lying-media-press-on-defensive-migration-refugees-attacks-sex-assault-nye/; Robert Spencer, “Google manipulates Search Results to Conceal Criticism of Islam and Jihad,” PJ Media, August 2, 2017, https://pjmedia.com/homeland-security/2017/08/02/google-manipulates-search-results-to-conceal-criticism-of-islam-and-jihad/; “Rome opens its gates to the modern barbarians,” Financial Times, May 15, 2018, https://www.ft.com/content/6348cc64-5764-11e8-b8b2-d6ceb45fa9d0. 35 Goodhart, The Road to Somewhere, 14. 36 Robert Samuelson, “The Middle Class Rocks—Again,” Real Clear Politics, September 18, 2017, https://www.realclearpolitics.com/articles/2017/09/18/the_middle_class_rocks-again_135014.html; Nate Silver, “Silver Bulletpoints: The Union Vote Could Swing the Election,” FiveThirtyEight, May 2, 2019, https://fivethirtyeight.com/features/silver-bulletpoints-the-union-vote-could-swing-the-election/. 37 Richard Florida, “Why Is Your State Red or Blue? Look to the Dominant Occupational Class,” City Lab, November 28, 2018, https://www.citylab.com/life/2018/11/state-voting-patterns-occupational-class-data-politics/575047/. 38 John Daniel Davidson, “Trump is no fascist.


pages: 331 words: 104,366

Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins by Garry Kasparov

3D printing, Ada Lovelace, AI winter, Albert Einstein, AltaVista, barriers to entry, Berlin Wall, business process, call centre, Charles Lindbergh, clean water, computer age, Daniel Kahneman / Amos Tversky, David Brooks, Donald Trump, Douglas Hofstadter, Drosophila, Elon Musk, Erik Brynjolfsson, factory automation, Freestyle chess, Gödel, Escher, Bach, job automation, Leonard Kleinrock, low earth orbit, Mikhail Gorbachev, Nate Silver, Norbert Wiener, packet switching, pattern recognition, Ray Kurzweil, Richard Feynman, rising living standards, rolodex, Second Machine Age, self-driving car, Silicon Valley, Silicon Valley startup, Skype, speech recognition, stem cell, Stephen Hawking, Steven Pinker, technological singularity, The Coming Technological Singularity, The Signal and the Noise by Nate Silver, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, zero-sum game

This hypothesis was proposed by Murray Campbell at least as early as Monty Newborn’s 2002 book on Deep Blue. The punch line to his theory was that Deep Blue’s mysterious move wasn’t profound at all; it was a blunder and the result of a yet another bug. Per Campbell and Hsu, the move was “random,” the result of a known bug they had failed to kill before the match began. This tale acquired new life when election analyst Nate Silver used it as the centerpiece for an entire chapter of his 2012 book, The Signal and the Noise. The narrative suggested by Frederic and spread by Campbell was irresistible: Kasparov lost to Deep Blue because of a bug! Writes Silver, “The bug was anything but unfortunate for Deep Blue: it was likely what allowed the computer to beat Kasparov.” TIME, Wired, and other outlets ran with breathless variations on this theme, each story containing more errors about chess and more silly assumptions about my mental state than the last.

I will not repeat here the stream of profanities in Russian, English, and languages not yet invented that escaped my lips when I first read that paragraph. What in the hell was this? Two paragraphs after Illescas says IBM had hired Russian speakers to spy on me, he says the team entered this critical line into Deep Blue’s book that morning? An obscure variation that I had only discussed with my team in the privacy of our suite at the Plaza Hotel that week in New York? I’m no Nate Silver, but the odds of winning the lottery are quite attractive in comparison to those of the Deep Blue team entering a specific variation I had never played before in my life into the computer’s book on the very same day it appeared on the board in the final game. And not only preparing the machine for the 4..Nd7 Caro-Kann—even during my brief dalliance with the Caro-Kann as a fifteen-year-old I played the 4..Bf5 line exclusively—but also forcing it to play 8.Nxe6 and doing this despite generally giving Deep Blue “a lot of freedom to play,” in Illescas’s own words.


pages: 185 words: 43,609

Zero to One: Notes on Startups, or How to Build the Future by Peter Thiel, Blake Masters

Airbnb, Albert Einstein, Andrew Wiles, Andy Kessler, Berlin Wall, cleantech, cloud computing, crony capitalism, discounted cash flows, diversified portfolio, don't be evil, Elon Musk, eurozone crisis, income inequality, Jeff Bezos, Lean Startup, life extension, lone genius, Long Term Capital Management, Lyft, Marc Andreessen, Mark Zuckerberg, minimum viable product, Nate Silver, Network effects, new economy, paypal mafia, Peter Thiel, pets.com, profit motive, Ralph Waldo Emerson, Ray Kurzweil, self-driving car, shareholder value, Silicon Valley, Silicon Valley startup, Singularitarianism, software is eating the world, Steve Jobs, strong AI, Ted Kaczynski, Tesla Model S, uber lyft, Vilfredo Pareto, working poor

But in an indefinite world, people actually prefer unlimited optionality; money is more valuable than anything you could possibly do with it. Only in a definite future is money a means to an end, not the end itself. Indefinite Politics Politicians have always been officially accountable to the public at election time, but today they are attuned to what the public thinks at every moment. Modern polling enables politicians to tailor their image to match preexisting public opinion exactly, so for the most part, they do. Nate Silver’s election predictions are remarkably accurate, but even more remarkable is how big a story they become every four years. We are more fascinated today by statistical predictions of what the country will be thinking in a few weeks’ time than by visionary predictions of what the country will look like 10 or 20 years from now. And it’s not just the electoral process—the very character of government has become indefinite, too.


pages: 428 words: 121,717

Warnings by Richard A. Clarke

active measures, Albert Einstein, algorithmic trading, anti-communist, artificial general intelligence, Asilomar, Asilomar Conference on Recombinant DNA, Bernie Madoff, cognitive bias, collateralized debt obligation, complexity theory, corporate governance, cuban missile crisis, data acquisition, discovery of penicillin, double helix, Elon Musk, failed state, financial thriller, fixed income, Flash crash, forensic accounting, friendly AI, Intergovernmental Panel on Climate Change (IPCC), Internet of things, James Watt: steam engine, Jeff Bezos, John Maynard Keynes: Economic Possibilities for our Grandchildren, knowledge worker, Maui Hawaii, megacity, Mikhail Gorbachev, money market fund, mouse model, Nate Silver, new economy, Nicholas Carr, nuclear winter, pattern recognition, personalized medicine, phenotype, Ponzi scheme, Ray Kurzweil, Richard Feynman, Richard Feynman: Challenger O-ring, risk tolerance, Ronald Reagan, Sam Altman, Search for Extraterrestrial Intelligence, self-driving car, Silicon Valley, smart grid, statistical model, Stephen Hawking, Stuxnet, technological singularity, The Future of Employment, the scientific method, The Signal and the Noise by Nate Silver, Tunguska event, uranium enrichment, Vernor Vinge, Watson beat the top human players on Jeopardy!, women in the workforce, Y2K

However, such a response does little to help prepare for future disaster or to ensure that Cassandra’s warnings do not again go unheeded. As it seemed we weren’t the only ones who had noticed Cassandras in our midst, we then wondered if there existed any scholarly research on the topic of predictions. In fact, prediction is something that academics have spent a lot of time studying and considering. The statistician Nate Silver has taken a highly quantitative approach to prediction, one that works for a certain class of event. The jurist Richard Posner examined the phenomenon of catastrophes in the years after 9/11. Psychologists like Dan Ariely and Tsachi Ein-Dor have probed the way our brains work (and don’t) through empirical observation and the study of warnings. Unquestionably one of the foundational works in this area, predictions within the social sciences, is Philip Tetlock’s Expert Political Judgment.

They did perform somewhat better than undergraduates subjected to the same exercises, and they outperformed the proverbial “chimp with a dart board,” but they didn’t come close to the predictive accuracy of formal statistical models. Later books have looked at Tetlock’s foundational results in some additional detail. Dan Gardner’s 2012 Future Babble draws on recent research in psychology, neuroscience, and behavioral economics to detail the biases and other cognitive processes that skew our judgment when we try to make predictions about the future. And building on a successful career in sports and political forecasting, Nate Silver discusses in his book, The Signal and the Noise, how thinking more probabilistically can help us distill more accurate predictions from a sea of raw data. Fundamentally, these books all identify the difficulties inherent in trying to see into the future. Predicting natural phenomena is stymied by the chaotic nature of the universe: natural processes are nonlinear systems driven by feedback loops that are often inherently unpredictable themselves.


pages: 588 words: 131,025

The Patient Will See You Now: The Future of Medicine Is in Your Hands by Eric Topol

23andMe, 3D printing, Affordable Care Act / Obamacare, Anne Wojcicki, Atul Gawande, augmented reality, bioinformatics, call centre, Clayton Christensen, clean water, cloud computing, commoditize, computer vision, conceptual framework, connected car, correlation does not imply causation, creative destruction, crowdsourcing, dark matter, data acquisition, disintermediation, disruptive innovation, don't be evil, Edward Snowden, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Firefox, global village, Google Glasses, Google X / Alphabet X, Ignaz Semmelweis: hand washing, information asymmetry, interchangeable parts, Internet of things, Isaac Newton, job automation, Julian Assange, Kevin Kelly, license plate recognition, lifelogging, Lyft, Mark Zuckerberg, Marshall McLuhan, meta analysis, meta-analysis, microbiome, Nate Silver, natural language processing, Network effects, Nicholas Carr, obamacare, pattern recognition, personalized medicine, phenotype, placebo effect, RAND corporation, randomized controlled trial, Second Machine Age, self-driving car, Silicon Valley, Skype, smart cities, Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia, Snapchat, social graph, speech recognition, stealth mode startup, Steve Jobs, the scientific method, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, Turing test, Uber for X, uber lyft, Watson beat the top human players on Jeopardy!, WikiLeaks, X Prize

Although Eisenstein stopped short of claiming that the first industrial revolution was an outgrowth of the printing press, many others have claimed this. Marshall McLuhan, in The Gutenberg Galaxy, wrote: “The invention of typography confirmed and extended the new visual stress of applied knowledge, providing the first uniformly repeatable commodity, the first assembly-line, and the first mass-production.”11 More recently, Nate Silver, in The Signal and the Noise, asserted that the industrial revolution of 1775 was sparked by the printing press, whereby the economic growth rate that was stagnant at 0.1 percent per year then grew faster than the growth rate of the population.12 But I prefer to principally assess the Gutenberg transformative effects by the specific attributes that they induced or cultivated instead of as a precursor for subsequent momentous periods in history.

Table 3.1 summarizes the predominant attributes affected by Gutenberg’s press and compares them to the smartphone, or what some term as the “phablet” (a combination of smartphone and tablet). At first glance you will note a check mark in every box. But here I’ll explain why the parallels can be seen as striking. Without question, APP was associated with an explosion of knowledge and too much information. So too is our era. Back in the fifteenth century, as Nate Silver summed up, “the amount of information was increasing much more rapidly than our understanding of what to do with it, or our ability to differentiate the useful information from the mistruths.”13 Here in the twenty-first century, we call that “big data,” with more data generated in the past two years than in the history of humankind. And an ever-increasing proportion of that is derived from and is passing through mobile devices.


pages: 500 words: 145,005

Misbehaving: The Making of Behavioral Economics by Richard H. Thaler

"Robert Solow", 3Com Palm IPO, Albert Einstein, Alvin Roth, Amazon Mechanical Turk, Andrei Shleifer, Apple's 1984 Super Bowl advert, Atul Gawande, Berlin Wall, Bernie Madoff, Black-Scholes formula, business cycle, capital asset pricing model, Cass Sunstein, Checklist Manifesto, choice architecture, clean water, cognitive dissonance, conceptual framework, constrained optimization, Daniel Kahneman / Amos Tversky, delayed gratification, diversification, diversified portfolio, Edward Glaeser, endowment effect, equity premium, Eugene Fama: efficient market hypothesis, experimental economics, Fall of the Berlin Wall, George Akerlof, hindsight bias, Home mortgage interest deduction, impulse control, index fund, information asymmetry, invisible hand, Jean Tirole, John Nash: game theory, John von Neumann, Kenneth Arrow, Kickstarter, late fees, law of one price, libertarian paternalism, Long Term Capital Management, loss aversion, market clearing, Mason jar, mental accounting, meta analysis, meta-analysis, money market fund, More Guns, Less Crime, mortgage debt, Myron Scholes, Nash equilibrium, Nate Silver, New Journalism, nudge unit, Paul Samuelson, payday loans, Ponzi scheme, presumed consent, pre–internet, principal–agent problem, prisoner's dilemma, profit maximization, random walk, randomized controlled trial, Richard Thaler, Robert Shiller, Robert Shiller, Ronald Coase, Silicon Valley, South Sea Bubble, Stanford marshmallow experiment, statistical model, Steve Jobs, Supply of New York City Cabdrivers, technology bubble, The Chicago School, The Myth of the Rational Market, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, Thomas Kuhn: the structure of scientific revolutions, transaction costs, ultimatum game, Vilfredo Pareto, Walter Mischel, zero-sum game

But a snappy retort to the “as if” critique was far from sufficient; to win the argument I would need hard empirical evidence that would convince economists. To this day, the phrase “survey evidence” is rarely heard in economics circles without the necessary adjective “mere,” which rhymes with “sneer.” This disdain is simply unscientific. Polling data, which just comes from asking people whether they are planning to vote and for whom, when carefully used by skilled statisticians such as Nate Silver, yield remarkably accurate predictions of elections. The most amusing aspect of this anti-survey attitude is that many important macroeconomic variables are produced by surveys! For instance, in America the press often obsesses over the monthly announcement of the latest “jobs” data, with serious-looking economists asked to weigh in about how to interpret the figures. Where do these jobs numbers come from?

Fans can follow the “New York Times 4th Down Bot” in real time and see what the math says a team should be doing. So what effect has this research plus a free app had on the behavior of football coaches? Essentially none. Since Romer wrote his paper, the frequency of going for it on fourth down has marginally gone down, meaning that teams have gotten dumber! (Similarly, there has been no noticeable change in teams’ draft strategy since our paper came out.) Nate Silver, the ex–sports analytics junkie who became famous for his political forecasts and for the excellent book The Signal and the Noise, estimates that bad fourth-down decisions cost a football team an average of half a win per season. The Times analysts estimate it to be closer to two-thirds of a win per year. That may not seem like a lot, but the season is only sixteen games. A team can win an extra game every other year just by making the smart decision two or three times a game, one they can even check online if they need help.¶ Of course, coaches are Humans.


pages: 475 words: 134,707

The Hype Machine: How Social Media Disrupts Our Elections, Our Economy, and Our Health--And How We Must Adapt by Sinan Aral

Airbnb, Albert Einstein, Any sufficiently advanced technology is indistinguishable from magic, augmented reality, Bernie Sanders, bitcoin, carbon footprint, Cass Sunstein, computer vision, coronavirus, correlation does not imply causation, COVID-19, Covid-19, crowdsourcing, cryptocurrency, death of newspapers, disintermediation, Donald Trump, Drosophila, Edward Snowden, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, experimental subject, facts on the ground, Filter Bubble, global pandemic, hive mind, illegal immigration, income inequality, Kickstarter, knowledge worker, longitudinal study, low skilled workers, Lyft, Mahatma Gandhi, Mark Zuckerberg, Menlo Park, meta analysis, meta-analysis, Metcalfe’s law, mobile money, move fast and break things, move fast and break things, multi-sided market, Nate Silver, natural language processing, Network effects, performance metric, phenotype, recommendation engine, Robert Bork, Robert Shiller, Robert Shiller, Second Machine Age, sentiment analysis, shareholder value, skunkworks, Snapchat, social graph, social intelligence, social software, social web, statistical model, stem cell, Stephen Hawking, Steve Jobs, Telecommunications Act of 1996, The Chicago School, The Wisdom of Crowds, theory of mind, Tim Cook: Apple, Uber and Lyft, uber lyft, WikiLeaks, Yogi Berra

Two studies released by the Senate Intelligence Committee in 2019 detail the reach and scope of Russian misinformation campaigns targeting hundreds of millions of U.S. citizens with the aim of affecting turnout and vote choice in the 2016 presidential election. The twin reports highlight, but do not answer, perhaps the most important question facing democracy in the digital age: to what extent are democratic elections vulnerable to social media manipulation? Journalists and academics have weighed in on this question, often with confident yet contradictory conclusions. Nate Silver, founder and editor in chief of the site FiveThirtyEight, remarked that “if you wrote out a list of the most important factors in the 2016 election, I’m not sure that Russian social media memes would be among the top 100.” Some academics are similarly skeptical, arguing that “Russia-sponsored content on social media likely did not decide the election” because Russian-linked spending and exposure to fake news were relatively small-scale.

Howard et al., The IRA, Social Media and Political Polarization in the United States, 2012–2018, Investigation of Russian Interference prepared for the U.S. Senate Select Committee on Intelligence (Graphika, 2019), https://int.nyt.com/​data/​documenthelper/​534-oxford-russia-internet-research-agency/​c6588b4a7b940c551c38/​optimized/​full.pdf. “if you wrote out a list of the most important factors”: Nate Silver, tweet, December 17, 2018, https://twitter.com/​natesilver538/​status/​1074833714931224582?lang=en. “Russia-sponsored content on social media likely”: John Sides, Michael Tesler, and Lynn Vavreck, Identity Crisis: The 2016 Presidential Campaign and the Battle for the Meaning of America (Princeton: Princeton University Press, 2018). Russian-linked spending and exposure to fake news: Hunt Allcott and Matthew Gentzkow, “Social Media and Fake News in the 2016 Election,” Journal of Economic Perspectives 31, no. 2 (2017): 211–36; Andrew Guess, Brendan Nyhan, and Jason Reifler, “Selective Exposure to Misinformation: Evidence from the Consumption of Fake News During the 2016 US Presidential Campaign,” European Research Council (2018), 9; N.


pages: 184 words: 53,625

Future Perfect: The Case for Progress in a Networked Age by Steven Johnson

Airbus A320, airport security, algorithmic trading, banking crisis, barriers to entry, Bernie Sanders, call centre, Captain Sullenberger Hudson, Cass Sunstein, Charles Lindbergh, cognitive dissonance, credit crunch, crowdsourcing, dark matter, Dava Sobel, David Brooks, Donald Davies, future of journalism, hive mind, Howard Rheingold, HyperCard, Jane Jacobs, John Gruber, John Harrison: Longitude, Joi Ito, Kevin Kelly, Kickstarter, lone genius, Mark Zuckerberg, mega-rich, meta analysis, meta-analysis, Naomi Klein, Nate Silver, Occupy movement, packet switching, peer-to-peer, Peter Thiel, planetary scale, pre–internet, RAND corporation, risk tolerance, shareholder value, Silicon Valley, Silicon Valley startup, social graph, Steve Jobs, Steven Pinker, Stewart Brand, The Death and Life of Great American Cities, Tim Cook: Apple, urban planning, US Airways Flight 1549, WikiLeaks, William Langewiesche, working poor, X Prize, your tax dollars at work

Sites such as Talking Points Memo and Politico did extensive direct reporting. Daily Kos provided in-depth surveys and field reports on state races that the Times would never have had the ink to cover. Individual bloggers such as Andrew Sullivan responded to each twist in the news cycle; The Huffington Post culled the most provocative opinion pieces from the rest of the blogosphere. The statistician Nate Silver at the website Five Thirty Eight did meta-analysis of polling that exceeded anything Bill Schneider dreamed of doing on CNN in 1992. When the banking crisis erupted in September 2008, I followed economist bloggers such as Brad DeLong to get their expert take on the candidates’ responses to the crisis. I watched the debates with a thousand virtual friends live-tweeting alongside me on the couch.


pages: 499 words: 144,278

Coders: The Making of a New Tribe and the Remaking of the World by Clive Thompson

2013 Report for America's Infrastructure - American Society of Civil Engineers - 19 March 2013, 4chan, 8-hour work day, Ada Lovelace, AI winter, Airbnb, Amazon Web Services, Asperger Syndrome, augmented reality, Ayatollah Khomeini, barriers to entry, basic income, Bernie Sanders, bitcoin, blockchain, blue-collar work, Brewster Kahle, Brian Krebs, Broken windows theory, call centre, cellular automata, Chelsea Manning, clean water, cloud computing, cognitive dissonance, computer vision, Conway's Game of Life, crowdsourcing, cryptocurrency, Danny Hillis, David Heinemeier Hansson, don't be evil, don't repeat yourself, Donald Trump, dumpster diving, Edward Snowden, Elon Musk, Erik Brynjolfsson, Ernest Rutherford, Ethereum, ethereum blockchain, Firefox, Frederick Winslow Taylor, game design, glass ceiling, Golden Gate Park, Google Hangouts, Google X / Alphabet X, Grace Hopper, Guido van Rossum, Hacker Ethic, HyperCard, illegal immigration, ImageNet competition, Internet Archive, Internet of things, Jane Jacobs, John Markoff, Jony Ive, Julian Assange, Kickstarter, Larry Wall, lone genius, Lyft, Marc Andreessen, Mark Shuttleworth, Mark Zuckerberg, Menlo Park, microservices, Minecraft, move fast and break things, move fast and break things, Nate Silver, Network effects, neurotypical, Nicholas Carr, Oculus Rift, PageRank, pattern recognition, Paul Graham, paypal mafia, Peter Thiel, pink-collar, planetary scale, profit motive, ransomware, recommendation engine, Richard Stallman, ride hailing / ride sharing, Rubik’s Cube, Ruby on Rails, Sam Altman, Satoshi Nakamoto, Saturday Night Live, self-driving car, side project, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, single-payer health, Skype, smart contracts, Snapchat, social software, software is eating the world, sorting algorithm, South of Market, San Francisco, speech recognition, Steve Wozniak, Steven Levy, TaskRabbit, the High Line, Travis Kalanick, Uber and Lyft, Uber for X, uber lyft, universal basic income, urban planning, Wall-E, Watson beat the top human players on Jeopardy!, WikiLeaks, women in the workforce, Y Combinator, Zimmermann PGP, éminence grise

Private Sector Myths (London: Anthem Press, 2015), 70. the general population: Peter Ryan, “Left, Right and Center: Crypto Isn’t Just for Libertarians Anymore,” CoinDesk, July 27, 2018, https://www.coindesk.com/no-crypto-isnt-just-for-libertarians-anymore; Nate Silver, “There Are Few Libertarians. But Many Americans Have Libertarian Views,” FiveThirtyEight, April 9, 2015, https://fivethirtyeight.com/features/there-are-few-libertarians-but-many-americans-have-libertarian-views/; both accessed October 7, 2018">https://www.coindesk.com/no-crypto-isnt-just-for-libertarians-anymore; Nate Silver, “There Are Few Libertarians. But Many Americans Have Libertarian Views,” FiveThirtyEight, April 9, 2015, https://fivethirtyeight.com/features/there-are-few-libertarians-but-many-americans-have-libertarian-views/; both accessed October 7, 2018.


pages: 590 words: 152,595

Army of None: Autonomous Weapons and the Future of War by Paul Scharre

active measures, Air France Flight 447, algorithmic trading, artificial general intelligence, augmented reality, automated trading system, autonomous vehicles, basic income, brain emulation, Brian Krebs, cognitive bias, computer vision, cuban missile crisis, dark matter, DARPA: Urban Challenge, DevOps, drone strike, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, facts on the ground, fault tolerance, Flash crash, Freestyle chess, friendly fire, IFF: identification friend or foe, ImageNet competition, Internet of things, Johann Wolfgang von Goethe, John Markoff, Kevin Kelly, Loebner Prize, loose coupling, Mark Zuckerberg, moral hazard, mutually assured destruction, Nate Silver, pattern recognition, Rodney Brooks, Rubik’s Cube, self-driving car, sensor fusion, South China Sea, speech recognition, Stanislav Petrov, Stephen Hawking, Steve Ballmer, Steve Wozniak, Stuxnet, superintelligent machines, Tesla Model S, The Signal and the Noise by Nate Silver, theory of mind, Turing test, universal basic income, Valery Gerasimov, Wall-E, William Langewiesche, Y2K, zero day

Unlike AlphaGo’s 1 in 10,000 surprise move that later turned out to be a stroke of brilliance, Kasparov could see right away that Deep Blue’s 44th move was tactically nonsensical. Deep Blue resigned the game one move later. Later that evening while pouring over a recreation of the final moves, Kasparov discovered that in 20 moves he would have checkmated Deep Blue. The implication was that Deep Blue made a nonsense move and resigned because it could see 20 moves ahead, a staggering advantage in chess. Nate Silver reports that this bug may have irreparably shaken Kasparov’s confidence. Nate Silver, The Signal and the Noise: Why So Many Predictions Fail (New York: Penguin, 2015), 276–289. 150 recent UNIDIR report on autonomous weapons and risk: UN Institute for Disarmament Research, “Safety, Unintentional Risk and Accidents in the Weaponization of Increasingly Autonomous Technologies,” 2016, http://www.unidir.org/files/publications/pdfs/safety-unintentional-risk-and-accidents-en-668.pdf.


pages: 558 words: 168,179

Dark Money: The Hidden History of the Billionaires Behind the Rise of the Radical Right by Jane Mayer

affirmative action, Affordable Care Act / Obamacare, American Legislative Exchange Council, anti-communist, Bakken shale, bank run, battle of ideas, Berlin Wall, Capital in the Twenty-First Century by Thomas Piketty, carried interest, centre right, clean water, Climategate, Climatic Research Unit, collective bargaining, corporate raider, crony capitalism, David Brooks, desegregation, diversified portfolio, Donald Trump, energy security, estate planning, Fall of the Berlin Wall, George Gilder, housing crisis, hydraulic fracturing, income inequality, Intergovernmental Panel on Climate Change (IPCC), invisible hand, job automation, low skilled workers, mandatory minimum, market fundamentalism, mass incarceration, Mont Pelerin Society, More Guns, Less Crime, Nate Silver, New Journalism, obamacare, Occupy movement, offshore financial centre, oil shale / tar sands, oil shock, plutocrats, Plutocrats, Powell Memorandum, Ralph Nader, Renaissance Technologies, road to serfdom, Robert Mercer, Ronald Reagan, school choice, school vouchers, The Bell Curve by Richard Herrnstein and Charles Murray, The Chicago School, the scientific method, University of East Anglia, Unsafe at Any Speed, War on Poverty, working poor

A series of loudspeakers formed a fence around an outdoor pavilion in which the donors met, emitting static toward the outside world, to prevent eavesdropping. Or so they thought until a reporter for Mother Jones, Brad Friedman, obtained an audio recording of the weekend’s highlights and published a transcript. As they gathered in the foothills of the Rockies, the donors had ample reason for optimism. The New York Times’s resident number cruncher, Nate Silver, who handicapped political odds with the unsentimental eye of a racetrack bookie, was openly asking, “Is Obama toast?” After analyzing Obama’s sagging approval rating and the economy’s lagging indicators, he concluded that Obama had gone from “a modest favorite to win re-election to, probably, a slight underdog.” If the Republicans chose a weak candidate or the economy miraculously revived, he noted, this could change.

“With no basis in fact”: Mann and Ornstein, It’s Even Worse Than It Looks, 23. Cantor later told: Ryan Lizza, “The House of Pain,” New Yorker, March 4, 2013. “I think he came in truly trying”: Neera Tanden, interview with author. CHAPTER TWELVE: MOTHER OF ALL WARS Or so they thought: Brad Friedman, “Inside the Koch Brothers’ 2011 Summer Seminar,” The Brad Blog, June 26, 2011. The New York Times’s resident: Nate Silver, “Is Obama Toast? Handicapping the 2012 Election,” New York Times Magazine, Nov. 3, 2011. “Wouldn’t it be easier”: Halperin and Heilemann, Double Down, 345. Four years later: For more on Christie’s record, see Cezary Podkul and Allan Sloan, “Christie Closed Budget Gaps with One-Shot Maneuvers,” Washington Post, April 18, 2015, A1. “Who knows?”: Friedman, “Inside the Koch Brothers’ 2011 Summer Seminar.”


pages: 250 words: 64,011

Everydata: The Misinformation Hidden in the Little Data You Consume Every Day by John H. Johnson

Affordable Care Act / Obamacare, Black Swan, business intelligence, Carmen Reinhart, cognitive bias, correlation does not imply causation, Daniel Kahneman / Amos Tversky, Donald Trump, en.wikipedia.org, Kenneth Rogoff, labor-force participation, lake wobegon effect, Long Term Capital Management, Mercator projection, Mercator projection distort size, especially Greenland and Africa, meta analysis, meta-analysis, Nate Silver, obamacare, p-value, PageRank, pattern recognition, publication bias, QR code, randomized controlled trial, risk-adjusted returns, Ronald Reagan, selection bias, statistical model, The Signal and the Noise by Nate Silver, Thomas Bayes, Tim Cook: Apple, wikimedia commons, Yogi Berra

“There is no scientifically plausible way of predicting the occurrence of a particular earthquake,” they note, adding that “prediction, as people expect it, requires predicting the magnitude, timing, and location of the future earthquake, which is not currently possible.”17 We simply don’t have the data, nor do we have the technology, to accurately predict quakes at this time. That said, the USGS does describe the places “most likely to produce earthquakes in the long term.” They call this forecasting, when they estimate the likelihood of a seismic event occurring over a period of time. This brings us to the distinction—or lack thereof—between a prediction and a forecast. As Nate Silver notes in The Signal and the Noise, the terms are used differently by some (most notably seismologists, who study earthquakes) but interchangeably by others. Some would argue that predictions are binary—something will or won’t happen—while forecasts are more probabilistic—there’s an X percent chance that something will happen. (To further complicate the issue, an estimate may be used when talking about past, current, or future data.)


Bulletproof Problem Solving by Charles Conn, Robert McLean

active transport: walking or cycling, Airbnb, Amazon Mechanical Turk, asset allocation, availability heuristic, Bayesian statistics, Black Swan, blockchain, business process, call centre, carbon footprint, cloud computing, correlation does not imply causation, Credit Default Swap, crowdsourcing, David Brooks, Donald Trump, Elon Musk, endowment effect, future of work, Hyperloop, Innovator's Dilemma, inventory management, iterative process, loss aversion, meta analysis, meta-analysis, Nate Silver, nudge unit, Occam's razor, pattern recognition, pets.com, prediction markets, principal–agent problem, RAND corporation, randomized controlled trial, risk tolerance, Silicon Valley, smart contracts, stem cell, the rule of 72, the scientific method, The Signal and the Noise by Nate Silver, time value of money, transfer pricing, Vilfredo Pareto, walkable city, WikiLeaks

Todd, and the ABC Research Group, Simple Heuristics That Make Us Smart (Oxford University Press, 2000). 3  Report prepared for the United Kingdom's Department of International Development by The Nature Conservancy, WWF, and the University of Manchester, “Improving Hydropower Outcomes through System Scale Planning, An Example from Myanmar,” 2016. 4  Warren Buffett, “My Philanthropic Pledge,” Fortune, June 16, 2010. 5  Our friend Barry Nalebuff of Yale points out that the actual rule is 69.3, but is usually rounded up to 72 because it is easier to do the division in your head. 6  CB Insights, May 25, 2015, www.cbinsights.com. 7  Nate Silver, The Signal and the Noise (Penguin, 2012). 8  Dan Lovallo, Carmina Clarke, and Colin Camerer, “Robust Analogizing and the Outside View: Two Empirical Tests of Case Based Decision Making,” Strategic Management Journal 33, no. 5 (2012): 496–512. 9  “‘Chainsaw Al’ Axed,” CNN Money, June 15, 1998. 10  This problem was suggested by Barry Nalebuff of Yale University. 11  Nicklas Garemo, Stefan Matzinger, and Robert Palter, “Megaprojects: The Good, the Bad, and the Better,” McKinsey Quarterly, July 2015 (quoting Bent Flyvberg, Oxford Saïd Business School). 12  Daniel Kahneman, Dan Lovallo, and Olivier Sibony, “Before You Make that Big Decision,” Harvard Business Review, June 2011. 13  Gerd Gigerenzer, Peter M.


pages: 254 words: 76,064

Whiplash: How to Survive Our Faster Future by Joi Ito, Jeff Howe

3D printing, Albert Michelson, Amazon Web Services, artificial general intelligence, basic income, Bernie Sanders, bitcoin, Black Swan, blockchain, Burning Man, buy low sell high, Claude Shannon: information theory, cloud computing, Computer Numeric Control, conceptual framework, crowdsourcing, cryptocurrency, data acquisition, disruptive innovation, Donald Trump, double helix, Edward Snowden, Elon Musk, Ferguson, Missouri, fiat currency, financial innovation, Flash crash, frictionless, game design, Gerolamo Cardano, informal economy, interchangeable parts, Internet Archive, Internet of things, Isaac Newton, Jeff Bezos, John Harrison: Longitude, Joi Ito, Khan Academy, Kickstarter, Mark Zuckerberg, microbiome, Nate Silver, Network effects, neurotypical, Oculus Rift, pattern recognition, peer-to-peer, pirate software, pre–internet, prisoner's dilemma, Productivity paradox, race to the bottom, RAND corporation, random walk, Ray Kurzweil, Ronald Coase, Ross Ulbricht, Satoshi Nakamoto, self-driving car, SETI@home, side project, Silicon Valley, Silicon Valley startup, Simon Singh, Singularitarianism, Skype, slashdot, smart contracts, Steve Ballmer, Steve Jobs, Steven Levy, Stewart Brand, Stuxnet, supply-chain management, technological singularity, technoutopianism, The Nature of the Firm, the scientific method, The Signal and the Noise by Nate Silver, There's no reason for any individual to have a computer in his home - Ken Olsen, Thomas Kuhn: the structure of scientific revolutions, universal basic income, unpaid internship, uranium enrichment, urban planning, WikiLeaks

., 13 34 Page is referring to the famous scene in the mockumentary This Is Spinal Tap in which the mentally addled lead guitarist, Nigel Tufnel, tries to explain the significance of an amplifier with the capacity to exceed the conventional 10 on the volume knob. “Well, it’s one louder, isn’t it?” 35 Quoted in Joichi Ito and Jeff Howe, “The Future: An Instruction Manual,” LinkedIn Pulse, October 2, 2012, https://www.linkedin.com/pulse/20121002120301-1391-the-future-an-instruction-manual. 36 Nate Silver, The Signal and the Noise: Why So Many Predictions Fail (New York: Penguin, 2012); Louis Menand, “Everybody’s an Expert,” New Yorker, December 5, 2005, http://www.newyorker.com/magazine/2005/12/05/everybodys-an-expert; Stephen J. Dubner, “The Folly of Prediction,” Freakonomics podcast, September 14, 2011, http://freakonomics.com/2011/09/14/new-freakonomics-radio-podcast-the-folly-of-prediction/. 37 National Council for Science and the Environment, The Climate Solutions Consensus: What We Know and What to Do About It, edited by David Blockstein and Leo Wiegman (Washington, D.C.: Island Press, 2012), 3. 38 Oxford Advanced Learner’s Dictionary, http://www.oxforddictionaries.com/us/definition/learner/medium. 39 The Media Lab’s website includes a comprehensive overview of the Lab’s funding model, current research, and history. http://media.mit.edu/about/about-the-lab. 40 Olivia Vanni.


pages: 268 words: 75,850

The Formula: How Algorithms Solve All Our Problems-And Create More by Luke Dormehl

3D printing, algorithmic trading, Any sufficiently advanced technology is indistinguishable from magic, augmented reality, big data - Walmart - Pop Tarts, call centre, Cass Sunstein, Clayton Christensen, commoditize, computer age, death of newspapers, deferred acceptance, disruptive innovation, Edward Lorenz: Chaos theory, Erik Brynjolfsson, Filter Bubble, Flash crash, Florence Nightingale: pie chart, Frank Levy and Richard Murnane: The New Division of Labor, Google Earth, Google Glasses, High speed trading, Internet Archive, Isaac Newton, Jaron Lanier, Jeff Bezos, job automation, John Markoff, Kevin Kelly, Kodak vs Instagram, lifelogging, Marshall McLuhan, means of production, Nate Silver, natural language processing, Netflix Prize, Panopticon Jeremy Bentham, pattern recognition, price discrimination, recommendation engine, Richard Thaler, Rosa Parks, self-driving car, sentiment analysis, Silicon Valley, Silicon Valley startup, Slavoj Žižek, social graph, speech recognition, Steve Jobs, Steven Levy, Steven Pinker, Stewart Brand, the scientific method, The Signal and the Noise by Nate Silver, upwardly mobile, Wall-E, Watson beat the top human players on Jeopardy!, Y Combinator

While I was first coming up with formulas at college, trying to mathematically determine whether we should go to the library to get some work done, deep down in the recesses of our dorky ids I think that what we were saying is that life is uncertain and we were trying to make it more certain. I’m not as disturbed by numbers providing answers as I am by the potential that there might not be answers.” What is it about the modern world that makes us demand easy answers? Is it that we are naturally pattern-seeking creatures, as the statistician Nate Silver argues in The Signal and the Noise? Or is there something about the effects of the march of technology that demands the kind of answers only an algorithm can provide? “[The algorithm does] seem to be a key metaphor for what matters now in terms of organizing the world,” acknowledges McKenzie Wark, a media theorist who has written about digital technologies for the last 20 years. “If one thinks of algorithms as processes which terminate and generate a result, there’s a moment when the process ceases and you have your answer.


pages: 319 words: 75,257

Trumpocalypse: Restoring American Democracy by David Frum

Affordable Care Act / Obamacare, anti-globalists, Bernie Sanders, centre right, coronavirus, currency manipulation / currency intervention, decarbonisation, Donald Trump, Edward Snowden, employer provided health coverage, illegal immigration, immigration reform, labor-force participation, manufacturing employment, mass immigration, Mikhail Gorbachev, Nate Silver, obamacare, offshore financial centre, Peter Thiel, plutocrats, Plutocrats, QAnon, rent-seeking, Ronald Reagan, Saturday Night Live, Silicon Valley

“Generic Congressional Ballot,” Rasmussen Reports, November 5, 2018, http://www.rasmussenreports.com/public_content/politics/mood_of_america/generic_congressional_ballot_nov05. President Trump has sometimes tweeted 50 percent numbers generated by the Rasmussen poll, but Rasmussen habitually screens for “likely” voters in a way that favors Republicans. This method produces headlines very appealing to Fox News between elections, but poor accuracy at elections. Nate Silver has rated Rasmussen consistently the least accurate of the major pollsters. Rasmussen enjoyed a fleeting triumph in 2016. That year its persistent habit of overestimating white turnout for once paid off. Driven by a surge in voting by white noncollege graduates, white turnout actually increased between 2012 and 2016, something that has not happened in memory: normally the US electorate becomes about slightly less white each election than the election before.


pages: 361 words: 81,068

The Internet Is Not the Answer by Andrew Keen

"Robert Solow", 3D printing, A Declaration of the Independence of Cyberspace, Airbnb, AltaVista, Andrew Keen, augmented reality, Bay Area Rapid Transit, Berlin Wall, bitcoin, Black Swan, Bob Geldof, Burning Man, Cass Sunstein, citizen journalism, Clayton Christensen, clean water, cloud computing, collective bargaining, Colonization of Mars, computer age, connected car, creative destruction, cuban missile crisis, David Brooks, disintermediation, disruptive innovation, Donald Davies, Downton Abbey, Edward Snowden, Elon Musk, Erik Brynjolfsson, Fall of the Berlin Wall, Filter Bubble, Francis Fukuyama: the end of history, Frank Gehry, Frederick Winslow Taylor, frictionless, full employment, future of work, gig economy, global village, Google bus, Google Glasses, Hacker Ethic, happiness index / gross national happiness, income inequality, index card, informal economy, information trail, Innovator's Dilemma, Internet of things, Isaac Newton, Jaron Lanier, Jeff Bezos, job automation, Joi Ito, Joseph Schumpeter, Julian Assange, Kevin Kelly, Kickstarter, Kodak vs Instagram, Lean Startup, libertarian paternalism, lifelogging, Lyft, Marc Andreessen, Mark Zuckerberg, Marshall McLuhan, Martin Wolf, Metcalfe’s law, move fast and break things, move fast and break things, Nate Silver, Nelson Mandela, Network effects, new economy, Nicholas Carr, nonsequential writing, Norbert Wiener, Norman Mailer, Occupy movement, packet switching, PageRank, Panopticon Jeremy Bentham, Paul Graham, peer-to-peer, peer-to-peer rental, Peter Thiel, plutocrats, Plutocrats, Potemkin village, precariat, pre–internet, RAND corporation, Ray Kurzweil, ride hailing / ride sharing, Robert Metcalfe, Second Machine Age, self-driving car, sharing economy, Silicon Valley, Silicon Valley ideology, Skype, smart cities, Snapchat, social web, South of Market, San Francisco, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, TaskRabbit, Ted Nelson, telemarketer, The Future of Employment, the medium is the message, the new new thing, Thomas L Friedman, Travis Kalanick, Tyler Cowen: Great Stagnation, Uber for X, uber lyft, urban planning, Vannevar Bush, Whole Earth Catalog, WikiLeaks, winner-take-all economy, working poor, Y Combinator

“So much for the borderless, gender-blind, class-blind and bank account–blind MOOCs,” notes the technology writer Jessica McKenzie. “If anything, this shows that MOOCs are widening the educational divide, not leveling the playing field.”19 This unequal economy is particularly pronounced in online journalism, where, amid the massive layoffs at regional newspapers, highly paid American reporters like Nate Silver, Ezra Klein, Matt Taibbi, and Glenn Greenwald represent what Emily Bell, the director of the Tow Center for Digital Journalism at Columbia University, calls “a one percent economy.”20 Ironically, for all the talk of how the Internet was supposed to diversify the news industry, the end result of the combination of a one percent economy and massive layoffs is less diversity in the newsroom, with minority employment for American journalists down almost 6% between 2006 and 2012.21 And as Bell also notes, the most recent wave of venture-capital-funded “personal brand” journalist startups are almost all supporting white male superstars like Greenwald, Taibbi, Silver, and Klein.22 But the most damaging discrimination is against paid work.


pages: 362 words: 83,464

The New Class Conflict by Joel Kotkin

2013 Report for America's Infrastructure - American Society of Civil Engineers - 19 March 2013, affirmative action, Affordable Care Act / Obamacare, American Society of Civil Engineers: Report Card, Bob Noyce, California gold rush, Capital in the Twenty-First Century by Thomas Piketty, carbon footprint, creative destruction, crony capitalism, David Graeber, deindustrialization, don't be evil, Downton Abbey, Edward Glaeser, Elon Musk, energy security, falling living standards, future of work, Gini coefficient, Google bus, housing crisis, income inequality, informal economy, Internet of things, Jane Jacobs, Jaron Lanier, Jeff Bezos, job automation, John Markoff, John von Neumann, Joseph Schumpeter, Kevin Kelly, labor-force participation, low-wage service sector, Marc Andreessen, Mark Zuckerberg, mass affluent, McJob, McMansion, medical bankruptcy, Nate Silver, New Economic Geography, new economy, New Urbanism, obamacare, offshore financial centre, Paul Buchheit, payday loans, Peter Calthorpe, plutocrats, Plutocrats, post-industrial society, RAND corporation, Ray Kurzweil, rent control, rent-seeking, Report Card for America’s Infrastructure, Richard Florida, Silicon Valley, Silicon Valley ideology, Steve Jobs, technoutopianism, The Death and Life of Great American Cities, Thomas L Friedman, too big to fail, transcontinental railway, trickle-down economics, Tyler Cowen: Great Stagnation, upwardly mobile, urban planning, urban sprawl, War on Poverty, women in the workforce, working poor, young professional

Hall, “With Help from Silicon Valley, America to Dominate the 21st Century,” Tech.pinions, August 12, 2013, http://techpinions.com/help-silicon-valley-america-dominate-21st-century/21550. 101. John B. Judis, “The GOP Plan to Crush Silicon Valley What Will Become of Steve Jobs’s Angel?” New Republic, August 20, 2013, http://www.newrepublic.com/article/114329/republican-budget-cut-would-crush-silicon-valley; Nate Silver, “In Silicon Valley, Technology Talent Gap Threatens G.O.P. Campaigns,” FiveThirtyEight (blog), New York Times, November 28, 2012, http://fivethirtyeight.blogs.nytimes.com/2012/11/28/in-silicon-valley-technology-talent-gap-threatens-g-o-p-campaigns. 102. Matthew Continetti, “The California Captivity of the Democratic Party,” Washington Free Beacon, May 31, 2013, http://freebeacon.com/columns/the-california-captivity-of-the-democratic-party. 103.


pages: 294 words: 82,438

Simple Rules: How to Thrive in a Complex World by Donald Sull, Kathleen M. Eisenhardt

Affordable Care Act / Obamacare, Airbnb, asset allocation, Atul Gawande, barriers to entry, Basel III, Berlin Wall, carbon footprint, Checklist Manifesto, complexity theory, Craig Reynolds: boids flock, Credit Default Swap, Daniel Kahneman / Amos Tversky, diversification, drone strike, en.wikipedia.org, European colonialism, Exxon Valdez, facts on the ground, Fall of the Berlin Wall, haute cuisine, invention of the printing press, Isaac Newton, Kickstarter, late fees, Lean Startup, Louis Pasteur, Lyft, Moneyball by Michael Lewis explains big data, Nate Silver, Network effects, obamacare, Paul Graham, performance metric, price anchoring, RAND corporation, risk/return, Saturday Night Live, sharing economy, Silicon Valley, Startup school, statistical model, Steve Jobs, TaskRabbit, The Signal and the Noise by Nate Silver, transportation-network company, two-sided market, Wall-E, web application, Y Combinator, Zipcar

., “Health on Impulse: When Low Self-Control Promotes Healthy Food Choices,” Health Psychology 33, no. 2 (2013): 103–9, http://www.medscape.com/medline/abstract/2347758. [>] In contrast, people: Brian Wansink, David R. Rust, and Collin R. Payne, “Mindless Eating and Healthy Heuristics for the Irrational,” American Economic Review: Papers and Proceedings 99, no. 2 (2009): 165–69. [>] Meteorologists make: Nate Silver, The Signal and the Noise (New York: Penguin, 2012), 126–27. [>] Japanese honeybees: Atsushi Ugajin et al., “Detection of Neural Activity in the Brains of Japanese Honeybee Workers During the Formation of a ‘Hot Defensive Bee Ball,’” PLoS One 7, no. 3 (2012), available at the website of the National Center for Biotechnology Information, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3303784/. [>] As an example of: Our account of the bees’ choice of new nest is based on the research of Thomas Seeley, especially Thomas D.


pages: 285 words: 86,174

Twilight of the Elites: America After Meritocracy by Chris Hayes

affirmative action, Affordable Care Act / Obamacare, asset-backed security, barriers to entry, Berlin Wall, Bernie Madoff, carried interest, circulation of elites, Climategate, Climatic Research Unit, collapse of Lehman Brothers, collective bargaining, creative destruction, Credit Default Swap, dark matter, David Brooks, David Graeber, deindustrialization, Fall of the Berlin Wall, financial deregulation, fixed income, full employment, George Akerlof, Gunnar Myrdal, hiring and firing, income inequality, Jane Jacobs, jimmy wales, Julian Assange, Kenneth Arrow, Mark Zuckerberg, mass affluent, mass incarceration, means of production, meta analysis, meta-analysis, money market fund, moral hazard, Naomi Klein, Nate Silver, peak oil, plutocrats, Plutocrats, Ponzi scheme, Ralph Waldo Emerson, rolodex, The Spirit Level, too big to fail, University of East Anglia, Vilfredo Pareto, We are the 99%, WikiLeaks, women in the workforce

Acknowledgments One of the more insidious aspects of meritocratic indoctrination is that it teaches you to believe that you wholly own your accomplishments. As I’ve tried to make clear in the book, it’s a tempting notion but a dangerous one. As is the case with any project of this size, this book is the product of the labor of far, far more people than the one person whose name appears on the cover. A Nate Silver post at his old site FiveThirtyEight, that showed General Social Survey data about America’s declining trust in their institutions over time, first crystallized the idea for the book. And it was during a long, loquacious walk I took with Dan Benaim through Washington, DC, in 2009 that I was first able to articulate just what it was I wanted to write a book about. Throughout the development of the book, I’ve had hundreds of conversations with friends and colleagues about its main themes, and those conversations inform every page of the work.


pages: 245 words: 83,272

Artificial Unintelligence: How Computers Misunderstand the World by Meredith Broussard

1960s counterculture, A Declaration of the Independence of Cyberspace, Ada Lovelace, AI winter, Airbnb, Amazon Web Services, autonomous vehicles, availability heuristic, barriers to entry, Bernie Sanders, bitcoin, Buckminster Fuller, Chris Urmson, Clayton Christensen, cloud computing, cognitive bias, complexity theory, computer vision, crowdsourcing, Danny Hillis, DARPA: Urban Challenge, digital map, disruptive innovation, Donald Trump, Douglas Engelbart, easy for humans, difficult for computers, Electric Kool-Aid Acid Test, Elon Musk, Firefox, gig economy, global supply chain, Google Glasses, Google X / Alphabet X, Hacker Ethic, Jaron Lanier, Jeff Bezos, John von Neumann, Joi Ito, Joseph-Marie Jacquard, life extension, Lyft, Mark Zuckerberg, mass incarceration, Minecraft, minimum viable product, Mother of all demos, move fast and break things, move fast and break things, Nate Silver, natural language processing, PageRank, payday loans, paypal mafia, performance metric, Peter Thiel, price discrimination, Ray Kurzweil, ride hailing / ride sharing, Ross Ulbricht, Saturday Night Live, school choice, self-driving car, Silicon Valley, speech recognition, statistical model, Steve Jobs, Steven Levy, Stewart Brand, Tesla Model S, the High Line, The Signal and the Noise by Nate Silver, theory of mind, Travis Kalanick, Turing test, Uber for X, uber lyft, Watson beat the top human players on Jeopardy!, Whole Earth Catalog, women in the workforce

It used to be that you would get stories by chatting to people in bars, and it still might be that you’ll do it that way some times. But now it’s also going to be about poring over data and equipping yourself with the tools to analyse it and picking out what’s interesting. And keeping it in perspective, helping people out by really seeing where it all fits together, and what’s going on in the country.”19 By the time Nate Silver launched FiveThirtyEight.com and published his book The Signal and the Noise in 2012, the term data journalism was in widespread use among investigative journalists.20 As computers have evolved, human nature has not. People need to be kept honest. I hope that this book will help you think like a data journalist so that you can challenge false claims about technology and uncover injustice and inequality embedded in today’s computational systems.


pages: 339 words: 88,732

The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies by Erik Brynjolfsson, Andrew McAfee

"Robert Solow", 2013 Report for America's Infrastructure - American Society of Civil Engineers - 19 March 2013, 3D printing, access to a mobile phone, additive manufacturing, Airbnb, Albert Einstein, Amazon Mechanical Turk, Amazon Web Services, American Society of Civil Engineers: Report Card, Any sufficiently advanced technology is indistinguishable from magic, autonomous vehicles, barriers to entry, basic income, Baxter: Rethink Robotics, British Empire, business cycle, business intelligence, business process, call centre, Charles Lindbergh, Chuck Templeton: OpenTable:, clean water, combinatorial explosion, computer age, computer vision, congestion charging, corporate governance, creative destruction, crowdsourcing, David Ricardo: comparative advantage, digital map, employer provided health coverage, en.wikipedia.org, Erik Brynjolfsson, factory automation, falling living standards, Filter Bubble, first square of the chessboard / second half of the chessboard, Frank Levy and Richard Murnane: The New Division of Labor, Freestyle chess, full employment, G4S, game design, global village, happiness index / gross national happiness, illegal immigration, immigration reform, income inequality, income per capita, indoor plumbing, industrial robot, informal economy, intangible asset, inventory management, James Watt: steam engine, Jeff Bezos, jimmy wales, job automation, John Markoff, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, Joseph Schumpeter, Kevin Kelly, Khan Academy, knowledge worker, Kodak vs Instagram, law of one price, low skilled workers, Lyft, Mahatma Gandhi, manufacturing employment, Marc Andreessen, Mark Zuckerberg, Mars Rover, mass immigration, means of production, Narrative Science, Nate Silver, natural language processing, Network effects, new economy, New Urbanism, Nicholas Carr, Occupy movement, oil shale / tar sands, oil shock, pattern recognition, Paul Samuelson, payday loans, post-work, price stability, Productivity paradox, profit maximization, Ralph Nader, Ray Kurzweil, recommendation engine, Report Card for America’s Infrastructure, Robert Gordon, Rodney Brooks, Ronald Reagan, Second Machine Age, self-driving car, sharing economy, Silicon Valley, Simon Kuznets, six sigma, Skype, software patent, sovereign wealth fund, speech recognition, statistical model, Steve Jobs, Steven Pinker, Stuxnet, supply-chain management, TaskRabbit, technological singularity, telepresence, The Bell Curve by Richard Herrnstein and Charles Murray, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, total factor productivity, transaction costs, Tyler Cowen: Great Stagnation, Vernor Vinge, Watson beat the top human players on Jeopardy!, winner-take-all economy, Y2K

., “Grading Student Loans,” Liberty Street Economics blog, Federal Reserve Bank of New York, March 5, 2012, http://libertystreeteconomics.newyorkfed.org/2012/03/grading-student-loans.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+LibertyStreetEconomics+(Liberty+Street+Economics). 21. Tim Hornyak, “Towel-folding Robot Won’t Do the Dishes,” CNET, March 31, 2010, http://news.cnet.com/8301-17938_105-10471898-1.html. 22. Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t, 1st ed. (New York: Penguin, 2012). Chapter 13 POLICY RECOMMENDATIONS 1. “Employment Level,” Economic Research—Federal Reserve Bank of St. Louis (U.S. Department of Labor, Bureau of Labor Statistics, August 2, 2013), http://research.stlouisfed.org/fred2/series/LNU02000000. 2. Claudia Goldin and Lawrence F. Katz, The Race Between Education and Technology (Cambridge, MA: Belknap Press of Harvard University Press, 2010). 3.


pages: 340 words: 92,904

Street Smart: The Rise of Cities and the Fall of Cars by Samuel I. Schwartz

2013 Report for America's Infrastructure - American Society of Civil Engineers - 19 March 2013, active transport: walking or cycling, Affordable Care Act / Obamacare, American Society of Civil Engineers: Report Card, autonomous vehicles, car-free, City Beautiful movement, collaborative consumption, congestion charging, crowdsourcing, desegregation, Enrique Peñalosa, Ford paid five dollars a day, Frederick Winslow Taylor, if you build it, they will come, Induced demand, intermodal, invention of the wheel, lake wobegon effect, Loma Prieta earthquake, longitudinal study, Lyft, Masdar, megacity, meta analysis, meta-analysis, moral hazard, Nate Silver, oil shock, Productivity paradox, Ralph Nader, rent control, ride hailing / ride sharing, Rosa Parks, self-driving car, skinny streets, smart cities, smart grid, smart transportation, the built environment, the map is not the territory, transportation-network company, Uber and Lyft, Uber for X, uber lyft, Unsafe at Any Speed, urban decay, urban planning, urban renewal, walkable city, Wall-E, white flight, white picket fence, Works Progress Administration, Yogi Berra, Zipcar

It works at almost every level of granularity: until a community—a county, a city, a town, or even a voting precinct—reaches a density of about eight hundred people a square mile, it’s as reliably Republican as Fox News. Once it exceeds that number, though, the voting patterns do a somersault. Anywhere under eight hundred people a square mile, there’s a two-thirds chance that a randomly selected voter went Republican; above it, that hypothetical voter pulled a Democratic lever two-thirds of the time. As the political prediction machine Nate Silver of 538.com tweeted in 2012, “If a place has sidewalks, it votes Democratic.” It’s not totally obvious whether people vote a certain way because of where they live, or whether they move to places where everyone votes the way they do. What is obvious though is that all the elements of a Street Smart transportation system depend on density. At first glance, this would appear to be a giant advantage for a Street Smart future, since every demographic indicator shows that America and the world are headed for a much more urbanized future.


The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences by Rob Kitchin

Bayesian statistics, business intelligence, business process, cellular automata, Celtic Tiger, cloud computing, collateralized debt obligation, conceptual framework, congestion charging, corporate governance, correlation does not imply causation, crowdsourcing, discrete time, disruptive innovation, George Gilder, Google Earth, Infrastructure as a Service, Internet Archive, Internet of things, invisible hand, knowledge economy, late capitalism, lifelogging, linked data, longitudinal study, Masdar, means of production, Nate Silver, natural language processing, openstreetmap, pattern recognition, platform as a service, recommendation engine, RFID, semantic web, sentiment analysis, slashdot, smart cities, Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia, smart grid, smart meter, software as a service, statistical model, supply-chain management, the scientific method, The Signal and the Noise by Nate Silver, transaction costs

Rubinstein, I.S. (2013) ‘Big data: the end of privacy or a new beginning?’, International Data Privacy Law, online first, http://idpl.oxfordjournals.org/content/early/2013/01/24/idpl.ips036.short (last accessed 15 July 2013). Ruppert, E. (2012) ‘The governmental topologies of database devices’, Theory, Culture Society, 29: 116–36. Ruppert, E. (2013) ‘Rethinking empirical social sciences’, Dialogues in Human Geography, 3(3): 268–73. Salmon, F. (2014) ‘Why the Nate Silvers of the world don’t know everything’, Wired, 7 January, http://www.wired.com/business/2014/01/quants-dont-know-everything/ (last accessed 8 January 2014). Salus, P. (1995) Casting the Net: From Arpanet to Internet and Beyond. Addison Wesley, Reading, MA. Sawyer, S. (2008) ‘Data wealth, data poverty, science and cyberinfrastructure’, Prometheus: Critical Studies in Innovation, 26(4): 355–71. Schnapp, J. and Presner, P. (2009) Digital Humanities Manifesto 2.0. http://www.humanitiesblast.com/manifesto/Manifesto_V2.pdf (last accessed 13 March 2013).


pages: 285 words: 86,853

What Algorithms Want: Imagination in the Age of Computing by Ed Finn

Airbnb, Albert Einstein, algorithmic trading, Amazon Mechanical Turk, Amazon Web Services, bitcoin, blockchain, Chuck Templeton: OpenTable:, Claude Shannon: information theory, commoditize, Credit Default Swap, crowdsourcing, cryptocurrency, disruptive innovation, Donald Knuth, Douglas Engelbart, Douglas Engelbart, Elon Musk, factory automation, fiat currency, Filter Bubble, Flash crash, game design, Google Glasses, Google X / Alphabet X, High speed trading, hiring and firing, invisible hand, Isaac Newton, iterative process, Jaron Lanier, Jeff Bezos, job automation, John Conway, John Markoff, Just-in-time delivery, Kickstarter, late fees, lifelogging, Loebner Prize, Lyft, Mother of all demos, Nate Silver, natural language processing, Netflix Prize, new economy, Nicholas Carr, Norbert Wiener, PageRank, peer-to-peer, Peter Thiel, Ray Kurzweil, recommendation engine, Republic of Letters, ride hailing / ride sharing, Satoshi Nakamoto, self-driving car, sharing economy, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, social graph, software studies, speech recognition, statistical model, Steve Jobs, Steven Levy, Stewart Brand, supply-chain management, TaskRabbit, technological singularity, technoutopianism, The Coming Technological Singularity, the scientific method, The Signal and the Noise by Nate Silver, The Structural Transformation of the Public Sphere, The Wealth of Nations by Adam Smith, transaction costs, traveling salesman, Turing machine, Turing test, Uber and Lyft, Uber for X, uber lyft, urban planning, Vannevar Bush, Vernor Vinge, wage slave

Something of this knowing spectacle remained in Kasparov’s matches with Deep Blue, including his persistent efforts to deploy openings and play styles that would throw off IBM’s algorithms.53 The human engineer was never entirely hidden behind the mechanism, as IBM employees tweaked the system between every game. Perhaps the best example of this relationship was a highly scrutinized move near the end of game one, the truth of which was only revealed in 2012 when IBM researcher Murray Campbell was interviewed by the popular statistician Nate Silver. The move in question had sent Kasparov “into a tizzy” as it seemed to reflect the ambiguity and refinement of a human-level intelligence, and many have suggested it threw off the grandmaster’s concentration for the second game, which he proceeded to lose.54 In fact, as Campbell revealed, the move had been a bug, one the engineers corrected after the first match.55 Kasparov himself had made magic out of the algorithm, inventing a sophisticated cultural explanation for what was in the end a random computational artifact.


We Need New Stories: Challenging the Toxic Myths Behind Our Age of Discontent by Nesrine Malik

affirmative action, Affordable Care Act / Obamacare, barriers to entry, Bernie Sanders, Boris Johnson, British Empire, centre right, cognitive dissonance, continuation of politics by other means, currency peg, Donald Trump, feminist movement, financial independence, Francis Fukuyama: the end of history, gender pay gap, ghettoisation, glass ceiling, illegal immigration, invisible hand, mass immigration, moral panic, Nate Silver, obamacare, old-boy network, payday loans, planetary scale, Ponzi scheme, race to the bottom, Ronald Reagan, Saturday Night Live, sexual politics, Steven Pinker, The Bell Curve by Richard Herrnstein and Charles Murray, Thomas L Friedman, transatlantic slave trade

In June 2017, Stephens publicly forswore Twitter, saying that the medium debased politics and that he would ‘intercede only to say nice things about the writing I admire, the people I like and the music I love’. In response to criticism of an article of his that misread the success of Democrats in the midterms (and which had to be quietly amended, twice, after more results came in, invalidating his thesis), Stephens tweeted at statistician Nate Silver of 538, ‘too bad you’re a Twitter troll’. Unprovoked, he tweeted at James Zogby, founder and president of the Arab American Institute, who took issue with a chef calling hummus Israeli food, saying it was part of a history of ‘cultural appropriation’ and a systematic effort to erase Palestinian history and culture. Stephens fired back at him: ‘Hummus seems to have first been mentioned as a Cairene food in the thirteenth century or so.


pages: 269 words: 104,430

Carjacked: The Culture of the Automobile and Its Effect on Our Lives by Catherine Lutz, Anne Lutz Fernandez

barriers to entry, car-free, carbon footprint, collateralized debt obligation, failed state, feminist movement, fudge factor, Gordon Gekko, housing crisis, illegal immigration, income inequality, inventory management, market design, market fundamentalism, mortgage tax deduction, Naomi Klein, Nate Silver, New Urbanism, oil shock, peak oil, Ralph Nader, Ralph Waldo Emerson, ride hailing / ride sharing, Thorstein Veblen, traffic fines, Unsafe at Any Speed, urban planning, white flight, women in the workforce, working poor, Zipcar

John Pucher and Lewis Dijkstra, “Promoting Safe Walking and Cycling to Improve Public Health: Lessons from the Netherlands and Germany,” American Journal of Public Health, 2004, 93 (9): 1509–16. Evans, Traffic Safety. CHAPTER 10 1. “Edmunds.com Forecasts December Auto Sales: SUV and Truck Sales to Outpace Cars for First Time,” December 19, 2008, www.edmunds.com NOTES 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 245 Nate Silver, “The End of Car Culture,” Esquire, May 6, 2009. “Recovery Accountability and Transparency Board. $8.4 Billion for Public Transit,” March 5, 2009, www.recovery.gov. Dylan Rivera, “U.S. Transportation Secretary Calls Portland’s Streetcar, Light Rail a ‘Model’ for Nation,” The Oregonian, April 14, 2009. Use the Edmunds.com’s True Cost to Own Calculator to see annual dollar depreciation on your current vehicle or a vehicle you are planning to buy in order to calculate exactly how much you would save by postponing trading it in or selling it to buy a new car.


pages: 357 words: 94,852

No Is Not Enough: Resisting Trump’s Shock Politics and Winning the World We Need by Naomi Klein

Airbnb, basic income, battle of ideas, Berlin Wall, Bernie Sanders, Brewster Kahle, Celebration, Florida, clean water, collective bargaining, Corrections Corporation of America, desegregation, Donald Trump, drone strike, Edward Snowden, Elon Musk, energy transition, financial deregulation, greed is good, high net worth, Howard Zinn, illegal immigration, income inequality, Internet Archive, Kickstarter, late capitalism, Mark Zuckerberg, market bubble, market fundamentalism, mass incarceration, Mikhail Gorbachev, moral panic, Naomi Klein, Nate Silver, new economy, Occupy movement, offshore financial centre, oil shale / tar sands, open borders, Peter Thiel, plutocrats, Plutocrats, private military company, profit motive, race to the bottom, Ralph Nader, Ronald Reagan, Saturday Night Live, sexual politics, sharing economy, Silicon Valley, too big to fail, trade liberalization, transatlantic slave trade, Triangle Shirtwaist Factory, trickle-down economics, Upton Sinclair, urban decay, women in the workforce, working poor

Trump and sexual harassment and assault allegations Megan Twohey, “Former ‘Apprentice’ Contestant Files Defamation Suit Against Trump,” New York Times, January 17, 2017, Trump and Ivana rape allegation Jane Mayer, “Documenting Trump’s Abuse of Women,” New Yorker, October 24, 2016, http://www.newyorker.com/​magazine/​2016/​10/​24/​documenting-trumps-abuse-of-women. The Problem with “Jobs Voters” Past 40 years has seen number of people behind bars in US increase by 500 percent Carl Vogel, Prison Brake, University of Chicago School of Social Service Administration, accessed April 11, 2017, https://ssa.uchicago.edu/​end-mass-incarceration. Trump voters: earn $50,000–$200,000 Nate Silver, “The Mythology of Trump’s ‘Working Class’ Support,” FiveThirtyEight, May 3, 2016, https://fivethirtyeight.com/​features/​the-mythology-of-trumps-working-class-support/. CNN analysis of exit polls “Exit polls,” CNN.com, November 23, 2016, http://www.cnn.com/​election/​results/​exit-polls. Insecure on Every Front Anne Case and Angus Deaton: “deaths of despair” Anne Case and Angus Deaton, “Mortality and Morbidity in the 21st Century,” Brookings Papers on Economic Activity, 2017, https://www.brookings.edu/​wp-content/​uploads/​2017/​03/​6_casedeaton.pdf.


pages: 493 words: 98,982

The Tyranny of Merit: What’s Become of the Common Good? by Michael J. Sandel

affirmative action, Affordable Care Act / Obamacare, anti-communist, Berlin Wall, Bernie Sanders, Boris Johnson, Capital in the Twenty-First Century by Thomas Piketty, centre right, coronavirus, COVID-19, Credit Default Swap, Deng Xiaoping, Donald Trump, ending welfare as we know it, facts on the ground, Fall of the Berlin Wall, financial deregulation, financial innovation, global supply chain, helicopter parent, High speed trading, immigration reform, income inequality, Khan Academy, laissez-faire capitalism, meta analysis, meta-analysis, Nate Silver, new economy, obamacare, Occupy movement, plutocrats, Plutocrats, Ronald Reagan, smart grid, Steve Jobs, Steven Levy, the market place, The Wealth of Nations by Adam Smith, Washington Consensus

On Trump share of non-college white voters, see 2016 exit polls, CNN, cnn.com/election/2016/results/exit-polls ; Clinton share of advanced degree holders is from Thomas Piketty, “Brahmin Left vs. Merchant Right: Rising Inequality & the Changing Structure of Political Conflict,” wid.world Working Paper Series, March 2018, piketty.pse.ens.fr/files/Piketty2018.pdf , Figure 3.3b; on education versus income, see Nate Silver, “Education, Not Income, Predicted Who Would Vote for Trump,” November 22, 2016, FiveThirtyEight.com , fivethirtyeight.com/features/education-not-income-predicted-who-would-vote-for-trump . 60. Silver, “Education, Not Income, Predicted Who Would Vote for Trump.” Trump quoted in Susan Page, “Trump Does the Impossible—Again,” USA Today , February 25, 2016, usatoday.com/story/news/politics/elections/2016/02/24/analysis-donald-trump-does-impossible-again/80843932 . 61.


pages: 1,034 words: 241,773

Enlightenment Now: The Case for Reason, Science, Humanism, and Progress by Steven Pinker

3D printing, access to a mobile phone, affirmative action, Affordable Care Act / Obamacare, agricultural Revolution, Albert Einstein, Alfred Russel Wallace, anti-communist, Anton Chekhov, Arthur Eddington, artificial general intelligence, availability heuristic, Ayatollah Khomeini, basic income, Berlin Wall, Bernie Sanders, Black Swan, Bonfire of the Vanities, business cycle, capital controls, Capital in the Twenty-First Century by Thomas Piketty, carbon footprint, clean water, clockwork universe, cognitive bias, cognitive dissonance, Columbine, conceptual framework, correlation does not imply causation, creative destruction, crowdsourcing, cuban missile crisis, Daniel Kahneman / Amos Tversky, dark matter, decarbonisation, deindustrialization, dematerialisation, demographic transition, Deng Xiaoping, distributed generation, diversified portfolio, Donald Trump, Doomsday Clock, double helix, effective altruism, Elon Musk, en.wikipedia.org, end world poverty, endogenous growth, energy transition, European colonialism, experimental subject, Exxon Valdez, facts on the ground, Fall of the Berlin Wall, first-past-the-post, Flynn Effect, food miles, Francis Fukuyama: the end of history, frictionless, frictionless market, germ theory of disease, Gini coefficient, Hans Rosling, hedonic treadmill, helicopter parent, Hobbesian trap, humanitarian revolution, Ignaz Semmelweis: hand washing, income inequality, income per capita, Indoor air pollution, Intergovernmental Panel on Climate Change (IPCC), invention of writing, Jaron Lanier, Joan Didion, job automation, Johannes Kepler, John Snow's cholera map, Kevin Kelly, Khan Academy, knowledge economy, l'esprit de l'escalier, Laplace demon, life extension, long peace, longitudinal study, Louis Pasteur, Martin Wolf, mass incarceration, meta analysis, meta-analysis, Mikhail Gorbachev, minimum wage unemployment, moral hazard, mutually assured destruction, Naomi Klein, Nate Silver, Nathan Meyer Rothschild: antibiotics, Nelson Mandela, New Journalism, Norman Mailer, nuclear winter, obamacare, open economy, Paul Graham, peak oil, Peter Singer: altruism, Peter Thiel, precision agriculture, prediction markets, purchasing power parity, Ralph Nader, randomized controlled trial, Ray Kurzweil, rent control, Republic of Letters, Richard Feynman, road to serfdom, Robert Gordon, Rodney Brooks, rolodex, Ronald Reagan, Rory Sutherland, Saturday Night Live, science of happiness, Scientific racism, Second Machine Age, secular stagnation, self-driving car, sharing economy, Silicon Valley, Silicon Valley ideology, Simon Kuznets, Skype, smart grid, sovereign wealth fund, stem cell, Stephen Hawking, Steven Pinker, Stewart Brand, Stuxnet, supervolcano, technological singularity, Ted Kaczynski, The Rise and Fall of American Growth, the scientific method, The Signal and the Noise by Nate Silver, The Spirit Level, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Thomas Kuhn: the structure of scientific revolutions, Thomas Malthus, total factor productivity, union organizing, universal basic income, University of East Anglia, Unsafe at Any Speed, Upton Sinclair, uranium enrichment, urban renewal, War on Poverty, We wanted flying cars, instead we got 140 characters, women in the workforce, working poor, World Values Survey, Y2K

In the American election, voters in the two lowest income brackets voted for Clinton 52–42, as did those who identified “the economy” as the most important issue. A majority of voters in the four highest income brackets voted for Trump, and Trump voters singled out “immigration” and “terrorism,” not “the economy,” as the most important issues.34 The twisted metal has turned up more promising clues. An article by the statistician Nate Silver began, “Sometimes statistical analysis is tricky, and sometimes a finding just jumps off the page.” That finding jumped right off the page and into the article’s headline: “Education, Not Income, Predicted Who Would Vote for Trump.”35 Why should education have mattered so much? Two uninteresting explanations are that the highly educated happen to affiliate with a liberal political tribe, and that education may be a better long-term predictor of economic security than current income.

So Tetlock pinned them down by stipulating events with unambiguous outcomes and deadlines (for example, “Will Russia annex additional Ukraine territory in the next three months?” “In the next year, will any country withdraw from the Eurozone?” “How many additional countries will report cases of the Ebola virus in the next eight months?”) and having them write down numerical probabilities. Tetlock also avoided the common fallacy of praising or ridiculing a single probabilistic prediction after the fact, as when the poll aggregator Nate Silver of FiveThirtyEight came under fire for giving Donald Trump just a 29 percent chance of winning the 2016 election.45 Since we cannot replay the election thousands of times and count up the number of times that Trump won, the question of whether the prediction was confirmed or disconfirmed is meaningless. What we can do, and what Tetlock did, is compare the set of each forecaster’s probabilities with the corresponding outcomes.


pages: 451 words: 103,606

Machine Learning for Hackers by Drew Conway, John Myles White

call centre, centre right, correlation does not imply causation, Debian, Erdős number, Nate Silver, natural language processing, Netflix Prize, p-value, pattern recognition, Paul Erdős, recommendation engine, social graph, SpamAssassin, statistical model, text mining, the scientific method, traveling salesman

Drew’s best recommendation is to follow Clay Shirky (cshirky), a professor at NYU who studies and writes on the role of technology and the Internet in society. Given what we have already learned about Drew’s bifurcated brain, this seems like a good match. Keeping this in mind, the rest of the recommendations fit one or both of Drew’s general interests. There is Danger Room (dangerroom); Wired’s National Security blog; Big Data (bigdata); and 538 (fivethirtyeight), the New York Times’ election forecasting blog by Nate Silver. And, of course, shitmydadsays. Although these recommendations are good—and since the writing of the first draft of this book, Drew has enjoyed following the names this engine presented—perhaps there is a better way to recommend people. Because we already know that a given seed user’s network has a lot of emergent structure, it might be useful to use this structure to recommend users that fit into those groups.


pages: 409 words: 105,551

Team of Teams: New Rules of Engagement for a Complex World by General Stanley McChrystal, Tantum Collins, David Silverman, Chris Fussell

Airbus A320, Albert Einstein, Atul Gawande, autonomous vehicles, bank run, barriers to entry, Black Swan, butterfly effect, call centre, Captain Sullenberger Hudson, Chelsea Manning, clockwork universe, crew resource management, crowdsourcing, Edward Snowden, Flash crash, Frederick Winslow Taylor, global supply chain, Henri Poincaré, high batting average, interchangeable parts, invisible hand, Isaac Newton, Jane Jacobs, job automation, job satisfaction, John Nash: game theory, knowledge economy, Mark Zuckerberg, Mohammed Bouazizi, Nate Silver, Pierre-Simon Laplace, RAND corporation, self-driving car, Silicon Valley, Silicon Valley startup, Skype, Steve Jobs, supply-chain management, The Wealth of Nations by Adam Smith, urban sprawl, US Airways Flight 1549, WikiLeaks, zero-sum game

Quoted in Ben Ramalingam, Harry Jones, Toussaint Reba, and John Young, Exploring the Science of Complexity: Ideas and Implications for Development and Humanitarian Efforts, vol. 285 (London: Overseas Development Institute, 2008), 10. “The evidence on the folly” . . . Jonathan Davis, “Folly of Forecasting and Useless Data,” Financial Times, January 17, 2010, http://www.ft.com/intl/cms/s/0/b8c4adfe-0202-11df-8b56-00144feabdc0.html#axzz36R3YiZLR. one-in-five-hundred chance . . . Nate Silver, “The Weatherman Is Not a Moron,” New York Times, September 7, 2012, http://www.nytimes.com/2012/09/09/magazine/the-weatherman-is-not-a-moron.html?pagewanted=all. increasingly shorter lifespan . . . http://www.forbes.com/sites/stevedenning/2011/11/19/peggy-noonan-on-steve-jobs-and-why-big-companies-die/. Fortune 500 list of 2011 . . . http://www.aei.org/publication/fortune-500-firms-in-1955-vs-2011-87-are-gone/.


pages: 350 words: 109,379

How to Run a Government: So That Citizens Benefit and Taxpayers Don't Go Crazy by Michael Barber

Affordable Care Act / Obamacare, Atul Gawande, battle of ideas, Berlin Wall, Black Swan, Checklist Manifesto, collapse of Lehman Brothers, collective bargaining, deliberate practice, facts on the ground, failed state, fear of failure, full employment, G4S, illegal immigration, invisible hand, libertarian paternalism, Mark Zuckerberg, Nate Silver, North Sea oil, obamacare, performance metric, Potemkin village, Ronald Reagan, school choice, The Signal and the Noise by Nate Silver, transaction costs, WikiLeaks

When I said to him I thought that was a rash statement, he replied with some cutting edge, ‘It is important that everyone takes responsibility, including me. And, by the way, if I go down, you’re coming with me.’ In other words, reputations are at risk. But David’s point was right; if you want thousands of public servants to take responsibility for their part in achieving a goal, you need to make it clear that you take your responsibility seriously too. Nate Silver, in his magisterial survey of ‘the art and science of prediction’, The Signal and the Noise, urges us to think probabilistically when we forecast: rather than ‘it’s going to rain tomorrow’, ‘there is a 90 per cent chance of rain tomorrow’. This is, of course, the right way to think analytically when a target is being discussed. He also makes another crucial point – that it’s dangerous to depend purely on the data.


pages: 416 words: 108,370

Hit Makers: The Science of Popularity in an Age of Distraction by Derek Thompson

Airbnb, Albert Einstein, Alexey Pajitnov wrote Tetris, always be closing, augmented reality, Clayton Christensen, Donald Trump, Downton Abbey, full employment, game design, Gordon Gekko, hindsight bias, indoor plumbing, industrial cluster, information trail, invention of the printing press, invention of the telegraph, Jeff Bezos, John Snow's cholera map, Kodak vs Instagram, linear programming, Lyft, Marc Andreessen, Mark Zuckerberg, Marshall McLuhan, Menlo Park, Metcalfe’s law, Minecraft, Nate Silver, Network effects, Nicholas Carr, out of africa, randomized controlled trial, recommendation engine, Robert Gordon, Ronald Reagan, Silicon Valley, Skype, Snapchat, statistical model, Steve Ballmer, Steve Jobs, Steven Levy, Steven Pinker, subscription business, telemarketer, the medium is the message, The Rise and Fall of American Growth, Uber and Lyft, Uber for X, uber lyft, Vilfredo Pareto, Vincenzo Peruggia: Mona Lisa, women in the workforce

Thank you to the wonderful team at Penguin Press: Virginia, Scott, Annie, Ann, and the whole publicity and marketing squad. Thanks to those whose work inspired this book, directly and implicitly: Raymond Loewy, Stanley Lieberson, Michael Kaminiski, Chris Taylor, Bill Bryson, Malcolm Gladwell, Jonah Berger, Steven Johnson, Tom Vanderbilt, Robert Gordon, David Suisman, Paul Barber, Elizabeth Margulis, John Seabrook, Charles Duhigg, Daniel Kahneman, Steven Pinker, Oliver Sacks, Michael Wolff, Nate Silver, Dan Ariely, Jonathan Franzen, Conor Sen, Felix Salmon, Matthew Yglesias, Ezra Klein, Chris Martin, Marc Andreessen, and Umberto Eco. Thanks to those whose conversations inspired this book, directly and implicitly: Drew Durbin, Lincoln Quirk, Michael Diamond, Jordan Weissmann, Robbie dePicciotto, Laura Martin, Maria Konnikova, Mark Harris, Spencer Kornhaber, Rececca Rosen, Alexis Madrigal, Bob Cohn, John Gould, Don Peck, James Bennet, Kevin Roose, Gabriel Rossman, Jesse Prinz, Duncan Watts, Anne Messitte, Andrew Golis, Aditya Sood, Nicholas Jackson, Seth Godin, the Diamonds, the Durbins, and Kira Thompson.


pages: 474 words: 120,801

The End of Power: From Boardrooms to Battlefields and Churches to States, Why Being in Charge Isn’t What It Used to Be by Moises Naim

additive manufacturing, barriers to entry, Berlin Wall, bilateral investment treaty, business cycle, business process, business process outsourcing, call centre, citizen journalism, Clayton Christensen, clean water, collapse of Lehman Brothers, collective bargaining, colonial rule, conceptual framework, corporate governance, creative destruction, crony capitalism, deskilling, disintermediation, disruptive innovation, don't be evil, failed state, Fall of the Berlin Wall, financial deregulation, Francis Fukuyama: the end of history, illegal immigration, immigration reform, income inequality, income per capita, intangible asset, intermodal, invisible hand, job-hopping, Joseph Schumpeter, Julian Assange, Kickstarter, liberation theology, Martin Wolf, mega-rich, megacity, Naomi Klein, Nate Silver, new economy, Northern Rock, Occupy movement, open borders, open economy, Peace of Westphalia, plutocrats, Plutocrats, price mechanism, price stability, private military company, profit maximization, Ronald Coase, Ronald Reagan, Silicon Valley, Skype, Steve Jobs, The Nature of the Firm, Thomas Malthus, too big to fail, trade route, transaction costs, Washington Consensus, WikiLeaks, World Values Survey, zero-sum game

(It should be noted, however, that David Wood, the Pulitzer Prize–winner at The Huffington Post, has decades of reporting experience.) Meanwhile, the ease of publishing on the Internet has turned blogs on everything from electoral politics to fiscal policy, rock music, and business travel into credible and revenue-earning specialty sources that often outperform beat reporters and magazine analysts. Consider the case of statistics geek Nate Silver, who applied the skills he honed crunching baseball numbers to the 2008 and 2012 US presidential campaigns on his site fivethirtyeight.com. Using his own model to aggregate polling data, Silver was able to predict the outcome of the Super Tuesday primaries between Barack Obama and Hillary Clinton; he went on to predict Obama’s defeat of John McCain as early as March 2008, and his detailed predictions on Election Night got forty-nine out of fifty states right, and in the 2012 elections also predicted accurately the results.


pages: 434 words: 117,327

Can It Happen Here?: Authoritarianism in America by Cass R. Sunstein

active measures, affirmative action, Affordable Care Act / Obamacare, airline deregulation, anti-communist, anti-globalists, availability heuristic, business cycle, Cass Sunstein, David Brooks, Donald Trump, Edward Snowden, Estimating the Reproducibility of Psychological Science, failed state, Filter Bubble, Francis Fukuyama: the end of history, ghettoisation, illegal immigration, immigration reform, Isaac Newton, job automation, Joseph Schumpeter, Long Term Capital Management, Nate Silver, Network effects, New Journalism, night-watchman state, obamacare, Potemkin village, random walk, Richard Thaler, road to serfdom, Ronald Reagan, the scientific method, War on Poverty, WikiLeaks, World Values Survey

Shanto Iyengar, Gaurav Sood, and Yohtach Lelkes, “Affect, Not Ideology: A Social Identity Perspective on Polarization,” Public Opinion Quarterly 76 (2012): 405–31. 37. Václav Havel, “The Power of the Powerless,” in The Power of the Powerless: Citizens against the State in Central-Eastern Europe, ed. John Keane, trans. Paul Wilson from 1979 orig. (Armonk, NY: M.E. Sharpe, 1985), 27–28. For an interpretation, see Kuran, Private Truths, chaps. 7 and 13. 38. Nate Silver, “Education, Not Income, Predicted Who Would Vote for Trump,” FiveThirtyEight, November 22, 2016; Alec Tyson and Shiva Maniam, “Behind Trump’s Victory: Divisions by Race, Gender, Education,” Pew Research Center Fact Tank, November 9, 2016. 39. According to exit polls, 81 percent of white evangelicals voted for Trump (Gregory A. Smith and Jessica Martinez, “How the Faithful Voted: A Preliminary 2016 Analysis,” Pew Research Center Fact Tank, November 9, 2016).


pages: 518 words: 49,555

Designing Social Interfaces by Christian Crumlish, Erin Malone

A Pattern Language, Amazon Mechanical Turk, anti-pattern, barriers to entry, c2.com, carbon footprint, cloud computing, collaborative editing, creative destruction, crowdsourcing, en.wikipedia.org, Firefox, game design, ghettoisation, Howard Rheingold, hypertext link, if you build it, they will come, Merlin Mann, Nate Silver, Network effects, Potemkin village, recommendation engine, RFC: Request For Comment, semantic web, SETI@home, Skype, slashdot, social graph, social software, social web, source of truth, stealth mode startup, Stewart Brand, telepresence, The Wisdom of Crowds, web application

Download at WoweBook.Com 214 Chapter 8: Share and Share Alike Users have come to expect these sorts of conveniences for grabbing and sharing content. Remember, everyone is overwhelmed with information and reminders to revisit or share information. If users can send or share content on impulse with immediate gratification, it is much more likely they will take the action and learn to expect to be able to do so (Figure 8-5). Figure 8-5. A reader of Nate Silver’s Fivethirtyeight.com blog asks him to add a sharing widget to his blog template. As a designer of social experiences, you have several potential avenues for employing a Share This widget: • You can design and make your own widget, and use it throughout your service and/ or encourage others to adopt it, thus driving at least some traffic back to your service. • You can publish icons, methods, and APIs for adding your service to existing or incumbent widgets (see Chapter 17). • You can embrace someone else’s widget if you simply want to incorporate its functionality and aren’t using sharing to drive direct participation in your own network or application.


pages: 336 words: 113,519

The Undoing Project: A Friendship That Changed Our Minds by Michael Lewis

Albert Einstein, availability heuristic, Cass Sunstein, choice architecture, complexity theory, Daniel Kahneman / Amos Tversky, Donald Trump, Douglas Hofstadter, endowment effect, feminist movement, framing effect, hindsight bias, John von Neumann, Kenneth Arrow, loss aversion, medical residency, Menlo Park, Murray Gell-Mann, Nate Silver, New Journalism, Paul Samuelson, Richard Thaler, Saturday Night Live, Stanford marshmallow experiment, statistical model, the new new thing, Thomas Bayes, Walter Mischel, Yom Kippur War

In 2004, after aping Oakland’s approach to baseball decision making, the Boston Red Sox won their first World Series in nearly a century. Using the same methods, they won it again in 2007 and 2013. But in 2016, after three disappointing seasons, they announced that they were moving away from the data-based approach and back to one where they relied upon the judgment of baseball experts. (“We have perhaps overly relied on numbers . . . ,” said owner John Henry.) The writer Nate Silver for several years enjoyed breathtaking success predicting U.S. presidential election outcomes for the New York Times, using an approach to statistics he learned writing about baseball. For the first time in memory, a newspaper seemed to have an edge in calling elections. But then Silver left the Times, and failed to predict the rise of Donald Trump—and his data-driven approach to predicting elections was called into question . . . by the New York Times!


pages: 450 words: 113,173

The Age of Entitlement: America Since the Sixties by Christopher Caldwell

1960s counterculture, affirmative action, Affordable Care Act / Obamacare, anti-communist, Bernie Sanders, big data - Walmart - Pop Tarts, blue-collar work, Cass Sunstein, choice architecture, computer age, crack epidemic, crony capitalism, Daniel Kahneman / Amos Tversky, David Attenborough, desegregation, disintermediation, disruptive innovation, Edward Snowden, Erik Brynjolfsson, Ferguson, Missouri, financial deregulation, financial innovation, Firefox, full employment, George Gilder, global value chain, Home mortgage interest deduction, illegal immigration, immigration reform, informal economy, Jeff Bezos, John Markoff, Kevin Kelly, libertarian paternalism, Mark Zuckerberg, Martin Wolf, mass immigration, mass incarceration, mortgage tax deduction, Nate Silver, new economy, Norman Mailer, post-industrial society, pre–internet, profit motive, reserve currency, Richard Thaler, Robert Bork, Robert Gordon, Robert Metcalfe, Ronald Reagan, Rosa Parks, Silicon Valley, Skype, South China Sea, Steve Jobs, Thomas Kuhn: the structure of scientific revolutions, Thomas L Friedman, too big to fail, transatlantic slave trade, transcontinental railway, War on Poverty, Whole Earth Catalog, zero-sum game

Only estates in the richest micro-fraction: Urban Institute and Brookings Institution Tax Policy Center, The Tax Policy Center Briefing Book: A Citizens’ Guide to the Tax System and Tax Policy (Washington, D.C.: Urban-Brookings Tax Policy Center, 2018), 322–23. Online at taxpolicycenter.org/briefing-book. In addition to their liquid assets: Kaplan, Then Comes Marriage, 95. was bankrolled by: From the “HRC Story” page of the Human Rights Campaign, https://www.hrc.org/hrc-story/corporate-partners. Accessed January 4, 2019. Google’s employees gave: Nate Silver, “How Rare Are Anti-Gay-Marriage Donations in Silicon Valley?,” FiveThirtyEight, April 4, 2014. Online at fivethirtyeight.com. Reuters discovered in 2014: Darel E. Paul, “Culture War as Class War,” First Things 285 (August–September 2018): 42. it permitted Windsor to tap: Kaplan, Then Comes Marriage, 56. pro–gay marriage activists pressured: Ibid., 148–49. Kaplan wrote: Ibid., 243–44. “felt like old home week”: Ibid., 318.


pages: 523 words: 112,185

Doing Data Science: Straight Talk From the Frontline by Cathy O'Neil, Rachel Schutt

Amazon Mechanical Turk, augmented reality, Augustin-Louis Cauchy, barriers to entry, Bayesian statistics, bioinformatics, computer vision, correlation does not imply causation, crowdsourcing, distributed generation, Edward Snowden, Emanuel Derman, fault tolerance, Filter Bubble, finite state, Firefox, game design, Google Glasses, index card, information retrieval, iterative process, John Harrison: Longitude, Khan Academy, Kickstarter, Mars Rover, Nate Silver, natural language processing, Netflix Prize, p-value, pattern recognition, performance metric, personalized medicine, pull request, recommendation engine, rent-seeking, selection bias, Silicon Valley, speech recognition, statistical model, stochastic process, text mining, the scientific method, The Wisdom of Crowds, Watson beat the top human players on Jeopardy!, X Prize

Keep in mind that the data generated by user behavior becomes the building blocks of data products, which simultaneously are used by users and influence user behavior. We see this in recommendation systems, ranking algorithms, friend suggestions, etc., and we will see it increasingly across sectors including education, finance, retail, and health. Things can go wrong with such feedback loops: keep the financial meltdown in mind as a cautionary example. Much is made about predicting the future (see Nate Silver), predicting the present (see Hal Varian), and exploring causal relationships from observed data (the past; see Sinan Aral). The next logical concept then is: models and algorithms are not only capable of predicting the future, but also of causing the future. That’s what we can look forward to, in the best of cases, and what we should fear in the worst. As an introduction to how to approach these issues ethically, let’s start with Emanuel Derman’s Hippocratic Oath of Modeling, which was made for financial modeling but fits perfectly into this framework: I will remember that I didn’t make the world and that it doesn’t satisfy my equations.


pages: 602 words: 120,848

Winner-Take-All Politics: How Washington Made the Rich Richer-And Turned Its Back on the Middle Class by Paul Pierson, Jacob S. Hacker

accounting loophole / creative accounting, active measures, affirmative action, asset allocation, barriers to entry, Bonfire of the Vanities, business climate, business cycle, carried interest, Cass Sunstein, clean water, collective bargaining, corporate governance, Credit Default Swap, David Brooks, desegregation, employer provided health coverage, financial deregulation, financial innovation, financial intermediation, fixed income, full employment, Home mortgage interest deduction, Howard Zinn, income inequality, invisible hand, knowledge economy, laissez-faire capitalism, Martin Wolf, medical bankruptcy, moral hazard, Nate Silver, new economy, night-watchman state, offshore financial centre, oil shock, Powell Memorandum, Ralph Nader, Ronald Reagan, shareholder value, Silicon Valley, The Wealth of Nations by Adam Smith, too big to fail, trickle-down economics, union organizing, very high income, War on Poverty, winner-take-all economy, women in the workforce

On the conflicting estimates of the crowd size, see www.politifact.com/truth-o-meter/article/2009/sep/14/tea-party-photo-shows-large-crowd-different-event/. 53 CNN Opinion Research Poll of 1,023 adult Americans, including 954 registered voters, February 12–15, 2010, http://i2.cdn.turner.com/cnn/2010/images/02/17/rel4b.pdf. 54 Pew Research Center survey of 1,003 adult Americans, January 14–17, 2010, http://people-press.org/reports/questionnaires/586.pdf. 55 CNN Opinion Research Poll of 1,160 adult Americans, December 16–20, 2009, http://i2.cdn.turner.com/cnn/2009/images/12/21/rel19a.pdf. 56 Kaiser Health Tracking Poll of 2,002 American adults, January 7–12, 2010, ww.kff.org/kaiserpolls/upload/8042-C.pdf; Nate Silver, “Health Care Polls: Opinion Gap or Information Gap?” FiveThirtyEight.com, January 23, 2010, www.fivethirtyeight.com/2010/01/health-care-polls-opinion-gap-or.html. Conclusion: Beating Winner-Take-All 1 The golden ticket analogy has been offered by Gregory Mankiw, “The Wealth Trajectory: Rewards for the Few,” New York Times, April 20, 2008. 2 Ronald D. Orol, “If Senate OKs Bank Bill, Expect a Year of Debate,” Marketwatch.com, March 17, 2010. 3 “Gohmert Calls for Amendment Convention by States,” March 23, 2010, http://gohmert.house.gov/index.cfm?


pages: 561 words: 120,899

The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant From Two Centuries of Controversy by Sharon Bertsch McGrayne

Bayesian statistics, bioinformatics, British Empire, Claude Shannon: information theory, Daniel Kahneman / Amos Tversky, double helix, Edmond Halley, Fellow of the Royal Society, full text search, Henri Poincaré, Isaac Newton, Johannes Kepler, John Markoff, John Nash: game theory, John von Neumann, linear programming, longitudinal study, meta analysis, meta-analysis, Nate Silver, p-value, Pierre-Simon Laplace, placebo effect, prediction markets, RAND corporation, recommendation engine, Renaissance Technologies, Richard Feynman, Richard Feynman: Challenger O-ring, Robert Mercer, Ronald Reagan, speech recognition, statistical model, stochastic process, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, traveling salesman, Turing machine, Turing test, uranium enrichment, Yom Kippur War

As a result, the only large computerized Bayesian study of a practical problem in the public domain during the Bayesian revival of the 1960s was the Mosteller–Wallace study of The Federalist in 1964. It would be 11 years before the next major Bayesian application appeared in public. And after Tukey stopped consulting for NBC in 1980, it would be 28 years before a presidential election poll utilized Bayesian techniques again. When Nate Silver at FiveThirtyEight.com used hierarchical Bayes during the presidential race in November 2008, he combined information from outside areas to strengthen small samples from low-population areas and from exit polls with low response rates. He weighted the results of other pollsters according to their track records and sample size and how up to date their data were. He also combined them with historical polling data.


pages: 480 words: 119,407

Invisible Women by Caroline Criado Perez

Affordable Care Act / Obamacare, augmented reality, Bernie Sanders, collective bargaining, crowdsourcing, Diane Coyle, Donald Trump, falling living standards, first-past-the-post, gender pay gap, gig economy, glass ceiling, Grace Hopper, Hacker Ethic, Indoor air pollution, informal economy, lifelogging, low skilled workers, mental accounting, meta analysis, meta-analysis, Nate Silver, new economy, obamacare, Oculus Rift, offshore financial centre, pattern recognition, phenotype, post-industrial society, randomized controlled trial, remote working, Silicon Valley, Simon Kuznets, speech recognition, stem cell, Stephen Hawking, Steven Levy, the built environment, urban planning, women in the workforce, zero-sum game

In fact the number of flexible workers in the US fell between 2015 and 2016 and several major US companies are rescinding their remote work policies.112 In the UK half of employees would like to work flexibly, but only 9.8% of job ads offer flexible working113 – and women in particular who request it report being penalised. Companies also still seem to conflate long hours in the office with job effectiveness, routinely and disproportionately rewarding employees who work long hours.114 This constitutes a bonus for men. Statistician Nate Silver found that in the US, the hourly wage for those working fifty hours or more – 70% of whom are men – has risen twice as fast since 1984 as hourly pay for those working a more typical thirty-five to forty-nine hours per week.115 And this invisible male bias is exacerbated in certain countries by tax systems that exempt overtime hours from tax116 – a bonus for being unencumbered117 that contrasts sharply with the tax relief on domestic services being trialled in Sweden.118 The long-hours bias is particularly acute in Japan where it is not unusual for employees to stay in the office past midnight.


pages: 503 words: 131,064

Liars and Outliers: How Security Holds Society Together by Bruce Schneier

airport security, barriers to entry, Berlin Wall, Bernie Madoff, Bernie Sanders, Brian Krebs, Broken windows theory, carried interest, Cass Sunstein, Chelsea Manning, commoditize, corporate governance, crack epidemic, credit crunch, crowdsourcing, cuban missile crisis, Daniel Kahneman / Amos Tversky, David Graeber, desegregation, don't be evil, Double Irish / Dutch Sandwich, Douglas Hofstadter, experimental economics, Fall of the Berlin Wall, financial deregulation, George Akerlof, hydraulic fracturing, impulse control, income inequality, invention of agriculture, invention of gunpowder, iterative process, Jean Tirole, John Nash: game theory, joint-stock company, Julian Assange, longitudinal study, mass incarceration, meta analysis, meta-analysis, microcredit, moral hazard, mutually assured destruction, Nate Silver, Network effects, Nick Leeson, offshore financial centre, patent troll, phenotype, pre–internet, principal–agent problem, prisoner's dilemma, profit maximization, profit motive, race to the bottom, Ralph Waldo Emerson, RAND corporation, rent-seeking, RFID, Richard Thaler, risk tolerance, Ronald Coase, security theater, shareholder value, slashdot, statistical model, Steven Pinker, Stuxnet, technological singularity, The Market for Lemons, The Nature of the Firm, The Spirit Level, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, theory of mind, too big to fail, traffic fines, transaction costs, ultimatum game, UNCLOS, union organizing, Vernor Vinge, WikiLeaks, World Values Survey, Y2K, zero-sum game

strip-search every Electronic Privacy Information Center (10 Mar 2011), “DHS: We Have the Authority to Routinely Strip-Search Air Travelers,” press release. Electronic Privacy Information Center (15 Jul 2011), “Federal Appeals Court: TSA Violated Federal Law, Must Take Public Comment on Body Scanners,” press release. EPIC v. DHS (15 Jul 2011), Opinion, Case No. 10N1157. DC Circuit Court of Appeals, filed November 1, 2010. September 11 attacks Nate Silver (4 Jan 2010), “The Skies Are as Friendly as Ever: 9/11, Al Qaeda Obscure Statistics on Airline Safety,” FiveThirtyEight.com. scale is too large Bruce Schneier (2008), “Seven Habits of Highly Unsuccessful Terrorists,” Wired News. Max Abrams (2008), “What Terrorists Really Want,” International Security, 32:78–105. regulatory capture Jean J. Laffont and Jean Tirole (1991), “The Politics of Government Decision-Making: A Theory of Regulatory Capture,” The Quarterly Journal of Economics, 106:1089–127.


pages: 504 words: 129,087

The Ones We've Been Waiting For: How a New Generation of Leaders Will Transform America by Charlotte Alter

"side hustle", 4chan, affirmative action, Affordable Care Act / Obamacare, basic income, Berlin Wall, Bernie Sanders, carbon footprint, clean water, collective bargaining, Columbine, corporate personhood, correlation does not imply causation, Credit Default Swap, crowdsourcing, David Brooks, Donald Trump, double helix, East Village, ending welfare as we know it, Fall of the Berlin Wall, feminist movement, Ferguson, Missouri, financial deregulation, Francis Fukuyama: the end of history, gig economy, glass ceiling, Google Hangouts, housing crisis, illegal immigration, immigration reform, income inequality, Intergovernmental Panel on Climate Change (IPCC), job-hopping, Kevin Kelly, knowledge economy, Lyft, mandatory minimum, Marc Andreessen, Mark Zuckerberg, mass incarceration, McMansion, medical bankruptcy, move fast and break things, move fast and break things, Nate Silver, obamacare, Occupy movement, passive income, pre–internet, race to the bottom, RAND corporation, Ronald Reagan, sexual politics, Silicon Valley, single-payer health, Snapchat, TaskRabbit, too big to fail, Uber and Lyft, uber lyft, universal basic income, unpaid internship, We are the 99%, white picket fence, working poor, Works Progress Administration

to weaken environmental regulations: Website of Senator Edward Markey, Report of the Senate Climate Change Task Force, “The Most Anti-Climate Administration in History,” n.d., markey.senate.gov/imo/media/doc/ANTI-CLIMATE%20REPORT%20.pdf. sixty-one in the Senate: “The 115th Congress Is Among the Oldest in History,” Quorum, n.d., quorum.us/data-driven-insights/the-115th-congress-is-among-the-oldest-in-history/175/. Republicans in the House: Nate Silver and Dhrumil Mehta, “Both Republicans and Democrats Have an Age Problem,” FiveThirtyEight, April 28, 2014, fivethirtyeight.com/features/both-republicans-and-democrats-have-an-age-problem/. hearing were an average: Emily Stewart, “Lawmakers Seem Confused About What Facebook Does—And How to Fix It,” Vox, April 10, 2018, vox.com/policy-and-politics/2018/4/10/17222062/mark-zuckerberg-testimony-graham-facebook-regulations.


pages: 515 words: 143,055

The Attention Merchants: The Epic Scramble to Get Inside Our Heads by Tim Wu

1960s counterculture, Affordable Care Act / Obamacare, AltaVista, Andrew Keen, anti-communist, Apple II, Apple's 1984 Super Bowl advert, barriers to entry, Bob Geldof, borderless world, Brownian motion, Burning Man, Cass Sunstein, citizen journalism, colonial rule, East Village, future of journalism, George Gilder, Golden Gate Park, Googley, Gordon Gekko, housing crisis, informal economy, Internet Archive, Jaron Lanier, Jeff Bezos, jimmy wales, Live Aid, Mark Zuckerberg, Marshall McLuhan, McMansion, Nate Silver, Network effects, Nicholas Carr, placebo effect, post scarcity, race to the bottom, road to serfdom, Saturday Night Live, science of happiness, self-driving car, side project, Silicon Valley, slashdot, Snapchat, Steve Jobs, Steve Wozniak, Steven Levy, Ted Nelson, telemarketer, the built environment, The Chicago School, the scientific method, The Structural Transformation of the Public Sphere, Tim Cook: Apple, Torches of Freedom, Upton Sinclair, upwardly mobile, white flight, zero-sum game

Bill Gueskin, Ava Seave, and Lucas Graves, “Chapter Six: Aggregation,” Columbia Journalism Review, May 10, 2011, http://www.cjr.org/​the_business_of_digital_journalism/​chapter_six_aggregation.php. For additional statistics on and details about The Huffington Post’s growth, see Michael Shapiro, “Six Degrees of Aggregation: How The Huffington Post Ate the Internet,” Columbia Journalism Review, June 2012, http://www.cjr.org/​cover_story/​six_degrees_of_aggregation.php. 8. As quoted in Joseph Turow, The Daily You (New Haven, CT: Yale University Press, 2011), 117. Nate Silver, “The Economics of Blogging and The Huffington Post,” New York Times, February 12, 2011, http://fivethirtyeight.blogs.nytimes.com/​2011/​02/​12/​the-economics-of-blogging-and-the-huffington-post/; Bill Keller, “All the Aggregation That’s Fit to Aggregate,” New York Times Magazine, March 10, 2011, http://www.nytimes.com/​2011/​03/​13/​magazine/​mag-13lede-t.html?_r=0; Mike Friedrichsen and Wolfgang Muhl-Benninghaus, eds., Handbook of Social Media Management: Value Chain and Business Models in Changing Media Markets (New York: Springer Heidelberg, 2013). 9.


pages: 389 words: 131,688

The Impossible Climb: Alex Honnold, El Capitan, and the Climbing Life by Mark Synnott

blue-collar work, California gold rush, Google Earth, index fund, Nate Silver, Skype, South China Sea, Steve Jobs, technological singularity, The Signal and the Noise by Nate Silver, trade route, Y2K

Alex wasn’t glowing and animated like I’d seen him after other big successful days—too much hadn’t gone well for everyone. But he was more chatty than usual, and a question I’d been pondering came to mind. He had read three books in Taghia. Open, The Push (Tommy’s autobiography that he shared with Alex in real time via thumb drive as he was writing it at the gîte), and The Signal and the Noise by Nate Silver. Somehow, he had also found time to watch at least three seasons of Spartacus. The Signal and the Noise is all about statistical probability and why most predictions fail. In the book, Silver explains what he calls the prediction paradox: “The more humility we have about our ability to make predictions, the more successful we can be in planning for the future.” I found it interesting that the world’s greatest free soloist was reading a book about probability in the weeks leading up to what could be called the ultimate gamble.


pages: 660 words: 141,595

Data Science for Business: What You Need to Know About Data Mining and Data-Analytic Thinking by Foster Provost, Tom Fawcett

Albert Einstein, Amazon Mechanical Turk, big data - Walmart - Pop Tarts, bioinformatics, business process, call centre, chief data officer, Claude Shannon: information theory, computer vision, conceptual framework, correlation does not imply causation, crowdsourcing, data acquisition, David Brooks, en.wikipedia.org, Erik Brynjolfsson, Gini coefficient, information retrieval, intangible asset, iterative process, Johann Wolfgang von Goethe, Louis Pasteur, Menlo Park, Nate Silver, Netflix Prize, new economy, p-value, pattern recognition, placebo effect, price discrimination, recommendation engine, Ronald Coase, selection bias, Silicon Valley, Skype, speech recognition, Steve Jobs, supply-chain management, text mining, The Signal and the Noise by Nate Silver, Thomas Bayes, transaction costs, WikiLeaks

Comparison against a random model establishes that there is some information to be extracted from the data. However, beating a random model may be easy (or may seem easy), so demonstrating superiority to it may not be very interesting or informative. A data scientist will often need to implement an alternative model, usually one that is simple but not simplistic, in order to justify continuing the data mining effort. In Nate Silver’s book on prediction, The Signal and the Noise (2012), he mentions the baseline issue with respect to weather forecasting: There are two basic tests that any weather forecast must pass to demonstrate its merit: It must do better than what meteorologists call persistence: the assumption that the weather will be the same tomorrow (and the next day) as it was today. It must also beat climatology, the long-term historical average of conditions on a particular date in a particular area.


pages: 598 words: 134,339

Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World by Bruce Schneier

23andMe, Airbnb, airport security, AltaVista, Anne Wojcicki, augmented reality, Benjamin Mako Hill, Black Swan, Boris Johnson, Brewster Kahle, Brian Krebs, call centre, Cass Sunstein, Chelsea Manning, citizen journalism, cloud computing, congestion charging, disintermediation, drone strike, Edward Snowden, experimental subject, failed state, fault tolerance, Ferguson, Missouri, Filter Bubble, Firefox, friendly fire, Google Chrome, Google Glasses, hindsight bias, informal economy, Internet Archive, Internet of things, Jacob Appelbaum, Jaron Lanier, John Markoff, Julian Assange, Kevin Kelly, license plate recognition, lifelogging, linked data, Lyft, Mark Zuckerberg, moral panic, Nash equilibrium, Nate Silver, national security letter, Network effects, Occupy movement, Panopticon Jeremy Bentham, payday loans, pre–internet, price discrimination, profit motive, race to the bottom, RAND corporation, recommendation engine, RFID, Ross Ulbricht, self-driving car, Shoshana Zuboff, Silicon Valley, Skype, smart cities, smart grid, Snapchat, social graph, software as a service, South China Sea, stealth mode startup, Steven Levy, Stuxnet, TaskRabbit, telemarketer, Tim Cook: Apple, transaction costs, Uber and Lyft, uber lyft, undersea cable, urban planning, WikiLeaks, zero day

Stewart (2011), Terror, Security, and Money: Balancing the Risks, Benefits, and Costs of Homeland Security, Oxford University Press, chap. 9, http://books.google.com/books? id=l1IrmjCdguYC&pg=PA172. it’s well past time to move beyond fear: I even wrote a book with that title. Bruce Schneier (2003), Beyond Fear: Thinking Sensibly about Security in an Uncertain World, Wiley, http://books.google.com/books/about/? id=wuNImmQufGsC. shift in Americans’ perceptions: Nate Silver (10 Jul 2013), “Public opinion shifts on security-liberty balance,” Fivethirtyeight, New York Times, http://fivethirtyeight.blogs.nytimes.com/2013/07/10/public-opinion-shifts-onsecurity-liberty-balance. Our personal definitions of privacy: New York University law professor Helen Nissenbaum argues that privacy can only be properly understood in terms of context and expectations. Helen Nissenbaum (Fall 2011), “A contextual approach to privacy online,” Daedalus 11, http://www.amacad.org/publications/daedalus/11_fall_nissenbaum.pdf.


Data Wrangling With Python: Tips and Tools to Make Your Life Easier by Jacqueline Kazil

Amazon Web Services, bash_history, cloud computing, correlation coefficient, crowdsourcing, data acquisition, database schema, Debian, en.wikipedia.org, Firefox, Google Chrome, job automation, Nate Silver, natural language processing, pull request, Ronald Reagan, Ruby on Rails, selection bias, social web, statistical model, web application, WikiLeaks

You also have a working knowledge of Python and numerous useful libraries at your fingertips. If you don’t yet have a passion for a particular field or dataset, you’ll want to discover ways to continue your progress and advancement as a data wrangler with new fields of study. There are many great data analysts out there writing inspirational stories. Here are a few: • FiveThirtyEight, once a blog started by Nate Silver for The New York Times, is now a site with numerous writers and analysts investigating a variety of topics. After the Ferguson grand jury decision to not indict Darren Wilson, FiveThir‐ tyEight published an article showing the outcome was an outlier. With controver‐ sial topics, being able to show a data trend or tendency can help take some of the emotions out of the story and reveal what the data is actually saying. • A study of income gaps by The Washington Post used tax and census data to con‐ clude the “ol’ boy network” was still alive in terms of job acquisition and initial salaries, but usually flattened or showed no correlation after those initial jobs were acquired. • We’ve studied some of the impacts of groups in Africa who use child labor, including for mining conflict minerals.


pages: 688 words: 147,571

Robot Rules: Regulating Artificial Intelligence by Jacob Turner

Ada Lovelace, Affordable Care Act / Obamacare, AI winter, algorithmic trading, artificial general intelligence, Asilomar, Asilomar Conference on Recombinant DNA, autonomous vehicles, Basel III, bitcoin, blockchain, brain emulation, Clapham omnibus, cognitive dissonance, corporate governance, corporate social responsibility, correlation does not imply causation, crowdsourcing, distributed ledger, don't be evil, Donald Trump, easy for humans, difficult for computers, effective altruism, Elon Musk, financial exclusion, financial innovation, friendly fire, future of work, hive mind, Internet of things, iterative process, job automation, John Markoff, John von Neumann, Loebner Prize, medical malpractice, Nate Silver, natural language processing, nudge unit, obamacare, off grid, pattern recognition, Peace of Westphalia, race to the bottom, Ray Kurzweil, Rodney Brooks, self-driving car, Silicon Valley, Stanislav Petrov, Stephen Hawking, Steve Wozniak, strong AI, technological singularity, Tesla Model S, The Coming Technological Singularity, The Future of Employment, The Signal and the Noise by Nate Silver, Turing test, Vernor Vinge

See Anna-Louise Taylor, “Why Infanticide Can Benefit Animals”, BBC Nature, 21 March 2012, http://​www.​bbc.​co.​uk/​nature/​18035811, accessed 1 June 2018. 100For proposals along these lines, see Oren Etzioni, “How to Regulate Artificial Intelligence”, The New York Times, 1 September 2017, https://​www.​nytimes.​com/​2017/​09/​01/​opinion/​artificial-intelligence-regulations-rules.​html, accessed 1 June 2018. 101Director of Public Prosecutions, “Suicide: Policy for Prosecutors in Respect of Cases of Encouraging or Assisting Suicide”, February 2010, updated October 2014, https://​www.​cps.​gov.​uk/​legal-guidance/​suicide-policy-prosecutors-respect-cases-encouraging-or-assisting-suicide, accessed 1 June 2018. 102As we explore in later chapters, the theoretical ability for AI to avoid human bias does not obviate the need to ensure that those humans originally programming AI or providing their seed data sets do not accidentally or intentionally imbue AI with human fallibilities or prejudice. 103Luciano Floridi, “A Fallacy that Will Hinder Advances in Artificial Intelligence”, The Financial Times, 1 June 2017, https://​www.​ft.​com/​content/​ee996846-4626-11e7-8d27-59b4dd6296b8, accessed 1 June 2018. See also Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (London: Penguin, 2012), 287–288. 104Philippa Foot, The Problem of Abortion and the Doctrine of the Double Effect in Virtues and Vices (Oxford: Basil Blackwell, 1978) (the article originally appeared in the Oxford Review, Number 5, 1967). 105See Judith Jarvis Thompson, “The Trolley Problem”, Yale Law Journal, Vol. 94, No. 6 (May, 1985), 1395–1415. 106In this book, the terms “self-driving” and “autonomous” when used in relation to vehicles refer to the delegation by humans of certain decision-making functions featuring in driving.


Not Working by Blanchflower, David G.

active measures, affirmative action, Affordable Care Act / Obamacare, Albert Einstein, bank run, banking crisis, basic income, Berlin Wall, Bernie Madoff, Bernie Sanders, Black Swan, Boris Johnson, business cycle, Capital in the Twenty-First Century by Thomas Piketty, Carmen Reinhart, Clapham omnibus, collective bargaining, correlation does not imply causation, credit crunch, declining real wages, deindustrialization, Donald Trump, estate planning, Fall of the Berlin Wall, full employment, George Akerlof, gig economy, Gini coefficient, Growth in a Time of Debt, illegal immigration, income inequality, indoor plumbing, inflation targeting, job satisfaction, John Bercow, Kenneth Rogoff, labor-force participation, liquidationism / Banker’s doctrine / the Treasury view, longitudinal study, low skilled workers, manufacturing employment, Mark Zuckerberg, market clearing, Martin Wolf, mass incarceration, meta analysis, meta-analysis, moral hazard, Nate Silver, negative equity, new economy, Northern Rock, obamacare, oil shock, open borders, Own Your Own Home, p-value, Panamax, pension reform, plutocrats, Plutocrats, post-materialism, price stability, prisoner's dilemma, quantitative easing, rent control, Richard Thaler, Robert Shiller, Robert Shiller, Ronald Coase, selection bias, selective serotonin reuptake inhibitor (SSRI), Silicon Valley, South Sea Bubble, Thorstein Veblen, trade liberalization, universal basic income, University of East Anglia, urban planning, working poor, working-age population, yield curve

I also found that the Trump-Romney difference was positively correlated with the heavy drinking rate and obesity rates, and negatively correlated with life-expectancy rates. By state the Trump vote is positively correlated with the suicide rate and the incidence of bad mental health. Places that were hurting the most voted for Trump. I report below that disadvantaged communities, on a host of measures, in the UK went for Brexit and in France for Le Pen. Nate Silver (2016) took a list of all 981 U.S. counties with 50,000 or more people based on data from the American Community Survey, and sorted it by the share of the population age 25 and over that had completed at least a four-year college degree. He found that Hillary Clinton improved on President Obama’s 2012 performance in 48 of the country’s 50 most well-educated counties. And on average, she improved on Obama’s margin of victory in these countries by almost 9 percentage points, even though Obama had done well in them to begin with.


pages: 554 words: 167,247

America's Bitter Pill: Money, Politics, Backroom Deals, and the Fight to Fix Our Broken Healthcare System by Steven Brill

Affordable Care Act / Obamacare, barriers to entry, Bernie Sanders, business process, call centre, collapse of Lehman Brothers, collective bargaining, crony capitalism, desegregation, Donald Trump, Edward Snowden, employer provided health coverage, medical malpractice, Menlo Park, Nate Silver, obamacare, Potemkin village, Ronald Reagan, Saturday Night Live, side project, Silicon Valley, the payments system, young professional

It sounds like the Obama campaign. And administration officials don’t shy away from the comparison. Even Jeanne Lambrew—who was supposed to be the White House official overseeing the website build—agreed that Simas’s work was what the launch was all about: “When I hear the conventional wisdom about Obamacare,” she told the Post, “this is the difference between the Karl Roves who put their fingers to the wind and the Nate Silvers of the world who looked at the numbers.” Implementing President Obama’s most important domestic policy seemed to be all about campaigning, not about governing. The officials at CMS and HHS whom I interviewed during the summer of 2013 were similarly focused. I was regaled with stories about the demographic targeting being done. In fact, the CMS staff claimed that its data, more than that supplied by Civis, was fueling the campaign.


pages: 593 words: 189,857

Stress Test: Reflections on Financial Crises by Timothy F. Geithner

Affordable Care Act / Obamacare, asset-backed security, Atul Gawande, bank run, banking crisis, Basel III, Bernie Madoff, Bernie Sanders, break the buck, Buckminster Fuller, Carmen Reinhart, central bank independence, collateralized debt obligation, correlation does not imply causation, creative destruction, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, currency manipulation / currency intervention, David Brooks, Doomsday Book, eurozone crisis, financial innovation, Flash crash, Goldman Sachs: Vampire Squid, housing crisis, Hyman Minsky, illegal immigration, implied volatility, Kickstarter, London Interbank Offered Rate, Long Term Capital Management, margin call, market fundamentalism, Martin Wolf, McMansion, Mexican peso crisis / tequila crisis, money market fund, moral hazard, mortgage debt, Nate Silver, negative equity, Northern Rock, obamacare, paradox of thrift, pets.com, price stability, profit maximization, pushing on a string, quantitative easing, race to the bottom, RAND corporation, regulatory arbitrage, reserve currency, Saturday Night Live, savings glut, selection bias, short selling, sovereign wealth fund, The Great Moderation, The Signal and the Noise by Nate Silver, Tobin tax, too big to fail, working poor

But the investors, reaching for yield, had shown little interest in the safest tranches, the “super-senior” CDOs that would pay out in full unless mortgage losses were so severe that investors in every tranche below them were wiped out. That seemed highly unlikely, so Merrill usually kept the super-seniors on its balance sheet. Their modest returns were still more than the cost of financing them, and they seemed almost bulletproof. Standard & Poor’s estimated a mere 0.12 percent chance that one of its AAA-rated CDOs would fail to pay out over five years—and super-seniors were considered safer than typical AAAs. But as Nate Silver noted in The Signal and the Noise, his excellent book about why many predictions fail, the actual default rate for AAA-rated tranches of CDOs would be 28 percent, more than two hundred times higher than S&P had predicted. Their perceived safety rested on all kinds of flawed assumptions, starting with the notion that housing prices would never fall simultaneously across the country. CDOs were often spliced together from geographically diverse piles of subprime mortgages, which was supposed to mitigate the effects of a housing slump in any one region.


pages: 677 words: 206,548

Future Crimes: Everything Is Connected, Everyone Is Vulnerable and What We Can Do About It by Marc Goodman

23andMe, 3D printing, active measures, additive manufacturing, Affordable Care Act / Obamacare, Airbnb, airport security, Albert Einstein, algorithmic trading, artificial general intelligence, Asilomar, Asilomar Conference on Recombinant DNA, augmented reality, autonomous vehicles, Baxter: Rethink Robotics, Bill Joy: nanobots, bitcoin, Black Swan, blockchain, borderless world, Brian Krebs, business process, butterfly effect, call centre, Charles Lindbergh, Chelsea Manning, cloud computing, cognitive dissonance, computer vision, connected car, corporate governance, crowdsourcing, cryptocurrency, data acquisition, data is the new oil, Dean Kamen, disintermediation, don't be evil, double helix, Downton Abbey, drone strike, Edward Snowden, Elon Musk, Erik Brynjolfsson, Filter Bubble, Firefox, Flash crash, future of work, game design, global pandemic, Google Chrome, Google Earth, Google Glasses, Gordon Gekko, high net worth, High speed trading, hive mind, Howard Rheingold, hypertext link, illegal immigration, impulse control, industrial robot, Intergovernmental Panel on Climate Change (IPCC), Internet of things, Jaron Lanier, Jeff Bezos, job automation, John Harrison: Longitude, John Markoff, Joi Ito, Jony Ive, Julian Assange, Kevin Kelly, Khan Academy, Kickstarter, knowledge worker, Kuwabatake Sanjuro: assassination market, Law of Accelerating Returns, Lean Startup, license plate recognition, lifelogging, litecoin, low earth orbit, M-Pesa, Mark Zuckerberg, Marshall McLuhan, Menlo Park, Metcalfe’s law, MITM: man-in-the-middle, mobile money, more computing power than Apollo, move fast and break things, move fast and break things, Nate Silver, national security letter, natural language processing, obamacare, Occupy movement, Oculus Rift, off grid, offshore financial centre, optical character recognition, Parag Khanna, pattern recognition, peer-to-peer, personalized medicine, Peter H. Diamandis: Planetary Resources, Peter Thiel, pre–internet, RAND corporation, ransomware, Ray Kurzweil, refrigerator car, RFID, ride hailing / ride sharing, Rodney Brooks, Ross Ulbricht, Satoshi Nakamoto, Second Machine Age, security theater, self-driving car, shareholder value, Silicon Valley, Silicon Valley startup, Skype, smart cities, smart grid, smart meter, Snapchat, social graph, software as a service, speech recognition, stealth mode startup, Stephen Hawking, Steve Jobs, Steve Wozniak, strong AI, Stuxnet, supply-chain management, technological singularity, telepresence, telepresence robot, Tesla Model S, The Future of Employment, The Wisdom of Crowds, Tim Cook: Apple, trade route, uranium enrichment, Wall-E, Watson beat the top human players on Jeopardy!, Wave and Pay, We are Anonymous. We are Legion, web application, Westphalian system, WikiLeaks, Y Combinator, zero day

On a daily basis, cyber attacks disrupt our financial system, thieves steal billions in intellectual property, foreign nations pilfer our military weapons plans, and hackers share online tips with one another on how to take over the industrial control systems that run everything from power plants to water and sewage treatment facilities. To paraphrase the renowned statistician and editor of the FiveThirtyEight blog, Nate Silver, our current lackadaisical approach to cyber security and the profound technological vulnerabilities before us has been until this point akin to applying sunscreen and claiming it protects us from a nuclear meltdown—wholly inadequate to the scale of the problem. It is time for a stone-cold somber rethinking of our current state of affairs. It’s time for a Manhattan Project for cyber security.


pages: 691 words: 203,236

Whiteshift: Populism, Immigration and the Future of White Majorities by Eric Kaufmann

4chan, affirmative action, Amazon Mechanical Turk, anti-communist, anti-globalists, augmented reality, battle of ideas, Berlin Wall, Bernie Sanders, Boris Johnson, British Empire, centre right, Chelsea Manning, cognitive dissonance, complexity theory, corporate governance, correlation does not imply causation, crowdsourcing, Daniel Kahneman / Amos Tversky, David Brooks, deindustrialization, demographic transition, Donald Trump, Elon Musk, en.wikipedia.org, facts on the ground, failed state, Fall of the Berlin Wall, first-past-the-post, Francis Fukuyama: the end of history, Haight Ashbury, illegal immigration, immigration reform, imperial preference, income inequality, knowledge economy, knowledge worker, liberal capitalism, longitudinal study, Lyft, mass immigration, meta analysis, meta-analysis, moral panic, Nate Silver, New Urbanism, Norman Mailer, open borders, phenotype, postnationalism / post nation state, Ralph Waldo Emerson, Republic of Letters, Ronald Reagan, Scientific racism, Silicon Valley, statistical model, Steven Pinker, the built environment, the scientific method, The Wisdom of Crowds, transcontinental railway, twin studies, uber lyft, upwardly mobile, urban sprawl, Washington Consensus, white flight, working-age population, World Values Survey, young professional

For instance, just 5 per cent of Brexit voters think inequality is the most important issue facing Britain while over 20 per cent of Remain voters do.95 In addition, the BES asks a battery of five questions on anti-elitism such as ‘the people, not politicians, should make our most important policy decisions’ or ‘politicians in the UK parliament need to follow the will of the people’. None of these items sorts Leavers from Remainers: socialist Corbyn supporters and Greens also tend to agree with them. As with the Trump phenomenon, opposition to a powerful, out-of-touch and wealthy elite does not explain the vote. SOCIAL PSYCHOLOGY AND THE BREXIT VOTE More sophisticated data journalists such as John Burn-Murdoch of the Financial Times in Britain or Nate Silver in the US were astute enough to spot that average education, not income, is the best census predictor of Brexit or Trump support in a district.96 Why education and not income? Education is a signal of worldview, not just material prosperity. Education and income are correlated, but it’s possible to be a successful building contractor with no degree or a penniless graduate. In the BES, for instance, 16 per cent of whites without degrees earn above-average incomes and 29 per cent of whites with degrees are in the lowest income bracket.


pages: 976 words: 235,576

The Meritocracy Trap: How America's Foundational Myth Feeds Inequality, Dismantles the Middle Class, and Devours the Elite by Daniel Markovits

"Robert Solow", 8-hour work day, activist fund / activist shareholder / activist investor, affirmative action, Anton Chekhov, asset-backed security, assortative mating, basic income, Bernie Sanders, big-box store, business cycle, capital asset pricing model, Capital in the Twenty-First Century by Thomas Piketty, carried interest, collateralized debt obligation, collective bargaining, computer age, corporate governance, corporate raider, crony capitalism, David Brooks, deskilling, Detroit bankruptcy, disruptive innovation, Donald Trump, Edward Glaeser, Emanuel Derman, equity premium, European colonialism, everywhere but in the productivity statistics, fear of failure, financial innovation, financial intermediation, fixed income, Ford paid five dollars a day, Frederick Winslow Taylor, full employment, future of work, gender pay gap, George Akerlof, Gini coefficient, glass ceiling, helicopter parent, high net worth, hiring and firing, income inequality, industrial robot, interchangeable parts, invention of agriculture, Jaron Lanier, Jeff Bezos, job automation, job satisfaction, John Maynard Keynes: Economic Possibilities for our Grandchildren, knowledge economy, knowledge worker, Kodak vs Instagram, labor-force participation, longitudinal study, low skilled workers, manufacturing employment, Mark Zuckerberg, Martin Wolf, mass incarceration, medical residency, minimum wage unemployment, Myron Scholes, Nate Silver, New Economic Geography, new economy, offshore financial centre, Paul Samuelson, payday loans, plutocrats, Plutocrats, Plutonomy: Buying Luxury, Explaining Global Imbalances, precariat, purchasing power parity, rent-seeking, Richard Florida, Robert Gordon, Robert Shiller, Robert Shiller, Ronald Reagan, savings glut, school choice, shareholder value, Silicon Valley, Simon Kuznets, six sigma, Skype, stakhanovite, stem cell, Steve Jobs, supply-chain management, telemarketer, The Bell Curve by Richard Herrnstein and Charles Murray, Thomas Davenport, Thorstein Veblen, too big to fail, total factor productivity, transaction costs, traveling salesman, universal basic income, unpaid internship, Vanguard fund, War on Poverty, Winter of Discontent, women in the workforce, working poor, young professional, zero-sum game

., “Election 2016: Exit Polls,” New York Times, November 8, 2016, accessed July 24, 2018, www.nytimes.com/interactive/2016/11/08/us/politics/election-exit-polls.html. For a largely congruent analysis of the sources of Trump’s support using a massive data set of preelection Gallup surveys, see Jonathan Rothwell and Pablo Diego-Rosell, “Explaining Nationalist Political Views: The Case of Donald Trump,” SSRN working paper (November 2, 2016), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2822059. over Romney’s 2012 results: See Nate Silver, “Education, Not Income, Predicted Who Would Vote for Trump,” FiveThirtyEight, November 22, 2016, accessed July 24, 2018, http://fivethirtyeight.com/features/education-not-income-predicted-who-would-vote-for-trump/. For a similar analysis, see Neera Tanden et al., “Towards a Marshall Plan for America,” Center for American Progress, May 16, 2017, accessed July 24, 2018, www.americanprogress.org/issues/economy/reports/2017/05/16/432499/toward-marshall-plan-america/.