big data - Walmart - Pop Tarts

17 results back to index


pages: 304 words: 82,395

Big Data: A Revolution That Will Transform How We Live, Work, and Think by Viktor Mayer-Schonberger, Kenneth Cukier

23andMe, Affordable Care Act / Obamacare, airport security, Apollo 11, barriers to entry, Berlin Wall, big data - Walmart - Pop Tarts, Black Swan, book scanning, book value, business intelligence, business process, call centre, cloud computing, computer age, correlation does not imply causation, dark matter, data science, double entry bookkeeping, Eratosthenes, Erik Brynjolfsson, game design, hype cycle, IBM and the Holocaust, index card, informal economy, intangible asset, Internet of things, invention of the printing press, Jeff Bezos, Joi Ito, lifelogging, Louis Pasteur, machine readable, machine translation, Marc Benioff, Mark Zuckerberg, Max Levchin, Menlo Park, Moneyball by Michael Lewis explains big data, Nate Silver, natural language processing, Netflix Prize, Network effects, obamacare, optical character recognition, PageRank, paypal mafia, performance metric, Peter Thiel, Plato's cave, post-materialism, random walk, recommendation engine, Salesforce, self-driving car, sentiment analysis, Silicon Valley, Silicon Valley startup, smart grid, smart meter, social graph, sparse data, speech recognition, Steve Jobs, Steven Levy, systematic bias, the scientific method, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, Thomas Davenport, Turing test, vertical integration, Watson beat the top human players on Jeopardy!

For a good description, see: Alex Howard, “Predictive data analytics is saving lives and taxpayer dollars in New York City,” O’Reilly Media, June 26, 2012 (http://strata.oreilly.com/2012/06/predictive-data-analytics-big-data-nyc.html). [>] Walmart and Pop-Tarts—Hays, “What Wal-Mart Knows About Customers’ Habits.” [>] Big data’s use in slums and in modeling refugee movements—Nathan Eagle, “Big Data, Global Development, and Complex Systems,” http://www.youtube.com/watch?v=yaivtqlu7iM. Perception of time—Benedict Anderson, Imagined Communities (Verso, 2006). [>] “What’s past is prologue”—William Shakespeare, “The Tempest,” Act 2, Scene I. [>] CERN experiment and data storage—Cukier email exchange with CERN researchers, November 2012.

But for many everyday needs, knowing what not why is good enough. And big-data correlations can point the way toward promising areas in which to explore causal relationships. These quick correlations let us save money on plane tickets, predict flu outbreaks, and know which manholes or overcrowded buildings to inspect in a resource-constrained world. They may enable health insurance firms to provide coverage without a physical exam and lower the cost of reminding the sick to take their medication. Languages are translated and cars drive themselves on the basis of predictions made through big-data correlations. Walmart can learn which flavor Pop-Tarts to stock at the front of the store before a hurricane.

They are often the consultancies, technology vendors, and analytics providers who have special expertise and do the work, but probably do not have the data themselves nor the ingenuity to come up with the most innovative uses for it. In the case of Walmart and Pop-Tarts, for example, the retailer turned to the specialists at Teradata, a data-analytics firm, to help tease out the insights. Third is the big-data mindset. For certain firms, the data and the know-how are not the main reasons for their success. What sets them apart is that their founders and employees have unique ideas about ways to tap data to unlock new forms of value.


pages: 337 words: 86,320

Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are by Seth Stephens-Davidowitz

affirmative action, AltaVista, Amazon Mechanical Turk, Asian financial crisis, Bernie Sanders, big data - Walmart - Pop Tarts, Black Lives Matter, Cass Sunstein, computer vision, content marketing, correlation does not imply causation, crowdsourcing, Daniel Kahneman / Amos Tversky, data science, desegregation, Donald Trump, Edward Glaeser, Filter Bubble, game design, happiness index / gross national happiness, income inequality, Jeff Bezos, Jeff Seder, John Snow's cholera map, longitudinal study, Mark Zuckerberg, Nate Silver, Nick Bostrom, peer-to-peer lending, Peter Thiel, price discrimination, quantitative hedge fund, Ronald Reagan, Rosa Parks, sentiment analysis, Silicon Valley, statistical model, Steve Jobs, Steven Levy, Steven Pinker, TaskRabbit, The Signal and the Noise by Nate Silver, working poor

And indeed, these Pop-Tarts sold well. Why Pop-Tarts? Probably because they don’t require refrigeration or cooking. Why strawberry? No clue. But when hurricanes hit, people turn to strawberry Pop-Tarts apparently. So in the days before a hurricane, Walmart now regularly stocks its shelves with boxes upon boxes of strawberry Pop-Tarts. The reason for the relationship doesn’t matter. But the relationship itself does. Maybe one day food scientists will figure out the association between hurricanes and toaster pastries filled with strawberry jam. But, while waiting for some such explanation, Walmart still needs to stock its shelves with strawberry Pop-Tarts when hurricanes are approaching and save the Rice Krispies treats for sunnier days.

And, in the prediction business, you just need to know that something works, not why. For example, Walmart uses data from sales in all their stores to know what products to shelve. Before Hurricane Frances, a destructive storm that hit the Southeast in 2004, Walmart suspected—correctly—that people’s shopping habits may change when a city is about to be pummeled by a storm. They pored through sales data from previous hurricanes to see what people might want to buy. A major answer? Strawberry Pop-Tarts. This product sells seven times faster than normal in the days leading up to a hurricane. Based on their analysis, Walmart had trucks loaded with strawberry Pop-Tarts heading down Interstate 95 toward stores in the path of the hurricane.

., 227, 228 127 Hours (movie), 90, 91 Optimal Decisions Group, 262 Or, Flora, 266 Ortiz, David “Big Papi,” 197–200, 200n, 203 “out-of-sample” tests, 250–51 Page, Larry, 60, 61, 62, 103 pancreatic cancer, Columbia University-Microsoft study of, 28–29 Pandora, 203 Pantheon project (Massachusetts Institute of Technology), 184–85 parents/parenting and child abuse, 145–47, 149–50, 161 and examples of Big Data searches, 22 and prejudice against children, 134–36, 135n Parks, Rosa, 93, 94 Parr, Ben, 153–54 Pathak, Parag, 235–36 PatientsLikeMe.com, 205 patterns, and data science as intuitive, 27, 33 Paul, Chris, 37 paying back loans, 257–61 PECOTA model, 199–200, 200n pedigrees of basketball players, 67 of horses, 66–67, 69, 71 pedometer, Chance emphasis on, 252–53 penis and Freud’s theories, 46 and phallic symbols in dreams, 46–47 size of, 17, 19, 123–24, 124n, 127 “penistrian,” 45, 46, 48, 50 Pennsylvania State University, income of graduates of, 237–39 Peysakhovich, Alex, 254 phallic symbols, in dreams, 46–48 Philadelphia Daily News, and words as data, 95 Philippines, cigarette economy in, 102 physical appearance and dating, 82, 120n and parents prejudice against children, 135–36 and truth about sex, 120, 120n, 125–26, 127 physics, as science, 272–73 pictures, as data, 97–102, 103 Pierson, Emma, 160n Piketty, Thomas, 283 Pinky Pizwaanski (horse), 70 pizza, information about, 77 PlentyOfFish (dating site), 139 Plomin, Robert, 249–50 political science, and digital revolution, 244, 274 politics and A/B testing, 211–14 complexity of, 273 and ignoring what people tell you, 157 and origin of political preferences, 169–71 and truth about the internet, 140–44 and words as data, 95–97 See also conservatives; Democrats; liberals; Republicans polls Google searches compared with, 9 and lying, 107 reliability of, 12 See also specific poll or topic Pop-Tarts, 72 Popp, Noah, 202 Popper, Karl, 45, 272, 273 PornHub (website), 14, 50–52, 54, 116, 120–22, 274 pornography as addiction, 219 and bias of social media, 151 and breastfeeding, 19 cartoon, 52 child, 121 and digital revolution, 279 and gays, 114–15, 114n, 116, 117, 119 honesty of data about, 53–54 and incest, 50–52 in India, 19 and lying, 110 popular videos on, 152 popularity of, 53, 151 and power of Big Data, 53 search engines for, 61n and truth about sex, 114–15, 117 unemployed and, 58, 59 Posada, Jorge, 200 poverty and life expectancy, 176–78 and words as data, 93, 94 See also income distribution predictions and data science as intuitive, 27 and getting the numbers right, 74 and what counts as data, 74 and what vs. why it works, 71 See also specific topic pregnancy, 20, 187–90 prejudice implicit, 132–34 of parents against children, 134–36, 135n subconscious, 134, 163 truth about, 128–40, 162–63 See also bias; hate; race/racism; Stormfront Premise, 101–2, 103 price discrimination, 262–65 prison conditions, and crime, 235 privacy issues, and danger of empowered government, 267–70 property rights, and words as data, 93, 94 proquest.com, 95 Prosper (lending site), 257 Psy, “Gangnam Style” video of, 152 psychics, 266 psychology and digital revolution, 274, 277–78, 279 as science, 273 as soft science, 273 and traditional research methods, 274 Quantcast, 137 questions asking the right, 21–22 and dating, 82–83 race/racism causes of, 18–19 elections of 2008 and, 2, 6–7, 12, 133 elections of 2012 and, 2–3, 8, 133 elections of 2016 and, 8, 11, 12, 14, 133 explicit, 133, 134 and Harvard Crimson editorial about Zuckerberg, 155 and lying, 109 map of, 7–9 and Obama, 2, 6–7, 8–9, 12, 133, 240, 243–44 and predicting success in basketball, 35, 36–37 and Republicans, 3, 7, 8 Stephens-Davidowitz’s study of, 2–3, 6–7, 12, 14, 243–44 and Trump, 8, 9, 11, 12, 14, 133 and truth about hate and prejudice, 129–34, 162–63 See also Muslims; “nigger” randomized controlled experiments and A/B testing, 209–21 and causality, 208–9 rape, 121–22, 190–91 Rawlings, Craig, 80 “rawtube” (porn site), 59 Reagan, Andy, 88, 90, 91 Reagan, Ronald, 227 regression discontinuity, 234–36 Reisinger, Joseph, 101–2, 103 relationships, lasting, 31–33 religion, and life expectancy, 177 Renaissance (hedge fund), 246 Republicans core principles of, 94 and origins of political preferences, 170–71 and racism, 3, 7, 8 and words as data, 93–97 See also specific person or election research and expansion of research methodology, 275–76 See also specific researcher or research reviews, of businesses, 265 “Rocket Tube” (gay porn site), 115 Rolling Stones, 278 Romney, Mitt, 10, 212 Roseau County, Minnesota, successful/notable Americans from, 186, 187 Runaway Bride (movie), 192, 195 sabermetricians, 198–99 San Bernardino, California, shooting in, 129–30 Sands, Emily, 202 science and Big Data, 273 and experiments, 272–73 real, 272–73 at scale, 276 soft, 273 search engines differentiation of Google from other, 60–62 for pornography, 61n reliability of, 60 word-count, 71 See also specific engine searchers, typing errors by, 48–50 searches negative words used in, 128–29 See also specific search “secrets about people,” 155–56 Seder, Jeff, 63–66, 68–70, 71, 74, 155, 256 segregation, 141–44.


pages: 239 words: 70,206

Data-Ism: The Revolution Transforming Decision Making, Consumer Behavior, and Almost Everything Else by Steve Lohr

"World Economic Forum" Davos, 23andMe, Abraham Maslow, Affordable Care Act / Obamacare, Albert Einstein, Alvin Toffler, Bear Stearns, behavioural economics, big data - Walmart - Pop Tarts, bioinformatics, business cycle, business intelligence, call centre, Carl Icahn, classic study, cloud computing, computer age, conceptual framework, Credit Default Swap, crowdsourcing, Daniel Kahneman / Amos Tversky, Danny Hillis, data is the new oil, data science, David Brooks, driverless car, East Village, Edward Snowden, Emanuel Derman, Erik Brynjolfsson, everywhere but in the productivity statistics, financial engineering, Frederick Winslow Taylor, Future Shock, Google Glasses, Ida Tarbell, impulse control, income inequality, indoor plumbing, industrial robot, informal economy, Internet of things, invention of writing, Johannes Kepler, John Markoff, John von Neumann, lifelogging, machine translation, Mark Zuckerberg, market bubble, meta-analysis, money market fund, natural language processing, obamacare, pattern recognition, payday loans, personalized medicine, planned obsolescence, precision agriculture, pre–internet, Productivity paradox, RAND corporation, rising living standards, Robert Gordon, Robert Solow, Salesforce, scientific management, Second Machine Age, self-driving car, Silicon Valley, Silicon Valley startup, SimCity, six sigma, skunkworks, speech recognition, statistical model, Steve Jobs, Steven Levy, The Design of Experiments, the scientific method, Thomas Kuhn: the structure of scientific revolutions, Tony Fadell, unbanked and underbanked, underbanked, Von Neumann architecture, Watson beat the top human players on Jeopardy!, yottabyte

Exploiting correlation is the first wave of the big-data phenomenon, and it can be extremely powerful. Indeed, useful and profitable observations increasingly do come from “listening to the data” to find correlations. A handful of large corporations have been at this for years, using their own data. A canonical example of this kind of data discovery is the Pop-Tarts-and-beer case at Walmart from a decade ago. The giant retailer, mining the historical purchasing data from its stores, found that consumers in the path of a predicted hurricane bought strawberry Pop-Tarts at seven times the usual rate and the best-selling item of all before a hurricane was beer.

Randall, 40 Mount Sinai Hospital, 8, 13–14, 15 data science and genomic research at, 163–65, 171, 173–81 medical data and human experience, 68–70 Mundie, Craig, 203 Nakashima, George, 65 Naked Society, The (Packard), 184 Narayanan, Arvind, 204 Nest learning thermostat, 143–45 Google and, 152–53 human behavior and, 147–52 Never-Ending Language Learning system (NELL), of Carnegie Mellon University, 110–11 New York State, Medicaid fraud prevention in, 48 Norvig, Peter, 116 Norway, 48 “notice and choice,” in data collection of personal information, 186, 187–88 Noyes, Eliot, 49 “numerical imagination,” of Hammerbacher, 13–14 Oak Ridge National Laboratory, 176 Obama administration, big data and, 203–4 O’Donnell, Tim, 180–81 OfficeMax, 188–89 Olmo, Harold, 126 Olson, Mike, 101 online advertising, 84–85 as “socio-technical construct,” 193–95 open-source code, IBM and, 9 operations research, 154 optimization, at IBM, 46 Packard, Vance, 184 Palmisano, Samuel, 49–51, 53 “Parable of Google Flu: Traps in Big Data Analysis, The” (Science), 108 Pattern Recognition (Gibson), 154 Paul, Sharoda, 135 payday lending market, 104–7 Pennebaker, James, 199 Pentland, Alex, 15, 203–4, 206 Perlich, Claudia, 120 personality traits, values, and needs, 198–99 personally identifying information, privacy concerns and, 187–92 Pieroni, Stephanie, 36 Pitts, Martha, 57 Pitts, Shereline, 57 Pop-Tarts, beer, and hurricane data, 104 precision agriculture, E. & J. Gallo’s use of, 123–33 Predix, 136 PricewaterhouseCoopers, 44 Principles of Scientific Management, The (Taylor), 208 Privacy Act (1974), 185 privacy concerns, 183–206 balancing privacy and data collection, 202–6 big data and personally identifying information, 187–92 cameras and, 183–86 data correlation and, 113 discrimination by statistical inference, 192–95 early computers and, 185–87 marketing and use of data, 195–97 social network data collection and, 197–202 productivity paradox, of computers, 72–75 Profiles in Performance: Business Intelligence Journeys and the Roadmap for Change (Dresner), 76 psycholinguistics, used for studying tweets, 199 Pulleyblank, William, 45–46, 47–48, 49 quantitative-to-qualitative transformation, data and, 7–8 “reality mining,” 206 Reisman, David, 155 Richardson, Tara, 106–7 Riedl, Paul, 156 Rock Health, 16 Rogers, Matt, 144 romantic relationships, social network research and, 87–88 Rometty, Virginia Haydock and, 156 IBM’s big data strategy and, 9, 42–45, 46, 47, 53–56 Rosenn, Itamar, 89–90, 94 Rotenberg, Marc, 204–5 Rothschild, Jeff, 86, 91–92, 98 Rubinsteyn, Alex, 180 Ruh, William, 134, 135–36 Sabre (Semi-Automated Business Research Environment), 46 Sage Bionetworks, 101–2, 170–71 SAS Institute, 52 satellite imagery, precision agriculture and, 129–32 Schadt, Eric background, 172–73 at Mount Sinai, 171–72, 173–74, 175 Sage Bionetworks and, 102 Schrage, Michael, 197 Science, 108 scientific management (Taylorism), 207–8 Seay, Mike, 188–89 Second Machine Age, The (Brynjolfsson and McAfee), 119–20 Shah, Rachana.

The giant retailer, mining the historical purchasing data from its stores, found that consumers in the path of a predicted hurricane bought strawberry Pop-Tarts at seven times the usual rate and the best-selling item of all before a hurricane was beer. Walmart’s store managers don’t care why that purchasing pattern occurs. They’re just going to stock up on beer and strawberry Pop-Tarts when hurricane warnings come their way. Today, this data-discovery game has opened up well beyond a select few big companies with deep pockets, vast pools of proprietary data, and corps of analysts to pore through it. Low-cost computing and software, combined with the explosion in data on the open Web and elsewhere, means small companies and start-ups can play as well. Consider start-up ZestFinance. Its cofounder and chief executive is Douglas Merrill, the former chief information officer of Google.


pages: 281 words: 71,242

World Without Mind: The Existential Threat of Big Tech by Franklin Foer

artificial general intelligence, back-to-the-land, Berlin Wall, big data - Walmart - Pop Tarts, Big Tech, big-box store, Buckminster Fuller, citizen journalism, Colonization of Mars, computer age, creative destruction, crowdsourcing, data is the new oil, data science, deep learning, DeepMind, don't be evil, Donald Trump, Double Irish / Dutch Sandwich, Douglas Engelbart, driverless car, Edward Snowden, Electric Kool-Aid Acid Test, Elon Musk, Evgeny Morozov, Fall of the Berlin Wall, Filter Bubble, Geoffrey Hinton, global village, Google Glasses, Haight Ashbury, hive mind, income inequality, intangible asset, Jeff Bezos, job automation, John Markoff, Kevin Kelly, knowledge economy, Law of Accelerating Returns, Marc Andreessen, Mark Zuckerberg, Marshall McLuhan, means of production, move fast and break things, new economy, New Journalism, Norbert Wiener, off-the-grid, offshore financial centre, PageRank, Peace of Westphalia, Peter Thiel, planetary scale, Ray Kurzweil, scientific management, self-driving car, Silicon Valley, Singularitarianism, software is eating the world, Steve Jobs, Steven Levy, Stewart Brand, strong AI, supply-chain management, TED Talk, the medium is the message, the scientific method, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Thomas L Friedman, Thorstein Veblen, Upton Sinclair, Vernor Vinge, vertical integration, We are as Gods, Whole Earth Catalog, yellow journalism

The essence of the algorithm is entirely uncomplicated: John MacCormick, Nine Algorithms That Changed the Future (Princeton University Press, 2012), 3–4. “We can stop looking for models. We can analyze the data without hypotheses”: Chris Anderson, “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete,” Wired, June 23, 2008. Walmart’s algorithms found that people desperately buy strawberry Pop-Tarts: Constance L. Hays, “What Wal-Mart Knows About Customers’ Habits,” New York Times, November 14, 2004. Sweeney conducted a study that found that users with African American names: Latanya Sweeney, “Discrimination in Online Ad Delivery,” Communications of the ACM 56, no. 5 (May 2013): 44–54.

We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.” On one level, this is undeniable. Algorithms can translate languages without understanding words, simply by uncovering the patterns that undergird the construction of sentences. They can find coincidences that humans might never even think to seek. Walmart’s algorithms found that people desperately buy strawberry Pop-Tarts as they prepare for massive storms. Still, even as an algorithm mindlessly implements its procedures—and even as it learns to see new patterns in the data—it reflects the minds of its creators, the motives of its trainers. Both Amazon and Netflix use algorithms to make recommendations about books and films.

Indeed, it earned a $429 million refund. According to Citizens for Tax Justice, Facebook bilked the treasury by taking a single deduction: It wrote off the stock options it gave to its executives. It’s hard to have sympathy with Walmart or Home Depot or the other big-box stores. They hardly pay the largest tax rates in the nation. Still, they cough up a reasonable sum. Over the last decade, Walmart, the supposed Beast of Bentonville, handed over about 30 percent of its income in taxes; Home Depot paid 38 percent. We can bemoan the fact that they don’t pay more, yet it seems reasonable to note that their prime competitor isn’t paying even half that rate.


pages: 343 words: 102,846

Trees on Mars: Our Obsession With the Future by Hal Niedzviecki

"World Economic Forum" Davos, Ada Lovelace, agricultural Revolution, Airbnb, Albert Einstein, Alvin Toffler, Amazon Robotics, anti-communist, big data - Walmart - Pop Tarts, big-box store, business intelligence, Charles Babbage, Colonization of Mars, computer age, crowdsourcing, data science, David Brooks, driverless car, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Evgeny Morozov, Flynn Effect, Ford Model T, Future Shock, Google Glasses, hive mind, Howard Zinn, if you build it, they will come, income inequality, independent contractor, Internet of things, invention of movable type, Jaron Lanier, Jeff Bezos, job automation, John von Neumann, knowledge economy, Kodak vs Instagram, life extension, Lyft, Marc Andreessen, Marc Benioff, Mark Zuckerberg, Marshall McLuhan, Neil Armstrong, One Laptop per Child (OLPC), Peter H. Diamandis: Planetary Resources, Peter Thiel, Pierre-Simon Laplace, Ponzi scheme, precariat, prediction markets, Ralph Nader, randomized controlled trial, Ray Kurzweil, ride hailing / ride sharing, rising living standards, Robert Solow, Ronald Reagan, Salesforce, self-driving car, shareholder value, sharing economy, Silicon Valley, Silicon Valley startup, Skype, Steve Jobs, TaskRabbit, tech worker, technological singularity, technological solutionism, technoutopianism, Ted Kaczynski, TED Talk, Thomas L Friedman, Tyler Cowen, Uber and Lyft, uber lyft, Virgin Galactic, warehouse robotics, working poor

From a data-crunching, owning-the-future perspective, converting the real into the virtual is becoming more and more desirable. Knowing what is going to happen is far more efficient than responding to events after the fact. And systematically controlling what is going to happen is most efficient of all. ° ° ° ° ° ° In the first phase of big data and information technology, Walmart deduced that before big storms, sales of Pop-Tarts skyrocket. Google analyzed millions of search terms and correlated searches for products and symptoms related to flu with actual flu outbreaks coming in the days and weeks ahead, ultimately providing faster and more accurate information about what areas were going to be ravaged by flu than the Centers for Disease Control and Prevention were able to achieve at the time.

We’ll watch it happen on the phone.”57 But maybe when we envision “progress” we should be picturing the cloistered aisles of the Walmart Supercenter where the real technological changes have affected billions of people—eliminating their jobs, supporting outsourcing to countries where it’s normal to have entire weeks when the smog is so thick you can’t see the sun, and all to give us shoppers slightly cheaper toasters and the Pop-Tarts to go with them. But, hey, things surely aren’t so very grim. For every takedown of Walmart—and really, what could be easier?—there’s a fabulous new business starting in the cloud that will eventually reverse the trend and make things good again. Everyone picks on Walmart. So let’s apply this same formula of information technology creating efficiencies that enhance productivity and profit but ultimately cost jobs and drop wages to a high-tech darling as far away from the Walmart aesthetic of buzzing fluorescent lights and glassy-eyed greeters as possible.

In 2012 the company pulled in a not-so-inconsequential $17-billion profit.52 What, you might ask, does a big-box chain store flogging everything from crayons to socks to flat-screen televisions have to do with IT? Well Walmart is, in fact, a pioneer in the kind of predict-the-future technology we talked about in our examination of the increasingly successful ways different actors are trying to own the future by knowing it sooner than everyone else. In fact, as the Big Data authors tells us, it was Walmart that revolutionized—or you might say disrupted retailing. In the 1990s, they developed a new tracking system called Retail Link. This massive IT system enabled and required suppliers to monitor what was selling where at Walmart stores. It then became their problem to keep up. “Wal-Mart used data to become, in effect, the world’s largest consignment shop.”53 Walmart’s world domination came by introducing cutting-edge, disruptive IT to its systems ranging from stock to shipping to employee monitoring.


pages: 660 words: 141,595

Data Science for Business: What You Need to Know About Data Mining and Data-Analytic Thinking by Foster Provost, Tom Fawcett

Albert Einstein, Amazon Mechanical Turk, Apollo 13, big data - Walmart - Pop Tarts, bioinformatics, business process, call centre, chief data officer, Claude Shannon: information theory, computer vision, conceptual framework, correlation does not imply causation, crowdsourcing, data acquisition, data science, David Brooks, en.wikipedia.org, Erik Brynjolfsson, Gini coefficient, Helicobacter pylori, independent contractor, information retrieval, intangible asset, iterative process, Johann Wolfgang von Goethe, Louis Pasteur, Menlo Park, Nate Silver, Netflix Prize, new economy, p-value, pattern recognition, placebo effect, price discrimination, recommendation engine, Ronald Coase, selection bias, Silicon Valley, Skype, SoftBank, speech recognition, Steve Jobs, supply-chain management, systems thinking, Teledyne, text mining, the long tail, The Signal and the Noise by Nate Silver, Thomas Bayes, transaction costs, WikiLeaks

., Answering Business Questions with These Techniques B bag of words approach, Bag of Words bags, Bag of Words base rates, Class Probability Estimation and Logistic “Regression”, Holdout Data and Fitting Graphs, Problems with Unbalanced Classes baseline classifiers, Advantages and Disadvantages of Naive Bayes baseline methods, of data science, Summary Basie, Count, Example: Jazz Musicians Bayes rate, Bias, Variance, and Ensemble Methods Bayes, Thomas, Bayes’ Rule Bayesian methods, Bayes’ Rule, Summary Bayes’ Rule, Bayes’ Rule–A Model of Evidence “Lift” beer and lottery example, Example: Beer and Lottery Tickets–Example: Beer and Lottery Tickets Beethoven, Ludwig van, Example: Evidence Lifts from Facebook “Likes” beginning cross-validation, From Holdout Evaluation to Cross-Validation behavior description, From Business Problems to Data Mining Tasks Being John Malkovich (film), Data Reduction, Latent Information, and Movie Recommendation Bellkors Pragmatic Chaos (Netflix Challenge team), Data Reduction, Latent Information, and Movie Recommendation benefit improvement, calculating, Costs and benefits benefits and underlying profit calculation, ROC Graphs and Curves data-driven decision-making, Data Science, Engineering, and Data-Driven Decision Making estimating, Costs and benefits in budgeting, Ranking Instead of Classifying nearest-neighbor methods, Computational efficiency bi-grams, N-gram Sequences bias errors, ensemble methods and, Bias, Variance, and Ensemble Methods–Bias, Variance, and Ensemble Methods Big Data data science and, Data Processing and “Big Data”–Data Processing and “Big Data” evolution of, From Big Data 1.0 to Big Data 2.0–From Big Data 1.0 to Big Data 2.0 on Amazon and Google, Thinking Data-Analytically, Redux big data technologies, Data Processing and “Big Data” state of, From Big Data 1.0 to Big Data 2.0 utilizing, Data Processing and “Big Data” Big Red proposal example, Example Data Mining Proposal–Flaws in the Big Red Proposal Bing, Why Text Is Important, Representation Black-Sholes model, Models, Induction, and Prediction blog postings, Why Text Is Important blog posts, Example: Targeting Online Consumers With Advertisements Borders (book retailer), Achieving Competitive Advantage with Data Science breast cancer example, Example: Logistic Regression versus Tree Induction–Example: Logistic Regression versus Tree Induction Brooks, David, What Data Can’t Do: Humans in the Loop, Revisited browser cookies, Example: Targeting Online Consumers With Advertisements Brubeck, Dave, Example: Jazz Musicians Bruichladdich single malt scotch, Understanding the Results of Clustering Brynjolfsson, Erik, Data Science, Engineering, and Data-Driven Decision Making, Data Processing and “Big Data” budget, Ranking Instead of Classifying budget constraints, Profit Curves building modeling labs, From Holdout Evaluation to Cross-Validation building models, Data Mining and Its Results, Business Understanding, From Holdout Evaluation to Cross-Validation Bunnahabhain single malt whiskey, Example: Whiskey Analytics, Hierarchical Clustering business news stories example, Example: Clustering Business News Stories–The news story clusters business problems changing definition of, to fit available data, Changing the Way We Think about Solutions to Business Problems–Changing the Way We Think about Solutions to Business Problems data exploration vs., Stepping Back: Solving a Business Problem Versus Data Exploration–Stepping Back: Solving a Business Problem Versus Data Exploration engineering problems vs., Other Data Science Tasks and Techniques evaluating in a proposal, Be Ready to Evaluate Proposals for Data Science Projects expected value framework, structuring with, The Expected Value Framework: Structuring a More Complicated Business Problem–The Expected Value Framework: Structuring a More Complicated Business Problem exploratory data mining vs., The Fundamental Concepts of Data Science unique context of, What Data Can’t Do: Humans in the Loop, Revisited using expected values to provide framework for, The Expected Value Framework: Decomposing the Business Problem and Recomposing the Solution Pieces–The Expected Value Framework: Decomposing the Business Problem and Recomposing the Solution Pieces business strategy, Data Science and Business Strategy–A Firm’s Data Science Maturity accepting creative ideas, Be Ready to Accept Creative Ideas from Any Source case studies, examining, Examine Data Science Case Studies competitive advantages, Achieving Competitive Advantage with Data Science–Achieving Competitive Advantage with Data Science, Sustaining Competitive Advantage with Data Science–Superior Data Science Management data scientists, evaluating, Superior Data Scientists–Superior Data Scientists evaluating proposals, Be Ready to Evaluate Proposals for Data Science Projects–Flaws in the Big Red Proposal historical advantages and, Formidable Historical Advantage intangible collateral assets and, Unique Intangible Collateral Assets intellectual property and, Unique Intellectual Property managing data scientists effectively, Superior Data Science Management–Superior Data Science Management maturity of the data science, A Firm’s Data Science Maturity–A Firm’s Data Science Maturity thinking data-analytically for, Thinking Data-Analytically, Redux–Thinking Data-Analytically, Redux C Caesars Entertainment, Data and Data Science Capability as a Strategic Asset call center example, Profiling: Finding Typical Behavior–Profiling: Finding Typical Behavior Capability Maturity Model, A Firm’s Data Science Maturity Capital One, Data and Data Science Capability as a Strategic Asset, From an Expected Value Decomposition to a Data Science Solution Case-Based Reasoning, How Many Neighbors and How Much Influence?

Data processing technologies are very important for many data-oriented business tasks that do not involve extracting knowledge or data-driven decision-making, such as efficient transaction processing, modern web system processing, and online advertising campaign management. “Big data” technologies (such as Hadoop, HBase, and MongoDB) have received considerable media attention recently. Big data essentially means datasets that are too large for traditional data processing systems, and therefore require new processing technologies. As with the traditional technologies, big data technologies are used for many tasks, including data engineering. Occasionally, big data technologies are actually used for implementing data mining techniques. However, much more often the well-known big data technologies are used for data processing in support of the data mining techniques and other data science activities, as represented in Figure 1-1.

A separate study, conducted by economist Prasanna Tambe of NYU’s Stern School, examined the extent to which big data technologies seem to help firms (Tambe, 2012). He finds that, after controlling for various possible confounding factors, using big data technologies is associated with significant additional productivity growth. Specifically, one standard deviation higher utilization of big data technologies is associated with 1%–3% higher productivity than the average firm; one standard deviation lower in terms of big data utilization is associated with 1%–3% lower productivity. This leads to potentially very large productivity differences between the firms at the extremes. From Big Data 1.0 to Big Data 2.0 One way to think about the state of big data technologies is to draw an analogy with the business adoption of Internet technologies.


pages: 252 words: 72,473

Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy by Cathy O'Neil

Affordable Care Act / Obamacare, Alan Greenspan, algorithmic bias, Bernie Madoff, big data - Walmart - Pop Tarts, call centre, Cambridge Analytica, carried interest, cloud computing, collateralized debt obligation, correlation does not imply causation, Credit Default Swap, credit default swaps / collateralized debt obligations, crowdsourcing, data science, disinformation, electronic logging device, Emanuel Derman, financial engineering, Financial Modelers Manifesto, Glass-Steagall Act, housing crisis, I will remember that I didn’t make the world, and it doesn’t satisfy my equations, Ida Tarbell, illegal immigration, Internet of things, late fees, low interest rates, machine readable, mass incarceration, medical bankruptcy, Moneyball by Michael Lewis explains big data, new economy, obamacare, Occupy movement, offshore financial centre, payday loans, peer-to-peer lending, Peter Thiel, Ponzi scheme, prediction markets, price discrimination, quantitative hedge fund, Ralph Nader, RAND corporation, real-name policy, recommendation engine, Rubik’s Cube, Salesforce, Sharpe ratio, statistical model, tech worker, Tim Cook: Apple, too big to fail, Unsafe at Any Speed, Upton Sinclair, Watson beat the top human players on Jeopardy!, working poor

American Express learned this the hard way: Ron Lieber, “American Express Kept a (Very) Watchful Eye on Charges,” New York Times, January 30, 2009, www.​nytimes.​com/​2009/​01/​31/​your-​money/credit-​and-​debit-​cards/​31money.​html. Douglas Merrill’s idea: Steve Lohr, “Big Data Underwriting for Payday Loans,” New York Times, January 19, 2015, http://​bits.​blogs.​nytimes.​com/​2015/​01/​19/​big-​data-​underwriting-​for-​payday-​loans/. On the company web page: Website ZestFinance.​com, accessed January 9, 2016, www.​zestfinance.​com/. A typical $500 loan: Lohr, “Big Data Underwriting.” ten thousand data points: Michael Carney, “Flush with $20M from Peter Thiel, ZestFinance Is Measuring Credit Risk Through Non-traditional Big Data,” Pando, July 31, 2013, https://​pando.​com/​2013/​07/​31/​flush-​with-​20m-​from-​peter-​thiel-​zestfinance-​is-​measuring-​credit-​risk-​through-​non-​traditional-​big-​data/.

The better solution would be to train the model over time, entering data every day on what I’d bought and cooked and noting the responses of each family member. I would also include parameters, or constraints. I might limit the fruits and vegetables to what’s in season and dole out a certain amount of Pop-Tarts, but only enough to forestall an open rebellion. I also would add a number of rules. This one likes meat, this one likes bread and pasta, this one drinks lots of milk and insists on spreading Nutella on everything in sight. If I made this work a major priority, over many months I might come up with a very good model.

Yet from the administrators’ perspective it provides an effective tool to ferret out hundreds of apparently underperforming teachers, even at the risk of misreading some of them. Here we see that models, despite their reputation for impartiality, reflect goals and ideology. When I removed the possibility of eating Pop-Tarts at every meal, I was imposing my ideology on the meals model. It’s something we do without a second thought. Our own values and desires influence our choices, from the data we choose to collect to the questions we ask. Models are opinions embedded in mathematics. Whether or not a model works is also a matter of opinion.


pages: 233 words: 67,596

Competing on Analytics: The New Science of Winning by Thomas H. Davenport, Jeanne G. Harris

always be closing, Apollo 13, big data - Walmart - Pop Tarts, business intelligence, business logic, business process, call centre, commoditize, data acquisition, digital map, en.wikipedia.org, fulfillment center, global supply chain, Great Leap Forward, high net worth, if you build it, they will come, intangible asset, inventory management, iterative process, Jeff Bezos, job satisfaction, knapsack problem, late fees, linear programming, Moneyball by Michael Lewis explains big data, Netflix Prize, new economy, performance metric, personalized medicine, quantitative hedge fund, quantitative trading / quantitative finance, recommendation engine, RFID, search inside the book, shareholder value, six sigma, statistical model, supply-chain management, text mining, The future is already here, the long tail, the scientific method, traveling salesman, yield management

So-called advanced planning and scheduling approaches also recognize material constraints in terms of current inventory and planned deliveries or allocations. As Wal-Mart’s data warehouse introduced additional information about customer behavior, applications using Wal-Mart’s massive database began to extend well beyond their supply chain. Wal-Mart now collects more data about more consumers than anyone in the private sector. Wal-Mart marketers mine this data to ensure that customers have the products they want, when they want them, and at the right price. For example, they’ve learned that before a hurricane, consumers stock up on food items that don’t require cooking or refrigeration. The top seller: Strawberry Pop Tarts. We expect that Wal-Mart asks Kellogg to rush shipments of them to stores just before a hurricane hits.

Connecting Customers and Suppliers The mother of all supply chain analytics competitors is Wal-Mart. The company collects massive amounts of sales and inventory data (583 terabytes as of April 2006) into a single integrated technology platform. Its managers routinely analyze manifold aspects of its supply chain, and store managers use analytical tools to optimize product assortment; they examine not only detailed sales data but also qualitative factors such as the opportunity to tailor assortments to local community needs.18 The most distinctive element of Wal-Mart’s supply chain data is its availability to suppliers. Wal-Mart buys products from more than 17,400 suppliers in eighty countries, and each one uses the company’s Retail Link system to track the movement of its products—in fact, the system’s use is mandatory.

At Netflix, the most strategic application may be predicting customer movie preferences, but the company also employs testing and detailed analysis in its supply chain and its advertising. Harrah’s started in loyalty and service but also does detailed analyses of its slot machine pricing and placement, the design of its Web site, and many other issues in its business. Wal-Mart, Progressive Insurance, and the hospital supply distributor Owens & Minor are all examples of firms that started with an internal analytical focus but have broadened it externally—to suppliers in the case of Wal-Mart and to customers for the other two firms. Analytical competitors need a primary focus for their analytical activity, but once an analytical, test-and-learn culture has been created, it’s impossible to stop it from spreading.


pages: 285 words: 58,517

The Network Imperative: How to Survive and Grow in the Age of Digital Business Models by Barry Libert, Megan Beck

active measures, Airbnb, Amazon Web Services, asset allocation, asset light, autonomous vehicles, big data - Walmart - Pop Tarts, business intelligence, call centre, Clayton Christensen, cloud computing, commoditize, crowdsourcing, data science, disintermediation, diversification, Douglas Engelbart, Douglas Engelbart, future of work, Google Glasses, Google X / Alphabet X, independent contractor, Infrastructure as a Service, intangible asset, Internet of things, invention of writing, inventory management, iterative process, Jeff Bezos, job satisfaction, John Zimmer (Lyft cofounder), Kevin Kelly, Kickstarter, Larry Ellison, late fees, Lyft, Mark Zuckerberg, Mary Meeker, Oculus Rift, pirate software, ride hailing / ride sharing, Salesforce, self-driving car, sharing economy, Silicon Valley, Silicon Valley startup, six sigma, software as a service, software patent, Steve Jobs, subscription business, systems thinking, TaskRabbit, Travis Kalanick, uber lyft, Wall-E, women in the workforce, Zipcar

It’s a wonderful opportunity to improve operations, decrease waste, and better serve our customers and the world. It’s time to figure out what opportunities are most exciting for you and your organization, and start the big data adventure. PRINCIPLE 9 BOARDS From Governance to Representation We need diversity of thought in the world to face the new challenges. —Tim Berners-Lee, inventor of the World Wide Web IT’S NOT EASY TO MANAGE A COMPANY YOU DON’T UNDERSTAND. In 2000, Kellogg Company, known for popular brands such as Froot Loops, Pop-Tarts, Frosted Flakes, and Pringles, purchased a small food company called Kashi. Kashi was a start-up that played in a similar part of the food market as Kellogg—with cereals, snack bars, crackers, and prepared foods—but Kashi had a mission focused on nutritious, plant-based foods and sustainable, ethical farming practices.

They often maintain a significant presence on all major social media platforms (Starbucks has more than a million followers on Instagram—pretty good for a coffee company), and they use big data analytics to learn about and better serve their customers. You might think that the whole world is moving online and to the digital network and that brick-and-mortar is going the way of the dinosaurs, but some traditional retailers have found that their physical assets can be used to complement their emerging technology and network business models. Macy’s and Walmart, along with several others, have become masters of omnichannel strategies. Their customers can shop at home, in stores, or even on their phones and receive the product through delivery or in-store pickup—whichever is most convenient.

Those that fail to meet worker needs will see their best and brightest head off in search of their next great role. PRINCIPLE 8 MEASUREMENT From Accounting to Big Data Not everything that counts can be counted, and not everything that can be counted counts. —William Bruce Cameron, sociologist YOU MIGHT NOT EXPECT a chain of barbecue joints with eleven people on its information technology staff to be an innovator in the use of big data. If so, you’re in for a surprise. Big data isn’t just for the Googles, Apples, Amazons, and Facebooks. Using big data doesn’t have to be a complicated, resource-heavy, yearlong endeavor. Big data, for our purposes, is nothing more than large sets of information that can be analyzed to understand useful patterns, often, but not always, related to human behavior.


pages: 268 words: 75,850

The Formula: How Algorithms Solve All Our Problems-And Create More by Luke Dormehl

3D printing, algorithmic bias, algorithmic trading, Alvin Toffler, Any sufficiently advanced technology is indistinguishable from magic, augmented reality, big data - Walmart - Pop Tarts, call centre, Cass Sunstein, classic study, Clayton Christensen, commoditize, computer age, death of newspapers, deferred acceptance, disruptive innovation, Edward Lorenz: Chaos theory, Erik Brynjolfsson, Evgeny Morozov, Filter Bubble, Flash crash, Florence Nightingale: pie chart, Ford Model T, Frank Levy and Richard Murnane: The New Division of Labor, fulfillment center, Google Earth, Google Glasses, High speed trading, Internet Archive, Isaac Newton, Jaron Lanier, Jeff Bezos, job automation, John Markoff, Kevin Kelly, Kodak vs Instagram, Lewis Mumford, lifelogging, machine readable, machine translation, Marshall McLuhan, means of production, Nate Silver, natural language processing, Netflix Prize, Panopticon Jeremy Bentham, Paradox of Choice, pattern recognition, price discrimination, recommendation engine, Richard Thaler, Rosa Parks, scientific management, self-driving car, sentiment analysis, Silicon Valley, Silicon Valley startup, Slavoj Žižek, social graph, speech recognition, stable marriage problem, Steve Jobs, Steven Levy, Steven Pinker, Stewart Brand, technological determinism, technological solutionism, TED Talk, the long tail, the scientific method, The Signal and the Noise by Nate Silver, upwardly mobile, Wall-E, Watson beat the top human players on Jeopardy!, Y Combinator

CHAPTER 3 Do Algorithms Dream of Electric Laws? Adecade ago, Walmart stumbled upon an oddball piece of information while using its data-mining algorithms to comb through the mountains of information generated by its 245 million weekly customers. What it discovered was that, alongside the expected emergency supplies of duct tape, beer and bottled water, no product saw more of an increase in demand during severe weather warnings than strawberry Pop-Tarts. To test this insight, when news broke about the impending Hurricane Frances in 2004, Walmart bosses ordered trucks stocked with the Kellogg’s snack to be delivered to all its stores in the hurricane’s path.

When these sold out just as quickly, Walmart bosses knew that they had gained a valuable glimpse into both consumer habits and the power of The Formula.1 Walmart executives weren’t alone in seeing the value of this discovery. At the time, psychologist Colleen McCue and Los Angeles police chief Charlie Beck were collaborating on a paper for the law-enforcement magazine The Police Chief. They too seized upon Walmart’s revelation as a way of reimagining police work in a form that would be more predictive and less reactive. Entitled “Predictive Policing: What Can We Learn from Walmart and Amazon about Fighting Crime in a Recession?

The Civilizing Process (New York: Urizen Books, 1978). 13 This wave metaphor was not, in itself, new: the German sociologist Norbert Elias had referred to “a wave of advancing integration over several centuries” in his book The Civilizing Process, as had other writers over the previous century. 14 Richtel, Matt. “How Big Data Is Playing Recruiter for Specialized Workers.” New York Times, April 27, 2013. nytimes.com/2013/04/28/technology/how-big-data-is-playing-recruiter-for-specialized-workers.html?_r=0. 15 Kwoh, Leslie. “Facebook Profiles Found to Predict Job Performance.” Wall Street Journal, February 21, 2012. online.wsj.com/news/articles/SB10001424052970204909104577235474086304212. 16 Bulmer, Michael.


pages: 230 words: 61,702

The Internet of Us: Knowing More and Understanding Less in the Age of Big Data by Michael P. Lynch

Affordable Care Act / Obamacare, Amazon Mechanical Turk, big data - Walmart - Pop Tarts, bitcoin, Cass Sunstein, Claude Shannon: information theory, cognitive load, crowdsourcing, data science, Edward Snowden, Firefox, Google Glasses, hive mind, income inequality, Internet of things, John von Neumann, meta-analysis, Nate Silver, new economy, Nick Bostrom, Panopticon Jeremy Bentham, patient HM, prediction markets, RFID, sharing economy, Steve Jobs, Steven Levy, the scientific method, The Wisdom of Crowds, Thomas Kuhn: the structure of scientific revolutions, Twitter Arab Spring, WikiLeaks

Similarly, Google Flu Trends doesn’t care why people are searching as they do; it just correlates the data. And Walmart doesn’t care why people buy more Pop-Tarts before a hurricane, nor do insurance companies care why certain credit scores correlate with certain medication adherences; they care only that they do. As Viktor Mayer-Schönberger and Kenneth Cukier put it, “predictions based on correlations lie at the heart of big data. Correlation analyses are now used so frequently that we sometimes fail to appreciate the inroads they have made. And the uses will only increase.” 4 Does the use of big data in this way however, really signal the end of theory, as Anderson alleged?

As a consequence of the increasing importance of data analytics, we might employ “big data” in a third sense—to refer to firms like Google or Amazon that utilize data analytics as an essential part of their business model, and government agencies like the NSA that use these techniques as an essential part of, well, their business model. In this third sense, Big Data is like Big Oil. Large oil conglomerates are powerful because they control how the world’s major energy resource is not only distributed but how it is extracted. The tech giants are similar. Energy is not information, but both are resources, and resources by which the world runs. And Big Data, like Big Oil, is big precisely because it can control access to data as well as the extraction of information and knowledge from that data.

Google has done more than perhaps any other single high-profile company or entity to usher in the brave new world of big data. As I noted in the first chapter, the term “big data” can refer to three different things. The first is the ever-expanding volume of data being collected by our digital devices. The second is analytical tools for extracting information from that data. And the third is the firms like Google that employ them. One of the lessons of previous chapters is that big data and our digital form of life, while sometimes making it easier to be a responsible and reasonable believer, often makes it harder as well—while at the same time setting up conditions that make reasonable belief more important than ever before.


pages: 223 words: 60,909

Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech by Sara Wachter-Boettcher

"Susan Fowler" uber, Abraham Maslow, Airbnb, airport security, algorithmic bias, AltaVista, big data - Walmart - Pop Tarts, Big Tech, Black Lives Matter, data science, deep learning, Donald Trump, fake news, false flag, Ferguson, Missouri, Firefox, Grace Hopper, Greyball, Hacker News, hockey-stick growth, independent contractor, job automation, Kickstarter, lifelogging, lolcat, Marc Benioff, Mark Zuckerberg, Max Levchin, Menlo Park, meritocracy, microaggression, move fast and break things, natural language processing, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, off-the-grid, pattern recognition, Peter Thiel, real-name policy, recommendation engine, ride hailing / ride sharing, Salesforce, self-driving car, Sheryl Sandberg, Silicon Valley, Silicon Valley startup, Snapchat, Steve Jobs, Tactical Technology Collective, TED Talk, Tim Cook: Apple, Travis Kalanick, upwardly mobile, Wayback Machine, women in the workforce, work culture , zero-sum game

According to Barron, tech products do this by employing “caretaker speech”—the linguistics term used to describe the way we talk to children. For example, when Seamless, a popular food delivery app, sends cutesy emails about the status of his order, he writes, “I picture a cool babysitter, Skylar, with his jean vest, telling me as he microwaves a pop-tart that ‘deliciousness is in the works,’ his tone just grazing the surface of mockery.” 16 But no matter how cool the babysitter—no matter how far past bedtime Skylar lets us stay up—at the end of the evening we’re still kids under someone else’s control. The result is an environment where we start to accept that the tech products we use, and the companies behind them, know best—and we’re just along for the ride.

Maggie Delano, “I Tried Tracking My Period and It Was Even Worse Than I Could Have Imagined,” Medium, February 23, 2015, https://medium.com/@maggied/i-tried-tracking-my-period-and-it-was-even-worse-than-i-could-have-imagined-bb46f869f45. 2. Glow, “About Glow,” Wayback Machine, September 21, 2013, https://web.archive.org/web/20130921143302/https://www.glowing.com/about. 3. Kia Kokalitcheva, “Glow Brings in $17M in New Funding, Puts Big Data to Work for Women’s Health,” VentureBeat, October 2, 2014, http://venturebeat.com/2014/10/02/glow-brings-in-17m-in-new-funding-as-puts-big-data-to-task-with-fertility-challenges. 4. Glow, “About Glow,” Wayback Machine, March 27, 2014, https://web.archive.org/web/20140327011628/https://glowing.com/about. 5. Erin Abler, Twitter post, January 31, 2017 (6:12 p.m.), https://twitter.com/erinabler/status/826614200114016256. 6.

Five seconds in, I’m already trying to ignore the app’s assumptions that pregnancy is why I want to track my period. The app also assumes that I’m sexually active with someone who can get me pregnant.1 The first screen in Glow’s onboarding process. What if none of these options apply to you? Delano’s experience with Glow might have made sense back in 2013, when Glow launched with the mission of using big data “to help get you pregnant.” 2 But in 2014, the founders realized that about half of Glow’s users were actually using the app to avoid getting pregnant.3 So, with $17 million in new funding in hand, the team set out to transform Glow from a narrow, fertility-focused experience to a product that could serve all women—including, it would seem, women like Delano.


pages: 421 words: 110,406

Platform Revolution: How Networked Markets Are Transforming the Economy--And How to Make Them Work for You by Sangeet Paul Choudary, Marshall W. van Alstyne, Geoffrey G. Parker

3D printing, Affordable Care Act / Obamacare, Airbnb, Alvin Roth, Amazon Mechanical Turk, Amazon Web Services, Andrei Shleifer, Apple's 1984 Super Bowl advert, autonomous vehicles, barriers to entry, Benchmark Capital, big data - Walmart - Pop Tarts, bitcoin, blockchain, business cycle, business logic, business process, buy low sell high, chief data officer, Chuck Templeton: OpenTable:, clean water, cloud computing, connected car, corporate governance, crowdsourcing, data acquisition, data is the new oil, data science, digital map, discounted cash flows, disintermediation, driverless car, Edward Glaeser, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, financial innovation, Free Software Foundation, gigafactory, growth hacking, Haber-Bosch Process, High speed trading, independent contractor, information asymmetry, Internet of things, inventory management, invisible hand, Jean Tirole, Jeff Bezos, jimmy wales, John Markoff, Kevin Roose, Khan Academy, Kickstarter, Lean Startup, Lyft, Marc Andreessen, market design, Max Levchin, Metcalfe’s law, multi-sided market, Network effects, new economy, PalmPilot, payday loans, peer-to-peer lending, Peter Thiel, pets.com, pre–internet, price mechanism, recommendation engine, RFID, Richard Stallman, ride hailing / ride sharing, Robert Metcalfe, Ronald Coase, Salesforce, Satoshi Nakamoto, search costs, self-driving car, shareholder value, sharing economy, side project, Silicon Valley, Skype, smart contracts, smart grid, Snapchat, social bookmarking, social contagion, software is eating the world, Steve Jobs, TaskRabbit, The Chicago School, the long tail, the payments system, Tim Cook: Apple, transaction costs, Travis Kalanick, two-sided market, Uber and Lyft, Uber for X, uber lyft, vertical integration, winner-take-all economy, zero-sum game, Zipcar

It referred to an unusual housing option for professionals who planned to attend the upcoming joint convention of two industrial design organizations, the International Congress of Societies of Industrial Design (ICSID) and the Industrial Designers Society of America (IDSA): If you’re heading out to the ICSID/IDSA World Congress/Connecting ’07 event in San Francisco next week and have yet to make accommodations, well, consider networking in your jam-jams. That’s right. For “an affordable alternative to hotels in the city,” imagine yourself in a fellow design industry person’s home, fresh awake from a snooze on the ol’ air mattress, chatting about the day’s upcoming events over Pop Tarts and OJ. The hosts for this “networking in your jam-jams” opportunity were Brian Chesky and Joe Gebbia, budding designers who’d moved to San Francisco only to find they couldn’t afford the rent on the loft they shared. Strapped for cash, they impulsively decided to make air mattresses and their own services as part-time tour guides available to convention attendees.

He lists eight markets with the potential to generate new multi-billion-dollar industries based on smart connections among industrial devices: • Security: using platform-based networks to protect industrial assets from attacks • Network: designing, building, and servicing the networks that will link and control industrial tools • Connected services: developing software and systems to manage the new networks • Product as a service: transitioning industrial companies from selling machines and tools to selling services facilitated by platform connections • Payments: implementing new ways to create and capture value from industrial equipment • Retrofits: equipping the $6.8 trillion worth of existing industrial machinery in the U.S. to participate in the new industrial Internet • Translation: teaching a wide array of devices and software systems to share data and communicate with one another • Vertical applications: finding ways to connect industrial tools at various places in the value chain to solve specific problems In total, Mount concludes (drawing on data from a World Economic Forum report) that the Industrial Awakening will generate $14.2 trillion of global output by 2030.13 Economist Jeremy Rifkin has deftly summarized this development, as well as some of its broader implications: There are now 11 billion sensors connecting devices to the internet of things. By 2030, 100 trillion sensors will be [in place] … continually sending big data to the communications, energy and logistics internets. Anyone will be able to access the internet of things and use big data and analytics to develop predictive algorithms that can speed efficiency, dramatically increase productivity and lower the marginal cost of producing and distributing physical things, including energy, products and services, to near zero, just as we now do with information goods.14 We may not be on the verge of seeing the majority of physical goods priced at or even near to zero—not yet.

The power of APIs to attract extension developers and the value they can create is enormous. Compare the financial results experienced by two major retailers: traditional giant Walmart and online platform Amazon. Amazon has some thirty-three open APIs as well as over 300 API “mashups” (i.e., combination tools that span two or more APIs), enabling e-commerce, cloud computing, messaging, search engine optimization, and payments. By contrast, Walmart has just one API, an e-commerce tool.14 Partly as a result of this difference, Amazon’s stock market capitalization exceeded that of Walmart for the first time in June 2015, reflecting Wall Street’s bullish view of Amazon’s future growth prospects.15 Other platform businesses have reaped similar benefits from their APIs.


pages: 569 words: 156,139

Amazon Unbound: Jeff Bezos and the Invention of a Global Empire by Brad Stone

activist fund / activist shareholder / activist investor, air freight, Airbnb, Amazon Picking Challenge, Amazon Robotics, Amazon Web Services, autonomous vehicles, Bernie Sanders, big data - Walmart - Pop Tarts, Big Tech, Black Lives Matter, business climate, call centre, carbon footprint, Clayton Christensen, cloud computing, Colonization of Mars, commoditize, company town, computer vision, contact tracing, coronavirus, corporate governance, COVID-19, crowdsourcing, data science, deep learning, disinformation, disintermediation, Donald Trump, Downton Abbey, Elon Musk, fake news, fulfillment center, future of work, gentrification, George Floyd, gigafactory, global pandemic, Greta Thunberg, income inequality, independent contractor, invisible hand, Jeff Bezos, John Markoff, Kiva Systems, Larry Ellison, lockdown, Mahatma Gandhi, Mark Zuckerberg, Masayoshi Son, mass immigration, minimum viable product, move fast and break things, Neal Stephenson, NSO Group, Paris climate accords, Peter Thiel, Ponzi scheme, Potemkin village, private spaceflight, quantitative hedge fund, remote working, rent stabilization, RFID, Robert Bork, Ronald Reagan, search inside the book, Sheryl Sandberg, Silicon Valley, Silicon Valley startup, Snapchat, social distancing, SoftBank, SpaceX Starlink, speech recognition, Steve Ballmer, Steve Jobs, Steven Levy, tech billionaire, tech bro, techlash, TED Talk, Tim Cook: Apple, Tony Hsieh, too big to fail, Tragedy of the Commons, two-pizza team, Uber for X, union organizing, warehouse robotics, WeWork

Armchair shoppers ignored Amazon’s proposed hashtag, #HappyPrimeDay and took to Twitter to criticize the company for sold-out items, discounts on trivial products like dishwashing detergent, and an abundance of underwhelming deals. “I keep going back to the Amazon #PrimeDay sale like a girlfriend who is convinced it’s going to get better,” said one typical Twitter post. “The best deal I’ve seen so far is 15% off a box of pop tarts,” read another. Rupp and Wulff and their team had turned a Seattle conference room in Amazon’s Arizona building into their designated war room. They spent two days and nights monitoring the traffic and promoting whatever deals they could. Wulff recalled going home to sleep for a few hours and then returning.

proposed a new set of rules… brokering more than 25 percent of the site’s total sales: Jon Russell, “New E-commerce Restrictions in India Just Ruined Christmas for Amazon and Walmart,” TechCrunch, December 27, 2018, https://techcrunch.com/2018/12/27/amazon-walmart-india-e-commerce-restrictions/ (January 20, 2021). Binny Bansal was promptly ousted: Saritha Rai and Matthew Boyle, “How Walmart Decided to Oust an Icon of India’s Tech Industry,” Bloomberg, November 15, 2018, https://www.bloomberg.com/news/articles/2018-11-15/how-walmart-decided-to-oust-an-icon-of-india-s-tech-industry?sref=dJuchiL5 (January 20, 2021); Saritha Rai, “Flipkart Billionaire Breaks His Silence After Walmart Ouster,” Bloomberg, February 4, 2019, https://www.bloomberg.com/news/articles/2019-02-05/flipkart-billionaire-breaks-his-silence-after-walmart-ouster?

After a process that stretched on for six months and had assaulted the calendars of everyone involved with never-ending conference calls, the fractious Flipkart board finally agreed to sell a stake to Walmart. At first the deal talks had called for the retailer to only take a minority position, but by then most of Flipkart’s weary investors wanted to sell their shares and cash out. Even at this late moment, drama dogged Flipkart; Sachin Bansal almost scuttled the deal by insisting Walmart guarantee him future control in the management of the company. Exasperated, the Flipkart board finally insisted that he leave the company for good. In May 2018, the companies announced that Walmart would pay $16 billion for a 77 percent stake in Flipkart. Walmart CEO Doug McMillon visited India after the deal was announced and told Flipkart employees, “It is our intention to just empower you and let you run.


pages: 450 words: 113,173

The Age of Entitlement: America Since the Sixties by Christopher Caldwell

1960s counterculture, affirmative action, Affordable Care Act / Obamacare, Alan Greenspan, Alvin Toffler, anti-communist, behavioural economics, Bernie Sanders, big data - Walmart - Pop Tarts, Black Lives Matter, blue-collar work, Cass Sunstein, choice architecture, classic study, computer age, crack epidemic, critical race theory, crony capitalism, Daniel Kahneman / Amos Tversky, David Attenborough, desegregation, disintermediation, disruptive innovation, Edward Snowden, Erik Brynjolfsson, Ferguson, Missouri, financial deregulation, financial innovation, Firefox, full employment, Future Shock, George Gilder, global value chain, Home mortgage interest deduction, illegal immigration, immigration reform, informal economy, James Bridle, Jeff Bezos, John Markoff, junk bonds, Kevin Kelly, Lewis Mumford, libertarian paternalism, Mark Zuckerberg, Martin Wolf, mass immigration, mass incarceration, messenger bag, mortgage tax deduction, Nate Silver, new economy, Norman Mailer, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, open immigration, opioid epidemic / opioid crisis, post-industrial society, pre–internet, profit motive, public intellectual, reserve currency, Richard Thaler, Robert Bork, Robert Gordon, Robert Metcalfe, Ronald Reagan, Rosa Parks, Silicon Valley, Skype, South China Sea, Steve Jobs, tech billionaire, Thomas Kuhn: the structure of scientific revolutions, Thomas L Friedman, too big to fail, transatlantic slave trade, transcontinental railway, W. E. B. Du Bois, War on Poverty, Whole Earth Catalog, zero-sum game

Now that people interacted with almost everything through a computer, their tiniest velleities could be tabulated. As Google, Facebook, and Amazon served customers, they were also harvesting and correlating information on them. Around 2010, this process came to be called Big Data. It was, at first, an entertaining curiosity. Walmart discovered through its algorithms that, when storms are coming, people buy more strawberry Pop-Tarts. Target could identify pregnant women from their tendency to buy unscented lotion in the third month of a pregnancy and then mineral supplements a few weeks thereafter. Marketers and advertisers now felt they held in their hands the same kind of esoteric, all-explaining truth that Alfred Kinsey’s Sexual Behavior in the Human Male had provided enthusiasts of sex in the late 1940s—a truth that is indifferent to what you say your morals or opinions are.

“Either we must allow”: Bertrand Russell, The Impact of Science on Society (London: George Allen & Unwin, 1952), 95. In hundreds of cities: Jeffrey Weiss, “Lunch Rush,” Dallas Morning News, October 12, 1993. Walmart discovered: Viktor Mayer-Schönberger and Kenneth Cukier, Big Data: A Revolution That Will Transform How We Live, Work, and Think (New York: Houghton Mifflin Harcourt, 2013), 54. Target could identify: Ibid., 58. “Society will need”: Mayer-Schönberger and Cukier, Big Data, 7. Google claimed to predict: Ibid., 15. SWIFT: Ibid. When pundits sought new ways: Fareed Zakaria, “Sanctions Russia Will Respect,” Washington Post, February 13, 2015.

Less well understood was that the internet approach to data, and to reality, undermined all types of thinking aimed at understanding systems from the outside—not just religion but also science, political ideology, and deductive reasoning. Big Data worked by correlation, not by logic. As the Oxford technology expert Viktor Mayer-Schönberger put it, “Society will need to shed some of its obsession for causality in exchange for simple correlations: not knowing why but only what.” Big Data was a reassertion by powerful corporations of a right that had been stripped from other Americans: the right to stereotype. If you’re the sort of person who does x, you’re the sort of person who’ll like y.


pages: 222 words: 70,132

Move Fast and Break Things: How Facebook, Google, and Amazon Cornered Culture and Undermined Democracy by Jonathan Taplin

"Friedman doctrine" OR "shareholder theory", "there is no alternative" (TINA), 1960s counterculture, affirmative action, Affordable Care Act / Obamacare, Airbnb, AlphaGo, Amazon Mechanical Turk, American Legislative Exchange Council, AOL-Time Warner, Apple's 1984 Super Bowl advert, back-to-the-land, barriers to entry, basic income, battle of ideas, big data - Walmart - Pop Tarts, Big Tech, bitcoin, Brewster Kahle, Buckminster Fuller, Burning Man, Clayton Christensen, Cody Wilson, commoditize, content marketing, creative destruction, crony capitalism, crowdsourcing, data is the new oil, data science, David Brooks, David Graeber, decentralized internet, don't be evil, Donald Trump, Douglas Engelbart, Douglas Engelbart, Dynabook, Edward Snowden, Elon Musk, equal pay for equal work, Erik Brynjolfsson, Fairchild Semiconductor, fake news, future of journalism, future of work, George Akerlof, George Gilder, Golden age of television, Google bus, Hacker Ethic, Herbert Marcuse, Howard Rheingold, income inequality, informal economy, information asymmetry, information retrieval, Internet Archive, Internet of things, invisible hand, Jacob Silverman, Jaron Lanier, Jeff Bezos, job automation, John Markoff, John Maynard Keynes: technological unemployment, John Perry Barlow, John von Neumann, Joseph Schumpeter, Kevin Kelly, Kickstarter, labor-force participation, Larry Ellison, life extension, Marc Andreessen, Mark Zuckerberg, Max Levchin, Menlo Park, Metcalfe’s law, military-industrial complex, Mother of all demos, move fast and break things, natural language processing, Network effects, new economy, Norbert Wiener, offshore financial centre, packet switching, PalmPilot, Paul Graham, paypal mafia, Peter Thiel, plutocrats, pre–internet, Ray Kurzweil, reality distortion field, recommendation engine, rent-seeking, revision control, Robert Bork, Robert Gordon, Robert Metcalfe, Ronald Reagan, Ross Ulbricht, Sam Altman, Sand Hill Road, secular stagnation, self-driving car, sharing economy, Silicon Valley, Silicon Valley ideology, Skinner box, smart grid, Snapchat, Social Justice Warrior, software is eating the world, Steve Bannon, Steve Jobs, Stewart Brand, tech billionaire, techno-determinism, technoutopianism, TED Talk, The Chicago School, the long tail, The Market for Lemons, The Rise and Fall of American Growth, Tim Cook: Apple, trade route, Tragedy of the Commons, transfer pricing, Travis Kalanick, trickle-down economics, Tyler Cowen, Tyler Cowen: Great Stagnation, universal basic income, unpaid internship, vertical integration, We are as Gods, We wanted flying cars, instead we got 140 characters, web application, Whole Earth Catalog, winner-take-all economy, women in the workforce, Y Combinator, you are the product

It’s also equally ludicrous to believe that—at the very least—this mass distraction and manipulation is not convenient for the people who are in charge. People are starving. They may not know it because they’re being fed mass-produced garbage. The packaging is colorful and loud, but it’s produced in the same factories that make Pop-Tarts and iPads by people sitting around thinking, “What can we do to get people to buy more of these?” And they’re very good at their jobs. But that’s what it is you’re getting, because that’s what they’re making. They’re selling you something. And the world is built on this now. Politics and government are built on this; corporations are built on this.

The very rich, when they get to be 130 years old or more, would be so fearful of ordinary causes of death—a car accident, a plane crash, a terrorist bomb—that, having spent millions of dollars on immortality, they might be afraid to leave their mansions for fear of losing money on their investment. I would say it takes no big leap to guess that both Peter Thiel and Larry Page truly believe that technology can deliver happiness. In a new book, The Internet of Us: Knowing More and Understanding Less in the Age of Big Data, Michael Patrick Lynch starts with a thought experiment: “Imagine a society where smartphones are miniaturized and hooked directly into a person’s brain.” Google’s Larry Page is already working on this. Then Lynch takes us several generations into the future, where we have stopped learning by observation and reason and have become totally dependent on the Google Now chip in our brains.

Yuval Levin, Fractured Republic: Renewing America’s Social Contract in the Age of Individualism (New York: Basic Books, 2016). Toni Morrison, Ta-Nehisi Coates, and Sonia Sanchez, “Art is Dangerous,” VOX, June 17, 2016, www.vox.com/2016/6/17/11955704/ta-nehisi-coates-toni-morrison-sonia-sanchez-in-conversation. Yuval Noah Harari, “Big Data, Google, and the End of Free Will,” Financial Times, August 26, 2016, www.ft.com/content/50bb4830-6a4c-11e6-ae5b-a7cc5dd5a28c. Thank you for buying this ebook, published by Hachette Digital. To receive special offers, bonus content, and news about our latest ebooks and apps, sign up for our newsletters.


pages: 282 words: 85,658

Ask Your Developer: How to Harness the Power of Software Developers and Win in the 21st Century by Jeff Lawson

Airbnb, AltaVista, Amazon Web Services, barriers to entry, big data - Walmart - Pop Tarts, Big Tech, big-box store, bitcoin, business process, call centre, Chuck Templeton: OpenTable:, cloud computing, coronavirus, COVID-19, create, read, update, delete, cryptocurrency, data science, David Heinemeier Hansson, deep learning, DevOps, Elon Musk, financial independence, global pandemic, global supply chain, Hacker News, Internet of things, Jeff Bezos, Kanban, Lean Startup, loose coupling, Lyft, Marc Andreessen, Marc Benioff, Mark Zuckerberg, microservices, minimum viable product, Mitch Kapor, move fast and break things, Paul Graham, peer-to-peer, ride hailing / ride sharing, risk tolerance, Ruby on Rails, Salesforce, side project, Silicon Valley, Silicon Valley startup, Skype, social distancing, software as a service, software is eating the world, sorting algorithm, Startup school, Steve Ballmer, Steve Jobs, Telecommunications Act of 1996, Toyota Production System, transaction costs, transfer pricing, two-pizza team, Uber and Lyft, uber lyft, ubercab, web application, Y Combinator

Those small independent shops are great when the sports are niche, but as these sports were gaining mainstream traction, Matt hypothesized that shoppers would want a big-box retail experience—predictable store hours, good return policies, wide product selection, like what REI provides for outdoor sports. We got to work wondering what such a store would look like for extreme sports. If REI was for the granola-eating sports, we would do it for the Pop-Tart–eating sports. Of all the thousand ideas we brainstormed, fueled by big technology trends happening on the Internet and mobile, the only idea we could all agree on was this brick-and-mortar retail idea. Odd, although it was also a sign of the times. In the wake of the dot-com crash, the idea of a business where you buy goods for X and sell them for more than X sounded pretty nice in contrast to the crazy business models of the Internet, which all seemed to be imploding in 2000–2001.

Unfortunately, much of GE’s revenue and profit came from parts and repairs, so the digital initiative was met with initial skepticism by other GE business leaders. However, Jeff knew that becoming a software business was the only way to protect and grow services. He had the right instincts, and thus he launched GE Digital to bolster these service businesses with big data, Internet of Things (IoT), and machine learning talent—and beat the variety of digital native companies to the punch. Launching the initiative, he committed hundreds of millions of dollars behind a big idea: a platform for industrial IoT applications. The commitment was impressive and, as he noted, necessary to let the company know he was serious.

When we got to Q&A time, someone in the crowd of about five thousand people got up and asked a question about retailing—I don’t even remember exactly what it was. But Jeff came back with an answer most of us didn’t expect. “Amazon,” he said, “is not a retailer. We’re a software company.” That seemed like a strange thing to say, especially given so many Amazon employees at the time came from either Walmart, an actual retailer, or from Microsoft, an actual software company. Both sets of employees were equally surprised. But Jeff insisted it was true. Most software companies at the time sold software pressed on CD-ROMs, packaged in boxes, and even still bought on a shelf at CompUSA. Jeff’s point was that Amazon was just as much a software company as Microsoft or Oracle or Adobe.


pages: 742 words: 137,937

The Future of the Professions: How Technology Will Transform the Work of Human Experts by Richard Susskind, Daniel Susskind

23andMe, 3D printing, Abraham Maslow, additive manufacturing, AI winter, Albert Einstein, Amazon Mechanical Turk, Amazon Robotics, Amazon Web Services, Andrew Keen, Atul Gawande, Automated Insights, autonomous vehicles, Big bang: deregulation of the City of London, big data - Walmart - Pop Tarts, Bill Joy: nanobots, Blue Ocean Strategy, business process, business process outsourcing, Cass Sunstein, Checklist Manifesto, Clapham omnibus, Clayton Christensen, clean water, cloud computing, commoditize, computer age, Computer Numeric Control, computer vision, Computing Machinery and Intelligence, conceptual framework, corporate governance, creative destruction, crowdsourcing, Daniel Kahneman / Amos Tversky, data science, death of newspapers, disintermediation, Douglas Hofstadter, driverless car, en.wikipedia.org, Erik Brynjolfsson, Evgeny Morozov, Filter Bubble, full employment, future of work, Garrett Hardin, Google Glasses, Google X / Alphabet X, Hacker Ethic, industrial robot, informal economy, information retrieval, interchangeable parts, Internet of things, Isaac Newton, James Hargreaves, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, Joseph Schumpeter, Khan Academy, knowledge economy, Large Hadron Collider, lifelogging, lump of labour, machine translation, Marshall McLuhan, Metcalfe’s law, Narrative Science, natural language processing, Network effects, Nick Bostrom, optical character recognition, Paul Samuelson, personalized medicine, planned obsolescence, pre–internet, Ray Kurzweil, Richard Feynman, Second Machine Age, self-driving car, semantic web, Shoshana Zuboff, Skype, social web, speech recognition, spinning jenny, strong AI, supply-chain management, Susan Wojcicki, tacit knowledge, TED Talk, telepresence, The Future of Employment, the market place, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Tragedy of the Commons, transaction costs, Turing test, Two Sigma, warehouse robotics, Watson beat the top human players on Jeopardy!, WikiLeaks, world market for maybe five computers, Yochai Benkler, young professional

In relation to the latter, on one view, the ‘proportion of the world’s data that comes from such sensors is expected to increase from 11 percent in 2005 to 42 percent in 2020’.40 The upshot of all of this is that great volumes of data are now at large, and the broad aim of data scientists is to develop methods for collecting, analysing, and exploiting these data. Case studies of success in Big Data abound. One (not entirely uncontroversial) illustration is Google Flu Trends, a system that can identify outbreaks of flu earlier than was possible in the past, by identifying geographical clustering of users whose search requests are made up of similar symptoms. Another is provided by Walmart, which analysed the buying habits of its customers prior to hurricanes and found not just that flashlights were in greater demand but so too were Pop-Tarts; and this insight enabled them to stock up accordingly when the next storm came round.

Liddy, that the future of audit was ‘the capacity to examine 100 percent of a client’s transactions’.275 This ambition of ‘100 per-cent testing’—using all available data, and not just a representative sample—is a particular case of a more general ambition very much in vogue in statistics, as discussed by Viktor Mayer-Schönberger and Kenneth Cukier in their book Big Data. One of the general features of Big Data, the authors argue, is precisely this move from taking small samples of data to using all the data instead (as they put it, ‘from some to all’).276 The next step on from 100 per cent testing is a phenomenon referred to by auditors at the vanguard as ‘continuous auditing’. Combining ongoing review of transactions and traditional financial accounts with platforms that can draw on more varied data sources, the aim is real-time insight into a company’s financial health. Again, this is a reflection of a general ambition in Big Data—to use data derived from many different sources, in different formats, and with less formal structure (not, for example, data that are carefully presented in a spreadsheet).

Coffee, Gatekeepers: The Role of the Professions and Corporate Governance (2006), 15. 273 ‘The Dozy Watchdogs’, Economist, 13 Dec. 2014. 274 James Shanteau, ‘Cognitive Heuristics and Biases in Behavioral Auditing: Review, Comments, and Observations’, Accounting, Organizations, and Society, 14: 1 (1989), 165–77. 275 James P. Liddy, ‘The Future of Audit’, Forbes, 4 Aug. 2014 <http://www.forbes.com> (accessed 8 March 2015). 276 Viktor Mayer-Schönberger and Kenneth Cukier, Big Data: A Revolution That Will Transform How we Live, Work, and Think (2013), 26. 277 Mayer-Schönberger and Cukier, Big Data, 32. 278 Mayer-Schönberger and Cukier, Big Data, and James Surowiekcki, ‘A Billion Prices Now’, New Yorker, 30 May 2011. 279 Michael Andersen, ‘Four crowdsourcing lessons from the Guardian’s (spectacular) expenses-scandal experiment’, NiemanLab, 23 June 2009 <http://www.niemanlab.org> (accessed 8 March 2015). 280 <https://www.xbrl.org>. 281 For instance, ‘the long shadow of the gentleman architect still hangs over the profession’, in Dickon Robinson et al., ‘The Future for Architects?’


pages: 606 words: 157,120

To Save Everything, Click Here: The Folly of Technological Solutionism by Evgeny Morozov

"World Economic Forum" Davos, 3D printing, algorithmic bias, algorithmic trading, Amazon Mechanical Turk, An Inconvenient Truth, Andrew Keen, augmented reality, Automated Insights, behavioural economics, Berlin Wall, big data - Walmart - Pop Tarts, Buckminster Fuller, call centre, carbon footprint, Cass Sunstein, choice architecture, citizen journalism, classic study, cloud computing, cognitive bias, creative destruction, crowdsourcing, data acquisition, Dava Sobel, digital divide, disintermediation, Donald Shoup, driverless car, East Village, en.wikipedia.org, Evgeny Morozov, Fall of the Berlin Wall, Filter Bubble, Firefox, Francis Fukuyama: the end of history, frictionless, future of journalism, game design, gamification, Gary Taubes, Google Glasses, Ian Bogost, illegal immigration, income inequality, invention of the printing press, Jane Jacobs, Jean Tirole, Jeff Bezos, jimmy wales, Julian Assange, Kevin Kelly, Kickstarter, license plate recognition, lifelogging, lolcat, lone genius, Louis Pasteur, machine readable, Mark Zuckerberg, market fundamentalism, Marshall McLuhan, moral panic, Narrative Science, Nelson Mandela, Nicholas Carr, packet switching, PageRank, Parag Khanna, Paul Graham, peer-to-peer, Peter Singer: altruism, Peter Thiel, pets.com, placebo effect, pre–internet, public intellectual, Ray Kurzweil, recommendation engine, Richard Thaler, Ronald Coase, Rosa Parks, self-driving car, Sheryl Sandberg, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, Skype, Slavoj Žižek, smart meter, social graph, social web, stakhanovite, Steve Jobs, Steven Levy, Stuxnet, surveillance capitalism, systems thinking, technoutopianism, TED Talk, the built environment, The Chicago School, The Death and Life of Great American Cities, the medium is the message, The Nature of the Firm, the scientific method, The Wisdom of Crowds, Thomas Kuhn: the structure of scientific revolutions, Thomas L Friedman, transaction costs, Twitter Arab Spring, urban decay, urban planning, urban sprawl, Vannevar Bush, warehouse robotics, WikiLeaks, work culture , Yochai Benkler

., 1972), 6. 182 ShotSpotter: Ethan Watters, “Shot Spotter,” Wired, March 2007, http://www.wired.com/wired/archive/15.04/shotspotter.html. 183 PredPol: on PredPol and predictive policing in general, see “Sci-fi Policing: Predicting Crime before It Occurs,” Associated Press, July 1, 2012; Joel Rubin, “Stopping Crime before It Starts,” Los Angeles Times, August 21, 2010, http://articles.latimes.com/2010/aug/21/local/la-me-predictcrime-20100427–1. 183 Consider the New York Police Department’s latest innovation: “NYPD, Microsoft Push Big Data Policing into Spotlight,” Informationweek, August 20, 2012, http://www.informationweek.com/security/privacy/nypd-microsoft-push-big-data-policing-in/240005838 . 183 “understand the unique groups in their customer base”: C. Beck and C. McCue, “Predictive Policing: What Can We Learn from Wal-Mart and Amazon About Fighting Crime in a Recession?,” Police Chief 76, no. 11 (2009), http://www.policechiefmagazine.org/magazine/index.cfm?fuseaction=print_display&article_id=1942&issue_id=112009. 185 “Predictive algorithms are not magic boxes”: Andrew Guthrie Ferguson, “Predictive Policing: The Future of Reasonable Suspicion,” Emory Law Journal, May 2, 2012, http://ssrn.com/abstract=2050001. 185 “the environmental vulnerability that encouraged”: ibid. 185 financial authorities in Hong Kong and Australia: for more on this, see Jeremy Grant, “Australia Clamps Down on ‘Algo’ Trading,” Financial Times, August 13, 2012, http://www.ft.com/intl/cms/s/0/ad11c4bc-e4f2–11e1–8e29–00144feab49a.html, and “Hong Kong Considers Annual Inspections of Algorithms,” Automated Trader, July 26, 2012, http://www.automatedtrader.net/headlines/129847/hong-kong-considers-annual-inspections-of-algorithms. 186 Facebook began using PhotoDNA: Riva Richmond, “Facebook’s New Way to Combat Child Pornography,” New York Times Gadgetwise, May 19, 2011, http://gadgetwise.blogs.nytimes.com/2011/05/19/facebook-to-combat-child-porn-using-microsofts-technology . 186 “We’ve never wanted to set up an environment”: Joseph Menn, “Social Networks Scan for Sexual Predators, with Uneven Results,” Reuters, July 12, 2012, http://www.reuters.com/article/2012/07/12/us-usa-internet-predators-idUSBRE86B05G20120712. 187 A headline that appeared in the Wall Street Journal: “Can Data Mining Stop the Killing?

That everything can be gamified does not mean that everything ought to be. Wired reports on how game theorist Jesse Schell, attempting to show that gamification has its limits, gave a conference talk describing “a world in which a person’s every action—brushing their teeth, showing up to work on time, tattooing an advertisement for Pop-Tarts onto their forearm—earned points.” Alas, Schell’s attempt to encourage more critical thinking by gamification apologists backfired. As Schell told Wired, “I’ve had dozens of people come to me saying, ‘Your talk was so influential to me that I started a company. . . All I can think is, oh God, don’t blame me for that.”

Of course, algorithms can be configured differently—and some independent labels might choose to release music that is bound to remain unpopular—but it’s hard to expect the major labels to pass up the opportunity to make more, and safer, money by deploying the algorithms. Surviving Big Data As we transition into the meme-saturated world of “algorithmic audiences,” it becomes very hard to remember the time when serious news media didn’t obsess over whether something was a “total bummer” and reported news that was important and worth caring about, regardless of how it affected the emotional well-being of the audience. To celebrate “the age of big data” and acquiesce to the ongoing invasion of journalism by various statistical measures and indicators is to give in to solutionism and endorse a very different, complacent kind of journalism.