42 results back to index

pages: 666 words: 181,495

In the Plex: How Google Thinks, Works, and Shapes Our Lives by Steven Levy


23andMe, AltaVista, Anne Wojcicki, Apple's 1984 Super Bowl advert, autonomous vehicles, book scanning, Brewster Kahle, Burning Man, business process, clean water, cloud computing, crowdsourcing, Dean Kamen, discounted cash flows, don't be evil, Douglas Engelbart, El Camino Real, fault tolerance, Firefox, Gerard Salton, Google bus, Google Chrome, Google Earth, Googley, HyperCard, hypertext link, IBM and the Holocaust, informal economy, information retrieval, Internet Archive, Jeff Bezos, Kevin Kelly, Mark Zuckerberg, Menlo Park, optical character recognition, PageRank, Paul Buchheit, Potemkin village, prediction markets, recommendation engine, risk tolerance, Sand Hill Road, Saturday Night Live, search inside the book, second-price auction, Silicon Valley, skunkworks, Skype, slashdot, social graph, social software, social web, spectrum auction, speech recognition, statistical model, Steve Ballmer, Steve Jobs, Steven Levy, Ted Nelson, telemarketer, trade route, traveling salesman, Vannevar Bush, web application, WikiLeaks, Y Combinator

The first assigned reading was their own paper, but later in the semester a class was devoted to a comparison of PageRank and Kleinberg’s work. In December, after the final projects were due, Page emailed the students a party invitation that also marked a milestone: “The Stanford Research Project is now The Next Generation Internet Search Company.” “Dress is Tiki Lounge wear,” the invitation read, “and bring something for the hot tub.” 2 “We want Google to be as smart as you.” Larry Page did not want to be Tesla’d. Google had quickly become a darling of everyone who used it to search the net. But at first so had AltaVista, and that search engine had failed to improve. How was Google, led by two talented but inexperienced youngsters, going to tackle the devilishly difficult problems of improving its service?

However, I hadn’t expected that instead of being attired in traditional T-shirts and jeans, the employees were decked out in costumes. I had come on Halloween. “Steven, meet Larry Page and Sergey Brin,” said Cindy, introducing me to the two young men who had founded the company as Stanford graduate students. Larry was dressed as a Viking, with a long-haired fur vest and a hat with long antlers protruding. Sergey was in a cow suit. On his chest was a rubber slab from which protruded huge, wart-specked teats. They greeted me cheerfully and we all retreated to a conference room where the Viking and the cow explained the miraculous powers of Google’s PageRank technology. That was the first of many interviews I would conduct at Google. Over the next few years, the company became a focus of my technology reporting at Newsweek. Google grew from the small start-up I had visited to a behemoth of more than 20,000 employees.

Explaining how a company that made an operating system for mobile phones fit into this mission would eventually present a challenge for Google’s publicists. But Larry Page interpreted Google’s mission in the broadest sense. What was good for the web was good for Google. What was good for the cloud was good for Google. So it made sense that what was good for the growing universe of wireless communication over mobile phone carrier networks would also be good for Google. Because the carriers tightly controlled the software that ran on phones using their networks, Google had reason to worry that it might not have the opportunity to place its services on those nets. An open network would give Google unlimited opportunity, so that even if Google spent millions of dollars to develop an operating system—and then gave it away for free—it would still come out ahead. If the move wound up putting Google in the path of a few more competitors, so be it.


pages: 532 words: 139,706

Googled: The End of the World as We Know It by Ken Auletta


23andMe, AltaVista, Anne Wojcicki, Apple's 1984 Super Bowl advert, bioinformatics, Burning Man, carbon footprint, citizen journalism, Clayton Christensen, cloud computing, Colonization of Mars, corporate social responsibility, death of newspapers, disintermediation, don't be evil, facts on the ground, Firefox, Frank Gehry, Google Earth, hypertext link, Innovator's Dilemma, Internet Archive, invention of the telephone, Jeff Bezos, jimmy wales, Kevin Kelly, knowledge worker, Long Term Capital Management, Mark Zuckerberg, Marshall McLuhan, Menlo Park, Network effects, new economy, Nicholas Carr, PageRank, Paul Buchheit, Peter Thiel, Ralph Waldo Emerson, Richard Feynman, Richard Feynman, Sand Hill Road, Saturday Night Live, semantic web, sharing economy, Silicon Valley, Skype, slashdot, social graph, spectrum auction, stealth mode startup, Stephen Hawking, Steve Ballmer, Steve Jobs, strikebreaker, telemarketer, the scientific method, The Wisdom of Crowds, Upton Sinclair, X Prize, yield management

When a question is typed into the Google search box, the task is to divine the searcher’s intention: when you wrote “Jobs” in the query box, did you mean employment or Steve Jobs? The query may produce thousands of links, but the promise of Google—what Google considers its secret sauce—is that the ones that appear near the top of the search results will be more relevant to you. The company’s algorithms not only rank those links that generate the most traffic, and therefore are presumed to be more reliable, they also assign a slightly higher qualitative ranking to more reliable sources—like, for instance, a New York Times story. By mapping how many people click on a link, or found it interesting enough to link to, Google determines whether the link is “relevant” and assigns it a value. This quantified value is known as PageRank, after Larry Page. All this was interesting enough, but where the Google executives really got Karmazin’s attention was when they described the company’s advertising business, which accounted for almost all its revenues.

The hybrids, wrote Lessig, are those that combine making money with sharing—as Red Hat did by offering Linux for free but selling consultant services to corporations; as Craigslist does by offering 99 percent of its listings for free; as YouTube does by allowing users to freely share videos; and as community-building sites like Facebook do. Google was free, but it was not building a community. While Google warily watched Facebook, a real skirmish broke out between Google and the bear that is the advertising industry. Ad executives had been uneasy for some time that Google would displace media-buying agencies. But there were additional concerns. How many more ad dollars would Google siphon from traditional media companies? Would Google disintermediate the sales forces of these companies? Might Google bypass advertising agencies and develop a direct relationship with advertisers? If Google’s automated auction system brought the cost efficiencies Larry Page touted, would it not inevitably lower old media’s advertising rates as well as the fees ad agencies charged clients?

Norman, Design of Everyday Things, Basic Books, 1988. 37 an obsession of Larry’s: author interview with Larry Page, March 25, 2008. 38 disdained games like golf: author interview with Omid Kordestani, April 15, 2008. 38 “two swords sharpening each other”: author interview with John Battelle, March 20, 2008. 38 “they were not”: author interview with Terry Winograd, September 25, 2007. 38 Page and Brin’s breakthrough: Search, John Battelle. 39 “they didn’t have this false respect”: author interview with Rajeev Motwani, October 12, 2007. 39 snuck onto the loading dock: author interview with Terry Winograd: September 16, 2008. 39 “We wanted to finish school”: Page and Schmidt appearance at Stanford, May 1, 2002, available on YouTube. 40 “You guys can always come back”: author interview with Larry Page, March 25, 2008; confirmed in a May 5, 2008 e-mail to the author from Jeffrey Ullman. 40 They chose the name Google: Sergey Brin interview with John Ince on PodVentureZone, January 2000. 40 “two important features”: Page and Brin, “The Anatomy of a Large-Scale Hypertextual Web Search Engine”; a printed version, “The PageRank Citation Ranking: Bringing Order to the Web,” was published January 29, 1998, and is available on the Web. 40 “Brin and Page . . . are expressing a desire”: Nicholas Carr, Big Switch: Rewiring the World, From Edison to Google, W. W. Norton & Company, 2008. 41 “They were . . . part of an engineering tribe”: author interview with Lawrence Lessig, March 30, 2009. 41 “This is going to change the way”: author interview with Rajeev Motwani, October 12, 2007. 41 “free of many of the old prejudices”: Nicholas Negroponte, Being Digital, Alfred A.


pages: 250 words: 73,574

Nine Algorithms That Changed the Future: The Ingenious Ideas That Drive Today's Computers by John MacCormick, Chris Bishop


Ada Lovelace, AltaVista, Claude Shannon: information theory, fault tolerance, information retrieval, Menlo Park, PageRank, pattern recognition, Richard Feynman, Richard Feynman, Silicon Valley, Simon Singh, sorting algorithm, speech recognition, Stephen Hawking, Steve Jobs, Steve Wozniak, traveling salesman, Turing machine, Turing test, Vannevar Bush

For readers with a computer science background, Search Engines: Information Retrieval in Practice, by Croft, Metzler, and Strohman, is a good option for learning more about indexing and many other aspects of search engines. PageRank (chapter 3). The opening quotation by Larry Page is taken from an interview by Ben Elgin, published in Businessweek, May 3, 2004. Vannevar Bush's “As We May Think” was, as mentioned above, originally published in The Atlantic magazine (July 1945). Bishop's lectures (see above) contain an elegant demonstration of PageRank using a system of water pipes to emulate hyperlinks. The original paper describing Google's architecture is “The Anatomy of a Large-Scale Hypertextual Web Search Engine,” written by Google's co-founders, Sergey Brin and Larry Page, and presented at the 1998 World Wide Web conference. The paper includes a brief description and analysis of PageRank. A much more technical, wide-ranging analysis appears in Langville and Meyer's Google's PageRank and Beyond—but this book requires college-level linear algebra.

And here is where our story really begins: in the words of PC Magazine, Google's elite status was awarded for its “uncanny knack for returning extremely relevant results.” You may recall from the last chapter that the first commercial search engines had been launched four years earlier, in 1994. How could the garage-bound Google overcome this phenomenal four-year deficit, leapfrogging the already-popular Lycos and AltaVista in terms of search quality? There is no simple answer to this question. But one of the most important factors, especially in those early days, was the innovative algorithm used by Google for ranking its search results: an algorithm known as PageRank. The name “PageRank” is a pun: it's an algorithm that ranks web pages, but it's also the ranking algorithm of Larry Page, its chief inventor. Page and Brin published the algorithm in 1998, in an academic conference paper, “The Anatomy of a Large-scale Hypertextual Web Search Engine.”

As we already know, efficient matching is only half the story for an effective search engine: the other grand challenge is to rank the matching pages. And as we will see in the next chapter, the emergence of a new type of ranking algorithm was enough to eclipse AltaVista, vaulting Google into the forefront of the world of web search. 3 PageRank: The Technology That Launched Google The Star Trek computer doesn't seem that interesting. They ask it random questions, it thinks for a while. I think we can do better than that. —LARRY PAGE (Google cofounder) Architecturally speaking, the garage is typically a humble entity. But in Silicon Valley, garages have a special entrepreneurial significance: many of the great Silicon Valley technology companies were born, or at least incubated, in a garage. This is not a trend that began in the dot-com boom of the 1990s.


pages: 368 words: 96,825

Bold: How to Go Big, Create Wealth and Impact the World by Peter H. Diamandis, Steven Kotler


3D printing, additive manufacturing, Airbnb, Amazon Mechanical Turk, Amazon Web Services, augmented reality, autonomous vehicles, cloud computing, crowdsourcing, Daniel Kahneman / Amos Tversky, dematerialisation, deskilling, Elon Musk,, Exxon Valdez, fear of failure, Firefox, Galaxy Zoo, Google Glasses, Google Hangouts, Google X / Alphabet X, gravity well, industrial robot, Internet of things, Jeff Bezos, John Harrison: Longitude, Jono Bacon, Just-in-time delivery, Kickstarter, Kodak vs Instagram, Law of Accelerating Returns, Lean Startup, life extension, loss aversion, Louis Pasteur, Mahatma Gandhi, Mark Zuckerberg, Mars Rover, meta analysis, meta-analysis, microbiome, minimum viable product, move fast and break things, Narrative Science, Netflix Prize, Network effects, Oculus Rift, optical character recognition, packet switching, PageRank, pattern recognition, performance metric, Peter H. Diamandis: Planetary Resources, Peter Thiel, pre–internet, Ray Kurzweil, recommendation engine, Richard Feynman, Richard Feynman, ride hailing / ride sharing, risk tolerance, rolodex, self-driving car, sentiment analysis, shareholder value, Silicon Valley, Silicon Valley startup, skunkworks, Skype, smart grid, stem cell, Stephen Hawking, Steve Jobs, Steven Levy, Stewart Brand, technoutopianism, telepresence, telepresence robot, Turing test, urban renewal, web application, X Prize, Y Combinator

v=9pmPa_KxsAM. 40 Joann Muller, “No Hands, No Feet: My Unnerving Ride in Google’s Driverless Car,” Forbes, March 21, 2013, 41 Robert Hof, “10 Breakthrough Technologies 2013: Deep Learning,” MIT Technology Review, April 23, 2013, 42 Steven Levy, “Google’s Larry Page on Why Moon Shots Matter,” Wired, January 17, 2013, 43 Larry Page, “Beyond Today—Larry Page—Zeitgeist 2012.” 44 Larry Page, “Google+: Calico Announcement,” Google+, September 2013, 45 Harry McCracken and Lev Grossman, “Google vs. Death,” Time, September 30, 2013, 46 Jason Calacanis, “#googlewinseverything (part 1),” Launch, October 30, 2013,

v=G-0KJF3uLP8. 31 “About Blue Origin,” Blue Origin, July 2014, 32 Alistair Barr, “Amazon testing delivery by drone, CEO Bezos Says,” USA Today, December 2, 2013, referencing a 60 Minutes interview with Jeff Bezos, 33 Jay Yarow, “Jeff Bezos’ Shareholder Letter Is Out,” Business Insider, April 10, 2014, 34 “Larry Page Biography,” Academy of Achievement, January 21, 2011, 35 Marcus Wohlsen, “Google Without Larry Page Would Not Be Like Apple Without Steve Jobs,” Wired, October 18, 2013, 36 Google Inc., 2012, Form 10-K 2012, retrieved from SEC Edgar website: 37 Larry Page, “Beyond Today—Larry Page—Zeitgeist 2012,” Google Zeitgeist, Zeitgeist Minds, May 22, 2012, 38 Matt Ridley, The Rational Optimist: How Prosperity Evolves (New York: HarperCollins, 2010). 39 Larry Page, “Google I/O 2013: Keynote,” Google I/O 2013, Google Developers, May 15, 2013,

This led to a partnership with another Stanford PhD student, Sergey Brin, and a research project nicknamed BackRub, which led to the page-rank algorithm that became Google. Not surprisingly, neither Brin nor Page ever finished their PhDs. Instead, in 1998, they dropped out and started up and changed history. The PageRank algorithm democratized access to information, or as a recent article in Wired put it: “Search, Google’s core product, is itself wondrous. Unlike shiny new gadgets, however, Google search has become such an expected part of the internet’s fabric that it has become mundane.”35 Meanwhile, YouTube became the dominant video platform on the web, Chrome the most popular browser, and Android the most prolific mobile phone operating system ever. To put this in perspective, today a Masai warrior in the heart of Kenya who has a smartphone and access to Google has—at his fingertips—access to the same level of information that the president of the United States did eighteen years ago.


pages: 496 words: 154,363

I'm Feeling Lucky: The Confessions of Google Employee Number 59 by Douglas Edwards


Albert Einstein, AltaVista, Any sufficiently advanced technology is indistinguishable from magic, barriers to entry, book scanning, Build a better mousetrap, Burning Man, business intelligence, call centre, crowdsourcing, don't be evil, Elon Musk, fault tolerance, Googley, gravity well, invisible hand, Jeff Bezos, job-hopping, Menlo Park, microcredit, music of the spheres, Network effects, P = NP, PageRank, performance metric,, Ralph Nader, risk tolerance, second-price auction, side project, Silicon Valley, Silicon Valley startup, slashdot, stem cell, Superbowl ad, Y2K

It's some kind of technical way to say "unrelated." I still don't really get it. But that didn't stop me from casually dropping it into conversations with engineers: "Oh, yeah, that press release is totally orthogonal to the ads we're running on Yahoo." Overture: The name assumed by the advertising network GoTo in October 2001. PageRank: An algorithm used for analyzing the relative importance of pages on the web. Written by, and named for, Google's co-founder Larry Page. PageRank's breakthrough approach was to look at the sites linking to a particular page to determine how many other websites deemed that page authoritative or important. Pay for inclusion: Some search engines accept payment from website owners to guarantee that their sites will be included in search results. These search engines don't necessarily guarantee the site prominent placement.

Except that when the "advanced features" were activated, they also gave Google a look at every page a user viewed. To tell you the PageRank of a site, Google needed to know what site you were visiting. The Toolbar sent that data back to Google if you let it, and Google would show you the green bar. The key was "if you let it," because you could also download a version of the toolbar that would not send any data back to Google. The user could make the choice, though Larry and the engineering team believed—and hoped—that most people wouldn't pass up the advanced features just because Google might learn their surfing habits. We're talking free extra data here. While knowing the PageRank of a page might have only nominal value to users, knowing the sites users visited would be tremendously valuable to Google. The PageRank indicator provided a justification for gathering it.

Next I connected the boxes and turned them into a pair of Ben Franklin–style glasses, with a spider hanging from a thread where the nose would be. "Once we have an index," Craig continued, "we assign a rank to each page based on its importance with our PageRank algorithm. PageRank is Google's secret sauce." "Secret sauce?" I leaned forward to learn what we had that was better than all the other search engines that our founders seemed so quick to dismiss. "PageRank looks at all the pages on the web and assigns a value to them based on who else links to them. The more credible the sites linking to them, the higher the PageRank. That's the first half of the recipe." I wrote "pageRank" under the Ben Franklin spectacles and drew an oval around it. It looked a little like a clown mouth, so I sketched a skull around it and added some Bozo hair on the sides. "The second half is how we determine which results are most relevant to the specific query we've received.


pages: 361 words: 81,068

The Internet Is Not the Answer by Andrew Keen


3D printing, A Declaration of the Independence of Cyberspace, Airbnb, AltaVista, Andrew Keen, augmented reality, Bay Area Rapid Transit, Berlin Wall, bitcoin, Black Swan, Burning Man, Cass Sunstein, citizen journalism, Clayton Christensen, clean water, cloud computing, collective bargaining, Colonization of Mars, computer age, connected car, cuban missile crisis, David Brooks, disintermediation, Downton Abbey, Edward Snowden, Elon Musk, Erik Brynjolfsson, Fall of the Berlin Wall, Filter Bubble, Francis Fukuyama: the end of history, Frank Gehry, Frederick Winslow Taylor, frictionless, full employment, future of work, gig economy, global village, Google bus, Google Glasses, Hacker Ethic, happiness index / gross national happiness, income inequality, index card, informal economy, information trail, Innovator's Dilemma, Internet of things, Isaac Newton, Jaron Lanier, Jeff Bezos, job automation, Joseph Schumpeter, Julian Assange, Kevin Kelly, Kickstarter, Kodak vs Instagram, Lean Startup, libertarian paternalism, Lyft, Mark Zuckerberg, Marshall McLuhan, Martin Wolf, move fast and break things, Nate Silver, Network effects, new economy, Nicholas Carr, nonsequential writing, Norbert Wiener, Occupy movement, packet switching, PageRank, Paul Graham, Peter Thiel, Plutocrats, plutocrats, Potemkin village, precariat, pre–internet, RAND corporation, Ray Kurzweil, ride hailing / ride sharing, Second Machine Age, self-driving car, sharing economy, Silicon Valley, Silicon Valley ideology, Skype, smart cities, Snapchat, social web, South of Market, San Francisco, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, TaskRabbit, Ted Nelson, telemarketer, the medium is the message, Thomas L Friedman, Tyler Cowen: Great Stagnation, Uber for X, urban planning, Vannevar Bush, Whole Earth Catalog, WikiLeaks, winner-take-all economy, working poor, Y Combinator

It seems like a win-win for everyone, of course—one of those supposedly virtuous circles that Sergey Brin and Larry Page built into PageRank. We all get free tools and the Internet entrepreneurs get to become superrich. KPCB cofounder Tom Perkins, whose venture fund has made billions from its investments in Google, Facebook, and Twitter, would no doubt claim that the achievement of what he called Silicon Valley’s “successful one percent” is resulting in more jobs and general prosperity. But as always with something that’s too good to be true, there’s a catch. The problem, of course, is that we are all working for Facebook and Google for free, manufacturing the very personal data that makes their companies so valuable. So Google, with its mid-2014 market cap of over $400 billion, needs to employ only 46,000 people.

They rose to $347 million in 2002, then to just under a billion dollars in 2003 and to almost $2 billion in 2004, when the six-year-old company went public in a $1.67 billion offering that valued it at $23 billion. By 2014, Google had become the world’s second most valuable company, after Apple, with a market cap of over $400 billion, and Brin and Page were the two wealthiest young men in the world, with fortunes of around $30 billion apiece. In vivid contrast with Amazon, Google’s profits were also astonishing. In 2012, its operational profits were just under $14 billion from revenues of $50 billion. In 2013, Google “demolished” Wall Street expectations and returned operational profits of over $15 billion from revenues of nearly $60 billion.71 Larry Page’s response to John Doerr’s question when they first met in 1999 had turned out to be a dramatic underestimation of just “how big” Google could become. And the company is still growing. By 2014, Google had joined Amazon as a winner-take-all company.

Hennessy, “The Tech Utopia Nobody Wants: Why the World Nerds Are Creating Will Be Awful,” Guardian, July 21, 2014. 60 Krissy Clark, “What Did the Tech CEO Say to the Worker He Wanted to Automate?,”, August 28, 2013. 61 Solnit, “Google Invades.” 62 Justine Sharrock, “How San Francisco Tech Companies Justify Their Tax Breaks,” Buzzfeed, October 8, 2013. 63 Sam Biddle, “This Asshole Misses the Shutdown,” Valleywag, October 17, 2013. 64 Max Read, “Oakland Residents Are Crowdfunding a Private Police Force,” Valleywag, October 4, 2013. 65 Lisa Fernandez, “Facebook Will Be First Private Company in U.S. to Pay for Full-Time Beat Cop,”, March 5, 2014. 66 Geoffrey A. Fowler and Brenda Cronin, “Freelancers Get Jobs Via Web Services,” Wall Street Journal, May 29, 2013. 67 Greg Kumparak, “Larry Page Wants Earth to Have a Mad Scientist Island,” TechCrunch, May 2015. 68 Sean Gallagher, “Larry Page Wants You to Stop Worrying and Let Him Fix the World,” Ars Technica, May 20, 2013. 69 “What Is Burning Man?


pages: 274 words: 75,846

The Filter Bubble: What the Internet Is Hiding From You by Eli Pariser


A Declaration of the Independence of Cyberspace, A Pattern Language, Amazon Web Services, augmented reality, back-to-the-land, Black Swan, borderless world, Build a better mousetrap, Cass Sunstein, citizen journalism, cloud computing, cognitive dissonance, crowdsourcing, Danny Hillis, data acquisition, disintermediation, don't be evil, Filter Bubble, Flash crash, fundamental attribution error, global village, Haight Ashbury, Internet of things, Isaac Newton, Jaron Lanier, Jeff Bezos, jimmy wales, Kevin Kelly, knowledge worker, Mark Zuckerberg, Marshall McLuhan, megacity, Netflix Prize, new economy, PageRank, paypal mafia, Peter Thiel, recommendation engine, RFID, sentiment analysis, shareholder value, Silicon Valley, Silicon Valley startup, social graph, social software, social web, speech recognition, Startup school, statistical model, stem cell, Steve Jobs, Steven Levy, Stewart Brand, technoutopianism, the scientific method, urban planning, Whole Earth Catalog, WikiLeaks, Y Combinator

But it would take two Stanford graduate students to apply the principles of machine learning to the whole world of online information. Click Signals As Jeff Bezos’s new company was getting off the ground, Larry Page and Sergey Brin, the founders of Google, were busy doing their doctoral research at Stanford. They were aware of Amazon’s success—in 1997, the dot-com bubble was in full swing, and Amazon, on paper at least, was worth billions. Page and Brin were math whizzes; Page, especially, was obsessed with AI. But they were interested in a different problem. Instead of using algorithms to figure out how to sell products more effectively, what if you could use them to sort through sites on the Web? Page had come up with a novel approach, and with a geeky predilection for puns, he called it PageRank. Most Web search companies at the time sorted pages using keywords and were very poor at figuring out which page for a given word was the most relevant.

LinkedIn Linux Lippmann, Walter Livingston, Jessica local-maximum problem lock-in “Long Live the Web” (Berners-Lee) Loopt Lovell, Todd Lowenstein, George luck Lynch, Zack Ma, Jack MacDougald, Harry machine learning Mancini, Paul Mark, David Mayer, Marissa Mayer-Schonberger, Viktor McLarty, Mack McLuhan, Marshall McPhie, Jonathan Meadowlands meaning threats mean world syndrome MeetUp Metcalfe, Bob Metcalfe’s law Microsoft Bob Live middleman, elimination of Migra corridos Milgram, Stanley Mill, John Stuart Minority Report Moore’s law MORIS Moses, Robert mousetraps MoveOn Mulvenon, James MySpace Nabokov, Vladimir National Rifle Association (NRA) National Security Agency (NSA) Nauman, Arthur Ne’eman, Yuval Negroponte, Nicholas Nemeth, Charlan Netflix Netflix Challenge Newmark, Craig news Facebook feeds Google News people-powered Yahoo News newspapers editorial ethics and ombudsmen and New York Times 9/11 Nordhaus, Ted Norvig, Peter Nosenko, Yuri Obama, Barack Oceana “Of Sirens and Amish Children” (Benkler) OkCupid Olson, Carrie Oswald, Lee Harvey overfitting Pabst Page, Larry PageRank Palmer, Chris Palo Alto Research Center (PARC) Pandora PanelDirector Pareles, Jon Parker, Sean Pattern Language, A (Alexander et al.) PayPal PeekYou persuasion profiling Phantom Public, The (Lippmann) Philby, Kim Phorm Piaget, Jean Picasa Picasso, Pablo PK List Management Plato politics electoral districts and partisans and programmers and voting Popper, Karl postmaterialism predictions present bias priming effect privacy Facebook and facial recognition and genetic Procter & Gamble product recommendations Proulx, Travis Pulitzer, Joseph push technology and pull technology Putnam, Robert Qiang, Xiao Rapleaf Rather, Dan Raz, Guy reality augmented Reality Hunger (Shields) Reddit Rendon, John (Sunstein) retargeting RFID chips robots Rodriguez de Montalvo, Garci Rolling Stone Roombas Rotenberg, Marc Rothstein, Mark Rove, Karl Royal Caribbean Rubel, Steve Rubicon Project Rumsfeld, Donald Rushkoff, Douglas Salam, Reihan Sandberg, Sheryl schemata Schmidt, Eric Schudson, Michael Schulz, Kathryn science Scientific American Scorpion sentiment analysis Sentry serendipity Shields, David Shirky, Clay Siegel, Lee signals click Simonton, Dean Singhal, Amit Sleepwalkers, The (Koestler) smart devices Smith, J.

The project is undoubtedly a great service to researchers and the casually curious public. But serving academia probably wasn’t Google’s only motive. Remember Larry Page’s declaration that he wanted to create a machine “that can understand anything,” which some people might call artificial intelligence? In Google’s approach to creating intelligence, the key is data, and the 5 million digitized books contain an awful lot of it. To grow your artificial intelligence, you need to keep it well fed. To get a sense of how this works, consider Google Translate, which can now do a passable job translating automatically among nearly sixty languages. You might imagine that Translate was built with a really big, really sophisticated set of translating dictionaries, but you’d be wrong. Instead, Google’s engineers took a probabilistic approach: They built software that could identify which words tended to appear in connection with which, and then sought out large chunks of data that were available in multiple languages to train the software on.


pages: 281 words: 95,852

The Googlization of Everything: by Siva Vaidhyanathan


1960s counterculture, AltaVista, barriers to entry, Berlin Wall, borderless world, Burning Man, Cass Sunstein, choice architecture, cloud computing, computer age, corporate social responsibility, correlation does not imply causation, data acquisition, death of newspapers, don't be evil, Firefox, Francis Fukuyama: the end of history, full text search, global village, Google Earth, Howard Rheingold, informal economy, information retrieval, Joseph Schumpeter, Kevin Kelly, knowledge worker, libertarian paternalism, market fundamentalism, Marshall McLuhan, means of production, Mikhail Gorbachev, Naomi Klein, Network effects, new economy, Nicholas Carr, PageRank, pirate software, Ray Kurzweil, Richard Thaler, Ronald Reagan, side project, Silicon Valley, Silicon Valley ideology, single-payer health, Skype, social web, Steven Levy, Stewart Brand, technoutopianism, The Nature of the Firm, The Structural Transformation of the Public Sphere, Thorstein Veblen, urban decay, web application

Unfortunately, universities have allowed Google to take the lead in and set the terms of the relationship. There is a strong cultural affinity between Google corporate culture and that of academia. Google’s founders, Sergey Brin and Larry Page, THE GOOGL I ZAT I ON OF ME MORY 187 met while pursuing PhDs in computer science at Stanford University.16 The foundational concept behind Google Web Search, the PageRank algorithm, emerged from an academic paper that Brin and Page wrote and published in 1999.17 Page did his undergraduate work at the University of Michigan and retains strong ties with that institution. Some of the most visionary Google employees, such as the University of California at Berkeley economist Hal Varian, suspended successful academic careers to join the company. So it’s not surprising that Google’s corporate culture reflects much of the best of academic work life: unstructured work time, horizontal management structures, multidirectional information and feedback flows, an altruistic sense of mission, recreation and physical activity integrated centrally into the “campus,” and an alarmingly relaxed dress code.

Google’s leaders—Sergey Brin, Larry Page, and Eric Schmidt—could leave the company because of illness or professional differences. Google might fail to make enough money to cover the costs of its commitments and liabilities. Governments might severely restrict Google’s ability to turn attention into cash or to dominate the search market. Anything is possible. And whereas institutions such as libraries, states, and universities tend to last for centuries, commercial firms rarely make it through one century. Most die or change unrecognizably within their first two decades. Google is halfway to that point. We should not count on the company being the same old Google, or even being around to serve as well as it has done so far, when it lurches through adolescence. Clearly, we should not trust Google to be the custodian of our most precious cultural and scientific resources.

Its verity is in fact an event, a process: the process namely of its verifying itself, its veri-fication.”24 James’s focus on the dynamism of truth—what Rorty later called “contingency”—is embodied in Google PageRank.25 Rank is assigned to a site through a dynamic process of verification by communal affirmation. The instrument of that affirmation is the hyperlink. The secondary instrument is the click on the hyperlink. The field in which the affirmations are transformed into contingent, temporary judgments of relevance or, as James might say, truth, is the PageRank algorithm. And this is the brilliance of PageRank and Google’s Web Search system in general: how else would one make sense of something as dynamic and messy as the World Wide Web? Just as pragmatism helps us understand what we mean when we say something in the world is “true” or that we “believe” something, Google sifts through an enormous array of documents and orders them in a way that reflects a rough—very rough—consensus among Web users.


pages: 413 words: 119,587

Machines of Loving Grace: The Quest for Common Ground Between Humans and Robots by John Markoff


A Declaration of the Independence of Cyberspace, AI winter, airport security, Apple II, artificial general intelligence, augmented reality, autonomous vehicles, Baxter: Rethink Robotics, Bill Duvall, bioinformatics, Brewster Kahle, Burning Man, call centre, cellular automata, Chris Urmson, Claude Shannon: information theory, Clayton Christensen, clean water, cloud computing, collective bargaining, computer age, computer vision, crowdsourcing, Danny Hillis, DARPA: Urban Challenge, data acquisition, Dean Kamen, deskilling, don't be evil, Douglas Engelbart, Douglas Hofstadter, Dynabook, Edward Snowden, Elon Musk, Erik Brynjolfsson, factory automation, From Mathematics to the Technologies of Life and Death, future of work, Galaxy Zoo, Google Glasses, Google X / Alphabet X, Grace Hopper, Gödel, Escher, Bach, Hacker Ethic, haute couture, hive mind, hypertext link, indoor plumbing, industrial robot, information retrieval, Internet Archive, Internet of things, invention of the wheel, Jacques de Vaucanson, Jaron Lanier, Jeff Bezos, job automation, John Conway, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John von Neumann, Kevin Kelly, knowledge worker, Kodak vs Instagram, labor-force participation, loose coupling, Mark Zuckerberg, Marshall McLuhan, medical residency, Menlo Park, Mother of all demos, natural language processing, new economy, Norbert Wiener, PageRank, pattern recognition, pre–internet, RAND corporation, Ray Kurzweil, Richard Stallman, Robert Gordon, Rodney Brooks, Sand Hill Road, Second Machine Age, self-driving car, semantic web, shareholder value, side project, Silicon Valley, Silicon Valley startup, Singularitarianism, skunkworks, Skype, social software, speech recognition, stealth mode startup, Stephen Hawking, Steve Ballmer, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, strong AI, superintelligent machines, technological singularity, Ted Nelson, telemarketer, telepresence, telepresence robot, Tenerife airport disaster, The Coming Technological Singularity, the medium is the message, Thorstein Veblen, Turing test, Vannevar Bush, Vernor Vinge, Watson beat the top human players on Jeopardy!, Whole Earth Catalog, William Shockley: the traitorous eight

Driving was the original metaphor for interactive computing, but today Google’s vision has changed the metaphor. The new analogy will be closer to traveling in an elevator or a train without human intervention. In Google’s world you will press a button and be taken to your destination. This conception of transportation undermines several notions that are deeply ingrained in American culture. In the last century the car became synonymous with the American ideal of freedom and independence. That era is now ending. What will replace it? It is significant that Google is instrumental in changing the metaphor. In one sense the company began as the quintessential intelligence augmentation, or IA, company. The PageRank algorithm Larry Page developed to improve Internet search results essentially mined human intelligence by using the crowd-sourced accumulation of human decisions about valuable information sources.

In the audience was Scott Hassan, the former Stanford graduate student who had done the original heavy lifting for Google as the first PageRank algorithm programmer, the basis for the company’s core search engine. It’s time to build an AI robot, Ng told the group. He said his dream was to put a robot in every home. The idea resonated with Hassan. A student in computer science first at the State University of New York at Buffalo, he then entered graduate programs in computer science at both Washington University in St. Louis and Stanford, but dropped out of both programs before receiving an advanced degree. Once he was on the West Coast, he had gotten involved with Brewster Kahle’s Internet Archive Project, which sought to save a copy of every Web page on the Internet. Larry Page and Sergey Brin had given Hassan stock for programming PageRank, and Hassan also sold E-Groups, another of his information retrieval projects, to Yahoo!

The idea felt like science fiction come to life, and Rose, who appeared stunned, did not ask hard questions. Google, however, unveiled its own drone delivery research project. Just days after the Amazon 60 Minutes extravaganza, the New York Times reported on Google’s robotic ambitions, which dwarfed what Bezos had sketched on the TV news show. Rubin had stepped down as head of Google’s Android phone division in the spring of 2013. Despite reports that he had lost a power struggle and was held in disfavor, exactly the opposite was true. Larry Page, Google’s chief executive, had opened the corporate checkbook and sent Rubin on a remarkable shopping spree. Rubin had spent hundreds of millions of dollars recruiting the best robotics talent and buying the best robotic technology in the world. In addition to Schaft, Google had also acquired Industrial Perception, Meka Robotics, and Redwood Robotics, a group of developers of humanoid robots and robot arms in San Francisco led by one of Rodney Brooks’s star students, and Bot & Dolly, a developer of robotic camera systems that had been used to create special effects in the movie Gravity.


pages: 294 words: 81,292

Our Final Invention: Artificial Intelligence and the End of the Human Era by James Barrat


3D printing, AI winter, Amazon Web Services, artificial general intelligence, Automated Insights, Bernie Madoff, Bill Joy: nanobots, brain emulation, cellular automata, cloud computing, cognitive bias, computer vision, cuban missile crisis, Daniel Kahneman / Amos Tversky, Danny Hillis, data acquisition, don't be evil, Extropian, finite state, Flash crash, friendly AI, friendly fire, Google Glasses, Google X / Alphabet X, Isaac Newton, Jaron Lanier, John von Neumann, Kevin Kelly, Law of Accelerating Returns, life extension, Loebner Prize, lone genius, mutually assured destruction, natural language processing, Nicholas Carr, optical character recognition, PageRank, pattern recognition, Peter Thiel, prisoner's dilemma, Ray Kurzweil, Rodney Brooks, Search for Extraterrestrial Intelligence, self-driving car, semantic web, Silicon Valley, Singularitarianism, Skype, smart grid, speech recognition, statistical model, stealth mode startup, stem cell, Stephen Hawking, Steve Jobs, Steve Wozniak, strong AI, Stuxnet, superintelligent machines, technological singularity, The Coming Technological Singularity, traveling salesman, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, zero day

Entrepreneur and AI maker Peter Voss speculated that had Aristotle possessed Einstein’s knowledge base, he could’ve come up with the theory of general relativity. The Google, Inc. search engine in particular has multiplied worker productivity, especially in occupations that call for research and writing. Tasks that formerly required time-consuming research—a trip to the library to pore over books and periodicals, perform Lexis/Nexis searches, and look up experts and write or phone them—are now fast, easy, and cheap. Much of this increased productivity is due, of course, to the Internet itself. But the vast ocean of information it holds is overwhelming without intelligent tools to extract the small fraction you need. How does Google do it? Google’s proprietary algorithm called PageRank gives every site on the entire Internet a score of 0 to 10. A score of 1 on PageRank (allegedly named after Google cofounder Larry Page, not because it ranks Web pages) means a page has twice the “quality” of a site with a PageRank of 0.

., July 12, 2011, (accessed August 28, 2012). People always make the assumption: Memepunks, “Google A.I. a Twinkle in Larry Page’s Eye,” May 26, 2006, (accessed May 3, 2011). Even the Google camera cars: Streitfeld, David, “Google Is Faulted for Impeding U.S. Inquiry on Data Collection,” New York Times, sec. technology, April 14, 2012, (accessed May 3, 2012). It doesn’t take Google glasses: In December 2012, Ray Kurzweil joined Google as Director of Engineering to work on projects involving machine learning and language processing. In the development of AGI, this is a landmark event, and a sobering one.

According to Nicholas Carr, author of the Big Switch, that’s what Google’s cofounder Larry Page has in mind for the search engine’s future. “The idea is that you no longer have to sit down at a keyboard to locate information,” said Carr. “It becomes automatic, a sort of machine-mind meld. Larry Page has discussed a scenario where you merely think of a question, and Google whispers the answer into your ear through your cell phone.” See for instance, the recent announcement of “Project Glass.” They are glasses that allow you to perform Google queries and see the results while you are walking down the street—right in your field of view. “Imagine a very near future when you don’t forget anything because the computer remembers,” said former Google CEO Eric Schmidt. “You are never lost. You are never lonely.”


pages: 390 words: 114,538

Digital Wars: Apple, Google, Microsoft and the Battle for the Internet by Charles Arthur


AltaVista, Build a better mousetrap, Burning Man, cloud computing, credit crunch, crowdsourcing, disintermediation, don't be evil,, Firefox, gravity well, Jeff Bezos, John Gruber, Mark Zuckerberg, Menlo Park, Network effects, PageRank, pre–internet, Robert X Cringely, Silicon Valley, Silicon Valley startup, skunkworks, Skype, slashdot, Snapchat, software patent, speech recognition, stealth mode startup, Steve Ballmer, Steve Jobs, the scientific method, Tim Cook: Apple, upwardly mobile

This ebook published in 2014 by Kogan Page Limited 2nd Floor, 45 Gee Street London EC1V 3RS UK © Charles Arthur, 2012, 2014 E-ISBN 978 0 7494 7204 7 Full imprint details Contents Introduction 01 1998 Bill Gates and Microsoft Steve Jobs and Apple Bill Gates and Steve Jobs Larry Page, Sergey Brin and Google Internet search Capital thinking 02 Microsoft antitrust Steve Ballmer The antitrust trial The outcome of the trial 03 Search: Google versus Microsoft The beginnings of search Google Search and Microsoft Bust Link to money Boom Random access Google and the public consciousness Project Underdog Preparing for battle Do it yourself Going public Competition Cultural differences Microsoft’s relaunched search engine Friends Microsoft’s bid for Yahoo Google’s identity The shadow of antitrust Still underdog 04 Digital music: Apple versus Microsoft The beginning of iTunes Gizmo, Tokyo iPod design Marketing the new product Meanwhile, in Redmond: Microsoft iPods and Windows Music, stored Celebrity marketing iTunes on Windows iPod mini The growth of iTunes Music Store Apple and the mobile phone Stolen!

At the end of the day it was worth $346.7 billion; Microsoft was worth $214.3 billion and Google $185.1 billion. Compared to the end of 1998 (Apple $5.54 billion, Microsoft $344.6 billion, Google $10 million), the aggregate wealth of the companies had more than doubled. Microsoft, though, had shrunk by 40 per cent, after being outdistanced first in search, then in digital music and then in smartphones – in the latter category by both companies. The companies had changed enormously. Google was soon to celebrate its 13th birthday, having roared from a three-person garage start-up to web giant; it was struggling too with having nearly 29,000 staff worldwide. Larry Page, once more the chief executive, was forcing the divisions to justify themselves, getting divisional heads to explain their projects in soundbite-length memos. His greatest concern was that Google was getting too big and slow to act: ‘Large companies are their own worst enemy’, he said in September.

id=1800514 Chapter Seven China 1 2 3 4 5 6 7 8 9 Chapter Eight 2011 1 2 3 References and further reading Auletta, Ken (2009) Googled: The end of the world as we know it, Virgin Books, London Battelle, John (2005) The Search: How Google and its rivals rewrote the rules of business and transformed our culture, Nicholas Brealey, London Deutschman, Alan (2000) The Second Coming of Steve Jobs, Broadway Books, New York Edwards, Douglas (2011) I’m Feeling Lucky: The confessions of Google employee number 59, Allen Lane, London Elliot, Jay (2011) The Steve Jobs Way: iLeadership for a new generation, Vanguard Press, New York Foley, Mary Jo (2008) Microsoft 2.0: How Microsoft plans to stay relevant in the post-Gates era, John Wiley, Hoboken, NJ Isaacson, Walter (2011) Steve Jobs, Little, Brown, London Kirkpatrick, David (2010) The Facebook Effect: The inside story of the company that is connecting the world, Simon & Schuster, New York Levis, Kieran (2009) Winners and Losers: Creators and casualties of the age of the internet, Atlantic Books, London Levy, Steven (2006) The Perfect Thing: How the iPod became the defining object of the 21st century, Ebury Press, London Levy, Steven (2011) In the Plex: How Google thinks, works and shapes our lives, Simon & Schuster, New York Lewis, Michael (1999) The New New Thing: A Silicon Valley story, Hodder & Stoughton, London Norman, Donald A (2004) Emotional Design: Why we love (or hate) everyday things, Basic Books, New York Wu, Tim (2011) The Master Switch: The rise and fall of information empires, Atlantic Books, London Acknowledgements First mention must go to Susannah Lear, who e-mailed me out of the blue with the Velcro-covered idea of a book about these three companies and their interactions.


pages: 299 words: 91,839

What Would Google Do? by Jeff Jarvis


23andMe, Amazon Mechanical Turk, Amazon Web Services, Anne Wojcicki, barriers to entry, Berlin Wall, business process, call centre, cashless society, citizen journalism, clean water, connected car, credit crunch, crowdsourcing, death of newspapers, disintermediation, diversified portfolio, don't be evil, fear of failure, Firefox, future of journalism, Google Earth, Googley, Howard Rheingold, informal economy, inventory management, Jeff Bezos, jimmy wales, Kevin Kelly, Mark Zuckerberg, moral hazard, Network effects, new economy, Nicholas Carr, PageRank, peer-to-peer lending, post scarcity, prediction markets, pre–internet, Ronald Coase, search inside the book, Silicon Valley, Skype, social graph, social software, social web, spectrum auction, speech recognition, Steve Jobs, the medium is the message, The Nature of the Firm, the payments system, The Wisdom of Crowds, transaction costs, web of trust, Y Combinator, Zipcar

The challenge is finding and supporting it. That is where Google comes in. Google can’t and shouldn’t do it all; we still need curators, editors, teachers—and ad salespeople—to find and nurture the best. But Google provides the infrastructure for a culture of choice. Google’s algorithms and its business model work because Google trusts us. That was the ding moment that led Sergey Brin and Larry Page to found their company: the realization that by tracking what we click on and link to, we would lead them to the good stuff and they, in turn, could lead others to it. “Good,” of course, is too relative and loaded a term. “Relevant” is a better description for what Google’s PageRank delivers. As the company explains on its site: PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page’s value.

It is an exercise in enlightened self-interest. Google and its megaplexes of servers are gigantic consumers of electricity with a growing impact on the economy and the earth. Google is not free of atoms’ drag. If Google can help create cleaner, cheaper electricity anywhere it operates, it will improve its own bottom line (the cost of power has been approaching that of the computers themselves in Google’s P&L). It will mitigate charges that Google is becoming a major contributor to carbon pollution. Google will have the flexibility to put servers most anywhere on earth, expanding its reach (Google has even patented the idea of wave-powered, water-cooled server farms on platforms in oceans). And the company will get due credit for helping to save the planet. “Our primary goal is not to fix the world,” Larry Page has said. But wouldn’t that be a nice fringe benefit?

It also had been rumored to be working on making its own Google phone. Instead, it created an open mobile operating system, which any phone manufacturer may use (T-Mobile released the first). In an effort to push the Federal Communications Commission and the mobile phone industry toward openness, Google bid in an auction of wireless spectrum in 2008, making a bargain with the government: Google would guarantee a minimum price of $4.6 billion if the FCC required openness—that is, that any device (such as those powered by Google’s operating system) could operate on much of the spectrum bought and run by the phone companies. Google didn’t win the auction—it won the point. For a few hours, though, it had the highest bid on the table and could have ended up with spectrum and a phone company. In a forum in Washington, D.C., Larry Page looked a bit dreamy and wistful as he recalled that for a day, his company was in the phone business.


pages: 407 words: 103,501

The Digital Divide: Arguments for and Against Facebook, Google, Texting, and the Age of Social Netwo Rking by Mark Bauerlein


Amazon Mechanical Turk, Andrew Keen, centre right, citizen journalism, collaborative editing, computer age, computer vision, corporate governance, crowdsourcing, David Brooks, disintermediation, Frederick Winslow Taylor, Howard Rheingold, invention of movable type, invention of the steam engine, invention of the telephone, Jaron Lanier, Jeff Bezos, jimmy wales, Kevin Kelly, knowledge worker, late fees, Mark Zuckerberg, Marshall McLuhan, means of production, meta analysis, meta-analysis, Network effects, new economy, Nicholas Carr, PageRank,, Results Only Work Environment, Saturday Night Live, search engine result page, semantic web, Silicon Valley, slashdot, social graph, social web, software as a service, speech recognition, Steve Jobs, Stewart Brand, technology bubble, Ted Nelson, The Wisdom of Crowds, Thorstein Veblen, web application

Second, because the blogging community is so highly self-referential, bloggers paying attention to other bloggers magnifies their visibility and power . . . like Wikipedia, blogging harnesses collective intelligence as a kind of filter . . . much as PageRank produces better results than analysis of any individual document, the collective attention of the blogosphere selects for value. PageRank is Google’s algorithm—its mathematical formula—for ranking search results. This is another contribution, according to its touters, to access to information, and therefore yet another boon to “democracy.” PageRank keeps track of websites that are the most linked to—that are the most popular. It is, in fact, the gold standard of popularity in Web culture. What O’Reilly is saying, in plain English, is that the more people blog, and the more blogs link to each other, the more highly ranked the most popular blogs will be.

The company has declared that its mission is “to organize the world’s information and make it universally accessible and useful.” It seeks to develop “the perfect search engine,” which it defines as something that “understands exactly what you mean and gives you back exactly what you want.” In Google’s view, information is a kind of commodity, a utilitarian resource that can be mined and processed with industrial efficiency. The more pieces of information we can “access” and the faster we can extract their gist, the more productive we become as thinkers. Where does it end? Sergey Brin and Larry Page, the gifted young men who founded Google while pursuing doctoral degrees in computer science at Stanford, speak frequently of their desire to turn their search engine into an artificial intelligence, a HAL-like machine that might be connected directly to our brains.

You use your iPhone camera to take a photo of a map that contains details not found on generic mapping applications such as Google maps—say, a trailhead map in a park, or another hiking map. Use the phone’s GPS to set your current location on the map. Walk a distance away, and set a second point. Now your iPhone can track your position on that custom map image as easily as it can on Google maps. Some of the most fundamental and useful services on the Web have been constructed in this way, by recognizing and then teaching the overlooked regularity of what at first appears to be unstructured data. Ti Kan, Steve Scherf, and Graham Toal, the creators of CDDB, realized that the sequence of track lengths on a CD formed a unique signature that could be correlated with artist, album, and song names. Larry Page and Sergey Brin realized that a link is a vote. Marc Hedlund at Wesabe realized that every credit card swipe is also a vote, that there is hidden meaning in repeated visits to the same merchant.


pages: 304 words: 82,395

Big Data: A Revolution That Will Transform How We Live, Work, and Think by Viktor Mayer-Schonberger, Kenneth Cukier


23andMe, Affordable Care Act / Obamacare, airport security, AltaVista, barriers to entry, Berlin Wall, big data - Walmart - Pop Tarts, Black Swan, book scanning, business intelligence, business process, call centre, cloud computing, computer age, correlation does not imply causation, dark matter, double entry bookkeeping, Eratosthenes, Erik Brynjolfsson, game design, IBM and the Holocaust, index card, informal economy, Internet of things, invention of the printing press, Jeff Bezos, Louis Pasteur, Mark Zuckerberg, Menlo Park, Moneyball by Michael Lewis explains big data, Nate Silver, natural language processing, Netflix Prize, Network effects, obamacare, optical character recognition, PageRank, performance metric, Peter Thiel, Post-materialism, post-materialism, random walk, recommendation engine, self-driving car, sentiment analysis, Silicon Valley, Silicon Valley startup, smart grid, smart meter, social graph, speech recognition, Steve Jobs, Steven Levy, the scientific method, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, Turing test, Watson beat the top human players on Jeopardy!

Falls to Lowest Level Since 2008,” Bloomberg, August 13, 2012 ( est-level-since-2008-as-vix-falls.html). [>] Google’s 24 petabytes per day—Thomas H. Davenport, Paul Barth, and Randy Bean, “How ‘Big Data’ Is Different,” Sloan Review, July 30, 2012, pp. 43–46 ( Facebook stats—Facebook IPO prospectus, “Form S-1 Registration Statement,” U.S. Securities and Exchange Commission, February 1, 2012 ( YouTube stats—Larry Page, “Update from the CEO,” Google, April 2012 ( Number of tweets—Tomio Geron, “Twitter’s Dick Costolo: Twitter Mobile Ad Revenue Beats Desktop on Some Days,” Forbes, June 6, 2012 (

With big data, these problems may arise more frequently or have larger consequences. Google, as we’ve shown in many examples, runs everything according to data. That strategy has obviously led to much of its success. But it also trips up the company from time to time. Its co-founders, Larry Page and Sergey Brin, long insisted on knowing all job candidates’ SAT scores and their grade point averages when they graduated from college. In their thinking, the first number measured potential and the second measured achievement. Accomplished managers in their forties who were being recruited were hounded for the scores, to their outright bafflement. The company even continued to demand the numbers long after its internal studies showed no correlation between the scores and job performance. Google ought to know better, to resist being seduced by data’s false charms.

See also imprecision and big data, [>]–[>], [>], [>], [>], [>] in database design, [>]–[>], [>] and measurement, [>]–[>], [>] necessary in sampling, [>], [>]–[>] Excite, [>] Experian, [>], [>], [>], [>], [>] expertise, subject-area: role in big data, [>]–[>] explainability: big data and, [>]–[>] Facebook, [>], [>], [>]–[>], [>]–[>], [>], [>], [>], [>] data processing by, [>] datafication by, [>], [>] IPO by, [>]–[>] market valuation of, [>]–[>] uses “data exhaust,” [>] Factual, [>] Fair Isaac Corporation (FICO), [>], [>] Farecast, [>]–[>], [>], [>], [>], [>], [>], [>], [>], [>] finance: big data in, [>]–[>], [>], [>] Fitbit, [>] Flickr, [>]–[>], [>]–[>] floor covering, touch-sensitive: and datafication, [>] Flowers, Mike: and government use of big data, [>]–[>], [>] flu: cell phone data predicts spread of, [>]–[>] Google predicts spread of, [>]–[>], [>], [>], [>], [>], [>], [>], [>] vaccine shots, [>]–[>], [>]–[>], [>]–[>] Ford, Henry, [>] Ford Motor Company, [>]–[>] Foursquare, [>], [>] Freakonomics (Leavitt), [>]–[>] free will: justice based on, [>]–[>] vs. predictive analytics, [>], [>], [>], [>]–[>] Galton, Sir Francis, [>] Gasser, Urs, [>] Gates, Bill, [>] Geographia (Ptolemy), [>] geospatial location: cell phone data and, [>]–[>], [>]–[>] commercial data applications, [>]–[>] datafication of, [>]–[>] insurance industry uses data, [>] UPS uses data, [>]–[>] Germany, East: as police state, [>], [>], [>] Global Positioning System (GPS) satellites, [>]–[>], [>], [>], [>] Gnip, [>] Goldblum, Anthony, [>] Google, [>], [>], [>], [>], [>], [>], [>], [>] artificial intelligence at, [>] as big-data company, [>] Books project, [>]–[>] data processing by, [>] data-reuse by, [>]–[>], [>], [>] Flu Trends, [>], [>], [>], [>], [>], [>] gathers GPS data, [>], [>], [>] Gmail, [>], [>] Google Docs, [>] and language translation, [>]–[>], [>], [>], [>], [>] MapReduce, [>], [>] maps, [>] PageRank, [>] page-ranking by, [>] predicts spread of flu, [>]–[>], [>], [>], [>], [>], [>], [>], [>] and privacy, [>]–[>] search-term analytics by, [>], [>], [>], [>], [>], [>] speech-recognition at, [>]–[>] spell-checking system, [>]–[>] Street View vehicles, [>], [>]–[>], [>], [>] uses “data exhaust,” [>]–[>] uses mathematical models, [>]–[>], [>] government: and open data, [>]–[>] regulation and big data, [>]–[>], [>] surveillance by, [>]–[>], [>]–[>] Graunt, John: and sampling, [>] Great Britain: open data in, [>] guilt by association: profiling and, [>]–[>] Gutenberg, Johannes, [>] Hadoop, [>], [>] Hammerbacher, Jeff, [>] Harcourt, Bernard, [>] health care: big data in, [>]–[>], [>], [>] cell phone data in, [>], [>]–[>] predictive analytics in, [>]–[>], [>] Health Care Cost Institute, [>] Hellend, Pat: “If You Have Too Much Data, Then ‘Good Enough’ Is Good Enough,” [>] Hilbert, Martin: attempts to measure information, [>]–[>] Hitwise, [>], [>] Hollerith, Herman: and punch cards, [>], [>] Hollywood films: profits predicted, [>]–[>] Honda, [>] Huberman, Bernardo: and social networking analysis, [>] human behavior: datafication and, [>]–[>], [>]–[>] human perceptions: big data changes, [>] IBM, [>] and electric automobiles, [>]–[>] founded, [>] and language translation, [>]–[>], [>] Project Candide, [>]–[>] ID3, [>] “If You Have Too Much Data, Then ‘Good Enough’ Is Good Enough” (Hellend), [>], [>] imprecision.


pages: 23 words: 5,264

Designing Great Data Products by Jeremy Howard, Mike Loukides, Margit Zwemer


AltaVista, Filter Bubble, PageRank, pattern recognition, recommendation engine, self-driving car, sentiment analysis, Silicon Valley, text mining

Our objective and available levers, what data we already have and what additional data we will need to collect, determine the models we can build. The models will take both the levers and any uncontrollable variables as their inputs; the outputs from the models can be combined to predict the final state for our objective. Step 4 of the Drivetrain Approach for Google is now part of tech history: Larry Page and Sergey Brin invented the graph traversal algorithm PageRank and built an engine on top of it that revolutionized search. But you don’t have to invent the next PageRank to build a great data product. We will show a systematic approach to step 4 that doesn’t require a PhD in computer science. The Model Assembly Line: A case study of Optimal Decisions Group Optimizing for an actionable outcome over the right predictive models can be a company’s most important strategic decision.

While their models were good at finding relevant websites, the answer the user was most interested in was often buried on page 100 of the search results. Then, Google came along and transformed online search by beginning with a simple question: What is the user’s main objective in typing in a search query? The four steps in the Drivetrain Approach. Google realized that the objective was to show the most relevant search result; for other companies, it might be increasing profit, improving the customer experience, finding the best path for a robot, or balancing the load in a data center. Once we have specified the goal, the second step is to specify what inputs of the system we can control, the levers we can pull to influence the final outcome. In Google’s case, they could control the ranking of the search results. The third step was to consider what new data they would need to produce such a ranking; they realized that the implicit information regarding which pages linked to which other pages could be used for this purpose.

We call it the Drivetrain Approach, inspired by the emerging field of self-driving vehicles. Engineers start by defining a clear objective: They want a car to drive safely from point A to point B without human intervention. Great predictive modeling is an important part of the solution, but it no longer stands on its own; as products become more sophisticated, it disappears into the plumbing. Someone using Google’s self-driving car is completely unaware of the hundreds (if not thousands) of models and the petabytes of data that make it work. But as data scientists build increasingly sophisticated products, they need a systematic design approach. We don’t claim that the Drivetrain Approach is the best or only method; our goal is to start a dialog within the data science and business communities to advance our collective vision.


pages: 204 words: 67,922

Elsewhere, U.S.A: How We Got From the Company Man, Family Dinners, and the Affluent Society to the Home Office, BlackBerry Moms,and Economic Anxiety by Dalton Conley


3D printing, call centre, clean water, dematerialisation, demographic transition, Edward Glaeser, extreme commuting, feminist movement, financial independence, Firefox, Frank Levy and Richard Murnane: The New Division of Labor, Home mortgage interest deduction, income inequality, informal economy, Jane Jacobs, John Maynard Keynes: Economic Possibilities for our Grandchildren, knowledge economy, knowledge worker, labor-force participation, late capitalism, low skilled workers, manufacturing employment, McMansion, mortgage tax deduction, new economy, oil shock, PageRank, Ponzi scheme, positional goods, post-industrial society, Post-materialism, post-materialism, principal–agent problem, recommendation engine, Richard Florida, rolodex, Ronald Reagan, Silicon Valley, Skype, statistical model, The Death and Life of Great American Cities, The Great Moderation, The Wealth of Nations by Adam Smith, Thomas Malthus, Thorstein Veblen, transaction costs, women in the workforce, Yom Kippur War

There were free bikes left at various stations. I hopped on one and found my way around to Building 40. As I wove through the main central area, I passed a huge dinosaur skeleton (bought by Google co-founder Larry Page on eBay) posed so that it was chasing a flock of pink lawn flamingos, a huge sandbox with a volleyball net strung over it, and an herb and tomato garden growing out of plastic “Earthboxes” meant for high-density soilless agriculture in the developing world. Above the Google reception desk a liquid crystal display offered a scrolling list of searches going on across the world. It is said that Google employees can see how news breaks across the globe in real time by watching searches on, say Britney Spears or a major earthquake, spreading across the globe like an information tsunami.

If you are popular—in other words, if you get a high page rank thanks to lots of other Web pages pointing toward yours within the category delineated by the search terms—then there is no need to pay for anything. But if you are trying to nudge your way into the top ten to get noticed, you have two options: try to game the Google algorithm (which evolves to outsmarting the gamers); or pay dollars. In this way, the abstract notion of network popularity—or rather, of positional status—becomes, like everything else, monetized. Another aspect of the new economics can be seen in these Google AdWords: the blunting of competition. The mechanism behind the auctioning off of paid links to be billed on a click-through basis was not invented by Google but by the company, which later became Overture Services Inc.

When it came time for them to leave their mark, their philanthropy became associated with colossal buildings such as Rockefeller Center, the New York Public Library, and Carnegie Hall. Contrast that to the efforts of Bill Gates, Warren Buffett, and Michael Bloomberg, who have dedicated their wealth—made from “soft” industries—to addressing such issues as malaria and education. It is to this new world of work built by folks like Bloomberg and Google’s Sergey Brin and Larry Page that we now turn. 1∗Never mind that with global reserves dwindling, oil is now increasingly dirty and difficult to extract. 2∗Its main purpose at the time? Helping to build the hydrogen bomb—only the single most destructive invention in human history still to this day. 3∗Americans work an average of 25.1 hours per week (averaged across all working-age persons) in contrast to Germans, for instance, who average 18.6 hours.


pages: 292 words: 85,151

Exponential Organizations: Why New Organizations Are Ten Times Better, Faster, and Cheaper Than Yours (And What to Do About It) by Salim Ismail, Yuri van Geest


23andMe, 3D printing, Airbnb, Amazon Mechanical Turk, Amazon Web Services, augmented reality, autonomous vehicles, Baxter: Rethink Robotics, bioinformatics, bitcoin, Black Swan, blockchain, Burning Man, business intelligence, business process, call centre, chief data officer, Clayton Christensen, clean water, cloud computing, cognitive bias, collaborative consumption, collaborative economy, corporate social responsibility, cross-subsidies, crowdsourcing, cryptocurrency, dark matter, Dean Kamen, dematerialisation, discounted cash flows, distributed ledger, Edward Snowden, Elon Musk,, ethereum blockchain, Galaxy Zoo, game design, Google Glasses, Google Hangouts, Google X / Alphabet X, gravity well, hiring and firing, Hyperloop, industrial robot, Innovator's Dilemma, Internet of things, Iridium satellite, Isaac Newton, Jeff Bezos, Kevin Kelly, Kickstarter, knowledge worker, Kodak vs Instagram, Law of Accelerating Returns, Lean Startup, life extension, loose coupling, loss aversion, Lyft, Mark Zuckerberg, market design, means of production, minimum viable product, natural language processing, Netflix Prize, Network effects, new economy, Oculus Rift, offshore financial centre, p-value, PageRank, pattern recognition, Paul Graham, Peter H. Diamandis: Planetary Resources, Peter Thiel, prediction markets, profit motive, publish or perish, Ray Kurzweil, recommendation engine, RFID, ride hailing / ride sharing, risk tolerance, Ronald Coase, Second Machine Age, self-driving car, sharing economy, Silicon Valley, skunkworks, Skype, smart contracts, Snapchat, social software, software is eating the world, speech recognition, stealth mode startup, Stephen Hawking, Steve Jobs, subscription business, supply-chain management, TaskRabbit, telepresence, telepresence robot, Tony Hsieh, transaction costs, Tyler Cowen: Great Stagnation, urban planning, WikiLeaks, winner-take-all economy, X Prize, Y Combinator

Recommendation: Hire both internal and external Black Ops teams and have them establish startups with a combined goal of defeating one another and disrupting the mother ship. Copy Google[X] At a Singularity University event three years ago, Larry Page told Salim he’d heard good things about Brickhouse and asked whether Google should set up something similar. Salim’s recommendation was no; he believed it would only evoke the same immune system response he’d experienced at Yahoo. Page’s response was cryptic: “What would a Brickhouse for atoms look like?” he asked. We now know what he meant. In launching the Google[X] lab, Google has taken the classic skunkworks approach to new product development further than anyone ever imagined. Google[X] offers two fascinating new extensions to the traditional approach. First, it aims for moonshot-quality ideas (e.g., life extension, autonomous vehicles, Google Glass, smart contact lenses, Project Loon, etc.).

Dependencies or Prerequisites • Increase loyalty to ExO • Drives exponential growth • Validates new ideas, and learning • Allows agility and rapid implementation • Amplifies ideation • MTP • Engagement • Authentic and transparent leadership • Low threshold to participate • P2P value creation Algorithms In 2002, Google’s revenues were less than a half-billion dollars. Ten years later, its revenues had jumped 125x and the company was generating a half-billion dollars every three days. At the heart of this staggering growth was the PageRank algorithm, which ranks the popularity of web pages. (Google doesn’t gauge which page is better from a human perspective; its algorithms simply respond to the pages that deliver the most clicks.) Google isn’t alone. Today, the world is pretty much run on algorithms. From automotive anti-lock braking to Amazon’s recommendation engine; from dynamic pricing for airlines to predicting the success of upcoming Hollywood blockbusters; from writing news posts to air traffic control; from credit card fraud detection to the 2 percent of posts that Facebook shows a typical user—algorithms are everywhere in modern life.

By design and desire, our students would be the world’s top entrepreneurs, as well as executives from Fortune 500 companies. Our mission: to help people positively impact the lives of a billion people. The idea for SU came together at a Founding Conference hosted at NASA’s Ames Research Center in Silicon Valley in September 2008. What I remember most clearly from the event was an impromptu speech given by Google co-founder Larry Page near the end of the first day. Standing before about one hundred attendees, Page made an impassioned speech calling for this new university to focus on addressing the world’s biggest problems: “I now have a very simple metric I use: Are you working on something that can change the world? Yes or no? The answer for 99.99999 percent of people is ‘no.’ I think we need to be training people on how to change the world.


pages: 229 words: 67,869

So You've Been Publicly Shamed by Jon Ronson


4chan, AltaVista, Berlin Wall, Broken windows theory, Burning Man, Clive Stafford Smith, cognitive dissonance, Desert Island Discs, don't be evil, Donald Trump, Google Hangouts, illegal immigration, Menlo Park, PageRank, Ralph Nader, Rosa Parks, Silicon Valley, Skype, Steve Jobs, urban planning, WikiLeaks

Which for me would be the most fantastic website to chance upon, but for everyone else, less so. But then two students at Stanford, Larry Page and Sergey Brin, had their idea. Why not build a search engine that ranked websites by popularity instead? If someone is linking to your page, that’s one vote. A link, they figured, is like a citation - a nod of respect. If the page linking to your page has a lot of links into it, then that page counts for more votes. An esteemed person bestowing their admiration upon you is worth more than some loner doing the same. And that was it. They called their invention PageRank, after Larry Page, and as soon as they turned the algorithm on, us early searchers were spellbound. This was why Farukh needed to create LinkedIn and Tumblr and Twitter pages for Lindsey. They come with a built-in high PageRank. The Google algorithm prejudges them as well liked.

It was a producer at Radiolab - Tim Howard - who put me in touch with their former contributor, Jonah Lehrer. So my thanks to them for that too. The Murderer Next Door was published by Penguin in 2005. Some background information on the Zumba prostitute ring in Kennebunk came from the story ‘Modern-Day Puritans Wring Hands Over Zumba Madam’s List Of Shame’ by Patrik Jonsson, which was published in the Christian Science Monitor on 13 October 2012. For more on Larry Page and Sergey Brin’s days at Stanford, I recommend ‘The Birth of Google’ by John Battelle, which was published in Wired magazine in August 2005. All my information about the Stasi came from Anna Funder’s brilliant Stasiland: Stories from Behind the Berlin Wall, published by Granta in 2003 and by Harper Perennial in 2011. My research into the terrible story of Lindsay Armstrong took me to ‘She Couldn’t Take Any More’, which was written by Kirsty Scott and published in the Guardian on 2 August 2002.

It’s about the companies that dominate the data flows of the Internet.’ Now, I suddenly wondered. Did Google make money from the destruction of Justine Sacco? Could a figure be calculated? And so I joined forces with a number-crunching researcher, Solvej Krause, and began writing to economists and analysts and online ad revenue people. Some things were known. In December 2013, the month of Justine’s annihilation, 12.2 billion Google searches took place - a figure that made me feel less worried about the possibility that people were sitting inside Google headquarters personally judging me. Google’s ad revenue for that month was $4.69 billion. Which meant they made an average of $0.38 for every search query. Every time we typed anything into Google: 38 cents to Google. Of those 12.2 billion searches that December, 1.2 million were people searching the name Justine Sacco.


pages: 239 words: 56,531

The Secret War Between Downloading and Uploading: Tales of the Computer as Culture Machine by Peter Lunenfeld


Albert Einstein, Andrew Keen, Apple II, Berlin Wall, British Empire, Brownian motion, Buckminster Fuller, Burning Man, butterfly effect, computer age, crowdsourcing, cuban missile crisis, Dissolution of the Soviet Union, don't be evil, Douglas Engelbart, Dynabook, East Village, Edward Lorenz: Chaos theory, Fall of the Berlin Wall, Francis Fukuyama: the end of history, Frank Gehry, Grace Hopper, gravity well, Guggenheim Bilbao, Honoré de Balzac, Howard Rheingold, invention of movable type, Isaac Newton, Jacquard loom, Jacquard loom, Jane Jacobs, Jeff Bezos, John von Neumann, Mark Zuckerberg, Marshall McLuhan, Mercator projection, Mother of all demos, mutually assured destruction, Network effects, new economy, Norbert Wiener, PageRank, pattern recognition, planetary scale, Plutocrats, plutocrats, Post-materialism, post-materialism, Potemkin village, RFID, Richard Feynman, Richard Feynman, Richard Stallman, Robert X Cringely, Schrödinger's Cat, Search for Extraterrestrial Intelligence, SETI@home, Silicon Valley, Skype, social software, spaced repetition, Steve Ballmer, Steve Jobs, Steve Wozniak, Ted Nelson, the built environment, The Death and Life of Great American Cities, the medium is the message, Thomas L Friedman, Turing machine, Turing test, urban planning, urban renewal, Vannevar Bush, walkable city, Watson beat the top human players on Jeopardy!, William Shockley: the traitorous eight

In the corporation’s own words, from “Ten Things Google Has Found to Be True,” available at <>: PageRank™ “evaluates all of the sites linking to a web page and assigns them a value, based in part on the sites linking to them. By analyzing the full structure of the web, Google is able to determine which sites have been ‘voted’ the best sources of information by those most interested in the information they offer. This technique actually improves as the web gets bigger, as each new site is another point of information and another vote to be counted.” 30. The advent of the more cluttered and text-rich iGoogle pages signal a shift in this approach. As of 2008, the upper menu allowed a user to toggle between iGoogle and the perhaps unintentionally funny “Classic Home,” the strippeddown, original concept.

The Web 1.0 bubble laid much “dark fiber” across the world, as companies built far broader networks than they could ever use profitably, and after the crash, others have since benefited from that infrastructure to restructure the ways we conceive of and engage with the Internet. No one company has so palpably benefited and defined this shift than Google, the search algorithm that became a company and then a verb, as noted earlier. Google was an intentional misspelling of the word “googol,” the mathematical term for 174 HOW THE COMPUTER BECAME OUR CULTURE MACHINE a one followed by ten zeros. The company became a networked Ourobors, that creature from Greek mythology that devours its own tail and encircles the world. What cofounders Larry Page and Sergey Brin created was a relentless innovation and acquisition machine, powered by users and advertisers alike. They challenged the old masters, from Microsoft to Yahoo! and infiltrated everything from libraries to desktops, enmeshing everyone from pornographers to cartographers, from anticorporate bloggers to CEOs.

Building on the installed base of all these users as the new millennium looms, the Hosts— World Wide Web inventor Tim Berners-Lee and open-source guru Linus Torvalds—link these disparate personal machines into a huge web, concentrating on communication as much as technology, pushing participation to the next level. The sixth generation, that of the Searchers—named after but hardly limited to Larry Page and Sergey Brin of Google, the search algorithm that became a company and then a verb—aggregated so much information and so many experiences that they rendered simulation and participation ubiquitous. There are three default ways of telling the history of computing, and the interesting thing is that people rarely tend to blend the narratives. There is the technical and scientific history of computing, which is frankly the least understood and disseminated.


pages: 202 words: 59,883

Age of Context: Mobile, Sensors, Data and the Future of Privacy by Robert Scoble, Shel Israel


Albert Einstein, Apple II, augmented reality, call centre, Chelsea Manning, cloud computing, connected car, Edward Snowden, Elon Musk, factory automation, Filter Bubble, Google Earth, Google Glasses, Internet of things, job automation, Kickstarter, Mars Rover, Menlo Park, New Urbanism, PageRank, pattern recognition, RFID, ride hailing / ride sharing, Saturday Night Live, self-driving car, sensor fusion, Silicon Valley, Skype, smart grid, social graph, speech recognition, Steve Jobs, Steve Wozniak, Steven Levy, Tesla Model S, Tim Cook: Apple, urban planning, Zipcar

Not Another Day Scoble was the 107th person to receive a Google Glass prototype. He put them on and immediately started posting short notes on his social networks about his experience. He wore them when he went to Europe, making presentations at tech conferences and letting hundreds of people give his Glass device a quick try. After two weeks, he posted his first review to Google+, the default social network for Google Glass users, declaring “I’m never going to live another day without a wearable computer on my face.” To illustrate his point, his wife Maryam photographed him in the shower wearing his Glass. Some scorned the stunt. “If Google Glass fails, it is Robert Scoble’s fault,” bemoaned author-speaker Peter Shankman in a blog post. Larry Page, Google’s CEO, told Scoble in front of a large audience that he “did not appreciate” the shower photo.

From a contextual perspective, we hold Google in particularly high regard, but the real game-changing development is the gadget Scoble is wearing on our back cover—Google Glass. Chapter 2 Through the Glass, Looking Right now, most of us look at the people with Google Glass like the dudes who first walked around with the big brick phones. Amber Naslund, SideraWorks The first of them went to Sergey Brin, Larry Page, and Eric Schmidt. Brin, who runs Project Glass, the company’s much-touted digital eyewear program, has rarely been seen in public again without them. Before anyone outside the company could actually touch the device, or see the world through its perspective, the hoopla had begun and has not stopped. Neither has the controversy. Google Glass is the flagship contextual device.

So next-generation companies like Google started building networks of gigantic data centers that employed millions of computers to host all the data being produced. Storing this data was the smaller of two challenges. The bigger one was figuring out how everyday people could extract the little spoonfuls they wanted from inside the new unstructured big data mountains. Google again led the way. Until 2012, the essence of its data search engine was Page Rank, which used complex mathematical equations, or algorithms, to understand connections between web pages and then rank them by relevance in search results. Before Google, we got back haystacks when we searched for needles. Then we had to sift through pages and pages of possible answers to find the one right for us. Page Rank started to understand the rudimentary context of a search.


pages: 298 words: 81,200

Where Good Ideas Come from: The Natural History of Innovation by Steven Johnson


Ada Lovelace, Albert Einstein, Alfred Russel Wallace, carbon-based life, Cass Sunstein, cleantech, complexity theory, conceptual framework, cosmic microwave background, crowdsourcing, data acquisition, digital Maoism, discovery of DNA, Dmitri Mendeleev, double entry bookkeeping, double helix, Douglas Engelbart, Drosophila, Edmond Halley, Edward Lloyd's coffeehouse, Ernest Rutherford, Geoffrey West, Santa Fe Institute, greed is good, Hans Lippershey, Henri Poincaré, hive mind, Howard Rheingold, hypertext link, invention of air conditioning, invention of movable type, invention of the printing press, invention of the telephone, Isaac Newton, Islamic Golden Age, Jacquard loom, James Hargreaves, James Watt: steam engine, Jane Jacobs, Jaron Lanier, John Snow's cholera map, Joseph Schumpeter, Joseph-Marie Jacquard, Kevin Kelly, lone genius, Louis Daguerre, Louis Pasteur, Mason jar, Mercator projection, On the Revolutions of the Heavenly Spheres, online collectivism, packet switching, PageRank, patent troll, pattern recognition, price mechanism, profit motive, Ray Oldenburg, Richard Florida, Richard Thaler, Ronald Reagan, side project, Silicon Valley, silicon-based life, six sigma, Solar eclipse in 1919, spinning jenny, Steve Jobs, Steve Wozniak, Stewart Brand, The Death and Life of Great American Cities, The Great Good Place, The Wisdom of Crowds, Thomas Kuhn: the structure of scientific revolutions, transaction costs, urban planning

A platform adapted for scholarship was exapted for shopping, and sharing photos, and watching pornography—along with a thousand other uses that would have astounded Berners-Lee when he created his first HTML-based directories in the early nineties. When Sergey Brin and Larry Page decided to use links between Web pages as digital votes endorsing the content of those pages, they were exapting Berners-Lee’s original design: they took a trait adapted for navigation—the hypertext link—and used it as a vehicle for assessing quality. The result was PageRank, the original algorithm that made Google into the behemoth that it is today. The literary historian Franco Moretti has persuasively documented the role of exaptation in the evolution of the novel. An author conceives a new kind of narrative device to address a specific, local need in a work he or she is writing.

The most telling contrast between Google and the FBI lies in the story of Krishna Bharat, who now holds the title of “principal scientist” at Google. In the weeks after 9/11, Bharat found himself overwhelmed by the amount of news information available about the attacks and the imminent war in Afghanistan. It occurred to him that it would be useful to create a software tool that could organize all those stories into useful clusters of relevance, so that you could see at a glance all the latest stories from around the Web about the search for bin Laden, or the cleanup efforts at Ground Zero, or the Bush administration’s case for military retaliation. Bharat decided to use his 20-percent time to build a system called StoryRank—modeled after the original PageRank algorithm that Google’s search engine relies on—to organize and cluster news items.

Nemeth, Charlan NeoNurture Neurotransmitters Neutrons Newcomen, Thomas Newlands, John Newton, Isaac New York City Police Department (NYPD) Nicholas of Cusa Niepce, Joseph Nicephore Nike Corporation Nitroglycerine Nobel, Alfred Nobel Prize Noise error and Non-Euclidian geometry Nucleotides Nylon Obama, Barack Ocean tides Octants Ogburn, William Ogle, Richard Ohain, Hans von Oldenburg, Ray Oldham, Richard Olszewski, Stanislav Oncogenes Onnes, Heike Kamerlingh Oral contraceptives Orbits of comets of electron around nucleus of atom of man-made satellites planetary O’Reilly, Tim Orkut Osborn, Alex Otis, Elisha Otto, Nikolaus Oughtred, William Oxford University Oxygen, isolation of Ozzie, Ray Pacemakers Pacioli, Luca Page, Larry PageRanks Pan Am International Flight Academy Pangea Papin, Denis Paracelsus Parachutes Paradigm shifts Paris Maternité de University of Parkes, Alexander Pascal, Blaise Pasteur, Louis Pelouze, J. T. Pencils Pendulums Penicillin Penzias, Arno Periodic table Perkins, Jacob Pescara, Raúl Pateras Phoenix memo Photography Photosynthesis Piano Pi Sheng Planck, Max Plante, Gaston Plant respiration Plastic Platforms city as emergent generative open stacked Pliny the Elder Poe, Edgar Allan Poincaré, Henri Poindexter, Admiral John Polaris nuclear missiles Portland cement Pressure cookers Prestero, Timothy Priestley, Joseph Princeton University Printing press Procter & Gamble Proust, Joseph Ptolemaic astronomy Public Broadcasting System (PBS) Public Enemy Pulmonary respiration Pulsars Punch cards Pyramids Quantum mechanics Quarter-power laws Quick-Time Radioactivity Radiocarbon dating Radios RAM (random access memory) Ramón y Cajal, Santiago Rangiroa atoll Raytheon Corporation Reagan, Ronald Red Sea Refrigerators Relativity theory REM sleep Renaissance Reproductive strategies Research and development (R&D) labs Respiration plant pulmonary Restriction enzymes Revolvers Richter, Claudio Riess, Adam RNA Roberts, Richard J.


pages: 538 words: 147,612

All the Money in the World by Peter W. Bernstein


Albert Einstein, anti-communist, Berlin Wall, Bill Gates: Altair 8800, call centre, corporate governance, currency peg, David Brooks, Donald Trump, estate planning, family office, financial innovation, George Gilder, high net worth, invisible hand, Jeff Bezos, job automation, job-hopping, Long Term Capital Management, Martin Wolf, Maui Hawaii, means of production, Menlo Park, Mikhail Gorbachev, new economy, PageRank, Peter Singer: altruism, pez dispenser, popular electronics, Renaissance Technologies, Rod Stewart played at Stephen Schwarzman birthday party, Ronald Reagan, Sand Hill Road, school vouchers, Search for Extraterrestrial Intelligence, shareholder value, Silicon Valley, Silicon Valley startup, stem cell, Stephen Hawking, Steve Ballmer, Steve Jobs, Steve Wozniak, Thorstein Veblen, too big to fail, traveling salesman, urban planning, William Shockley: the traitorous eight, women in the workforce

One of the youngest people16 ever to make the list was Steve Jobs (number 49 on the 2006 Forbes 400 list), who at age twenty-seven boasted a $100 million fortune, thanks to his success with Apple Computer. Bill Gates was also one of the youngest; in 1986 he joined the list at age thirty, with $315 million. And then came the Google guys: In 1998 Google’s founders17, Larry Page (number 13 on the 2006 Forbes 400 list) and Sergey Brin (number 12 on the 2006 list), both then just in their mid-twenties, formally incorporated Google and hired their first employee while working on a graduate student project at Stanford University. This became the prototype for the phenomenally successful search engine. In 2004, a year after Google went public, Brin and Page joined the list, each with a fortune of $4 billion that has since ballooned to $14.1 billion and $14 billion, respectively. * * * At Home in Woodside Once they make their fortunes, many of the most successful Silicon Valley entrepreneurs head to the historic Silicon Valley town of Woodside.

Four years later, his fortune had increased to $4.9 billion. Sergey Brin Larry Page August 1, 1973 December 1,1972 Google 2004 The Google guys weren’t rich enough to make the Forbes list when they were 30, but at 31 and 32, respectively, they were each worth $4 billion. In 2006, Brin, 33, and Page, 34, were each worth $14.1 billion. * * * Before long, everyone at Stanford was Googling. And it was not much longer before the venture capitalists, many of whom were headquartered just a few miles up the road from Stanford on Sand Hill Road, came knocking with proposals in hand. One of them, Vinod Khosla50 of Kleiner Perkins Caufield & Byers, showed up with an offer from Excite, a company in which he was invested, to buy Google for $750,000. But Brin and Page held out for $1.6 million, and the deal fell through.

It worked so well that he took the company public in 1978 and grew it into the world’s largest overnight delivery service. He more than recouped his investment: FedEx brought him a personal net worth of $2.2 billion in 2006. Yahoo, the popular Web portal, and Google were both born at Stanford, under strikingly similar circumstances. Yahoo founders David Filo and Jerry Yang were Stanford graduate students when they designed a system for operating an Internet directory. The duo found the idea so compelling that they put their PhDs on hold in the mid-1990s to devote full attention to the Yahoo project. Now Filo and Yang are each billionaires twice over. Meanwhile, in 1998 Google cofounders Larry Page and Sergey Brin were working toward their PhDs in computer science at Stanford when they started running the now wildly popular search engine. The pair currently shares the company’s presidency; their 2006 net worth came to about $14 billion each.


pages: 903 words: 235,753

The Stack: On Software and Sovereignty by Benjamin H. Bratton


1960s counterculture, 3D printing, 4chan, Ada Lovelace, additive manufacturing, airport security, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, algorithmic trading, Amazon Mechanical Turk, Amazon Web Services, augmented reality, autonomous vehicles, Berlin Wall, bioinformatics, bitcoin, blockchain, Buckminster Fuller, Burning Man, call centre, carbon footprint, carbon-based life, Cass Sunstein, Celebration, Florida, charter city, clean water, cloud computing, connected car, corporate governance, crowdsourcing, cryptocurrency, dark matter, David Graeber, deglobalization, dematerialisation, disintermediation, distributed generation, don't be evil, Douglas Engelbart, Edward Snowden, Elon Musk,, Eratosthenes, ethereum blockchain, facts on the ground, Flash crash, Frank Gehry, Frederick Winslow Taylor, future of work, Georg Cantor, gig economy, global supply chain, Google Earth, Google Glasses, Guggenheim Bilbao, High speed trading, Hyperloop, illegal immigration, industrial robot, information retrieval, intermodal, Internet of things, invisible hand, Jacob Appelbaum, Jaron Lanier, Jony Ive, Julian Assange, Khan Academy, linked data, Mark Zuckerberg, market fundamentalism, Marshall McLuhan, Masdar, McMansion, means of production, megacity, megastructure, Menlo Park, Minecraft, Monroe Doctrine, Network effects, new economy, offshore financial centre, oil shale / tar sands, packet switching, PageRank, pattern recognition, peak oil, performance metric, personalized medicine, Peter Thiel, phenotype, place-making, planetary scale, RAND corporation, recommendation engine, reserve currency, RFID, Sand Hill Road, self-driving car, semantic web, sharing economy, Silicon Valley, Silicon Valley ideology, Slavoj Žižek, smart cities, smart grid, smart meter, social graph, software studies, South China Sea, sovereign wealth fund, special economic zone, spectrum auction, Startup school, statistical arbitrage, Steve Jobs, Steven Levy, Stewart Brand, Stuxnet, Superbowl ad, supply-chain management, supply-chain management software, TaskRabbit, the built environment, The Chicago School, the scientific method, Torches of Freedom, transaction costs, Turing complete, Turing machine, Turing test, universal basic income, urban planning, Vernor Vinge, Washington Consensus, web application, WikiLeaks, working poor, Y Combinator

In the longer term, a Google Energy may be a key player in the retail and wholesaling of renewables and the management of both consumer and municipality-facing smart grids. The company sees the pairing of bits and electrons as part of its vocation in ways that others simply cannot: Google Energy, Google Glass, Google Ideas, Google Car, Google Robotics, Google Earth. Google Space. Google Time. Google AI. Google Grossraum. Google Sovereignty, Google World. For the Google platform model for the Cloud Polis, these are all based on a grand vision encompassing (at least) information cosmopolitanism, search, advertising, physicalized information, and global infrastructure. Google (and now Alphabet) is a company founded on an algorithm.62 The original PageRank algorithm was Larry Page's attempt to organize the entire World Wide Web according to something like the peer citation models that quantify which academic papers are most influential and relevant.

It's not my interest to revisit or revitalize Cold War ideologies (or evangelize twentieth-century economic ideologies, as should be clear by now) and so will offer instead an update and correction of this joke.56 The most significant indirect contribution was not Apollo; rather it was Google. Is that still even a joke? The PageRank algorithm that formed the initial core of Search was based on “collective evaluation” as opposed to expert evaluation, which would be more expensive, slower and less reliable when dealing with massive amounts of unstructured and dynamic data. As described by Massimo Franceschet in his article “PageRank: Standing on the Shoulders of Giants,” which locates Google's algorithmic methods in the long and diverse history of econometric, sociometric, and bibliometric information evaluation and calculative techniques, “PageRank introduced an original notion of quality of information found on the Web: the collective intelligence of the Web, formed by the opinions of the millions of people that populate this universe, is exploited to determine the importance, and ultimately the quality, of that information.”

It will take some time for platform robotics to invent new infrastructural systems that are unique to how its capacities can be designed instead of merely automating what already exists. Toward this, the ambition of Google's Cloud Polis model already extends into the design and deployment of global infrastructures, and this interest is not a recent addition to the company's founding vision, but in many ways precedes it. When Larry Page was a student at the University of Michigan, he was fascinated with the idea of an Ann Arbor monorail that would make local transportation more efficient. Google's interest in transportation-as-platform extends, as discussed, into the Google driverless car project, now a focus of Sergey Brin's interest and attention (more on these in the User chapter). One corollary of governing the Cloud as a global infrastructure for the integration of multiple economies under the rubric of rationalized information is a reframing of existing infrastructures as local instances for the application of a new information systems engineering program.


pages: 396 words: 117,149

The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World by Pedro Domingos


3D printing, Albert Einstein, Amazon Mechanical Turk, Arthur Eddington, Benoit Mandelbrot, bioinformatics, Black Swan, Brownian motion, cellular automata, Claude Shannon: information theory, combinatorial explosion, computer vision, constrained optimization, correlation does not imply causation, crowdsourcing, Danny Hillis, data is the new oil, double helix, Douglas Hofstadter, Erik Brynjolfsson, experimental subject, Filter Bubble, future of work, global village, Google Glasses, Gödel, Escher, Bach, information retrieval, job automation, John Snow's cholera map, John von Neumann, Joseph Schumpeter, Kevin Kelly, lone genius, mandelbrot fractal, Mark Zuckerberg, Moneyball by Michael Lewis explains big data, Narrative Science, Nate Silver, natural language processing, Netflix Prize, Network effects, NP-complete, P = NP, PageRank, pattern recognition, phenotype, planetary scale, pre–internet, random walk, Ray Kurzweil, recommendation engine, Richard Feynman, Richard Feynman, Second Machine Age, self-driving car, Silicon Valley, speech recognition, statistical model, Stephen Hawking, Steven Levy, Steven Pinker, superintelligent machines, the scientific method, The Signal and the Noise by Nate Silver, theory of mind, transaction costs, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, white flight

“First links in the Markov chain,” by Brian Hayes (American Scientist, 2013), recounts Markov’s invention of the eponymous chains. “Large language models in machine translation,”* by Thorsten Brants et al. (Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, 2007), explains how Google Translate works. “The PageRank citation ranking: Bringing order to the Web,”* by Larry Page, Sergey Brin, Rajeev Motwani, and Terry Winograd (Stanford University technical report, 1998), describes the PageRank algorithm and its interpretation as a random walk over the web. Statistical Language Learning,* by Eugene Charniak (MIT Press, 1996), explains how hidden Markov models work. Statistical Methods for Speech Recognition,* by Fred Jelinek (MIT Press, 1997), describes their application to speech recognition.

The result is complete gibberish, of course, but if we let each letter depend on several previous letters instead of just one, it starts to sound more like the ramblings of a drunkard, locally coherent even if globally meaningless. Still not enough to pass the Turing test, but models like this are a key component of machine-translation systems, like Google Translate, which lets you see the whole web in English (or almost), regardless of the language the pages were originally written in. PageRank, the algorithm that gave rise to Google, is itself a Markov chain. Larry Page’s idea was that web pages with many incoming links are probably more important than pages with few, and links from important pages should themselves count for more. This sets up an infinite regress, but we can handle it with a Markov chain. Imagine a web surfer going from page to page by randomly following links: the states of this Markov chain are web pages instead of characters, making it a vastly larger problem, but the math is the same.

., 128 building blocks and, 128–129, 134 schemas, 129 survival of the fittest programs, 131–134 The Genetical Theory of Natural Selection (Fisher), 122 Genetic programming, 52, 131–133, 240, 244, 245, 252, 303–304 sex and, 134–137 Genetic Programming (Koza), 136 Genetic search, 241, 243, 249 Genome, poverty of, 27 Gentner, Dedre, 199 Ghani, Rayid, 17 The Ghost Map (Johnson), 182–183 Gibson, William, 289 Gift economy, 279 Gleevec, 84 Global Alliance for Genomics and Health, 261 Gödel, Escher, Bach (Hofstadter), 200 Good, I. J., 286 Google, 9, 44, 291 A/B testing and, 227 AdSense system, 160 communication with learner, 266–267 data gathering, 272 DeepMind and, 222 knowledge graph, 255 Master Algorithm and, 282 Naïve Bayes and, 152 PageRank and, 154, 305 problem of induction and, 61 relational learning and, 227–228 search results, 13 value of data, 274 value of learning algorithms, 10, 12 Google Brain network, 117 Google Translate, 154, 304 Gould, Stephen Jay, 127 GPS, 212–214, 216, 277 Gradient descent, 109–110, 171, 189, 193, 241, 243, 249, 252, 257–258 Grammars, formal, 36–37 Grandmother cell, perceptron and, 99–100 Graphical models, 240, 245–250 Graphical user interfaces, 236 The Guns of August (Tuchman), 178 Handwritten digit recognition, 189, 195 Hart, Peter, 185 Hawking, Stephen, 47, 283 Hawkins, Jeff, 28, 118 Hebb, Donald, 93, 94 Hebb’s rule, 93, 94, 95 Heckerman, David, 151–152, 159–160 Held-out data, accuracy of, 75–76 Help desks, 198 Hemingway, Ernest, 106 Heraclitus, 48 Hidden Markov model (HMM), 154–155, 159, 210, 305 Hierarchical structure, Markov logic network with, 256–257 Hill climbing, 135, 136, 169, 189, 252 Hillis, Danny, 135 Hinton, Geoff, 103, 104, 112, 115, 137, 139 The Hitchhiker’s Guide to the Galaxy (Adams), 130 HIV testing, Bayes’ theorem and, 147–148 HMM.


pages: 286 words: 82,065

Curation Nation by Rosenbaum, Steven


Amazon Mechanical Turk, Andrew Keen, barriers to entry, citizen journalism, cognitive dissonance, crowdsourcing, disintermediation,, future of journalism, Jason Scott:, means of production, PageRank, pattern recognition, postindustrial economy, pre–internet, Sand Hill Road, Silicon Valley, Skype, social graph, social web, Steve Jobs, Tony Hsieh, Yogi Berra

Today Google runs over one million servers in data centers around the world and processes over one billion search requests and 20 petabytes of user-generated data every day. Dewey was a human system, with a rigid digital classification. Google replaced human classification with digital discovery and a “black box” formula that ranked pages based on a complex and changing algorithm that let Google determine a page of data’s relative value for a particular search term. The concept of page rank was powerful, and it resulted in a taxonomy that created an entire industry of consultants and advisors who helped Web-content makers increase search engine optimization (SEO). That Larry Page, one of the Google cofounders, understood that the unit of measure for Web content was pages rather than domains, URLs, articles, authors, sources, or any other dimension helped to shape the Web for almost 10 years.

(See also Advertising) Brin, Sergey Brinkley, Alan Britain’s Got Talent (TV program) BROADCAST: New York (TV program) Brogan, Chris Broken, nature of Brooklyn Flea swap Burn Rate (Wolff) Business Insider Cable television Cablevision Calacanis, Jason CameraPlanet 9/11 Archive Carnegie, Andrew Carolla, Adam Carr, Paul Cars Direct CBS News CBS Radio CD-ROMs Chaos Scenarios, The (Garfield) Chen, Steve Citizens, of Curation Nation City Winery Civic leaders, of Curation Nation Clean, Steven Clinton, Bill CNBC CNN Cognitive Surplus (Shirky) Collins, Shawn Comcast Commerce, nature of Commission Junction Community antenna television (CATV) Community information Compete Consistency Consumer conversations Content creation by brands content entrepreneurs in machines versus humans in magazines and Content farms Content Generation Content strategy brands and cupcake analogy for curation in curation mix and emergence of nature of publishing and social media and stakeholders in Content Strategy (Halvorson) Contests Cooper, Frank Corradina, Linda Cost per acquisition (CPA) Cost per click (CPC) Cost per sale (CPS) Craigslist Creative artists Creative Commons Credit card information Crenshaw, Marshall Crowd Fusion Cruise ships Cuban, Mark Cult of the Amateur, The (Keen) Curated networks BlogHer Glam Media human repeaters in SB Nation Curation accidental as adding value aggregation versus applications of of consumer conversations content entrepreneurs and critics of curation economy curation manifesto defined history of human element of impact of legal issues in low-value moral issues in nature of need for origins of in shift from industrial to information age trend toward varieties of Curation nation CurationStation Curiosity Curse of the Mogul, The (Seave) Data mining Davola, Joe Daylife Dell Dell, Michael Demand Media Demby, Eric Democratization trend Denton, Nick Des Jardins, Jory Dewey, Melvil Dewey Decimal System DEWmocracy Diesel Digg Digital Millennium Copyright Act (DMCA) Digital natives Diller, Barry DJs Domain names Donohue, Joe Döpfner, Mathias Dorsey, Jack DoubleClick Drudge, Matt Drudge Report Dvorkin, Lewis DVR Dyson, Esther Earned media eBay Edelman PR Worldwide Editorial calendars Eliason, Frank Engadget Engage (Solis) Entertainment Weekly Entrepreneur magazine Etsy Facebook data mining and Facebook Connect Facebook Places Like button Open Graph origins of Fair use Fast Company Fett, Boba Film critics Finance First-person publishing Flickr Flipboard Flip cams Food critics Forbes magazine Ford, Henry Forry, Clinton Foursquare FOX News Frankfurt Kurnit Klein & Selz, PC Free (Anderson) Free content curation Friend-curated information F*cked Company Future of Privacy Forum Garfield, Bob Gartner, Gideon Gartner Group Gates, Bill Gawker Gelman, Lauren General Mills Generation C Gilt Group Giuliani, Rudy Gizmodo Glam Media Global Business Network Godin, Seth Google Google Ad Sense Google Affiliate Network Google Images Google Index Google Maps Google News Google Reader keyword tool page rank algorithm Gowalla Grub Street Hadden, Briton Hall, Colby Halvorson, Kristina Hampton University Ham radio Hansell, Saul Harvard University Here Comes Everybody (Shirky) Hewitt, Perry Heywood, Jamie Hileman, Kristen Hippeau, Eric Hirschhorn, Jason Hitwise Holt, Courtney Huffington, Arianna Huffington Post HUGE design Hulu Hurley, Chad Internet, launch of commercial iPad iPhone iTunes iVillage Jarvis, Jeff Jobs, Steve Journalism bionic financial machines versus humans in SB Nation and Joystiq Kaboodle Kaplan, Dina Kaplan, Philip Kashi Kasprzak, Michelle Kawaja, Terrence Keen, Andrew Keyword search terms Kinsley, Michael Kissinger, Henry Kurnit, Rick Kurnit, Scott Law of unintended consequences Lego Libraries Lincoln Center Library for the Performing Arts (New York City) Lindzon, Howard Linked economy LinkedIn Linked stories LinkShare Listenomics Livingston, Troy M.

Kurnit explains it this way: “What Congress was doing with the Digital Millennium Copyright Act is admitting that they can’t establish regulations. If Congress tried to sit down and write a copyright law for this medium, it would be out of date before it got enacted, and they essentially recognized that. So one could posit that Google’s response was, ‘We just want to proliferate content because we’re the curator; we don’t even care about being paid for content, we give away our content.’” Google is, after all, free. It gives away links to content that it aggregates via Web crawlers and curates using its page rank algorithm. Free content curation is at the core of Google’s business model. And by all accounts, it’s doing pretty well. Viacom is now appealing the judge’s ruling. But, in the industry broadly, the claim that aggregation is stealing seems to be fading into history. There are plenty of skirmishes about where the lines should be drawn, with folks like Nick Denton claiming that Huffington is stealing from Gawker, or the Newser versus The Wrap kerfuffle that I wrote about in chapter 3.


pages: 265 words: 74,000

The Numerati by Stephen Baker


Berlin Wall, Black Swan, business process, call centre, correlation does not imply causation, Drosophila, full employment, illegal immigration, index card, Isaac Newton, job automation, job satisfaction, McMansion, natural language processing, PageRank, personalized medicine, recommendation engine, RFID, Silicon Valley, Skype, statistical model, Watson beat the top human players on Jeopardy!

The Numerati are in control. They'll have their way with us. Wrong. Even the greatest and most powerful of the Numerati only master certain domains. Everywhere else, they'll be just like the rest of us: objects of study. Larry Page, for example, is a cofounder of Google and a titan in the world of the Numerati. His scientists are building machines to crunch hundreds of billions of our search queries and clicks, and to sell us, in neatly organized buckets, to advertisers. But when Josh Gotbaum's political program pours through consumer data and classifies millions of California voters, it plunks Larry Page into a bucket of Still Waters or Right Clicks. Whether they're patients with a genetic predisposition for blindness or supermarket shoppers with a sky-high tendency to throw a candy bar in the cart, the Numerati are sitting in the databases with the rest of us.

His point is that mathematicians model misunderstandings of the world, often using the data at hand instead of chasing down the hidden facts. He tells the story of a drunk looking for his keys on a dark night under a streetlight. He's looking for them under that lamp not necessarily because he dropped them there but because it's the only place with light. Later that afternoon, I'm sitting at an outdoor patio with Craig Silverstein, Google's chief technologist. He was the number-one employee at Google. The founders, Larry Page and Sergey Brin, hired him because neither one of them, for all their brilliant ideas, knew much about search engines. It's sunny and the wind is blowing the pages of my notebook, and I tell Silverstein the story about the drunk looking for his keys. He smiles. He's heard it many times before. He recalls a science fair in junior high, where his project featured lots of good data he'd come up with.

Spam blogs, or splogs, they called them. The purpose of splogs was to use the immense power of Google to cash in on the fast-growing field of blog advertising. Google offered a service called Adsense. If you signed up for it, Google would automatically place relevant advertisements onto your blog or Web page. If you wrote about weddings, the system would detect this and drop in ad banners, say, for flowers, gowns, and tuxedos. If a reader clicked the banner, the advertiser would pay Google a few cents, and Google would share the take with the blogger. For bloggers, it looked like a great way to bring in advertising revenue with absolutely no sales staff. Just click the box, blog energetically, and wait for the check from Google. But when I surveyed bloggers that spring and asked how they were faring, most of them complained.


pages: 459 words: 103,153

Adapt: Why Success Always Starts With Failure by Tim Harford


Andrew Wiles, banking crisis, Basel III, Berlin Wall, Bernie Madoff, Black Swan, car-free, carbon footprint, Cass Sunstein, charter city, Clayton Christensen, clean water, cloud computing, cognitive dissonance, complexity theory, corporate governance, correlation does not imply causation, credit crunch, Credit Default Swap, crowdsourcing, cuban missile crisis, Daniel Kahneman / Amos Tversky, Dava Sobel, Deep Water Horizon, Deng Xiaoping, double entry bookkeeping, Edmond Halley,, Erik Brynjolfsson, experimental subject, Fall of the Berlin Wall, Fermat's Last Theorem, Firefox, food miles, Gerolamo Cardano, global supply chain, Isaac Newton, Jane Jacobs, Jarndyce and Jarndyce, Jarndyce and Jarndyce, John Harrison: Longitude, knowledge worker, loose coupling, Martin Wolf, Menlo Park, Mikhail Gorbachev, mutually assured destruction, Netflix Prize, New Urbanism, Nick Leeson, PageRank, Piper Alpha, profit motive, Richard Florida, Richard Thaler, rolodex, Shenzhen was a fishing village, Silicon Valley, Silicon Valley startup, South China Sea, special economic zone, spectrum auction, Steve Jobs, supply-chain management, the market place, The Wisdom of Crowds, too big to fail, trade route, Tyler Cowen: Great Stagnation, web application, X Prize

Yet it was hard to forget seeing peer monitoring in action: the instant correction of a problem, no matter how small and no matter what the hierarchical relationship might be between head of safety and tea lady. 4 Google’s corporate strategy: have no corporate strategy At Hinkley Point, the key priority is ensuring that the power station operates exactly as planned, without deviation. But at other companies, the challenge is to do something new every day, and nowhere is this truer than at Google. The company’s CEO Eric Schmidt had a surprise when he walked into Larry Page’s office in 2002. Page is the co-creator of Google and the man who gave his name to the idea at the company’s foundation: its PageRank search algorithm. But Page had something rather different to show Schmidt: a machine he’d built himself which cut off the bindings of books and then scanned their pages into a digital format. Page had been trying to figure out whether it might be possible for Google to scan the world’s books into searchable form.

If any company can be said to embrace trying new things in the expectation that many will fail, it is Google. Marissa Mayer, the vice-president who helped Larry Page bodge together the first book scanner, says that 80 per cent of Google’s products will fail – but that doesn’t matter, because people will remember the ones that stick. Fair enough: Google’s image seems to be untarnished by the indifferent performances of Knol, a Google service vaguely similar to Wikipedia which didn’t seem to catch on; or SearchMash, a testbed for alternative Google search products which was labelled ‘Google’s Worst Ever Product’ by one search expert and has now been discontinued. According to the influential TechRepublic website, two of the five worst technology products of 2009 came from Google – and they were major Google products at that, Google Wave and the Android 1.0 operating system for mobile phones.

Rather than instructing an intern to rig something up, or commissioning analysis from a consulting firm, he teamed up with Marissa Mayer, a Google vice-president, to see how fast two people could produce an image of a 300-page book. Armed with a plywood frame, a pair of clamps, a metronome and a digital camera, two of Google’s most senior staff tried out the project themselves. (The book went from paper to pixels in forty minutes.) Larry Page regarded the time he devoted to the project not as something he could do because he was Google’s founder and could do whatever he wanted, but as something to which he was entitled because every engineer at Google had the same deal. Famous, Google has a ‘20 per cent time’ policy: any engineer (and some other employees) is allowed to spend one fifth of his or her time on any project that seems worthwhile. Google News, Google Suggest, Adsense and the social networking site Orkut are all projects that emerged from these personal projects, along with half of all Google’s successful products – and an astonishing portfolio of failures.


pages: 606 words: 157,120

To Save Everything, Click Here: The Folly of Technological Solutionism by Evgeny Morozov


3D printing, algorithmic trading, Amazon Mechanical Turk, Andrew Keen, augmented reality, Automated Insights, Berlin Wall, big data - Walmart - Pop Tarts, Buckminster Fuller, call centre, carbon footprint, Cass Sunstein, choice architecture, citizen journalism, cloud computing, cognitive bias, crowdsourcing, data acquisition, Dava Sobel, disintermediation, East Village,, Fall of the Berlin Wall, Filter Bubble, Firefox, Francis Fukuyama: the end of history, frictionless, future of journalism, game design, Gary Taubes, Google Glasses, illegal immigration, income inequality, invention of the printing press, Jane Jacobs, Jean Tirole, Jeff Bezos, jimmy wales, Julian Assange, Kevin Kelly, Kickstarter, license plate recognition, lone genius, Louis Pasteur, Mark Zuckerberg, market fundamentalism, Marshall McLuhan, Narrative Science, Nicholas Carr, packet switching, PageRank, Paul Graham, Peter Singer: altruism, Peter Thiel,, placebo effect, pre–internet, Ray Kurzweil, recommendation engine, Richard Thaler, Ronald Coase, Rosa Parks, self-driving car, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, Skype, Slavoj Žižek, smart meter, social graph, social web, stakhanovite, Steve Jobs, Steven Levy, Stuxnet, technoutopianism, the built environment, The Chicago School, The Death and Life of Great American Cities, the medium is the message, The Nature of the Firm, the scientific method, The Wisdom of Crowds, Thomas Kuhn: the structure of scientific revolutions, Thomas L Friedman, transaction costs, urban decay, urban planning, urban sprawl, Vannevar Bush, WikiLeaks

See National Endowment for the Arts Nelson, Mark Networks News industry and international news, and technological intervention Newspaper industry Newspapers Newton, Sir Isaac Nietzsche, Friedrich Noise-abatement campaigns Norms adaptation of, and technology revision of, and technological enforcement Nostalgia Noveck, Beth Nuclear age Nudging Numeric imagination Nussbaum, Martha Nutrition, and quantification Nyberg, David Oakeshott, Michael Obama, Barack and open government and the Pirates Obesity Object-recognition technology Occupy Wall Street On-line shopping O’Neill, Onora Online data, longevity of Online learning Online profiling Online/offline divide Open government Open-government data Open Handset Alliance Openness and Google See also Transparency Openness fundamentalism Originality Ortega y Gasset, José Otter, Chris Page, Larry PageRank (Google) Palantir Paparazzi Pariser, Eli Parking system (California) Parks, Rosa Pasteur, Louis Paul, Ron Payer, Peter Payne, Brent PayPal Paywall Peppet, Scott Perfection, and situational crime prevention Personal analytics Perversity-futility-jeopardy triad Peters, John Durham Pharmaceutical industry Philips company Philosophy vs. psychology PhotoDNA (Microsoft) Pirates Places (Google) Plant watering system Play, and games Pocket registrator Political backpacking Political change Political information Political parties Political Reform Act of 1974 (California) Politics and ambiguity and consumerism and fact checking and gamification and hypocrisy and imperfection and mendacity and networks and the Pirates and proxy voting and technocracy and technology and technorationalists and technoscapists and transparency and two-party system Politwoops Poole, Steven Populism vs. expertise Post, David Potholes, and smartphones Power, Michael Power and control Predictive policing dangers of and Facebook and social networks, surveillance of See also Crime prevention PredPol Print culture Printing press as agent of change and the Internet Privacy and digital natives Internet and online data, longevity of and self-tracking and tracking See also Self-disclosure Problem solving Professors Profiling, online Project Glass goggles (Google) Projectors Proposition 8 (California) Protestant Reformation Proust, Marcel Proxy voting Pseudo-crime Psychology vs. philosophy Public broadcasting Public engagement Public information Public information databases Public life, and memes Public relations industry Publishing industry, and gatekeepers Putin, Vladimir Quantification critique of deficiency in and education ethics of in the future and marketing budgets and narrative imagination vs. numeric imagination and needs/desires/necessities and nutrition and water and energy consumption feedback devices and water and energy consumption metering systems Quantified Self movement and authenticity beginning of and correlations and hunches and narrative imagination See also Lifelogging; Self-tracking Quick Response Codes Racial discrimination Radical agenda Radio erratic appliance Rand, Ayn Rapid Content Analysis for Law Enforcement Rate My Professors (website) RateMyDrive Rational-choice theory (RCT) RCT.

See Perversity-futility-jeopardy triad Galileo Galison, Peter Galton, Francis Gambling addiction Game mechanics Games and gamification and humanitarianism and smartphones vs. reality Gamification and adversarial design and degrading environment, enjoyment in and efficiency vs. inefficiency and games and games vs. reality literature and motivation and rewards vs. citizenship Gardner, James Garland, David Gasto Público Bahiense (website) Gatekeepers Gates, Bill Gates, Kelly Gawker Gender discrimination Generativity theory Genetic engineering Gertner, Joe Ghonim, Wael Gillespie, Tarleton Global Integrity Godin, Benoit Google AdSense and advertising and algorithms and algorithms, and democracy and algorithms, neutrality and objectivity of and badges and citizenship and content business and ethics GPS-enabled Android phones and Huffington Post and information organization and legal challenges and mirror imagery and openness PageRank Places and predictive policing and privacy Project Glass goggles and scientific credentials and self-driving cars values and WiFi networks and Zagat Google+ Google Autocomplete Google Buzz Google News and badges Google Scholar Government and networks role of Government, US, and WikiLeaks GPS driving data GPS-enabled Android phones (Google) Grafton, Anthony Graham, Paul Grant, Ruth Green, Donald Green, Shane Greenwald, Glenn Guernica Gutenberg, Johannes Gutshot-detection systems Hanrahan, Nancy Harvey, David Hayek, Friedrich Heald, David Health and gamification monitoring device Heller, Nathaniel Hibbing, John Hierarchies, and networks Hieronymi, Pamela Highlighting and shading Hildebrandt, Mireille Hill, Kashmir Hirschman, Albert Historians, and Internet debate History as irrelevant of technology Hoffman, Reid Holiday, Ryan Holocaust Horkheimer, Max Howard Dean for Iowa Game Huffington Post, The Humanitarianism, and games Hypocrisy Illich, Ivan Image-recognition software Imperfection Impermium Incentives Information cascades theory Information consumption self-tracking of Information emperors Information industries and government history of Information organization Information-processing imperative Information reductionism Information technology InfoWorld (website) Innovation and justice and technology unintended consequences of Innovation talk Institutions, and networks Intel Intermediaries.

The technician who believes that he has arrived at a full understanding of a question is always surprised and often grieved when he encounters opposition to his theories; inevitably, he is tempted to attribute this to ignorance or ill-will.” When the benefits of one’s solutions look so obvious, how can people fail to recognize them? Meynaud’s speculation that technocrats believe they have superior access to truth, seeing those who disagree with them as ignoramuses, can help us explain, among other things, Google’s befuddlement over the massive opposition to its quest to scan all the world’s books. Here is how Douglas Edwards, Google’s brand manager between 1999 and 2005, describes it: “For Larry [Page] and Sergey [Brin], truth was often self-evident and unassailable. The inability of others to recognize truth did not make it any less absolute. Obviously, it’s a good idea to make as much information as possible available to as many people as possible. Obviously, a lot of valuable information is in books.


pages: 320 words: 87,853

The Black Box Society: The Secret Algorithms That Control Money and Information by Frank Pasquale


Affordable Care Act / Obamacare, algorithmic trading, Amazon Mechanical Turk, asset-backed security, Atul Gawande, bank run, barriers to entry, Berlin Wall, Bernie Madoff, Black Swan, bonus culture, Brian Krebs, call centre, Capital in the Twenty-First Century by Thomas Piketty, Chelsea Manning, cloud computing, collateralized debt obligation, corporate governance, Credit Default Swap, credit default swaps / collateralized debt obligations, crowdsourcing, cryptocurrency, Debian, don't be evil, Edward Snowden,, Fall of the Berlin Wall, Filter Bubble, financial innovation, Flash crash, full employment, Goldman Sachs: Vampire Squid, Google Earth, Hernando de Soto, High speed trading, hiring and firing, housing crisis, informal economy, information retrieval, interest rate swap, Internet of things, invisible hand, Jaron Lanier, Jeff Bezos, job automation, Julian Assange, Kevin Kelly, knowledge worker, Kodak vs Instagram, kremlinology, late fees, London Interbank Offered Rate, London Whale, Mark Zuckerberg, mobile money, moral hazard, new economy, Nicholas Carr, offshore financial centre, PageRank, pattern recognition, precariat, profit maximization, profit motive, quantitative easing, race to the bottom, recommendation engine, regulatory arbitrage, risk-adjusted returns, search engine result page, shareholder value, Silicon Valley, Snapchat, Spread Networks laid a new fibre optics cable between New York and Chicago, statistical arbitrage, statistical model, Steven Levy, the scientific method, too big to fail, transaction costs, two-sided market, universal basic income, Upton Sinclair, value at risk, WikiLeaks

Mark Walters, “How Does Google Rank Websites?” SEOmark. Available at /how-does-google-rank-websites/. Amy N. Langville and Carl D. Meyer, “Deeper inside PageRank,” Internet Mathematics, 1 (2004): 335–380. Langville and Meyer, Google’s PageRank and Beyond. 41. Siva Vaidhyanathan, The Googlization of Everything (And Why We Should Worry) (Berkeley: University of California Press, 2010). 42. Ibid. 43. “Trust Us—We’re Geniuses and You’re Not—The Arrival of Google,” Searchless in Paradise (blog), February 19, 2013, /page/2/. Google’s current mission statement is “to organize the world’s information and make it universally accessible and useful.” “Company Overview,” Google. Available at /about /company/. For Google’s recent dominance, see Matt McGee, “Google Now #1 Search Engine In Czech Republic; 5 Countries to Go for Global Domination” (Jan. 2011).

It rates sites on relevance and on importance. The more web pages link to a given page, the more authoritative Google deems it. (For those who need to connect to a page but don’t want to promote it, Google promises not to count links that include a “rel:nofollow” tag.) The voting is weighted; web pages that are themselves linked to by many other pages have more authority than unconnected ones. This is the core of the patented “PageRank” method behind Google’s success.36 PageRank’s hybrid of egalitarianism (anyone can link) and elitism (some links count more than others) both reflected and inspired powerful modes of ordering web content.37 It also caused new problems. The more Google revealed about its ranking algorithms, the easier it was to manipulate them.38 Thus THE HIDDEN LOGICS OF SEARCH 65 began the endless cat-and-mouse game of “search engine optimization,” and with it the rush to methodological secrecy that makes search the black box business that it is.

But the truth of the great Internet firms is closer to the oligopolistic dominance of AT&T, Verizon, and Comcast. In 2008, I testified before a congressional committee about Google’s market power. Just about every representative who questioned me assumed that a clique of twenty-somethings could at that very moment be developing an alternative. They didn’t know much about the Internet, but they knew that Larry Page and Sergey Brin had risen from grad students to billionaires by building a corporate colossus out of old servers and ingenuity. In their imaginations, Google’s own rags-to-riches story foreshadowed its eventual displacement.135 Even law professors who ought to know better buy into this myth. “No one’s even going to care about Google in five years!” one heatedly told me. That was six years ago. Too many still believe that the digital economy is by its nature open, competitive, and subject to the disruption that it preaches for other fields.136 82 THE BLACK BOX SOCIETY But how realistic is this?


pages: 519 words: 102,669

Programming Collective Intelligence by Toby Segaran


correlation coefficient, Debian,, Firefox, full text search, information retrieval, PageRank, prediction markets, recommendation engine, slashdot, web application

It's also possible that people are more interested in results that have attracted the attention of very popular sites. Next, you'll see how to make links from popular pages worth more in calculating rankings. The PageRank Algorithm The PageRank algorithm was invented by the founders of Google, and variations on the idea are now used by all the large search engines. This algorithm assigns every page a score that indicates how important that page is. The importance of the page is calculated from the importance of all the other pages that link to it and from the number of links each of the other pages has. Tip In theory, PageRank (named after one of its inventors, Larry Page) calculates the probability that someone randomly clicking on links will arrive at a certain page. The more inbound links the page has from other popular pages, the more likely it is that someone will end up there purely by chance.

The nature of many machine-learning algorithms is that they can continue to learn as new information arrives. Real-Life Examples There are many sites on the Internet currently collecting data from many different people and using machine learning and statistical methods to benefit from it. Google is likely the largest effort—it not only uses web links to rank pages, but it constantly gathers information on when advertisements are clicked by different users, which allows Google to target the advertising more effectively. In Chapter 4 you'll learn about search engines and the PageRank algorithm, an important part of Google's ranking system. Other examples include web sites with recommendation systems. Sites like Amazon and Netflix use information about the things people buy or rent to determine which people or items are similar to one another, and then make recommendations based on purchase history.

The solution is to set all the PageRanks to an initial arbitrary value (the code will use 1.0, but the actual value doesn't make any difference), and repeat the calculation over several iterations. After each iteration, the PageRank for each page gets closer to its true PageRank value. The number of iterations needed varies with the number of pages, but in the small set you're working with, 20 should be sufficient. Because the PageRank is time-consuming to calculate and stays the same no matter what the query is, you'll be creating a function that precomputes the PageRank for every URL and stores it in a table. This function will recalculate all the PageRanks every time it is run. Add this function to the crawler class: def calculatepagerank(self,iterations=20): # clear out the current PageRank tables self.con.execute('drop table if exists pagerank') self.con.execute('create table pagerank(urlid primary key,score)') # initialize every url with a PageRank of 1 self.con.execute('insert into pagerank select rowid, 1.0 from urllist') self.dbcommit( ) for i in range(iterations): print "Iteration %d" % (i) for (urlid,) in self.con.execute('select rowid from urllist'): pr=0.15 # Loop through all the pages that link to this one for (linker,) in self.con.execute( 'select distinct fromid from link where toid=%d' % urlid): # Get the PageRank of the linker linkingpr=self.con.execute( 'select score from pagerank where urlid=%d' % linker).fetchone( )[0] # Get the total number of links from the linker linkingcount=self.con.execute( 'select count(*) from link where fromid=%d' % linker).fetchone( )[0]pr+=0.85*(linkingpr/linkingcount) self.con.execute( 'update pagerank set score=%f where urlid=%d' % (pr,urlid)) self.dbcommit( ) This function initially sets the PageRank of every page to 1.0.


pages: 370 words: 105,085

Joel on Software by Joel Spolsky


barriers to entry,, George Gilder, index card, Jeff Bezos, knowledge worker, Metcalfe's law, Network effects, new economy, PageRank, Paul Graham, profit motive, Robert X Cringely, shareholder value, Silicon Valley, Silicon Valley startup, six sigma, slashdot, Steve Ballmer, Steve Jobs, the scientific method, thinkpad, VA Linux, web application

In defense of the computer scientists, this is something nobody even noticed until they starting indexing gigantic corpora the size of the Internet. But somebody noticed. Larry Page and Sergey Brin over at Google realized that ranking the pages in the right order was more important than grabbing every possible page. Their PageRank algorithm1 is a great way to sort the zillions of results so that the one you want is probably in the top ten. Indeed, search for Joel on Software on Google and you'll see that it comes up first. On Altavista, it's not even on the first five pages, after which I gave up looking for it. __________ 1. See Antialiased Text Antialiasing was invented way back in 1972 at the Architecture Machine Group of MIT, which was later incorporated into the famous Media Lab.

Web sites become flexible services that can interact, and exchange and leverage each other's data. That's a "feature" of this exciting .NET architecture. The fact that it is so broad, vague, and high level that it doesn't mean anything at all doesn't seem to be bothering anyone. Or how about: Microsoft .NET makes it possible to find services and people with which to interact. Oh, joy! Five years after Altavista went live, and two years after Larry Page and Sergei Brin actually invented a radically better search engine (Google), Microsoft is pretending like there's no way to search on the Internet and they're going to solve this problem for us. The whole document is exactly like that. There are two things going on here. Microsoft has some great thinkers. When great thinkers think about problems, they start to see patterns. They look at the problem of people sending each other word-processor files, and then they look at the problem of people sending each other spreadsheets, and they realize that there's a general pattern: sending files.

Not-Invented-Here syndrome–2nd Old New Thing weblog Oliver, Jamie on-site, in-person interviews one step builds online discussion forums open issues in functional specifications–2nd open source software–2nd, 3rd–4th HP IBM Java–2nd Netscape–2nd Sun–2nd operating systems APIs. See APIs history–2nd opportunity cost options, stock expensing–2nd value of organic business model original estimates in software schedules–2nd OS X output from successful programs oversimplifying condescension own products, using–2nd P Page, Larry PageRank algorithm Palmerston, Lord, quote by paper companies paper prototyping–2nd Pascal language productivity in–2nd strings in passionate employees, looking for Paterson, Tim patterns pay incentive–2nd, 3rd programmer service PC-DOS operating system history licensing Peopleware–2nd, 3rd performance measurement system based string concatenation–2nd XML data with SELECT statements–2nd performance reviews–2nd PhDs as employees phone screening pictures in functional specifications Pipeline online service–2nd pivot tables plain text plane travel planning in Extreme Programming, 2nd functional specs in.


The Economic Singularity: Artificial intelligence and the death of capitalism by Calum Chace


3D printing, additive manufacturing, agricultural Revolution, AI winter, Airbnb, artificial general intelligence, augmented reality, autonomous vehicles, banking crisis, Baxter: Rethink Robotics, Berlin Wall, Bernie Sanders, bitcoin, blockchain, call centre, Chris Urmson, congestion charging, credit crunch, David Ricardo: comparative advantage, Elon Musk,, Erik Brynjolfsson, Flynn Effect, full employment, future of work, gender pay gap, gig economy, Google Glasses, Google X / Alphabet X, income inequality, industrial robot, Internet of things, invention of the telephone, invisible hand, James Watt: steam engine, Jaron Lanier, Jeff Bezos, job automation, John Maynard Keynes: technological unemployment, John von Neumann, Kevin Kelly, knowledge worker, lump of labour, Lyft, Mark Zuckerberg, Martin Wolf, McJob, means of production, Milgram experiment, Narrative Science, natural language processing, new economy, Occupy movement, Oculus Rift, PageRank, pattern recognition, post scarcity, post-industrial society, precariat, prediction markets, QWERTY keyboard, railway mania, RAND corporation, Ray Kurzweil, RFID, Rodney Brooks, Satoshi Nakamoto, Second Machine Age, self-driving car, sharing economy, Silicon Valley, Skype, software is eating the world, speech recognition, Stephen Hawking, Steve Jobs, TaskRabbit, technological singularity, Thomas Malthus, transaction costs, Tyler Cowen: Great Stagnation, Uber for X, universal basic income, Vernor Vinge, working-age population, Y Combinator, young professional

The pages were indexed by an algorithm called PageRank, which scored each web page according to how many other web pages linked to it. This algorithm, while ingenious, was not itself an example of artificial intelligence. Over time, Google Search has become unquestionably AI-powered. In August 2013, Google executed a major update of its search function by introducing Hummingbird, which enables the service to respond appropriately to questions phrased in natural language, such as, “what's the quickest route to Australia?”[lxxix] It combines AI techniques of natural language processing with colossal information resources (including Google's own Knowledge Graph, and of course Wikipedia) to analyse the context of the search query and make the response more relevant. PageRank wasn't dropped, but instead became just one of the 200 or so techniques that are now deployed to provide answers.

In February 2015 he told a CBS interviewer “Eventually I think most jobs will be replaced, like 75-80% of people are probably not going to work for a living... There are a few people starting to talk about it.”[xlv] Federico Pistono Federico Pistono is a young Italian lecturer and social entrepreneur. He attracted considerable attention with his 2012 book “Robots Will Steal Your Job, But That's OK”. A range of eminent people, including Google's Larry Page, were drawn to its optimistic and discursive style. (Google re-named itself Alphabet in October 2015, but most people still call it Google, so in this book I’ll mostly follow that convention.) After making a forceful case that future automation will render most people unemployed, Pistono argues that there is no need to worry. Much of the book is taken up with musing on the nature of happiness – the word features in the titles of a quarter of its chapters.

As recently as the late 20th century, knowledge workers could spend hours each day looking for information. Today, less than twenty years after Google was incorporated in 1998, we have something close to omniscience. At the press of a button or two, you can access pretty much any knowledge that humans have ever recorded. To our great-grandparents, this would surely have been more astonishing than flying cars. (Some people are so impressed by Google Search that they have established a Church of Google, and offer nine proofs that Google is God, including its omnipresence, near-onmiscience, potential immortality, and responses to prayer.[lxxvii] Admittedly, at the time of writing, there are only 427 registered devotees, or “readers”, at their meeting-place, a page on the internet community site Reddit.[lxxviii]) In the early days, Google Search was achieved by indexing large amounts of the web with software agents called crawlers, or spiders.


Remix: Making Art and Commerce Thrive in the Hybrid Economy by Lawrence Lessig


Amazon Web Services, Andrew Keen, Benjamin Mako Hill, Berlin Wall, Bernie Sanders, Brewster Kahle, Cass Sunstein, collaborative editing, disintermediation, don't be evil, Erik Brynjolfsson, Internet Archive, invisible hand, Jeff Bezos, jimmy wales, Kevin Kelly, late fees, Netflix Prize, Network effects, new economy, optical character recognition, PageRank, recommendation engine, revision control, Richard Stallman, Ronald Coase, Saturday Night Live, SETI@home, sharing economy, Silicon Valley, Skype, slashdot, Steve Jobs, The Nature of the Firm, thinkpad, transaction costs, VA Linux

That innovation rewards others and Amazon both. 80706 i-xxiv 001-328 r4nk.indd 126 8/12/08 1:55:16 AM T W O EC O NO MIE S: C O MMERC I A L A ND SH A RING 127 Google Without a doubt, the most famous example of Internet success is Google. Founded at Stanford by two students (the first URL was, the company radically improved the effectiveness of Internet searches. Rather than selling placement (which can often corrupt the results) or relying upon humans to index (which would be impossible given the vast scale of the Internet), the first Google algorithms ordered search results based upon how the Net linked to the results—a process called PageRank, referring not to “page” as in Web page, but “Page” as in Larry Page, Google cofounder and developer of the technique.11 If many Web sites linked to a particular site, that site would be ranked higher in the returned list than another Web site that had few links. Google thus built upon the knowledge the Web revealed to deliver back to the Web a product of extraordinary value.

., “,”, available at link #57 (last visited July 31, 2007). These numbers reflect sales only. According to reports, Amazon’s net deficit is still high— $2 billion as of 2005. 10. Ibid., available at link #58 (last visited July 31, 2007). 11. Wikipedia contributors, “Larry Page,” Wikipedia: The Free Encyclopedia, available at link #59 (last visited July 31, 2007). 12. Verne Kopytoff, “Google Shares Top $400: Search Engine No. 3 in Market Cap Among Firms in Bay Area,” San Francisco Chronicle, November 18, 2005; Yahoo! Finance, “GOOG: Key Statistics for Google Inc,” Capital IQ, available at link #60 (last visited July 5, 2007). 13. Keen, The Cult of the Amateur, 135. 14. The point was made long before by Nicholas Negroponte. “A best-seller in 1990, Nicholas Negroponte’s Being Digital drew a sharp contrast between ‘passive old media’ and ‘interactive new media,’ predicting the collapse of broadcast networks in favor of an era of narrowcasting and niche media on demand: ‘What will happen to broadcast television over the next five years is so phenomenal that it’s difficult to comprehend.’ ” Jenkins, Convergence Culture, 5. 15.

But it is false if it suggests that da Vinci wasn’t responsible for the great value the Mona Lisa is. Like Amazon, Google also offers its tools as a platform for others to build upon. We’ll see this more below as we consider Google Application Programming Interfaces (APIs). And more successfully than anyone, Google has built an advertising business into the heart of technology. Web pages can be served with very smartly selected ads; users can buy searches in Google to promote their own products. The complete range of Google products is vast. But one feature of all of them is central to the argument I want to make here. Practically everything Google offers helps Google build an extraordinary database of knowledge about what people want, and how those wants relate to the Web. Every click you make in the Google universe adds to that database. With each click, Google gets smarter. Three Keys to These Three Successes These familiar stories of Internet success reveal three keys to success in this digital economy.


pages: 298 words: 43,745

Understanding Sponsored Search: Core Elements of Keyword Advertising by Jim Jansen


AltaVista, barriers to entry, Black Swan, bounce rate, business intelligence, butterfly effect, call centre, Claude Shannon: information theory, complexity theory, correlation does not imply causation,, first-price auction, information retrieval, inventory management, life extension, linear programming, megacity, Nash equilibrium, Network effects, PageRank, place-making, price mechanism, psychological pricing, random walk, Schrödinger's Cat, sealed-bid auction, search engine result page, second-price auction, second-price sealed-bid, sentiment analysis, social web, software as a service, stochastic process, telemarketer, the market place, The Present Situation in Quantum Mechanics, the scientific method, The Wisdom of Crowds, Vickrey auction, yield management

Page jacking: theft of a page from the original site and publication of a copy (or nearcopy) at another site (Source: Marketing (see Chapter 2 model). Page request: the opportunity for an HTML document to appear on a browser window as a direct result of a user’s interaction with a Web site (Source: IAB) (see Chapter 2 model). Page view: request to load a single HTML page (Source: Marketing (see Chapter 2 model). PageRank (PR): the Google technology developed at Stanford University for placing importance on pages and Web sites. At one point, PageRank (PR) was a major factor in rankings. Today it is one of hundreds of factors in the algorithm that determines a page’s rankings (Source: SEMPO) (see Chapter 2 model). Paid Inclusion: refers to the process of paying a fee to a search engine in order to be included in that search engine or directory. Also known as guaranteed inclusion.

Hart does a statistical analysis using attributes of the people on the list, noting that there is a clustering by location and time. Hart credits this clustering to particular societies’ ability to communicate more effectively. This increased ability to communicate has a positive effect on the society’s ability to innovate. With this viewpoint, sponsored search (as the economy engine of the Web) is a significant social enhancer. Given that Google was the search platform that really took the sponsoredsearch concept and made it the economic engine of the Web, Sergey Brin and Larry Page really deserve credit for shaping the Web and Internet as we know it. Their efforts were most influential. By the way, there were two other interesting correlations that Hart discovered with the people on his list: There were high occurrences of gout and no living descendents. Nothing to do with sponsored search, but I found it interesting.

Berlin: Springer, pp. 177–206. [11] Voge, K. and McCaffrey, C. 2000. Google Launches Self-Service Advertising Program. (October 23). Retrieved January 6, 2011, from [12] Krane, D. and McCaffrey, C. 2002. Google Introduces New Pricing For Popular Self-Service Online Advertising Program. (February 20). Retrieved January 6, 2011, from http://www. [13] Google. 2010. Google, Corporate Information, Our Philosophy. Retrieved July 13, 2010, from [14] Saracevic, T. 1975. “Relevance: A Review of and a Framework for the Thinking on the Notion in Information Science.” Journal of the American Society of Information Science, vol. 26(6), pp. 321–343. [15] Battelle, J. 2005. The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed Our Culture.


pages: 317 words: 84,400

Automate This: How Algorithms Came to Rule Our World by Christopher Steiner


23andMe, Ada Lovelace, airport security, Al Roth, algorithmic trading, backtesting, big-box store, Black-Scholes formula, call centre, cloud computing, collateralized debt obligation, Credit Default Swap, credit default swaps / collateralized debt obligations, delta neutral, Donald Trump, Douglas Hofstadter, dumpster diving, Flash crash, Gödel, Escher, Bach, High speed trading, Howard Rheingold, index fund, Isaac Newton, John Maynard Keynes: technological unemployment, knowledge economy, late fees, Mark Zuckerberg, market bubble, medical residency, Narrative Science, PageRank, pattern recognition, Paul Graham, prediction markets, quantitative hedge fund, Renaissance Technologies, ride hailing / ride sharing, risk tolerance, Sergey Aleynikov, side project, Silicon Valley, Skype, speech recognition, Spread Networks laid a new fibre optics cable between New York and Chicago, transaction costs, upwardly mobile, Watson beat the top human players on Jeopardy!, Y Combinator

Being able to identify pecking-order top dogs automatically on a wide scale could lead to new approaches in politics, management, sales, and marketing. This model of ranking things based on small clues of influence is the same calculus that drives PageRank, Google’s algorithm, named after cofounder Larry Page, which steers Web traffic to sites the Web regards as authoritative on the subject being searched. Important Web sites are called hubs and influencers. Google gives more credence in its search results to sites that are often linked to by influential sites and hubs. If these sites commonly refer to, say, a particular flight-booking search engine as the best one while concurrently linking to it, it’s likely that this Web site will rise to the top of Google’s results. By looking at where the influential sites link, Google’s algorithm can quickly determine what to show for any query a user might type in. Sorting humans can be done much the same way.

., 3, 218 “Explanation of Binary Arithmetic” (Leibniz), 58 ExxonMobil, 50 Facebook, 198–99, 204–6, 214 graph theory and, 70 face-reading algorithms, 129, 161 Falchuk, Myron, 157 Farmville, 206 fat tails, 63–64 FBI, 137 FedEx, 116 Ferguson, Lynne, 87 Fermat, Pierre, 66–67 fiber: dark, 114–20, 122 lit, 114 fiber optic cables, 117, 124, 192 Fibonacci, Leonardo, 56–57 Fibonacci sequence, 57 Fidelity, 50 finance, probability theory and, 66 financial markets, algorithms’ domination of, 24 financial sector, expansion of, 184, 191 see also Wall Street Finkel, Eli, 145 Finland, 130 First New York Securities, 4 Fisher, Helen, 144 Flash Crash of 2010, 2–5, 48–49, 64, 184 Forbes magazine, 8 foreign exchange, golden mean and, 57 Fortran, 12, 38 Fortune 500 companies, Kahler’s methods at, 176 Fourier, Joseph, 105–6 Fourier series, 105–7 Fourier transforms, 82 401K plans, 50 Fox News, 137 fractal geometry, 56 France, 61, 66, 80, 121, 147 Frankfurt, 121 fraud, eLoyalty bots and, 193 French-English translation software, 178–79 From Darkness, Light, 99 galaxies, orbital patterns of, 56 gambling: algorithms and, 127–35 probability theory and, 66, 67 game theory, 58 algorithms and, 129–31 and fall of Soviet Union, 136 in organ donor networks, 147–49 in politics, 136 sports betting and, 133–35 terrorism prevention by, 135–40 gastroenterology, 157 Gauss, Carl Friedrich, 61–65 Gaussian copula, 65, 189 Gaussian distributions, 63–64 Gaussian functions, 53 GE, 209, 213 Geffen, 87 General Mills, 130 General Motors, 201 genes, algorithmic scanning of, 159, 160 geometry, 55 of carbon, 70 fractal, 56 George IV, king of England, 62 Germany, 26, 61, 90 West, 19 Getco, 49, 116, 118 Glenn, John, 175 gluten, 157 Gmail, 71, 196 Gödel, Escher, Bach: An Eternal Golden Braid (Hofstadter), 97 gold, 21, 27 Gold and Stock Telegraph Company, 123 Goldberg, David, 219 golden mean, 56–57 Goldman Sachs, 116, 119, 204, 213 bailout of, 191 engineering and science talent hired by, 179, 186, 187, 189 Hull Trading bought by, 46 Peterffy’s buyout offer from, 46 Gomez, Dominic, 87 goodwill, 27 Google, 47, 71, 124, 192, 196, 207, 213, 219 algorithm-driven cars from, 215 PageRank algorithm of, 213–14 Gorbachev, Mikhail, 136 Göttingen, 122 Göttingen, University of, 59, 65 grain prices, hedging algorithm for, 130 grammar, algorithms for, 54 Grammy awards, 83 graph theory, 69–70 Great Depression, 123 Greatest Trade Ever, The (Zuckerman), 202 Greece, rioting in, 2–3 Greenlight Capital, 128 Greenwich, Conn., 47, 48 Griffin, Blake, 142 Griffin, Ken, 128, 190 Groopman, Jerome, 156 Groupon, 199 growth prospects, 27 Guido of Arezzo, 91 guitars: Harrison’s twelve–string Rickenbacker, 104–5, 107–9 Lennon’s six–string, 104, 107–8 hackers: as algorithm creators, 8, 9, 178 chat rooms for, 53, 124 as criminals, 7–8 for gambling, 135 Leibniz as, 60 Lovelace as, 73 online, 53 poker played by, 128 Silicon Valley, 8 on Wall Street, 17–18, 49, 124, 160, 179, 185, 201 Wall Street, dawn of hacker era on, 24–27 haiku, algorithm-composed, 100–101 Haise, Fred, 165–67 Hal 9000, 7 Hammerbacher, Jeffrey, 201–6, 209, 216 Handel, George Frideric, 68, 89, 91 Hanover, 62 Hanto, Ruthanne, 151 Hardaway, Penny, 143 “Hard Day’s Night, A,” opening chord of, 104–10 hardware: escalating war of, 119–25 Leibniz’s binary system and, 61 Harrah’s, 135 Harrison, George, 103–5, 107–10 on Yahoo!

., 140, 165 Nobel Prize, 23, 106 North Carolina, 48, 204 Northwestern University, 145, 186 Kellogg School of Management at, 10 Novak, Ben, 77–79, 83, 85, 86 NSA, 137 NuclearPhynance, 124 nuclear power, 139 nuclear weapons, in Iran, 137, 138–39 number theory, 65 numerals: Arabic-Indian, 56 Roman, 56 NYSE composite index, 40, 41 Oakland Athletics, 141 Obama, Barack, 46, 218–19 Occupy Wall Street, 210 O’Connor & Associates, 40, 46 OEX, see S&P 100 index Ohio, 91 oil prices, 54 OkCupid, 144–45 Olivetti home computers, 27 opera, 92, 93, 95 Operation Match, 144 opinions-driven people, 173, 174, 175 OptionMonster, 119 option prices, probability and statistics in, 27 options: Black-Scholes formula and, 23 call, 21–22 commodities, 22 definition of, 21 pricing of, 22 put, 22 options contracts, 30 options trading, 36 algorithms in, 22–23, 24, 114–15 Oregon, University of, 96–97 organ donor networks: algorithms in, 149–51, 152, 214 game theory in, 147–49 oscilloscopes, 32 Outkast, 102 outliers, 63 musical, 102 outputs, algorithmic, 54 Pacific Exchange, 40 Page, Larry, 213 PageRank, 213–14 pairs matching, 148–51 pairs trading, 31 Pakistan, 191 Pandora, 6–7, 83 Papanikolaou, Georgios, 153 Pap tests, 152, 153–54 Parham, Peter, 161 Paris, 56, 59, 121 Paris Stock Exchange, 122, 201 partial differential equations, 23 Pascal, Blaise, 59, 66–67 pathologists, 153 patient data, real-time, 158–59 patterns, in music, 89, 93, 96 Patterson, Nick, 160–61 PayPal, 188 PCs, Quotron data for, 33, 37, 39 pecking orders, social, 212–14 Pennsylvania, 115, 116 Pennsylvania, University of, 49 pension funds, 202 Pentagon, 168, 144 Perry, Katy, 89 Persia, 54 Peru, 91 Peterffy, Thomas: ambitions of, 27 on AMEX, 28–38 automated trading by, 41–42, 47–48, 113, 116 background and early career of, 18–20 Correlator algorithm of, 42–45 early handheld computers developed by, 36–39, 41, 44–45 earnings of, 17, 37, 46, 48, 51 fear that algorithms have gone too far by, 51 hackers hired by, 24–27 independence retained by, 46–47 on index funds, 41–46 at Interactive Brokers, 47–48 as market maker, 31, 35–36, 38, 51 at Mocatta, 20–28, 31 Nasdaq and, 11–18, 32, 42, 47–48, 185 new technology innovated by, 15–16 options trading algorithm of, 22–23, 24 as outsider, 31–32 profit guidelines of, 29 as programmer, 12, 15–16, 17, 20–21, 26–27, 38, 48, 62 Quotron hack of, 32–35 stock options algorithm as goal of, 27 Timber Hill trading operation of, see Timber Hill traders eliminated by, 12–18 trading floor methods of, 28–34 trading instincts of, 18, 26 World Trade Center offices of, 11, 39, 42, 43, 44 Petty, Tom, 84 pharmaceutical companies, 146, 155, 186 pharmacists, automation and, 154–56 Philips, 159 philosophy, Leibniz on, 57 phone lines: cross-country, 41 dedicated, 39, 42 phones, cell, 124–25 phosphate levels, 162 Physicians’ Desk Reference (PDR), 146 physicists, 62, 157 algorithms and, 6 on Wall Street, 14, 37, 119, 185, 190, 207 pianos, 108–9 Pincus, Mark, 206 Pisa, 56 pitch, 82, 93, 106 Pittsburgh International Airport, security algorithm at, 136 Pittsburgh Pirates, 141 Pius II, Pope, 69 Plimpton, George, 141–42 pneumonia, 158 poetry, composed by algorithm, 100–101 poker, 127–28 algorithms for, 129–35, 147, 150 Poland, 69, 91 Polyphonic HMI, 77–79, 82–83, 85 predictive algorithms, 54, 61, 62–65 prescriptions, mistakes with, 151, 155–56 present value, of future money streams, 57 pressure, thriving under, 169–70 prime numbers, general distribution pattern of, 65 probability theory, 66–68 in option prices, 27 problem solving, cooperative, 145 Procter & Gamble, 3 programmers: Cope as, 92–93 at eLoyalty, 182–83 Peterffy as, 12, 15–16, 17, 20–21, 26–27, 38, 48, 62 on Wall Street, 13, 14, 24, 46, 47, 53, 188, 191, 203, 207 programming, 188 education for, 218–20 learning, 9–10 simple algorithms in, 54 Progress Energy, 48 Project TACT (Technical Automated Compatibility Testing), 144 proprietary code, 190 proprietary trading, algorithmic, 184 Prussia, 69, 121 PSE, 40 pseudocholinesterase deficiency, 160 psychiatry, 163, 171 psychology, 178 Pu, Yihao, 190 Pulitzer Prize, 97 Purdue University, 170, 172 put options, 22, 43–45 Pythagorean algorithm, 64 quadratic equations, 63, 65 quants (quantitative analysts), 6, 46, 124, 133, 198, 200, 202–3, 204, 205 Leibniz as, 60 Wall Street’s monopoly on, 183, 190, 191, 192 Queen’s College, 72 quizzes, and OkCupid’s algorithms, 145 Quotron machine, 32–35, 37 Rachmaninoff, Sergei, 91, 96 Radiohead, 86 radiologists, 154 radio transmitters, in trading, 39, 41 railroad rights-of-way, 115–17 reactions-based people, 173–74, 195 ReadyForZero, 207 real estate, 192 on Redfin, 207 recruitment, of math and engineering students, 24 Redfin, 192, 206–7, 210 reflections-driven people, 173, 174, 182 refraction, indexes of, 15 regression analysis, 62 Relativity Technologies, 189 Renaissance Technologies, 160, 179–80, 207–8 Medallion Fund of, 207–8 retirement, 50, 214 Reuter, Paul Julius, 122 Rhode Island hold ‘em poker, 131 rhythms, 82, 86, 87, 89 Richmond, Va., 95 Richmond Times-Dispatch, 95 rickets, 162 ride sharing, algorithm for, 130 riffs, 86 Riker, William H., 136 Ritchie, Joe, 40, 46 Rochester, N.Y., 154 Rolling Stones, 86 Rondo, Rajon, 143 Ross, Robert, 143–44 Roth, Al, 147–49 Rothschild, Nathan, 121–22 Royal Society, London, 59 RSB40, 143 runners, 39, 122 Russia, 69, 193 intelligence of, 136 Russian debt default of 1998, 64 Rutgers University, 144 Ryan, Lee, 79 Saint Petersburg Academy of Sciences, 69 Sam Goody, 83 Sandberg, Martin (Max Martin), 88–89 Sandholm, Tuomas: organ donor matching algorithm of, 147–51 poker algorithm of, 128–33, 147, 150 S&P 100 index, 40–41 S&P 500 index, 40–41, 51, 114–15, 218 Santa Cruz, Calif., 90, 95, 99 satellites, 60 Savage Beast, 83 Saverin, Eduardo, 199 Scholes, Myron, 23, 62, 105–6 schools, matching algorithm for, 147–48 Schubert, Franz, 98 Schwartz, Pepper, 144 science, education in, 139–40, 218–20 scientists, on Wall Street, 46, 186 Scott, Riley, 9 scripts, algorithms for writing, 76 Seattle, Wash., 192, 207 securities, 113, 114–15 mortgage-backed, 203 options on, 21 Securities and Exchange Commission (SEC), 185 semiconductors, 60, 186 sentence structure, 62 Sequoia Capital, 158 Seven Bridges of Königsberg, 69, 111 Shannon, Claude, 73–74 Shuruppak, 55 Silicon Valley, 53, 81, 90, 116, 188, 189, 215 hackers in, 8 resurgence of, 198–211, 216 Y Combinator program in, 9, 207 silver, 27 Simons, James, 179–80, 208, 219 Simpson, O.


pages: 224 words: 64,156

You Are Not a Gadget by Jaron Lanier


1960s counterculture, accounting loophole / creative accounting, additive manufacturing, Albert Einstein, call centre, cloud computing, crowdsourcing, death of newspapers, digital Maoism, Douglas Hofstadter, Extropian, follow your passion, hive mind, Internet Archive, Jaron Lanier, jimmy wales, John Conway, John von Neumann, Kevin Kelly, Long Term Capital Management, Network effects, new economy, packet switching, PageRank, pattern recognition, Ponzi scheme, Ray Kurzweil, Richard Stallman, Silicon Valley, Silicon Valley startup, slashdot, social graph, stem cell, Steve Jobs, Stewart Brand, Ted Nelson, telemarketer, telepresence, The Wisdom of Crowds, trickle-down economics, Turing test, Vernor Vinge, Whole Earth Catalog

It is utterly strange to hear my many old friends in the world of digital culture claim to be the true sons of the Renaissance without realizing that using computers to reduce individual expression is a primitive, retrograde activity, no matter how sophisticated your tools are. Rejection of the Idea of Quality Results in a Loss of Quality The fragments of human effort that have flooded the internet are perceived by some to form a hive mind, or noosphere. These are some of the terms used to describe what is thought to be a new superintelligence that is emerging on a global basis on the net. Some people, like Larry Page, one of the Google founders, expect the internet to come alive at some point, while others, like science historian George Dyson, think that might already have happened. Popular derivative terms like “blogosphere” have become commonplace. A fashionable idea in technical circles is that quantity not only turns into quality at some extreme of scale, but also does so according to principles we already understand.

Visualize, if you will, the most transcendently messy, hirsute, and otherwise eccentric pair of young nerds on the planet. They were in their early twenties. The scene was an uproariously messy hippie apartment in Cambridge, Massachusetts, in the vicinity of MIT. I was one of these men; the other was Richard Stallman. Why are so many of the more sophisticated examples of code in the online world—like the page-rank algorithms in the top search engines or like Adobe’s Flash—the results of proprietary development? Why did the adored iPhone come out of what many regard as the most closed, tyrannically managed software-development shop on Earth? An honest empiricist must conclude that while the open approach has been able to create lovely, polished copies, it hasn’t been so good at creating notable originals.

When businesses rushed in to capitalize on what had happened, there was something of a problem, in that the content aspect of the web, the cultural side, was functioning rather well without a business plan. Google came along with the idea of linking advertising and searching, but that business stayed out of the middle of what people actually did online. It had indirect effects, but not direct ones. The early waves of web activity were remarkably energetic and had a personal quality. People created personal “homepages,” and each of them was different, and often strange. The web had flavor. Entrepreneurs naturally sought to create products that would inspire demand (or at least hypothetical advertising opportunities that might someday compete with Google) where there was no lack to be addressed and no need to be filled, other than greed. Google had discovered a new permanently entrenched niche enabled by the nature of digital technology.


Everydata: The Misinformation Hidden in the Little Data You Consume Every Day by John H. Johnson


Affordable Care Act / Obamacare, Black Swan, business intelligence, Carmen Reinhart, cognitive bias, correlation does not imply causation, Daniel Kahneman / Amos Tversky, Donald Trump,, Kenneth Rogoff, labor-force participation, lake wobegon effect, Long Term Capital Management, Mercator projection, Mercator projection distort size, especially Greenland and Africa, meta analysis, meta-analysis, Nate Silver, obamacare, p-value, PageRank, pattern recognition, randomized controlled trial, risk-adjusted returns, Ronald Reagan, statistical model, The Signal and the Noise by Nate Silver, Tim Cook: Apple, wikimedia commons, Yogi Berra

” Washington Post website, April 22, 2015, http://­how-​­does‑a‑­teachers -​­race-​­affect-​­which-​­students-​­get‑to‑be‑identified‑as‑gifted/. 20. Emily Oster, “Take Back Your Pregnancy,” Wall Street Journal website, August 9, 2013, 404578652091268307904. 21. “The Value of Google Result Positioning,” Chitika website, June 7, 2013,­google-​­positioning-​­value. 22. “Algorithms,” Google website, accessed April 20, 2015, .com/insidesearch/howsearchworks/algorithms.html. Here, you’ll also find a link to “The Anatomy of a ­Large-​­Scale Hypertextual Web Search Engine,” in which Sergey Brin and Larry Page presented Google. 23. “Search Engine Ranking Factors 2015,” Moz website, accessed September 1, 2015,­search-​­ranking-​­factors/correlations. 24. “Search Engine Ranking Factors 2015, Expert Survey and Correlation Data,” Moz website, accessed September 1, 2015,­search-​­ranking-​­factors/ correlations. 25.

What if you’re truly unable to determine what the omitted variables are? Here’s an example. If you run a business, you would probably love to nearly double the traffic to your company’s website. After all, the ­number-​­one spot on Google search results gets almost twice the traffic that the ­number-​­two spot does.21 Depending on your business, moving up just one spot in Google rankings could bring millions of additional visitors. So how do you improve your ranking? According to Google, the engine determines search results using algorithms that rely on “more than 200 unique signals or ‘clues’ that make it possible to guess what you might really be looking for.”22 The problem is that Google doesn’t give you details about what those ­200-​­plus signals ­are—​­perhaps because it doesn’t want to give away its competitive advantage. How do you deal with more than 200 omitted variables?

How do you deal with more than 200 omitted variables? Well, if you click over to, you’ll see charts showing how more than 160 factors correlate to search engine rankings.23 It’s interesting stuff, and probably very useful if you’re looking for ways to increase your page ranking. But it’s not definitive, because it’s based largely on correlations. To its credit, uses the word “correlation” 12 times on the page.24 In a separate blog post, it goes even further, explaining that “correlation data isn’t (necessarily) showing us ranking factors.”25 Sometimes, you simply can’t get your hands on the omitted variables. Maybe the data is proprietary. Maybe it was accidentally destroyed, or never recorded in the first place. In these cases, you can try to reverse engineer the data and tease out some correlations.


pages: 528 words: 146,459

Computer: A History of the Information Machine by Martin Campbell-Kelly, William Aspray, Nathan L. Ensmenger, Jeffrey R. Yost


Ada Lovelace, air freight, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Apple's 1984 Super Bowl advert, barriers to entry, Bill Gates: Altair 8800, borderless world, Buckminster Fuller, Build a better mousetrap, Byte Shop, card file, cashless society, cloud computing, combinatorial explosion, computer age, deskilling, don't be evil, Douglas Engelbart, Dynabook, fault tolerance, Fellow of the Royal Society, financial independence, Frederick Winslow Taylor, game design, garden city movement, Grace Hopper, informal economy, interchangeable parts, invention of the wheel, Jacquard loom, Jacquard loom, Jeff Bezos, jimmy wales, John von Neumann, linked data, Mark Zuckerberg, Marshall McLuhan, Menlo Park, natural language processing, Network effects, New Journalism, Norbert Wiener, Occupy movement, optical character recognition, packet switching, PageRank, pattern recognition, pirate software, popular electronics, prediction markets, pre–internet, QWERTY keyboard, RAND corporation, Robert X Cringely, Silicon Valley, Silicon Valley startup, Steve Jobs, Steven Levy, Stewart Brand, Ted Nelson, the market place, Turing machine, Vannevar Bush, Von Neumann architecture, Whole Earth Catalog, William Shockley: the traitorous eight, women in the workforce, young professional

Using a “web crawler” to gather back-link data (that is, the websites that linked to a particular site), Page, now teamed up with Brin, created their “PageRank” algorithm based on back-links ranked by importance—the more prominent the linking site, the more influence it would have on the linked site’s page rank. They insightfully reasoned that this would provide the basis for more useful web searches than any existing tools and, moreover, that there would be no need to hire a corps of indexing staff. Thus was born their “search engine,” Backrub, renamed Google shortly before they launched the URL in September 1997. The name was a modification of a friend’s suggestion of googol—a term referring to the number 1 followed by 100 zeros. Brin misspelled the term as google, but the Internet address for googol was already taken so the catchy misspelling stuck.

was not without competition: Lycos, Excite, and a dozen others had come up with the same concept, and listing and information search services became one of the first established categories of the web. One question remained: How to pay for the service? The choices included subscriptions, sponsorship, commissions, or advertising. As with early broadcasting, advertising was the obvious choice. Another firm focused on helping users find information on the web—Google Inc.—soon demonstrated how lucrative web advertising could be. Yahoo! was already well established when two other Stanford University doctoral students, Larry Page and Sergey Brin, began work on the Stanford Digital Library Project (funded in part by the National Science Foundation)—research that would not only forever change the process of finding things on the Internet but also, in time, lead to an unprecedentedly successful web advertising model. Page became interested in a dissertation project on the mathematical properties of the web, and found strong support from his adviser Terry Winograd, a pioneer of artificial intelligence research on natural language processing.

The symbolic return of Silicon Valley to glory came with the success of Google. In 2004 Google’s public offering valued the company at more than $26 billion. By 2007 Google facilitated more searches than all other search and listing services combined. That year Google achieved revenue of $16.6 billion and net income of $4.2 billion. Google continues to dominate the search field with 1.7 trillion annual searches (in 2011, representing roughly a two-thirds share). While search-based advertising revenue remained its primary source of income, Google successfully moved into e-mail services (Gmail), maps and satellite photos, Internet video (with its 2006 acquisition of YouTube), cloud computing, digitizing books, and other endeavors. More recently, Google has also been an important participant in open-source mobile platforms that are transforming computing.


pages: 829 words: 186,976

The Signal and the Noise: Why So Many Predictions Fail-But Some Don't by Nate Silver


airport security, availability heuristic, Benoit Mandelbrot, Berlin Wall, Bernie Madoff, big-box store, Black Swan, Broken windows theory, Carmen Reinhart, Claude Shannon: information theory, Climategate, Climatic Research Unit, cognitive dissonance, collapse of Lehman Brothers, collateralized debt obligation, complexity theory, computer age, correlation does not imply causation, Credit Default Swap, credit default swaps / collateralized debt obligations, cuban missile crisis, Daniel Kahneman / Amos Tversky, diversification, Donald Trump, Edmond Halley, Edward Lorenz: Chaos theory,, equity premium, Eugene Fama: efficient market hypothesis, everywhere but in the productivity statistics, fear of failure, Fellow of the Royal Society, Freestyle chess, fudge factor, George Akerlof, haute cuisine, Henri Poincaré, high batting average, housing crisis, income per capita, index fund, Internet Archive, invention of the printing press, invisible hand, Isaac Newton, James Watt: steam engine, John Nash: game theory, John von Neumann, Kenneth Rogoff, knowledge economy, locking in a profit, Loma Prieta earthquake, market bubble, Mikhail Gorbachev, Moneyball by Michael Lewis explains big data, Monroe Doctrine, mortgage debt, Nate Silver, new economy, Norbert Wiener, PageRank, pattern recognition,, prediction markets, Productivity paradox, random walk, Richard Thaler, Robert Shiller, Robert Shiller, Rodney Brooks, Ronald Reagan, Saturday Night Live, savings glut, security theater, short selling, Skype, statistical model, Steven Pinker, The Great Moderation, The Market for Lemons, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Kuhn: the structure of scientific revolutions, too big to fail, transaction costs, transfer pricing, University of East Anglia, Watson beat the top human players on Jeopardy!, wikimedia commons

This metaphor is borrowed from Bill Wyman, a music critic for the Chicago Reader, who ranked it as the greatest moment in rock history. Bill Wyman, “The 100 Greatest Moments in Rock History,” Chicago Reader, September 28, 1995. 44. Campbell, Hoane Jr., and Feng-hsiung, “Deep Blue.” 45. Larry Page, “PageRank: Bringing Order to the Web,” Stanford Digital Library Project, August 18, 1997. 46. “How Search Works,” by Google via YouTube, March 4, 2010. 47. Per interview with Vasik Rajlich. 48. “Amateurs beat GMs in PAL / CSS Freestyle,” ChessBase News. 49. Kasparov, “The Chess Master and the Computer.”

You probably should have formed a better search query, but since you didn’t, Google can convene a panel of 1,000 people who made the same request, show them a wide variety of Web pages, and have them rate the utility of each one on a scale of 0 to 10. Then Google would display the pages to you in order of the highest to lowest average rating. Google cannot do this for every search request, of course—not when they receive hundreds of millions of search requests per day. But, Varian told me, they do use human evaluators on a series of representative search queries. Then they see which statistical measurements are best correlated with these human judgments about relevance and usefulness. Google’s best-known statistical measurement of a Web site is PageRank,45 a score based on how many other Web pages link to the one you might be seeking out. But PageRank is just one of two hundred signals that Google uses46 to approximate the human evaluators’ judgment.

Ziobrowski, “Abnormal Returns from the Common Stock Investments of the U.S. Senate,” Journal of Financial and Quantiative Analysis, 39, no. 4 (December 2004). 34. Google Scholar search. 35. Google Scholar search. 36. Google Scholar search. 37. John Aidan Byrne, “Elkins/McSherry—Global Transaction Costs Decline Despite High Frequency Trading,” Institutional Investor, November 1, 2010.


pages: 669 words: 210,153

Tools of Titans: The Tactics, Routines, and Habits of Billionaires, Icons, and World-Class Performers by Timothy Ferriss


Airbnb, artificial general intelligence, asset allocation, Atul Gawande, augmented reality, back-to-the-land, Bernie Madoff, Bertrand Russell: In Praise of Idleness, Black Swan, blue-collar work, Buckminster Fuller, business process, Cal Newport, call centre, Checklist Manifesto, cognitive bias, cognitive dissonance, Colonization of Mars, Columbine, correlation does not imply causation, David Brooks, David Graeber, diversification, diversified portfolio, Donald Trump, effective altruism, Elon Musk, fault tolerance, fear of failure, Firefox, follow your passion, future of work, Google X / Alphabet X, Howard Zinn, Hugh Fearnley-Whittingstall, Jeff Bezos, job satisfaction, Johann Wolfgang von Goethe, Kevin Kelly, Kickstarter, Lao Tzu, life extension, Mahatma Gandhi, Mark Zuckerberg, Mason jar, Menlo Park, Mikhail Gorbachev, Nicholas Carr, optical character recognition, PageRank, passive income, pattern recognition, Paul Graham, Peter H. Diamandis: Planetary Resources, Peter Singer: altruism, Peter Thiel, phenotype, post scarcity, premature optimization, QWERTY keyboard, Ralph Waldo Emerson, Ray Kurzweil, recommendation engine, rent-seeking, Richard Feynman, Richard Feynman, risk tolerance, Ronald Reagan, sharing economy, side project, Silicon Valley, skunkworks, Skype, Snapchat, social graph, software as a service, software is eating the world, stem cell, Stephen Hawking, Steve Jobs, Stewart Brand, superintelligent machines, Tesla Model S, The Wisdom of Crowds, Thomas L Friedman, Wall-E, Washington Consensus, Whole Earth Catalog, Y Combinator

“When 99% of people doubt you, you’re either gravely wrong or about to make history.” “I saw this the other day, and this comes from Scott Belsky [page 359], who was a founder of Behance.” “The best way to become a billionaire is to help a billion people.” Peter co-founded Singularity University with Ray Kurzweil. In 2008, at their founding conference at NASA Ames Research Center in Mountain View, California, Google co-founder Larry Page spoke. Among other things, he underscored how he assesses projects: “I now have a very simple metric I use: Are you working on something that can change the world? Yes or no? The answer for 99.99999% of people is ‘no.’ I think we need to be training people on how to change the world.” Origins of the XPRIZE and “SuperCredibility” “The fact of the matter is I read this book, The Spirit of St.

Here are my three primary responses to online criticism: Starve it of oxygen (ignore it)—90% Pour gasoline on it (promote it)—8% Engage with trolls after too much wine (and really regret it)—2% I’m not going to cover option number three, but the first two are worth explaining. The reason that you would want to starve 90% of oxygen is because doing otherwise gives your haters extra Google juice. In other words, if you reply publicly—worst-case scenario, you put something on another site with high page rank and link to the critic—all you’re going to do is gift them powerful inbound links, increase traffic, and ensure the persistence and prominence of the piece. In some cases, I’ve had to bite my tongue for months at a time to wait for something (infuriating BS that I could easily refute) to drop off the front page or even the second page of Google results. It’s very, very hard to stay silent, and it’s very, very important to have that self-control. Rewatch the “Hoooold! Hooooooold!” scene from Braveheart. But what about pouring gasoline on 8% of the negative?

That you are working on a unique problem that people are not solving elsewhere. “When Elon Musk started SpaceX, they set out the mission to go to Mars. You may agree or disagree with that as a mission statement, but it was a problem that was not going to be solved outside of SpaceX. All of the people working there knew that, and it motivated them tremendously.” TF: Peter has written elsewhere, “The next Bill Gates will not build an operating system. The next Larry Page or Sergey Brin won’t make a search engine. And the next Mark Zuckerberg won’t create a social network. If you are copying these guys, you aren’t learning from them.” ✸ How would you reply to someone who says that your position on college and higher education is hypocritical since you, yourself, went to Stanford for both undergraduate and law school? [Context: Many people see Peter as “anti-college” due to his Thiel Fellowship, which “gives $100,000 to young people who want to build new things instead of sitting in a classroom.”]


pages: 418 words: 128,965

The Master Switch: The Rise and Fall of Information Empires by Tim Wu


accounting loophole / creative accounting, Alfred Russel Wallace, Apple II, barriers to entry, British Empire, Burning Man, Cass Sunstein, Clayton Christensen, don't be evil, Douglas Engelbart, Howard Rheingold, Hush-A-Phone, informal economy, intermodal, Internet Archive, invention of movable type, invention of the telephone, invisible hand, Jane Jacobs, Joseph Schumpeter, Menlo Park, open economy, packet switching, PageRank, profit motive, road to serfdom, Ronald Coase, shareholder value, Silicon Valley, Skype, Steve Jobs, Steve Wozniak, Telecommunications Act of 1996, The Chicago School, The Death and Life of Great American Cities, the market place, The Wisdom of Crowds, too big to fail, Upton Sinclair, urban planning

This may seem an improbably shaky foundation to build a firm on, but perhaps that is the genius of it. If that seems a bit abstract, it is well to remember that Google is an unusually academic company in origins and sensibility. Larry Page, one of the two founders, described his personal ambitions this way: “I decided I was either going to be a professor or start a company.” Just as Columbia University effectively financed FM radio in the 1930s, Stanford got Google started. With its original Web address, the operation relied on university hardware and software and the efforts of graduate students. “At one point,” as John Battelle writes in The Search, the early Google “consumed nearly half of Stanford’s entire network bandwidth.”15 Google’s corporate design remains both its greatest strength and its most serious vulnerability. It is what makes the firm so remarkably well adapted to the Internet environment, as a native species, so to speak.

The firm harvests the best of the Internet, organizing the worldwide chaos in a useful way, and asks its users to navigate this order via their own connections; by relying on the sweat of others for content and carriage, Google can focus on its central mission: search. From its founding, the firm was dedicated to performing that function with clear superiority; it famously pioneered an algorithm called PageRank, which arranged search hits by importance rather than sheer numerical incidence, thereby making search more intelligent. The company resolved to stand or fall on the strength of that competitive edge. As Google’s CEO, Eric Schmidt, explained to me once, firms like the old AT&T or Western Union “had to build the entire supply chain. We are specialized. We understand that infrastructure is not the same thing as content. And we do infrastructure better than anyone else.” Google, between content and transport Unlike AOL Time Warner, Google doesn’t need to try to steer users anywhere in particular.

That’s the advantage. On the other hand, Google’s lack of vertical integration leaves it vulnerable, rather like a medieval city without a wall.* He who controls the wires or airwaves can potentially destroy Google, for it is only via these means that Google reaches its customers. To use the search engine and other utilities, you need Internet access, not a service Google now provides (with trivial exceptions). To have such access, you need to pay an Internet Service Provider—typically your telephone or cable company. Meanwhile, Google itself must also pay for Internet service, a fact that, conceptually at least, puts the firm and its customers on an equal footing: both are subscription users of the Internet. And so whoever controls those connection services can potentially block Google—or any other site or content, as well as the individual user, for that matter.


pages: 855 words: 178,507

The Information: A History, a Theory, a Flood by James Gleick


Ada Lovelace, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, AltaVista, bank run, bioinformatics, Brownian motion, butterfly effect, citation needed, Claude Shannon: information theory, clockwork universe, computer age, conceptual framework, crowdsourcing, death of newspapers, discovery of DNA, double helix, Douglas Hofstadter,, Eratosthenes, Fellow of the Royal Society, Gödel, Escher, Bach, Henri Poincaré, Honoré de Balzac, index card, informal economy, information retrieval, invention of the printing press, invention of writing, Isaac Newton, Jacquard loom, Jacquard loom, Jaron Lanier, jimmy wales, John von Neumann, Joseph-Marie Jacquard, Louis Daguerre, Marshall McLuhan, Menlo Park, microbiome, Milgram experiment, Network effects, New Journalism, Norbert Wiener, On the Economy of Machinery and Manufactures, PageRank, pattern recognition, phenotype, pre–internet, Ralph Waldo Emerson, RAND corporation, reversible computing, Richard Feynman, Richard Feynman, Simon Singh, Socratic dialogue, Stephen Hawking, Steven Pinker, stochastic process, talking drums, the High Line, The Wisdom of Crowds, transcontinental railway, Turing machine, Turing test, women in the workforce

When the publishers of the Oxford English Dictionary began digitizing its contents in 1987 (120 typists; an IBM mainframe), they estimated its size at a gigabyte. A gigabyte also encompasses the entire human genome. A thousand of those would fill a terabyte. A terabyte was the amount of disk storage Larry Page and Sergey Brin managed to patch together with the help of $15,000 spread across their personal credit cards in 1998, when they were Stanford graduate students building a search-engine prototype, which they first called BackRub and then renamed Google. A terabyte is how much data a typical analog television station broadcasts daily, and it was the size of the United States government’s database of patent and trademark records when it went online in 1998. By 2010, one could buy a terabyte disc drive for a hundred dollars and hold it in the palm of one hand.

Kolmogorov in Perspective. History of Mathematics, vol. 20. Translated by Harold H. McFaden. N.p.: American Mathematical Society, London Mathematical Society, 2000. Krutch, Joseph Wood. Edgar Allan Poe: A Study in Genius. New York: Knopf, 1926. Kubát, Libor, and Jirí Zeman. Entropy and Information in Science and Philosophy. Amsterdam: Elsevier, 1975. Langville, Amy N., and Carl D. Meyer. Google’s Page Rank and Beyond: The Science of Search Engine Rankings. Princeton, N.J.: Princeton University Press, 2006. Lanier, Jaron. You Are Not a Gadget. New York: Knopf, 2010. Lanouette, William. Genius in the Shadows. New York: Scribner’s, 1992. Lardner, Dionysius. “Babbage’s Calculating Engines.” Edinburgh Review 59, no. 120 (1834): 263–327. ———. The Electric Telegraph. Revised and rewritten by Edward B.

At any given moment a thousand such slips sat on Simpson’s desk, and within a stone’s throw were millions more, filling metal files and wooden boxes with the ink of two centuries. But the word-slips had gone obsolete. They had become treeware. Treeware had just entered the OED as “computing slang, freq. humorous”; blog was recognized in 2003, dot-commer in 2004, cyberpet in 2005, and the verb to Google in 2006. Simpson himself Googled often. Beside the word-slips his desk held conduits into the nervous system of the language: instantaneous connection to a worldwide network of proxy amateur lexicographers and access to a vast, interlocking set of databases growing asymptotically toward the ideal of All Previous Text. The dictionary had met cyberspace, and neither would be the same thereafter. However much Simpson loved the OED’s roots and legacy, he was leading a revolution, willy-nilly—in what it was, what it knew, what it saw.