packet switching

107 results back to index


pages: 675 words: 141,667

Open Standards and the Digital Age: History, Ideology, and Networks (Cambridge Studies in the Emergence of Global Enterprise) by Andrew L. Russell

American ideology, animal electricity, barriers to entry, borderless world, Chelsea Manning, computer age, creative destruction, disruptive innovation, Donald Davies, Edward Snowden, Frederick Winslow Taylor, Hacker Ethic, Howard Rheingold, Hush-A-Phone, interchangeable parts, invisible hand, John Markoff, Joseph Schumpeter, Leonard Kleinrock, means of production, Menlo Park, Network effects, new economy, Norbert Wiener, open economy, packet switching, pre–internet, RAND corporation, RFC: Request For Comment, Richard Stallman, Ronald Coase, Ronald Reagan, Silicon Valley, Steve Crocker, Steven Levy, Stewart Brand, technoutopianism, Ted Nelson, The Nature of the Firm, Thomas L Friedman, Thorstein Veblen, transaction costs, web of trust

A growing number of networking experiments outside the United States had already begun to take shape, including two significant projects in France and Great Britain. Louis Pouzin, the computer scientist working for IRIA, was designing a packet-switched network called Cyclades; Donald Davies, the computer scientist at Great Britain’s NPL, had begun his packet-switching experiments in the mid-1960s. Additionally, several PTT national monopolies in Europe were evaluating packet-switching technology, and the European Common Market had asked Derek Barber from Britain’s NPL to direct the creation of a European Informatics Network. These researchers came together to pursue their shared objective: to design new network standards for a new era of digital, packet-switched communication. The network researchers assembled in Washington, D.C. in October 1972 – including Pouzin, Davies, Barber, and leaders of the Arpanet community – took advantage of the unprecedented occasion to form the International Network Working Group (INWG).

By the late 1970s, three competing communities of researchers – Arpanet engineers, telecommunications professionals in the ITU, and a loose alliance of American and European computer professionals – were seeking to establish their own designs as the definitive architecture for packet-switched networks. They would press on with the benefit of Pouzin’s insights but without the presence of the sage of datagrams himself. INWG member John Day summarized the significance of Pouzin’s technical work, suggesting that the conventional wisdom about the “invention” of packet-switching and internetworking is incomplete without reference to Pouzin and the young cohort of computer researchers whom he inspired: The real breakthrough in networking is not packet switching (Baran and Davies independently), but datagram packet switching (Pouzin). I have always found it somewhat interesting that every project Baran and Larry Roberts have been involved in since the ARPANet have been connection-oriented networks, not connectionless ones.

Rather, the principles of openness and consensus have international origins and global consequences. In Chapter 6, I describe a series of international collaborations in the 1970s among American, British, and French computer researchers who tried, and ultimately failed, to agree on a single design for packet-switched computer networks. Instead, the future of computer networking appeared to be on the verge of a battle between IBM’s proprietary System Network Architecture and public data networks based on the X.25 standard produced by the International Telecommunications Union. These foes proved to be too powerful for the packet-switched researchers. In 1976, the packet-switching research community splintered into two groups: one inspired by the French computer scientists Louis Pouzin and Hubert Zimmermann, and the other funded by the American Department of Defense and led by Vinton Cerf and Robert Kahn.


pages: 352 words: 96,532

Where Wizards Stay Up Late: The Origins of the Internet by Katie Hafner, Matthew Lyon

air freight, Bill Duvall, computer age, conceptual framework, Donald Davies, Douglas Engelbart, Douglas Engelbart, fault tolerance, Hush-A-Phone, information retrieval, John Markoff, Kevin Kelly, Leonard Kleinrock, Marc Andreessen, Menlo Park, natural language processing, packet switching, RAND corporation, RFC: Request For Comment, Robert Metcalfe, Ronald Reagan, Silicon Valley, speech recognition, Steve Crocker, Steven Levy

It was the first and only time the computers went down. The phone company executives’first reaction was to laugh. “I looked up in pain,” said Metcalfe, “and I caught them smiling, delighted that packet-switching was flaky. This I will never forget. It confirmed for them that circuit-switching technology was here to stay, and this packet-switching stuff was an unreliable toy that would never have much impact in the commercial world, and now they could go home to New Jersey. It was clear to me they were tangled up in the past.” Had they looked beyond the luckless Metcalfe and the failed demo, the AT&T executives would have seen the exuberance in other corners of the room. Not only did packet-switching work but it made wondrous things possible. Some of the most ingenious demonstrations involved English-language conversational programs. These were elaborate programs constructed to engage a user in a verbal dialogue with a machine.

In 1954 Davies won a fellowship to spend a year in the United States; part of that year, he was at MIT. He then returned to England, rose swiftly at the NPL, and in 1966, after describing his pioneering work on packet-switching, he was appointed head of the computer science division. The technical similarity between Davies’ and Baran’s work was striking. Not only were their ideas roughly parallel in concept, but by coincidence they had even chosen the same packet size and data-transmission rate. Independently, Davies also came up with a routing scheme that was adaptive, like Baran’s, but different in detail. There was just one major difference in their approaches. The motivation that led Davies to conceive of a packet-switching network had nothing to do with the military concerns that had driven Baran. Davies simply wanted to create a new public communications network.

Before settling on the word, he asked two linguists from a research team in his lab to confirm that there were cognates in other languages. When they reported back that it was a good choice, he fixed on it. Packet-switching. It was precise, economic, and very British. And it was far easier on the ear than Baran’s “distributed adaptive message block switching.” Davies met Baran for the first time several years later. He told Baran that he had been thoroughly embarrassed to hear of Baran’s work after he had finished his own, and then added, “Well, you may have got there first, but I got the name.” Mapping It Out In December 1966, when Larry Roberts arrived at the Pentagon, he knew Donald Davies from his trip to London the previous year, but didn’t know about Davies’ subsequent work in packet switching. And he had never heard the name Paul Baran. A few years earlier, Roberts had decided that computing was getting old and everything worth doing inside a computer had already been done.


pages: 720 words: 197,129

The Innovators: How a Group of Inventors, Hackers, Geniuses and Geeks Created the Digital Revolution by Walter Isaacson

1960s counterculture, Ada Lovelace, AI winter, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, AltaVista, Apple II, augmented reality, back-to-the-land, beat the dealer, Bill Gates: Altair 8800, bitcoin, Bob Noyce, Buckminster Fuller, Byte Shop, c2.com, call centre, citizen journalism, Claude Shannon: information theory, Clayton Christensen, commoditize, computer age, crowdsourcing, cryptocurrency, Debian, desegregation, Donald Davies, Douglas Engelbart, Douglas Engelbart, Douglas Hofstadter, Dynabook, El Camino Real, Electric Kool-Aid Acid Test, en.wikipedia.org, Firefox, Google Glasses, Grace Hopper, Gödel, Escher, Bach, Hacker Ethic, Haight Ashbury, Howard Rheingold, Hush-A-Phone, HyperCard, hypertext link, index card, Internet Archive, Jacquard loom, Jaron Lanier, Jeff Bezos, jimmy wales, John Markoff, John von Neumann, Joseph-Marie Jacquard, Leonard Kleinrock, Marc Andreessen, Mark Zuckerberg, Marshall McLuhan, Menlo Park, Mitch Kapor, Mother of all demos, new economy, New Journalism, Norbert Wiener, Norman Macrae, packet switching, PageRank, Paul Terrell, pirate software, popular electronics, pre–internet, RAND corporation, Ray Kurzweil, RFC: Request For Comment, Richard Feynman, Richard Stallman, Robert Metcalfe, Rubik’s Cube, Sand Hill Road, Saturday Night Live, self-driving car, Silicon Valley, Silicon Valley startup, Skype, slashdot, speech recognition, Steve Ballmer, Steve Crocker, Steve Jobs, Steve Wozniak, Steven Levy, Steven Pinker, Stewart Brand, technological singularity, technoutopianism, Ted Nelson, The Coming Technological Singularity, The Nature of the Firm, The Wisdom of Crowds, Turing complete, Turing machine, Turing test, Vannevar Bush, Vernor Vinge, Von Neumann architecture, Watson beat the top human players on Jeopardy!, Whole Earth Catalog, Whole Earth Review, wikimedia commons, William Shockley: the traitorous eight

He would later alienate many of the other developers of the Internet by asserting that, in his PhD thesis and his paper proposing it (both written after Baran began formulating packet switching at RAND), he had “developed the basic principles of packet switching” and “the mathematical theory of packet networks, the technology underpinning the Internet.”65 Beginning in the mid-1990s, he began an energetic campaign to be recognized “as the Father of Modern Data Networking.”66 He claimed in a 1996 interview, “My dissertation laid out the basic principles for packet switching.”67 This led to an outcry among many of the other Internet pioneers, who publicly attacked Kleinrock and said that his brief mention of breaking messages into smaller pieces did not come close to being a proposal for packet switching. “Kleinrock is a prevaricator,” said Bob Taylor. “His claim to have anything to do with the invention of packet switching is typical incorrigible self-promotion, which he has been guilty of from day one.”68 (Countered Kleinrock, “Taylor is disgruntled because he never got the recognition he thought he deserved.”69) Donald Davies, the British researcher who coined the term packet, was a gentle and reticent researcher who never boasted of his accomplishments.

In 1968 Donald Davies at the National Physical Laboratories in England was beginning to write about packet-switched networks.”73 Likewise, in a 1979 paper describing the development of distributed networks, Kleinrock neither mentioned nor cited his own work from the early 1960s. As late as 1990 he was still declaring that Baran was the first to conceive of packet switching: “I would credit him [Baran] with the first ideas.”74 However, when Kleinrock’s 1979 paper was reprinted in 2002, he wrote a new introduction that claimed, “I developed the underlying principles of packet switching, having published the first paper on the subject in 1961.”75 In fairness to Kleinrock, whether or not he had claimed that his work in the early 1960s devised packet switching, he would have been (and still should be) accorded great respect as an Internet pioneer.

“His claim to have anything to do with the invention of packet switching is typical incorrigible self-promotion, which he has been guilty of from day one.”68 (Countered Kleinrock, “Taylor is disgruntled because he never got the recognition he thought he deserved.”69) Donald Davies, the British researcher who coined the term packet, was a gentle and reticent researcher who never boasted of his accomplishments. People called him humble to a fault. But as he was dying, he wrote a paper to be published posthumously that attacked Kleinrock in surprisingly strong terms. “The work of Kleinrock before and up to 1964 gives him no claim to have originated packet switching,” Davies wrote after an exhaustive analysis. “The passage in his book on time-sharing queue discipline, if pursued to a conclusion, might have led him to packet switching, but it did not. . . . I can find no evidence that he understood the principles of packet switching.”70 Alex McKenzie, an engineer who managed BBN’s network control center, would later be even more blunt: “Kleinrock claims to have introduced the idea of packetization. This is utter nonsense; there is NOTHING in the entire 1964 book that suggests, analyzes, or alludes to the idea of packetization.”


pages: 463 words: 118,936

Darwin Among the Machines by George Dyson

Ada Lovelace, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, anti-communist, British Empire, carbon-based life, cellular automata, Claude Shannon: information theory, combinatorial explosion, computer age, Danny Hillis, Donald Davies, fault tolerance, Fellow of the Royal Society, finite state, IFF: identification friend or foe, invention of the telescope, invisible hand, Isaac Newton, Jacquard loom, James Watt: steam engine, John Nash: game theory, John von Neumann, low earth orbit, Menlo Park, Nash equilibrium, Norbert Wiener, On the Economy of Machinery and Manufactures, packet switching, pattern recognition, phenotype, RAND corporation, Richard Feynman, spectrum auction, strong AI, the scientific method, The Wealth of Nations by Adam Smith, Turing machine, Von Neumann architecture, zero-sum game

All the switches in the world could never keep up. But with packet-switched data communications, collective computation scales gracefully as the number of processors (both electronic and biological) grows. Thanks to “hot-potato” routing algorithms, individual messages—the raw material from which intelligence is formed—are broken into smaller pieces, told where they are going but not how to get there, and reassembled after finding their own way to the destination address. Consensual protocols, running on all the processors in the net, maintain the appearance of robust connections between all the elements at once. The resulting free market for information and computational resources determines which connection pathways will be strengthened and which languish or die out. By the introduction of packet switching on an epidemic scale, the computational landscape is infiltrated by virtual circuitry, cultivating a haphazard, dendritic architecture reminiscent more of nature’s design than of our own.

We see the wires plugged into the wall and think of the architecture as constrained by the hardwired topology that the physical connection represents, whereas computationally, our machines belong to a diffuse, untethered cloud of the kind that Good envisioned as the basis of an ultraintelligent machine. All our networking protocols—packet switching, token ring, Ethernet, time-division multiplexing, asynchronous transfer mode, and so on—are simply a way of allowing hundreds of millions of individual processors to tune selectively to each others’ signals, free of interference, as they wish. Paul Baran, pioneer of packet switching, sees the relations between computers and communications advancing along similar, wireless lines. You can plug only so many things at one time into your wall. As everything from taxicabs to telephones to televisions to personal digital assistants becomes connected to the network, universal—and microminiature—wireless is the only way to disentangle the communications web.

Baran founded Com21, Inc., in 1991 to develop the ultrafast packet switches and strategic alliances necessary to deliver a broadband digital communication spectrum over coaxial cable to the home, with a fiber-optic backbone linking the head ends of the local tails. Among the various schemes offering to provide broadband network growth, hybrid fiber-coaxial offers the path of least resistance because much of the infrastructure already is in place. What to do with all this bandwidth is a different problem, but history has shown that as bandwidth becomes available, the digital ecology swiftly takes root and grows. Baran also founded (in 1985) a company named Metricom, better known by the name of its wireless network, Ricochet. A wireless, packet-switched, spread-spectrum digital communications network, Ricochet takes an extreme grassroots approach.


pages: 218 words: 63,471

How We Got Here: A Slightly Irreverent History of Technology and Markets by Andy Kessler

Albert Einstein, Andy Kessler, animal electricity, automated trading system, bank run, Big bang: deregulation of the City of London, Bob Noyce, Bretton Woods, British Empire, buttonwood tree, Claude Shannon: information theory, Corn Laws, Douglas Engelbart, Edward Lloyd's coffeehouse, fiat currency, fixed income, floating exchange rates, Fractional reserve banking, full employment, Grace Hopper, invention of the steam engine, invention of the telephone, invisible hand, Isaac Newton, Jacquard loom, James Hargreaves, James Watt: steam engine, John von Neumann, joint-stock company, joint-stock limited liability company, Joseph-Marie Jacquard, Kickstarter, Leonard Kleinrock, Marc Andreessen, Maui Hawaii, Menlo Park, Metcalfe's law, Metcalfe’s law, Mitch Kapor, packet switching, price mechanism, probability theory / Blaise Pascal / Pierre de Fermat, profit motive, railway mania, RAND corporation, Robert Metcalfe, Silicon Valley, Small Order Execution System, South Sea Bubble, spice trade, spinning jenny, Steve Jobs, supply-chain management, supply-chain management software, trade route, transatlantic slave trade, tulip mania, Turing machine, Turing test, undersea cable, William Shockley: the traitorous eight

The best description is often attributed to Baran, but I don’t think he ever said it, in fact I’m not sure who did (I got it, appropriately, off the Internet), but it is revealing: "Packet switching is the breaking down of data into datagrams or packets that are labeled to indicate the origin and the destination of the information and the forwarding of these packets from one computer to another computer until the information arrives at its final destination computer. This was crucial to the realization of a computer network. If packets are lost at any given point, the message can be resent by the originator." So there you have it. If you could install a computer at various points in the circuit-switched phone network, it would become a packet-switched network, and would withstand not only broken links, but a full scale nu-cu-ler winter. Still a theory, though. Larry Roberts at MIT proposed a collection of computers hooked together via packet switching, which turned into ARPANET.

Payne eventually would go the entrepreneur route, and form Southampton Photonics Inc. in June 2000, a little late to be funding optical ventures. *** Oddly, WDM brought the communications industry full circle. Phone calls began on a circuit-switched network; you just took the whole voice signal and switched it from one wire to another to complete the call. Then packet switching came in, at first to prevent the vulnerability of a nuclear attack. But then packet switching took off, as the most efficient method to handle voice calls, data packets and the transport of Web pages. Packet switching is entirely electrical - a switch SOFTWARE AND NETWORKS 155 or router looks at the header of each packet and decides where to send it. But now with WDM, we are back to circuits. Going from electrical to optical penalizes both cost and speed. An all-optical network means keeping the signal optical as long as possible, until the very last minute when the information is needed.

One of the issues of the day was the idea that a nuclear blast (I’ve learned never to trust anyone that pronounces it nu-cu-ler) would wipe out the phone network and all communications lines and disable the command and control structure of U.S. defense. The president could order a launch, but if no one could get the message, what would be the use? In 1961, Leonard Kleinrock at MIT proposed a PhD thesis called “Information Flow in Large Communication Nets,” and this provided the theory and proof for packet switching, although it wasn’t called packet switching, not yet, and it was still a theory. The North American Aerospace Defense Command or NORAD was in charge of early warning and control. It didn’t want no stinking 144 HOW WE GOT HERE theories, it wanted something it could use. NORAD was nervous about being out of touch, especially with its command center dug into the mountains near Cheyenne. So the Air Force sprinkled money around for research on ways to resolve the vulnerability of communications networks, which were dependant on centralized phone switches.


pages: 518 words: 107,836

How Not to Network a Nation: The Uneasy History of the Soviet Internet (Information Policy) by Benjamin Peters

Albert Einstein, American ideology, Andrei Shleifer, Benoit Mandelbrot, bitcoin, Brownian motion, Claude Shannon: information theory, cloud computing, cognitive dissonance, computer age, conceptual framework, continuation of politics by other means, crony capitalism, crowdsourcing, cuban missile crisis, Daniel Kahneman / Amos Tversky, David Graeber, Dissolution of the Soviet Union, Donald Davies, double helix, Drosophila, Francis Fukuyama: the end of history, From Mathematics to the Technologies of Life and Death, hive mind, index card, informal economy, information asymmetry, invisible hand, Jacquard loom, John von Neumann, Kevin Kelly, knowledge economy, knowledge worker, linear programming, mandelbrot fractal, Marshall McLuhan, means of production, Menlo Park, Mikhail Gorbachev, mutually assured destruction, Network effects, Norbert Wiener, packet switching, Pareto efficiency, pattern recognition, Paul Erdős, Peter Thiel, Philip Mirowski, RAND corporation, rent-seeking, road to serfdom, Ronald Coase, scientific mainstream, Steve Jobs, Stewart Brand, stochastic process, technoutopianism, The Structural Transformation of the Public Sphere, transaction costs, Turing machine

The result of Baran’s conversations was packet switching, a technology that broke messages into “packets,” which allowed digital “bursts” of data to be rerouted around damaged parts of a network—just as the brain can reroute neural impulses around damaged neural matter. Similarly, Baran’s observation was that, due to network effects, the brilliance of a distributed network, whether neural or national, is that it does not need each of the average eighty-six billion neurons in the human brain to connect to every other (and the number of possible connections between eighty-six billion neurons is so incomprehensibly large that the need for robust reconnection becomes obvious).42 Rather, attaching to a couple of other nodes allows a distributed packet-switched network to reroute in real time around damaged territory, whether neural or national.

With no such “competent organization” in sight and after spending six years aggressively publishing his network research internationally to ensure maximum circulation about how survivable communication networks could help ensure mutual deterrence, Baran despaired at the local prospects and turned his attention elsewhere.43 The popularity of the phrase packet switching, which was Davies’s term, and the obscurity of Baran’s initial coinage block switching are evidence that it took outside competition to spur local authorities to take packet switching seriously. The U.S. ARPANET, despite the efforts of its own network entrepreneurs, was inspired by foreign founders. To the degree that Stigler’s law of eponymy holds—“no scientific discovery is named after its original discoverer” (a law that Stigler attributes with a grin to Robert Merton)—Baran’s case rehearses not the exception but the rule that international communication networks precede national computer networks.

Perhaps the cardinal mistake of the socialist imagination of technology is not to dream the celebrated dream of social justice but to bulldoze the rutted world of human relations with the private interest logics of the oikos (military, corporations, states, and individuals that seek only their own survival). The Soviet OGAS figured out the “why?” (socialist utopia) but not the “how?” for their large computer network projects, and researchers at the U.S. ARPANET knew the “how?” (packet-switching networks) but not the “why?” of modern networking. The Soviets’ missing “how?” lasted for the duration of the project, and the absence of the Western “why?” remains both its historical attraction and the contemporary challenge to computer network culture. The Western network “how?” has sped many unfinished attempts at answering the network “why?” The technical openness of packet-switching networks to diverse actors has afforded the Internet astonishing and well-documented successes of technical energy, commercial innovation, and cultural creativity. At the same time, the open-ended “why?”


pages: 523 words: 143,139

Algorithms to Live By: The Computer Science of Human Decisions by Brian Christian, Tom Griffiths

4chan, Ada Lovelace, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, algorithmic trading, anthropic principle, asset allocation, autonomous vehicles, Bayesian statistics, Berlin Wall, Bill Duvall, bitcoin, Community Supported Agriculture, complexity theory, constrained optimization, cosmological principle, cryptocurrency, Danny Hillis, David Heinemeier Hansson, delayed gratification, dematerialisation, diversification, Donald Knuth, double helix, Elon Musk, fault tolerance, Fellow of the Royal Society, Firefox, first-price auction, Flash crash, Frederick Winslow Taylor, George Akerlof, global supply chain, Google Chrome, Henri Poincaré, information retrieval, Internet Archive, Jeff Bezos, Johannes Kepler, John Nash: game theory, John von Neumann, Kickstarter, knapsack problem, Lao Tzu, Leonard Kleinrock, linear programming, martingale, Nash equilibrium, natural language processing, NP-complete, P = NP, packet switching, Pierre-Simon Laplace, prediction markets, race to the bottom, RAND corporation, RFC: Request For Comment, Robert X Cringely, Sam Altman, sealed-bid auction, second-price auction, self-driving car, Silicon Valley, Skype, sorting algorithm, spectrum auction, Stanford marshmallow experiment, Steve Jobs, stochastic process, Thomas Bayes, Thomas Malthus, traveling salesman, Turing machine, urban planning, Vickrey auction, Vilfredo Pareto, Walter Mischel, Y Combinator, zero-sum game

But from a packet-switching point of view, the phone wires are just a means to an end; the sender and receiver don’t actually care how the packets get delivered. The ability to operate agnostically over any number of diverse media would be packet switching’s great virtue. After early networks in the late ’60s and early ’70s, such as the ARPANET, proved the viability of the concept, networks of all types began sprouting across the country, doing packet switching not only over copper phone wires, but over satellites and over radio. In 2001, a group of computer scientists in the Norwegian city of Bergen briefly even implemented a packet-switching network over “Avian Carriers”—that is, packets written down on paper and tied to pigeons’ feet. Of course, packet switching would not be without its own problems. For starters, one of the first questions for any protocol, human or machine, is, quite simply: how do you know your messages are getting through?

“utter heresy”: Jacobson, “A New Way to Look at Networking.” “So little boy went away”: Kleinrock, “Computing Conversations.” would become known as packet switching: The term “packet switching” comes from Donald W. Davies of the National Physical Laboratory, another key contributor to packet switching research at the time. “a consensual illusion between the two endpoints”: Stuart Cheshire, personal interview, February 26, 2015. communications could survive a nuclear attack: Baran, “On Distributed Communications.” a growing network becomes a virtue: For elaboration on this point, and a broader reflection on the history of networking (including its current problems), see Jacobson, “A New Way to Look at Networking.” a packet-switching network over “Avian Carriers”: See Waitzman, A Standard for the Transmission of IP Datagrams on Avian Carriers, Waitzman, IP Over Avian Carriers with Quality of Service, and Carpenter and Hinden, Adaptation of RFC 1149 for IPv6 for descriptions of the avian protocol, and see http://www.blug.linux.no/rfc1149 for details of the actual implementation performed in Bergen, Norway, on April 28, 2001.

In circuit-switched networks, a call fails if any one of its links gets disrupted—which means that reliability goes down exponentially as a network grows larger. In packet switching, on the other hand, the proliferation of paths in a growing network becomes a virtue: there are now that many more ways for data to flow, so the reliability of the network increases exponentially with its size. Still, as Van Jacobson tells it, even after packet switching was devised, the phone companies were unimpressed. “All the telco people said, with very loud voices, that’s not a network! That’s just a crummy way to use our network! You’re taking our wires, you’re sending on the paths that we create! And you’re putting a lot of extra gunk on it so that you use it really inefficiently.” But from a packet-switching point of view, the phone wires are just a means to an end; the sender and receiver don’t actually care how the packets get delivered.


The Dream Machine: J.C.R. Licklider and the Revolution That Made Computing Personal by M. Mitchell Waldrop

Ada Lovelace, air freight, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, anti-communist, Apple II, battle of ideas, Berlin Wall, Bill Duvall, Bill Gates: Altair 8800, Byte Shop, Claude Shannon: information theory, computer age, conceptual framework, cuban missile crisis, Donald Davies, double helix, Douglas Engelbart, Douglas Engelbart, Dynabook, experimental subject, fault tolerance, Frederick Winslow Taylor, friendly fire, From Mathematics to the Technologies of Life and Death, Haight Ashbury, Howard Rheingold, information retrieval, invisible hand, Isaac Newton, James Watt: steam engine, Jeff Rulifson, John von Neumann, Leonard Kleinrock, Marc Andreessen, Menlo Park, New Journalism, Norbert Wiener, packet switching, pink-collar, popular electronics, RAND corporation, RFC: Request For Comment, Robert Metcalfe, Silicon Valley, Steve Crocker, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, Ted Nelson, Turing machine, Turing test, Vannevar Bush, Von Neumann architecture, Wiener process, zero-sum game

Davies's name for the scheme as a whole was "packet switching." From there, said Scantlebury, Davies and his group at Teddington had con- tinued to develop the packet-switching idea with computer simulations. They had even scraped together enough money to build a "one-node" network, con- sisting of a single Honeywell computer connected to a lot of terminals through a special interface. It wasn't much, admittedly. But it did demonstrate the switch- ing principle: you could type in text on one terminal and have it print out on any other terminal you specified. And that, explained Scantlebury, was the sad part of the story: the powers- that-be at the British Postal Service, which had absolute control over the U.K. telecommunications system, had flatly refused to fund Davies's vision of nation- wide packet switching. They couldn't even see the point of a demonstration.

The ICCC demonstration did what it was intended to do, which was make the world sit up and take notice of packet switching. It was what Metcalfe calls the Arpanet's debut-its coming-out party, its coming of age. "Up until that point you couldn't see it anywhere," says Kahn. "All you could do was read an arbitrary abstract paper somewhere that said, 'Here is this new way to do computer communications.' But ICCC was the watershed event that made people suddenly realize that packet switching was a real technology." DIASPORA Looking back on it, there were any number of ways that the Arpanet project could have failed. It could have been snuffed out by the Vietnam-era budget crunch before it even got started, as Pentagon officials scrounged for money high and low. It could have been crushed by the mainstream telecommunica- tions community, which saw packet switching as utterly wrongheaded at best 330 THE DREAM MACHINE and a competitor at worst.

Conversely, a commander in chief blessed with survivable com- mand and control could afford to wait it out, see what developed, and make some effort at a measured response. Indeed, Baran and his colleagues even advocated sharing the packet- switching technology with the Soviets, on the grounds that having survivable communica- tions on both sides would be the most stable configuration of all. THE INTERGALACTIC NETWORK 277 could see that the essence of Baran's network-packets, a decentralized architec- ture, computer routing-was the same as his. So why hadn't Baran's plan been adopted already? Because it was too far ahead of its time, apparently. AT&T engineers, most of whom had spent a life- time perfecting their circuit-switching network, found Baran's packet-switching concept ludicrous ("Son," Baran remembers one telling him with exaggerated patience, "this is how a telephone works. . ."). Worse, Pentagon politics dic- tated that the network would have had to be implemented by the newly orga- nized Defense Communications Agency, which was also staffed by old-line telephone engineers and which simply did not have the technical competence to pull it off.


pages: 323 words: 92,135

Running Money by Andy Kessler

Andy Kessler, Apple II, bioinformatics, Bob Noyce, British Empire, business intelligence, buy and hold, buy low sell high, call centre, Corn Laws, Douglas Engelbart, family office, full employment, George Gilder, happiness index / gross national happiness, interest rate swap, invisible hand, James Hargreaves, James Watt: steam engine, joint-stock company, joint-stock limited liability company, knowledge worker, Leonard Kleinrock, Long Term Capital Management, mail merge, Marc Andreessen, margin call, market bubble, Maui Hawaii, Menlo Park, Metcalfe’s law, Mitch Kapor, Network effects, packet switching, pattern recognition, pets.com, railway mania, risk tolerance, Robert Metcalfe, Sand Hill Road, Silicon Valley, South China Sea, spinning jenny, Steve Jobs, Steve Wozniak, Toyota Production System, zero-sum game

Larry Roberts at MIT proposed a collection of computers hooked together via packet switching.” “OK. But who invented packets?” I asked. “Paul Baran at Rand in Santa Monica gets a lot of credit for packets.” I know about Rand. It is a Santa Monica, California, think tank spun out of Douglas Aircraft after World War II. It’s still around. Packet Racket 185 “NORAD, you know, the North American Air Defense Command,” Kleinrock continued, “shoehorned in that mountain in Cheyenne, was worried about getting cut off from Washington, so the Air Force commissioned a study on how to resolve the vulnerability of communications networks. Baran wrote a paper in 1964 called ‘On Distributed Computing.’ It’s on the Web. You can find it.” “So that was the start of packet switching.” “Sort of. Baran describes standard message blocks and store and forward transmissions and hot potato routing.

The best description I’ve read goes something like this: “Packet switching is the breaking down of data into datagrams or packets that are labeled to indicate the origin and the destination of the information and the forwarding of these packets from one computer to another computer until the information arrives at its final destination computer. This is crucial to the realization of a computer network. If packets are lost at any given point, the message can be resent by the originator.” “So how did you get involved?” I asked. “Well, my thesis proposal at MIT back in 1961 was called ‘Information Flow in Large Communication Nets.’ So they got me involved,” Eddie, I mean Leonard, answered. “So, wait, 1961. It was you that invented packet switching.” “It was a lot of us.” “But what happened in 1969?” “Oh, right.

That is one of those six nine’s, or four to the right of the decimal point, or 99.9999% reliability. As the story goes, researcher Bob Metcalfe was in the middle of demonstrating the packet network when, like any good demo, it crashed. This put smiles on the faces of those 10 AT&T execu-humps, and they merrily skipped back to headquarters singing the stillbirth of packet switching. Of course, they were right for another 30 years, but packet switching would eventually be trouble for circuit-switched phone networks. Metcalfe got back at them. With the success of its new packet network, ARPA became “D for Defense” DARPA, to remind everyone it was your defense dollars at work, keeping communications alive in the event of a nuclear war. Bob Metcalfe moved from DARPA to the Xerox Palo Alto Research Center. He was playing around with a bunch of new Xerox Alto workstations, trying to devise a fast network both to hook them together, and more importantly, to connect them to laser printers that Xerox was hoping to sell in large numbers.


pages: 290 words: 94,968

Writing on the Wall: Social Media - the First 2,000 Years by Tom Standage

Bill Duvall, British Empire, Edmond Halley, Edward Lloyd's coffeehouse, invention of the printing press, invention of writing, Isaac Newton, knowledge worker, Leonard Kleinrock, Marc Andreessen, Mark Zuckerberg, Menlo Park, Mohammed Bouazizi, New Journalism, packet switching, place-making, Republic of Letters, sexual politics, social intelligence, The Structural Transformation of the Public Sphere, The Wealth of Nations by Adam Smith, theory of mind, yellow journalism

Taylor drew up a detailed plan for the network, proposing that it use a promising new theoretical approach called “packet switching.” Instead of connecting all the machines on the network directly to each other with leased across time and space and the do lines, this involved breaking data down into small, uniform “packets” that could be passed from one machine to another until they reached the appropriate destination. A computer could then talk to a distant machine via several intermediaries, without needing a direct connection. This approach would greatly reduce the number of leased lines needed to interconnect a given number of computers, and would also make efficient use of network capacity by interleaving traffic between multiple sources and destinations. Packet switching was originally proposed as a way to build networks that would keep working in the event of a nuclear attack, because when part of a packet-switching network is disabled, packets can simply be routed around the problem, finding another path to their destination.

Packet switching was originally proposed as a way to build networks that would keep working in the event of a nuclear attack, because when part of a packet-switching network is disabled, packets can simply be routed around the problem, finding another path to their destination. But given the cost and unreliability of network links and computer hardware in the 1960s, computer scientists realized that packet switching was also a good way to build reliable networks for general use. Taylor invited one hundred and forty companies to bid for the contract to build special interface boxes, called “Interface Message Processors” (IMPs), which would be plugged into computers at different sites and linked up by leased lines. Industry giants IBM, the biggest provider of mainframes, and AT&T, America’s telecoms monopoly, declined to bid. Rather than interconnecting and sharing separate computers, IBM imagined a future of ever-larger mainframes, with remote terminals connected by AT&T’s lines, as the best way to bring many users together.

By early December they had both been linked to a third IMP at the University of California, Santa Barbara, and the Stanford IMP had been connected to the fourth IMP at the University of Utah. The packet-switching system meant that users at each of the four sites could access any of the four connected computers, even when there was not a direct link between their carried out in thIQrespective IMPs. (Network traffic between UCLA and the University of Utah, for example, traveled via Stanford or Santa Barbara.) ARPANET was extended to the east coast in March 1970 and continued to grow as more computers, connected by more IMPs, were added to the network. In 1975, when ARPANET was declared fully operational, rather than being an experimental project, there were 57 IMPs, including one across the Atlantic in London. By 1981 there were 213 computers attached to the network, with another being added, on average, every twenty days. In January 1983 the packet-switching protocol used by the IMPs, known as NCP, was retired in favor of a more robust standard called TCP/IP, which had been developed by Robert E.


pages: 287 words: 86,919

Protocol: how control exists after decentralization by Alexander R. Galloway

Ada Lovelace, airport security, Berlin Wall, bioinformatics, Bretton Woods, computer age, Craig Reynolds: boids flock, discovery of DNA, Donald Davies, double helix, Douglas Engelbart, Douglas Engelbart, easy for humans, difficult for computers, Fall of the Berlin Wall, Grace Hopper, Hacker Ethic, informal economy, John Conway, John Markoff, Kevin Kelly, Kickstarter, late capitalism, linear programming, Marshall McLuhan, means of production, Menlo Park, moral panic, mutually assured destruction, Norbert Wiener, old-boy network, packet switching, Panopticon Jeremy Bentham, phenotype, post-industrial society, profit motive, QWERTY keyboard, RAND corporation, Ray Kurzweil, RFC: Request For Comment, Richard Stallman, semantic web, SETI@home, stem cell, Steve Crocker, Steven Levy, Stewart Brand, Ted Nelson, telerobotics, the market place, theory of mind, urban planning, Vannevar Bush, Whole Earth Review, working poor

If we are indeed living in a post-industrial, postmodern, postdemocratic society, how does one account for political agency in situations in which agency appears to be either caught in networks of power or distributed across multiple agencies? By looking closely and carefully at the technical specifications of TCP/IP and DNS, Protocol suggests that power relations are in the process of being transformed in a way that is resonant with the flexibility and constraints of information technology. The Internet is not simply “open” or “closed” but above all a form that is modulated. The very concept of packet-switching demonstrates this on several levels, from the efficiency standards of routing during a download, to the ways in which each individual datagram is tagged for delivery to your email account or hard drive. Information does flow, but it does so in a highly regulated manner. This dual property (regulated flow) is central to Protocol’s analysis of the Internet as a political technology. Isomorphic Biopolitics As a final comment, it is worthwhile to note that the concept of “protocol” is related to a biopolitical production, a production of the possibility for experience in control societies.

In August 1964, he published an eleven-volume memorandum for the Rand Corporation outlining his research.6 Baran’s network was based on a technology called packet-switching7 that allows messages to break themselves apart into small fragments. Each fragment, or packet, is able to find its own way to its destination. Once there, the packets reassemble to create the original message. In 1969, the Advanced Research Projects Agency (ARPA) at the U.S. Department of Defense started the ARPAnet, the first network to use Baran’s packet-switching technology. The ARPAnet allowed academics to share resources and transfer files. In its early years, the ARPAnet (later renamed DARPAnet) existed unnoticed by the outside world, with only a few hundred participating computers, or “hosts.” All addressing for this network was maintained by a single machine located at the Stanford Research Institute in Menlo Park, California. By 1984 the network had grown larger.

The term protocol is most known today in its military context, as a method of correct behavior under a given chain of command. On the Internet, the meaning of protocol is slightly different. In fact, the reason why the Internet would withstand nuclear attack is precisely because its internal protocols are the enemy of bureaucracy, of rigid hierarchy, and of centralization. As I show in this chapter, the material substrate of network protocols is highly flexible, distributed, and resistive of hierarchy. The packet-switching technologies behind the Internet provided a very different “solution” to nuclear attack than did common military protocol during the Cold War. For example, in 1958 the Royal Canadian Air Force and the U.S. Air Force entered into agreement under the North American Aerospace Defense Command (NORAD). NORAD is a radar surveillance system ringing North America that provides early warnings of missile or other air attacks against Canada and the United States.


pages: 528 words: 146,459

Computer: A History of the Information Machine by Martin Campbell-Kelly, William Aspray, Nathan L. Ensmenger, Jeffrey R. Yost

Ada Lovelace, air freight, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Apple's 1984 Super Bowl advert, barriers to entry, Bill Gates: Altair 8800, borderless world, Buckminster Fuller, Build a better mousetrap, Byte Shop, card file, cashless society, cloud computing, combinatorial explosion, computer age, deskilling, don't be evil, Donald Davies, Douglas Engelbart, Douglas Engelbart, Dynabook, fault tolerance, Fellow of the Royal Society, financial independence, Frederick Winslow Taylor, game design, garden city movement, Grace Hopper, informal economy, interchangeable parts, invention of the wheel, Jacquard loom, Jeff Bezos, jimmy wales, John Markoff, John von Neumann, Kickstarter, light touch regulation, linked data, Marc Andreessen, Mark Zuckerberg, Marshall McLuhan, Menlo Park, Mitch Kapor, natural language processing, Network effects, New Journalism, Norbert Wiener, Occupy movement, optical character recognition, packet switching, PageRank, pattern recognition, Pierre-Simon Laplace, pirate software, popular electronics, prediction markets, pre–internet, QWERTY keyboard, RAND corporation, Robert X Cringely, Silicon Valley, Silicon Valley startup, Steve Jobs, Steven Levy, Stewart Brand, Ted Nelson, the market place, Turing machine, Vannevar Bush, Von Neumann architecture, Whole Earth Catalog, William Shockley: the traitorous eight, women in the workforce, young professional

The third problem Roberts faced was how to link together all the computer systems, which came from different manufacturers and used many varieties of operating software that had taken several years to develop. Enough was known about the software crisis at this stage to want to avoid the extensive rewriting of operating systems. Unknown to Roberts, a solution to the first two problems had already been invented. Known as “store-and-forward packet switching,” the idea was first put forward by Paul Baran of the RAND Corporation in 1961 and was independently reinvented in 1965 at the National Physical Laboratory in England by Donald Davies, who coined the term packet switching. Davies recognized the packet-switching concept to be similar to an older telegraph technology. In telegraph networks, engineers had already solved the problem of how to avoid having every city connected to every other. Connectivity was achieved by using a number of switching centers located in major cities.

In the 1930s these manual switching centers were mechanized in “torn-tape offices,” where incoming messages were automatically recorded on perforated paper tape and then retransmitted mechanically. In the 1960s the same functions were being computerized using disk stores instead of paper tape as the storage medium. Store-and-forward packet switching was a simple elaboration of these old telegraph ideas. Instead of having every computer connected to every other, store-and-forward technology would be used to route messages through the network; there would be a single “backbone” communications line that connected the computers together, with other connections being added as the need arose. Packet-switching technology addressed the problem of making economic use of the high-speed communications lines. So that a single user did not monopolize a line, data would be shuttled around the network in packets. A packet was rather like a short telegram, with each packet having the address of the destination.

The computers that acted as the switching centers—called nodes in the Arpanet—would simply receive packets and pass them along to the next node on the route toward the destination. The computer at the destination would be responsible for reconstituting the original message from the packets. In effect, by enabling many users to share a communications line simultaneously, packet switching did for telecommunications what time-sharing had done for computing. All of this was unknown to Roberts until he attended an international meeting of computer network researchers in Gatlinburg, Tennessee, in October 1967. There he learned of the packet-switching concept from one of Donald Davies’s English colleagues. He later described this as a kind of revelation: “Suddenly I learned how to route packets.” The final problem that remained for Roberts was how to avoid the horrendous software problems of getting the different computers to handle the network traffic.


Smart Cities, Digital Nations by Caspar Herzberg

Asian financial crisis, barriers to entry, business climate, business cycle, business process, carbon footprint, clean water, cloud computing, corporate social responsibility, Dean Kamen, demographic dividend, Edward Glaeser, Edward Snowden, hive mind, Internet of things, knowledge economy, Masdar, megacity, New Urbanism, packet switching, QR code, remote working, RFID, rising living standards, risk tolerance, Ronald Reagan, shareholder value, Silicon Valley, Silicon Valley startup, smart cities, Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia, smart meter, social software, special economic zone, Stephen Hawking, telepresence, too big to fail, trade route, transcontinental railway, upwardly mobile, urban planning, urban sprawl, women in the workforce, working poor, X Prize

Many legacy businesses and institutions are forced to operate both circuit-switching and modern packet-switching networks to conduct daily traffic over phone and computer. Packet switching is the future, but it is a technology in transition, having been essential to the development of the cloud and the Internet in the twenty-first century. Because it requires time and expense to transition entirely to packet-switching networks, many organizations will phase out circuit-switching networks. This usually means that voice and data configurations are kept separate, with voice networks remaining on legacy circuit-switching networks even as packet switching is brought in to handle the massive flood of data (which will increase as the Internet of Everything expands). In this case, City of Dreams could jump directly to a single packet-switch network with support for voice and data protocol.


Bootstrapping: Douglas Engelbart, Coevolution, and the Origins of Personal Computing (Writing Science) by Thierry Bardini

Apple II, augmented reality, Bill Duvall, conceptual framework, Donald Davies, Douglas Engelbart, Douglas Engelbart, Dynabook, experimental subject, Grace Hopper, hiring and firing, hypertext link, index card, information retrieval, invention of hypertext, Jaron Lanier, Jeff Rulifson, John von Neumann, knowledge worker, Leonard Kleinrock, Menlo Park, Mother of all demos, new economy, Norbert Wiener, Norman Mailer, packet switching, QWERTY keyboard, Ralph Waldo Emerson, RAND corporation, RFC: Request For Comment, Sapir-Whorf hypothesis, Silicon Valley, Steve Crocker, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, stochastic process, Ted Nelson, the medium is the message, theory of mind, Turing test, unbiased observer, Vannevar Bush, Whole Earth Catalog

Davies, a colleague of Scantleburry's at NPL, in a seminar on time-sharing at MIT in 1965 and had discussed with him and Licklider "networking and the inadequacy of data communication facilities for both time-sharing and networking" (Roberts 1988, 144). After Scantleburry's presentation, Roberts had a discussion with him and considered his suggestion that "packet switching offered a solution to his problem" (Norberg and O'Neill 1996, 166). After his return to Washing- ton, Roberts read Baran's reports on packet switching and initiated contact with him. S In June 1968, Roberts described the ARPANET as a demonstration of the kind of distributed network recommended by Baran in his study. During the winter and spring of 1968, Roberts contracted Elmer Shapiro at SRI, who was only distantly associated with the ARC laboratory, to study the "design and specifications of a computer network."

And finally, the stand-alone workstations of the pre- vious phase are connected into a network. Such a way to describe the evolution of computing focuses on the specific characteristics of the computer at a given time and usually puts the emphasis on a technological innovation that allowed the passage from one phase to the next: the time-sharing operating system, for example, the desktop meta phor of the human-computer interface, or packet switching network technologies. 1 While these innovations obviously contributed greatly to shaping the history of computing, the dynamic of personalization that characterizes the evolution of computing since the late 1940'S played an equally important role. I describe the progressive construction of the user as a person, or, what sometimes amounted to the same thing, how the computer eventually got a personality.

Licklider proposed a network linking the IPTO contractor sites, the main purpose of the network was to be resource sharing, since Licklider and Taylor were concerned about the costs of the multiplication of the infrastruc- ture investments funded by their office (Norberg and O'Neill 1996, 163). This contradicts the often-stated myth of its origin that claims the u.s. Department of Defense wanted a computer network reliable and robust enough to survive a nuclear attack. The effort to achieve the efficiencies of resource sharing drew on technology that made it possible to carry data on leased dedicated phone lines. That technology was packet switching. Robert Taylor had enrolled Larry Roberts to take charge of the ARPANET project, and Roberts started at the IPTO in December 1966, as Taylor's assis- tant director. At the IPTO contractors' meeting at the University of Michigan in Ann Arbor in April 1967, Wes Clark had proposed to organize the network around small computers interfacing the main computer at each site to the com- munication network (Salus 1995,20-21; Hafner and Lyon 1996,72-74).


Howard Rheingold by The Virtual Community Homesteading on the Electronic Frontier-Perseus Books (1993)

Apple II, Brewster Kahle, Buckminster Fuller, commoditize, conceptual framework, Douglas Engelbart, Douglas Engelbart, Electric Kool-Aid Acid Test, experimental subject, George Gilder, global village, Hacker Ethic, Haight Ashbury, Howard Rheingold, HyperCard, John Markoff, Kevin Kelly, knowledge worker, license plate recognition, loose coupling, Marshall McLuhan, Menlo Park, meta analysis, meta-analysis, Mitch Kapor, packet switching, Panopticon Jeremy Bentham, profit motive, RAND corporation, Ray Oldenburg, rent control, RFC: Request For Comment, Ronald Reagan, Saturday Night Live, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, technoutopianism, Ted Nelson, telepresence, The Great Good Place, The Hackers Conference, urban decay, Whole Earth Catalog, Whole Earth Review, young professional

If it could be done through the short wire that connected all the terminals to the central time-shared computer, it could, in theory, be done with a wire as long as a continent. The telecommunications orthodoxy of the 1960s was as pessimistic about the ARPA quest as the computer orthodoxy had been uninterested in interactive computing. The ARPA planners adopted a particular way of sending chunks of computer information over a network, a scheme known as packet-switching. IP Packet-switching is yet another case of a technology invented for one purpose evolving into purposes beyond the intentions of the inventors. It started in the 1950s, when the RAND Corporation performed top-secret studies on thermonuclear war scenarios. 26-04-2012 21:43 howard rheingold's | the virtual community 11 de 43 http://www.rheingold.com/vc/book/3.html They focused on the survivability of the communications system that made command and control possible on local and national levels.

The National Physical Laboratory in Great Britain tested packetswitching principles in practice in 1968. At the same time, ARPA issued a request for proposals (RFP) for a system to link geographically remote research computers into a network. Robert Taylor hired Lawrence Roberts from MIT's Lincoln Laboratory to write the RFP and choose the sites for the first network nodes. Roberts made the decision to use the packet-switching scheme. Robert Kahn, a mathematics professor at MIT, took a leave to work at Bolt, Beranek, and Newman (BBN), a government-funded think tank that ended up designing and running key components of ARPANET. Kahn wrote the proposal that won the first contract from ARPA for BBN. The first node was delivered to UCLA in 26-04-2012 21:43 howard rheingold's | the virtual community 12 de 43 http://www.rheingold.com/vc/book/3.html 1969, and the network expanded to four nodes by the end of the year.

In 1970, Harvard and MIT came online. By the middle of 1971, more than thirty different computers (and their communities) were linked to the network. Many of the people involved in funding and building the first network, such as Robert Taylor and Robert Kahn, are still actively involved in creating the next generation of network technology, more than twenty years later. The significance of packet-switching technology to nontechnologists is twofold . First, this invention creates the building block for a communications system with no central control because you don't need a central controller when each packet and the entire network of routers all know how to get information around. Second, as the world's information becomes digitized, those packets can carry everything that humans can perceive and machines can process--voice, high-fidelity sound, text, high-resolution color graphics, computer programs, data, full-motion video.


Masters of Deception: The Gang That Ruled Cyberspace by Michelle Slatalla, Joshua Quittner

dumpster diving, East Village, Hacker Ethic, hacker house, job automation, Mitch Kapor, packet switching, Stewart Brand, Whole Earth Review

It's the talk of the hacker elite: Phiber Optik got into a feud with Erik Bloodaxe, and to hear Erik Bloodaxe tell it, Phiber Optik lost. Here's how it happened. One day in 1989, while Chris is working on his big hacker project, a directory of the computers on a large data network known as Telenet, the phone rings. The caller is LOD member Mark Abene, up in New York City. Mark is really upset. His account on the NYNEX Packet Switched Network was killed. Can you imagine? Phiber Optik without access to the NYNEX Packet Switched Network. It was like James Dean without a motorcycle. Mark desperately wants to get back in the system, and knows that Chris has a secret route to the computer. Mark asks for it. Now, Chris knows Mark has access to a list of addresses of certain phone company computers that you can reach over Telenet, Chris wants to include those addresses in his directory.

He's intuitively hacking out the most complex programs and commands you can imagine. He's learning new things, going new places every day. By himself. He's the dude. The funny thing about Phiber is, he's so far into the phone system that when he wants to hit a switch, he does it the hard way. He doesn't just dial the switch in question and connect. No, he logs in through something called the NYNEX Packet Switched Network. This network of computers is much more potent than any single switch. In fact, this network ties together every switch in the New York-New England telephone region. Each is one pearl on the necklace and Phiber has his hands on the clasp. But, ironically, he has never possessed a single specific phone number for any one of the switches. Of course, he hasn't exactly been lusting for one.

As they log into the Laurelton switch to start exploring, he describes every command they're typing even the commands they already know in precise, easy-tounderstand language. He knows everything. And Mark is just as excited by this session as they are, because he senses that finally he's met two other hackers who can ride at his pace. For his part, Mark will always think of this evening as "a meeting of the minds. " They forgot who they were, and where they were, and thought only about where they were headed. Mark has shown them how to use the NYNEX Packet Switched Network to jump off into other switches as well, and tonight they traipse around in the Hollis switch system for a while. In earlier phone conversations, Mark has told them different ways he's found to get into phone company computers, and Paul took it all in. So tonight Mark never has to repeat a phone number, never has to explain the meaning of a command to Paul. Mark types it and Paul absorbs it, because the progression of commands on the monitor is distinctly logical.


pages: 378 words: 94,468

Drugs 2.0: The Web Revolution That's Changing How the World Gets High by Mike Power

air freight, Alexander Shulgin, banking crisis, bitcoin, blockchain, Buckminster Fuller, Burning Man, cloud computing, credit crunch, crowdsourcing, death of newspapers, Donald Davies, double helix, Douglas Engelbart, Electric Kool-Aid Acid Test, fiat currency, Firefox, Fractional reserve banking, frictionless, Haight Ashbury, John Bercow, John Markoff, Kevin Kelly, Leonard Kleinrock, means of production, Menlo Park, moral panic, Mother of all demos, Network effects, nuclear paranoia, packet switching, pattern recognition, PIHKAL and TIHKAL, pre–internet, QR code, RAND corporation, Satoshi Nakamoto, selective serotonin reuptake inhibitor (SSRI), sexual politics, Skype, Stephen Hawking, Steve Jobs, Stewart Brand, trade route, Whole Earth Catalog, Zimmermann PGP

In the early 1960s, American computer scientist Leonard Kleinrock of the Massachusetts Institute of Technology and Paul Baran of the Rand Corporation, and, later, Britain’s Donald Davies, a physician at the UK’s National Physical Library in Teddington, independently conceived of the same way to send data around a telephone network efficiently by splitting it into chunks and routing it through nodes around the network to later arrive, reassembled, in the right place. These deliberate first steps towards cyberspace had a greater impact on the history of mankind than the simple stroll on a rock high above our heads two years later. This ‘packet-switching’ concept was to become the central structure in international telecommunications and, later, data networks. Four months after the moon landing, on 29 October 1969, the Advanced Research Projects Agency Network (ARPANET), the world’s first packet-switching data network, consisting of four computers in separate university sites, jumped into life. The first message ever sent was meant to say ‘login’, but the system crashed, and the first word ever sent from one computer to another was the accidentally portentous ‘Lo’.

ARPANET is often described as the birth of the internet, and is equally often reported to have been designed to survive a thermonuclear strike, meaning that if one node or cell of the network were destroyed, the others would gather the digital slack and reroute the information around the surviving nodes. However, the aim of ARPANET was not to preserve national security in the event of warfare, but to allow university researchers separated by geography to share information; the net’s roots were indisputably collaborative and altruistic. Its technological cornerstone – the packet-switching network – underpinned all the later digital developments that would enable the reeling madness and quotidian mundanity that comprises a day online today – a day that includes buying groceries, paying bills, sharing photos and ideas, updating the world on your latest hairstyle choices, and, for many more people than is currently acknowledged, talking about and buying drugs. Few involved in the early days of the internet could ever have imagined how central to billions of people’s lives it was to become, but some of them dreamed of it.

Wikipedia soon became an essential resource for those looking for information about new drugs, and the site started publishing entries on the drugs along with their Chemical Abstracts Service (CAS) numbers – the unique identifying code that serves as a chemical Dewey Decimal system. With this information, would-be vendors or users could easily find chemical firms that carried the compounds they wanted. In 2003 MySpace opened the web to a whole generation of teenagers, to whom the concepts of packet-switching were as alien as the concept of not using the net as their first port of call for entertainment or communication. The site, with its super-vernacular design, clumsy layouts and clashing colours, was as riotous and impenetrable as any poster-adorned bedroom wall of previous eras. It wasn’t until late 2004 that the phrase Web 2.0 was officially coined by technologist Tim O’Reilly, who correctly identified the future of the web – it would become a model driven by user-generated content, mass collaboration, global sharing and cross-border participation.2 That year, The Facebook, a web version of the college yearbook popular at American universities, launched.


pages: 326 words: 103,170

The Seventh Sense: Power, Fortune, and Survival in the Age of Networks by Joshua Cooper Ramo

Airbnb, Albert Einstein, algorithmic trading, barriers to entry, Berlin Wall, bitcoin, British Empire, cloud computing, crowdsourcing, Danny Hillis, defense in depth, Deng Xiaoping, drone strike, Edward Snowden, Fall of the Berlin Wall, Firefox, Google Chrome, income inequality, Isaac Newton, Jeff Bezos, job automation, Joi Ito, market bubble, Menlo Park, Metcalfe’s law, Mitch Kapor, natural language processing, Network effects, Norbert Wiener, Oculus Rift, packet switching, Paul Graham, price stability, quantitative easing, RAND corporation, recommendation engine, Republic of Letters, Richard Feynman, road to serfdom, Robert Metcalfe, Sand Hill Road, secular stagnation, self-driving car, Silicon Valley, Skype, Snapchat, social web, sovereign wealth fund, Steve Jobs, Steve Wozniak, Stewart Brand, Stuxnet, superintelligent machines, technological singularity, The Coming Technological Singularity, The Wealth of Nations by Adam Smith, too big to fail, Vernor Vinge, zero day

So Baran expected a friendly reception. After all, he’d be telling a bunch of men with a uranium death sentence that he’d found a way to get them off the Soviet target plan. His new “mesh” network would mean that bombing AT&T would be largely pointless. It wouldn’t blind U.S. commanders. If only they’d redesign their network, the AT&T engineers might save their own lives. They thought he was insane. “I tried to explain packet switching to a senior telephone company executive. In midsentence he interrupted me,” Baran recalled. “The old analog engineer looked stunned. He looked at his colleagues in the room while his eyeballs rolled up, sending a signal of his utter disbelief. He paused for a while, and then said, ‘Son, here’s how a telephone works.’” Of course Paul Baran knew how a telephone worked. You jacked one point to a switch to another point.

Other scientists had been chasing the idea as well, but the design suited Baran’s problem particularly well: a network with no central control, survivable, uncuttable. The earliest large network built on Baran’s principles became known as ARPANET, the Advanced Research Projects Agency Network—a mesh of connections that even today serves as the backbone for parts of the Internet. Even with the risk of nuclear war hopefully long gone, packet-switching designs of one sort or another still account for most of the data moving in the world. Think of how true an idea must be to endure more than fifty years of technological change. And all the efficiencies Baran first predicted fifty years ago are still at work. Every time you make a call, share a video, or ask a machine to think for you, that transaction likely takes place through fishnet-routed packets.

Data flows could be monitored with the ease of watching a subway turnstile. The far-flung, wild creativity of our plug-and-play, connected world would be stifled. Each additional connection to the system would demand bureaucratic, centralized approval by the Switch Despots, concerned more with their own power than with our survival. Instead we have a slice-resistant mesh that has grown a billion times over, with its original architecture largely intact. Packet-switched systems such as the Internet give anyone with some string and an ability to tie knots (which, in techno-speak, is anyone with some blinking fiber-optic cables and a TCP/IP connection) the power to weave themselves into the global web. This is why you can so easily turn on your phone or tablet and more or less instantly touch a whole world of data. Every minute now, an additional ten thousand devices get connected to the Internet—not just wired citizens, smartphones, laptops, and tablets but also medical tools, Bitcoin mines, and airplane diagnostic systems.


pages: 268 words: 76,702

The System: Who Owns the Internet, and How It Owns Us by James Ball

Bill Duvall, bitcoin, blockchain, Chelsea Manning, cryptocurrency, don't be evil, Donald Trump, Douglas Engelbart, Edward Snowden, en.wikipedia.org, Firefox, Frank Gehry, Internet of things, invention of movable type, Jeff Bezos, jimmy wales, Julian Assange, Kickstarter, Leonard Kleinrock, Marc Andreessen, Mark Zuckerberg, Menlo Park, Minecraft, Mother of all demos, move fast and break things, move fast and break things, Network effects, Oculus Rift, packet switching, patent troll, Peter Thiel, pre–internet, ransomware, RFC: Request For Comment, risk tolerance, Ronald Reagan, Rubik’s Cube, self-driving car, Shoshana Zuboff, Silicon Valley, Silicon Valley startup, Skype, Snapchat, Steve Crocker, Stuxnet, The Chicago School, undersea cable, uranium enrichment, WikiLeaks, yield management, zero day

One man with a good idea of the answer was Steve Lukasik, who – as first deputy director, and then director of ARPA – was the man who signed the cheques for most of the early days of ARPANET, and who in a later reflection described the project as a ‘high-risk gamble’ suitable to the agency’s mandate, and one which definitely had ‘unexpected results’.13 Unsurprisingly, according to Lukasik’s account ARPA was not especially concerned with computer use at a handful of US universities. The problem the agency was grappling with was command and control – partly for the US military as a whole, but primarily for the country’s Cold War nuclear arsenal. Packet switching provided an opportunity for the nuclear deterrent: if its promise of splitting up signals so they could be sent dynamically across the country, or across the world, even if some pathways were disrupted or destroyed, was confirmed, that would have huge potential. Even if most of the connections were destroyed by an adversary, signals could still get through if some remained – exactly what you want from a weapons system of last resort.

By 1975, DARPA handed official government oversight of the ARPANET over to the Defense Communications Agency, who attempted to prevent unauthorised access to the network, despite it having no built-in tools to track accesses or use. Eventually, the military aspects of the network – parts which later became the basis for modern secure networks for diplomatic and intelligence information – were split from ARPANET, which was then wound down as its successor, the internet we know now, supplanted it. ‘ARPA’s intent had been to demonstrate the utility of packet switching for military command and control, and in that it succeeded brilliantly,’ Lukasik’s paper ruefully notes. ‘As a consequence unanticipated by its sponsor, the ARPANET and its successor, the Internet, further demonstrated the general utility of networking for “command and control” far beyond the needs of the DoD.’ Overall, Lukasik seems unsure about the long-term results of the project he funded.

noredirect=on&utm_term=.e7adba67bfe6 3https://www.bbc.co.uk/news/business-42745853 1 THE ARCHITECTS 1US broadband speed taken from http://fortune.com/2017/06/02/internet-speed-akamai-survey/ 2The narrative of the first internet message is taken from this (charming and very readable) transcript: https://archive.icann.org/meetings/losangeles2014/en/schedule/mon-crocker-kleinrock/transcript-crocker-kleinrock-13oct14-en.pdf 3https://www.internethalloffame.org//inductees/steve-crocker 4https://ai.google/research/people/author32412 5Wired have a great feature with much more detail on ‘the mother of all demos’ here: https://www.wired.com/2010/12/1209computer-mouse-mother-of-all-demos/ 6This was the recollection of Bob Taylor, who secured the funding (https://www.computer.org/csdl/magazine/an/2011/03/man2011030004/13rRUxly9fL), but was disputed by Charles Herzfeld, who said he had agreed the funding, but had taken more than twenty minutes’ persuasion (https://www.wired.com/2012/08/herzfeld/). 7Full video and transcript: http://opentranscripts.org/transcript/steve-crocker-internet-hall-fame-2012-profile/ 8This is also from Kleinrock’s 2014 transcript: https://archive.icann.org/meetings/losangeles2014/en/schedule/mon-crocker-kleinrock/transcript-crocker-kleinrock-13oct14-en.pdf 9https://www.ietf.org/rfc/rfc0675.txt 10ARPANET had operated as a packet switching network from its inception – TCP is just a specific implantation of the concept, and the one which came to be the standard. 11This paragraph borrows key dates from https://www.webfx.com/blog/web-design/the-history-of-the-internet-in-a-nutshell/ 12Everything from Steve Lukasik comes from his paper ‘Why the ARPANET Was Built’, published online here: https://www.academia.edu/34728504/WHY_THE_ARPANET_WAS_BUILT 13This is from the Crocker/Kleinrock discussion. 14https://webfoundation.org/about/vision/history-of-the-web/ 15These are sourced to the Internet Services Consortium, but most easily viewed on Wikipedia https://en.wikipedia.org/wiki/Global_Internet_usage 16https://www.bbc.co.uk/news/technology-32884867 17https://www.statista.com/statistics/471264/iot-number-of-connected-devices-worldwide/ 18This stat comes from TeleGeography (https://www2.telegeography.com/submarine-cable-faqs-frequently-asked-questions) – their map of the main undersea internet cables is well worth a look: https://www.submarinecablemap.com/ 2 THE CABLE GUYS 1http://www.washingtonpost.com/wp-dyn/content/article/2007/10/17/AR2007101702359.html?


pages: 302 words: 74,350

I Hate the Internet: A Novel by Jarett Kobek

Anne Wojcicki, Burning Man, disruptive innovation, East Village, Edward Snowden, Golden Gate Park, Google bus, Google Glasses, Google X / Alphabet X, immigration reform, indoor plumbing, informal economy, Jeff Bezos, liberation theology, Mark Zuckerberg, MITM: man-in-the-middle, Norman Mailer, nuclear winter, packet switching, PageRank, Peter Thiel, quantitative easing, Ray Kurzweil, rent control, Ronald Reagan, Silicon Valley, Steve Jobs, technological singularity, Triangle Shirtwaist Factory, union organizing, V2 rocket, Vernor Vinge, wage slave, Whole Earth Catalog

Not if you use the Internet! “Women must develop their own Internet! They must group together and engineer a new, gynocentric Internet and they must exclude all the stupid assumptions of men in its implementation and design! They must not repeat its mistakes! No bullshit about freedom of speech, no bullshit about individual liberties, no bullshit reimagining of juvenile literature! No IPv4! No packet switching! Packet switching is the root of all evil! When women have finished engineering their own Internet, they must ban men from it! For at least ten years until the bugs are worked out!” J. Karacehennem stopped talking. “That’s all I’ve got,” he said. “How do you feel?” “It didn’t really do much. I guess it was worth trying.” One of the tourists walked up to J. Karacehennem. She was a young woman, maybe twenty-two years old.

Other than Apple, the primary revenue stream of every other company is advertising. There is no way to make money off the Internet itself other than advertising. We are living in the biggest advertising economy that the world has ever seen, and no one will admit it.” “I have a theory,” said J. Karacehennem, “That all money and technology is embedded with the ideology of its origin. You should Google ‘packet switching.’ It will explain everything. Advertising itself explains why everyone in the Bay Area is so full of shit and no one can tell the truth.” “Because they are advertisers,” said Christine. “They can’t say that they work in advertising. So they lie about what they do. Google wants us to believe that they’re changing the world and offering a million services for free and that we’re all part of the same team, but they’re lying.

Men are the shit of the world and all of our political systems and philosophies were created and devised without the input of women! Half of the world’s population lives beneath systems of government and technological innovation into which their gender had zero input! Democracy is a bullshit ideology that a bunch of slaveholding Greek men constructed between rounds of beating their wives! All the presumed ideologies of men were taken for inescapable actualities and designed into the Internet! Packet switching is an incredible evil! “The Internet is the last stand of the Patriarchy. It was designed by warmongering men to systematically dehumanize women! The whole thing is fucked! It’s where straight men are hosting their final battle! They’ve discovered the grim truth of their own obsolescence! They lost control of the complex systems we call society, so they created a new one! A new one where they could play by their own rules!


Smart Mobs: The Next Social Revolution by Howard Rheingold

A Pattern Language, augmented reality, barriers to entry, battle of ideas, Brewster Kahle, Burning Man, business climate, citizen journalism, computer vision, conceptual framework, creative destruction, Douglas Engelbart, Douglas Engelbart, experimental economics, experimental subject, Extropian, Hacker Ethic, Hedy Lamarr / George Antheil, Howard Rheingold, invention of the telephone, inventory management, John Markoff, John von Neumann, Joi Ito, Joseph Schumpeter, Kevin Kelly, Metcalfe's law, Metcalfe’s law, more computing power than Apollo, New Urbanism, Norbert Wiener, packet switching, Panopticon Jeremy Bentham, pattern recognition, peer-to-peer, peer-to-peer model, pez dispenser, planetary scale, pre–internet, prisoner's dilemma, RAND corporation, recommendation engine, Renaissance Technologies, RFID, Richard Stallman, Robert Metcalfe, Robert X Cringely, Ronald Coase, Search for Extraterrestrial Intelligence, SETI@home, sharing economy, Silicon Valley, skunkworks, slashdot, social intelligence, spectrum auction, Steven Levy, Stewart Brand, the scientific method, transaction costs, ultimatum game, urban planning, web of trust, Whole Earth Review, zero-sum game

The attack on the World Trade Center on September 11, 2001, influenced American attitudes about the devices Americans call “cell phones.” First came dramatic reports that people trapped in burning buildings and hijacked airplanes had used their personal telephones to call their families.75 Then came reports that two-way text pagers, which made use of a packet-switched data network, remained useful as a lifeline when most voice calls became impossible in Lower Manhattan.76 (Packet-switching had been invented originally as a means of communicating during a nuclear war.77) As more Americans start using mobile phones and two-way messaging for safety in the United States, research from Scandinavia suggests that they will quickly adopt the devices for social communication.78 Although it isn’t clear yet which company will become the IBM, Microsoft, or AOL of the wireless Internet, or whether American users will grow a mainstream texting culture, it’s clear that mobile telephony, texting, and mobile Internet services are already affecting social relationships.

The unexpected success of texting was also a sign that people were once again appropriating a communication technology for social purposes, as they had done with voice telephony and with the Minitel in France, where the chat tool was literally stolen from operators by the users, and with email, where it was the driving force behind the growth of the landlocked Internet.49 A technical and economic advantage of text messaging is that it is “packet-switched” rather than “circuit-switched.” This technical distinction divides the telegraph-telephone era analog network from the Internet and mobile era digital network. Circuit-switched telephone connections require a series of physical switches to link a continuous wired circuit between both parties—think of early twentieth-century films of operators who closed those circuits by plugging jacks into a switchboard.

If you video record your child’s first steps, and a Mickey Mouse cartoon is playing on a TV in the background, the cop chip will shut down your recorder. If you talk on your mobile phone while walking down the street and somebody drives by with their window open and their car radio is playing a copyrighted song, the cop chip will shut down your phone. Finally, Hollywood is calling for a redesign of the Internet to stop p2p file sharing, which amounts to a proposed ban on decentralized packet switching in favor of centralized networks that can be monitored for acts of infringement.62 Recent legal and regulatory actions are the first moves of a thus far successful campaign to lock down the formerly freewheeling Internet and return to the days of three television networks and one telephone company, when customers were consumers and no one sliced into profits with their own businesses or challenged old technologies with new ones.63 This time, the dinosaurs are well aware of the dangers from the mammals and are taking big thumping steps to protect themselves.


pages: 361 words: 81,068

The Internet Is Not the Answer by Andrew Keen

"Robert Solow", 3D printing, A Declaration of the Independence of Cyberspace, Airbnb, AltaVista, Andrew Keen, augmented reality, Bay Area Rapid Transit, Berlin Wall, bitcoin, Black Swan, Bob Geldof, Burning Man, Cass Sunstein, citizen journalism, Clayton Christensen, clean water, cloud computing, collective bargaining, Colonization of Mars, computer age, connected car, creative destruction, cuban missile crisis, David Brooks, disintermediation, disruptive innovation, Donald Davies, Downton Abbey, Edward Snowden, Elon Musk, Erik Brynjolfsson, Fall of the Berlin Wall, Filter Bubble, Francis Fukuyama: the end of history, Frank Gehry, Frederick Winslow Taylor, frictionless, full employment, future of work, gig economy, global village, Google bus, Google Glasses, Hacker Ethic, happiness index / gross national happiness, income inequality, index card, informal economy, information trail, Innovator's Dilemma, Internet of things, Isaac Newton, Jaron Lanier, Jeff Bezos, job automation, Joi Ito, Joseph Schumpeter, Julian Assange, Kevin Kelly, Kickstarter, Kodak vs Instagram, Lean Startup, libertarian paternalism, lifelogging, Lyft, Marc Andreessen, Mark Zuckerberg, Marshall McLuhan, Martin Wolf, Metcalfe’s law, move fast and break things, move fast and break things, Nate Silver, Nelson Mandela, Network effects, new economy, Nicholas Carr, nonsequential writing, Norbert Wiener, Norman Mailer, Occupy movement, packet switching, PageRank, Panopticon Jeremy Bentham, Paul Graham, peer-to-peer, peer-to-peer rental, Peter Thiel, plutocrats, Plutocrats, Potemkin village, precariat, pre–internet, RAND corporation, Ray Kurzweil, ride hailing / ride sharing, Robert Metcalfe, Second Machine Age, self-driving car, sharing economy, Silicon Valley, Silicon Valley ideology, Skype, smart cities, Snapchat, social web, South of Market, San Francisco, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, TaskRabbit, Ted Nelson, telemarketer, The Future of Employment, the medium is the message, the new new thing, Thomas L Friedman, Travis Kalanick, Tyler Cowen: Great Stagnation, Uber for X, uber lyft, urban planning, Vannevar Bush, Whole Earth Catalog, WikiLeaks, winner-take-all economy, working poor, Y Combinator

The second revolutionary aspect of Baran’s survivable system was its method for communicating information from computer to computer. Rather than sending a single message, Baran’s new system broke up this content into many digital pieces, flooding the network with what he called “message blocks,” which would travel arbitrarily across its many nodes and be reassembled by the receiving computer into readable form. Coined as “packet switching” by Donald Davies, a government-funded information scientist at Britain’s National Physical Laboratory, who had serendipitously been working on a remarkably similar set of ideas, the technology was driven by a process Baran called “hot potato routing,” which rapidly sent packets of information from node to node, guaranteeing the security of the message from spies. “We shape our tools and thereafter our tools shape us,” McLuhan said.

We already know how to do it,” Taylor promised. “Great idea,” Herzfeld said. “Get it going. You’ve got a million dollars more in your budget right now. Go.”30 And Taylor did indeed get it going. He assembled a team of engineers including Paul Baran and Wesley Clark, the programmer who had gotten J. C. R. Licklider hooked on the TX-2 computer back in the fifties. Relying on Baran’s distributed packet-switching technology, the team developed a plan to develop a trial network of four sites—UCLA, Stanford Research Institute (SRI), the University of Utah, and the University of California, Santa Barbara. They were linked together by something called an Interface Message Processor (IMP), which today we call routers—those little boxes with blinking lights that connect up the networked devices in our homes.

“Not true,” Taylor interrupted, insisting that the “Internet’s roots” lay with the ARPANET.31 Both Taylor and Kahn are, in a sense, correct. The Internet would never have been built without ARPANET. Growing from its four original IMPs in 1969, it reached 29 by 1972, 57 by 1975, and 213 IMPs by 1981 before it was shut down and replaced as the Internet’s backbone by the National Science Foundation Network (NSFNET) in 1985. But the problem was that ARPANET’s success led to the creation of other packet-switching networks—such as the commercial TELENET, the French CYCLADES, the radio-based PRNET, and the satellite network SATNET—which complicated internetworked communication. So Kahn was right. ARPANET wasn’t the Internet. And he was right, too, about TCP/IP, the two protocols that finally realized Licklider’s dream of an intergalactic computer network. Bob Kahn and Vint Cerf met at UCLA in 1970 while working on the ARPANET project.


pages: 297 words: 83,651

The Twittering Machine by Richard Seymour

4chan, anti-communist, augmented reality, Bernie Sanders, Cal Newport, Cass Sunstein, Chelsea Manning, citizen journalism, colonial rule, correlation does not imply causation, credit crunch, crowdsourcing, don't be evil, Donald Trump, Elon Musk, Erik Brynjolfsson, Filter Bubble, Google Chrome, Google Earth, hive mind, informal economy, Internet of things, invention of movable type, invention of writing, Jaron Lanier, Jony Ive, Kevin Kelly, knowledge economy, late capitalism, liberal capitalism, Mark Zuckerberg, Marshall McLuhan, meta analysis, meta-analysis, Mohammed Bouazizi, moral panic, move fast and break things, move fast and break things, Network effects, new economy, packet switching, patent troll, Philip Mirowski, post scarcity, post-industrial society, RAND corporation, Rat Park, rent-seeking, replication crisis, sentiment analysis, Shoshana Zuboff, Silicon Valley, Silicon Valley ideology, smart cities, Snapchat, Steve Jobs, Stewart Brand, Stuxnet, TaskRabbit, technoutopianism, the scientific method, Tim Cook: Apple, undersea cable, upwardly mobile, white flight, Whole Earth Catalog, WikiLeaks

For example, Arpanet used a distributed network, rather than a centralized information system. It implemented a form of packet-switching, wherein a message is broken into bits of data, distributed over optimal routes and reassembled at destination, and which is still used today in the foundational protocols of the internet. These systems were chosen, in part, for their military virtue. An alluring myth of the internet’s origins has it that it was essentially invented by Paul Baran, of the RAND Corporation, as a way for communications to survive nuclear war.11 The Arpanet system was actually designed separately, without Baran’s direct involvement. Nonetheless, it used remarkably similar ideas, and Baran was one of the major inventors of the distributed network and the packet-switching method. The underlying idea for a ‘distributed network’ of writing, published in a 1964 article, was that in the event of a nuclear strike, the communications system would best survive if it wasn’t centralized.12 This necessitated plenty of redundancy in the network.

It was desperate to outdo Britain and Germany by modernizing the French economy first.14 The Gaullist state was ploughing money into advanced technological research, to update its telecommunications system. In 1973, engineers at the Institut de Recherche en lnformatique et en Automatique had developed a network to rival Arpanet: CYCLADES. It was based on similar practices of decentralized networking. And it deployed its own version of packet-switching, using the ‘datagram’ invented by Louis Pouzin, which had been a major influence on the Arpanet design. The first CYCLADES terminal with television and keyboard calling was made public in 1974. This combination of telephone and computer was the earliest echo of what would later be called, with official gusto, ‘telematics’. The public sector threw its weight behind the development of this system.

Gérard Théry, the French director general of telecommunications, drew up plans to develop a ‘telephone for all’ system as the infrastructural basis for ‘computing for all’.15 In 1978, a government report anticipated the ‘computerization of society’ and called for state investment to expedite the future. CYCLADES was ultimately defeated, however, by the internal politics of the public sector. France’s telecommunications department, Postes, télégraphes et téléphones (PTT), had been developing its own system, Transpac. Transpac used, instead of packet-switching, a circuit-switched network. Circuit-switching is a much older system initially designed in 1878 for handling phone calls, using dedicated point-to-point connections for the duration of the call. At the insistence of the PTT, CYCLADES was defunded and the Transpac system was the basis for the emerging videotex service, Minitel. Minitel was pioneered in 1981, in a small experiment in Velizy, connecting 2,500 homes with an experimental range of services.


pages: 184 words: 53,625

Future Perfect: The Case for Progress in a Networked Age by Steven Johnson

Airbus A320, airport security, algorithmic trading, banking crisis, barriers to entry, Bernie Sanders, call centre, Captain Sullenberger Hudson, Cass Sunstein, Charles Lindbergh, cognitive dissonance, credit crunch, crowdsourcing, dark matter, Dava Sobel, David Brooks, Donald Davies, future of journalism, hive mind, Howard Rheingold, HyperCard, Jane Jacobs, John Gruber, John Harrison: Longitude, Joi Ito, Kevin Kelly, Kickstarter, lone genius, Mark Zuckerberg, mega-rich, meta analysis, meta-analysis, Naomi Klein, Nate Silver, Occupy movement, packet switching, peer-to-peer, Peter Thiel, planetary scale, pre–internet, RAND corporation, risk tolerance, shareholder value, Silicon Valley, Silicon Valley startup, social graph, Steve Jobs, Steven Pinker, Stewart Brand, The Death and Life of Great American Cities, Tim Cook: Apple, urban planning, US Airways Flight 1549, WikiLeaks, William Langewiesche, working poor, X Prize, your tax dollars at work

A few years later, the Welsh computer scientist Donald Davies hit upon a similar scheme, independent of Baran. He anointed the message fragments with the slightly more Anglo name of “packets,” and the general approach “packet switching.” The metaphors stuck. Today, the vast majority of data circling around the globe comes in the form of message fragments that we still call packets. Years after both Baran and Davies had published their seminal papers, Davies jokingly said to Baran, “Well, you may have got there first, but I got the name.” In the late 1960s, packet switching became the foundation of ARPANET, the research network that laid the groundwork for the Internet. The ARPANET design relied on several radical principles that broke with existing computing paradigms. ARPANET was what we would now call a peer-to-peer network, as opposed to a client-server or mainframe-terminal network.

See also 311 system open exchange of information collaboration, 29, 118, 140, 190–192, 209, 213 as defining feature of Internet, 119 in human prehistory, 209 incentives for, 138–140, 190–192 information productivity, 92–95 inhibition by patents, 129–131, 138 in peer networks, 26, 92, 131 positive outcomes, 24–25, 49–50, 121–122 in Renaissance trading towns, 27–28 in social architecture of Web, 47 undesirable consequences, 109–114, 120, 121, 122 Orteig Prize, 147–148 packet switching, 13–14 Page, Scott E., 98 paleolithic-era social networks, 208–209 patents decline in U.S. share, 184 disallowance of, in prize-backed challenges, 129, 135, 138–140 in industrial capitalism, 129–131 versus normal market forces, 137 peer progressive views on, 130–131, 132, 137–138 peer-to-patent review process, 132–133 on pharmaceuticals, 136–138 in profit incentive, 136–137 Paul, Ron, 202, 207 peer networks.


pages: 105 words: 34,444

The Open Revolution: New Rules for a New World by Rufus Pollock

Airbnb, discovery of penicillin, Donald Davies, Donald Trump, double helix, Hush-A-Phone, informal economy, Internet of things, invention of the wheel, Isaac Newton, Kickstarter, Live Aid, openstreetmap, packet switching, RAND corporation, Richard Stallman, software patent, speech recognition

Suddenly, he controlled a bigger budget for computer science research than the combined budgets of all other such efforts in America, and he used it to fund some of the most imaginative blue-sky research of the time. Central to his vision was the idea that if computers were truly to enhance human thinking, ways had to be found to communicate with them and between them. The earliest glimmerings of the possibilities of networked computers had been identified. Licklider stepped down after two years at ARPA, but his ideas gathered momentum thanks not only to his successor, Bob Taylor, but to Paul Baran’s packet-switching ideas at Rand Corporation, and to many others. In August 1968, the tender went out to build the first prototype implementation – to be called the Arpanet – which a few years later became the seedling of the internet we know today. The contract to build it went to a small consulting firm with a reputation for brilliance and informality, Bolt, Beranek & Newman (BBN), founded by Licklider’s old Harvard colleague, Leo Beranek.

Thanks to careful nurturing and a strong base in academia, it was resilient, and by the mid 1980s it was poised to take over the world. How was this possible? How could the internet be so different? Much of the credit must go to the fact of government funding. The beginnings of the internet were almost entirely paid for by government research funds in the US and to a much smaller extent in the UK (where work at the National Physical Laboratory under Donald Davies was crucial to the development of packet-switching). Even more important was the form of the government funding. Today, ARPA is a legend of what is possible for a public agency. It was staffed by outsiders and free to make bold bets with a minimum of bureaucracy. Funding from ARPA helped to create not only the internet but other aspects of digital life that we now take for granted, from user interfaces to the mouse. This was money with a vision, and a vision that something unprecedented was possible.


pages: 352 words: 120,202

Tools for Thought: The History and Future of Mind-Expanding Technology by Howard Rheingold

Ada Lovelace, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, card file, cellular automata, Claude Shannon: information theory, combinatorial explosion, computer age, conceptual framework, Conway's Game of Life, Douglas Engelbart, Dynabook, experimental subject, Hacker Ethic, Howard Rheingold, interchangeable parts, invention of movable type, invention of the printing press, Jacquard loom, John von Neumann, knowledge worker, Marshall McLuhan, Menlo Park, Norbert Wiener, packet switching, pattern recognition, popular electronics, post-industrial society, RAND corporation, Robert Metcalfe, Silicon Valley, speech recognition, Steve Jobs, Steve Wozniak, Stewart Brand, Ted Nelson, telemarketer, Turing machine, Turing test, Vannevar Bush, Von Neumann architecture

This scheme embeds local networks in more global supernetworks. Today's network technologies use the packet-switching techniques originally developed during the creation of the ARPAnet -- exactly the kind of coding of information that Shannon predicted in 1948. Information is transported and processed in packets of information -- bursts of coded on-off pulses -- that carry, in addition to the core data of the message, information on how the message is to be transmitted and received. If your computer uses the right kind of hardware and software translators, your data will find its own way through the network according to the control and routing information embedded in the packets. The technical details of packet switching won't matter to the vast majority of the people who will end up using network systems in the future, but the notion of "distributed computing" signals an important change to a new phase in the evolution of computation.

Instead of a huge host computer in the center of it all that received a stream of information from one computer, translated the stream into a form that could be decoded by another computer, and relayed the translated information to the receiving computer, the smaller imps at each node would accept and pass along information packets that had been translated into a common format by the imp connected to the originating computer. The controlling agent in a "packet switched" network like the ARPAnet was not connected to a central computer somewhere, nor even the "message processors" that mediated between the computers, but the packets of information, the messages themselves. Like the addresses on letters, the code by which information was packaged for transmission put into each packet all the information necessary for getting the message from origin to destination, and for translating between different kinds of computers and computer languages.

But by 1971, when Taylor recruited fifty or sixty of the best people in the field for the Computer Science Laboratory at PARC, the cream of the interactive computer designers had enough engineering and software research behind them from the time-sharing and ARPAnet projects to make them confident that such a utopian scenario might be possible -- especially if a corporation with the resources of Xerox was willing to take the high-stakes gamble. The people who built the first interactive, multiaccess computers, the first intellectual augmentation systems, and the first packet-switching computer networks were gathering under the same roof for the first time, in order to turn those dreams into prototypes as soon as possible. Butler Lampson, Chuck Thacker, Jim Mitchell, Ed McCreight, Bob Sproull, Jim Morris, Chuck Geschke, Alan Kay, Bob Metcalfe, Peter Deutsch, Bill English -- to those who knew anything about the esoteric world of computer design, the PARC computer science founders constituted an unprecedented collection of talents.


pages: 903 words: 235,753

The Stack: On Software and Sovereignty by Benjamin H. Bratton

1960s counterculture, 3D printing, 4chan, Ada Lovelace, additive manufacturing, airport security, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, algorithmic trading, Amazon Mechanical Turk, Amazon Web Services, augmented reality, autonomous vehicles, basic income, Benevolent Dictator For Life (BDFL), Berlin Wall, bioinformatics, bitcoin, blockchain, Buckminster Fuller, Burning Man, call centre, carbon footprint, carbon-based life, Cass Sunstein, Celebration, Florida, charter city, clean water, cloud computing, connected car, corporate governance, crowdsourcing, cryptocurrency, dark matter, David Graeber, deglobalization, dematerialisation, disintermediation, distributed generation, don't be evil, Douglas Engelbart, Douglas Engelbart, Edward Snowden, Elon Musk, en.wikipedia.org, Eratosthenes, Ethereum, ethereum blockchain, facts on the ground, Flash crash, Frank Gehry, Frederick Winslow Taylor, future of work, Georg Cantor, gig economy, global supply chain, Google Earth, Google Glasses, Guggenheim Bilbao, High speed trading, Hyperloop, illegal immigration, industrial robot, information retrieval, Intergovernmental Panel on Climate Change (IPCC), intermodal, Internet of things, invisible hand, Jacob Appelbaum, Jaron Lanier, Joan Didion, John Markoff, Joi Ito, Jony Ive, Julian Assange, Khan Academy, liberal capitalism, lifelogging, linked data, Mark Zuckerberg, market fundamentalism, Marshall McLuhan, Masdar, McMansion, means of production, megacity, megastructure, Menlo Park, Minecraft, MITM: man-in-the-middle, Monroe Doctrine, Network effects, new economy, offshore financial centre, oil shale / tar sands, packet switching, PageRank, pattern recognition, peak oil, peer-to-peer, performance metric, personalized medicine, Peter Eisenman, Peter Thiel, phenotype, Philip Mirowski, Pierre-Simon Laplace, place-making, planetary scale, RAND corporation, recommendation engine, reserve currency, RFID, Robert Bork, Sand Hill Road, self-driving car, semantic web, sharing economy, Silicon Valley, Silicon Valley ideology, Slavoj Žižek, smart cities, smart grid, smart meter, social graph, software studies, South China Sea, sovereign wealth fund, special economic zone, spectrum auction, Startup school, statistical arbitrage, Steve Jobs, Steven Levy, Stewart Brand, Stuxnet, Superbowl ad, supply-chain management, supply-chain management software, TaskRabbit, the built environment, The Chicago School, the scientific method, Torches of Freedom, transaction costs, Turing complete, Turing machine, Turing test, undersea cable, universal basic income, urban planning, Vernor Vinge, Washington Consensus, web application, Westphalian system, WikiLeaks, working poor, Y Combinator

The standards wars of this era divided phone companies, which preferred a system that would support discrete circuits between one sender and receiver, like older telephony networks, versus many computing companies, such as IBM, which lobbied hard for packet switching technologies that could treat all messages (e.g., voice, data, image) as recombinant bits flowing over whatever future hardware that could connect with the network. The models of communication (equally technical and social) posed by both options contain profound downstream implications for the geopolitics of an information society. A polity of circuits and a polity of packets are in epistemological and functional opposition. For the circuit model, its stack is a bounded utility for which use is metered by monopolistic caretakers who, by guaranteeing the circuit between sender and receiver, retain de facto sovereignty over the channel. For the packet switching model, at least in the minds of Cerf's group, the platform would prioritize the edges of the network, asking them to do more of the work to reassemble transmitted packets and calculate the content of messages.

We appreciate the role of railroads, telegraphy, and telephony networks as the infrastructure of globalization, and their speed for the acceleration of the modernities of space and time, but perhaps we underappreciate the metastructuring importance of mundane anonymous standards to turn isolated mechanical inventions into infrastructural innovations (e.g., railroad gauges and spike lengths, timetable templates, the semiotics of graphical interface feedback conventions, transmission line materials, arbitrary telegraphic languages, packet-switching protocols, country codes and area codes, the fixed numeration of money itself, and so on). The centrifugal standardization of how individual components interrelate and assemble into higher-order systems, whether physical or informational, is as important as what any part or component may be. This is how platforms can scale up. To engineer systems that coordinate the shuttling of units from one point to another with efficiency, adaptability, and flexibility is to compose within the rules laid down by other systems, larger and smaller, with which interaction is required.

For example, the formal urban grid in a major city is for the most part rigid and inflexible, but precisely because of this linear and universally authoritarian topography, it affords both maximum tumult of dynamic horizontal interchange in the street plan as well as vertical recombinant programmatic complexity in the skyscrapers that pop up in each of its cells (more on this in the City layer chapter).8 Similarly, it is the legal and practical standard size of the humble paper envelope that makes it possible for it to shuttle messages both discrete and discreet; like the urban grid, the envelope's power is in its dumbness. In the 1970s as the world's cities began to more fully merge into the networked hierarchies of today with the widespread standardization of very-large-scale envelopes, made of steel instead of paper, in the form of fixed proportion and attribute shipping containers. Containerization migrated the packet switching from telecommunications onto the transit of physical objects (or perhaps the other way around). It traded the standardized, linear traffic program of the grounded asphalt grid for another, now smoothed into liquid shipping lanes, pacing big packets of objects back and forth across the avenues of oceans. 10.  How Platforms Work Platforms centralize and decentralize at once, drawing many actors into a common infrastructure.


pages: 843 words: 223,858

The Rise of the Network Society by Manuel Castells

"Robert Solow", Apple II, Asian financial crisis, barriers to entry, Big bang: deregulation of the City of London, Bob Noyce, borderless world, British Empire, business cycle, capital controls, complexity theory, computer age, computerized trading, creative destruction, Credit Default Swap, declining real wages, deindustrialization, delayed gratification, dematerialisation, deskilling, disintermediation, double helix, Douglas Engelbart, Douglas Engelbart, edge city, experimental subject, financial deregulation, financial independence, floating exchange rates, future of work, global village, Gunnar Myrdal, Hacker Ethic, hiring and firing, Howard Rheingold, illegal immigration, income inequality, Induced demand, industrial robot, informal economy, information retrieval, intermodal, invention of the steam engine, invention of the telephone, inventory management, James Watt: steam engine, job automation, job-hopping, John Markoff, knowledge economy, knowledge worker, labor-force participation, laissez-faire capitalism, Leonard Kleinrock, longitudinal study, low skilled workers, manufacturing employment, Marc Andreessen, Marshall McLuhan, means of production, megacity, Menlo Park, moral panic, new economy, New Urbanism, offshore financial centre, oil shock, open economy, packet switching, Pearl River Delta, peer-to-peer, planetary scale, popular capitalism, popular electronics, post-industrial society, postindustrial economy, prediction markets, Productivity paradox, profit maximization, purchasing power parity, RAND corporation, Robert Gordon, Robert Metcalfe, Shoshana Zuboff, Silicon Valley, Silicon Valley startup, social software, South China Sea, South of Market, San Francisco, special economic zone, spinning jenny, statistical model, Steve Jobs, Steve Wozniak, Ted Nelson, the built environment, the medium is the message, the new new thing, The Wealth of Nations by Adam Smith, Thomas Kuhn: the structure of scientific revolutions, total factor productivity, trade liberalization, transaction costs, urban renewal, urban sprawl, zero-sum game

Furthermore, the extraordinary increase of transmission capacity with broadband communication technology provided the opportunity to use the Internet, or Internet-related communication technologies, to transmit voice, as well as data, through packet switching, thus revolutionizing telecommunications – and the telecommunications industry. According to Vinton Cerf, “Today you go through a circuit switch to get a packet switch. Tomorrow you’ll go through a packet switch to get a circuit switch.”55 In another technological vision, Cerf asserted that “during the latter half of the next decade – that is around 2005–2010 – there will be a new (technological) driver: billions of devices attached to the Internet.”56 So, ultimately, the communications network will be packet switched, with data transmission accounting for the overwhelming share of traffic, and voice transmission being but one, specialized service.

When in the late 1950s the launching of the first Sputnik alarmed the American high-tech military establishment, ARPA undertook a number of bold initiatives, some of which changed the history of technology and ushered in the Information Age on a grand scale. One of these strategies, developing an idea conceived by Paul Baran at Rand Corporation in 1960–4, was to design a communications system invulnerable to nuclear attack. Based on packet-switching communication technology, the system made the network independent of command and control centers, so that message units would find their own routes along the network, being reassembled in coherent meaning at any point in the network. When, later on, digital technology allowed the packaging of all kind of messages, including sound, images, and data, a network was formed that was able to communicate its nodes without using control centers.

Furthermore, in terms of technological system, one element cannot be imagined without the other: computers are largely determined by chip power, and both the design and the parallel processing of microprocessors depend on computer architecture. Telecommunications is now but one form of processing information; transmission and linkage technologies are at the same time increasingly diversified and integrated into the same network, operated by computers.90 As I analyzed above, the development of the Internet is reversing the relationship between circuit switching and packet switching in communication technologies, so that data transmission becomes the predominant, universal form of communication. And data transmission is based on software instructions of coding and decoding. Technological convergence increasingly extends to growing interdependence between the biological and micro-electronics revolutions, both materially and methodologically. Thus, decisive advances in biological research, such as the identification of human genes or segments of human DNA, can only proceed because of massive computing power.91 Nanotechnology may allow sending tiny microprocessors into the systems of living organisms, including humans.92 On the other hand, the use of biological materials in micro-electronics, although still very far from a generalized application, was already at the experimentation stage in the late 1990s.


pages: 319 words: 89,477

The Power of Pull: How Small Moves, Smartly Made, Can Set Big Things in Motion by John Hagel Iii, John Seely Brown

Albert Einstein, Andrew Keen, barriers to entry, Black Swan, business process, call centre, Clayton Christensen, cleantech, cloud computing, commoditize, corporate governance, creative destruction, disruptive innovation, Elon Musk, en.wikipedia.org, future of work, game design, George Gilder, intangible asset, Isaac Newton, job satisfaction, Joi Ito, knowledge economy, knowledge worker, loose coupling, Louis Pasteur, Malcom McLean invented shipping containers, Maui Hawaii, medical residency, Network effects, old-boy network, packet switching, pattern recognition, peer-to-peer, pre–internet, profit motive, recommendation engine, Ronald Coase, shareholder value, Silicon Valley, Skype, smart transportation, software as a service, supply-chain management, The Nature of the Firm, the new new thing, too big to fail, trade liberalization, transaction costs

The second key technological innovation involved the introduction of standards for packet-switched networks. Previous generations of communication networks had involved circuit switching. In these earlier networks, any communication required the establishment of a dedicated circuit across the parties that wanted to communicate. This circuit would be completely dedicated to the parties involved until they were finished communicating, even if the circuit did not have a lot of traffic at specific points in time. In contrast, packet-switched networks, enabled by digital technology, broke down longer communications into discrete packets of data that could be sent across shared networks. This represented a far more efficient use of network capacity. Although the concept of packet switching as an alternative technology for communicating had emerged in the early 1960s, it was not until the mid-1970s that researchers began to define the TCP/IP standard (Transmission Control Protocol/Internet Protocol) that ultimately provided a foundation for connecting a wide variety of digital networks together.


pages: 335 words: 107,779

Some Remarks by Neal Stephenson

airport security, augmented reality, barriers to entry, British Empire, cable laying ship, call centre, cellular automata, edge city, Eratosthenes, Fellow of the Royal Society, Hacker Ethic, impulse control, Iridium satellite, Isaac Newton, Jaron Lanier, John von Neumann, Just-in-time delivery, Kevin Kelly, music of the spheres, Norbert Wiener, offshore financial centre, oil shock, packet switching, pirate software, Richard Feynman, Saturday Night Live, shareholder value, Silicon Valley, Skype, slashdot, social web, Socratic dialogue, South China Sea, special economic zone, Stephen Hawking, the scientific method, trade route, Turing machine, undersea cable, uranium enrichment, Vernor Vinge, X Prize

., where you’ve got thousands of miles of nearly empty interstate highways and railroad lines and huge chunks of rolling stock to carry stuff around. The latter approach works in a place like Shanghai. The same problems of distribution arise in computer networks. As networks get bigger and as the machines that make them up become more equal, the whole approach to moving information around changes from centralized to distributed. The packet-switching system that makes things like the Internet work would be immediately familiar to the Chinese. Instead of requisitioning a hunk of optical fiber between Point A and Point B and slamming the data down it in one big shipment, the packet data network breaks the data down into tiny pieces and sends them out separately, just as a Chinese enterprise might break a large shipment down into small pieces and send each one out on a separate bicycle, knowing that each one might take a different route but that they’d all get there eventually.

Gao, bless him, was the only government official who would talk to me the whole trip—the PRC was still pissed off at the Great Hegemon (as they now call the U.S.) about that incident in the Persian Gulf a few months back when our guys stopped and boarded a Chinese freighter allegedly full of chemical warfare ingredients. They found nothing. Gao calmly rattled off a fairly staggering list of statistics on how rapidly the phone system there is growing—half to three-quarters of a million lines added per year for the foreseeable future. All of their local exchanges are webbed together with fiber, and they’re running fiber down the coast toward Shenzhen. They’re setting up packet-switching networks for their customers who want them—banks, import/export houses, and the like. The cellular and CT2 networks are also growing as rapidly as technology allows. He buys scads of high-bandwidth technology from the West and is actually trying to set up a sort of clearinghouse near Shanghai where Western manufacturers could gain access to the potentially stupendous Chinese market through a single point, instead of having to traffic separately with each regional PTA.

And unlike the ones who built FLAG, they will have the benefit of knowing about the Internet, and perhaps of understanding, at some level, that they are not merely stringing fancy telephone lines but laying down new traces on the circuit board of The Computer. That understanding may lead them to create vast amounts of bandwidth that would blow the minds of the entrenched telecrats and to adopt business models designed around packet-switching instead of the circuits that the telecrats are stuck on. If the network is The Computer, then its motherboard is the crust of Planet Earth. This may be the single biggest drag on the growth of The Computer, because Mother Earth was not designed to be a motherboard. There is too much water and not enough dirt. Water favors a few companies that know how to lay cable and have the ships to do it.


Speaking Code: Coding as Aesthetic and Political Expression by Geoff Cox, Alex McLean

4chan, Amazon Mechanical Turk, augmented reality, bash_history, bitcoin, cloud computing, computer age, computer vision, crowdsourcing, dematerialisation, Donald Knuth, Douglas Hofstadter, en.wikipedia.org, Everything should be made as simple as possible, finite state, Gödel, Escher, Bach, Jacques de Vaucanson, Larry Wall, late capitalism, means of production, natural language processing, new economy, Norbert Wiener, Occupy movement, packet switching, peer-to-peer, Richard Stallman, Ronald Coase, Slavoj Žižek, social software, social web, software studies, speech recognition, stem cell, Stewart Brand, The Nature of the Firm, Turing machine, Turing test, Vilfredo Pareto, We are Anonymous. We are Legion, We are the 99%, WikiLeaks

This informs the logic of its development, of course, funded by the Defense Advanced Research Projects Agency (DARPA) of the United States Department of Defense to provide a robust communications infrastructure able to withstand local failures.72 Its mechanism of command and control was expressed through distributed networks that would supersede centralized or decentralized ones, as these were considered far too vulnerable. The adoption of packet-switching networks, which informed the development of Arpanet, came to characterize the Internet as well.73 The idea was to break messages into units (like a cut-up poem) and then route each message unit along a functioning path to its ultimate destination, where it would be reassembled to form a coherent message once more. But once again the focus on technical development alone, at the expense of wider cultural and ethical considerations, leaves little 82 Chapter 3 room for a wider conception of agency that, if considered embedded in object-subjects under dynamic network conditions (as Latour thinks), is evident in all aspects of the network.

DARPA’s motivation was to develop a robust communications infrastructure for use by its projects at universities and research laboratories in the US, but whether it was developed to withstand attack from a nuclear strike, feared at the time of the Cold War, is rather more contentious. See http://en.wikipedia.org/wiki/ARPANET. 73. Ibid. The Advanced Research Projects Agency Network (ARPANET), initially known as ARPA (1963), was the world’s first operational packet-switching network. The first message ever sent via the ARPANET was at 10:30 PM, October 29, 1969. 74. Alex Galloway, Protocol: How Control Exists after Decentralization (Cambridge, MA: MIT Press, 2004), 7. 75. Ibid. 76. Richard Wray, “EU Says Internet Could Fall Apart,” Guardian, 12 October 2005; available at http://www.guardian.co.uk/business/2005/oct/12/newmedia.media. 77. Ned Rossiter, Organized Networks: Media Theory, Creative Labour, New Institutions (Rotterdam: NAi, in association with the Institute of Network Cultures, 2006). 78.


pages: 615 words: 168,775

Troublemakers: Silicon Valley's Coming of Age by Leslie Berlin

AltaVista, Apple II, Asilomar, Asilomar Conference on Recombinant DNA, beat the dealer, Bill Gates: Altair 8800, Bob Noyce, Byte Shop, Clayton Christensen, cloud computing, computer age, discovery of DNA, don't be evil, Donald Knuth, double helix, Douglas Engelbart, Douglas Engelbart, Dynabook, Edward Thorp, El Camino Real, fear of failure, Fellow of the Royal Society, financial independence, game design, Haight Ashbury, hiring and firing, industrial robot, informal economy, Internet of things, inventory management, John Markoff, Kickstarter, Kitchen Debate, Leonard Kleinrock, manufacturing employment, Mark Zuckerberg, Menlo Park, Minecraft, Mother of all demos, packet switching, Ralph Nader, Robert Metcalfe, rolodex, Ronald Reagan, Sand Hill Road, Silicon Valley, Silicon Valley startup, Snapchat, software as a service, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, Ted Nelson, union organizing, upwardly mobile, William Shockley: the traitorous eight, women in the workforce

Taylor kick-started the network. He got it funded. He helped to cajole researchers to contribute to it. No wonder Paul Baran, the inventor of packet switching, called the Arpanet Taylor’s “baby.”39 But after the Ann Arbor meeting, once it was clear the network was going to be built, Taylor stepped back and Roberts stepped forward. Roberts spent hours sketching out possible network configurations and meeting with principal investigators in subgroups that addressed topics ranging from software protocols to hardware design, to appropriate bandwidth allocations, to packet storage and routing. By summer 1967, a number of key issues had been resolved. The network would be packet-switched and run on a subnet system of the sort Clark proposed. The first four nodes would be at UCLA; Stanford Research Institute; the University of California, Santa Barbara; and the University of Utah.

“ARPA was trying to give away the ARPANET at one time, to get anybody to take it,” recalled Frank, who, along with Taylor’s successor Larry Roberts, met with AT&T in an effort to convince it to run the network. Frank describes AT&T’s reaction to the offer to take ARPANET technology to the public as “complete lack of interest, because they couldn’t imagine why anybody would want to send data in the network.” AT&T also did not believe packet switching had a future. 44. Notation in Jerry Elkind and Bob Taylor to George Pake, “Activity Report for June 13, 1972, through December 31, 1972,” records that on May 22, 1972, Elkind wrote a memo to Jack Goldman (“Xerox Acquisition of the ARPA Network”) “discussing the future sale of the ARPANET and the possible implications to and for Xerox,” RWT. On June 9, Elkind again wrote to Goldman (“ARPA Network”) “suggesting that a group be formed to analyze the opportunity for buying the ARPANET, and recommending the action Xerox should take.”

., 59, 81, 168, 230 New Yorker, 246, 351 New York Stock Exchange, 4–5, 317 New York Times, 133, 157, 172, 187, 257, 264n, 347, 371 NeXT, 372 Nintendo, 348 Nixon, Richard, 37, 120, 143, 158 NLS (oNLine System), 24 Nobel Prize, 133, 144, 188, 190, 193, 258, 263–64 Noyce, Robert, xi, 51, 54, 126n–27n, 129, 149, 190n, 235, 254n, 285, 302 Nutting Associates, 116 Office of Naval Research, 58 Opalka, Josephine, 138–40, 204 “Open Letter to Hobbyists” (Gates), 212 Oracle, 38, 78, 185, 364, 371 orchards, 4, 43, 46, 49, 149, 180, 370 order-processing systems, 148–49, 247 original equipment manufacturers (OEMs), 186 Ornstein, Severo, 95, 100 Orwell, George, 158 Osborne Computer, 358 Oshman, Ken, 44–46, 165, 319–22, 371 Packard, David, xi, 86, 254n, 255 packet switching, 21 PAC-MAN game, 345 Page, Larry, xii, 351 Pake, George, 94–95, 101, 217–23, 335–39 Palevsky, Max, 93n, 101 Palo Alto, Calif., 74, 80–84, 89, 93, 179, 207, 216–18, 221, 225, 235, 334, 342, 369 Palo Alto Research Center (PARC), xii–xiii Pake as director of, 94–95, 101, 217–21 Scientific Data Systems and, 90, 93, 101–2, 149, 215, 249 Sun technology and, 364 and women in workforce, 99–101 Palo Alto Research Center (PARC), Computer Science Laboratory at, 89–106, 146, 157, 213–14, 215–24, 285, 289 Alto system and, 92, 101–6, 141, 146, 215–17, 221–24, 285–88, 334 “dealer” meetings at, 99–103, 226, 340 personal computers and, 91–92, 101–6 paperless office, 224 Patent and Trademark Office, U.S., 144 patents, 59–62, 112, 132–37, 142–44, 157, 187–89, 257, 263, 349, 374 Peddle, Chuck, 212n People’s Park, 33–35, 55–56, 211, 226 peripheral devices, 209–10, 230, 246 Perkins, Tom, 128, 190–93, 197, 200–201, 254–60, 263 personal computers, xii, xv, 74, 91, 101–6, 141, 148, 213, 231, 239, 246–50, 269, 285, 301–3, 358, 366 pharmaceutical companies, 189, 235, 256–58, 266, 375 Philco-Ford, 58–59 pinball machines, 109, 113–14, 117–18, 273 Pitfall!


Smart Grid Standards by Takuro Sato

business cycle, business process, carbon footprint, clean water, cloud computing, data acquisition, decarbonisation, demand response, distributed generation, energy security, factory automation, information retrieval, Intergovernmental Panel on Climate Change (IPCC), Internet of things, Iridium satellite, iterative process, knowledge economy, life extension, linear programming, low earth orbit, market design, MITM: man-in-the-middle, off grid, oil shale / tar sands, packet switching, performance metric, RFC: Request For Comment, RFID, smart cities, smart grid, smart meter, smart transportation, Thomas Davenport

The following documents are included in IEC 61850-6: • • • • • • • IEC 60870-6-1 Application context and organization of standards IEC 60870-6-2 Use of basic standards (OSI layers 1–4) IEC 60870-6-501 TASE.1 Service definitions IEC 60870-6-502 TASE.1 Protocol definitions IEC 60870-6-503 TASE.2 Services and protocol IEC 60870-6-504 TASE.1 User conventions IEC 60870-6-601 Functional profile for providing the connection-oriented transport service in an end system connected via permanent access to a packet switched data network • IEC 60870-6-602 TASE transport profiles • IEC 60870-6-701 Functional profile for providing the TASE.1 application service in end systems • IEC 60870-6-702 Functional profile for providing the TASE.2 application service in end systems. 3.3.3.2 Architecture and Network Model The TASE.2 protocol relies on the use of MMS services (and hence the underlying MMS protocol) to implement data exchange among control center.

TASE.1 Protocol Definitions, International Electrotechnical Commission. [30] IEC (2002) IEC 60870-6-503. TASE.2 Services and Protocol, International Electrotechnical Commission. [31] IEC (1995) IEC 60870-6-504. TASE.1 User Conventions, International Electrotechnical Commission. [32] IEC (1998) IEC 60870-6-601. Functional Profile for Providing the Connection-oriented Transport Service in an End System Connected Via Permanent Access to a Packet Switched Data Network, International Electrotechnical Commission. [33] IEC (1998) IEC 60870-6-602. TASE Transport Profiles, International Electrotechnical Commission. [34] IEC IEC 60870-6-701. Functional Profile for Providing the TASE.1 Application Service in End Systems, International Electrotechnical Commission. [35] IEC IEC 60870-6-702. Functional Profile for Providing the TASE.2 Application Service in End Systems, International Electrotechnical Commission. 142 Smart Grid Standards [36] IEC (2005) IEC 61970-1 Ed.: Energy Management System Application Program Interface (EMS-API) Part 1: Guidelines and General Requirement, International Electrotechnical Commission. [37] IEC (2004) IEC 61970-2.

Similar to D-AMPS, Japan has developed its own 2G system based on TDMA, called Personal Digital Cellular (PDC). PDC used Frequency Division Duplex – Time Division Multiple Access (FDD-TDMA), 25 kHz carrier spacing and three full rate (or six half-rate) channels Communications in the Smart Grid 287 per carrier. PDC operated in the 800-MHz and 1.5-GHz bands, offering voice and data services, supplementary services such as call waiting, call forwarding, and voice mail. The maximum data rate for packet-switched data was 28.8 kbps and for circuit-switched data service 9.6 kbps. At the same time, the Global System for Mobile communication (GSM) has been deployed in Europe and later adopted also in the United States. While D-AMPS and PDC digital cellular systems have already been shut down, GSM survived and it is still actively used by service providers worldwide. 6.4.4.1 GSM Family of Standards GSM is currently the most widely deployed and globally accepted system.


pages: 395 words: 116,675

The Evolution of Everything: How New Ideas Emerge by Matt Ridley

"Robert Solow", affirmative action, Affordable Care Act / Obamacare, Albert Einstein, Alfred Russel Wallace, AltaVista, altcoin, anthropic principle, anti-communist, bank run, banking crisis, barriers to entry, bitcoin, blockchain, Boris Johnson, British Empire, Broken windows theory, Columbian Exchange, computer age, Corn Laws, cosmological constant, creative destruction, Credit Default Swap, crony capitalism, crowdsourcing, cryptocurrency, David Ricardo: comparative advantage, demographic transition, Deng Xiaoping, discovery of DNA, Donald Davies, double helix, Downton Abbey, Edward Glaeser, Edward Lorenz: Chaos theory, Edward Snowden, endogenous growth, epigenetics, Ethereum, ethereum blockchain, facts on the ground, falling living standards, Ferguson, Missouri, financial deregulation, financial innovation, Frederick Winslow Taylor, Geoffrey West, Santa Fe Institute, George Gilder, George Santayana, Gunnar Myrdal, Henri Poincaré, hydraulic fracturing, imperial preference, income per capita, indoor plumbing, interchangeable parts, Intergovernmental Panel on Climate Change (IPCC), invisible hand, Isaac Newton, Jane Jacobs, Jeff Bezos, joint-stock company, Joseph Schumpeter, Kenneth Arrow, Kevin Kelly, Khan Academy, knowledge economy, land reform, Lao Tzu, long peace, Lyft, M-Pesa, Mahatma Gandhi, Mark Zuckerberg, means of production, meta analysis, meta-analysis, mobile money, money: store of value / unit of account / medium of exchange, Mont Pelerin Society, moral hazard, Necker cube, obamacare, out of africa, packet switching, peer-to-peer, phenotype, Pierre-Simon Laplace, price mechanism, profit motive, RAND corporation, random walk, Ray Kurzweil, rent-seeking, reserve currency, Richard Feynman, rising living standards, road to serfdom, Ronald Coase, Ronald Reagan, Satoshi Nakamoto, Second Machine Age, sharing economy, smart contracts, South Sea Bubble, Steve Jobs, Steven Pinker, The Wealth of Nations by Adam Smith, Thorstein Veblen, transaction costs, twin studies, uber lyft, women in the workforce

There is a long and sterile argument to be had about who deserves credit for inventing the internet – government or private industry. Barack Obama is in no doubt that, as he put it in a speech in 2012, ‘The Internet didn’t get invented on its own. Government research created the Internet.’ He was referring to the fact that the decentralised network we know today began life as the Arpanet, a project funded by the Pentagon, and that relied on an idea called packet switching, dreamt up by Paul Baran at the RAND Corporation, whose motive was chiefly to make something that could survive a Soviet first strike and still transmit messages to missile bases to retaliate. Hence the decentralised nature of the network. That’s nonsense, say others. The internet is more than package-switching. It requires computers, communications, all sorts of software and other protocols, many of which the government-funded research projects would have bought from private enterprise.

A handbook for users of the Arpanet at MIT in the 1980s reminded them that ‘sending electronic messages over the ARPAnet for commercial profit or political purposes is both antisocial and illegal’. The internet revolution might have happened ten years earlier if academics had not been dependent on a government network antipathetic to commercial use. Well, then, perhaps we should forget about who was funding the work, and at least give credit to the individuals without whom the internet would never have happened. Paul Baran was first with the notion of packet switching, Vint Cerf invented the TCP/IP protocols that proved crucial to allowing different programs to run on the internet, and Sir Tim Berners Lee developed the worldwide web. Yet there is a problem here, too. Can anybody really think that these things – or their equivalents – would not have come into existence in the 1990s if these undoubtedly brilliant men had never been born? Given all we know about the ubiquitous phenomenon of simultaneous invention, and the inevitability of the next step in innovation once a technology is ripe (see Chapter 7), it is inconceivable that the twentieth century would have ended without a general, open means of connecting computers to each other so that people could see what was on other nodes than their own hard drive.

Given all we know about the ubiquitous phenomenon of simultaneous invention, and the inevitability of the next step in innovation once a technology is ripe (see Chapter 7), it is inconceivable that the twentieth century would have ended without a general, open means of connecting computers to each other so that people could see what was on other nodes than their own hard drive. Indeed, the notion of packet switching – and even the name we now use for it – occurred independently to a Welshman named Donald Davies just a short time after Baran stumbled on it. Vint Cerf shares the credit for TCP/IP with Bob Kahn. So, while we should honour individuals for their contributions, we should not really think that they made something come into existence that would not have otherwise. The names would be different, and some of the procedures too, but an alternative internet would exist today whoever had lived.


pages: 247 words: 62,845

VoIP Telephony with Asterisk by Unknown

call centre, Debian, framing effect, packet switching, telemarketer

Asterisk can be used for many things and has features includin Private Branch Exchange (PBX) Voicemail Services with Directory Conferencing Server Packet Voice Server Encryption of Telephone or Fax Calls Heterogeneous Voice over IP gateway (H.323, SIP, MGCP, IAX) Custom Interactive Voice Response (IVR) system Soft switch Number Translation Calling Card Server Predictive Dialer Call Queueing with Remote Agents Gateway and Aggregation for Legacy PBX systems Remote Office or User Telephone Services PBX long distance Gateway Telemarketing Block Standalone Voicemail System Many of the world's largest telephone companies have committed to replacing their existing circuit switched systems with packet switched voice over IP systems. Many phone companies are alread transporting a significant portion of their traffic with IP. Many calls made over telephone compan equipment are already being transported with IP. Packet switched voice over IP systems are in principle as efficient as a synchronous circuit switched systems, but only recently have they had the potential to achieve the same level of reliability as the public switched telephone network or proprietaryPBX equipment. With the invention and implementation of RTP (real time protocol) and SIP (session initiation protocol,) voice over IP has the technological base to obsolete the circuit switched public switched telephone network.


pages: 272 words: 64,626

Eat People: And Other Unapologetic Rules for Game-Changing Entrepreneurs by Andy Kessler

23andMe, Andy Kessler, bank run, barriers to entry, Berlin Wall, Bob Noyce, British Empire, business cycle, business process, California gold rush, carbon footprint, Cass Sunstein, cloud computing, collateralized debt obligation, collective bargaining, commoditize, computer age, creative destruction, disintermediation, Douglas Engelbart, Eugene Fama: efficient market hypothesis, fiat currency, Firefox, Fractional reserve banking, George Gilder, Gordon Gekko, greed is good, income inequality, invisible hand, James Watt: steam engine, Jeff Bezos, job automation, Joseph Schumpeter, Kickstarter, knowledge economy, knowledge worker, libertarian paternalism, low skilled workers, Mark Zuckerberg, McMansion, Netflix Prize, packet switching, personalized medicine, pets.com, prediction markets, pre–internet, profit motive, race to the bottom, Richard Thaler, risk tolerance, risk-adjusted returns, Silicon Valley, six sigma, Skype, social graph, Steve Jobs, The Wealth of Nations by Adam Smith, transcontinental railway, transfer pricing, wealth creators, Yogi Berra

But really, they’re just political entrepreneurs. Big whoop. They are successful with their hands in my pocket. Not a stitch of productivity to be found. Moguls are not Free Radicals. AND THEN THE Internet came along. Oops. Move along—no scarcity here. Cisco routers, and all the other packet-switching network equipment composing the Internet cloud ending up at that broadband router in your basement, send packets of data around to wherever folks want them—no moguls needed. Market entrepreneurs used the chaos of that packet switching to deliver text and pictures to Web sites or phones or even TVs. Then bandwidth got cheap enough to move music around too, shattering the record labels’ control of distribution. And now as bandwidth gets even cheaper and more plentiful, video starts to move around this wild packet network.


pages: 510 words: 120,048

Who Owns the Future? by Jaron Lanier

3D printing, 4chan, Affordable Care Act / Obamacare, Airbnb, augmented reality, automated trading system, barriers to entry, bitcoin, book scanning, Burning Man, call centre, carbon footprint, cloud computing, commoditize, computer age, crowdsourcing, David Brooks, David Graeber, delayed gratification, digital Maoism, Douglas Engelbart, en.wikipedia.org, Everything should be made as simple as possible, facts on the ground, Filter Bubble, financial deregulation, Fractional reserve banking, Francis Fukuyama: the end of history, George Akerlof, global supply chain, global village, Haight Ashbury, hive mind, if you build it, they will come, income inequality, informal economy, information asymmetry, invisible hand, Jaron Lanier, Jeff Bezos, job automation, John Markoff, Kevin Kelly, Khan Academy, Kickstarter, Kodak vs Instagram, life extension, Long Term Capital Management, Marc Andreessen, Mark Zuckerberg, meta analysis, meta-analysis, Metcalfe’s law, moral hazard, mutually assured destruction, Network effects, new economy, Norbert Wiener, obamacare, packet switching, Panopticon Jeremy Bentham, Peter Thiel, place-making, plutocrats, Plutocrats, Ponzi scheme, post-oil, pre–internet, race to the bottom, Ray Kurzweil, rent-seeking, reversible computing, Richard Feynman, Ronald Reagan, scientific worldview, self-driving car, side project, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, Skype, smart meter, stem cell, Steve Jobs, Steve Wozniak, Stewart Brand, Ted Nelson, The Market for Lemons, Thomas Malthus, too big to fail, trickle-down economics, Turing test, Vannevar Bush, WikiLeaks, zero-sum game

They broke up and reconciled repeatedly, and were perpetually on the verge of presenting the ultimate software project, Xanadu, in some formulation, which would have been remembered as the first implementation of the Web, or perhaps even the Internet itself. To be clear, the key technical insight that allowed networking to become decentralized and scale was packet switching, and that insight did not arise from Ted Nelson or the Xanadu project. Instead it arose just a little later than Ted’s earliest work, from the very different world of elite universities, government labs, and military research funding. However, at least the functionality of something like packet switching is foreseen in Ted’s early thinking. Ted published outrageous books. One was a big floppy book composed of montages of nearly indecipherable small print snippets flung in all directions, called Computer Lib/Dream Machines. If you turned it one way and started reading, it was what Che would have been reading in the jungle if he had been a computer nerd.

., 75, 91, 266–67 New York Times, 109 Nobel Prize, 40, 118, 143n nodes, network, 156, 227, 230, 241–43, 350 “no free lunch” principle, 55–56, 59–60 nondeterministic music, 23n nonlinear solutions, 149–50 nonprofit share sites, 59n, 94–95 nostalgia, 129–32 NRO, 199–200 nuclear power, 133 nuclear weapons, 127, 296 nursing, 97–100, 123, 296n nursing homes, 97–100, 269 Obama, Barack, 79, 100 “Obamacare,” 100n obsolescence, 89, 95 oil resources, 43, 133 online stores, 171 Ono, Yoko, 212 ontologies, 124n, 196 open-source applications, 206, 207, 272, 310–11 optical illusions, 121 optimism, 32–35, 45, 130, 138–40, 218, 230n, 295 optimization, 144–47, 148, 153, 154–55, 167, 202, 203 Oracle, 265 Orbitz, 63, 64, 65 organ donors, 190, 191 ouroboros, 154 outcomes, economic, 40–41, 144–45 outsourcing, 177–78, 185 Owens, Buck, 256 packet switching, 228–29 Palmer, Amanda, 186–87 Pandora, 192 panopticons, 308 papacy, 190 paper money, 34n parallel computers, 147–48, 149, 151 paranoia, 309 Parrish, Maxfield, 214 particle interactions, 196 party machines, 202 Pascal, Blaise, 132, 139 Pascal’s Wager, 139 passwords, 307, 309 “past-oriented money,” 29–31, 35, 284–85 patterns, information, 178, 183, 184, 188–89 Paul, Ron, 33n Pauli exclusion principle, 181, 202 PayPal, 60, 93, 326 peasants, 565 pensions, 95, 99 Perestroika (Kushner), 165 “perfect investments,” 59–67, 77–78 performances, musical, 47–48, 51, 186–87, 253 perpetual motion, 55 Persian Gulf, 86 personal computers (PCs), 158, 182n, 214, 223, 229 personal information systems, 110, 312–16, 317 Pfizer, 265 pharmaceuticals industry, 66–67, 100–106, 123, 136, 203 philanthropy, 117 photography, 53, 89n, 92, 94, 309–11, 318, 319, 321 photo-sharing services, 53 physical trades, 292 physicians, 66–67 physics, 88, 153n, 167n Picasso, Pablo, 108 Pinterest, 180–81, 183 Pirate Party, 49, 199, 206, 226, 253, 284, 318 placebos, 112 placement fees, 184 player pianos, 160–61 plutocracy, 48, 291–94, 355 police, 246, 310, 311, 319–21, 335 politics, 13–18, 21, 22–25, 47–48, 85, 122, 124–26, 128, 134–37, 149–51, 155, 167, 199–234, 295–96, 342 see also conservatism; liberalism; libertarianism Ponzi schemes, 48 Popper, Karl, 189n popular culture, 111–12, 130, 137–38, 139, 159 “populating the stack,” 273 population, 17, 34n, 86, 97–100, 123, 125, 132, 133, 269, 296n, 325–26, 346 poverty, 37–38, 42, 44, 53–54, 93–94, 137, 148, 167, 190, 194, 253, 256, 263, 290, 291–92 power, personal, 13–15, 53, 60, 62–63, 86, 114, 116, 120, 122, 158, 166, 172–73, 175, 190, 199, 204, 207, 208, 278–79, 290, 291, 302–3, 308–9, 314, 319, 326, 344, 360 Presley, Elvis, 211 Priceline, 65 pricing strategies, 1–2, 43, 60–66, 72–74, 145, 147–48, 158, 169–74, 226, 261, 272–75, 289, 317–24, 331, 337–38 printers, 90, 99, 154, 162, 212, 269, 310–11, 316, 331, 347, 348, 349 privacy, 1–2, 11, 13–15, 25, 50–51, 64, 99, 108–9, 114–15, 120–21, 152, 177n, 199–200, 201, 204, 206–7, 234–35, 246, 272, 291, 305, 309–13, 314, 315–16, 317, 319–24 privacy rights, 13–15, 25, 204, 305, 312–13, 314, 315–16, 321–22 product design and development, 85–89, 117–20, 128, 136–37, 145, 154, 236 productivity, 7, 56–57, 134–35 profit margins, 59n, 71–72, 76–78, 94–95, 116, 177n, 178, 179, 207, 258, 274–75, 321–22 progress, 9–18, 20, 21, 37, 43, 48, 57, 88, 98, 123, 124–40, 130–37, 256–57, 267, 325–31, 341–42 promotions, 62 property values, 52 proprietary hardware, 172 provenance, 245–46, 247, 338 pseudo-asceticism, 211–12 public libraries, 293 public roads, 79–80 publishers, 62n, 92, 182, 277–78, 281, 347, 352–60 punishing vs. rewarding network effects, 169–74, 182, 183 quants, 75–76 quantum field theory, 167n, 195 QuNeo, 117, 118, 119 Rabois, Keith, 185 “race to the bottom,” 178 radiant risk, 61–63, 118–19, 120, 156, 183–84 Ragnarok, 30 railroads, 43, 172 Rand, Ayn, 167, 204 randomness, 143 rationality, 144 Reagan, Ronald, 149 real estate, 33, 46, 49–52, 61, 78, 95–96, 99, 193, 224, 227, 239, 245, 255, 274n, 289n, 296, 298, 300, 301 reality, 55–56, 59–60, 124n, 127–28, 154–56, 161, 165–68, 194–95, 203–4, 216–17, 295–303, 364–65 see also Virtual Reality (VR) reason, 195–96 recessions, economic, 31, 54, 60, 76–77, 79, 151–52, 167, 204, 311, 336–37 record labels, 347 recycling, 88, 89 Reddit, 118n, 186, 254 reductionism, 184 regulation, economic, 37–38, 44, 45–46, 49–50, 54, 56, 69–70, 77–78, 266n, 274, 299–300, 311, 321–22, 350–51 relativity theory, 167n religion, 124–25, 126, 131, 139, 190, 193–95, 211–17, 293, 300n, 326 remote computers, 11–12 rents, 144 Republican Party, 79, 202 research and development, 40–45, 85–89, 117–20, 128, 136–37, 145, 154, 215, 229–30, 236 retail sector, 69, 70–74, 95–96, 169–74, 272, 349–51, 355–56 retirement, 49, 150 revenue growth plans, 173n revenues, 149, 149, 150, 151, 173n, 225, 234–35, 242, 347–48 reversible computers, 143n revolutions, 199, 291, 331 rhythm, 159–62 Rich Dad, Poor Dad (Kiyosaki), 46 risk, 54, 55, 57, 59–63, 71–72, 85, 117, 118–19, 120, 156, 170–71, 179, 183–84, 188, 242, 277–81, 284, 337, 350 externalization of, 59n, 117, 277–81 risk aversion, 188 risk pools, 277–81, 284 risk radiation, 61–63, 118–19, 120, 156, 183–84 robo call centers, 177n robotic cars, 90–92 robotics, robots, 11, 12, 17, 23, 42, 55, 85–86, 90–92, 97–100, 111, 129, 135–36, 155, 157, 162, 260, 261, 269, 296n, 342, 359–60 Roman Empire, 24–25 root nodes, 241 Rousseau, Jean-Jacques, 129 Rousseau humor, 126, 129, 130–31 routers, 171–72 royalties, 47, 240, 254, 263–64, 323, 338 Rubin, Edgar, 121 rupture, 66–67 salaries, 10, 46–47, 50–54, 152, 178, 270–71, 287–88, 291–94, 338–39, 365 sampling, 71–72, 191, 221, 224–26, 259 San Francisco, University of, 190 satellites, 110 savings, 49, 72–74 scalable solutions, 47 scams, 119–21, 186, 275n, 287–88, 299–300 scanned books, 192, 193 SceneTap, 108n Schmidt, Eric, 305n, 352 Schwartz, Peter, 214 science fiction, 18, 126–27, 136, 137–38, 139, 193, 230n, 309, 356n search engines, 51, 60, 70, 81, 120, 191, 267, 289, 293 Second Life, 270, 343 Secret, The (Byrne), 216 securitization, 76–78, 99, 289n security, 14–15, 175, 239–40, 305–8, 345 self-actualization, 211–17 self-driving vehicles, 90–92, 98, 311, 343, 367 servants, 22 servers, 12n, 15, 31, 53–57, 71–72, 95–96, 143–44, 171, 180, 183, 206, 245, 358 see also Siren Servers “Sexy Sadie,” 213 Shakur, Tupac, 329 Shelley, Mary, 327 Short History of Progress, A (Wright), 132 “shrinking markets,” 66–67 shuttles, 22, 23n, 24 signal-processing algorithms, 76–78, 148 silicon chips, 10, 86–87 Silicon Valley, 12, 13, 14, 21, 34n, 56, 59, 60, 66–67, 70, 71, 75–76, 80, 93, 96–97, 100, 102, 108n, 125n, 132, 136, 154, 157, 162, 170, 179–89, 192, 193, 200, 207, 210, 211–18, 228, 230, 233, 258, 275n, 294, 299–300, 325–31, 345, 349, 352, 354–58 singularity, 22–25, 125, 215, 217, 327–28, 366, 367 Singularity University, 193, 325, 327–28 Sirenic Age, 66n, 354 Siren Servers, 53–57, 59, 61–64, 65, 66n, 69–78, 82, 91–99, 114–19, 143–48, 154–56, 166–89, 191, 200, 201, 203, 210n, 216, 235, 246–50, 258, 259, 269, 271, 272, 280, 285, 289, 293–94, 298, 301, 302–3, 307–10, 314–23, 326, 336–51, 354, 365, 366 Siri, 95 skilled labor, 99–100 Skout, 280n Skype, 95, 129 slavery, 22, 23, 33n Sleeper, 130 small businesses, 173 smartphones, 34n, 39, 162, 172, 192, 269n, 273 Smith, Adam, 121, 126 Smolin, Lee, 148n social contract, 20, 49, 247, 284, 288, 335, 336 social engineering, 112–13, 190–91 socialism, 14, 128, 254, 257, 341n social mobility, 66, 97, 292–94 social networks, 18, 51, 56, 60, 70, 81, 89, 107–9, 113, 114, 129, 167–68, 172–73, 179, 180, 190, 199, 200–201, 202, 204, 227, 241, 242–43, 259, 267, 269n, 274–75, 280n, 286, 307–8, 317, 336, 337, 343, 349, 358, 365–66 see also Facebook social safety nets, 10, 44, 54, 202, 251, 293 Social Security, 251, 345 software, 7, 9, 11, 14, 17, 68, 86, 99, 100–101, 128, 129, 147, 154, 155, 165, 172–73, 177–78, 182, 192, 234, 236, 241–42, 258, 262, 273–74, 283, 331, 347, 357 software-mediated technology, 7, 11, 14, 86, 100–101, 165, 234, 236, 258, 347 South Korea, 133 Soviet Union, 70 “space elevator pitch,” 233, 342, 361 space travel, 233, 266 Spain, 159–60 spam, 178, 275n spending levels, 287–88 spirituality, 126, 211–17, 325–31, 364 spreadsheet programs, 230 “spy data tax,” 234–35 Square, 185 Stalin, Joseph, 125n Stanford Research Institute (SRI), 215 Stanford University, 60, 75, 90, 95, 97, 101, 102, 103, 162, 325 Starr, Ringo, 256 Star Trek, 138, 139, 230n startup companies, 39, 60, 69, 93–94, 108n, 124n, 136, 179–89, 265, 274n, 279–80, 309–10, 326, 341, 343–45, 348, 352, 355 starvation, 123 Star Wars, 137 star (winner-take-all) system, 38–43, 50, 54–55, 204, 243, 256–57, 263, 329–30 statistics, 11, 20, 71–72, 75–78, 90–91, 93, 110n, 114–15, 186, 192 “stickiness,” 170, 171 stimulus, economic, 151–52 stoplights, 90 Strangelove humor, 127 student debt, 92, 95 “Study 27,” 160 “Study 36,” 160 Sumer, 29 supergoop, 85–89 supernatural phenomena, 55, 124–25, 127, 132, 192, 194–95, 300 supply chain, 70–72, 174, 187 Supreme Court, U.S., 104–5 surgery, 11–13, 17, 18, 98, 157–58, 363 surveillance, 1–2, 11, 14, 50–51, 64, 71–72, 99, 108–9, 114–15, 120–21, 152, 177n, 199–200, 201, 206–7, 234–35, 246, 272, 291, 305, 309–11, 315, 316, 317, 319–24 Surviving Progress, 132 sustainable economies, 235–37, 285–87 Sutherland, Ivan, 221 swarms, 99, 109 synthesizers, 160 synthetic biology, 162 tablets, 85, 86, 87, 88, 113, 162, 229 Tahrir Square, 95 Tamagotchis, 98 target ads, 170 taxation, 44, 45, 49, 52, 60, 74–75, 77, 82, 149, 149, 150, 151, 202, 210, 234–35, 263, 273, 289–90 taxis, 44, 91–92, 239, 240, 266–67, 269, 273, 311 Teamsters, 91 TechCrunch, 189 tech fixes, 295–96 technical schools, 96–97 technologists (“techies”), 9–10, 15–16, 45, 47–48, 66–67, 88, 122, 124, 131–32, 134, 139–40, 157–62, 165–66, 178, 193–94, 295–98, 307, 309, 325–31, 341, 342, 356n technology: author’s experience in, 47–48, 62n, 69–72, 93–94, 114, 130, 131–32, 153, 158–62, 178, 206–7, 228, 265, 266–67, 309–10, 325, 328, 343, 352–53, 362n, 364, 365n, 366 bio-, 11–13, 17, 18, 109–10, 162, 330–31 chaos and, 165–66, 273n, 331 collusion in, 65–66, 72, 169–74, 255, 350–51 complexity of, 53–54 costs of, 8, 18, 72–74, 87n, 136–37, 170–71, 176–77, 184–85 creepiness of, 305–24 cultural impact of, 8–9, 21, 23–25, 53, 130, 135–40 development and emergence of, 7–18, 21, 53–54, 60–61, 66–67, 85–86, 87, 97–98, 129–38, 157–58, 182, 188–90, 193–96, 217 digital, 2–3, 7–8, 15–16, 18, 31, 40, 43, 50–51, 132, 208 economic impact of, 1–3, 15–18, 29–30, 37, 40, 53–54, 60–66, 71–74, 79–110, 124, 134–37, 161, 162, 169–77, 181–82, 183, 184–85, 218, 254, 277–78, 298, 335–39, 341–51, 357–58 educational, 92–97 efficiency of, 90, 118, 191 employment in, 56–57, 60, 71–74, 79, 123, 135, 178 engineering for, 113–14, 123–24, 192, 194, 217, 218, 326 essential vs. worthless, 11–12 failure of, 188–89 fear of (technophobia), 129–32, 134–38 freedom as issue in, 32–33, 90–92, 277–78, 336 government influence in, 158, 199, 205–6, 234–35, 240, 246, 248–51, 307, 317, 341, 345–46, 350–51 human agency and, 8–21, 50–52, 85, 88, 91, 124–40, 144, 165–66, 175–78, 191–92, 193, 217, 253–64, 274–75, 283–85, 305–6, 328, 341–51, 358–60, 361, 362, 365–67 ideas for, 123, 124, 158, 188–89, 225, 245–46, 286–87, 299, 358–60 industrial, 49, 83, 85–89, 123, 132, 154, 343 information, 7, 32–35, 49, 66n, 71–72, 109, 110, 116, 120, 125n, 126, 135, 136, 254, 312–16, 317 investment in, 66, 181, 183, 184, 218, 277–78, 298, 348 limitations of, 157–62, 196, 222 monopolies for, 60, 65–66, 169–74, 181–82, 187–88, 190, 202, 326, 350 morality and, 50–51, 72, 73–74, 188, 194–95, 262, 335–36 motivation and, 7–18, 85–86, 97–98, 216 nano-, 11, 12, 17, 162 new vs. old, 20–21 obsolescence of, 89, 97 political impact of, 13–18, 22–25, 85, 122, 124–26, 128, 134–37, 199–234, 295–96, 342 progress in, 9–18, 20, 21, 37, 43, 48, 57, 88, 98, 123, 124–40, 130–37, 256–57, 267, 325–31, 341–42 resources for, 55–56, 157–58 rupture as concept in, 66–67 scams in, 119–21, 186, 275n, 287–88, 299–300 singularity of, 22–25, 125, 215, 217, 327–28, 366, 367 social impact of, 9–21, 124–40, 167n, 187, 280–81, 310–11 software-mediated, 7, 11, 14, 86, 100–101, 165, 234, 236, 258, 347 startup companies in, 39, 60, 69, 93–94, 108n, 124n, 136, 179–89, 265, 274n, 279–80, 309–10, 326, 341, 343–45, 348, 352, 355 utopian, 13–18, 21, 31, 37–38, 45–46, 96, 128, 130, 167, 205, 207, 265, 267, 270, 283, 290, 291, 308–9, 316 see also specific technologies technophobia, 129–32, 134–38 television, 86, 185–86, 191, 216, 267 temperature, 56, 145 Ten Commandments, 300n Terminator, The, 137 terrorism, 133, 200 Tesla, Nikola, 327 Texas, 203 text, 162, 352–60 textile industry, 22, 23n, 24, 135 theocracy, 194–95 Theocracy humor, 124–25 thermodynamics, 88, 143n Thiel, Peter, 60, 93, 326 thought experiments, 55, 139 thought schemas, 13 3D printers, 7, 85–89, 90, 99, 154, 162, 212, 269, 310–11, 316, 331, 347, 348, 349 Thrun, Sebastian, 94 Tibet, 214 Time Machine, The (Wells), 127, 137, 261, 331 topology, network, 241–43, 246 touchscreens, 86 tourism, 79 Toyota Prius, 302 tracking services, 109, 120–21, 122 trade, 29 traffic, 90–92, 314 “tragedy of the commons,” 66n Transformers, 98 translation services, 19–20, 182, 191, 195, 261, 262, 284, 338 transparency, 63–66, 74–78, 118, 176, 190–91, 205–6, 278, 291, 306–9, 316, 336 transportation, 79–80, 87, 90–92, 123, 258 travel agents, 64 Travelocity, 65 travel sites, 63, 64, 65, 181, 279–80 tree-shaped networks, 241–42, 243, 246 tribal dramas, 126 trickle-down effect, 148–49, 204 triumphalism, 128, 157–62 tropes (humors), 124–40, 157, 170, 230 trust, 32–34, 35, 42, 51–52 Turing, Alan, 127–28, 134 Turing’s humor, 127–28, 191–94 Turing Test, 330 Twitter, 128, 173n, 180, 182, 188, 199, 200n, 201, 204, 245, 258, 259, 349, 365n 2001: A Space Odyssey, 137 two-way links, 1–2, 227, 245, 289 underemployment, 257–58 unemployment, 7–8, 22, 79, 85–106, 117, 151–52, 234, 257–58, 321–22, 331, 343 “unintentional manipulation,” 144 United States, 25, 45, 54, 79–80, 86, 138, 199–204 universities, 92–97 upper class, 45, 48 used car market, 118–19 user interface, 362–63, 364 utopianism, 13–18, 21, 30, 31, 37–38, 45–46, 96, 128, 130, 167, 205, 207, 265, 267, 270, 283, 290, 291, 308–9, 316 value, economic, 21, 33–35, 52, 61, 64–67, 73n, 108, 283–90, 299–300, 321–22, 364 value, information, 1–3, 15–16, 20, 210, 235–43, 257–58, 259, 261–63, 271–75, 321–24, 358–60 Values, Attitudes, and Lifestyles (VALS), 215 variables, 149–50 vendors, 71–74 venture capital, 66, 181, 218, 277–78, 298, 348 videos, 60, 100, 162, 185–86, 204, 223, 225, 226, 239, 240, 242, 245, 277, 287, 329, 335–36, 349, 354, 356 Vietnam War, 353n vinyl records, 89 viral videos, 185–86 Virtual Reality (VR), 12, 47–48, 127, 129, 132, 158, 162, 214, 283–85, 312–13, 314, 315, 325, 343, 356, 362n viruses, 132–33 visibility, 184, 185–86, 234, 355 visual cognition, 111–12 VitaBop, 100–106, 284n vitamins, 100–106 Voice, The, 185–86 “voodoo economics,” 149 voting, 122, 202–4, 249 Wachowski, Lana, 165 Wall Street, 49, 70, 76–77, 181, 184, 234, 317, 331, 350 Wal-Mart, 69, 70–74, 89, 174, 187, 201 Warhol, Andy, 108 War of the Worlds, The (Wells), 137 water supplies, 17, 18 Watts, Alan, 211–12 Wave, 189 wealth: aggregate or concentration of, 9, 42–43, 53, 60, 61, 74–75, 96, 97, 108, 115, 148, 157–58, 166, 175, 201, 202, 208, 234, 278–79, 298, 305, 335, 355, 360 creation of, 32, 33–34, 46–47, 50–51, 57, 62–63, 79, 92, 96, 120, 148–49, 210, 241–43, 270–75, 291–94, 338–39, 349 inequalities and redistribution of, 20, 37–45, 65–66, 92, 97, 144, 254, 256–57, 274–75, 286–87, 290–94, 298, 299–300 see also income levels weather forecasting, 110, 120, 150 weaving, 22, 23n, 24 webcams, 99, 245 websites, 80, 170, 200, 201, 343 Wells, H.


pages: 263 words: 75,610

Delete: The Virtue of Forgetting in the Digital Age by Viktor Mayer-Schönberger

en.wikipedia.org, Erik Brynjolfsson, Firefox, full text search, George Akerlof, information asymmetry, information retrieval, information trail, Internet Archive, invention of movable type, invention of the printing press, John Markoff, Joi Ito, lifelogging, moveable type in China, Network effects, packet switching, Panopticon Jeremy Bentham, pattern recognition, RFID, slashdot, Steve Jobs, Steven Levy, The Market for Lemons, The Structural Transformation of the Public Sphere, Vannevar Bush

By 2008, more than 330 million people had broadband connections, receiving in excess of 100,000 bytes per second.35 To achieve such a staggering 15-fold increase, connection speeds must have doubled roughly every fifteen months, thus outpacing the already phenomenal growth of both processing power and storage capacity. At the same time, monthly connection fees have remained relatively flat, resulting in an equally amazing decrease in communication costs. Moreover, because almost all broadband connections are offered for a flat monthly fee, they create a further economic incentive for users to maximize utilization of the network. Three drivers have facilitated this development. The first is the packet-switched structure of the Internet. Unlike the telephone system, which directly connects two communication parties, information on the Internet travels in small information packets that find the fastest way from sender to recipient independently of each other. This leads to a much better utilization of the available network infrastructure. Second, a huge amount of fiber optic cable ideal for broadband connections has been laid.

See meta-information Microsoft, 6, 8, 50, 51, 159, 176–78, 179 Miller, Arthur, 11, 100 misinterpretation: danger of, 90 Moore, Gordon, 63–64 Moore’s law, 64 MyLifeBits, 50–51 MySpace, 1, 2, 84, 102, 131 Negroponte, Nicholas, 53 network: fiber optic, 80–81 global, 79 social, 84 network externalities, 85 neurons, 16–17 newspapers. See periodicals Nissenbaum, Helen, 142 noise, 53–55, 60 Nozick, Robert, 91 Nye, Joseph, 98 online travel sites, 8 Orbitz, 8 original, 34, 56 Orkut, 2 Orwell, George, 120–21 packet-switching, 80 page numbers, 73–74 painting, 29 advantages and disadvantages of, 30–31 Palfrey, John 3, 130 panopticon, 11–12, 111–12, 165, 197 spatial, 111–12 temporal, 111–12 paper, 39–42 cost of, 39–42 papyri, 33 peer-production. See information: peer-production of penny press, 41 perfect memory, 4, 5 benefits of, 10 chilling effect of, 5, 12 periodicals, 41, 42, 43–44 Pew Research, 3 photography, 46–47 pixel, 54, 55, 57 Plato, 28 printing press, 37 privacy, 11, 135, 137.


The Ages of Globalization by Jeffrey D. Sachs

Admiral Zheng, British Empire, Cape to Cairo, colonial rule, Columbian Exchange, Commentariolus, coronavirus, COVID-19, Covid-19, cuban missile crisis, decarbonisation, demographic transition, Deng Xiaoping, domestication of the camel, Donald Trump, en.wikipedia.org, endogenous growth, European colonialism, global supply chain, greed is good, income per capita, invention of agriculture, invention of gunpowder, invention of movable type, invention of the steam engine, invisible hand, Isaac Newton, James Watt: steam engine, job automation, John von Neumann, joint-stock company, Louis Pasteur, low skilled workers, mass immigration, Nikolai Kondratiev, out of africa, packet switching, Pax Mongolica, precision agriculture, profit maximization, profit motive, purchasing power parity, South China Sea, spinning jenny, The inhabitant of London could order by telephone, sipping his morning tea in bed, the various products of the whole earth, The Wealth of Nations by Adam Smith, trade route, transatlantic slave trade, Turing machine, Turing test, urban planning, Watson beat the top human players on Jeopardy!, wikimedia commons

As computers began to penetrate scientific, military, and business work, the U.S. Department of Defense asked a basic question: How can computers communicate with each other, and do so in a resilient way that would survive the disruption of networks in a war? The answer was a method for sending data packets (bits of 0s and 1s) between computers according to flexible routing, a method known as “packet switching,” that became the basis for a new Internet. Initially a U.S. government project, the Internet was later made available to a group of participating U.S. universities before it was opened for commercial use in 1987. In 1965, Gordon Moore, then the head of Intel, an early manufacturer of integrated circuitry that would become the global pacesetter, noticed that the transistor count etched into a microchip of silicon was doubling roughly every one to two years.

.), 106 Noyce, Robert, 171 nuclear powers, 30 ocean acidification, 188 Ocean Age, 2, 4, 7, 195; global reach in, 11; lessons from, 126–27 ocean navigation, 97–101 ocean shipping, 134 Ogedei Khan, 91 oil reserves, 18 Old World, 100–101, 228n10 Old World technologies, 21 Opium War, 146–47 organic economy, 133 Ottoman Empire, 89, 111, 158 Our Common Future (report), 197 overland transport, 25 oxen, 47 ozone depletion, 188 pack animals, 56 packet switching, 171 Paleolithic Age, 2–3, 7, 195; dog domestication in, 54–55; human dispersal during, 35; hunter/gatherers in, 15–16; lessons from, 40; Middle Paleolithic of, 34; migration and human settlement in, 10–11; productive activity in, 15; sub-periods of, 226n1 papermaking, 82 Paris Climate Agreement, 232n4 Parthasarathi, Prasannan, 149 Parthian Empire, 82–83 Partial Nuclear Test Ban Treaty (1963), 213 Patent Cooperation Treaty (PCT), 182 patents, 182 pathogens, 101, 101–2 Paul III (pope), 106 Pax Mongolica, 92 PCT.


pages: 489 words: 148,885

Accelerando by Stross, Charles

business cycle, call centre, carbon-based life, cellular automata, cognitive dissonance, commoditize, Conway's Game of Life, dark matter, dumpster diving, Extropian, finite state, Flynn Effect, glass ceiling, gravity well, John von Neumann, Kickstarter, knapsack problem, Kuiper Belt, Magellanic Cloud, mandelbrot fractal, market bubble, means of production, MITM: man-in-the-middle, orbital mechanics / astrodynamics, packet switching, performance metric, phenotype, planetary scale, Pluto: dwarf planet, reversible computing, Richard Stallman, SETI@home, Silicon Valley, Singularitarianism, slashdot, South China Sea, stem cell, technological singularity, telepresence, The Chicago School, theory of mind, Turing complete, Turing machine, Turing test, upwardly mobile, Vernor Vinge, Von Neumann architecture, web of trust, Y2K, zero-sum game

From the alien?" The cat, claws extended, delicately picks its way down to her lap and waits to be held and stroked. It never once takes its eyes off him. "Where else?" she asks. "Doctor, I didn't get the Franklin Trust to loan me the wherewithal to build this castle just in return for some legal paperwork, and some, ah, interesting legal waivers from Brussels. We've known for years there's a whole alien packet-switching network out there, and we're just getting spillover from some of their routers. It turns out there's a node not far away from here, in real space. Helium-three, separate jurisdictions, heavy industrialization on Io – there is a purpose to all this activity." Sadeq licks his suddenly dry lips. "You're going to narrowcast a reply?" "No, much better than that: we're going to visit them. Cut the delay cycle down to real-time.

"A network of point-to-point wormholes linking routers, self-replicating communication hubs, in orbit around most of the brown dwarfs of the galaxy. That's what the brochure said, right? That's what we expected. Limited bandwidth, not a lot of use to a mature superintelligence that has converted the free mass of its birth solar system into computronium, but sufficient to allow it to hold conversations with its neighbors. Conversations carried out via a packet-switched network in real time, not limited by the speed of light, but bound together by a common reference frame and the latency between network hops." "That's about the size of it," she agrees from the carved-ruby throne beside him. "Except there's a trade delegation waiting for us. In fact, they're coming aboard already. And I don't buy it – something about the whole setup stinks." Pierre's brow wrinkles.

But in only about ten gigaseconds, the infestation has turned the dead brown dwarf system upside down. They strip-mined the chilly planets to make environments suitable for their own variety of carbon life. They rearranged moons, building massive structures the size of asteroids. They ripped wormhole endpoints free of the routers and turned them into their own crude point-to-point network, learned how to generate new wormholes, then ran their own packet-switched polities over them. Wormhole traffic now supports an ever-expanding mesh of interstellar human commerce, but always in the darkness between the lit stars and the strange, metal-depleted dwarfs with the suspiciously low-entropy radiation. The sheer temerity of the project is mind-boggling: notwithstanding that canned apes are simply not suited to life in the interstellar void, especially in orbit around a brown dwarf whose planets make Pluto seem like a tropical paradise, they've taken over the whole damn system.


Principles of Protocol Design by Robin Sharp

accounting loophole / creative accounting, business process, discrete time, fault tolerance, finite state, Gödel, Escher, Bach, information retrieval, loose coupling, MITM: man-in-the-middle, packet switching, RFC: Request For Comment, stochastic process

Comput. 3, 146–158 (1989) 27. Daemen, J., Rijmen, V.: AES Proposal: Rijndael (1999). Available via URL http://csrc.nist.gov/encryption/aes/rijndael/. Selected as the NIST Advanced Encryption Standard algorithm. 28. Dally, W.J., Seitz, C.L.: Deadlock-free message routing in multiprocessor interconnection networks. IEEE Trans. Comput. C-36(5), 547–553 (1987) 29. Davies, D.W.: The control of congestion in packet switching networks. IEEE Trans. on Communications COM-20(3), 546–550 (1972) 30. Diffie, W., Hellman, M.E.: New directions in cryptography. IEEE Trans. on Inf. Theory IT-22(6), 644–654 (1976) 31. Diffie, W., van Oorschot, P.C., Wiener, M.J.: Authentication and authenticated key exchanges. Designs, Codes and Cryptography 2, 107–125 (1992) 32. Dijsktra, E.W.: A note on two problems in connexion with graphs.

IEEE Trans. on Software Engineering SE-6(5), 435–440 (1980) 53. Gordon, J.: Strong RSA keys. Electronics Letters 20(5), 514–516 (1984) 54. Gray, J.: Notes on data base operating systems. In: R. Bayer, et al. (eds.) Operating Systems – An Advanced Course, Lecture Notes in Computer Science, vol. 60, pp. 393–481. SpringerVerlag (1978) 55. Griffiths, J.M.: ISDN Explained, second edn. John Wiley & Sons (1992) 56. Günther, K.D.: Prevention of deadlocks in packet-switched data transport systems. IEEE Trans. on Communications COM-29(4), 512–524 (1981) 57. Hailpern, B.: Verifying Concurrent Processes Using Temporal Logic, Lecture Notes in Computer Science, vol. 129. Springer-Verlag (1982) 58. Halsall, F.: Computer Networking and the Internet, fifth edn. Addison-Wesley (2005). ISBN 0-321-26358-8 59. Hayes, J.P., Mudge, T.: Hypercube supercomputers. Proc. IEEE 77(12), 1829–1841 (1989) 60.

.: A calculus of total correctness for communicating processes. Sci. Comput. Program. 1, 49–72 (1981) 64. Hoare, C.A.R.: Communicating Sequential Processes. Prentice-Hall International (1985) 65. Holzmann, G.: Design and Validation of Computer Protocols. Prentice-Hall International (1991) 66. Hull, R., Su, J.: Tools for composite web services: A short overview. SIGMOD Record 34(2), 86–95 (2005) 67. Irland, M.I.: Buffer management in a packet switch. IEEE Trans. on Communications COM26(3), 328–327 (1978) 68. Jacobsen, V.: Congestion avoidance and control. In: Proc. ACM SIGCOMM’88, Stanford, California, pp. 314–329. ACM (1988) 69. Jain, R.: Congestion control in computer networks: Issues and trends. IEEE Network Magazine pp. 24–30 (1990) 70. Jain, R.: Congestion control and traffic management in ATM networks: Recent advances and a survey.


pages: 282 words: 92,998

Cyber War: The Next Threat to National Security and What to Do About It by Richard A. Clarke, Robert Knake

barriers to entry, complexity theory, data acquisition, Just-in-time delivery, MITM: man-in-the-middle, nuclear winter, packet switching, RAND corporation, Robert Hanssen: Double agent, Ronald Reagan, Silicon Valley, smart grid, South China Sea, Steve Jobs, trade route, undersea cable, Y2K, zero day

Data could be scanned for malware and backed up in redundant data farms, some of which would always be disconnected from the network in case of a corrupting system failure. All of these new intranets could use constant scanning technologies to detect and prevent anomalous activity, intrusions, identity theft, malicious software, or unauthorized exporting of data. The intranets could encrypt all data and require that a user prove with two or three reliable methods who he is before he could access the intranet. If the new nets were “packet switched,” as the Internet is now, the user’s authenticated identity could be embedded in each packet. Most important, these networks could constantly monitor for and prevent connectivity to the Internet. A lot of people will hate that idea. Many of the Internet’s earliest advocates strongly believe that information should be free and freely disseminated, and that essential to that freedom is the right to access information anonymously.

Crossing the boundary is an escalatory step that may lead to the war spiraling out of control. DARPA (also seen as ARPA): The Defense Advanced Research Projects Agency is a component of the U.S. Defense Department charged with funding innovative research to meet the needs of the U.S. military. DARPA funded the initial research that created the Internet. In 1969 ARPANET became the first packet-switched network connecting four universities. Deep-Packet Inspection: A procedure that scans the packets of data that make up an e-mail, webpage, or other Internet traffic. Normally only the “header” of a packet is scanned, the top part that gives the to and from information. A deep inspection would scan the digital pattern in the content but would not convert that content into text. The inspection looks only for digital patterns that are identical or highly similar to known malware or hacking tools.


pages: 371 words: 93,570

Broad Band: The Untold Story of the Women Who Made the Internet by Claire L. Evans

"side hustle", 4chan, Ada Lovelace, Albert Einstein, British Empire, colonial rule, computer age, crowdsourcing, dark matter, dematerialisation, Doomsday Book, Douglas Engelbart, Douglas Engelbart, Douglas Hofstadter, East Village, Edward Charles Pickering, game design, glass ceiling, Grace Hopper, Gödel, Escher, Bach, Haight Ashbury, Harvard Computers: women astronomers, Honoré de Balzac, Howard Rheingold, HyperCard, hypertext link, index card, information retrieval, Internet Archive, Jacquard loom, John von Neumann, Joseph-Marie Jacquard, knowledge worker, Leonard Kleinrock, Mahatma Gandhi, Mark Zuckerberg, Menlo Park, Mother of all demos, Network effects, old-boy network, On the Economy of Machinery and Manufactures, packet switching, pets.com, rent control, RFC: Request For Comment, rolodex, semantic web, Silicon Valley, Skype, South of Market, San Francisco, Steve Jobs, Steven Levy, Stewart Brand, subscription business, technoutopianism, Ted Nelson, telepresence, Whole Earth Catalog, Whole Earth Review, women in the workforce, Works Progress Administration, Y2K

The Crowthers’ maps were simplified line plots, but they represent some of the earliest efforts to computerize caves, a leap in technical sophistication made possible by the hardware to which they had access: a PDP-1 mainframe and a Honeywell 316, a sixteen-bit minicomputer, both far beyond consumer-grade. Will Crowther’s employer was Bolt, Beranek and Newman (BBN), a Massachusetts company that specialized in advanced research. In 1969, BBN was contracted by the U.S. government to help build the ARPANET, the military and academic packet-switching network that spawned our present-day Internet. A few years after they used it to plot their cave maps, the Honeywell 316 minicomputer was repurposed and ruggedized to become an Interface Message Processor, or IMP—what we now call a router. These routers formed a subnetwork of smaller computers within the ARPANET, shuffling data around and translating between primary nodes, a vital component of the Internet then and now.

As one historian points out, before computers became accessible remotely, “a scientist who needed to use a distant computer might find it easier to get on a plane and fly to the machine’s location to use it in person.” The ARPANET, by linking a group of useful “distant computers” together, changed all that. With network access, a scientist at MIT could run programs on a machine in California just as easily as if they were in the room punching the keys themselves. The ARPANET was a packet-switching network, as the Internet remains today: by breaking information into bite-size “packets” and sending it across the network in measured hops, the ARPANET insured itself against system-wide failure. If any node along the network were to go down, the packets could easily reroute themselves before reassembling upon arrival. The ARPANET’s earliest users were its builders: mathematicians, computer scientists, and engineers at places like Bolt, Beranek and Newman, where Pat Crowther printed her cave maps and Will Crowther wrote router code; MIT; Carnegie Mellon; UCLA; and, up in Northern California, Berkeley, Stanford, and the Stanford Research Institute in Menlo Park.


pages: 889 words: 433,897

The Best of 2600: A Hacker Odyssey by Emmanuel Goldstein

affirmative action, Apple II, call centre, don't be evil, Firefox, game design, Hacker Ethic, hiring and firing, information retrieval, John Markoff, late fees, license plate recognition, Mitch Kapor, MITM: man-in-the-middle, optical character recognition, packet switching, pirate software, place-making, profit motive, QWERTY keyboard, RFID, Robert Hanssen: Double agent, rolodex, Ronald Reagan, Silicon Valley, Skype, spectrum auction, statistical model, Steve Jobs, Steve Wozniak, Steven Levy, Telecommunications Act of 1996, telemarketer, undersea cable, Y2K

Those magical moments when Matthew Broderick managed to get inside that computer system, or when he figured out how to make the free call from the pay phone, or when he was apprehended by the feds—all of us who found ourselves messing around with phones and computers at the time felt like we were living that story because in many cases we were. That thrill—and that fear—is something that never really leaves you. And those of us who experienced it at that relatively early stage of the game were really quite privileged, even though it sure didn’t feel like it at the time. In a big way, the Internet would be the death knell for the kind of hacking most popular in the 1980s. Back then, the most attractive targets were the big packet switched networks like Telenet and Tymnet. These systems allowed you to connect to computers all over the world once you dialed into a local node. Unlike the Internet, it was geared primarily toward businesses and institutions. So if you wanted to play around with it, you pretty much had to break in. We couldn’t get accounts as individuals and we sure couldn’t quell our curiosity. Nor could we effectively explain this to most people.

And apart from all of that, there were massive amounts of new toys to play with as the landscape continued to change. Something as simple as a fax machine or a new consumer service for modem users like PC Pursuit was enough to captivate our attention for huge amounts of time. 94192c04.qxd 6/4/08 3:37 AM Page 121 The Early Days of the Net Hacking on Telenet (February, 1984) Telenet. Or, to be more specific, GTE Telenet. A massive network formed by the people and technology that were used to develop packet switching for the Department of Defense. Telenet was purchased by GTE in 1979 and has been growing in size and revenue ever since. There are quite a few data networks in existence today. Datapac, Autonet, Tymnet, ARPANET, to name some of the better known. A data network is basically a collection of mainframes, specialized minis, and high-speed lines. Through Telenet, you can connect to literally thousands of computers, all over the country, even the world if you know the proper procedures.

Pursuit for People (September, 1985) On August 7, GTE Telenet announced a new service that, if handled properly, will usher in a whole new phase of computer communications. 94192c05.qxd 6/3/08 3:31 PM Page 165 Corporate History The service is called PC Pursuit and it enables people to connect their computers to other computers for $25 a month (plus a start-up fee of $25). In other words, a hobbyist in New York can connect his computer to a bulletin board in California and not have to pay for a long-distance call. The “computer conversation” goes through GTE Telenet, a packet-switching network for computers, previously used exclusively by large corporations. “To access the service,” GTE’s press release explains, “a user calls his PC Pursuit access number and is prompted to enter his home phone number and make a request for a destination phone number in a distant city. If the user’s telephone number is not authorized, the phone call is terminated and a record of the call is generated.


Martin Kleppmann-Designing Data-Intensive Applications. The Big Ideas Behind Reliable, Scalable and Maintainable Systems-O’Reilly (2017) by Unknown

active measures, Amazon Web Services, bitcoin, blockchain, business intelligence, business process, c2.com, cloud computing, collaborative editing, commoditize, conceptual framework, cryptocurrency, database schema, DevOps, distributed ledger, Donald Knuth, Edward Snowden, Ethereum, ethereum blockchain, fault tolerance, finite state, Flash crash, full text search, general-purpose programming language, informal economy, information retrieval, Internet of things, iterative process, John von Neumann, Kubernetes, loose coupling, Marc Andreessen, microservices, natural language processing, Network effects, packet switching, peer-to-peer, performance metric, place-making, premature optimization, recommendation engine, Richard Feynman, self-driving car, semantic web, Shoshana Zuboff, social graph, social web, software as a service, software is eating the world, sorting algorithm, source of truth, SPARQL, speech recognition, statistical model, undersea cable, web application, WebSocket, wikimedia commons

., an email or a web page), and it will try to transfer it in the shortest time possible. While a TCP connection is idle, it doesn’t use any bandwidth.ii If datacenter networks and the internet were circuit-switched networks, it would be possible to establish a guaranteed maximum round-trip time when a circuit was set up. However, they are not: Ethernet and IP are packet-switched protocols, which suf‐ fer from queueing and thus unbounded delays in the network. These protocols do not have the concept of a circuit. Why do datacenter networks and the internet use packet switching? The answer is that they are optimized for bursty traffic. A circuit is good for an audio or video call, which needs to transfer a fairly constant number of bits per second for the duration of the call. On the other hand, requesting a web page, sending an email, or transfer‐ ring a file doesn’t have any particular bandwidth requirement—we just want it to complete as quickly as possible.

If you guess too high, the circuit cannot be set up (because the net‐ work cannot allow a circuit to be created if its bandwidth allocation cannot be guar‐ anteed). Thus, using circuits for bursty data transfers wastes network capacity and makes transfers unnecessarily slow. By contrast, TCP dynamically adapts the rate of data transfer to the available network capacity. There have been some attempts to build hybrid networks that support both circuit switching and packet switching, such as ATM.iii InfiniBand has some similarities [35]: it implements end-to-end flow control at the link layer, which reduces the need for ii. Except perhaps for an occasional keepalive packet, if TCP keepalive is enabled. iii. Asynchronous Transfer Mode (ATM) was a competitor to Ethernet in the 1980s [32], but it didn’t gain much adoption outside of telephone network core switches. It has nothing to do with automatic teller machines (also known as cash machines), despite sharing an acronym.

The opposite of bounded. 558 | Glossary Index A aborts (transactions), 222, 224 in two-phase commit, 356 performance of optimistic concurrency con‐ trol, 266 retrying aborted transactions, 231 abstraction, 21, 27, 222, 266, 321 access path (in network model), 37, 60 accidental complexity, removing, 21 accountability, 535 ACID properties (transactions), 90, 223 atomicity, 223, 228 consistency, 224, 529 durability, 226 isolation, 225, 228 acknowledgements (messaging), 445 active/active replication (see multi-leader repli‐ cation) active/passive replication (see leader-based rep‐ lication) ActiveMQ (messaging), 137, 444 distributed transaction support, 361 ActiveRecord (object-relational mapper), 30, 232 actor model, 138 (see also message-passing) comparison to Pregel model, 425 comparison to stream processing, 468 Advanced Message Queuing Protocol (see AMQP) aerospace systems, 6, 10, 305, 372 aggregation data cubes and materialized views, 101 in batch processes, 406 in stream processes, 466 aggregation pipeline query language, 48 Agile, 22 minimizing irreversibility, 414, 497 moving faster with confidence, 532 Unix philosophy, 394 agreement, 365 (see also consensus) Airflow (workflow scheduler), 402 Ajax, 131 Akka (actor framework), 139 algorithms algorithm correctness, 308 B-trees, 79-83 for distributed systems, 306 hash indexes, 72-75 mergesort, 76, 402, 405 red-black trees, 78 SSTables and LSM-trees, 76-79 all-to-all replication topologies, 175 AllegroGraph (database), 50 ALTER TABLE statement (SQL), 40, 111 Amazon Dynamo (database), 177 Amazon Web Services (AWS), 8 Kinesis Streams (messaging), 448 network reliability, 279 postmortems, 9 RedShift (database), 93 S3 (object storage), 398 checking data integrity, 530 amplification of bias, 534 of failures, 364, 495 Index | 559 of tail latency, 16, 207 write amplification, 84 AMQP (Advanced Message Queuing Protocol), 444 (see also messaging systems) comparison to log-based messaging, 448, 451 message ordering, 446 analytics, 90 comparison to transaction processing, 91 data warehousing (see data warehousing) parallel query execution in MPP databases, 415 predictive (see predictive analytics) relation to batch processing, 411 schemas for, 93-95 snapshot isolation for queries, 238 stream analytics, 466 using MapReduce, analysis of user activity events (example), 404 anti-caching (in-memory databases), 89 anti-entropy, 178 Apache ActiveMQ (see ActiveMQ) Apache Avro (see Avro) Apache Beam (see Beam) Apache BookKeeper (see BookKeeper) Apache Cassandra (see Cassandra) Apache CouchDB (see CouchDB) Apache Curator (see Curator) Apache Drill (see Drill) Apache Flink (see Flink) Apache Giraph (see Giraph) Apache Hadoop (see Hadoop) Apache HAWQ (see HAWQ) Apache HBase (see HBase) Apache Helix (see Helix) Apache Hive (see Hive) Apache Impala (see Impala) Apache Jena (see Jena) Apache Kafka (see Kafka) Apache Lucene (see Lucene) Apache MADlib (see MADlib) Apache Mahout (see Mahout) Apache Oozie (see Oozie) Apache Parquet (see Parquet) Apache Qpid (see Qpid) Apache Samza (see Samza) Apache Solr (see Solr) Apache Spark (see Spark) 560 | Index Apache Storm (see Storm) Apache Tajo (see Tajo) Apache Tez (see Tez) Apache Thrift (see Thrift) Apache ZooKeeper (see ZooKeeper) Apama (stream analytics), 466 append-only B-trees, 82, 242 append-only files (see logs) Application Programming Interfaces (APIs), 5, 27 for batch processing, 403 for change streams, 456 for distributed transactions, 361 for graph processing, 425 for services, 131-136 (see also services) evolvability, 136 RESTful, 133 SOAP, 133 application state (see state) approximate search (see similarity search) archival storage, data from databases, 131 arcs (see edges) arithmetic mean, 14 ASCII text, 119, 395 ASN.1 (schema language), 127 asynchronous networks, 278, 553 comparison to synchronous networks, 284 formal model, 307 asynchronous replication, 154, 553 conflict detection, 172 data loss on failover, 157 reads from asynchronous follower, 162 Asynchronous Transfer Mode (ATM), 285 atomic broadcast (see total order broadcast) atomic clocks (caesium clocks), 294, 295 (see also clocks) atomicity (concurrency), 553 atomic increment-and-get, 351 compare-and-set, 245, 327 (see also compare-and-set operations) replicated operations, 246 write operations, 243 atomicity (transactions), 223, 228, 553 atomic commit, 353 avoiding, 523, 528 blocking and nonblocking, 359 in stream processing, 360, 477 maintaining derived data, 453 for multi-object transactions, 229 for single-object writes, 230 auditability, 528-533 designing for, 531 self-auditing systems, 530 through immutability, 460 tools for auditable data systems, 532 availability, 8 (see also fault tolerance) in CAP theorem, 337 in service level agreements (SLAs), 15 Avro (data format), 122-127 code generation, 127 dynamically generated schemas, 126 object container files, 125, 131, 414 reader determining writer’s schema, 125 schema evolution, 123 use in Hadoop, 414 awk (Unix tool), 391 AWS (see Amazon Web Services) Azure (see Microsoft) B B-trees (indexes), 79-83 append-only/copy-on-write variants, 82, 242 branching factor, 81 comparison to LSM-trees, 83-85 crash recovery, 82 growing by splitting a page, 81 optimizations, 82 similarity to dynamic partitioning, 212 backpressure, 441, 553 in TCP, 282 backups database snapshot for replication, 156 integrity of, 530 snapshot isolation for, 238 use for ETL processes, 405 backward compatibility, 112 BASE, contrast to ACID, 223 bash shell (Unix), 70, 395, 503 batch processing, 28, 389-431, 553 combining with stream processing lambda architecture, 497 unifying technologies, 498 comparison to MPP databases, 414-418 comparison to stream processing, 464 comparison to Unix, 413-414 dataflow engines, 421-423 fault tolerance, 406, 414, 422, 442 for data integration, 494-498 graphs and iterative processing, 424-426 high-level APIs and languages, 403, 426-429 log-based messaging and, 451 maintaining derived state, 495 MapReduce and distributed filesystems, 397-413 (see also MapReduce) measuring performance, 13, 390 outputs, 411-413 key-value stores, 412 search indexes, 411 using Unix tools (example), 391-394 Bayou (database), 522 Beam (dataflow library), 498 bias, 534 big ball of mud, 20 Bigtable data model, 41, 99 binary data encodings, 115-128 Avro, 122-127 MessagePack, 116-117 Thrift and Protocol Buffers, 117-121 binary encoding based on schemas, 127 by network drivers, 128 binary strings, lack of support in JSON and XML, 114 BinaryProtocol encoding (Thrift), 118 Bitcask (storage engine), 72 crash recovery, 74 Bitcoin (cryptocurrency), 532 Byzantine fault tolerance, 305 concurrency bugs in exchanges, 233 bitmap indexes, 97 blockchains, 532 Byzantine fault tolerance, 305 blocking atomic commit, 359 Bloom (programming language), 504 Bloom filter (algorithm), 79, 466 BookKeeper (replicated log), 372 Bottled Water (change data capture), 455 bounded datasets, 430, 439, 553 (see also batch processing) bounded delays, 553 in networks, 285 process pauses, 298 broadcast hash joins, 409 Index | 561 brokerless messaging, 442 Brubeck (metrics aggregator), 442 BTM (transaction coordinator), 356 bulk synchronous parallel (BSP) model, 425 bursty network traffic patterns, 285 business data processing, 28, 90, 390 byte sequence, encoding data in, 112 Byzantine faults, 304-306, 307, 553 Byzantine fault-tolerant systems, 305, 532 Byzantine Generals Problem, 304 consensus algorithms and, 366 C caches, 89, 553 and materialized views, 101 as derived data, 386, 499-504 database as cache of transaction log, 460 in CPUs, 99, 338, 428 invalidation and maintenance, 452, 467 linearizability, 324 CAP theorem, 336-338, 554 Cascading (batch processing), 419, 427 hash joins, 409 workflows, 403 cascading failures, 9, 214, 281 Cascalog (batch processing), 60 Cassandra (database) column-family data model, 41, 99 compaction strategy, 79 compound primary key, 204 gossip protocol, 216 hash partitioning, 203-205 last-write-wins conflict resolution, 186, 292 leaderless replication, 177 linearizability, lack of, 335 log-structured storage, 78 multi-datacenter support, 184 partitioning scheme, 213 secondary indexes, 207 sloppy quorums, 184 cat (Unix tool), 391 causal context, 191 (see also causal dependencies) causal dependencies, 186-191 capturing, 191, 342, 494, 514 by total ordering, 493 causal ordering, 339 in transactions, 262 sending message to friends (example), 494 562 | Index causality, 554 causal ordering, 339-343 linearizability and, 342 total order consistent with, 344, 345 consistency with, 344-347 consistent snapshots, 340 happens-before relationship, 186 in serializable transactions, 262-265 mismatch with clocks, 292 ordering events to capture, 493 violations of, 165, 176, 292, 340 with synchronized clocks, 294 CEP (see complex event processing) certificate transparency, 532 chain replication, 155 linearizable reads, 351 change data capture, 160, 454 API support for change streams, 456 comparison to event sourcing, 457 implementing, 454 initial snapshot, 455 log compaction, 456 changelogs, 460 change data capture, 454 for operator state, 479 generating with triggers, 455 in stream joins, 474 log compaction, 456 maintaining derived state, 452 Chaos Monkey, 7, 280 checkpointing in batch processors, 422, 426 in high-performance computing, 275 in stream processors, 477, 523 chronicle data model, 458 circuit-switched networks, 284 circular buffers, 450 circular replication topologies, 175 clickstream data, analysis of, 404 clients calling services, 131 pushing state changes to, 512 request routing, 214 stateful and offline-capable, 170, 511 clocks, 287-299 atomic (caesium) clocks, 294, 295 confidence interval, 293-295 for global snapshots, 294 logical (see logical clocks) skew, 291-294, 334 slewing, 289 synchronization and accuracy, 289-291 synchronization using GPS, 287, 290, 294, 295 time-of-day versus monotonic clocks, 288 timestamping events, 471 cloud computing, 146, 275 need for service discovery, 372 network glitches, 279 shared resources, 284 single-machine reliability, 8 Cloudera Impala (see Impala) clustered indexes, 86 CODASYL model, 36 (see also network model) code generation with Avro, 127 with Thrift and Protocol Buffers, 118 with WSDL, 133 collaborative editing multi-leader replication and, 170 column families (Bigtable), 41, 99 column-oriented storage, 95-101 column compression, 97 distinction between column families and, 99 in batch processors, 428 Parquet, 96, 131, 414 sort order in, 99-100 vectorized processing, 99, 428 writing to, 101 comma-separated values (see CSV) command query responsibility segregation (CQRS), 462 commands (event sourcing), 459 commits (transactions), 222 atomic commit, 354-355 (see also atomicity; transactions) read committed isolation, 234 three-phase commit (3PC), 359 two-phase commit (2PC), 355-359 commutative operations, 246 compaction of changelogs, 456 (see also log compaction) for stream operator state, 479 of log-structured storage, 73 issues with, 84 size-tiered and leveled approaches, 79 CompactProtocol encoding (Thrift), 119 compare-and-set operations, 245, 327 implementing locks, 370 implementing uniqueness constraints, 331 implementing with total order broadcast, 350 relation to consensus, 335, 350, 352, 374 relation to transactions, 230 compatibility, 112, 128 calling services, 136 properties of encoding formats, 139 using databases, 129-131 using message-passing, 138 compensating transactions, 355, 461, 526 complex event processing (CEP), 465 complexity distilling in theoretical models, 310 hiding using abstraction, 27 of software systems, managing, 20 composing data systems (see unbundling data‐ bases) compute-intensive applications, 3, 275 concatenated indexes, 87 in Cassandra, 204 Concord (stream processor), 466 concurrency actor programming model, 138, 468 (see also message-passing) bugs from weak transaction isolation, 233 conflict resolution, 171, 174 detecting concurrent writes, 184-191 dual writes, problems with, 453 happens-before relationship, 186 in replicated systems, 161-191, 324-338 lost updates, 243 multi-version concurrency control (MVCC), 239 optimistic concurrency control, 261 ordering of operations, 326, 341 reducing, through event logs, 351, 462, 507 time and relativity, 187 transaction isolation, 225 write skew (transaction isolation), 246-251 conflict-free replicated datatypes (CRDTs), 174 conflicts conflict detection, 172 causal dependencies, 186, 342 in consensus algorithms, 368 in leaderless replication, 184 Index | 563 in log-based systems, 351, 521 in nonlinearizable systems, 343 in serializable snapshot isolation (SSI), 264 in two-phase commit, 357, 364 conflict resolution automatic conflict resolution, 174 by aborting transactions, 261 by apologizing, 527 convergence, 172-174 in leaderless systems, 190 last write wins (LWW), 186, 292 using atomic operations, 246 using custom logic, 173 determining what is a conflict, 174, 522 in multi-leader replication, 171-175 avoiding conflicts, 172 lost updates, 242-246 materializing, 251 relation to operation ordering, 339 write skew (transaction isolation), 246-251 congestion (networks) avoidance, 282 limiting accuracy of clocks, 293 queueing delays, 282 consensus, 321, 364-375, 554 algorithms, 366-368 preventing split brain, 367 safety and liveness properties, 365 using linearizable operations, 351 cost of, 369 distributed transactions, 352-375 in practice, 360-364 two-phase commit, 354-359 XA transactions, 361-364 impossibility of, 353 membership and coordination services, 370-373 relation to compare-and-set, 335, 350, 352, 374 relation to replication, 155, 349 relation to uniqueness constraints, 521 consistency, 224, 524 across different databases, 157, 452, 462, 492 causal, 339-348, 493 consistent prefix reads, 165-167 consistent snapshots, 156, 237-242, 294, 455, 500 (see also snapshots) 564 | Index crash recovery, 82 enforcing constraints (see constraints) eventual, 162, 322 (see also eventual consistency) in ACID transactions, 224, 529 in CAP theorem, 337 linearizability, 324-338 meanings of, 224 monotonic reads, 164-165 of secondary indexes, 231, 241, 354, 491, 500 ordering guarantees, 339-352 read-after-write, 162-164 sequential, 351 strong (see linearizability) timeliness and integrity, 524 using quorums, 181, 334 consistent hashing, 204 consistent prefix reads, 165 constraints (databases), 225, 248 asynchronously checked, 526 coordination avoidance, 527 ensuring idempotence, 519 in log-based systems, 521-524 across multiple partitions, 522 in two-phase commit, 355, 357 relation to consensus, 374, 521 relation to event ordering, 347 requiring linearizability, 330 Consul (service discovery), 372 consumers (message streams), 137, 440 backpressure, 441 consumer offsets in logs, 449 failures, 445, 449 fan-out, 11, 445, 448 load balancing, 444, 448 not keeping up with producers, 441, 450, 502 context switches, 14, 297 convergence (conflict resolution), 172-174, 322 coordination avoidance, 527 cross-datacenter, 168, 493 cross-partition ordering, 256, 294, 348, 523 services, 330, 370-373 coordinator (in 2PC), 356 failure, 358 in XA transactions, 361-364 recovery, 363 copy-on-write (B-trees), 82, 242 CORBA (Common Object Request Broker Architecture), 134 correctness, 6 auditability, 528-533 Byzantine fault tolerance, 305, 532 dealing with partial failures, 274 in log-based systems, 521-524 of algorithm within system model, 308 of compensating transactions, 355 of consensus, 368 of derived data, 497, 531 of immutable data, 461 of personal data, 535, 540 of time, 176, 289-295 of transactions, 225, 515, 529 timeliness and integrity, 524-528 corruption of data detecting, 519, 530-533 due to pathological memory access, 529 due to radiation, 305 due to split brain, 158, 302 due to weak transaction isolation, 233 formalization in consensus, 366 integrity as absence of, 524 network packets, 306 on disks, 227 preventing using write-ahead logs, 82 recovering from, 414, 460 Couchbase (database) durability, 89 hash partitioning, 203-204, 211 rebalancing, 213 request routing, 216 CouchDB (database) B-tree storage, 242 change feed, 456 document data model, 31 join support, 34 MapReduce support, 46, 400 replication, 170, 173 covering indexes, 86 CPUs cache coherence and memory barriers, 338 caching and pipelining, 99, 428 increasing parallelism, 43 CRDTs (see conflict-free replicated datatypes) CREATE INDEX statement (SQL), 85, 500 credit rating agencies, 535 Crunch (batch processing), 419, 427 hash joins, 409 sharded joins, 408 workflows, 403 cryptography defense against attackers, 306 end-to-end encryption and authentication, 519, 543 proving integrity of data, 532 CSS (Cascading Style Sheets), 44 CSV (comma-separated values), 70, 114, 396 Curator (ZooKeeper recipes), 330, 371 curl (Unix tool), 135, 397 cursor stability, 243 Cypher (query language), 52 comparison to SPARQL, 59 D data corruption (see corruption of data) data cubes, 102 data formats (see encoding) data integration, 490-498, 543 batch and stream processing, 494-498 lambda architecture, 497 maintaining derived state, 495 reprocessing data, 496 unifying, 498 by unbundling databases, 499-515 comparison to federated databases, 501 combining tools by deriving data, 490-494 derived data versus distributed transac‐ tions, 492 limits of total ordering, 493 ordering events to capture causality, 493 reasoning about dataflows, 491 need for, 385 data lakes, 415 data locality (see locality) data models, 27-64 graph-like models, 49-63 Datalog language, 60-63 property graphs, 50 RDF and triple-stores, 55-59 query languages, 42-48 relational model versus document model, 28-42 data protection regulations, 542 data systems, 3 about, 4 Index | 565 concerns when designing, 5 future of, 489-544 correctness, constraints, and integrity, 515-533 data integration, 490-498 unbundling databases, 499-515 heterogeneous, keeping in sync, 452 maintainability, 18-22 possible faults in, 221 reliability, 6-10 hardware faults, 7 human errors, 9 importance of, 10 software errors, 8 scalability, 10-18 unreliable clocks, 287-299 data warehousing, 91-95, 554 comparison to data lakes, 415 ETL (extract-transform-load), 92, 416, 452 keeping data systems in sync, 452 schema design, 93 slowly changing dimension (SCD), 476 data-intensive applications, 3 database triggers (see triggers) database-internal distributed transactions, 360, 364, 477 databases archival storage, 131 comparison of message brokers to, 443 dataflow through, 129 end-to-end argument for, 519-520 checking integrity, 531 inside-out, 504 (see also unbundling databases) output from batch workflows, 412 relation to event streams, 451-464 (see also changelogs) API support for change streams, 456, 506 change data capture, 454-457 event sourcing, 457-459 keeping systems in sync, 452-453 philosophy of immutable events, 459-464 unbundling, 499-515 composing data storage technologies, 499-504 designing applications around dataflow, 504-509 566 | Index observing derived state, 509-515 datacenters geographically distributed, 145, 164, 278, 493 multi-tenancy and shared resources, 284 network architecture, 276 network faults, 279 replication across multiple, 169 leaderless replication, 184 multi-leader replication, 168, 335 dataflow, 128-139, 504-509 correctness of dataflow systems, 525 differential, 504 message-passing, 136-139 reasoning about, 491 through databases, 129 through services, 131-136 dataflow engines, 421-423 comparison to stream processing, 464 directed acyclic graphs (DAG), 424 partitioning, approach to, 429 support for declarative queries, 427 Datalog (query language), 60-63 datatypes binary strings in XML and JSON, 114 conflict-free, 174 in Avro encodings, 122 in Thrift and Protocol Buffers, 121 numbers in XML and JSON, 114 Datomic (database) B-tree storage, 242 data model, 50, 57 Datalog query language, 60 excision (deleting data), 463 languages for transactions, 255 serial execution of transactions, 253 deadlocks detection, in two-phase commit (2PC), 364 in two-phase locking (2PL), 258 Debezium (change data capture), 455 declarative languages, 42, 554 Bloom, 504 CSS and XSL, 44 Cypher, 52 Datalog, 60 for batch processing, 427 recursive SQL queries, 53 relational algebra and SQL, 42 SPARQL, 59 delays bounded network delays, 285 bounded process pauses, 298 unbounded network delays, 282 unbounded process pauses, 296 deleting data, 463 denormalization (data representation), 34, 554 costs, 39 in derived data systems, 386 materialized views, 101 updating derived data, 228, 231, 490 versus normalization, 462 derived data, 386, 439, 554 from change data capture, 454 in event sourcing, 458-458 maintaining derived state through logs, 452-457, 459-463 observing, by subscribing to streams, 512 outputs of batch and stream processing, 495 through application code, 505 versus distributed transactions, 492 deterministic operations, 255, 274, 554 accidental nondeterminism, 423 and fault tolerance, 423, 426 and idempotence, 478, 492 computing derived data, 495, 526, 531 in state machine replication, 349, 452, 458 joins, 476 DevOps, 394 differential dataflow, 504 dimension tables, 94 dimensional modeling (see star schemas) directed acyclic graphs (DAGs), 424 dirty reads (transaction isolation), 234 dirty writes (transaction isolation), 235 discrimination, 534 disks (see hard disks) distributed actor frameworks, 138 distributed filesystems, 398-399 decoupling from query engines, 417 indiscriminately dumping data into, 415 use by MapReduce, 402 distributed systems, 273-312, 554 Byzantine faults, 304-306 cloud versus supercomputing, 275 detecting network faults, 280 faults and partial failures, 274-277 formalization of consensus, 365 impossibility results, 338, 353 issues with failover, 157 limitations of distributed transactions, 363 multi-datacenter, 169, 335 network problems, 277-286 quorums, relying on, 301 reasons for using, 145, 151 synchronized clocks, relying on, 291-295 system models, 306-310 use of clocks and time, 287 distributed transactions (see transactions) Django (web framework), 232 DNS (Domain Name System), 216, 372 Docker (container manager), 506 document data model, 30-42 comparison to relational model, 38-42 document references, 38, 403 document-oriented databases, 31 many-to-many relationships and joins, 36 multi-object transactions, need for, 231 versus relational model convergence of models, 41 data locality, 41 document-partitioned indexes, 206, 217, 411 domain-driven design (DDD), 457 DRBD (Distributed Replicated Block Device), 153 drift (clocks), 289 Drill (query engine), 93 Druid (database), 461 Dryad (dataflow engine), 421 dual writes, problems with, 452, 507 duplicates, suppression of, 517 (see also idempotence) using a unique ID, 518, 522 durability (transactions), 226, 554 duration (time), 287 measurement with monotonic clocks, 288 dynamic partitioning, 212 dynamically typed languages analogy to schema-on-read, 40 code generation and, 127 Dynamo-style databases (see leaderless replica‐ tion) E edges (in graphs), 49, 403 property graph model, 50 edit distance (full-text search), 88 effectively-once semantics, 476, 516 Index | 567 (see also exactly-once semantics) preservation of integrity, 525 elastic systems, 17 Elasticsearch (search server) document-partitioned indexes, 207 partition rebalancing, 211 percolator (stream search), 467 usage example, 4 use of Lucene, 79 ElephantDB (database), 413 Elm (programming language), 504, 512 encodings (data formats), 111-128 Avro, 122-127 binary variants of JSON and XML, 115 compatibility, 112 calling services, 136 using databases, 129-131 using message-passing, 138 defined, 113 JSON, XML, and CSV, 114 language-specific formats, 113 merits of schemas, 127 representations of data, 112 Thrift and Protocol Buffers, 117-121 end-to-end argument, 277, 519-520 checking integrity, 531 publish/subscribe streams, 512 enrichment (stream), 473 Enterprise JavaBeans (EJB), 134 entities (see vertices) epoch (consensus algorithms), 368 epoch (Unix timestamps), 288 equi-joins, 403 erasure coding (error correction), 398 Erlang OTP (actor framework), 139 error handling for network faults, 280 in transactions, 231 error-correcting codes, 277, 398 Esper (CEP engine), 466 etcd (coordination service), 370-373 linearizable operations, 333 locks and leader election, 330 quorum reads, 351 service discovery, 372 use of Raft algorithm, 349, 353 Ethereum (blockchain), 532 Ethernet (networks), 276, 278, 285 packet checksums, 306, 519 568 | Index Etherpad (collaborative editor), 170 ethics, 533-543 code of ethics and professional practice, 533 legislation and self-regulation, 542 predictive analytics, 533-536 amplifying bias, 534 feedback loops, 536 privacy and tracking, 536-543 consent and freedom of choice, 538 data as assets and power, 540 meaning of privacy, 539 surveillance, 537 respect, dignity, and agency, 543, 544 unintended consequences, 533, 536 ETL (extract-transform-load), 92, 405, 452, 554 use of Hadoop for, 416 event sourcing, 457-459 commands and events, 459 comparison to change data capture, 457 comparison to lambda architecture, 497 deriving current state from event log, 458 immutability and auditability, 459, 531 large, reliable data systems, 519, 526 Event Store (database), 458 event streams (see streams) events, 440 deciding on total order of, 493 deriving views from event log, 461 difference to commands, 459 event time versus processing time, 469, 477, 498 immutable, advantages of, 460, 531 ordering to capture causality, 493 reads as, 513 stragglers, 470, 498 timestamp of, in stream processing, 471 EventSource (browser API), 512 eventual consistency, 152, 162, 308, 322 (see also conflicts) and perpetual inconsistency, 525 evolvability, 21, 111 calling services, 136 graph-structured data, 52 of databases, 40, 129-131, 461, 497 of message-passing, 138 reprocessing data, 496, 498 schema evolution in Avro, 123 schema evolution in Thrift and Protocol Buffers, 120 schema-on-read, 39, 111, 128 exactly-once semantics, 360, 476, 516 parity with batch processors, 498 preservation of integrity, 525 exclusive mode (locks), 258 eXtended Architecture transactions (see XA transactions) extract-transform-load (see ETL) F Facebook Presto (query engine), 93 React, Flux, and Redux (user interface libra‐ ries), 512 social graphs, 49 Wormhole (change data capture), 455 fact tables, 93 failover, 157, 554 (see also leader-based replication) in leaderless replication, absence of, 178 leader election, 301, 348, 352 potential problems, 157 failures amplification by distributed transactions, 364, 495 failure detection, 280 automatic rebalancing causing cascading failures, 214 perfect failure detectors, 359 timeouts and unbounded delays, 282, 284 using ZooKeeper, 371 faults versus, 7 partial failures in distributed systems, 275-277, 310 fan-out (messaging systems), 11, 445 fault tolerance, 6-10, 555 abstractions for, 321 formalization in consensus, 365-369 use of replication, 367 human fault tolerance, 414 in batch processing, 406, 414, 422, 425 in log-based systems, 520, 524-526 in stream processing, 476-479 atomic commit, 477 idempotence, 478 maintaining derived state, 495 microbatching and checkpointing, 477 rebuilding state after a failure, 478 of distributed transactions, 362-364 transaction atomicity, 223, 354-361 faults, 6 Byzantine faults, 304-306 failures versus, 7 handled by transactions, 221 handling in supercomputers and cloud computing, 275 hardware, 7 in batch processing versus distributed data‐ bases, 417 in distributed systems, 274-277 introducing deliberately, 7, 280 network faults, 279-281 asymmetric faults, 300 detecting, 280 tolerance of, in multi-leader replication, 169 software errors, 8 tolerating (see fault tolerance) federated databases, 501 fence (CPU instruction), 338 fencing (preventing split brain), 158, 302-304 generating fencing tokens, 349, 370 properties of fencing tokens, 308 stream processors writing to databases, 478, 517 Fibre Channel (networks), 398 field tags (Thrift and Protocol Buffers), 119-121 file descriptors (Unix), 395 financial data, 460 Firebase (database), 456 Flink (processing framework), 421-423 dataflow APIs, 427 fault tolerance, 422, 477, 479 Gelly API (graph processing), 425 integration of batch and stream processing, 495, 498 machine learning, 428 query optimizer, 427 stream processing, 466 flow control, 282, 441, 555 FLP result (on consensus), 353 FlumeJava (dataflow library), 403, 427 followers, 152, 555 (see also leader-based replication) foreign keys, 38, 403 forward compatibility, 112 forward decay (algorithm), 16 Index | 569 Fossil (version control system), 463 shunning (deleting data), 463 FoundationDB (database) serializable transactions, 261, 265, 364 fractal trees, 83 full table scans, 403 full-text search, 555 and fuzzy indexes, 88 building search indexes, 411 Lucene storage engine, 79 functional reactive programming (FRP), 504 functional requirements, 22 futures (asynchronous operations), 135 fuzzy search (see similarity search) G garbage collection immutability and, 463 process pauses for, 14, 296-299, 301 (see also process pauses) genome analysis, 63, 429 geographically distributed datacenters, 145, 164, 278, 493 geospatial indexes, 87 Giraph (graph processing), 425 Git (version control system), 174, 342, 463 GitHub, postmortems, 157, 158, 309 global indexes (see term-partitioned indexes) GlusterFS (distributed filesystem), 398 GNU Coreutils (Linux), 394 GoldenGate (change data capture), 161, 170, 455 (see also Oracle) Google Bigtable (database) data model (see Bigtable data model) partitioning scheme, 199, 202 storage layout, 78 Chubby (lock service), 370 Cloud Dataflow (stream processor), 466, 477, 498 (see also Beam) Cloud Pub/Sub (messaging), 444, 448 Docs (collaborative editor), 170 Dremel (query engine), 93, 96 FlumeJava (dataflow library), 403, 427 GFS (distributed file system), 398 gRPC (RPC framework), 135 MapReduce (batch processing), 390 570 | Index (see also MapReduce) building search indexes, 411 task preemption, 418 Pregel (graph processing), 425 Spanner (see Spanner) TrueTime (clock API), 294 gossip protocol, 216 government use of data, 541 GPS (Global Positioning System) use for clock synchronization, 287, 290, 294, 295 GraphChi (graph processing), 426 graphs, 555 as data models, 49-63 example of graph-structured data, 49 property graphs, 50 RDF and triple-stores, 55-59 versus the network model, 60 processing and analysis, 424-426 fault tolerance, 425 Pregel processing model, 425 query languages Cypher, 52 Datalog, 60-63 recursive SQL queries, 53 SPARQL, 59-59 Gremlin (graph query language), 50 grep (Unix tool), 392 GROUP BY clause (SQL), 406 grouping records in MapReduce, 406 handling skew, 407 H Hadoop (data infrastructure) comparison to distributed databases, 390 comparison to MPP databases, 414-418 comparison to Unix, 413-414, 499 diverse processing models in ecosystem, 417 HDFS distributed filesystem (see HDFS) higher-level tools, 403 join algorithms, 403-410 (see also MapReduce) MapReduce (see MapReduce) YARN (see YARN) happens-before relationship, 340 capturing, 187 concurrency and, 186 hard disks access patterns, 84 detecting corruption, 519, 530 faults in, 7, 227 sequential write throughput, 75, 450 hardware faults, 7 hash indexes, 72-75 broadcast hash joins, 409 partitioned hash joins, 409 hash partitioning, 203-205, 217 consistent hashing, 204 problems with hash mod N, 210 range queries, 204 suitable hash functions, 203 with fixed number of partitions, 210 HAWQ (database), 428 HBase (database) bug due to lack of fencing, 302 bulk loading, 413 column-family data model, 41, 99 dynamic partitioning, 212 key-range partitioning, 202 log-structured storage, 78 request routing, 216 size-tiered compaction, 79 use of HDFS, 417 use of ZooKeeper, 370 HDFS (Hadoop Distributed File System), 398-399 (see also distributed filesystems) checking data integrity, 530 decoupling from query engines, 417 indiscriminately dumping data into, 415 metadata about datasets, 410 NameNode, 398 use by Flink, 479 use by HBase, 212 use by MapReduce, 402 HdrHistogram (numerical library), 16 head (Unix tool), 392 head vertex (property graphs), 51 head-of-line blocking, 15 heap files (databases), 86 Helix (cluster manager), 216 heterogeneous distributed transactions, 360, 364 heuristic decisions (in 2PC), 363 Hibernate (object-relational mapper), 30 hierarchical model, 36 high availability (see fault tolerance) high-frequency trading, 290, 299 high-performance computing (HPC), 275 hinted handoff, 183 histograms, 16 Hive (query engine), 419, 427 for data warehouses, 93 HCatalog and metastore, 410 map-side joins, 409 query optimizer, 427 skewed joins, 408 workflows, 403 Hollerith machines, 390 hopping windows (stream processing), 472 (see also windows) horizontal scaling (see scaling out) HornetQ (messaging), 137, 444 distributed transaction support, 361 hot spots, 201 due to celebrities, 205 for time-series data, 203 in batch processing, 407 relieving, 205 hot standbys (see leader-based replication) HTTP, use in APIs (see services) human errors, 9, 279, 414 HyperDex (database), 88 HyperLogLog (algorithm), 466 I I/O operations, waiting for, 297 IBM DB2 (database) distributed transaction support, 361 recursive query support, 54 serializable isolation, 242, 257 XML and JSON support, 30, 42 electromechanical card-sorting machines, 390 IMS (database), 36 imperative query APIs, 46 InfoSphere Streams (CEP engine), 466 MQ (messaging), 444 distributed transaction support, 361 System R (database), 222 WebSphere (messaging), 137 idempotence, 134, 478, 555 by giving operations unique IDs, 518, 522 idempotent operations, 517 immutability advantages of, 460, 531 Index | 571 deriving state from event log, 459-464 for crash recovery, 75 in B-trees, 82, 242 in event sourcing, 457 inputs to Unix commands, 397 limitations of, 463 Impala (query engine) for data warehouses, 93 hash joins, 409 native code generation, 428 use of HDFS, 417 impedance mismatch, 29 imperative languages, 42 setting element styles (example), 45 in doubt (transaction status), 358 holding locks, 362 orphaned transactions, 363 in-memory databases, 88 durability, 227 serial transaction execution, 253 incidents cascading failures, 9 crashes due to leap seconds, 290 data corruption and financial losses due to concurrency bugs, 233 data corruption on hard disks, 227 data loss due to last-write-wins, 173, 292 data on disks unreadable, 309 deleted items reappearing, 174 disclosure of sensitive data due to primary key reuse, 157 errors in transaction serializability, 529 gigabit network interface with 1 Kb/s throughput, 311 network faults, 279 network interface dropping only inbound packets, 279 network partitions and whole-datacenter failures, 275 poor handling of network faults, 280 sending message to ex-partner, 494 sharks biting undersea cables, 279 split brain due to 1-minute packet delay, 158, 279 vibrations in server rack, 14 violation of uniqueness constraint, 529 indexes, 71, 555 and snapshot isolation, 241 as derived data, 386, 499-504 572 | Index B-trees, 79-83 building in batch processes, 411 clustered, 86 comparison of B-trees and LSM-trees, 83-85 concatenated, 87 covering (with included columns), 86 creating, 500 full-text search, 88 geospatial, 87 hash, 72-75 index-range locking, 260 multi-column, 87 partitioning and secondary indexes, 206-209, 217 secondary, 85 (see also secondary indexes) problems with dual writes, 452, 491 SSTables and LSM-trees, 76-79 updating when data changes, 452, 467 Industrial Revolution, 541 InfiniBand (networks), 285 InfiniteGraph (database), 50 InnoDB (storage engine) clustered index on primary key, 86 not preventing lost updates, 245 preventing write skew, 248, 257 serializable isolation, 257 snapshot isolation support, 239 inside-out databases, 504 (see also unbundling databases) integrating different data systems (see data integration) integrity, 524 coordination-avoiding data systems, 528 correctness of dataflow systems, 525 in consensus formalization, 365 integrity checks, 530 (see also auditing) end-to-end, 519, 531 use of snapshot isolation, 238 maintaining despite software bugs, 529 Interface Definition Language (IDL), 117, 122 intermediate state, materialization of, 420-423 internet services, systems for implementing, 275 invariants, 225 (see also constraints) inversion of control, 396 IP (Internet Protocol) unreliability of, 277 ISDN (Integrated Services Digital Network), 284 isolation (in transactions), 225, 228, 555 correctness and, 515 for single-object writes, 230 serializability, 251-266 actual serial execution, 252-256 serializable snapshot isolation (SSI), 261-266 two-phase locking (2PL), 257-261 violating, 228 weak isolation levels, 233-251 preventing lost updates, 242-246 read committed, 234-237 snapshot isolation, 237-242 iterative processing, 424-426 J Java Database Connectivity (JDBC) distributed transaction support, 361 network drivers, 128 Java Enterprise Edition (EE), 134, 356, 361 Java Message Service (JMS), 444 (see also messaging systems) comparison to log-based messaging, 448, 451 distributed transaction support, 361 message ordering, 446 Java Transaction API (JTA), 355, 361 Java Virtual Machine (JVM) bytecode generation, 428 garbage collection pauses, 296 process reuse in batch processors, 422 JavaScript in MapReduce querying, 46 setting element styles (example), 45 use in advanced queries, 48 Jena (RDF framework), 57 Jepsen (fault tolerance testing), 515 jitter (network delay), 284 joins, 555 by index lookup, 403 expressing as relational operators, 427 in relational and document databases, 34 MapReduce map-side joins, 408-410 broadcast hash joins, 409 merge joins, 410 partitioned hash joins, 409 MapReduce reduce-side joins, 403-408 handling skew, 407 sort-merge joins, 405 parallel execution of, 415 secondary indexes and, 85 stream joins, 472-476 stream-stream join, 473 stream-table join, 473 table-table join, 474 time-dependence of, 475 support in document databases, 42 JOTM (transaction coordinator), 356 JSON Avro schema representation, 122 binary variants, 115 for application data, issues with, 114 in relational databases, 30, 42 representing a résumé (example), 31 Juttle (query language), 504 K k-nearest neighbors, 429 Kafka (messaging), 137, 448 Kafka Connect (database integration), 457, 461 Kafka Streams (stream processor), 466, 467 fault tolerance, 479 leader-based replication, 153 log compaction, 456, 467 message offsets, 447, 478 request routing, 216 transaction support, 477 usage example, 4 Ketama (partitioning library), 213 key-value stores, 70 as batch process output, 412 hash indexes, 72-75 in-memory, 89 partitioning, 201-205 by hash of key, 203, 217 by key range, 202, 217 dynamic partitioning, 212 skew and hot spots, 205 Kryo (Java), 113 Kubernetes (cluster manager), 418, 506 L lambda architecture, 497 Lamport timestamps, 345 Index | 573 Large Hadron Collider (LHC), 64 last write wins (LWW), 173, 334 discarding concurrent writes, 186 problems with, 292 prone to lost updates, 246 late binding, 396 latency instability under two-phase locking, 259 network latency and resource utilization, 286 response time versus, 14 tail latency, 15, 207 leader-based replication, 152-161 (see also replication) failover, 157, 301 handling node outages, 156 implementation of replication logs change data capture, 454-457 (see also changelogs) statement-based, 158 trigger-based replication, 161 write-ahead log (WAL) shipping, 159 linearizability of operations, 333 locking and leader election, 330 log sequence number, 156, 449 read-scaling architecture, 161 relation to consensus, 367 setting up new followers, 155 synchronous versus asynchronous, 153-155 leaderless replication, 177-191 (see also replication) detecting concurrent writes, 184-191 capturing happens-before relationship, 187 happens-before relationship and concur‐ rency, 186 last write wins, 186 merging concurrently written values, 190 version vectors, 191 multi-datacenter, 184 quorums, 179-182 consistency limitations, 181-183, 334 sloppy quorums and hinted handoff, 183 read repair and anti-entropy, 178 leap seconds, 8, 290 in time-of-day clocks, 288 leases, 295 implementation with ZooKeeper, 370 574 | Index need for fencing, 302 ledgers, 460 distributed ledger technologies, 532 legacy systems, maintenance of, 18 less (Unix tool), 397 LevelDB (storage engine), 78 leveled compaction, 79 Levenshtein automata, 88 limping (partial failure), 311 linearizability, 324-338, 555 cost of, 335-338 CAP theorem, 336 memory on multi-core CPUs, 338 definition, 325-329 implementing with total order broadcast, 350 in ZooKeeper, 370 of derived data systems, 492, 524 avoiding coordination, 527 of different replication methods, 332-335 using quorums, 334 relying on, 330-332 constraints and uniqueness, 330 cross-channel timing dependencies, 331 locking and leader election, 330 stronger than causal consistency, 342 using to implement total order broadcast, 351 versus serializability, 329 LinkedIn Azkaban (workflow scheduler), 402 Databus (change data capture), 161, 455 Espresso (database), 31, 126, 130, 153, 216 Helix (cluster manager) (see Helix) profile (example), 30 reference to company entity (example), 34 Rest.li (RPC framework), 135 Voldemort (database) (see Voldemort) Linux, leap second bug, 8, 290 liveness properties, 308 LMDB (storage engine), 82, 242 load approaches to coping with, 17 describing, 11 load testing, 16 load balancing (messaging), 444 local indexes (see document-partitioned indexes) locality (data access), 32, 41, 555 in batch processing, 400, 405, 421 in stateful clients, 170, 511 in stream processing, 474, 478, 508, 522 location transparency, 134 in the actor model, 138 locks, 556 deadlock, 258 distributed locking, 301-304, 330 fencing tokens, 303 implementation with ZooKeeper, 370 relation to consensus, 374 for transaction isolation in snapshot isolation, 239 in two-phase locking (2PL), 257-261 making operations atomic, 243 performance, 258 preventing dirty writes, 236 preventing phantoms with index-range locks, 260, 265 read locks (shared mode), 236, 258 shared mode and exclusive mode, 258 in two-phase commit (2PC) deadlock detection, 364 in-doubt transactions holding locks, 362 materializing conflicts with, 251 preventing lost updates by explicit locking, 244 log sequence number, 156, 449 logic programming languages, 504 logical clocks, 293, 343, 494 for read-after-write consistency, 164 logical logs, 160 logs (data structure), 71, 556 advantages of immutability, 460 compaction, 73, 79, 456, 460 for stream operator state, 479 creating using total order broadcast, 349 implementing uniqueness constraints, 522 log-based messaging, 446-451 comparison to traditional messaging, 448, 451 consumer offsets, 449 disk space usage, 450 replaying old messages, 451, 496, 498 slow consumers, 450 using logs for message storage, 447 log-structured storage, 71-79 log-structured merge tree (see LSMtrees) replication, 152, 158-161 change data capture, 454-457 (see also changelogs) coordination with snapshot, 156 logical (row-based) replication, 160 statement-based replication, 158 trigger-based replication, 161 write-ahead log (WAL) shipping, 159 scalability limits, 493 loose coupling, 396, 419, 502 lost updates (see updates) LSM-trees (indexes), 78-79 comparison to B-trees, 83-85 Lucene (storage engine), 79 building indexes in batch processes, 411 similarity search, 88 Luigi (workflow scheduler), 402 LWW (see last write wins) M machine learning ethical considerations, 534 (see also ethics) iterative processing, 424 models derived from training data, 505 statistical and numerical algorithms, 428 MADlib (machine learning toolkit), 428 magic scaling sauce, 18 Mahout (machine learning toolkit), 428 maintainability, 18-22, 489 defined, 23 design principles for software systems, 19 evolvability (see evolvability) operability, 19 simplicity and managing complexity, 20 many-to-many relationships in document model versus relational model, 39 modeling as graphs, 49 many-to-one and many-to-many relationships, 33-36 many-to-one relationships, 34 MapReduce (batch processing), 390, 399-400 accessing external services within job, 404, 412 comparison to distributed databases designing for frequent faults, 417 diversity of processing models, 416 diversity of storage, 415 Index | 575 comparison to stream processing, 464 comparison to Unix, 413-414 disadvantages and limitations of, 419 fault tolerance, 406, 414, 422 higher-level tools, 403, 426 implementation in Hadoop, 400-403 the shuffle, 402 implementation in MongoDB, 46-48 machine learning, 428 map-side processing, 408-410 broadcast hash joins, 409 merge joins, 410 partitioned hash joins, 409 mapper and reducer functions, 399 materialization of intermediate state, 419-423 output of batch workflows, 411-413 building search indexes, 411 key-value stores, 412 reduce-side processing, 403-408 analysis of user activity events (exam‐ ple), 404 grouping records by same key, 406 handling skew, 407 sort-merge joins, 405 workflows, 402 marshalling (see encoding) massively parallel processing (MPP), 216 comparison to composing storage technolo‐ gies, 502 comparison to Hadoop, 414-418, 428 master-master replication (see multi-leader replication) master-slave replication (see leader-based repli‐ cation) materialization, 556 aggregate values, 101 conflicts, 251 intermediate state (batch processing), 420-423 materialized views, 101 as derived data, 386, 499-504 maintaining, using stream processing, 467, 475 Maven (Java build tool), 428 Maxwell (change data capture), 455 mean, 14 media monitoring, 467 median, 14 576 | Index meeting room booking (example), 249, 259, 521 membership services, 372 Memcached (caching server), 4, 89 memory in-memory databases, 88 durability, 227 serial transaction execution, 253 in-memory representation of data, 112 random bit-flips in, 529 use by indexes, 72, 77 memory barrier (CPU instruction), 338 MemSQL (database) in-memory storage, 89 read committed isolation, 236 memtable (in LSM-trees), 78 Mercurial (version control system), 463 merge joins, MapReduce map-side, 410 mergeable persistent data structures, 174 merging sorted files, 76, 402, 405 Merkle trees, 532 Mesos (cluster manager), 418, 506 message brokers (see messaging systems) message-passing, 136-139 advantages over direct RPC, 137 distributed actor frameworks, 138 evolvability, 138 MessagePack (encoding format), 116 messages exactly-once semantics, 360, 476 loss of, 442 using total order broadcast, 348 messaging systems, 440-451 (see also streams) backpressure, buffering, or dropping mes‐ sages, 441 brokerless messaging, 442 event logs, 446-451 comparison to traditional messaging, 448, 451 consumer offsets, 449 replaying old messages, 451, 496, 498 slow consumers, 450 message brokers, 443-446 acknowledgements and redelivery, 445 comparison to event logs, 448, 451 multiple consumers of same topic, 444 reliability, 442 uniqueness in log-based messaging, 522 Meteor (web framework), 456 microbatching, 477, 495 microservices, 132 (see also services) causal dependencies across services, 493 loose coupling, 502 relation to batch/stream processors, 389, 508 Microsoft Azure Service Bus (messaging), 444 Azure Storage, 155, 398 Azure Stream Analytics, 466 DCOM (Distributed Component Object Model), 134 MSDTC (transaction coordinator), 356 Orleans (see Orleans) SQL Server (see SQL Server) migrating (rewriting) data, 40, 130, 461, 497 modulus operator (%), 210 MongoDB (database) aggregation pipeline, 48 atomic operations, 243 BSON, 41 document data model, 31 hash partitioning (sharding), 203-204 key-range partitioning, 202 lack of join support, 34, 42 leader-based replication, 153 MapReduce support, 46, 400 oplog parsing, 455, 456 partition splitting, 212 request routing, 216 secondary indexes, 207 Mongoriver (change data capture), 455 monitoring, 10, 19 monotonic clocks, 288 monotonic reads, 164 MPP (see massively parallel processing) MSMQ (messaging), 361 multi-column indexes, 87 multi-leader replication, 168-177 (see also replication) handling write conflicts, 171 conflict avoidance, 172 converging toward a consistent state, 172 custom conflict resolution logic, 173 determining what is a conflict, 174 linearizability, lack of, 333 replication topologies, 175-177 use cases, 168 clients with offline operation, 170 collaborative editing, 170 multi-datacenter replication, 168, 335 multi-object transactions, 228 need for, 231 Multi-Paxos (total order broadcast), 367 multi-table index cluster tables (Oracle), 41 multi-tenancy, 284 multi-version concurrency control (MVCC), 239, 266 detecting stale MVCC reads, 263 indexes and snapshot isolation, 241 mutual exclusion, 261 (see also locks) MySQL (database) binlog coordinates, 156 binlog parsing for change data capture, 455 circular replication topology, 175 consistent snapshots, 156 distributed transaction support, 361 InnoDB storage engine (see InnoDB) JSON support, 30, 42 leader-based replication, 153 performance of XA transactions, 360 row-based replication, 160 schema changes in, 40 snapshot isolation support, 242 (see also InnoDB) statement-based replication, 159 Tungsten Replicator (multi-leader replica‐ tion), 170 conflict detection, 177 N nanomsg (messaging library), 442 Narayana (transaction coordinator), 356 NATS (messaging), 137 near-real-time (nearline) processing, 390 (see also stream processing) Neo4j (database) Cypher query language, 52 graph data model, 50 Nephele (dataflow engine), 421 netcat (Unix tool), 397 Netflix Chaos Monkey, 7, 280 Network Attached Storage (NAS), 146, 398 network model, 36 Index | 577 graph databases versus, 60 imperative query APIs, 46 Network Time Protocol (see NTP) networks congestion and queueing, 282 datacenter network topologies, 276 faults (see faults) linearizability and network delays, 338 network partitions, 279, 337 timeouts and unbounded delays, 281 next-key locking, 260 nodes (in graphs) (see vertices) nodes (processes), 556 handling outages in leader-based replica‐ tion, 156 system models for failure, 307 noisy neighbors, 284 nonblocking atomic commit, 359 nondeterministic operations accidental nondeterminism, 423 partial failures in distributed systems, 275 nonfunctional requirements, 22 nonrepeatable reads, 238 (see also read skew) normalization (data representation), 33, 556 executing joins, 39, 42, 403 foreign key references, 231 in systems of record, 386 versus denormalization, 462 NoSQL, 29, 499 transactions and, 223 Notation3 (N3), 56 npm (package manager), 428 NTP (Network Time Protocol), 287 accuracy, 289, 293 adjustments to monotonic clocks, 289 multiple server addresses, 306 numbers, in XML and JSON encodings, 114 O object-relational mapping (ORM) frameworks, 30 error handling and aborted transactions, 232 unsafe read-modify-write cycle code, 244 object-relational mismatch, 29 observer pattern, 506 offline systems, 390 (see also batch processing) 578 | Index stateful, offline-capable clients, 170, 511 offline-first applications, 511 offsets consumer offsets in partitioned logs, 449 messages in partitioned logs, 447 OLAP (online analytic processing), 91, 556 data cubes, 102 OLTP (online transaction processing), 90, 556 analytics queries versus, 411 workload characteristics, 253 one-to-many relationships, 30 JSON representation, 32 online systems, 389 (see also services) Oozie (workflow scheduler), 402 OpenAPI (service definition format), 133 OpenStack Nova (cloud infrastructure) use of ZooKeeper, 370 Swift (object storage), 398 operability, 19 operating systems versus databases, 499 operation identifiers, 518, 522 operational transformation, 174 operators, 421 flow of data between, 424 in stream processing, 464 optimistic concurrency control, 261 Oracle (database) distributed transaction support, 361 GoldenGate (change data capture), 161, 170, 455 lack of serializability, 226 leader-based replication, 153 multi-table index cluster tables, 41 not preventing write skew, 248 partitioned indexes, 209 PL/SQL language, 255 preventing lost updates, 245 read committed isolation, 236 Real Application Clusters (RAC), 330 recursive query support, 54 snapshot isolation support, 239, 242 TimesTen (in-memory database), 89 WAL-based replication, 160 XML support, 30 ordering, 339-352 by sequence numbers, 343-348 causal ordering, 339-343 partial order, 341 limits of total ordering, 493 total order broadcast, 348-352 Orleans (actor framework), 139 outliers (response time), 14 Oz (programming language), 504 P package managers, 428, 505 packet switching, 285 packets corruption of, 306 sending via UDP, 442 PageRank (algorithm), 49, 424 paging (see virtual memory) ParAccel (database), 93 parallel databases (see massively parallel pro‐ cessing) parallel execution of graph analysis algorithms, 426 queries in MPP databases, 216 Parquet (data format), 96, 131 (see also column-oriented storage) use in Hadoop, 414 partial failures, 275, 310 limping, 311 partial order, 341 partitioning, 199-218, 556 and replication, 200 in batch processing, 429 multi-partition operations, 514 enforcing constraints, 522 secondary index maintenance, 495 of key-value data, 201-205 by key range, 202 skew and hot spots, 205 rebalancing partitions, 209-214 automatic or manual rebalancing, 213 problems with hash mod N, 210 using dynamic partitioning, 212 using fixed number of partitions, 210 using N partitions per node, 212 replication and, 147 request routing, 214-216 secondary indexes, 206-209 document-based partitioning, 206 term-based partitioning, 208 serial execution of transactions and, 255 Paxos (consensus algorithm), 366 ballot number, 368 Multi-Paxos (total order broadcast), 367 percentiles, 14, 556 calculating efficiently, 16 importance of high percentiles, 16 use in service level agreements (SLAs), 15 Percona XtraBackup (MySQL tool), 156 performance describing, 13 of distributed transactions, 360 of in-memory databases, 89 of linearizability, 338 of multi-leader replication, 169 perpetual inconsistency, 525 pessimistic concurrency control, 261 phantoms (transaction isolation), 250 materializing conflicts, 251 preventing, in serializability, 259 physical clocks (see clocks) pickle (Python), 113 Pig (dataflow language), 419, 427 replicated joins, 409 skewed joins, 407 workflows, 403 Pinball (workflow scheduler), 402 pipelined execution, 423 in Unix, 394 point in time, 287 polyglot persistence, 29 polystores, 501 PostgreSQL (database) BDR (multi-leader replication), 170 causal ordering of writes, 177 Bottled Water (change data capture), 455 Bucardo (trigger-based replication), 161, 173 distributed transaction support, 361 foreign data wrappers, 501 full text search support, 490 leader-based replication, 153 log sequence number, 156 MVCC implementation, 239, 241 PL/pgSQL language, 255 PostGIS geospatial indexes, 87 preventing lost updates, 245 preventing write skew, 248, 261 read committed isolation, 236 recursive query support, 54 representing graphs, 51 Index | 579 serializable snapshot isolation (SSI), 261 snapshot isolation support, 239, 242 WAL-based replication, 160 XML and JSON support, 30, 42 pre-splitting, 212 Precision Time Protocol (PTP), 290 predicate locks, 259 predictive analytics, 533-536 amplifying bias, 534 ethics of (see ethics) feedback loops, 536 preemption of datacenter resources, 418 of threads, 298 Pregel processing model, 425 primary keys, 85, 556 compound primary key (Cassandra), 204 primary-secondary replication (see leaderbased replication) privacy, 536-543 consent and freedom of choice, 538 data as assets and power, 540 deleting data, 463 ethical considerations (see ethics) legislation and self-regulation, 542 meaning of, 539 surveillance, 537 tracking behavioral data, 536 probabilistic algorithms, 16, 466 process pauses, 295-299 processing time (of events), 469 producers (message streams), 440 programming languages dataflow languages, 504 for stored procedures, 255 functional reactive programming (FRP), 504 logic programming, 504 Prolog (language), 61 (see also Datalog) promises (asynchronous operations), 135 property graphs, 50 Cypher query language, 52 Protocol Buffers (data format), 117-121 field tags and schema evolution, 120 provenance of data, 531 publish/subscribe model, 441 publishers (message streams), 440 punch card tabulating machines, 390 580 | Index pure functions, 48 putting computation near data, 400 Q Qpid (messaging), 444 quality of service (QoS), 285 Quantcast File System (distributed filesystem), 398 query languages, 42-48 aggregation pipeline, 48 CSS and XSL, 44 Cypher, 52 Datalog, 60 Juttle, 504 MapReduce querying, 46-48 recursive SQL queries, 53 relational algebra and SQL, 42 SPARQL, 59 query optimizers, 37, 427 queueing delays (networks), 282 head-of-line blocking, 15 latency and response time, 14 queues (messaging), 137 quorums, 179-182, 556 for leaderless replication, 179 in consensus algorithms, 368 limitations of consistency, 181-183, 334 making decisions in distributed systems, 301 monitoring staleness, 182 multi-datacenter replication, 184 relying on durability, 309 sloppy quorums and hinted handoff, 183 R R-trees (indexes), 87 RabbitMQ (messaging), 137, 444 leader-based replication, 153 race conditions, 225 (see also concurrency) avoiding with linearizability, 331 caused by dual writes, 452 dirty writes, 235 in counter increments, 235 lost updates, 242-246 preventing with event logs, 462, 507 preventing with serializable isolation, 252 write skew, 246-251 Raft (consensus algorithm), 366 sensitivity to network problems, 369 term number, 368 use in etcd, 353 RAID (Redundant Array of Independent Disks), 7, 398 railways, schema migration on, 496 RAMCloud (in-memory storage), 89 ranking algorithms, 424 RDF (Resource Description Framework), 57 querying with SPARQL, 59 RDMA (Remote Direct Memory Access), 276 read committed isolation level, 234-237 implementing, 236 multi-version concurrency control (MVCC), 239 no dirty reads, 234 no dirty writes, 235 read path (derived data), 509 read repair (leaderless replication), 178 for linearizability, 335 read replicas (see leader-based replication) read skew (transaction isolation), 238, 266 as violation of causality, 340 read-after-write consistency, 163, 524 cross-device, 164 read-modify-write cycle, 243 read-scaling architecture, 161 reads as events, 513 real-time collaborative editing, 170 near-real-time processing, 390 (see also stream processing) publish/subscribe dataflow, 513 response time guarantees, 298 time-of-day clocks, 288 rebalancing partitions, 209-214, 556 (see also partitioning) automatic or manual rebalancing, 213 dynamic partitioning, 212 fixed number of partitions, 210 fixed number of partitions per node, 212 problems with hash mod N, 210 recency guarantee, 324 recommendation engines batch process outputs, 412 batch workflows, 403, 420 iterative processing, 424 statistical and numerical algorithms, 428 records, 399 events in stream processing, 440 recursive common table expressions (SQL), 54 redelivery (messaging), 445 Redis (database) atomic operations, 243 durability, 89 Lua scripting, 255 single-threaded execution, 253 usage example, 4 redundancy hardware components, 7 of derived data, 386 (see also derived data) Reed–Solomon codes (error correction), 398 refactoring, 22 (see also evolvability) regions (partitioning), 199 register (data structure), 325 relational data model, 28-42 comparison to document model, 38-42 graph queries in SQL, 53 in-memory databases with, 89 many-to-one and many-to-many relation‐ ships, 33 multi-object transactions, need for, 231 NoSQL as alternative to, 29 object-relational mismatch, 29 relational algebra and SQL, 42 versus document model convergence of models, 41 data locality, 41 relational databases eventual consistency, 162 history, 28 leader-based replication, 153 logical logs, 160 philosophy compared to Unix, 499, 501 schema changes, 40, 111, 130 statement-based replication, 158 use of B-tree indexes, 80 relationships (see edges) reliability, 6-10, 489 building a reliable system from unreliable components, 276 defined, 6, 22 hardware faults, 7 human errors, 9 importance of, 10 of messaging systems, 442 Index | 581 software errors, 8 Remote Method Invocation (Java RMI), 134 remote procedure calls (RPCs), 134-136 (see also services) based on futures, 135 data encoding and evolution, 136 issues with, 134 using Avro, 126, 135 using Thrift, 135 versus message brokers, 137 repeatable reads (transaction isolation), 242 replicas, 152 replication, 151-193, 556 and durability, 227 chain replication, 155 conflict resolution and, 246 consistency properties, 161-167 consistent prefix reads, 165 monotonic reads, 164 reading your own writes, 162 in distributed filesystems, 398 leaderless, 177-191 detecting concurrent writes, 184-191 limitations of quorum consistency, 181-183, 334 sloppy quorums and hinted handoff, 183 monitoring staleness, 182 multi-leader, 168-177 across multiple datacenters, 168, 335 handling write conflicts, 171-175 replication topologies, 175-177 partitioning and, 147, 200 reasons for using, 145, 151 single-leader, 152-161 failover, 157 implementation of replication logs, 158-161 relation to consensus, 367 setting up new followers, 155 synchronous versus asynchronous, 153-155 state machine replication, 349, 452 using erasure coding, 398 with heterogeneous data systems, 453 replication logs (see logs) reprocessing data, 496, 498 (see also evolvability) from log-based messaging, 451 request routing, 214-216 582 | Index approaches to, 214 parallel query execution, 216 resilient systems, 6 (see also fault tolerance) response time as performance metric for services, 13, 389 guarantees on, 298 latency versus, 14 mean and percentiles, 14 user experience, 15 responsibility and accountability, 535 REST (Representational State Transfer), 133 (see also services) RethinkDB (database) document data model, 31 dynamic partitioning, 212 join support, 34, 42 key-range partitioning, 202 leader-based replication, 153 subscribing to changes, 456 Riak (database) Bitcask storage engine, 72 CRDTs, 174, 191 dotted version vectors, 191 gossip protocol, 216 hash partitioning, 203-204, 211 last-write-wins conflict resolution, 186 leaderless replication, 177 LevelDB storage engine, 78 linearizability, lack of, 335 multi-datacenter support, 184 preventing lost updates across replicas, 246 rebalancing, 213 search feature, 209 secondary indexes, 207 siblings (concurrently written values), 190 sloppy quorums, 184 ring buffers, 450 Ripple (cryptocurrency), 532 rockets, 10, 36, 305 RocksDB (storage engine), 78 leveled compaction, 79 rollbacks (transactions), 222 rolling upgrades, 8, 112 routing (see request routing) row-oriented storage, 96 row-based replication, 160 rowhammer (memory corruption), 529 RPCs (see remote procedure calls) Rubygems (package manager), 428 rules (Datalog), 61 S safety and liveness properties, 308 in consensus algorithms, 366 in transactions, 222 sagas (see compensating transactions) Samza (stream processor), 466, 467 fault tolerance, 479 streaming SQL support, 466 sandboxes, 9 SAP HANA (database), 93 scalability, 10-18, 489 approaches for coping with load, 17 defined, 22 describing load, 11 describing performance, 13 partitioning and, 199 replication and, 161 scaling up versus scaling out, 146 scaling out, 17, 146 (see also shared-nothing architecture) scaling up, 17, 146 scatter/gather approach, querying partitioned databases, 207 SCD (slowly changing dimension), 476 schema-on-read, 39 comparison to evolvable schema, 128 in distributed filesystems, 415 schema-on-write, 39 schemaless databases (see schema-on-read) schemas, 557 Avro, 122-127 reader determining writer’s schema, 125 schema evolution, 123 dynamically generated, 126 evolution of, 496 affecting application code, 111 compatibility checking, 126 in databases, 129-131 in message-passing, 138 in service calls, 136 flexibility in document model, 39 for analytics, 93-95 for JSON and XML, 115 merits of, 127 schema migration on railways, 496 Thrift and Protocol Buffers, 117-121 schema evolution, 120 traditional approach to design, fallacy in, 462 searches building search indexes in batch processes, 411 k-nearest neighbors, 429 on streams, 467 partitioned secondary indexes, 206 secondaries (see leader-based replication) secondary indexes, 85, 557 partitioning, 206-209, 217 document-partitioned, 206 index maintenance, 495 term-partitioned, 208 problems with dual writes, 452, 491 updating, transaction isolation and, 231 secondary sorts, 405 sed (Unix tool), 392 self-describing files, 127 self-joins, 480 self-validating systems, 530 semantic web, 57 semi-synchronous replication, 154 sequence number ordering, 343-348 generators, 294, 344 insufficiency for enforcing constraints, 347 Lamport timestamps, 345 use of timestamps, 291, 295, 345 sequential consistency, 351 serializability, 225, 233, 251-266, 557 linearizability versus, 329 pessimistic versus optimistic concurrency control, 261 serial execution, 252-256 partitioning, 255 using stored procedures, 253, 349 serializable snapshot isolation (SSI), 261-266 detecting stale MVCC reads, 263 detecting writes that affect prior reads, 264 distributed execution, 265, 364 performance of SSI, 265 preventing write skew, 262-265 two-phase locking (2PL), 257-261 index-range locks, 260 performance, 258 Serializable (Java), 113 Index | 583 serialization, 113 (see also encoding) service discovery, 135, 214, 372 using DNS, 216, 372 service level agreements (SLAs), 15 service-oriented architecture (SOA), 132 (see also services) services, 131-136 microservices, 132 causal dependencies across services, 493 loose coupling, 502 relation to batch/stream processors, 389, 508 remote procedure calls (RPCs), 134-136 issues with, 134 similarity to databases, 132 web services, 132, 135 session windows (stream processing), 472 (see also windows) sessionization, 407 sharding (see partitioning) shared mode (locks), 258 shared-disk architecture, 146, 398 shared-memory architecture, 146 shared-nothing architecture, 17, 146-147, 557 (see also replication) distributed filesystems, 398 (see also distributed filesystems) partitioning, 199 use of network, 277 sharks biting undersea cables, 279 counting (example), 46-48 finding (example), 42 website about (example), 44 shredding (in relational model), 38 siblings (concurrent values), 190, 246 (see also conflicts) similarity search edit distance, 88 genome data, 63 k-nearest neighbors, 429 single-leader replication (see leader-based rep‐ lication) single-threaded execution, 243, 252 in batch processing, 406, 421, 426 in stream processing, 448, 463, 522 size-tiered compaction, 79 skew, 557 584 | Index clock skew, 291-294, 334 in transaction isolation read skew, 238, 266 write skew, 246-251, 262-265 (see also write skew) meanings of, 238 unbalanced workload, 201 compensating for, 205 due to celebrities, 205 for time-series data, 203 in batch processing, 407 slaves (see leader-based replication) sliding windows (stream processing), 472 (see also windows) sloppy quorums, 183 (see also quorums) lack of linearizability, 334 slowly changing dimension (data warehouses), 476 smearing (leap seconds adjustments), 290 snapshots (databases) causal consistency, 340 computing derived data, 500 in change data capture, 455 serializable snapshot isolation (SSI), 261-266, 329 setting up a new replica, 156 snapshot isolation and repeatable read, 237-242 implementing with MVCC, 239 indexes and MVCC, 241 visibility rules, 240 synchronized clocks for global snapshots, 294 snowflake schemas, 95 SOAP, 133 (see also services) evolvability, 136 software bugs, 8 maintaining integrity, 529 solid state drives (SSDs) access patterns, 84 detecting corruption, 519, 530 faults in, 227 sequential write throughput, 75 Solr (search server) building indexes in batch processes, 411 document-partitioned indexes, 207 request routing, 216 usage example, 4 use of Lucene, 79 sort (Unix tool), 392, 394, 395 sort-merge joins (MapReduce), 405 Sorted String Tables (see SSTables) sorting sort order in column storage, 99 source of truth (see systems of record) Spanner (database) data locality, 41 snapshot isolation using clocks, 295 TrueTime API, 294 Spark (processing framework), 421-423 bytecode generation, 428 dataflow APIs, 427 fault tolerance, 422 for data warehouses, 93 GraphX API (graph processing), 425 machine learning, 428 query optimizer, 427 Spark Streaming, 466 microbatching, 477 stream processing on top of batch process‐ ing, 495 SPARQL (query language), 59 spatial algorithms, 429 split brain, 158, 557 in consensus algorithms, 352, 367 preventing, 322, 333 using fencing tokens to avoid, 302-304 spreadsheets, dataflow programming capabili‐ ties, 504 SQL (Structured Query Language), 21, 28, 43 advantages and limitations of, 416 distributed query execution, 48 graph queries in, 53 isolation levels standard, issues with, 242 query execution on Hadoop, 416 résumé (example), 30 SQL injection vulnerability, 305 SQL on Hadoop, 93 statement-based replication, 158 stored procedures, 255 SQL Server (database) data warehousing support, 93 distributed transaction support, 361 leader-based replication, 153 preventing lost updates, 245 preventing write skew, 248, 257 read committed isolation, 236 recursive query support, 54 serializable isolation, 257 snapshot isolation support, 239 T-SQL language, 255 XML support, 30 SQLstream (stream analytics), 466 SSDs (see solid state drives) SSTables (storage format), 76-79 advantages over hash indexes, 76 concatenated index, 204 constructing and maintaining, 78 making LSM-Tree from, 78 staleness (old data), 162 cross-channel timing dependencies, 331 in leaderless databases, 178 in multi-version concurrency control, 263 monitoring for, 182 of client state, 512 versus linearizability, 324 versus timeliness, 524 standbys (see leader-based replication) star replication topologies, 175 star schemas, 93-95 similarity to event sourcing, 458 Star Wars analogy (event time versus process‐ ing time), 469 state derived from log of immutable events, 459 deriving current state from the event log, 458 interplay between state changes and appli‐ cation code, 507 maintaining derived state, 495 maintenance by stream processor in streamstream joins, 473 observing derived state, 509-515 rebuilding after stream processor failure, 478 separation of application code and, 505 state machine replication, 349, 452 statement-based replication, 158 statically typed languages analogy to schema-on-write, 40 code generation and, 127 statistical and numerical algorithms, 428 StatsD (metrics aggregator), 442 stdin, stdout, 395, 396 Stellar (cryptocurrency), 532 Index | 585 stock market feeds, 442 STONITH (Shoot The Other Node In The Head), 158 stop-the-world (see garbage collection) storage composing data storage technologies, 499-504 diversity of, in MapReduce, 415 Storage Area Network (SAN), 146, 398 storage engines, 69-104 column-oriented, 95-101 column compression, 97-99 defined, 96 distinction between column families and, 99 Parquet, 96, 131 sort order in, 99-100 writing to, 101 comparing requirements for transaction processing and analytics, 90-96 in-memory storage, 88 durability, 227 row-oriented, 70-90 B-trees, 79-83 comparing B-trees and LSM-trees, 83-85 defined, 96 log-structured, 72-79 stored procedures, 161, 253-255, 557 and total order broadcast, 349 pros and cons of, 255 similarity to stream processors, 505 Storm (stream processor), 466 distributed RPC, 468, 514 Trident state handling, 478 straggler events, 470, 498 stream processing, 464-481, 557 accessing external services within job, 474, 477, 478, 517 combining with batch processing lambda architecture, 497 unifying technologies, 498 comparison to batch processing, 464 complex event processing (CEP), 465 fault tolerance, 476-479 atomic commit, 477 idempotence, 478 microbatching and checkpointing, 477 rebuilding state after a failure, 478 for data integration, 494-498 586 | Index maintaining derived state, 495 maintenance of materialized views, 467 messaging systems (see messaging systems) reasoning about time, 468-472 event time versus processing time, 469, 477, 498 knowing when window is ready, 470 types of windows, 472 relation to databases (see streams) relation to services, 508 search on streams, 467 single-threaded execution, 448, 463 stream analytics, 466 stream joins, 472-476 stream-stream join, 473 stream-table join, 473 table-table join, 474 time-dependence of, 475 streams, 440-451 end-to-end, pushing events to clients, 512 messaging systems (see messaging systems) processing (see stream processing) relation to databases, 451-464 (see also changelogs) API support for change streams, 456 change data capture, 454-457 derivative of state by time, 460 event sourcing, 457-459 keeping systems in sync, 452-453 philosophy of immutable events, 459-464 topics, 440 strict serializability, 329 strong consistency (see linearizability) strong one-copy serializability, 329 subjects, predicates, and objects (in triplestores), 55 subscribers (message streams), 440 (see also consumers) supercomputers, 275 surveillance, 537 (see also privacy) Swagger (service definition format), 133 swapping to disk (see virtual memory) synchronous networks, 285, 557 comparison to asynchronous networks, 284 formal model, 307 synchronous replication, 154, 557 chain replication, 155 conflict detection, 172 system models, 300, 306-310 assumptions in, 528 correctness of algorithms, 308 mapping to the real world, 309 safety and liveness, 308 systems of record, 386, 557 change data capture, 454, 491 treating event log as, 460 systems thinking, 536 T t-digest (algorithm), 16 table-table joins, 474 Tableau (data visualization software), 416 tail (Unix tool), 447 tail vertex (property graphs), 51 Tajo (query engine), 93 Tandem NonStop SQL (database), 200 TCP (Transmission Control Protocol), 277 comparison to circuit switching, 285 comparison to UDP, 283 connection failures, 280 flow control, 282, 441 packet checksums, 306, 519, 529 reliability and duplicate suppression, 517 retransmission timeouts, 284 use for transaction sessions, 229 telemetry (see monitoring) Teradata (database), 93, 200 term-partitioned indexes, 208, 217 termination (consensus), 365 Terrapin (database), 413 Tez (dataflow engine), 421-423 fault tolerance, 422 support by higher-level tools, 427 thrashing (out of memory), 297 threads (concurrency) actor model, 138, 468 (see also message-passing) atomic operations, 223 background threads, 73, 85 execution pauses, 286, 296-298 memory barriers, 338 preemption, 298 single (see single-threaded execution) three-phase commit, 359 Thrift (data format), 117-121 BinaryProtocol, 118 CompactProtocol, 119 field tags and schema evolution, 120 throughput, 13, 390 TIBCO, 137 Enterprise Message Service, 444 StreamBase (stream analytics), 466 time concurrency and, 187 cross-channel timing dependencies, 331 in distributed systems, 287-299 (see also clocks) clock synchronization and accuracy, 289 relying on synchronized clocks, 291-295 process pauses, 295-299 reasoning about, in stream processors, 468-472 event time versus processing time, 469, 477, 498 knowing when window is ready, 470 timestamp of events, 471 types of windows, 472 system models for distributed systems, 307 time-dependence in stream joins, 475 time-of-day clocks, 288 timeliness, 524 coordination-avoiding data systems, 528 correctness of dataflow systems, 525 timeouts, 279, 557 dynamic configuration of, 284 for failover, 158 length of, 281 timestamps, 343 assigning to events in stream processing, 471 for read-after-write consistency, 163 for transaction ordering, 295 insufficiency for enforcing constraints, 347 key range partitioning by, 203 Lamport, 345 logical, 494 ordering events, 291, 345 Titan (database), 50 tombstones, 74, 191, 456 topics (messaging), 137, 440 total order, 341, 557 limits of, 493 sequence numbers or timestamps, 344 total order broadcast, 348-352, 493, 522 consensus algorithms and, 366-368 Index | 587 implementation in ZooKeeper and etcd, 370 implementing with linearizable storage, 351 using, 349 using to implement linearizable storage, 350 tracking behavioral data, 536 (see also privacy) transaction coordinator (see coordinator) transaction manager (see coordinator) transaction processing, 28, 90-95 comparison to analytics, 91 comparison to data warehousing, 93 transactions, 221-267, 558 ACID properties of, 223 atomicity, 223 consistency, 224 durability, 226 isolation, 225 compensating (see compensating transac‐ tions) concept of, 222 distributed transactions, 352-364 avoiding, 492, 502, 521-528 failure amplification, 364, 495 in doubt/uncertain status, 358, 362 two-phase commit, 354-359 use of, 360-361 XA transactions, 361-364 OLTP versus analytics queries, 411 purpose of, 222 serializability, 251-266 actual serial execution, 252-256 pessimistic versus optimistic concur‐ rency control, 261 serializable snapshot isolation (SSI), 261-266 two-phase locking (2PL), 257-261 single-object and multi-object, 228-232 handling errors and aborts, 231 need for multi-object transactions, 231 single-object writes, 230 snapshot isolation (see snapshots) weak isolation levels, 233-251 preventing lost updates, 242-246 read committed, 234-238 transitive closure (graph algorithm), 424 trie (data structure), 88 triggers (databases), 161, 441 implementing change data capture, 455 implementing replication, 161 588 | Index triple-stores, 55-59 SPARQL query language, 59 tumbling windows (stream processing), 472 (see also windows) in microbatching, 477 tuple spaces (programming model), 507 Turtle (RDF data format), 56 Twitter constructing home timelines (example), 11, 462, 474, 511 DistributedLog (event log), 448 Finagle (RPC framework), 135 Snowflake (sequence number generator), 294 Summingbird (processing library), 497 two-phase commit (2PC), 353, 355-359, 558 confusion with two-phase locking, 356 coordinator failure, 358 coordinator recovery, 363 how it works, 357 issues in practice, 363 performance cost, 360 transactions holding locks, 362 two-phase locking (2PL), 257-261, 329, 558 confusion with two-phase commit, 356 index-range locks, 260 performance of, 258 type checking, dynamic versus static, 40 U UDP (User Datagram Protocol) comparison to TCP, 283 multicast, 442 unbounded datasets, 439, 558 (see also streams) unbounded delays, 558 in networks, 282 process pauses, 296 unbundling databases, 499-515 composing data storage technologies, 499-504 federation versus unbundling, 501 need for high-level language, 503 designing applications around dataflow, 504-509 observing derived state, 509-515 materialized views and caching, 510 multi-partition data processing, 514 pushing state changes to clients, 512 uncertain (transaction status) (see in doubt) uniform consensus, 365 (see also consensus) uniform interfaces, 395 union type (in Avro), 125 uniq (Unix tool), 392 uniqueness constraints asynchronously checked, 526 requiring consensus, 521 requiring linearizability, 330 uniqueness in log-based messaging, 522 Unix philosophy, 394-397 command-line batch processing, 391-394 Unix pipes versus dataflow engines, 423 comparison to Hadoop, 413-414 comparison to relational databases, 499, 501 comparison to stream processing, 464 composability and uniform interfaces, 395 loose coupling, 396 pipes, 394 relation to Hadoop, 499 UPDATE statement (SQL), 40 updates preventing lost updates, 242-246 atomic write operations, 243 automatically detecting lost updates, 245 compare-and-set operations, 245 conflict resolution and replication, 246 using explicit locking, 244 preventing write skew, 246-251 V validity (consensus), 365 vBuckets (partitioning), 199 vector clocks, 191 (see also version vectors) vectorized processing, 99, 428 verification, 528-533 avoiding blind trust, 530 culture of, 530 designing for auditability, 531 end-to-end integrity checks, 531 tools for auditable data systems, 532 version control systems, reliance on immutable data, 463 version vectors, 177, 191 capturing causal dependencies, 343 versus vector clocks, 191 Vertica (database), 93 handling writes, 101 replicas using different sort orders, 100 vertical scaling (see scaling up) vertices (in graphs), 49 property graph model, 50 Viewstamped Replication (consensus algo‐ rithm), 366 view number, 368 virtual machines, 146 (see also cloud computing) context switches, 297 network performance, 282 noisy neighbors, 284 reliability in cloud services, 8 virtualized clocks in, 290 virtual memory process pauses due to page faults, 14, 297 versus memory management by databases, 89 VisiCalc (spreadsheets), 504 vnodes (partitioning), 199 Voice over IP (VoIP), 283 Voldemort (database) building read-only stores in batch processes, 413 hash partitioning, 203-204, 211 leaderless replication, 177 multi-datacenter support, 184 rebalancing, 213 reliance on read repair, 179 sloppy quorums, 184 VoltDB (database) cross-partition serializability, 256 deterministic stored procedures, 255 in-memory storage, 89 output streams, 456 secondary indexes, 207 serial execution of transactions, 253 statement-based replication, 159, 479 transactions in stream processing, 477 W WAL (write-ahead log), 82 web services (see services) Web Services Description Language (WSDL), 133 webhooks, 443 webMethods (messaging), 137 WebSocket (protocol), 512 Index | 589 windows (stream processing), 466, 468-472 infinite windows for changelogs, 467, 474 knowing when all events have arrived, 470 stream joins within a window, 473 types of windows, 472 winners (conflict resolution), 173 WITH RECURSIVE syntax (SQL), 54 workflows (MapReduce), 402 outputs, 411-414 key-value stores, 412 search indexes, 411 with map-side joins, 410 working set, 393 write amplification, 84 write path (derived data), 509 write skew (transaction isolation), 246-251 characterizing, 246-251, 262 examples of, 247, 249 materializing conflicts, 251 occurrence in practice, 529 phantoms, 250 preventing in snapshot isolation, 262-265 in two-phase locking, 259-261 options for, 248 write-ahead log (WAL), 82, 159 writes (database) atomic write operations, 243 detecting writes affecting prior reads, 264 preventing dirty writes with read commit‐ ted, 235 WS-* framework, 133 (see also services) WS-AtomicTransaction (2PC), 355 590 | Index X XA transactions, 355, 361-364 heuristic decisions, 363 limitations of, 363 xargs (Unix tool), 392, 396 XML binary variants, 115 encoding RDF data, 57 for application data, issues with, 114 in relational databases, 30, 41 XSL/XPath, 45 Y Yahoo!


pages: 1,237 words: 227,370

Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems by Martin Kleppmann

active measures, Amazon Web Services, bitcoin, blockchain, business intelligence, business process, c2.com, cloud computing, collaborative editing, commoditize, conceptual framework, cryptocurrency, database schema, DevOps, distributed ledger, Donald Knuth, Edward Snowden, Ethereum, ethereum blockchain, fault tolerance, finite state, Flash crash, full text search, general-purpose programming language, informal economy, information retrieval, Infrastructure as a Service, Internet of things, iterative process, John von Neumann, Kubernetes, loose coupling, Marc Andreessen, microservices, natural language processing, Network effects, packet switching, peer-to-peer, performance metric, place-making, premature optimization, recommendation engine, Richard Feynman, self-driving car, semantic web, Shoshana Zuboff, social graph, social web, software as a service, software is eating the world, sorting algorithm, source of truth, SPARQL, speech recognition, statistical model, undersea cable, web application, WebSocket, wikimedia commons

., an email or a web page), and it will try to transfer it in the shortest time possible. While a TCP connection is idle, it doesn’t use any bandwidth.ii If datacenter networks and the internet were circuit-switched networks, it would be possible to establish a guaranteed maximum round-trip time when a circuit was set up. However, they are not: Ethernet and IP are packet-switched protocols, which suffer from queueing and thus unbounded delays in the network. These protocols do not have the concept of a circuit. Why do datacenter networks and the internet use packet switching? The answer is that they are optimized for bursty traffic. A circuit is good for an audio or video call, which needs to transfer a fairly constant number of bits per second for the duration of the call. On the other hand, requesting a web page, sending an email, or transferring a file doesn’t have any particular bandwidth requirement—we just want it to complete as quickly as possible.

If you guess too high, the circuit cannot be set up (because the network cannot allow a circuit to be created if its bandwidth allocation cannot be guaranteed). Thus, using circuits for bursty data transfers wastes network capacity and makes transfers unnecessarily slow. By contrast, TCP dynamically adapts the rate of data transfer to the available network capacity. There have been some attempts to build hybrid networks that support both circuit switching and packet switching, such as ATM.iii InfiniBand has some similarities [35]: it implements end-to-end flow control at the link layer, which reduces the need for queueing in the network, although it can still suffer from delays due to link congestion [36]. With careful use of quality of service (QoS, prioritization and scheduling of packets) and admission control (rate-limiting senders), it is possible to emulate circuit switching on packet networks, or provide statistically bounded delay [25, 32].

, Glossaryanalytics queries versus, The Output of Batch Workflows workload characteristics, Actual Serial Execution one-to-many relationships, The Object-Relational MismatchJSON representation, The Object-Relational Mismatch online systems, Batch Processing(see also services) Oozie (workflow scheduler), MapReduce workflows OpenAPI (service definition format), Web services OpenStackNova (cloud infrastructure)use of ZooKeeper, Membership and Coordination Services Swift (object storage), MapReduce and Distributed Filesystems operability, Operability: Making Life Easy for Operations operating systems versus databases, Unbundling Databases operation identifiers, Operation identifiers, Multi-partition request processing operational transformation, Custom conflict resolution logic operators, Dataflow enginesflow of data between, Graphs and Iterative Processing in stream processing, Processing Streams optimistic concurrency control, Pessimistic versus optimistic concurrency control Oracle (database)distributed transaction support, XA transactions GoldenGate (change data capture), Trigger-based replication, Multi-datacenter operation, Implementing change data capture lack of serializability, Isolation leader-based replication, Leaders and Followers multi-table index cluster tables, Data locality for queries not preventing write skew, Characterizing write skew partitioned indexes, Partitioning Secondary Indexes by Term PL/SQL language, Pros and cons of stored procedures preventing lost updates, Automatically detecting lost updates read committed isolation, Implementing read committed Real Application Clusters (RAC), Locking and leader election recursive query support, Graph Queries in SQL snapshot isolation support, Snapshot Isolation and Repeatable Read, Repeatable read and naming confusion TimesTen (in-memory database), Keeping everything in memory WAL-based replication, Write-ahead log (WAL) shipping XML support, The Object-Relational Mismatch ordering, Ordering Guarantees-Implementing total order broadcast using linearizable storageby sequence numbers, Sequence Number Ordering-Timestamp ordering is not sufficient causal ordering, Ordering and Causality-Capturing causal dependenciespartial order, The causal order is not a total order limits of total ordering, The limits of total ordering total order broadcast, Total Order Broadcast-Implementing total order broadcast using linearizable storage Orleans (actor framework), Distributed actor frameworks outliers (response time), Describing Performance Oz (programming language), Designing Applications Around Dataflow P package managers, The move toward declarative query languages, Separation of application code and state packet switching, Can we not simply make network delays predictable? packetscorruption of, Weak forms of lying sending via UDP, Direct messaging from producers to consumers PageRank (algorithm), Graph-Like Data Models, Graphs and Iterative Processing paging (see virtual memory) ParAccel (database), The divergence between OLTP databases and data warehouses parallel databases (see massively parallel processing) parallel executionof graph analysis algorithms, Parallel execution queries in MPP databases, Parallel Query Execution Parquet (data format), Column-Oriented Storage, Archival storage(see also column-oriented storage) use in Hadoop, Philosophy of batch process outputs partial failures, Faults and Partial Failures, Summarylimping, Summary partial order, The causal order is not a total order partitioning, Partitioning-Summary, Glossaryand replication, Partitioning and Replication in batch processing, Summary multi-partition operations, Multi-partition data processingenforcing constraints, Multi-partition request processing secondary index maintenance, Maintaining derived state of key-value data, Partitioning of Key-Value Data-Skewed Workloads and Relieving Hot Spotsby key range, Partitioning by Key Range skew and hot spots, Skewed Workloads and Relieving Hot Spots rebalancing partitions, Rebalancing Partitions-Operations: Automatic or Manual Rebalancingautomatic or manual rebalancing, Operations: Automatic or Manual Rebalancing problems with hash mod N, How not to do it: hash mod N using dynamic partitioning, Dynamic partitioning using fixed number of partitions, Fixed number of partitions using N partitions per node, Partitioning proportionally to nodes replication and, Distributed Data request routing, Request Routing-Parallel Query Execution secondary indexes, Partitioning and Secondary Indexes-Partitioning Secondary Indexes by Termdocument-based partitioning, Partitioning Secondary Indexes by Document term-based partitioning, Partitioning Secondary Indexes by Term serial execution of transactions and, Partitioning Paxos (consensus algorithm), Consensus algorithms and total order broadcastballot number, Epoch numbering and quorums Multi-Paxos (total order broadcast), Consensus algorithms and total order broadcast percentiles, Describing Performance, Glossarycalculating efficiently, Describing Performance importance of high percentiles, Describing Performance use in service level agreements (SLAs), Describing Performance Percona XtraBackup (MySQL tool), Setting Up New Followers performancedescribing, Describing Performance of distributed transactions, Distributed Transactions in Practice of in-memory databases, Keeping everything in memory of linearizability, Linearizability and network delays of multi-leader replication, Multi-datacenter operation perpetual inconsistency, Timeliness and Integrity pessimistic concurrency control, Pessimistic versus optimistic concurrency control phantoms (transaction isolation), Phantoms causing write skewmaterializing conflicts, Materializing conflicts preventing, in serializability, Predicate locks physical clocks (see clocks) pickle (Python), Language-Specific Formats Pig (dataflow language), Beyond MapReduce, High-Level APIs and Languagesreplicated joins, Broadcast hash joins skewed joins, Handling skew workflows, MapReduce workflows Pinball (workflow scheduler), MapReduce workflows pipelined execution, Discussion of materializationin Unix, The Unix Philosophy point in time, Unreliable Clocks polyglot persistence, The Birth of NoSQL polystores, The meta-database of everything PostgreSQL (database)BDR (multi-leader replication), Multi-datacenter operationcausal ordering of writes, Multi-Leader Replication Topologies Bottled Water (change data capture), Implementing change data capture Bucardo (trigger-based replication), Trigger-based replication, Custom conflict resolution logic distributed transaction support, XA transactions foreign data wrappers, The meta-database of everything full text search support, Combining Specialized Tools by Deriving Data leader-based replication, Leaders and Followers log sequence number, Setting Up New Followers MVCC implementation, Implementing snapshot isolation, Indexes and snapshot isolation PL/pgSQL language, Pros and cons of stored procedures PostGIS geospatial indexes, Multi-column indexes preventing lost updates, Automatically detecting lost updates preventing write skew, Characterizing write skew, Serializable Snapshot Isolation (SSI) read committed isolation, Implementing read committed recursive query support, Graph Queries in SQL representing graphs, Property Graphs serializable snapshot isolation (SSI), Serializable Snapshot Isolation (SSI) snapshot isolation support, Snapshot Isolation and Repeatable Read, Repeatable read and naming confusion WAL-based replication, Write-ahead log (WAL) shipping XML and JSON support, The Object-Relational Mismatch, Convergence of document and relational databases pre-splitting, Dynamic partitioning Precision Time Protocol (PTP), Clock Synchronization and Accuracy predicate locks, Predicate locks predictive analytics, Predictive Analytics-Feedback loopsamplifying bias, Bias and discrimination ethics of (see ethics) feedback loops, Feedback loops preemptionof datacenter resources, Designing for frequent faults of threads, Process Pauses Pregel processing model, The Pregel processing model primary keys, Other Indexing Structures, Glossarycompound primary key (Cassandra), Partitioning by Hash of Key primary-secondary replication (see leader-based replication) privacy, Privacy and Tracking-Legislation and self-regulationconsent and freedom of choice, Consent and freedom of choice data as assets and power, Data as assets and power deleting data, Limitations of immutability ethical considerations (see ethics) legislation and self-regulation, Legislation and self-regulation meaning of, Privacy and use of data surveillance, Surveillance tracking behavioral data, Privacy and Tracking probabilistic algorithms, Describing Performance, Stream analytics process pauses, Process Pauses-Limiting the impact of garbage collection processing time (of events), Reasoning About Time producers (message streams), Transmitting Event Streams programming languagesdataflow languages, Designing Applications Around Dataflow for stored procedures, Pros and cons of stored procedures functional reactive programming (FRP), Designing Applications Around Dataflow logic programming, Designing Applications Around Dataflow Prolog (language), The Foundation: Datalog(see also Datalog) promises (asynchronous operations), Current directions for RPC property graphs, Property GraphsCypher query language, The Cypher Query Language Protocol Buffers (data format), Thrift and Protocol Buffers-Datatypes and schema evolutionfield tags and schema evolution, Field tags and schema evolution provenance of data, Designing for auditability publish/subscribe model, Messaging Systems publishers (message streams), Transmitting Event Streams punch card tabulating machines, Batch Processing pure functions, MapReduce Querying putting computation near data, Distributed execution of MapReduce Q Qpid (messaging), Message brokers compared to databases quality of service (QoS), Can we not simply make network delays predictable?


pages: 561 words: 157,589

WTF?: What's the Future and Why It's Up to Us by Tim O'Reilly

4chan, Affordable Care Act / Obamacare, Airbnb, Alvin Roth, Amazon Mechanical Turk, Amazon Web Services, artificial general intelligence, augmented reality, autonomous vehicles, barriers to entry, basic income, Bernie Madoff, Bernie Sanders, Bill Joy: nanobots, bitcoin, blockchain, Bretton Woods, Brewster Kahle, British Empire, business process, call centre, Capital in the Twenty-First Century by Thomas Piketty, Captain Sullenberger Hudson, Chuck Templeton: OpenTable:, Clayton Christensen, clean water, cloud computing, cognitive dissonance, collateralized debt obligation, commoditize, computer vision, corporate governance, corporate raider, creative destruction, crowdsourcing, Danny Hillis, data acquisition, deskilling, DevOps, Donald Davies, Donald Trump, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Filter Bubble, Firefox, Flash crash, full employment, future of work, George Akerlof, gig economy, glass ceiling, Google Glasses, Gordon Gekko, gravity well, greed is good, Guido van Rossum, High speed trading, hiring and firing, Home mortgage interest deduction, Hyperloop, income inequality, index fund, informal economy, information asymmetry, Internet Archive, Internet of things, invention of movable type, invisible hand, iterative process, Jaron Lanier, Jeff Bezos, jitney, job automation, job satisfaction, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, Kevin Kelly, Khan Academy, Kickstarter, knowledge worker, Kodak vs Instagram, Lao Tzu, Larry Wall, Lean Startup, Leonard Kleinrock, Lyft, Marc Andreessen, Mark Zuckerberg, market fundamentalism, Marshall McLuhan, McMansion, microbiome, microservices, minimum viable product, mortgage tax deduction, move fast and break things, move fast and break things, Network effects, new economy, Nicholas Carr, obamacare, Oculus Rift, packet switching, PageRank, pattern recognition, Paul Buchheit, peer-to-peer, peer-to-peer model, Ponzi scheme, race to the bottom, Ralph Nader, randomized controlled trial, RFC: Request For Comment, Richard Feynman, Richard Stallman, ride hailing / ride sharing, Robert Gordon, Robert Metcalfe, Ronald Coase, Sam Altman, school choice, Second Machine Age, secular stagnation, self-driving car, SETI@home, shareholder value, Silicon Valley, Silicon Valley startup, skunkworks, Skype, smart contracts, Snapchat, Social Responsibility of Business Is to Increase Its Profits, social web, software as a service, software patent, spectrum auction, speech recognition, Stephen Hawking, Steve Ballmer, Steve Jobs, Steven Levy, Stewart Brand, strong AI, TaskRabbit, telepresence, the built environment, The Future of Employment, the map is not the territory, The Nature of the Firm, The Rise and Fall of American Growth, The Wealth of Nations by Adam Smith, Thomas Davenport, transaction costs, transcontinental railway, transportation-network company, Travis Kalanick, trickle-down economics, Uber and Lyft, Uber for X, uber lyft, ubercab, universal basic income, US Airways Flight 1549, VA Linux, Watson beat the top human players on Jeopardy!, We are the 99%, web application, Whole Earth Catalog, winner-take-all economy, women in the workforce, Y Combinator, yellow journalism, zero-sum game, Zipcar

But add the idea of fitness functions and a fitness landscape to his insight that simple systems are able to evolve in ways that surprise their creators and you have a powerful tool for seeing and understanding how computer networks and marketplaces work. The Internet itself proves the point. In the 1960s, Paul Baran, Donald Davies, Leonard Kleinrock, and others had developed a theoretical alternative called packet switching to the circuit-switched networks that had characterized the telephone and telegraph. Rather than creating a physical circuit between the two endpoints for the duration of a communication, messages are broken up into small, standardized chunks, shipped by whatever route is most convenient for each packet, and reassembled at their destination. Networks such as NPL in the United Kingdom and ARPANET in the United States were the first packet-switched networks, but by the early 1970s there were dozens, if not hundreds, of incompatible networks, and it had become obvious that some method of interoperability was needed.

See also data Information, The (tech report), 287 innovation waves, xxiii, 46–47, 339 Innovator’s Dilemma, The (Christensen), 351 Instagram, 96–97, 102 Intel, 12–13, 33 Internet and business organization changes, 123–24 commercializing process, 79–81 communications role, 90 cybercrime, 208–9 economic value of, 97 file sharing between users, 25–26 freedom leads to growth, 100–101 free software people and, 15 and GNN, 28–29, 38–39, 79–81, 89, 276 as neutral platform, 202–3 open source infrastructure, 19, 20 as operating system, 27–28, 35, 41 packet switching, 106–7 peer-to-peer file sharing, 26–27 programmers work from inside the application, 120–24 proprietary applications running on open source software, 25 SETI@home project, 26 survey of users, 81 TCP/IP development, 107–8 web spidering, 110 See also World Wide Web Internet Creators Guild, 289 Internet in a Box, 81 invention obvious in retrospect, 71–75 invisible hand theory, 262–70 iPhone, xiii, 32, 128, 136 iPhone App Store, 101, 128, 136 issue-tracking systems, 118–19 “It’s Still Day 1” (Bezos), 124 iTunes, 31 Jacobsen, Mark, 285 Janeway, Bill, 104–5, 115, 238, 247, 263, 274, 277–78, 284 Jefferson, Thomas, 130 Jensen, Michael, 240–41 jobs, xxvi, 301–3, 308, 320–21 and AI, xx–xxi, 91–92, 232–33 caring and sharing aspects, 308–11, 323–24, 332–33 creativity-based, 312–19 displacement and transformation of, 94 and education/training, 303, 304 independent contractor status at Uber and Lyft, 59 labor globalization, 67 and new technology, xvii optimism about the future, 298–302 reducing work hours, 304, 308–11 replacing with higher-value tasks, 94–95 universal basic income for, 305–6, 307–11 See also augmented workers; employees Jobs, Steve, 70, 313 Johnson, Bryan, 330 Johnson, Clay, 149 Johnson, Samuel, 313 Johri, Akhil, 256 Just for Fun (Torvalds), 14 Kahn, Bob, 107 Kalanick, Travis, 54, 69, 75 Kaplan, Esther, 193 Kasriel, Stephane, 333–34 Katsuyama, Brad, 237–38 Kernighan, Brian, 105–6 Kettl, Donald, 129 Keynes, John Maynard, 271–72, 298 Kickstarter, 291–92 Kilpi, Esko, 89–90 Kim, Gene, 122–23 Klein, Ezra, 143 knowledge, sharing vs. hoarding, 296–97, 323–25 Korea, 134 Korzybski, Alfred, 20, 195, 211, 314 Kressel, Henry, 284 Krol, Ed, 28 Kromhout, Peter, 116–17 Kwak, James, 258 labor globalization, 67 labor movement, 262–63 Lang, David, 183 language, 20–21, 323–24 language translation, 155–56, 165–66 Lanier, Jaron, 96 laser eye surgery, xvii Launchbury, John, 209 LaVecchia, Olivia, 103 Law of Conservation of Attractive Profits (Christensen), 24–25, 33–34, 331 Lazonick, William, 245, 247 Learning by Doing (Bessen), 345–47 LeCun, Yann, 164–65, 167, 234, 297 leisure time, 309–10, 314 Lessig, Larry, 130–31 Lessin, Jessica, 287 Lessin, Sam, 331 Levi, Margaret, 60 Levie, Aaron, 85–86 Lewis, Michael, 237 Lincoln, Abraham, 150, 323 Linux Kongress, Würzburg, Germany, 8–11 Linux operating system, xii, 7, 8, 23, 24 Long Now Foundation, 355–56 Loosemore, Tom, 186–87 “Looting” (Akerlof and Romer), 249 Lopez, Nadia, 371 Loukides, Mike, 38 Lucovsky, Mark, 119 Lyft, 47, 54–55, 58, 70, 77, 94, 183, 262, 318.


Digital Accounting: The Effects of the Internet and Erp on Accounting by Ashutosh Deshmukh

accounting loophole / creative accounting, AltaVista, business continuity plan, business intelligence, business process, call centre, computer age, conceptual framework, corporate governance, data acquisition, dumpster diving, fixed income, hypertext link, interest rate swap, inventory management, iterative process, late fees, money market fund, new economy, New Journalism, optical character recognition, packet switching, performance metric, profit maximization, semantic web, shareholder value, six sigma, statistical model, supply-chain management, supply-chain management software, telemarketer, transaction costs, value at risk, web application, Y2K

The development of the Telegraph (communicating as dots and dashes), Transatlantic Cable (as a communication medium) and Telephone all contributed to the development of the Internet. The development of the Advanced Research Projects Agency Network (ARPANET) in the late 1960s heralded the era of interconnected computers. The primary objective of ARPANET was to develop a network that would provide numerous alternate network paths to the ultimate destination. The packet switching mechanism for data Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. A Framework for Digital Accounting 5 transfer was the key. This mechanism split data into different packets and routed it to the destination using different network paths. Thus, if one part of the network was down an alternate network path could be taken, and data flow remained unbroken.

The National Science Foundation (NSF) built a backbone (56 KB) that was primitive by today’s standards, but it was a start. In the 1980s, the NSF took off commercial restrictions for the use of NSFNet, which was by then a primary backbone for carrying Internet messages. Exhibit 3. Timeline for the Internet 1836 1858-’66 1876 1962-’68 1969 1971 1972 1976-’79 1982 1986 1988-’90 1991 1991-’92 1993 1994-’96 1996-’99 2000-’02 ≥? 2004 Telegraph Transatlantic Cable Telephone Packet switching networks developed. • ARPANET. Department of Defense (DOD) establishes nodes at UCLA, Stanford Research Institute and University of Utah. The objective is research into networking. • The idea of Electronic Data Interchange begins to emerge from various industry initiatives. Individuals go online for the first time. E-mail invented. E-mail goes international (Norway and England). Telnet protocol specified.

The resulting EDI data Exhibit 19. Differences in traditional EDI and EDIINT Traditional EDI EDIINT Formatting of the message ANSI X.12 or EDIFACT ANSI X.12 or EDIFACT Proprietary flat files, Web forms (HTML or XML) Enveloping layer X.12.5 Protocol for ANSI X.12 and ISO9735 for EDIFACT MIME envelope on top of the EDI envelope, FTP, HTTP Transport layer VAN protocol, generally X.25 packet switching networks TCP/IP Physical layer Direct connections Dial-up lines VANs Direct connections, dial-up lines, VANs, the Internet Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. Electronic Data Interchange 115 is generally enveloped by X.12.5 protocol in ANSI X.12 standard and ISO 9735 in EDIFACT standard.


Data and the City by Rob Kitchin,Tracey P. Lauriault,Gavin McArdle

A Declaration of the Independence of Cyberspace, bike sharing scheme, bitcoin, blockchain, Bretton Woods, Chelsea Manning, citizen journalism, Claude Shannon: information theory, clean water, cloud computing, complexity theory, conceptual framework, corporate governance, correlation does not imply causation, create, read, update, delete, crowdsourcing, cryptocurrency, dematerialisation, digital map, distributed ledger, fault tolerance, fiat currency, Filter Bubble, floating exchange rates, global value chain, Google Earth, hive mind, Internet of things, Kickstarter, knowledge economy, lifelogging, linked data, loose coupling, new economy, New Urbanism, Nicholas Carr, open economy, openstreetmap, packet switching, pattern recognition, performance metric, place-making, RAND corporation, RFID, Richard Florida, ride hailing / ride sharing, semantic web, sentiment analysis, sharing economy, Silicon Valley, Skype, smart cities, Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia, smart contracts, smart grid, smart meter, social graph, software studies, statistical model, TaskRabbit, text mining, The Chicago School, The Death and Life of Great American Cities, the market place, the medium is the message, the scientific method, Toyota Production System, urban planning, urban sprawl, web application

In a way, this means reading Law against himself: how could the cleancut ‘elemental’ topologies (as in fire, fluid or solid networks) ever account for the full complexity – for the ‘messiness’ (Law 2004) – of the various spatio-temporal behaviours of digital matter? For example, consider a topological reading of the Web’s quintessential immutable mobile: the Internet Protocol (IP) is the technology that breaks up data (e.g. HTML files or bitmaps) into packets of manageable size, which are then independently routed through a network of computers, and finally reassembled at the recipient’s end. Packet switching technology lies at the heart of how the internet works. IP packets are (super‑)mobile in the sense that they traverse the World Wide Web in a matter of microseconds. They are (super‑)immutable in that they require strict adherence to a predefined protocol. In IPv4 the destination address is determined by reading bits 128 through 159 of the packet. If one were to change just one bit in the header, the packet would become meaningless ‘noise in a pipe’.

Even its ‘time to live’ that prevents packets to travel around in circles (defined in bits 64–82, and originally 164 T. Straube intended as an integer value of seconds) has in practice become a ‘hop count’ reduced by one each time it passes through a node in the network. When the count reaches zero, the packet is discarded. Taking cues from Callon’s (2007; 2009) notion of performation, it can be said that this is an instance of an assemblage of digital matter performing the topologies of packet switching technology. These time-spaces are arbitrary in that they overflow – or rather are ‘skew’, or at odds with – preconceived notions of time and space (linear, Euclidian or other). To unearth them, to submit them to critical analysis, they need to be read for what they are rather than moulded to something that is already understood. Post-human time-spaces Consider another example: Git is a popular free and open source versioning system developed by Linus Torvalds.


pages: 394 words: 108,215

What the Dormouse Said: How the Sixties Counterculture Shaped the Personal Computer Industry by John Markoff

Any sufficiently advanced technology is indistinguishable from magic, Apple II, back-to-the-land, beat the dealer, Bill Duvall, Bill Gates: Altair 8800, Buckminster Fuller, California gold rush, card file, computer age, computer vision, conceptual framework, cuban missile crisis, different worldview, Donald Knuth, Douglas Engelbart, Douglas Engelbart, Dynabook, Edward Thorp, El Camino Real, Electric Kool-Aid Acid Test, general-purpose programming language, Golden Gate Park, Hacker Ethic, hypertext link, informal economy, information retrieval, invention of the printing press, Jeff Rulifson, John Markoff, John Nash: game theory, John von Neumann, Kevin Kelly, knowledge worker, Mahatma Gandhi, Menlo Park, Mother of all demos, Norbert Wiener, packet switching, Paul Terrell, popular electronics, QWERTY keyboard, RAND corporation, RFC: Request For Comment, Richard Stallman, Robert X Cringely, Sand Hill Road, Silicon Valley, Silicon Valley startup, South of Market, San Francisco, speech recognition, Steve Crocker, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, Ted Nelson, The Hackers Conference, Thorstein Veblen, Turing test, union organizing, Vannevar Bush, Whole Earth Catalog, William Shockley: the traitorous eight

Ultimately, it was Telnet, electronic mail, and ftp, and not NLS, that would generate the demand that led to the dramatic expansion of the computer network. During 1972, Watson also led the charge at ARC to make NLS more useful to the ARPAnet community. ARPA was under some pressure to show that its new network was actually viable, and articles had already appeared in the computer trade press questioning the entire notion of the packet switching that was at its heart. This was a technique for breaking up digital data into small “packets” so that each packet could be routed separately through a computer network and then resent if necessary. It made it possible to route around network nodes that had stopped functioning, making the network more reliable. Roberts had decreed that in October 1972 there would be an event in Washington, D.C., that would show off the network, in much the same fashion that Engelbart had shown off NLS in 1968 in San Francisco.

Albrecht, Bob est and folk dancing of Homebrew and Moore and Aldus Manutius algorithms Allen, Don Allen, Mary Allen, Paul Allison, Dennis Alpert, Richard Altair Alternatives conference Alto American Documentation Institute Ames Research Laboratory Ampex LSD and Andrews, Don Andrews, Paul antiwar activism Augment lab and Brand and Diffie and draft resistance Duvall and Felsenstein and militancy in Moore and Stanford and Apple Computer Alto and ARPA (Advanced Research Projects Agency) Augment funded by Augment funding terminated by SAIL funded by ARPAnet e-commerce on expansion of file-sharing in launch of Network Information Center (NIC) packet switching and Super AI computer for artificial intelligence (AI) golden years of McCarthy and; see also McCarthy, John modeling human intelligence and superbrain and Turing test and see also Stanford Artificial Intelligence Laboratory Art of Computer Programming, The (Knuth) ASCII AT&T Atari Atlantic Monthly augmentation complexity in Augmentation Research Center (Augmented Human Intellect Research Center) antiwar viewpoint and ARPAnet launch and ARPA’s funding of ARPA’s termination of funding of business manager hired at counterculture and departures from division of Engelbart’s Brooks Hall demonstration Engelbart’s loss of control of est and Fadiman and growth of hippie vibe at Kay at Moore and name change of NLS in, see NLS refashioning of SAIL and social experimentation at teenagers at text editing and Tymshare purchase of Xerox and Baer, Steve Baez, Joan Bakalinsky, Eric Bank of America Barringer, Felicity BASIC “borrowed” copy of Interaccess Tiny Bass, Walter Bates, Roger Baum, Allen Beach, Scott Beautiful Mind, A (Nasar) be-ins Bell, Gordon Bell Laboratories Bender, Dorothy Bennion, Dave Berkeley, Calif.


pages: 492 words: 118,882

The Blockchain Alternative: Rethinking Macroeconomic Policy and Economic Theory by Kariappa Bheemaiah

accounting loophole / creative accounting, Ada Lovelace, Airbnb, algorithmic trading, asset allocation, autonomous vehicles, balance sheet recession, bank run, banks create money, Basel III, basic income, Ben Bernanke: helicopter money, bitcoin, blockchain, Bretton Woods, business cycle, business process, call centre, capital controls, Capital in the Twenty-First Century by Thomas Piketty, cashless society, cellular automata, central bank independence, Claude Shannon: information theory, cloud computing, cognitive dissonance, collateralized debt obligation, commoditize, complexity theory, constrained optimization, corporate governance, creative destruction, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, crowdsourcing, cryptocurrency, David Graeber, deskilling, Diane Coyle, discrete time, disruptive innovation, distributed ledger, diversification, double entry bookkeeping, Ethereum, ethereum blockchain, fiat currency, financial innovation, financial intermediation, Flash crash, floating exchange rates, Fractional reserve banking, full employment, George Akerlof, illegal immigration, income inequality, income per capita, inflation targeting, information asymmetry, interest rate derivative, inventory management, invisible hand, John Maynard Keynes: technological unemployment, John von Neumann, joint-stock company, Joseph Schumpeter, Kenneth Arrow, Kenneth Rogoff, Kevin Kelly, knowledge economy, large denomination, liquidity trap, London Whale, low skilled workers, M-Pesa, Marc Andreessen, market bubble, market fundamentalism, Mexican peso crisis / tequila crisis, MITM: man-in-the-middle, money market fund, money: store of value / unit of account / medium of exchange, mortgage debt, natural language processing, Network effects, new economy, Nikolai Kondratiev, offshore financial centre, packet switching, Pareto efficiency, pattern recognition, peer-to-peer lending, Ponzi scheme, precariat, pre–internet, price mechanism, price stability, private sector deleveraging, profit maximization, QR code, quantitative easing, quantitative trading / quantitative finance, Ray Kurzweil, Real Time Gross Settlement, rent control, rent-seeking, Satoshi Nakamoto, Satyajit Das, savings glut, seigniorage, Silicon Valley, Skype, smart contracts, software as a service, software is eating the world, speech recognition, statistical model, Stephen Hawking, supply-chain management, technology bubble, The Chicago School, The Future of Employment, The Great Moderation, the market place, The Nature of the Firm, the payments system, the scientific method, The Wealth of Nations by Adam Smith, Thomas Kuhn: the structure of scientific revolutions, too big to fail, trade liberalization, transaction costs, Turing machine, Turing test, universal basic income, Von Neumann architecture, Washington Consensus

There is no rule that states that someone has to do this, but it remains a protocol of communication that is commonly followed. In the same vein, TCP/IP was developed as a way for any computer to connect and communicate with the ARPANet. More importantly, by using data packet-switching, Cerf and Khan had found a way to eliminate a single point of failure. As there were many computers attached to the network (the ARPANet), a message could be broken into pieces and sent via various channels. If one of the computers was not connected to the network, the message could still be sent via the other computers. This distributed task-management and communication system is the basis of today’s decentralized ledgers. Based on the fundamental principles of packet-switching and a protocol for decentralized communication, other protocols like HTTP, SMTP and VoIP were developed for specific communication purposes. As protocols evolved, they went on to create a digital, decentralized, and distributed environment that was fertile for innovation.


pages: 587 words: 117,894

Cybersecurity: What Everyone Needs to Know by P. W. Singer, Allan Friedman

4chan, A Declaration of the Independence of Cyberspace, Apple's 1984 Super Bowl advert, barriers to entry, Berlin Wall, bitcoin, blood diamonds, borderless world, Brian Krebs, business continuity plan, Chelsea Manning, cloud computing, crowdsourcing, cuban missile crisis, data acquisition, do-ocracy, drone strike, Edward Snowden, energy security, failed state, Fall of the Berlin Wall, fault tolerance, global supply chain, Google Earth, Internet of things, invention of the telegraph, John Markoff, Julian Assange, Khan Academy, M-Pesa, MITM: man-in-the-middle, mutually assured destruction, Network effects, packet switching, Peace of Westphalia, pre–internet, profit motive, RAND corporation, ransomware, RFC: Request For Comment, risk tolerance, rolodex, Silicon Valley, Skype, smart grid, Steve Jobs, Stuxnet, uranium enrichment, We are Anonymous. We are Legion, web application, WikiLeaks, zero day, zero-sum game

What makes the Internet distinct from prior communication networks like the old telegraphs and then telephone networks, however, is that it is packet-switched instead of circuit-switched. Packets are small digital envelopes of data. At the beginning of each packet, essentially the “outside” of the envelope, is the header, which contains details about the network source, destination, and some basic information about the packet contents. By breaking up flows of data into smaller components, each can be delivered in an independent and decentralized fashion, then reassembled at the endpoint. The network routes each packet as it arrives, a dynamic architecture that creates both flexibility and resiliency. Packet-switching was not developed to allow the United States to maintain communications even in the event of a nuclear attack, a common myth.


pages: 781 words: 226,928

Commodore: A Company on the Edge by Brian Bagnall

Apple II, belly landing, Bill Gates: Altair 8800, Byte Shop, Claude Shannon: information theory, computer age, Douglas Engelbart, Douglas Engelbart, Firefox, game design, index card, inventory management, Isaac Newton, low skilled workers, Menlo Park, packet switching, pink-collar, popular electronics, prediction markets, pre–internet, QWERTY keyboard, Robert Metcalfe, Robert X Cringely, Silicon Valley, special economic zone, Steve Jobs, Steve Wozniak, Ted Nelson

Throughout late 1975 and 1976, Peddle travelled the country promoting the 6502. His microcontroller became part of an important new technology called the ARPANET (later renamed the Internet). A company named Telenet (later acquired by GTE) created its own commercial network. “GTE had their Telenet packet switching,” recalls Mensch. “Telenet was one of the early fiber providers for the early Internet.” Packet switching required fast processors, and originally the company used costly minicomputers as switches, but then decided to move to something less expensive. “Telenet was a very high performance packet switching for communications systems,” says Mensch. “They tried all the other chips and said the only one that can do it is the 6502.” GTE settled on the 6502 and incorporated the processor in its switches throughout the network, making the 6502 an integral part of the early Internet.


pages: 731 words: 134,263

Talk Is Cheap: Switching to Internet Telephones by James E. Gaskin

Debian, packet switching, peer-to-peer, peer-to-peer model, Silicon Valley, Skype, speech recognition, telemarketer

After all, the Post Office isn't held responsible for what individuals mail, and the telephone companies have never been held responsible for what people say to each other over the phone. If the people mail or say something illegal, they may get caught, but the Post Office and telephone company are not considered coconspirators. Bits are bits. And bits in the late 1970s were bits, at least to a study by the Department of Defense. Their own widely publicized report said clearly that a packet switched network, supporting packetized voice traffic, would be cheaper than circuit switched voice, the only technology in place at that time. Bits are bits, and 30 years ago, bits were bits. 1.3.4. Regulatory Issues We can't blame AT&T engineers because they didn't reinvent the telephone on a regular basis; that wasn't their job. The U.S. Government made a deal with AT&T to grant them a monopoly in exchange for building the world's best telephone system.

After all, the Post Office isn't held responsible for what individuals mail, and the telephone companies have never been held responsible for what people say to each other over the phone. If the people mail or say something illegal, they may get caught, but the Post Office and telephone company are not considered coconspirators. Bits are bits. And bits in the late 1970s were bits, at least to a study by the Department of Defense. Their own widely publicized report said clearly that a packet switched network, supporting packetized voice traffic, would be cheaper than circuit switched voice, the only technology in place at that time. Bits are bits, and 30 years ago, bits were bits. 1.3.4. Regulatory Issues We can't blame AT&T engineers because they didn't reinvent the telephone on a regular basis; that wasn't their job. The U.S. Government made a deal with AT&T to grant them a monopoly in exchange for building the world's best telephone system.


pages: 494 words: 142,285

The Future of Ideas: The Fate of the Commons in a Connected World by Lawrence Lessig

AltaVista, Andy Kessler, barriers to entry, business process, Cass Sunstein, commoditize, computer age, creative destruction, dark matter, disintermediation, disruptive innovation, Donald Davies, Erik Brynjolfsson, George Gilder, Hacker Ethic, Hedy Lamarr / George Antheil, Howard Rheingold, Hush-A-Phone, HyperCard, hypertext link, Innovator's Dilemma, invention of hypertext, inventory management, invisible hand, Jean Tirole, Jeff Bezos, Joseph Schumpeter, Kenneth Arrow, Larry Wall, Leonard Kleinrock, linked data, Marc Andreessen, Menlo Park, Mitch Kapor, Network effects, new economy, packet switching, peer-to-peer, peer-to-peer model, price mechanism, profit maximization, RAND corporation, rent control, rent-seeking, RFC: Request For Comment, Richard Stallman, Richard Thaler, Robert Bork, Ronald Coase, Search for Extraterrestrial Intelligence, SETI@home, Silicon Valley, smart grid, software patent, spectrum auction, Steve Crocker, Steven Levy, Stewart Brand, Ted Nelson, Telecommunications Act of 1996, The Chicago School, transaction costs, zero-sum game

He was certain that the system it had built would not withstand a nuclear attack. The network was too concentrated; it had no effective redundancy. So he continued to press his idea for a different telecommunications system. He had a different design for telecommunications, and he wanted AT&T to help him build it. This different model was not the Internet, but it was close to the Internet. Baran proposed a kind of packet-switching technology to replace the persistent circuits around which the telephone system was built. Under AT&T's design, when you called someone in Paris, a circuit was opened between you and Paris. In principle, you could trace the line of copper that linked you to Paris; along that line of copper, all your conversation would travel. Baran's idea was fundamentally different. If you digitized a conversation—translating it from waves to bits—and then chopped the resulting stream into packets, these packets could flow independently across a network and create the impression of a real-time connection on the other end.

., 1956). 20 The idea is developed in Kleinrock's dissertation: Leonard Kleinrock, Message Delay in Communication Nets with Storage (1962, unpublished Ph.D. dissertation, Massachusetts Institute of Technology), which was later published in a modified form. See Leonard Kleinrock, Communication Nets: Stochastic Message Flow and Delay (New York: McGraw-Hill, 1964). See also John Naughton, A Brief History of the Future: The Origins of the Internet (London: Weidenfeld & Nicolson, 1999), 92, 118-19 (discussing other earlier contributors to the Internet). 21 Baran attributes to him the discovery of the term. Interview with Paul Baran (“The term 'packet switching' was first used by Donald Davies of the National Physical Laboratory in England, who independently came up with the same general concept in November 1965.”). 22 Baran confirmed this history to me in an interview. “So the first level of objections was about technology—that I didn't understand how the telephone system worked, [and] that what I'm proposing could not possibly work.” Interview with Paul Baran. 23 Naughton, 107.


pages: 474 words: 130,575

Surveillance Valley: The Rise of the Military-Digital Complex by Yasha Levine

23andMe, activist fund / activist shareholder / activist investor, Airbnb, AltaVista, Amazon Web Services, Anne Wojcicki, anti-communist, Apple's 1984 Super Bowl advert, bitcoin, borderless world, British Empire, call centre, Chelsea Manning, cloud computing, collaborative editing, colonial rule, computer age, computerized markets, corporate governance, crowdsourcing, cryptocurrency, digital map, don't be evil, Donald Trump, Douglas Engelbart, Douglas Engelbart, drone strike, Edward Snowden, El Camino Real, Electric Kool-Aid Acid Test, Elon Musk, fault tolerance, George Gilder, ghettoisation, global village, Google Chrome, Google Earth, Google Hangouts, Howard Zinn, hypertext link, IBM and the Holocaust, index card, Jacob Appelbaum, Jeff Bezos, jimmy wales, John Markoff, John von Neumann, Julian Assange, Kevin Kelly, Kickstarter, life extension, Lyft, Mark Zuckerberg, market bubble, Menlo Park, Mitch Kapor, natural language processing, Network effects, new economy, Norbert Wiener, packet switching, PageRank, Paul Buchheit, peer-to-peer, Peter Thiel, Philip Mirowski, plutocrats, Plutocrats, private military company, RAND corporation, Ronald Reagan, Ross Ulbricht, Satoshi Nakamoto, self-driving car, sentiment analysis, shareholder value, side project, Silicon Valley, Silicon Valley startup, Skype, slashdot, Snapchat, speech recognition, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, Telecommunications Act of 1996, telepresence, telepresence robot, The Bell Curve by Richard Herrnstein and Charles Murray, The Hackers Conference, uber lyft, Whole Earth Catalog, Whole Earth Review, WikiLeaks

At the same time, Kahn pushed into packet satellite networking, setting up an experimental network called SATNET that linked Maryland, West Virginia, England, and Norway; the system was initially designed to carry seismic data from remote installations set up to detect Soviet nuclear tests. ARPANET’s data packet technology worked remarkably well in a wireless setting. But there was one problem: although they were based on the same fundamental data-packet-switching designs, PRNET, SATNET, and ARPANET all used slightly different protocols to run and so could not connect to each other. For all practical purposes, they were standalone networks, which went against the whole concept of networking and minimized their usefulness to the military. ARPA needed all three networks to function as one.65 The question was: How to bring them all together in a simple way?

BBN a few decades later became one of the largest Internet service providers in the country. 62. Guy Raz, “‘Lo’ And Behold: A Communication Revolution,” All Things Considered, NPR, October 29, 2009. 63. Hafner and Lyon, Where Wizards Stay Up Late, 153. 64. Vinton Cerf, interview by Judy O’Neill, April 24, 1990, https://web.archive.org/web/20170104132550/http://americanhistory.si.edu/comphist/vc1.html. 65. Lawrence G. Roberts, “The Evolution of Packet Switching,” November 1978, https://web.archive.org/web/20170310205146/http://www.packet.cc/files /ev-packet-sw.html. 66. “250 Jam Harvard Office,” Boston Globe, September 27, 1969. 67. Victor McElheny, “Sympathy for Protests, but… : How MIT Authorities See Student Scene,” Boston Globe, October 5, 1969. 68. Project Cam Exposed, pamphlet 1, 1969, MIT Institute Archives and Special Collections, Cambridge Project. 69.


pages: 200 words: 47,378

The Internet of Money by Andreas M. Antonopoulos

AltaVista, altcoin, bitcoin, blockchain, clean water, cognitive dissonance, cryptocurrency, disruptive innovation, Ethereum, ethereum blockchain, financial exclusion, global reserve currency, litecoin, London Interbank Offered Rate, Marc Andreessen, Oculus Rift, packet switching, peer-to-peer lending, Ponzi scheme, QR code, ransomware, reserve currency, Satoshi Nakamoto, self-driving car, Skype, smart contracts, the medium is the message, trade route, underbanked, WikiLeaks, zero-sum game

The internet was still failing to scale. But by now, it had been failing to scale for more than a decade, very gracefully, very successfully. 11.1.5. VOIP Will Destroy the Internet Then, someone invented Voice Over IP. Some other people decided, why don’t we just replace the entire phone system with the internet? That was a crazy idea. The phone companies then started this massive campaign to inform us of why packet-switched networks could never carry voice. They said, really, the true quality approach to voice was always going to be hierarchical switch networks owned by national monopoly telecom companies because the internet couldn’t possibly scale to carry the world’s phone calls. Those same phone companies (the ones still in business) now route all of their phone calls over the internet. First, they didn’t want the internet on their phone networks.


pages: 188 words: 9,226

Collaborative Futures by Mike Linksvayer, Michael Mandiberg, Mushon Zer-Aviv

4chan, AGPL, Benjamin Mako Hill, British Empire, citizen journalism, cloud computing, collaborative economy, corporate governance, crowdsourcing, Debian, en.wikipedia.org, Firefox, informal economy, jimmy wales, Kickstarter, late capitalism, loose coupling, Marshall McLuhan, means of production, Naomi Klein, Network effects, optical character recognition, packet switching, postnationalism / post nation state, prediction markets, Richard Stallman, semantic web, Silicon Valley, slashdot, Slavoj Žižek, stealth mode startup, technoutopianism, the medium is the message, The Wisdom of Crowds, web application, WikiLeaks

It doesn't have to be heroic; maybe you just want to browse the most milquetoast sites on the Internet with complete privacy. By using Tor, you join a bunch of strangers in declaring everybody has the right to complete privacy and collaborate anonymously to grant yourself and others that constitutional right. 74 21. Problematizing Attribution “I get credit for a lot of things I didn’t do. I just did a li le piece on packet switching and I get blamed for the whole goddamned Internet, you know? Technology reaches a certain ripeness and the pieces are available and the need is there and the economics look good—it’s going to get invented by somebody.” Paul Baran A few years ago, the unofficial fanclub website of a very popular Spanish band became notorious for reasons beyond their commitment to the band. As is customary, the site included a page with all the lyrics from all the songs recorded by the band over the years, listed in chronological and alphabetical order.


pages: 462 words: 150,129

The Rational Optimist: How Prosperity Evolves by Matt Ridley

"Robert Solow", 23andMe, agricultural Revolution, air freight, back-to-the-land, banking crisis, barriers to entry, Bernie Madoff, British Empire, call centre, carbon footprint, Cesare Marchetti: Marchetti’s constant, charter city, clean water, cloud computing, cognitive dissonance, collateralized debt obligation, colonial exploitation, colonial rule, Corn Laws, creative destruction, credit crunch, David Ricardo: comparative advantage, decarbonisation, dematerialisation, demographic dividend, demographic transition, double entry bookkeeping, Edward Glaeser, en.wikipedia.org, everywhere but in the productivity statistics, falling living standards, feminist movement, financial innovation, Flynn Effect, food miles, Gordon Gekko, greed is good, Hans Rosling, happiness index / gross national happiness, haute cuisine, hedonic treadmill, Hernando de Soto, income inequality, income per capita, Indoor air pollution, informal economy, Intergovernmental Panel on Climate Change (IPCC), invention of agriculture, invisible hand, James Hargreaves, James Watt: steam engine, Jane Jacobs, John Nash: game theory, joint-stock limited liability company, Joseph Schumpeter, Kevin Kelly, Kickstarter, knowledge worker, Kula ring, Mark Zuckerberg, meta analysis, meta-analysis, mutually assured destruction, Naomi Klein, Northern Rock, nuclear winter, oil shale / tar sands, out of africa, packet switching, patent troll, Pax Mongolica, Peter Thiel, phenotype, plutocrats, Plutocrats, Ponzi scheme, Productivity paradox, profit motive, purchasing power parity, race to the bottom, Ray Kurzweil, rent-seeking, rising living standards, Silicon Valley, spice trade, spinning jenny, stem cell, Steve Jobs, Steven Pinker, Stewart Brand, supervolcano, technological singularity, Thales and the olive presses, Thales of Miletus, The Wealth of Nations by Adam Smith, Thorstein Veblen, trade route, transaction costs, ultimatum game, upwardly mobile, urban sprawl, Vernor Vinge, Vilfredo Pareto, wage slave, working poor, working-age population, Y2K, Yogi Berra, zero-sum game

Or, second, you can capture the first-mover advantage, as Sam Walton, the founder of Wal-Mart, did throughout his career. Even as his retailing rivals were catching up, he was forging ahead with new cost-cutting tactics. Intel’s dominance of the microchip industry, and 3M’s of the diversified technology industry, were based not on protecting their inventions so much as on improving them faster than everyone else. Packet switching was the invention that made the internet possible, yet nobody made any royalties out of it. The way to keep your customers, if you are Michael Dell, Steve Jobs or Bill Gates, is to keep making your own products obsolete. The third way to profit from invention is a patent, a copyright or a trademark. The various mechanisms of intellectual property are eerily echoed in the apparently lawless and highly competitive world of real recipes, recipes devised by French chefs for their restaurants.

Abbasids 161, 178 Abelard, Peter 358 aborigines (Australian): division of labour 62, 63, 76; farming 127; technological regress 78–84; trade 90–91, 92 abortion, compulsory 203 Abu Hureyra 127 Acapulco 184 accounting systems 160, 168, 196 Accra 189 Acemoglu, Daron 321 Ache people 61 Acheulean tools 48–9, 50, 275, 373 Achuar people 87 acid rain 280, 281, 304–6, 329, 339 acidification of oceans 280, 340–41 Adams, Henry 289 Aden 177 Adenauer, Konrad 289 Aegean sea 168, 170–71 Afghanistan 14, 208–9, 315, 353 Africa: agriculture 145, 148, 154–5, 326; AIDS epidemic 14, 307–8, 316, 319, 320, 322; colonialism 319–20, 321–2; demographic transition 210, 316, 328; economic growth 315, 326–8, 332, 347; international aid 317–19, 322, 328; lawlessness 293, 320; life expectancy 14, 316, 422; per capita income 14, 315, 317, 320; poverty 314–17, 319–20, 322, 325–6, 327–8; prehistoric 52–5, 65–6, 83, 123, 350; property rights 320, 321, 323–5; trade 187–8, 320, 322–3, 325, 326, 327–8; see also individual countries African-Americans 108 agricultural employment: decline in 42–3; hardships of 13, 219–20, 285–6 agriculture: early development of 122–30, 135–9, 352, 387, 388; fertilisers, development of 135, 139–41, 142, 146, 147, 337; genetically modified (GM) crops 28, 32, 148, 151–6, 283, 358; hybrids, development of 141–2, 146, 153; and trade 123, 126, 127–33, 159, 163–4; and urbanisation 128, 158–9, 163–4, 215; see also farming; food supply Agta people 61–2 aid, international 28, 141, 154, 203, 317–19, 328 AIDS 8, 14, 307–8, 310, 316, 319, 320, 322, 331, 353 AIG (insurance corporation) 115 air conditioning 17 air pollution 304–5 air travel: costs of 24, 37, 252, 253; speed of 253 aircraft 257, 261, 264, 266 Akkadian empire 161, 164–5 Al-Ghazali 357 Al-Khwarizmi, Muhammad ibn Musa 115 Al-Qaeda 296 Albania 187 Alcoa (corporation) 24 Alexander the Great 169, 171 Alexander, Gary 295 Alexandria 171, 175, 270 Algeria 53, 246, 345 alphabet, invention of 166, 396 Alps 122, 178 altruism 93–4, 97 aluminium 24, 213, 237, 303 Alyawarre aborigines 63 Amalfi 178 Amazon (corporation) 21, 259, 261 Amazonia 76, 138, 145, 250–51 amber 71, 92 ambition 45–6, 351 Ames, Bruce 298–9 Amish people 211 ammonia 140, 146 Amsterdam 115–16, 169, 259, 368 Amsterdam Exchange Bank 251 Anabaptists 211 Anatolia 127, 128, 164, 165, 166, 167 Ancoats, Manchester 214 Andaman islands 66–7, 78 Andes 123, 140, 163 Andrew, Deroi Kwesi 189 Angkor Wat 330 Angola 316 animal welfare 104, 145–6 animals: conservation 324, 339; extinctions 17, 43, 64, 68, 69–70, 243, 293, 302, 338–9; humans’ differences from other 1, 2–4, 6, 56, 58, 64 Annan, Kofi 337 Antarctica 334 anti-corporatism 110–111, 114 anti-slavery 104, 105–6, 214 antibiotics 6, 258, 271, 307 antimony 213 ants 75–6, 87–8, 192 apartheid 108 apes 56–7, 59–60, 62, 65, 88; see also chimpanzees; orang-utans ‘apocaholics’ 295, 301 Appalachia 239 Apple (corporation) 260, 261, 268 Aquinas, St Thomas 102 Arabia 66, 159, 176, 179 Arabian Sea 174 Arabs 89, 175, 176–7, 180, 209, 357 Aral Sea 240 Arcadia Biosciences (company) 31–2 Archimedes 256 Arctic Ocean 125, 130, 185, 334, 338–9 Argentina 15, 186, 187 Arikamedu 174 Aristotle 115, 250 Arizona 152, 246, 345 Arkwright, Sir Richard 227 Armenians 89 Arnolfini, Giovanni 179 art: cave paintings 2, 68, 73, 76–7; and commerce 115–16; symbolism in 136; as unique human trait 4 Ashur, Assyria 165 Asimov, Isaac 354 Asoka the Great 172–3 aspirin 258 asset price inflation 24, 30 Assyrian empire 161, 165–6, 167 asteroid impacts, risk of 280, 333 astronomy 221, 270, 357 Athabasca tar sands, Canada 238 Athens 115, 170, 171 Atlantic Monthly 293 Atlantic Ocean 125, 170 Attica 171 Augustus, Roman emperor 174 Augustus the Strong, Elector of Saxony 184–5 Australia: climate 127, 241, 300, 334; prehistoric 66, 67, 69–70, 127; trade 187; see also aborigines (Australian); Tasmania Austria 132 Ausubel, Jesse 239, 346, 409 automobiles see cars axes: copper 123, 131, 132, 136, 271; stone 2, 5, 48–9, 50, 51, 71, 81, 90–91, 92, 118–19, 271 Babylon 21, 161, 166, 240, 254, 289 Bacon, Francis 255 bacteria: cross fertilisation 271; and pest control 151; resistance to antibiotics 6, 258, 271, 307; symbiosis 75 Baghdad 115, 177, 178, 357 Baines, Edward 227 Baird, John Logie 38 baking 124, 130 ‘balance of nature’, belief in 250–51 Balazs, Etienne 183 bald eagles 17, 299 Bali 66 Baltic Sea 71, 128–9, 180, 185 Bamako 326 bananas 92, 126, 149, 154, 392 Bangladesh 204, 210, 426 Banks, Sir Joseph 221 Barigaza (Bharuch) 174 barley 32, 124, 151 barrels 176 bartering vii, 56–60, 65, 84, 91–2, 163, 356 Basalla, George 272 Basra 177 battery farming 104, 145–6 BBC 295 beads 53, 70, 71, 73, 81, 93, 162 beef 186, 224, 308; see also cattle bees, killer 280 Beijing 17 Beinhocker, Eric 112 Bell, Alexander Graham 38 Bengal famine (1943) 141 benzene 257 Berlin 299 Berlin, Sir Isaiah 288 Bernard of Clairvaux, St 358 Berners-Lee, Sir Tim 38, 273 Berra, Yogi 354 Besant, Annie 208 Bhutan 25–6 Bible 138, 168, 396 bicycles 248–9, 263, 269–70 bin Laden, Osama 110 biofuels 149, 236, 238, 239, 240–43, 246, 300, 339, 343, 344, 346, 393 Bird, Isabella 197–8 birds: effects of pollution on 17, 299; killed by wind turbines 239, 409; nests 51; sexual differences 64; songbirds 55; see also individual species bireme galleys 167 Birmingham 223 birth control see contraception birth rates: declining 204–212; and food supply 192, 208–9; and industrialisation 202; measurement of 205, 403; population control policies 202–4, 208; pre-industrial societies 135, 137; and television 234; and wealth 200–201, 204, 205–6, 209, 211, 212; see also population growth Black Death 181, 195–6, 197, 380 Black Sea 71, 128, 129, 170, 176, 180 blogging 257 Blombos Cave, South Africa 53, 83 blood circulation, discovery of 258 Blunt, John 29 boat-building 167, 168, 177; see also canoes; ship-building Boers 321, 322 Bohemia 222 Bolivia 315, 324 Bolsheviks 324 Borlaug, Norman 142–3, 146 Borneo 339 Bosch, Carl 140, 412 Botswana 15, 316, 320–22, 326 Bottger, Johann Friedrich 184–5 Boudreaux, Don 21, 214 Boulton, Matthew 221, 256, 413–14 bows and arrows 43, 62, 70, 82, 137, 251, 274 Boxgrove hominids 48, 50 Boyer, Stanley 222, 405 Boyle, Robert 256 Bradlaugh, Charles 208 brain size 3–4, 48–9, 51, 55 Bramah, Joseph 221 Branc, Slovakia 136 Brand, Stewart 154, 189, 205 Brando, Marlon 110 brass 223 Brazil 38, 87, 123, 190, 240, 242, 315, 358 bread 38, 124, 140, 158, 224, 286, 392 bridges, suspension 283 Brin, Sergey 221, 405 Britain: affluence 12, 16, 224–5, 236, 296–7; birth rates 195, 200–201, 206, 208, 227; British exceptionalism 200–202, 221–2; climate change policy 330–31; consumer prices 24, 224–5, 227, 228; copyright system 267; enclosure acts 226, 323, 406; energy use 22, 231–2, 232–3, 342–3, 368, 430; ‘glorious revolution’ (1688) 223; income equality 18–19, 218; industrial revolution 201–2, 216–17, 220–32, 255–6, 258–9; life expectancy 15, 17–18; National Food Service 268; National Health Service 111, 261; parliamentary reform 107; per capita income 16, 218, 227, 285, 404–5; productivity 112; property rights 223, 226, 323–4; state benefits 16; tariffs 185–6, 186–7, 223; see also England; Scotland; Wales British Empire 161, 322 bronze 164, 168, 177 Brosnan, Sarah 59 Brown, Lester 147–8, 281–2, 300–301 Brown, Louise 306 Bruges 179 Brunel, Sir Marc 221 Buddhism 2, 172, 357 Buddle, John 412 Buffett, Warren 106, 268 Bulgaria 320 Burkina Faso 154 Burma 66, 67, 209, 335 Bush, George W. 161 Butler, Eamonn 105, 249 Byblos 167 Byzantium 176, 177, 179 cabbages 298 ‘Caesarism’ 289 Cairo 323 Calcutta 190, 315 Calico Act (1722) 226 Califano, Joseph 202–3 California: agriculture 150; Chumash people 62, 92–3; development of credit card 251, 254; Mojave Desert 69; Silicon Valley 221–2, 224, 257, 258, 259, 268 Cambodia 14, 315 camels 135, 176–7 camera pills 270–71 Cameroon 57 Campania 174, 175 Canaanites 166, 396 Canada 141, 169, 202, 238, 304, 305 Canal du Midi 251 cancer 14, 18, 293, 297–9, 302, 308, 329 Cannae, battle of 170 canning 186, 258 canoes 66, 67, 79, 82 capitalism 23–4, 101–4, 110, 115, 133, 214, 258–62, 291–2, 311; see also corporations; markets ‘Captain Swing’ 283 capuchin monkeys 96–7, 375 Caral, Peru 162–3 carbon dioxide emissions 340–47; absorption of 217; and agriculture 130, 337–8; and biofuels 242; costs of 331; and economic growth 315, 332; and fossil fuels 237, 315; and local sourcing of goods 41–2; taxes 346, 356 Cardwell’s Law 411 Caribbean see West Indies Carnegie, Andrew 23 Carney, Thomas 173 carnivorism 51, 60, 62, 68–9, 147, 156, 241, 376 carrots 153, 156 cars: biofuel for 240, 241; costs of 24, 252; efficiency of 252; future production 282, 355; hybrid 245; invention of 189, 270, 271; pollution from 17, 242; sport-utility vehicles 45 The Rational Optimist 424 Carson, Rachel 152, 297–8 Carter, Jimmy 238 Carthage 169, 170, 173 Cartwright, Edmund 221, 263 Castro, Fidel 187 Catalhoyuk 127 catallaxy 56, 355–9 Catholicism 105, 208, 306 cattle 122, 132, 145, 147, 148, 150, 197, 321, 336; see also beef Caucasus 237 cave paintings 2, 68, 73, 76–7 Cavendish, Henry 221 cement 283 central heating 16, 37 cereals 124–5, 125–6, 130–31, 143–4, 146–7, 158, 163; global harvests 121 Champlain, Samuel 138–9 charcoal 131, 216, 229, 230, 346 charitable giving 92, 105, 106, 295, 318–19, 356 Charles V: king of Spain 30–31; Holy Roman Emperor 184 Charles, Prince of Wales 291, 332 Chauvet Cave, France 2, 68, 73, 76–7 Chernobyl 283, 308, 345, 421 Chicago World Fair (1893) 346 chickens 122–3, 145–6, 147, 148, 408 chickpeas 125 Childe, Gordon 162 children: child labour 104, 188, 218, 220, 292; child molestation 104; childcare 2, 62–3; childhood diseases 310; mortality rates 14, 15, 16, 208–9, 284 Chile 187 chimpanzees 2, 3, 4, 6, 29, 59–60, 87, 88, 97 China: agriculture 123, 126, 148, 152, 220; birth rate 15, 200–201; coal supplies 229–30; Cultural Revolution 14, 201; diet 241; economic growth and industrialisation 17, 109, 180–81, 187, 201, 219, 220, 281–2, 300, 322, 324–5, 328, 358; economic and technological regression 180, 181–2, 193, 229–30, 255, 321, 357–8; energy use 245; income equality 19; innovations 181, 251; life expectancy 15; Longshan culture 397; Maoism 16, 187, 296, 311; Ming empire 117, 181–4, 260, 311; per capita income 15, 180; prehistoric 68, 123, 126; serfdom 181–2; Shang dynasty 166; Song dynasty 180–81; trade 172, 174–5, 177, 179, 183–4, 187, 225, 228 chlorine 296 cholera 40, 310 Chomsky, Noam 291 Christianity 172, 357, 358, 396; see also Catholicism; Church of England; monasteries Christmas 134 Chumash people 62, 92–3 Church of England 194 Churchill, Sir Winston 288 Cicero 173 Cilicia 173 Cisco Systems (corporation) 268 Cistercians 215 civil rights movement 108, 109 Clairvaux Abbey 215 Clark, Colin 146, 227 Clark, Gregory 193, 201, 401, 404 Clarke, Arthur C. 354 climate change 328–47, 426–30; costs of mitigation measures 330–32, 333, 338, 342–4; death rates associated with 335–7; and ecological dynamism 250, 329–30, 335, 339; and economic growth 315, 331–3, 341–3, 347; effects on ecosystems 338–41; and food supply 337–8; and fossil fuels 243, 314, 342, 346, 426; historic 194, 195, 329, 334, 426–7; pessimism about 280, 281, 314–15, 328–9; prehistoric 54, 65, 125, 127, 130, 160, 329, 334, 339, 340, 352; scepticism about 111, 329–30, 426; solutions to 8, 315, 345–7 Clinton, Bill 341 Clippinger, John 99 cloth trade 75, 159, 160, 165, 172, 177, 180, 194, 196, 225, 225–9, 232 clothes: Britain 224, 225, 227; early homo sapiens 71, 73; Inuits 64; metal age 122; Tasmanian natives 78 clothing prices 20, 34, 37, 40, 227, 228 ‘Club of Rome’ 302–3 coal: and economic take-off 201, 202, 213, 214, 216–17; and generation of electricity 233, 237, 239, 240, 304, 344; and industrialisation 229–33, 236, 407; prices 230, 232, 237; supplies 302–3 coal mining 132, 230–31, 237, 239, 257, 343 Coalbrookdale 407 Cobb, Kelly 35 Coca-Cola (corporation) 111, 263 coffee 298–9, 392 Cohen, Mark 135 Cold War 299 collective intelligence 5, 38–9, 46, 56, 83, 350–52, 355–6 Collier, Paul 315, 316–17 colonialism 160, 161, 187, 321–2; see also imperialism Colorado 324 Columbus, Christopher 91, 184 combine harvesters 158, 392 combined-cycle turbines 244, 410 commerce see trade Commoner, Barry 402 communism 106, 336 Compaq (corporation) 259 computer games 273, 292 computers 2, 3, 5, 211, 252, 260, 261, 263–4, 268, 282; computing power costs 24; information storage capacities 276; silicon chips 245, 263, 267–8; software 99, 257, 272–3, 304, 356; Y2K bug 280, 290, 341; see also internet Confucius 2, 181 Congo 14–15, 28, 307, 316 Congreve, Sir William 221 Connelly, Matthew 204 conservation, nature 324, 339; see also wilderness land, expansion of conservatism 109 Constantinople 175, 177 consumer spending, average 39–40 containerisation 113, 253, 386 continental drift 274 contraception 208, 210; coerced 203–4 Cook, Captain James 91 cooking 4, 29, 38, 50, 51, 52, 55, 60–61, 64, 163, 337 copper 122, 123, 131–2, 160, 162, 164, 165, 168, 213, 223, 302, 303 copyright 264, 266–7, 326 coral reefs 250, 339–40, 429–30 Cordoba 177 corn laws 185–6 Cornwall 132 corporations 110–116, 355; research and development budgets 260, 262, 269 Cosmides, Leda 57 Costa Rica 338 cotton 37, 108, 149, 151–2, 162, 163, 171, 172, 202, 225–9, 230, 407; calico 225–6, 232; spinning and weaving 184, 214, 217, 219–20, 227–8, 232, 256, 258, 263, 283 Coughlin, Father Charles 109 Craigslist (website) 273, 356 Crapper, Thomas 38 Crathis river 171 creationists 358 creative destruction 114, 356 credit cards 251, 254 credit crunch (2008) 8–10, 28–9, 31, 100, 102, 316, 355, 399, 411 Cree Indians 62 Crete 167, 169 Crichton, Michael 254 Crick, Francis 412 crime: cyber-crime 99–100, 357; falling rates 106, 201; false convictions 19–20; homicide 14, 20, 85, 88, 106, 118, 201; illegal drugs 106, 186; pessimism about 288, 293 Crimea 171 crocodiles, deaths by 40 Crompton, Samuel 227 Crookes, Sir William 140, 141 cruelty 104, 106, 138–9, 146 crusades 358 Cuba 187, 299 ‘curse of resources’ 31, 320 cyber-crime 99–100, 357 Cyprus 132, 148, 167, 168 Cyrus the Great 169 Dalkon Shield (contraceptive device) 203 Dalton, John 221 Damascus 127 Damerham, Wiltshire 194 Danube, River 128, 132 Darby, Abraham 407 Darfur 302, 353 Dark Ages 164, 175–6, 215 Darwin, Charles 77, 81, 91–2, 105, 116, 350, 415 Darwin, Erasmus 256 Darwinism 5 Davy, Sir Humphry 221, 412 Dawkins, Richard 5, 51 DDT (pesticide) 297–8, 299 de Geer, Louis 184 de Soto, Hernando 323, 324, 325 de Waal, Frans 88 Dean, James 110 decimal system 173, 178 deer 32–3, 122 deflation 24 Defoe, Daniel 224 deforestation, predictions of 304–5, 339 Delhi 189 Dell (corporation) 268 Dell, Michael 264 demographic transition 206–212, 316, 328, 402 Denmark 200, 344, 366; National Academy of Sciences 280 Dennett, Dan 350 dentistry 45 depression (psychological) 8, 156 depressions (economic) 3, 31, 32, 186–7, 192, 289; see also economic crashes deserts, expanding 28, 280 Detroit 315, 355 Dhaka 189 diabetes 156, 274, 306 Diamond, Jared 293–4, 380 diamonds 320, 322 Dickens, Charles 220 Diesel, Rudolf 146 Digital Equipment Corporation 260, 282 digital photography 114, 386 Dimawe, battle of (1852) 321 Diocletian, Roman emperor 175, 184 Diodorus 169 diprotodons 69 discount merchandising 112–14 division of labour: Adam Smith on vii, 80; and catallaxy 56; and fragmented government 172; in insects 75–6, 87–8; and population growth 211; by sex 61–5, 136, 376; and specialisation 7, 33, 38, 46, 61, 76–7, 175; among strangers and enemies 87–9; and trust 100; and urbanisation 164 DNA: forensic use 20; gene transfer 153 dogs 43, 56, 61, 84, 125 Doll, Richard 298 Dolphin, HMS 169 dolphins 3, 87 Domesday Book 215 Doriot, Georges 261 ‘dot-communism’ 356 Dover Castle 197 droughts: modern 241, 300, 334; prehistoric 54, 65, 334 drug crime 106, 186 DuPont (corporation) 31 dyes 167, 225, 257, 263 dynamos 217, 233–4, 271–2, 289 dysentery 157, 353 eagles 17, 239, 299, 409 East India Company 225, 226 Easter Island 380 Easterbrook, Greg 294, 300, 370 Easterlin, Richard 26 Easterly, William 318, 411 eBay (corporation) 21, 99, 100, 114, 115 Ebla, Syria 164 Ebola virus 307 economic booms 9, 29, 216 economic crashes 7–8, 9, 193; credit crunch (2008) 8–10, 28–9, 31, 100, 102, 316, 355, 399, 411; see also depressions (economic) ecosystems, dynamism of 250–51, 303, 410 Ecuador 87 Edinburgh Review 285 Edison, Thomas 234, 246, 272, 412 education: Africa 320; Japan 16; measuring value of 117; and population control 209, 210; universal access 106, 235; women and 209, 210 Edwards, Robert 306 Eemian interglacial period 52–3 Egypt: ancient 161, 166, 167, 170, 171, 192, 193, 197, 270, 334; Mamluk 182; modern 142, 154, 192, 301, 323; prehistoric 44, 45, 125, 126; Roman 174, 175, 178 Ehrenreich, Barbara 291 Ehrlich, Anne 203, 301–2 Ehrlich, Paul 143, 190, 203, 207, 301–2, 303 electric motors 271–2, 283 electricity 233–5, 236, 237, 245–6, 337, 343–4; costs 23; dynamos 217, 233–4, 271–2, 289 elephants 51, 54, 69, 303, 321 Eliot, T.S. 289 email 292 emigration 199–200, 202; see also migrations empathy 94–8 empires, trading 160–61; see also imperialism enclosure acts 226, 323, 406 endocrine disruptors 293 Engels, Friedrich 107–8, 136 England: agriculture 194–6, 215; infant mortality 284; law 118; life expectancy 13, 284; medieval population 194–7; per capita income 196; scientific revolution 255–7; trade 75, 89, 104, 106, 118, 169, 194; see also Britain Enron (corporation) 29, 111, 385 Erie, Lake 17 Erie Canal 139, 283 ethanol 240–42, 300 Ethiopia 14, 316, 319; prehistoric 52, 53, 129 eugenics 288, 329 Euphrates river 127, 158, 161, 167, 177 evolution, biological 5, 6, 7, 49–50, 55–6, 75, 271, 350 Ewald, Paul 309 exchange: etiquette and ritual of 133–4; and innovation 71–2, 76, 119, 167–8, 251, 269–74; and pre-industrial economies 133–4; and property rights 324–5; and rule of law 116, 117–18; and sexual division of labour 65; and specialisation 7, 10, 33, 35, 37–8, 46, 56, 58, 75, 90, 132–3, 350–52, 355, 358–9; and trust 98–100, 103, 104; as unique human trait 56–60; and virtue 100–104; see also bartering; markets; trade executions 104 extinctions 17, 43, 64, 68, 69–70, 243, 293, 302, 338–9 Exxon (corporation) 111, 115 eye colour 129 Ezekiel 167, 168 Facebook (website) 262, 268, 356 factories 160, 214, 218, 219–20, 221, 223, 256, 258–9, 284–5 falcons 299 family formation 195, 209–210, 211, 227 famines: modern 141, 143, 154, 199, 203, 302; pessimism about 280, 281, 284, 290, 300–302, 314; pre-industrial 45, 139, 195, 197 Faraday, Michael 271–2 Fargione, Joseph 242 farming: battery 104, 145–6; free-range 146, 308; intensive 143–9; organic 147, 149–52, 393; slash-and-burn 87, 129, 130; subsidies 188, 328; subsistence 87, 138, 175–6, 189, 192, 199–200; see also agriculture; food supply fascism 289 Fauchart, Emmanuelle 264 fax machines 252 Feering, Essex 195 Fehr, Ernst 94–6 female emancipation 107, 108–9, 209 feminism 109 Ferguson, Adam 1 Ferguson, Niall 85 Fermat’s Last Theorem 275 fermenting 130, 241 Ferranti, Sebastian de 234 Fertile Crescent 126, 251 fertilisation, in-vitro 306 fertilisers 32, 129, 135, 139–41, 142, 143, 145, 146, 147, 148, 149–50, 152, 155, 200, 337 Fibonacci 178 figs 125, 129 filariasis 310 Finland 15, 35, 261 fire, invention of 4, 50, 51, 52, 60, 274 First World War 289, 309 fish, sex-change 280, 293 fish farming 148, 155 fishing 62, 63–4, 71, 78–9, 81–2, 125, 127, 129, 136, 159, 162, 163, 327 Fishman, Charles 113 Flanders 179, 181, 194 flight, powered 257, 261, 264, 266 Flinders Island 81, 84 floods 128, 250, 329, 331, 334, 335, 426 Florence 89, 103, 115, 178 flowers, cut 42, 327, 328 flu, pandemic 28, 145–6, 308–310 Flynn, James 19 Fontaine, Hippolyte 233–4 food aid 28, 141, 154, 203 food miles 41–2, 353, 392; see also local sourcing food preservation 139, 145, 258 food prices 20, 22, 23, 34, 39, 40, 42, 240, 241, 300 food processing 29–30, 60–61, 145; see also baking; cooking food retailing 36, 112, 148, 268; see also supermarkets food sharing 56, 59–60, 64 food supply: and biofuels 240–41, 243, 300; and climate change 337–8; and industrialisation 139, 201–2; pessimism about 280, 281, 284, 290, 300–302; and population growth 139, 141, 143–4, 146–7, 192, 206, 208–9, 300–302 Ford, Ford Maddox 188 Ford, Henry 24, 114, 189, 271 Forester, Jay 303 forests, fears of depletion 304–5, 339 fossil fuels: and ecology 237, 240, 304, 315, 342–3, 345–6; fertilisers 143, 150, 155, 237; and industrialisation 214, 216–17, 229–33, 352; and labour saving 236–7; and productivity 244–5; supplies 216–17, 229–30, 237–8, 245, 302–3; see also charcoal; coal; gas, natural; oil; peat Fourier analysis 283 FOXP2 (gene) 55, 375 fragmentation, political 170–73, 180–81, 184, 185 France: capital markets 259; famine 197; infant mortality 16; population growth 206, 208; revolution 324; trade 184, 186, 222 Franco, Francisco 186 Frank, Robert 95–6 Franken, Al 291 Franklin, Benjamin 107, 256 Franks 176 Fray Bentos 186 free choice 27–8, 107–110, 291–2 free-range farming 146, 308 French Revolution 324 Friedel, Robert 224 Friedman, Milton 111 Friend, Sir Richard 257 Friends of the Earth 154, 155 Fry, Art 261 Fuji (corporation) 114, 386 Fujian, China 89, 183 fur trade 169, 180 futurology 354–5 Gadir (Cadiz) 168–9, 170 Gaelic language 129 Galbraith, J.K. 16 Galdikas, Birute 60 Galilee, Sea of 124 Galileo 115 Gandhi, Indira 203, 204 Gandhi, Sanjay 203–4 Ganges, River 147, 172 gas, natural 235, 236, 237, 240, 302, 303, 337 Gates, Bill 106, 264, 268 GDP per capita (world), increases in 11, 349 Genentech (corporation) 259, 405 General Electric Company 261, 264 General Motors (corporation) 115 generosity 86–7, 94–5 genetic research 54, 151, 265, 306–7, 310, 356, 358 genetically modified (GM) crops 28, 32, 148, 151–6, 283, 358 Genghis Khan 182 Genoa 89, 169, 178, 180 genome sequencing 265 geothermal power 246, 344 Germany: Great Depression (1930s) 31; industrialisation 202; infant mortality 16; Nazism 109, 289; population growth 202; predicted deforestation 304, 305; prehistoric 70, 138; trade 179–80, 187; see also West Germany Ghana 187, 189, 316, 326 Gibraltar, Strait of 180 gift giving 87, 92, 133, 134 Gilbert, Daniel 4 Gilgamesh, King 159 Ginsberg, Allen 110 Gintis, Herb 86 Gladstone, William 237 Glaeser, Edward 190 Glasgow 315 glass 166, 174–5, 177, 259 glass fibre 303 Global Humanitarian Forum 337 global warming see climate change globalisation 290, 358 ‘glorious revolution’ (1688) 223 GM (genetically modified) crops 28, 148, 151–6, 283, 358 goats 122, 126, 144, 145, 197, 320 Goethe, Johann von 104 Goklany, Indur 143–4, 341, 426 gold 165, 177, 303 golden eagles 239, 409 golden toads 338 Goldsmith, Edward 291 Google (corporation) 21, 100, 114, 259, 260, 268, 355 Gore, Al 233, 291 Goths 175 Gott, Richard 294 Gramme, Zénobe Théophile 233–4 Grantham, George 401 gravity, discovery of 258 Gray, John 285, 291 Great Barrier Reef 250 Greece: ancient 115, 128, 161, 170–71, 173–4; modern 186 greenhouse gases 152, 155, 242, 329; see also carbon dioxide emissions Greenland: ice cap 125, 130, 313, 334, 339, 426; Inuits 61; Norse 380 Greenpeace 154, 155, 281, 385 Grottes des Pigeons, Morocco 53 Groves, Leslie 412 Growth is Good for the Poor (World Bank study) 317 guano 139–40, 302 Guatemala 209 Gujarat 162, 174 Gujaratis 89 Gustavus Adolphus, King of Sweden 184 Gutenberg, Johann 184, 253 Guth, Werner 86 habeas corpus 358 Haber, Fritz 140, 412 Hadza people 61, 63, 87 Haiti 14, 301, 315 Halaf people 130 Hall, Charles Martin 24 Halley, Edmond 256 HANPP (human appropriation of net primary productivity) number 144–5 Hanseatic merchants 89, 179–80, 196 Hansen, James 426 hanta virus 307 happiness 25–8, 191 Harappa, Indus valley 161–2 Hardin, Garrett 203 harems 136 Hargreaves, James 227, 256 Harlem, Holland 215–16 Harper’s Weekly 23 Harvey, William 256 hay 214–15, 216, 239, 408–9 Hayek, Friedrich 5, 19, 38, 56, 250, 280, 355 heart disease 18, 156, 295 ‘hedonic treadmill’ 27 height, average human 16, 18 Heller, Michael 265–6 Hellespont 128, 170 Henrich, Joe 77, 377 Henry II, King of England 118 Henry, Joseph 271, 272 Henry, William 221 Heraclitus 251 herbicides 145, 152, 153–4 herding 130–31 Hero of Alexandria 270 Herschel, Sir William 221 Hesiod 292 Hippel, Eric von 273 hippies 26, 110, 175 Hiroshima 283 Hitler, Adolf 16, 184, 296 Hittites 166, 167 HIV/AIDS 8, 14, 307–8, 310, 316, 319, 320, 322, 331, 353 Hiwi people 61 Hobbes, Thomas 96 Hock, Dee 254 Hohle Fels, Germany 70 Holdren, John 203, 207, 311 Holland: agriculture 153; golden age 185, 201, 215–16, 223; horticulture 42; industrialisation 215–16, 226; innovations 264; trade 31, 89, 104, 106, 185, 223, 328 Holy Roman Empire 178, 265–6 Homer 2, 102, 168 Homestead Act (1862) 323 homicide 14, 20, 85, 88, 106, 118, 201 Homo erectus 49, 68, 71, 373 Homo heidelbergensis 49, 50–52, 373 Homo sapiens, emergence of 52–3 Hong Kong 31, 83, 158, 169, 187, 219, 328 Hongwu, Chinese emperor 183 Hood, Leroy 222, 405 Hooke, Robert 256 horses 48, 68, 69, 129, 140, 197, 215, 282, 408–9; shoes and harnesses 176, 215 housing costs 20, 25, 34, 39–40, 234, 368 Hoxha, Enver 187 Hrdy, Sarah 88 Huber, Peter 244, 344 Hueper, Wilhelm 297 Huguenots 184 Huia (birds) 64 human sacrifice 104 Hume, David 96, 103, 104, 170 humour 2 Hunan 177 Hungary 222 Huns 175 hunter-gatherers: consumption and production patterns 29–30, 123; division of labour 61–5, 76, 136; famines 45, 139; limitations of band size 77; modern societies 66–7, 76, 77–8, 80, 87, 135–6, 136–7; nomadism 130; nostalgia for life of 43–5, 135, 137; permanent settlements 128; processing of food 29, 38, 61; technological regress 78–84; trade 72, 77–8, 81, 92–3, 123, 136–7; violence and warfare 27, 44–5, 136, 137 hunting 61–4, 68–70, 125–6, 130, 339 Huron Indians 138–9 hurricanes 329, 335, 337 Hurst, Blake 152 Hutterites 211 Huxley, Aldous 289, 354 hydroelectric power 236, 239, 343, 344, 409 hyenas 43, 50, 54 IBM (corporation) 260, 261, 282 Ibn Khaldun 182 ice ages 52, 127, 329, 335, 340, 388 ice caps 125, 130, 313, 314, 334, 338–9, 426 Iceland 324 Ichaboe island 140 ‘idea-agora’ 262 imitation 4, 5, 6, 50, 77, 80 imperialism 104, 162, 164, 166, 172, 182, 319–20, 357; see also colonialism in-vitro fertilisation 306 income, per capita: and economic freedom 117; equality 18–19, 218–19; increases in 14, 15, 16–17, 218–19, 285, 331–2 India: agriculture 126, 129, 141, 142–3, 147, 151–2, 156, 301; British rule 160; caste system 173; economic growth 187, 358; energy use 245; income equality 19; infant mortality 16; innovations 172–3, 251; Mauryan empire 172–3, 201, 357; mobile phone use 327; population growth 202, 203–4; prehistoric 66, 126, 129; trade 174–5, 175, 179, 186–7, 225, 228, 232; urbanisation 189 Indian Ocean 174, 175 Indonesia 66, 87, 89, 177 Indus river 167 Indus valley civilisation 161–2, 164 industrialisation: and capital investment 258–9; and end of slavery 197, 214; and food production 139, 201–2; and fossil fuels 214, 216–17, 229–33, 352; and innovation 38, 220–24, 227–8; and living standards 217–20, 226–7, 258; pessimistic views of 42, 102–3, 217–18, 284–5; and productivity 227–8, 230–31, 232, 235–6, 244–5; and science 255–8; and trade 224–6; and urbanisation 188, 226–7 infant mortality 14, 15, 16, 208–9, 284 inflation 24, 30, 169, 289 influenza see flu, pandemic Ingleheart, Ronald 27 innovation: and capital investment 258–62, 269; and exchange 71–2, 76, 119, 167–8, 251, 269–74; and government spending programmes 267–9; increasing returns of 248–55, 274–7, 346, 354, 358–9; and industrialisation 38, 220–24, 227–8; and intellectual property 262–7, 269; limitlessness 374–7; and population growth 252; and productivity 227–8; and science 255–8, 412; and specialisation 56, 71–2, 73–4, 76–7, 119, 251; and trade 168, 171 insect-resistant crops 154–5 insecticides 151–2 insects 75–6, 87–8 insulin 156, 274 Intel (corporation) 263, 268 intellectual property 262–7; see also copyright; patents intensive farming 143–9 Intergovernmental Panel on Climate Change (IPCC) 330, 331, 332, 333–4, 338, 342, 347, 425, 426, 427, 428 internal combustion engine 140, 146, 244 International Planned Parenthood Foundation 203 internet: access to 253, 268; blogging 257; and charitable giving 318–19, 356; cyber-crime 99–100, 357; development of 263, 268, 270, 356; email 292; free exchange 105, 272–3, 356; packet switching 263; problem-solving applications 261–2; search engines 245, 256, 267; shopping 37, 99, 107, 261; social networking websites 262, 268, 356; speed of 252, 253; trust among users 99–100, 356; World Wide Web 273, 356 Inuits 44, 61, 64, 126 IPCC (Intergovernmental Panel on Climate Change) 330, 331, 332, 333–4, 338, 342, 347, 349, 425, 426, 427, 428 IQ levels 19 Iran 162 Iraq 31, 158, 161 Ireland 24, 129, 199, 227 iron 166, 167, 169, 181, 184, 223, 229, 230, 302, 407 irradiated food 150–51 irrigation 136, 147–8, 159, 161, 163, 198, 242, 281 Isaac, Glyn 64 Isaiah 102, 168 Islam 176, 357, 358 Israel 53, 69, 124, 148 Israelites 168 Italy: birth rate 208; city states 178–9, 181, 196; fascism 289; Greek settlements 170–71, 173–4; infant mortality 15; innovations 196, 251; mercantilism 89, 103, 178–9, 180, 196; prehistoric 69 ivory 70, 71, 73, 167 Jacob, François 7 Jacobs, Jane 128 Jamaica 149 James II, King 223 Japan: agriculture 197–8; birth rates 212; dictatorship 109; economic development 103, 322, 332; economic and technological regression 193, 197–9, 202; education 16; happiness 27; industrialisation 219; life expectancy 17, 31; trade 31, 183, 184, 187, 197 Jarawa tribe 67 Java 187 jealousy 2, 351 Jebel Sahaba cemeteries, Egypt 44, 45 Jefferson, Thomas 247, 249, 269 Jenner, Edward 221 Jensen, Robert 327 Jericho 127, 138 Jevons, Stanley 213, 237, 245 Jews 89, 108, 177–8, 184 Jigme Singye Wangchuck, King of Bhutan 25–6 Jobs, Steve 221, 264, 405 John, King of England 118 Johnson, Lyndon 202–3 Jones, Rhys 79 Jordan 148, 167 Jordan river 127 Joyce, James 289 justice 19–20, 116, 320, 358 Kalahari desert 44, 61, 76 Kalkadoon aborigines 91 Kanesh, Anatolia 165 Kangaroo Island 81 kangaroos 62, 63, 69–70, 84, 127 Kant, Immanuel 96 Kaplan, Robert 293 Kay, John 184, 227 Kazakhstan 206 Kealey, Terence 172, 255, 411 Kelly, Kevin 356 Kelvin, William Thomson, 1st Baron 412 Kenya 42, 87, 155, 209, 316, 326, 336, 353 Kerala 327 Kerouac, Jack 110 Khoisan people 54, 61, 62, 67, 116, 321 Kim Il Sung 187 King, Gregory 218 Kingdon, Jonathan 67 Kinneret, Lake 124 Klasies River 83 Klein, Naomi 291 Kleiner Perkins Caufield & Byers (venture capitalists) 259 knowledge, increasing returns of 248–50, 274–7 Kodak (corporation) 114, 386 Kohler, Hans-Peter 212 Korea 184, 197, 300; see also North Korea; South Korea Kuhn, Steven 64, 69 kula (exchange system) 134 !


pages: 547 words: 160,071

Underground by Suelette Dreyfus

airport security, invisible hand, John Markoff, Julian Assange, Loma Prieta earthquake, packet switching, pirate software, profit motive, publish or perish, RFC: Request For Comment, Ronald Reagan, Stephen Hawking, Steven Levy, Stuxnet, uranium enrichment, urban decay, WikiLeaks, zero day

What maintenance unit? Nibbler hadn’t mentioned any problems with any of the motel’s lines, but Par checked with him. No problems with the telephones. Par felt nervous. In addition to messing around with the phone company’s networks, he had been hacking into a Russian computer network from the computer chalet. The Soviet network was a shiny new toy. It had only been connected to the rest of the world’s global packet-switched network for about a month, which made it particularly attractive virgin territory. Nibbler called in a friend to check the motel’s phones. The friend, a former telephone company technician turned freelancer, came over to look at the equipment. He told Nibbler and Par that something weird was happening in the motel’s phone system. The line voltages were way off. Par realised instantly what was going on.

At least one of them – and often more – had already broken into systems belonging to the European Community in Luxembourg, The Financial Times (owners of the FTSE 100 share index), the British Ministry of Defence, the Foreign Office, NASA, the investment bank SG Warburg in London, the American computer database software manufacturer Oracle, and more machines on the JANET network than they could remember. Pad had also penetrated a classified military network containing a NATO system. They moved through British Telecom’s Packet Switched Stream Network (PSS), which was similar to the Tymnet X.25 network, with absolute ease.2 Gandalf’s motto was, ‘If it moves, hack it’. On 27 June 1991, Pad was sitting in the front room of his parents’ comfortable home in Greater Manchester watching the last remnants of daylight disappear on one of the longest days of the year. He loved summer, loved waking up to streaks of sunlight sneaking through the cracks in his bedroom curtain.


How to Make a Spaceship: A Band of Renegades, an Epic Race, and the Birth of Private Spaceflight by Julian Guthrie

Albert Einstein, Any sufficiently advanced technology is indistinguishable from magic, Ayatollah Khomeini, Berlin Wall, Charles Lindbergh, cosmic microwave background, crowdsourcing, Doomsday Book, Elon Musk, fear of failure, Frank Gehry, gravity well, high net worth, Iridium satellite, Isaac Newton, Jacquard loom, Jeff Bezos, Johannes Kepler, Leonard Kleinrock, life extension, low earth orbit, Mark Shuttleworth, Menlo Park, meta analysis, meta-analysis, Murray Gell-Mann, Oculus Rift, orbital mechanics / astrodynamics, packet switching, Peter H. Diamandis: Planetary Resources, pets.com, private space industry, Richard Feynman, Richard Feynman: Challenger O-ring, Ronald Reagan, side project, Silicon Valley, South of Market, San Francisco, stealth mode startup, stem cell, Stephen Hawking, Steve Jobs, urban planning

“In the first second, he figures out, the fuel burned is more than ten times as much as he had used flying his Spirit of St. Louis from New York to Paris.” Charles typed the Life story up for Anne, using two index fingers at a time. *O’Neill invented the storage ring technique for particle colliders, which led to the building of the Stanford Linear Accelerator Center. He also invented a mass driver to move materials mined on the Moon into Earth orbit. *The ARPANET was the first packet-switched network. Packet-switched networks were the work of many hands: Leonard Kleinrock (UCLA) and Paul Baran (RAND), as well as Bob Kahn (DARPA), who is related to futurist and nuclear strategist Herman Kahn, and Vint Cerf, who connected with Kleinrock at UCLA, worked with Kahn at DARPA, and works at Google. ARPANET was all about breaking down messages into little self-contained packets like postcards that have a “from” and “to” address and can shuttle through a heterogeneous network of cooperating computers.


pages: 467 words: 149,632

If Then: How Simulmatics Corporation Invented the Future by Jill Lepore

A Declaration of the Independence of Cyberspace, anti-communist, Buckminster Fuller, computer age, coronavirus, cuban missile crisis, desegregation, don't be evil, Donald Trump, Elon Musk, game design, George Gilder, Grace Hopper, Hacker Ethic, Howard Zinn, index card, information retrieval, Jaron Lanier, Jeff Bezos, Jeffrey Epstein, job automation, land reform, linear programming, Mahatma Gandhi, Marc Andreessen, Mark Zuckerberg, mass incarceration, Maui Hawaii, Menlo Park, New Journalism, New Urbanism, Norbert Wiener, Norman Mailer, packet switching, Peter Thiel, profit motive, RAND corporation, Robert Bork, Ronald Reagan, Rosa Parks, self-driving car, Silicon Valley, smart cities, South China Sea, Stewart Brand, technoutopianism, Telecommunications Act of 1996, urban renewal, War on Poverty, white flight, Whole Earth Catalog

Nearly all of what Licklider described in Libraries of the Future later came to pass: the digitization of printed material, the networking of library catalogs and their contents, the development of sophisticated, natural language–based information-retrieval and search mechanisms.23 Licklider described, with a contagious amazement, what would become, in the twenty-first century, the Internet at its very best. In 1962, Licklider left Bolt Beranek and Newman for ARPA, where his many duties included funding behavioral science projects, including Pool’s Project ComCom. At ARPA, Licklider also funded research that produced the building blocks of his imagined intergalactic computer network, which came to be called ARPANET. (Paul Baran, at RAND, pioneered the technology known as packet switching, in which data is broken into smaller pieces for faster transfer.) In 1964, not long before Simulmatics began its work in Vietnam, Licklider left ARPA for IBM. Four years later, with nearly all the necessary pieces of ARPANET in place, ARPA awarded a contract to Licklider’s old Cambridge research firm, BBN, to build it.24 By the time Licklider returned to MIT as a professor of electrical engineering and then director of Project MAC, he’d known Pool for years.

   — attempts to halt Pentagon Papers publication, 305    — Checkers speech, 22, 273    — efforts to punish Pentagon Papers leakers, 306–8    — hate and fear used in campaign, 39    — House Un-American Activities Committee, 39, 55    — loss in 1962 gubernatorial election, 165, 193, 272    — nominating speech for Goldwater, 193, 272    — presidential debates in 1960, 115, 121    — presidential election of 1952, 17, 22    — presidential election of 1960, 102, 106, 115, 123–24, 272    — presidential election of 1968, 272–74, 285–86, 289    — presidential election of 1972, 308, 309, 313    — punishment of political enemies, 306–8, 313    — resignation, 313    — Second Inaugural Address, 313    — silent majority and, 274    — Simultron Project and, 272–74, 287, 381n    — speaking style, 19    — Stevenson and, 22    — willingness to spy on American citizens, 313 Norman, Craig & Kummel (NCK), 44 Obama, Barack, 304 O’Brien, Lawrence, 107, 117, 191, 308, 367n O’Brien, Tim, 238 Ochs, Phil, 200, 252 Octoputer, 296 One Flew over the Cuckoo’s Nest (Kesey), 90 Operation Crossroads Africa, 14 Oswald, Lee Harvey, 179 Packard, Vance, 43–44, 82, 280 packet switching, 284 Page, Larry, 55 Palo Alto, CA, postwar boom, 30 Parks, Rosa, 73 Patil, D. J., 389n Patterson, John, 240–41, 243 Pauling, Linus, 157 Peace Corps, 14, 238 Pei, I. M., 317 Penner, Ann, 224, 255, 256, 259, 272, 303 Pentagon Papers    — attempts to halt publication, 305    — commissioned by McNamara, 243, 305    — efforts to punish leakers, 306–8    — grand jury investigations, 306–7, 309–10    — indictment of Ellsberg and Russo, 306    — leaked by Ellsberg, 245, 294, 295, 305    — Russo and, 245, 294, 306, 310 People Machine    — funding of, 132    — limitations, 321    — origins of idea, 2, 4, 14, 73, 319    — presidential election of 1960 and, 103, 121, 126–30, 132, 275    — see also Project Macroscope “People-Machine, The” (Morgan), 125–27, 128, 129, 141, 180 personal computers, 310, 313, 318 Peter, Paul and Mary, 252 Peyton Place (film), 88–89 Peyton Place (Metalious), 3, 88, 144–45 Phillips, Kevin, 274 Plath, Sylvia, 80, 83–84 Poe, Edgar Allan, 72 Politics: Who Gets What, When, How (Lasswell), 32 Pool, Adam, 77, 144, 147, 212, 217, 292 Pool, David de Sola, 50, 51 Pool, Ithiel de Sola    — accused of being Communist, 48–49, 50–51, 52, 54, 55–56, 211    — accused of being war criminal, 49, 295, 322–23    — advice to the Humphrey campaign, 273, 287–88    — antiwar movement and, 242    — arguments against regulating communi­cation technology, 315–17, 318    — attempts to sell Simulmatics, 293–94, 384n    — book about Simulmatics and 1960 election, 131, 182, 183, 186, 188–89    — at Bundy’s visit to MIT, 289    — Center for Advanced Study in the Behavioral Sciences, 76–77    — childhood, 51    — Chomsky battle with, 289–90, 291, 293, 295    — commitment to engage with critics, 292–93    — content analysis, 54–55, 262, 263, 268, 379n, 380n    — counterinsurgency program, 49, 200, 209, 290    — Cuc Thu Duong’s request for help, 303    — death, 317    — Democratic National Convention of 1960, 111–12    — distribution of notes on Vietnam visits, 253–54    — divorce and remarriage, 58, 344n    — Ellsberg friendship with, 230, 231 Pool, Ithiel de Sola (continued)    — Greenfield and, 46, 49, 61–62, 242, 345n    — at Hobart College, 54, 57, 343n    — at Hoover Institute, 54, 57, 58    — Humphrey campaign and, 273    — IBM 704 and, 70    — insufficiency of data for models, 139, 142, 357n    — Kennedy campaign in 1960, 110, 116, 117–18    — Kerner Commission research, 262–63    — Kochen and, 76–77, 347n    — Lasswell and, 52, 53–55, 343nn    — Lederer and, 182    — Lerner and, 54–55, 58, 344n    — letter of support for Popkin, 310    — letter to Kissinger, 290    — marketing for Simulmatics, 138, 147, 152–53, 359n    — Media-Mix simulation, 142, 146    — Moynihan and, 260    — New Left and, 313    — Nixon 1972 campaign, 273, 308–9    — Nixon help with security clearance, 56, 59, 273, 308    — opinion surveys in Vietnam, 217    — optimism about Vietnam War, 231, 253, 266, 266n, 277    — at the Pentagon, 46, 47–48, 290    — preparation for 1962 Times election coverage, 164–65, 166, 167    — Project Cambridge, 283, 284–85, 292–93, 295–96, 298, 299, 315    — Project Camelot, 209, 210–11    — Project ComCom, 169–72, 173, 239, 262, 284, 295    — Project Renaissance proposal and, 244    — Project TROY, 58    — RAND Corporation employment offer, 50, 54    — reaction to Harper’s story, 147    — resigns as chair of MIT’s Political Science Department, 291    — reviews of The 480 clipped by, 191    — Robert Kennedy and, 116, 117    — on role of social scientists, 212, 324    — Schlesinger as neighbor, 94, 109, 117, 134, 147, 212    — search for scientists willing to go to Vietnam, 236–37    — second-rate scientists employed in Vietnam, 221–25, 245    — sense of humor, 218    — Simulmatics’ Cambridge operation, 242, 254    — Simulmatics Corporation founded, 49, 77    — on Simulmatics people, 1962 Election Night, 165    — Simulmatics reports for Kennedy campaign, 117–18, 120, 121, 129, 131    — Simulmatics’ security lapses and, 241, 244, 373–74n    — Simulmatics stock offering and, 138–39    — Simultron Project and, 273, 287, 381n    — socialism, 52–53, 56    — social networks, 9, 62, 77, 170, 319, 345n    — at Stanford, 54–55    — Stevenson 1952 campaign, 308    — Stevenson 1956 campaign, 49, 63, 345n    — Stevenson campaign of 1960 and, 104, 107    — student attacks on Pool and Simulmatics, 276, 285, 292–93, 295–96, 298–300, 303, 315, 385n    — study of propaganda, 53    — summer of 1961 at Wading River, 146    — teaching at MIT, 46, 47, 58, 170–72, 173, 288, 344n    — on Tet Offensive, 267    — “TV Hamlets” study, 246–47, 264    — in Vietnam, 207–8, 217, 230–31, 243–45, 248    — Vietnam as social-science laboratory, 205, 216, 231    — Vietnamese language and, 220    — in Washington, 228    — writing about emerging technologies, 277–79, 299, 316–17, 318–19, 323 Pool, Jean MacKenzie, 58, 77, 144–45, 147, 217, 344nn Pool, Jeremy, 54, 58, 118, 144, 237–38 Pool, Jonathan, 54, 58, 238 Pool, Tamar de Sola, 51, 342–43n Popkin, Sam    — admiration of Pool, 218    — book written with Pool, 182, 186    — in China, 242    — Ellsberg as neighbor, 294, 306    — on failure of pacification, 371n    — in handcuffs, 301    — at Harvard, 294    — in MIT’s PhD program, 172, 294    — Penner and, 256    — Pentagon Papers investigation and, 301, 306–7, 309–10, 313, 387–88n    — Project ComCom, 170–71    — in Saigon, 205, 218–19, 222    — work for Simulmatics, 171 Porter, Sylvia, 141–42 Powell, Adam Clayton, Jr., 43 Praise and Patience (McPhee), 303 prediction and prophecy, 5, 36, 323–24 prediction as goal of social sciences, 35–37 predictive analytics, 5, 327 presidential election of 1948, 24 presidential election of 1952    — advertising, 14, 15, 20, 21–23    — CBS News, 24–25, 69, 150    — computer predictions, 24–26, 69, 122, 150    — Democratic primaries and convention, 17–19    — Edward L.


pages: 229 words: 68,426

Everyware: The Dawning Age of Ubiquitous Computing by Adam Greenfield

augmented reality, business process, defense in depth, demand response, demographic transition, facts on the ground, game design, Howard Rheingold, Internet of things, James Dyson, knowledge worker, late capitalism, Marshall McLuhan, new economy, Norbert Wiener, packet switching, pattern recognition, profit motive, QR code, recommendation engine, RFID, Steve Jobs, technoutopianism, the built environment, the scientific method

This is not to say that social, juridical, and political forces do not exert shaping influences that are at least as significant—otherwise we really would have architected our cities around the Segway, and RU-486 would be dispensed over every drugstore counter in the land. But it wouldn't have taken a surplus of imagination, even ahead of the fact, to discern the original Napster in Paul Baran's first paper on packet-switched networks, the Manhattan skyline in the Otis safety elevator patent, or the suburb and the strip mall latent in the heart of the internal combustion engine. Let's draw three emerging technologies from the alphabet soup of new standards and specifications we face at the moment and take a look at what they seem to "want." First, RFID, the tiny radio-frequency transponders that are already doing so much to revolutionize logistics.


pages: 238 words: 46

When Things Start to Think by Neil A. Gershenfeld

3D printing, Ada Lovelace, Bretton Woods, cellular automata, Claude Shannon: information theory, Dynabook, Hedy Lamarr / George Antheil, I think there is a world market for maybe five computers, invention of movable type, Iridium satellite, Isaac Newton, Jacquard loom, Johannes Kepler, John von Neumann, low earth orbit, means of production, new economy, Nick Leeson, packet switching, RFID, speech recognition, Stephen Hawking, Steve Jobs, telemarketer, the medium is the message, Turing machine, Turing test, Vannevar Bush

Disney, for example, is interested in personalization. A theme park attraction can't respond appropriately if it knows only your average height and weight, not your language and gender. How can a one-cent ticket contain that information and be read from a distance? Federal Express has something of a mainframe model, sending all the packages to a central hub for processing. Just as the Internet introduced packet switching, routing chunks of data wher- 204 + WHEN THINGS START TO THINK ever they need to go, FedEx would like a package-switched network. How can a one-cent envelope route itself? Steelcase's customers want the file cabinets to find the file folders: how can a one-cent file folder communicate its contents? Becton Dickinson made a billion medical syringes last year that are sterile, sharp, and cost a penny.


pages: 224 words: 64,156

You Are Not a Gadget by Jaron Lanier

1960s counterculture, accounting loophole / creative accounting, additive manufacturing, Albert Einstein, call centre, cloud computing, commoditize, crowdsourcing, death of newspapers, different worldview, digital Maoism, Douglas Hofstadter, Extropian, follow your passion, hive mind, Internet Archive, Jaron Lanier, jimmy wales, John Conway, John von Neumann, Kevin Kelly, Long Term Capital Management, Network effects, new economy, packet switching, PageRank, pattern recognition, Ponzi scheme, Ray Kurzweil, Richard Stallman, Silicon Valley, Silicon Valley startup, slashdot, social graph, stem cell, Steve Jobs, Stewart Brand, Ted Nelson, telemarketer, telepresence, The Wisdom of Crowds, trickle-down economics, Turing test, Vernor Vinge, Whole Earth Catalog

(BitTorrent is used for a variety of content, but a primary motivation to use it is that it is suitable for distributing large files, such as television shows and feature-length movies.) The internet was, of course, originally conceived during the Cold War to be capable of surviving a nuclear attack. Parts of it can be destroyed without destroying the whole, but that also means that parts can be known without knowing the whole. The core idea is called “packet switching.” A packet is a tiny portion of a file that is passed between nodes on the internet in the way a baton is passed between runners in a relay race. The packet has a destination address. If a particular node fails to acknowledge receipt of a packet, the node trying to pass the packet to it can try again elsewhere. The route is not specified, only the destination. This is how the internet can hypothetically survive an attack.


pages: 222 words: 70,132

Move Fast and Break Things: How Facebook, Google, and Amazon Cornered Culture and Undermined Democracy by Jonathan Taplin

1960s counterculture, affirmative action, Affordable Care Act / Obamacare, Airbnb, Amazon Mechanical Turk, American Legislative Exchange Council, Apple's 1984 Super Bowl advert, back-to-the-land, barriers to entry, basic income, battle of ideas, big data - Walmart - Pop Tarts, bitcoin, Brewster Kahle, Buckminster Fuller, Burning Man, Clayton Christensen, commoditize, creative destruction, crony capitalism, crowdsourcing, data is the new oil, David Brooks, David Graeber, don't be evil, Donald Trump, Douglas Engelbart, Douglas Engelbart, Dynabook, Edward Snowden, Elon Musk, equal pay for equal work, Erik Brynjolfsson, future of journalism, future of work, George Akerlof, George Gilder, Google bus, Hacker Ethic, Howard Rheingold, income inequality, informal economy, information asymmetry, information retrieval, Internet Archive, Internet of things, invisible hand, Jaron Lanier, Jeff Bezos, job automation, John Markoff, John Maynard Keynes: technological unemployment, John von Neumann, Joseph Schumpeter, Kevin Kelly, Kickstarter, labor-force participation, life extension, Marc Andreessen, Mark Zuckerberg, Menlo Park, Metcalfe’s law, Mother of all demos, move fast and break things, move fast and break things, natural language processing, Network effects, new economy, Norbert Wiener, offshore financial centre, packet switching, Paul Graham, paypal mafia, Peter Thiel, plutocrats, Plutocrats, pre–internet, Ray Kurzweil, recommendation engine, rent-seeking, revision control, Robert Bork, Robert Gordon, Robert Metcalfe, Ronald Reagan, Ross Ulbricht, Sam Altman, Sand Hill Road, secular stagnation, self-driving car, sharing economy, Silicon Valley, Silicon Valley ideology, smart grid, Snapchat, software is eating the world, Steve Jobs, Stewart Brand, technoutopianism, The Chicago School, The Market for Lemons, The Rise and Fall of American Growth, Tim Cook: Apple, trade route, transfer pricing, Travis Kalanick, trickle-down economics, Tyler Cowen: Great Stagnation, universal basic income, unpaid internship, We wanted flying cars, instead we got 140 characters, web application, Whole Earth Catalog, winner-take-all economy, women in the workforce, Y Combinator

DARPA was a direct response to the Soviet launch of Sputnik 1 in 1957 and was set up to fund technology research projects that would expand the frontiers of expertise beyond the immediate and specific requirements of the military and its laboratories. It was an extremely flat organization, characterized as 100 scientists and a travel agent, that set out to give major university computer science labs the economic support to conduct basic research that would lead to US technological superiority in computers and networked connectivity. One of its first successful projects was the progenitor of the Internet—ARPANET, the world’s first packet switching network developed in 1962 between four university campuses. Here is the paradox that libertarians just don’t get: the Internet was conceived and paid for by the US government. It was not a product of the free market as we think of it today—the realization of some young entrepreneur’s dreams. It was painstakingly researched and executed by a bunch of academics for whom IPO billions weren’t a reason to work.


Blindside: How to Anticipate Forcing Events and Wild Cards in Global Politics by Francis Fukuyama

Asian financial crisis, banking crisis, Berlin Wall, Bretton Woods, British Empire, business cycle, capital controls, Carmen Reinhart, cognitive bias, cuban missile crisis, energy security, flex fuel, global pandemic, income per capita, informal economy, Intergovernmental Panel on Climate Change (IPCC), invisible hand, John von Neumann, mass immigration, Menlo Park, Mikhail Gorbachev, moral hazard, Norbert Wiener, oil rush, oil shale / tar sands, oil shock, packet switching, RAND corporation, Ray Kurzweil, reserve currency, Ronald Reagan, The Wisdom of Crowds, trade route, Vannevar Bush, Vernor Vinge, Yom Kippur War

There was the notion of interactive comput- 2990-7 ch11 waldrop 7/23/07 12:13 PM innovation and adaptation Page 125 125 ing, for example, in which a computer would respond to the user’s input immediately (as opposed to generating a stack of fanfold printout hours later); this idea dated back to the Whirlwind project, an experiment in real-time computing that began at MIT in the 1940s.13 There were the twin notions of individually controlled computing (having a computer apparently under the control of a single user) and home computing (having a computer in your own house); both emerged in the 1960s from MIT’s Project MAC, an early experiment in time-sharing.14 And then there was the notion of a computer as an open system, meaning that a user could modify it, add to it, and upgrade it however he or she wanted; that practice was already standard in the minicomputer market, which was pioneered by the Digital Equipment Corporation in the 1960s.15 —The Internet as we know it today represents the convergence of (among other ideas) the notion of packet-switched networking from the 1960s;16 the notion of internetworking (as embodied in the TCP/IP protocol), which was developed in the 1970s to allow packets to pass between different networks;17 and the notion of hypertext—which, of course, goes back to Vannevar Bush’s article on the memex in 1945. 2990-7 ch11 waldrop 7/23/07 12:13 PM Page 126 2990-7 ch12 kurth 7/23/07 12:14 PM Page 127 Part IV What Could Be 2990-7 ch12 kurth 7/23/07 12:14 PM Page 128 2990-7 ch12 kurth 7/23/07 12:14 PM Page 129 12 Cassandra versus Pollyanna A Debate between James Kurth and Gregg Easterbrook James Kurth: I am an optimist about the current pessimism, but a pessimist overall.


pages: 298 words: 81,200

Where Good Ideas Come from: The Natural History of Innovation by Steven Johnson

Ada Lovelace, Albert Einstein, Alfred Russel Wallace, carbon-based life, Cass Sunstein, cleantech, complexity theory, conceptual framework, cosmic microwave background, creative destruction, crowdsourcing, data acquisition, digital Maoism, digital map, discovery of DNA, Dmitri Mendeleev, double entry bookkeeping, double helix, Douglas Engelbart, Douglas Engelbart, Drosophila, Edmond Halley, Edward Lloyd's coffeehouse, Ernest Rutherford, Geoffrey West, Santa Fe Institute, greed is good, Hans Lippershey, Henri Poincaré, hive mind, Howard Rheingold, hypertext link, invention of air conditioning, invention of movable type, invention of the printing press, invention of the telephone, Isaac Newton, Islamic Golden Age, James Hargreaves, James Watt: steam engine, Jane Jacobs, Jaron Lanier, Johannes Kepler, John Snow's cholera map, Joseph Schumpeter, Joseph-Marie Jacquard, Kevin Kelly, lone genius, Louis Daguerre, Louis Pasteur, Mason jar, mass immigration, Mercator projection, On the Revolutions of the Heavenly Spheres, online collectivism, packet switching, PageRank, patent troll, pattern recognition, price mechanism, profit motive, Ray Oldenburg, Richard Florida, Richard Thaler, Ronald Reagan, side project, Silicon Valley, silicon-based life, six sigma, Solar eclipse in 1919, spinning jenny, Steve Jobs, Steve Wozniak, Stewart Brand, The Death and Life of Great American Cities, The Great Good Place, The Wisdom of Crowds, Thomas Kuhn: the structure of scientific revolutions, transaction costs, urban planning

GRAPHICAL USER INTERFACE (1968—1974) The use of visual metaphors to represent data on a computer screen, along with the concept of a mouse as pointing device, dates back to a legendary demo by the Stanford professor Douglas Engelbart. Elements of the GUI were also evident in Ivan Sutherland’s 1963 program Sketchpad. The idea was refined and expanded by the Xerox PARC lab in the early 1970s. INTERNET (1970--1975) Assisted by many other computer scientists, the American Vinton Cerf designed and created the original model of the Internet, building on his early research and experiments with packet-switching networks, supported by the U.S. Department of Defense Advanced Research Projects Agency. CT SCAN (1971) Using a grant provided by the British Department of Health and Social Services, British electrical engineer Godfrey Hounsfield conceived and designed the first CT scan (computerized axial tomography), which sent multiple X-ray beams through the human body, providing a near three-dimensional image.


pages: 264 words: 79,589

Kingpin: How One Hacker Took Over the Billion-Dollar Cybercrime Underground by Kevin Poulsen

Apple II, Brian Krebs, Burning Man, corporate governance, dumpster diving, Exxon Valdez, Hacker Ethic, hive mind, index card, Kickstarter, McMansion, Mercator projection, offshore financial centre, packet switching, pirate software, Ponzi scheme, Robert Hanssen: Double agent, Saturday Night Live, Silicon Valley, Steve Jobs, Steve Wozniak, Steven Levy, traffic fines, web application, WikiLeaks, zero day, Zipcar

In its plain, unadorned text, Max could follow the exploits of editors Taran King and Knight Lightning, and contributors like Phone Phanatic, Crimson Death, and Sir Hackalot. The first generation to come of age in the home computing era was tasting the power at its fingertips, and Phrack was a jolt of subversive, electric information from a world far beyond Meridian’s sleepy borders. A typical issue was packed with tutorials on packet-switched networks like Telenet and Tymnet, guides to telephone-company computers like COSMOS, and inside looks at large-scale operating systems powering mainframe and mini-computers in air-conditioned equipment rooms around the globe. Phrack also diligently tracked news reports from the frontier battleground between hackers and their opponents in state and federal law enforcement, who were just beginning to meet the challenges posed by recreational hackers.


pages: 314 words: 83,631

Tubes: A Journey to the Center of the Internet by Andrew Blum

air freight, cable laying ship, call centre, Donald Davies, global village, Hibernia Atlantic: Project Express, if you build it, they will come, inflight wifi, invisible hand, John Markoff, Kevin Kelly, Leonard Kleinrock, Marc Andreessen, Mark Zuckerberg, Menlo Park, Mercator projection, Network effects, New Urbanism, packet switching, Ralph Waldo Emerson, RAND corporation, side project, Silicon Valley, Skype, South of Market, San Francisco, Steve Crocker, Steve Jobs, Steven Levy, undersea cable, urban planning, WikiLeaks, zero-sum game

Kleinrock looked over the top of his reading glasses and waved me toward a chair. Then he clicked. And clicked again. Now try, the voice said. He winced. “It says I’m not connected to the Internet. That’s what it says!” Then he laughed so hard his shoulders shook. Kleinrock is the father of the Internet—or rather, a father, as success has many. In 1961, while a graduate student at MIT, he published the first paper on “packet switching,” the idea that data could be transmitted efficiently in small chunks rather than a continuous stream—one of the key notions behind the Internet. The idea was already in the air. A professor at the British National Physical Laboratory named Donald Davies had, unbeknownst to Kleinrock, been independently refining similar concepts, as had Paul Baran, a researcher at the RAND Corporation in Los Angeles.


pages: 322 words: 84,752

Pax Technica: How the Internet of Things May Set Us Free or Lock Us Up by Philip N. Howard

Affordable Care Act / Obamacare, Berlin Wall, bitcoin, blood diamonds, Bretton Woods, Brian Krebs, British Empire, butter production in bangladesh, call centre, Chelsea Manning, citizen journalism, clean water, cloud computing, corporate social responsibility, creative destruction, crowdsourcing, digital map, Edward Snowden, en.wikipedia.org, failed state, Fall of the Berlin Wall, feminist movement, Filter Bubble, Firefox, Francis Fukuyama: the end of history, Google Earth, Howard Rheingold, income inequality, informal economy, Internet of things, Julian Assange, Kibera, Kickstarter, land reform, M-Pesa, Marshall McLuhan, megacity, Mikhail Gorbachev, mobile money, Mohammed Bouazizi, national security letter, Nelson Mandela, Network effects, obamacare, Occupy movement, packet switching, pension reform, prediction markets, sentiment analysis, Silicon Valley, Skype, spectrum auction, statistical model, Stuxnet, trade route, undersea cable, uranium enrichment, WikiLeaks, zero day

Today, democracy is a form of open society in which people in authority use the internet for public goods and human security in ways that have been widely reviewed and publicly approved. Democracy occurs when the rules and norms of mass surveillance have been developed openly, and state practices are acknowledged by the government. Information policy has not only come to define what kind of government a country has; the political decision to disconnect information infrastructure now delineates a regime on the edge of collapse. Net watchers report instantly when packet switching through a nation’s digital switches stops and the country “goes dark.” Public protests in an authoritarian regime can be a sign of political instability. A defining feature of political, military, and security crisis is the moment when a ruler orders the mobile-phone company and internet-service providers to shut down. Going dark has become the modern mark of a regime in crisis, and the indicator that a state is close to collapse.


pages: 239 words: 80,319

Lurking: How a Person Became a User by Joanne McNeil

4chan, A Declaration of the Independence of Cyberspace, Ada Lovelace, Airbnb, AltaVista, Amazon Mechanical Turk, Burning Man, Chelsea Manning, Chris Wanstrath, citation needed, cloud computing, crowdsourcing, delayed gratification, dematerialisation, don't be evil, Donald Trump, drone strike, Edward Snowden, Elon Musk, feminist movement, Firefox, Google Earth, Google Glasses, Google Hangouts, helicopter parent, Internet Archive, invention of the telephone, Jeff Bezos, jimmy wales, l'esprit de l'escalier, Marc Andreessen, Mark Zuckerberg, Marshall McLuhan, means of production, Menlo Park, moral panic, move fast and break things, move fast and break things, Network effects, packet switching, PageRank, pre–internet, profit motive, QAnon, recommendation engine, Saturday Night Live, Shoshana Zuboff, Silicon Valley, slashdot, Snapchat, social graph, Stephen Hawking, Steve Jobs, Steven Levy, Stewart Brand, technoutopianism, Ted Nelson, Tim Cook: Apple, trade route, Turing complete, We are the 99%, web application, white flight, Whole Earth Catalog

I would enter the password (not challenging—my dog’s name, probably, or the name of my elementary school typed backward) and wait patiently through the duration of my electrifying commute. The ding ding bong bong wooosh-woosh dinggg sound trembled through my skin, in discord with my heartbeat, like the rattling of rails and wind against a train cabin. Then I was ready at the landing page to dive in and hide. * * * The first internet message was sent over a packet-switching network in 1969. Two decades later, the launch of the World Wide Web added another gust of excitement. This development was more accessible and customizable than previous online functions. Tim Berners-Lee humbly announced his new “hypertext browser/editor” in several posts to Usenet newsgroups in 1991. “This project is experimental and of course comes without any warranty whatsoever. However, it could start a revolution in information access,” he offered.


pages: 315 words: 93,522

How Music Got Free: The End of an Industry, the Turn of the Century, and the Patient Zero of Piracy by Stephen Witt

4chan, barriers to entry, Berlin Wall, big-box store, cloud computing, collaborative economy, crowdsourcing, game design, Internet Archive, invention of movable type, inventory management, iterative process, Jason Scott: textfiles.com, job automation, late fees, mental accounting, moral panic, packet switching, pattern recognition, peer-to-peer, pirate software, Ronald Reagan, security theater, sharing economy, side project, Silicon Valley, software patent, Steve Jobs, zero day

His name was Bram Cohen, and he called his invention BitTorrent. Born in Manhattan, Cohen was a gifted programmer who competed in recreational mathematics tournaments in his spare time. He wore his hair long and his eyebrows thick, his voice came fast and nasal, and he had the hard-geek habit of nervously chuckling at things that weren’t really funny, like the inefficiencies of standard Internet packet switching, or the believability of reported file transfer download speeds. His laugh was startling and staccato, and always felt forced, and when he talked he bounced in his seat and didn’t meet your eyes. These were classic symptoms of Asperger’s syndrome, an autism spectrum disorder that Cohen claimed to have—although, he admitted, this wasn’t a professional diagnosis, merely one he’d assigned to himself.


pages: 327 words: 103,336

Everything Is Obvious: *Once You Know the Answer by Duncan J. Watts

active measures, affirmative action, Albert Einstein, Amazon Mechanical Turk, Black Swan, business cycle, butterfly effect, Carmen Reinhart, Cass Sunstein, clockwork universe, cognitive dissonance, coherent worldview, collapse of Lehman Brothers, complexity theory, correlation does not imply causation, crowdsourcing, death of newspapers, discovery of DNA, East Village, easy for humans, difficult for computers, edge city, en.wikipedia.org, Erik Brynjolfsson, framing effect, Geoffrey West, Santa Fe Institute, George Santayana, happiness index / gross national happiness, high batting average, hindsight bias, illegal immigration, industrial cluster, interest rate swap, invention of the printing press, invention of the telescope, invisible hand, Isaac Newton, Jane Jacobs, Jeff Bezos, Joseph Schumpeter, Kenneth Rogoff, lake wobegon effect, Laplace demon, Long Term Capital Management, loss aversion, medical malpractice, meta analysis, meta-analysis, Milgram experiment, natural language processing, Netflix Prize, Network effects, oil shock, packet switching, pattern recognition, performance metric, phenotype, Pierre-Simon Laplace, planetary scale, prediction markets, pre–internet, RAND corporation, random walk, RFID, school choice, Silicon Valley, social intelligence, statistical model, Steve Ballmer, Steve Jobs, Steve Wozniak, supply-chain management, The Death and Life of Great American Cities, the scientific method, The Wisdom of Crowds, too big to fail, Toyota Production System, ultimatum game, urban planning, Vincenzo Peruggia: Mona Lisa, Watson beat the top human players on Jeopardy!, X Prize

And that event included not only the days up until July 23, but also the subsequent repercussions, like the bizarre mass panic, often called the Great Fear, that gripped the provinces over the next week, and the famous legislative session that lasted the entire night of August 4, during which the entire social and political order of the old regime was dismantled.16 The more you want to explain about a black swan event like the storming of the Bastille, in other words, the broader you have to draw the boundaries around what you consider to be the event itself. This is true not only for political events but also for “technological black swans,” like the computer, the Internet, and the laser. For example, it might be true that the Internet was a black swan, but what does that mean? Does it mean that the invention of packet-switched networks was a black swan? Or was the black swan the growth of this original network into something much larger, eventually forming what would at first be called the ARPANET and then this thing called the Internet? Was it solely the development of the physical infrastructure on which other technological innovations, such as the Web and voice-over IP, were built? Or was it that these technologies, in turn, led to new business models and modes of social interaction?


pages: 342 words: 95,013

The Zenith Angle by Bruce Sterling

airport security, Burning Man, cuban missile crisis, digital map, glass ceiling, Grace Hopper, half of the world's population has never made a phone call, Iridium satellite, market bubble, new economy, packet switching, pirate software, profit motive, RFID, Richard Feynman, Richard Feynman: Challenger O-ring, Ronald Reagan, Silicon Valley, Steve Jobs, thinkpad, Y2K

She looks like a Muppet.” “That would be Fawn Glickleister.” “I know her name. If I need help, I know where to get it.” “Glickleister!” Jeb insisted. “She’s not twelve, she’s twenty-six. She’s Glickleister’s daughter.” Recognition dawned. “The Glickleister? Hyman Glickleister?” “Do you know any other Glickleisters?” Van took a breath. Hyman Glickleister. Legendary computer visionary. ARPANET. Packet-switching guru. A man thirty years ahead of his time. Glickleister had spent the last fifteen years of his life in a wheelchair, dying of some obscure neuromuscular disease, and that had only made him concentrate more fiercely. Van had been crushed when Glickleister had died. It was as if some vast blazing bonfire had gone out. There ought to be bronze statues to Glickleister in front of every router station in the world.


Scratch Monkey by Stross, Charles

carbon-based life, defense in depth, fault tolerance, gravity well, Kuiper Belt, packet switching, phenotype, telepresence

If Anubis had the full use of his faculties, he would be more than prepared for anything she could do. But if he was cut off, just a shadow of his full intellect, she might stand a chance. "I'm coming," she said. Her voice echoed from the walls. There was no answering volley of automatic fire; but she felt a sudden prickle throughout her climb-spider's nerves. You are being probed. Mechanism indeterminate and quantum-encrypted. EPR-privileged technology in use. Dreamtime packet-switched scan in use. There is a possibility of viral attack ... Her wisdom base screamed more warnings until she winced it off. "What's going on?" she demanded, firing off a flurry of active radar pulses to map out the dimensions of the killing jar. "I demand to know!" "She demands to know," crooned Anubis. He barked like a dog: feral laughter. Oshi took another step towards the light. "It is a long time since anyone demanded anything of Anubis!


pages: 368 words: 96,825

Bold: How to Go Big, Create Wealth and Impact the World by Peter H. Diamandis, Steven Kotler

3D printing, additive manufacturing, Airbnb, Amazon Mechanical Turk, Amazon Web Services, augmented reality, autonomous vehicles, Charles Lindbergh, cloud computing, creative destruction, crowdsourcing, Daniel Kahneman / Amos Tversky, dematerialisation, deskilling, disruptive innovation, Elon Musk, en.wikipedia.org, Exxon Valdez, fear of failure, Firefox, Galaxy Zoo, Google Glasses, Google Hangouts, gravity well, ImageNet competition, industrial robot, Internet of things, Jeff Bezos, John Harrison: Longitude, John Markoff, Jono Bacon, Just-in-time delivery, Kickstarter, Kodak vs Instagram, Law of Accelerating Returns, Lean Startup, life extension, loss aversion, Louis Pasteur, low earth orbit, Mahatma Gandhi, Marc Andreessen, Mark Zuckerberg, Mars Rover, meta analysis, meta-analysis, microbiome, minimum viable product, move fast and break things, Narrative Science, Netflix Prize, Network effects, Oculus Rift, optical character recognition, packet switching, PageRank, pattern recognition, performance metric, Peter H. Diamandis: Planetary Resources, Peter Thiel, pre–internet, Ray Kurzweil, recommendation engine, Richard Feynman, ride hailing / ride sharing, risk tolerance, rolodex, self-driving car, sentiment analysis, shareholder value, Silicon Valley, Silicon Valley startup, skunkworks, Skype, smart grid, stem cell, Stephen Hawking, Steve Jobs, Steven Levy, Stewart Brand, superconnector, technoutopianism, telepresence, telepresence robot, Turing test, urban renewal, web application, X Prize, Y Combinator, zero-sum game

Back then, there were only a few major computing centers on the planet. All those researchers who didn’t happen to work at MIT or Caltech—well, they were just out of luck. Then, in April 1963, a computer scientist named J. C. R. Licklider wrote a memo to his colleagues proposing an “Intergalactic Computer Network”—a network that replaced traditional circuit-switching technology with the then new development of packet switching, allowing any researcher with a terminal and a phone line to connect to one of the computing centers they so desperately needed.4 This was the birth of the Advanced Research Projects Agency Network (ARPANET), the foundational network that has since become today’s Internet. ARPANET became operational in 1975. It was mostly text-based, fairly complicated to navigate, and used primarily by scientists.


pages: 268 words: 109,447

The Cultural Logic of Computation by David Golumbia

Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, American ideology, Benoit Mandelbrot, borderless world, business process, cellular automata, citizen journalism, Claude Shannon: information theory, computer age, corporate governance, creative destruction, en.wikipedia.org, finite state, future of work, Google Earth, Howard Zinn, IBM and the Holocaust, iterative process, Jaron Lanier, jimmy wales, John von Neumann, Joseph Schumpeter, late capitalism, means of production, natural language processing, Norbert Wiener, packet switching, RAND corporation, Ray Kurzweil, RFID, Richard Stallman, semantic web, Shoshana Zuboff, Slavoj Žižek, social web, stem cell, Stephen Hawking, Steve Ballmer, Stewart Brand, strong AI, supply-chain management, supply-chain management software, Ted Nelson, telemarketer, The Wisdom of Crowds, theory of mind, Turing machine, Turing test, Vannevar Bush, web application

Of course this is highly odd, since nothing of the sort is true, even if computer technology does enable an acceleration of worldwide communication. Yet perhaps even more than communication itself, what computerized networking entails is the pinpoint location of each object and individual in a worldwide grid. The cellphone, a vanguard technology of computerization although not always recognized as such, relies on digital technologies such as packet-switching to transfer partial messages from point-to-point. We are told that cellphones free us to move anywhere we wish, like nomads, no longer tethered to a central location; no doubt there is some truth to this. But at the same time the cellphone itself, precisely demarcated via a numeric identity akin to the Internet’s IP number, becomes an inescapable marker of personal location, so much so that with much more frequency than land-line phones, it is routine for cellphone users to be asked why, at any hour of the day or night, they failed to answer their phone—as if the responsibility for such communications lies with the recipient and not with the maker of the call.


pages: 317 words: 101,074

The Road Ahead by Bill Gates, Nathan Myhrvold, Peter Rinearson

Albert Einstein, Apple's 1984 Super Bowl advert, Berlin Wall, Bill Gates: Altair 8800, Bob Noyce, Bonfire of the Vanities, business process, California gold rush, Claude Shannon: information theory, computer age, Donald Knuth, first square of the chessboard, first square of the chessboard / second half of the chessboard, glass ceiling, global village, informal economy, invention of movable type, invention of the printing press, invention of writing, John von Neumann, knowledge worker, medical malpractice, Mitch Kapor, new economy, packet switching, popular electronics, Richard Feynman, Ronald Reagan, speech recognition, Steve Ballmer, Steve Jobs, Steven Pinker, Ted Nelson, telemarketer, the scientific method, The Wealth of Nations by Adam Smith, transaction costs, Turing machine, Turing test, Von Neumann architecture

He may not understand that to implement full Internet access and maintain censorship, you would almost have to have someone looking over the shoulder of every user. In France, the pioneering on-line service, Minitel, has fostered a community of information publishers and stimulated broad familiarity with on-line systems generally. Even though both terminals and bandwidth are limited, Minitel's success has fostered innovations and provided lessons. France Telecom is investing in a packet switch data network. In Germany, Deutsche Telekom lowered the price of ISDN service dramatically in 1995. This has led to a significant increase in the number of users connecting personal computers. Bringing down ISDN prices was clever, because lower prices will foster the development of applications that will help hasten the arrival of a broadband system. The level of PC penetration in business is even higher in the Nordic countries than in the United States.


Cataloging the World: Paul Otlet and the Birth of the Information Age by Alex Wright

1960s counterculture, Ada Lovelace, barriers to entry, British Empire, business climate, business intelligence, Cape to Cairo, card file, centralized clearinghouse, corporate governance, crowdsourcing, Danny Hillis, Deng Xiaoping, don't be evil, Douglas Engelbart, Douglas Engelbart, Electric Kool-Aid Acid Test, European colonialism, Frederick Winslow Taylor, hive mind, Howard Rheingold, index card, information retrieval, invention of movable type, invention of the printing press, Jane Jacobs, John Markoff, Kevin Kelly, knowledge worker, Law of Accelerating Returns, linked data, Livingstone, I presume, lone genius, Menlo Park, Mother of all demos, Norman Mailer, out of africa, packet switching, profit motive, RAND corporation, Ray Kurzweil, Scramble for Africa, self-driving car, semantic web, Silicon Valley, speech recognition, Steve Jobs, Stewart Brand, Ted Nelson, The Death and Life of Great American Cities, the scientific method, Thomas L Friedman, urban planning, Vannevar Bush, Whole Earth Catalog

In an era when almost anyone with a mobile phone can press a few keys to search the contents of the world’s libraries, when millions of people negotiate their personal relationships via online social networks, and when institutions of all stripes find their operations disrupted by the sometimes wrenching effects of networks, it scarcely seems like hyperbole—and has even become cliché—to suggest that the advent of the Internet ranks as an event of epochal significance. While Otlet did not by any stretch of the imagination “invent” the Internet—working as he did in an age before digital computers, magnetic storage, or packet-switching networks—nonetheless his vision looks nothing short of prophetic. In Otlet’s day, microfilm may have qualified as the most advanced information storage technology, and the closest thing anyone had ever seen to a database was a drawer full of index cards. Yet despite these analog limitations, he 14 I ntrod u ction envisioned a global network of interconnected institutions that would alter the flow of information around the world, and in the process lead to profound social, cultural, and political transformations.


pages: 340 words: 97,723

The Big Nine: How the Tech Titans and Their Thinking Machines Could Warp Humanity by Amy Webb

Ada Lovelace, AI winter, Airbnb, airport security, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, artificial general intelligence, Asilomar, autonomous vehicles, Bayesian statistics, Bernie Sanders, bioinformatics, blockchain, Bretton Woods, business intelligence, Cass Sunstein, Claude Shannon: information theory, cloud computing, cognitive bias, complexity theory, computer vision, crowdsourcing, cryptocurrency, Daniel Kahneman / Amos Tversky, Deng Xiaoping, distributed ledger, don't be evil, Donald Trump, Elon Musk, Filter Bubble, Flynn Effect, gig economy, Google Glasses, Grace Hopper, Gödel, Escher, Bach, Inbox Zero, Internet of things, Jacques de Vaucanson, Jeff Bezos, Joan Didion, job automation, John von Neumann, knowledge worker, Lyft, Mark Zuckerberg, Menlo Park, move fast and break things, move fast and break things, natural language processing, New Urbanism, one-China policy, optical character recognition, packet switching, pattern recognition, personalized medicine, RAND corporation, Ray Kurzweil, ride hailing / ride sharing, Rodney Brooks, Rubik’s Cube, Sand Hill Road, Second Machine Age, self-driving car, SETI@home, side project, Silicon Valley, Silicon Valley startup, skunkworks, Skype, smart cities, South China Sea, sovereign wealth fund, speech recognition, Stephen Hawking, strong AI, superintelligent machines, technological singularity, The Coming Technological Singularity, theory of mind, Tim Cook: Apple, trade route, Turing machine, Turing test, uber lyft, Von Neumann architecture, Watson beat the top human players on Jeopardy!, zero day

But public goods can also be created in markets, and as time wears on, market-borne public goods can produce unintended consequences. We’re living with one great example of what happens when we generalize technology as a platform: the internet. The internet began as a concept—a way to improve communication and work that would ultimately benefit society. Our modern-day web evolved from a 20-year collaboration between many different researchers: in the earliest days as a packet-switching network developed by the Department of Defense and then as a wider academic network for researchers to share their work. Tim Berners-Lee, a software engineer based at CERN, wrote a proposal that expanded the network using a new set of technologies and protocols that would allow others to contribute: the uniform resource locator (URL), hypertext markup language (HTML), and hypertext transfer protocol (HTTP).


pages: 383 words: 105,021

Dark Territory: The Secret History of Cyber War by Fred Kaplan

Cass Sunstein, computer age, data acquisition, drone strike, dumpster diving, Edward Snowden, game design, hiring and firing, index card, Internet of things, Jacob Appelbaum, John Markoff, John von Neumann, kremlinology, Mikhail Gorbachev, millennium bug, national security letter, packet switching, pre–internet, RAND corporation, Ronald Reagan, Silicon Valley, Skype, Stuxnet, uranium enrichment, Y2K, zero day

The briefers raised these questions on their own, because, just one week earlier, President Obama had declassified a ruling, back in October 2011, by a FISA Court judge named John Bates, excoriating the NSA for the Section 702 intercepts generally. The fact that domestic communications were caught up in these “upstream collections,” as they were called, was no accident, Bates wrote in his ruling; it was an inherent part of the program, an inherent part of packet-switching technology. Unavoidably, then, the NSA was collecting “tens of thousands of wholly domestic communications” each year, and, as such, this constituted a blatant violation of the Fourth Amendment. “The government,” Bates concluded, “has failed to demonstrate that it has struck a reasonable balance between its foreign intelligence needs and the requirement that information concerning U.S. persons be protected.”


pages: 416 words: 106,532

Cryptoassets: The Innovative Investor's Guide to Bitcoin and Beyond: The Innovative Investor's Guide to Bitcoin and Beyond by Chris Burniske, Jack Tatar

Airbnb, altcoin, asset allocation, asset-backed security, autonomous vehicles, bitcoin, blockchain, Blythe Masters, business cycle, business process, buy and hold, capital controls, Carmen Reinhart, Clayton Christensen, clean water, cloud computing, collateralized debt obligation, commoditize, correlation coefficient, creative destruction, Credit Default Swap, credit default swaps / collateralized debt obligations, cryptocurrency, disintermediation, distributed ledger, diversification, diversified portfolio, Donald Trump, Elon Musk, en.wikipedia.org, Ethereum, ethereum blockchain, fiat currency, financial innovation, fixed income, George Gilder, Google Hangouts, high net worth, Jeff Bezos, Kenneth Rogoff, Kickstarter, Leonard Kleinrock, litecoin, Marc Andreessen, Mark Zuckerberg, market bubble, money market fund, money: store of value / unit of account / medium of exchange, moral hazard, Network effects, packet switching, passive investing, peer-to-peer, peer-to-peer lending, Peter Thiel, pets.com, Ponzi scheme, prediction markets, quantitative easing, RAND corporation, random walk, Renaissance Technologies, risk tolerance, risk-adjusted returns, Robert Shiller, Robert Shiller, Ross Ulbricht, Satoshi Nakamoto, Sharpe ratio, Silicon Valley, Simon Singh, Skype, smart contracts, social web, South Sea Bubble, Steve Jobs, transaction costs, tulip mania, Turing complete, Uber for X, Vanguard fund, WikiLeaks, Y2K

According to one of the Internet’s progenitors, Paul Baran, the key to accomplishing such resilience was decentralization.2 J. C. R. Licklider proselytized the concept of an “Intergalactic Computer Network,” convincing his colleagues at DARPA—which is responsible for investigating and developing new technologies for the U.S. military—of its importance.3 Leonard Kleinrock, an MIT professor, was doing work on packet switching—the technology underpinning the Internet—that would lead to the first book on the subject: Communication Nets. Ironically, though they were all working on a means to connecting the world, many of the early researchers in this period were unaware of one another. But their dream has been realized. Every day more than 3.5 billion Google search queries are made,4 18.7 billion text messages are sent (that doesn’t even include WhatsApp and Facebook Messenger, which combine for more than 60 billion messages per day),5 and 269 billion emails are sent.6 Interestingly, however, the Internet has become increasingly centralized over time, potentially endangering its original conception as a “highly survivable system.”


pages: 385 words: 111,113

Augmented: Life in the Smart Lane by Brett King

23andMe, 3D printing, additive manufacturing, Affordable Care Act / Obamacare, agricultural Revolution, Airbnb, Albert Einstein, Amazon Web Services, Any sufficiently advanced technology is indistinguishable from magic, Apple II, artificial general intelligence, asset allocation, augmented reality, autonomous vehicles, barriers to entry, bitcoin, blockchain, business intelligence, business process, call centre, chief data officer, Chris Urmson, Clayton Christensen, clean water, congestion charging, crowdsourcing, cryptocurrency, deskilling, different worldview, disruptive innovation, distributed generation, distributed ledger, double helix, drone strike, Elon Musk, Erik Brynjolfsson, Fellow of the Royal Society, fiat currency, financial exclusion, Flash crash, Flynn Effect, future of work, gig economy, Google Glasses, Google X / Alphabet X, Hans Lippershey, Hyperloop, income inequality, industrial robot, information asymmetry, Internet of things, invention of movable type, invention of the printing press, invention of the telephone, invention of the wheel, James Dyson, Jeff Bezos, job automation, job-hopping, John Markoff, John von Neumann, Kevin Kelly, Kickstarter, Kodak vs Instagram, Leonard Kleinrock, lifelogging, low earth orbit, low skilled workers, Lyft, M-Pesa, Mark Zuckerberg, Marshall McLuhan, megacity, Metcalfe’s law, Minecraft, mobile money, money market fund, more computing power than Apollo, Network effects, new economy, obamacare, Occupy movement, Oculus Rift, off grid, packet switching, pattern recognition, peer-to-peer, Ray Kurzweil, RFID, ride hailing / ride sharing, Robert Metcalfe, Satoshi Nakamoto, Second Machine Age, selective serotonin reuptake inhibitor (SSRI), self-driving car, sharing economy, Shoshana Zuboff, Silicon Valley, Silicon Valley startup, Skype, smart cities, smart grid, smart transportation, Snapchat, social graph, software as a service, speech recognition, statistical model, stem cell, Stephen Hawking, Steve Jobs, Steve Wozniak, strong AI, TaskRabbit, technological singularity, telemarketer, telepresence, telepresence robot, Tesla Model S, The Future of Employment, Tim Cook: Apple, trade route, Travis Kalanick, Turing complete, Turing test, uber lyft, undersea cable, urban sprawl, V2 rocket, Watson beat the top human players on Jeopardy!, white picket fence, WikiLeaks

The first ARPANET link was established between the University of California, Los Angeles (UCLA), and the Stanford Research Institute (SRI) at 22:30 on 29th October 1969. “We set up a telephone connection between us and the guys at SRI. We typed the L and we asked on the phone, “Do you see the L?” “Yes, we see the L,” came the response. We typed the O, and asked, “Do you see the O?” “Yes, we see the O.” Then we typed the G, and the system crashed...”5 Prof. Leonard Kleinrock, UCLA, from an interview on the first ARPANET packet-switching test in 1969 In parallel to the development of early computer networks, various computer manufacturers set about shrinking and personalising computer technology so that it could be used at home or in the office. Contrary to popular belief, IBM wasn’t the first company to create a personal computer (PC). In the early 1970s, Steve Jobs and Steve Wozniak had been busy working on their own version of the personal computer.


pages: 379 words: 108,129

An Optimist's Tour of the Future by Mark Stevenson

23andMe, Albert Einstein, Andy Kessler, augmented reality, bank run, carbon footprint, carbon-based life, clean water, computer age, decarbonisation, double helix, Douglas Hofstadter, Elon Musk, flex fuel, Gödel, Escher, Bach, Hans Rosling, Intergovernmental Panel on Climate Change (IPCC), Internet of things, invention of agriculture, Isaac Newton, Jeff Bezos, Kevin Kelly, Law of Accelerating Returns, Leonard Kleinrock, life extension, Louis Pasteur, low earth orbit, mutually assured destruction, Naomi Klein, off grid, packet switching, peak oil, pre–internet, Ray Kurzweil, Richard Feynman, Rodney Brooks, self-driving car, Silicon Valley, smart cities, social intelligence, stem cell, Stephen Hawking, Steven Pinker, Stewart Brand, strong AI, the scientific method, Wall-E, X Prize

I’m hoping Vint can give me the big picture on our increasing interconnectedness. After all, he was in at the ground floor of the Internet and now works on the top one. He’s a man with a career-length view on the technology, which for a technology as young as the Net is about the longest view you can have. As a graduate student, Cerf worked under Professor Leonard Kleinrock, who in 1969 oversaw the first computer-to-computer message to be sent using the ‘packet switching’ method that underlies the Internet. Actually, it was two-thirds of a message. Another of Kleinrock’s students, Charley Kline, hoped to send a three-letter message ‘LOG’ to a receiving machine (this being the code for logging on to that computer). The ‘L’ and the ‘O’ worked but the ‘G’ crashed the system. ‘So the first message on the Internet was LO,’ said Professor Kleinrock. ‘Or “Hello,” crash!’


pages: 331 words: 104,366

Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins by Garry Kasparov

3D printing, Ada Lovelace, AI winter, Albert Einstein, AltaVista, barriers to entry, Berlin Wall, business process, call centre, Charles Lindbergh, clean water, computer age, Daniel Kahneman / Amos Tversky, David Brooks, Donald Trump, Douglas Hofstadter, Drosophila, Elon Musk, Erik Brynjolfsson, factory automation, Freestyle chess, Gödel, Escher, Bach, job automation, Leonard Kleinrock, low earth orbit, Mikhail Gorbachev, Nate Silver, Norbert Wiener, packet switching, pattern recognition, Ray Kurzweil, Richard Feynman, rising living standards, rolodex, Second Machine Age, self-driving car, Silicon Valley, Silicon Valley startup, Skype, speech recognition, stem cell, Stephen Hawking, Steven Pinker, technological singularity, The Coming Technological Singularity, The Signal and the Noise by Nate Silver, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, zero-sum game

A few weeks after that, two more computers had been added, in Santa Barbara and Salt Lake City. I was familiar with the basic facts of the story and had used the ARPANET story to rebut audiences who wanted to claim the Internet wholly for the 1990s. Being able to meet the man himself was an unexpected honor. Kleinrock, who received the 2007 National Medal of Science in the United States, developed the mathematical background for packet switching, the most elemental network building block of the Internet. His theoretical work on routing network traffic is what today’s World Wide Web operates on. He points out that while it took considerable time to build the hardware and software required for the early networks, the ambition of the people working on the project was always global in scope despite the primitive nature of their early inventions.


pages: 302 words: 82,233

Beautiful security by Andy Oram, John Viega

Albert Einstein, Amazon Web Services, business intelligence, business process, call centre, cloud computing, corporate governance, credit crunch, crowdsourcing, defense in depth, Donald Davies, en.wikipedia.org, fault tolerance, Firefox, loose coupling, Marc Andreessen, market design, MITM: man-in-the-middle, Monroe Doctrine, new economy, Nicholas Carr, Nick Leeson, Norbert Wiener, optical character recognition, packet switching, peer-to-peer, performance metric, pirate software, Robert Bork, Search for Extraterrestrial Intelligence, security theater, SETI@home, Silicon Valley, Skype, software as a service, statistical model, Steven Levy, The Wisdom of Crowds, Upton Sinclair, web application, web of trust, zero day, Zimmermann PGP

The Egyptians carved obfuscated hieroglyphs into monuments; the Spartans used sticks and wound messages called scytales to exchange military plans; and the Romans’ Caesar ciphers are well documented in school textbooks. Many historians attribute the victory in the Second World War directly to the code breakers at Bletchley Park who deciphered the famous Enigma machine, yet even this monumental technological event, which ended the World War and changed history forever, may pale into insignificance next to changes to come. The packet switching network invented by Donald Davies in 1970 also changed the world forever when the sudden ability of computers to talk to other computers with which they previously had no relationship opened up new possibilities for previously isolated computing power. Although the early telegraph networks almost a century before may have aroused the dream of an electronically connected planet, it was only in the 1970s, 1980s, and 1990s that we started to wire the world together definitively with copper cables and later with fiber-optic technology.


pages: 549 words: 116,200

With a Little Help by Cory Efram Doctorow, Jonathan Coulton, Russell Galen

autonomous vehicles, big-box store, Burning Man, call centre, carbon footprint, death of newspapers, don't be evil, game design, Google Earth, high net worth, lifelogging, margin call, Mark Shuttleworth, offshore financial centre, packet switching, Ponzi scheme, rolodex, Sand Hill Road, sensible shoes, skunkworks, Skype, traffic fines, traveling salesman, Turing test, urban planning, Y2K

She was a collection of trademark affectations: a jacket with built-up shoulders, a monocle, a string tie, nipple tassles, and tattooed cross-hatching on her face that made her look like a woodcut of a Victorian counting-house clerk. Rainer loathed her -- she'd been on the committee to which he'd defended his Philosophy of Networks thesis, and she'd busted his balls so hard that they still ached a decade later when he saw her on the tube. 947 The pundit explained the packet-switching, using trains versus automobiles as a metaphor: "In a circuit uniwerse, every communication gets its own dedicated line, like a train on a track. Ven I vant to talk to you, ve build a circuit -- a train track -- betveen our dewices. No one else can use those tracks, even if ve're not talking. But packet-svitching is like a freevay. Ve break the information up into packets and ve give every packet its own little car, and it finds its own vay to the other end.


pages: 349 words: 114,038

Culture & Empire: Digital Revolution by Pieter Hintjens

4chan, airport security, AltaVista, anti-communist, anti-pattern, barriers to entry, Bill Duvall, bitcoin, blockchain, business climate, business intelligence, business process, Chelsea Manning, clean water, commoditize, congestion charging, Corn Laws, correlation does not imply causation, cryptocurrency, Debian, Edward Snowden, failed state, financial independence, Firefox, full text search, German hyperinflation, global village, GnuPG, Google Chrome, greed is good, Hernando de Soto, hiring and firing, informal economy, intangible asset, invisible hand, James Watt: steam engine, Jeff Rulifson, Julian Assange, Kickstarter, M-Pesa, mass immigration, mass incarceration, mega-rich, MITM: man-in-the-middle, mutually assured destruction, Naomi Klein, national security letter, Nelson Mandela, new economy, New Urbanism, Occupy movement, offshore financial centre, packet switching, patent troll, peak oil, pre–internet, private military company, race to the bottom, rent-seeking, reserve currency, RFC: Request For Comment, Richard Feynman, Richard Stallman, Ross Ulbricht, Satoshi Nakamoto, security theater, selection bias, Skype, slashdot, software patent, spectrum auction, Steve Crocker, Steve Jobs, Steven Pinker, Stuxnet, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, trade route, transaction costs, twin studies, union organizing, wealth creators, web application, WikiLeaks, Y2K, zero day, Zipf's Law

It took only four decades to go from three terminals on a local network to almost seven billion mobile phones, of which two billion are smartphones, on a global network. In the 1960's, mainframes ruled. These were huge expensive machines run like private empires. People were experimenting with simple networks. In 1962, I was born, and someone also invented network packets. These are like envelopes of information that could be sent around different routes to get to their destination. The military began developing packet-switched networks that could survive a lot of damage. Around 1965, people invented mainframe electronic mail; in 1969, the first RFC was written; and in 1971, the @ sign was born. The first Internet was actually built out of smaller networks like Arpanet, which had a whopping 213 hosts in 1981, and Usenet, which had 940 hosts by 1984. The Internet doubled in size every eighteen months. The Internet Protocol (IP) made it possible to route packets between networks (not just inside single networks) and after Big Brother failed to appear in 1984 (except in Apple adverts), the Internet grew into a worldwide research network that reached most places except Africa.


pages: 385 words: 118,314

Cities Are Good for You: The Genius of the Metropolis by Leo Hollis

Airbnb, banking crisis, Berlin Wall, Boris Johnson, Broken windows theory, Buckminster Fuller, call centre, car-free, carbon footprint, cellular automata, clean water, cloud computing, complexity theory, congestion charging, creative destruction, credit crunch, Credit Default Swap, crowdsourcing, Deng Xiaoping, digital map, East Village, Edward Glaeser, Enrique Peñalosa, Firefox, Frank Gehry, Geoffrey West, Santa Fe Institute, Gini coefficient, Google Earth, Guggenheim Bilbao, haute couture, Hernando de Soto, housing crisis, illegal immigration, income inequality, informal economy, Internet of things, invisible hand, Jane Jacobs, Kickstarter, knowledge economy, knowledge worker, Long Term Capital Management, M-Pesa, Mahatma Gandhi, Mark Zuckerberg, Masdar, mass immigration, megacity, negative equity, new economy, New Urbanism, Occupy movement, openstreetmap, packet switching, Panopticon Jeremy Bentham, place-making, Ray Oldenburg, Richard Florida, sharing economy, Silicon Valley, Skype, smart cities, smart grid, spice trade, Steve Jobs, technoutopianism, the built environment, The Chicago School, The Death and Life of Great American Cities, The Great Good Place, the High Line, The Spirit Level, The Wisdom of Crowds, Thomas Malthus, trade route, traveling salesman, urban planning, urban renewal, urban sprawl, walkable city, white flight, Y2K, Yom Kippur War

Weaver’s work set out the template of the science of self-organised systems; in time, the ideas opened new avenues of enquiry in biology, technology, physics, cybernetics and chemistry. His fascination with systems became the language expressed in E. O. Wilson’s groundbreaking study of anthills and the development of his socio-biological ideas of the super-organism. Complexity Theory became central to the development of the packet-switch method that underpins the internet. The theory has also been the driving force behind the Black-Scholes algorithm that raised Long-Term Capital Management to the peaks of financial success in the 1990s, and its eventual collapse in 2000; as well as James Lovelock’s theory of the earth as a self-organised structure, Gaia. It has even been used to study the power of social networks as well as an exciting new means to map the brain.


pages: 405 words: 117,219

In Our Own Image: Savior or Destroyer? The History and Future of Artificial Intelligence by George Zarkadakis

3D printing, Ada Lovelace, agricultural Revolution, Airbnb, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, animal electricity, anthropic principle, Asperger Syndrome, autonomous vehicles, barriers to entry, battle of ideas, Berlin Wall, bioinformatics, British Empire, business process, carbon-based life, cellular automata, Claude Shannon: information theory, combinatorial explosion, complexity theory, continuous integration, Conway's Game of Life, cosmological principle, dark matter, dematerialisation, double helix, Douglas Hofstadter, Edward Snowden, epigenetics, Flash crash, Google Glasses, Gödel, Escher, Bach, income inequality, index card, industrial robot, Internet of things, invention of agriculture, invention of the steam engine, invisible hand, Isaac Newton, Jacquard loom, Jacques de Vaucanson, James Watt: steam engine, job automation, John von Neumann, Joseph-Marie Jacquard, Kickstarter, liberal capitalism, lifelogging, millennium bug, Moravec's paradox, natural language processing, Norbert Wiener, off grid, On the Economy of Machinery and Manufactures, packet switching, pattern recognition, Paul Erdős, post-industrial society, prediction markets, Ray Kurzweil, Rodney Brooks, Second Machine Age, self-driving car, Silicon Valley, social intelligence, speech recognition, stem cell, Stephen Hawking, Steven Pinker, strong AI, technological singularity, The Coming Technological Singularity, The Future of Employment, the scientific method, theory of mind, Turing complete, Turing machine, Turing test, Tyler Cowen: Great Stagnation, Vernor Vinge, Von Neumann architecture, Watson beat the top human players on Jeopardy!, Y2K

A way therefore had to be found to make telecommunications indestructible in the instance of nuclear war. Telecommunication networks had to become decentralised and distributed, and guided by switching systems able to reroute traffic along whichever connections provided the optimal routes. In around 1965, DARPA commissioned the study of decentralised switching systems, which led to the development of the ARPANET15 packet switching research network, which later grew into the public Internet. ARPANET sent its first email in 1971. Email was thus the Internet’s first ‘killer app’. By the early 1990s, modems made email widely available. Computers began increasingly to connect to the Internet. The ocean was transforming into a new continent where information became a commodity. The invention of the World Wide Web (‘Web’ for short) by English computer scientist Sir Tim Berners-Lee provided a way for computers to share information.


pages: 501 words: 114,888

The Future Is Faster Than You Think: How Converging Technologies Are Transforming Business, Industries, and Our Lives by Peter H. Diamandis, Steven Kotler

Ada Lovelace, additive manufacturing, Airbnb, Albert Einstein, Amazon Mechanical Turk, augmented reality, autonomous vehicles, barriers to entry, bitcoin, blockchain, blood diamonds, Burning Man, call centre, cashless society, Charles Lindbergh, Clayton Christensen, clean water, cloud computing, Colonization of Mars, computer vision, creative destruction, crowdsourcing, cryptocurrency, Dean Kamen, delayed gratification, dematerialisation, digital twin, disruptive innovation, Edward Glaeser, Edward Lloyd's coffeehouse, Elon Musk, en.wikipedia.org, epigenetics, Erik Brynjolfsson, Ethereum, ethereum blockchain, experimental economics, food miles, game design, Geoffrey West, Santa Fe Institute, gig economy, Google X / Alphabet X, gravity well, hive mind, housing crisis, Hyperloop, indoor plumbing, industrial robot, informal economy, Intergovernmental Panel on Climate Change (IPCC), Internet of things, invention of the telegraph, Isaac Newton, Jaron Lanier, Jeff Bezos, job automation, Joseph Schumpeter, Kevin Kelly, Kickstarter, late fees, Law of Accelerating Returns, life extension, lifelogging, loss aversion, Lyft, M-Pesa, Mary Lou Jepsen, mass immigration, megacity, meta analysis, meta-analysis, microbiome, mobile money, multiplanetary species, Narrative Science, natural language processing, Network effects, new economy, New Urbanism, Oculus Rift, out of africa, packet switching, peer-to-peer lending, Peter H. Diamandis: Planetary Resources, Peter Thiel, QR code, RAND corporation, Ray Kurzweil, RFID, Richard Feynman, Richard Florida, ride hailing / ride sharing, risk tolerance, Satoshi Nakamoto, Second Machine Age, self-driving car, Silicon Valley, Skype, smart cities, smart contracts, smart grid, Snapchat, sovereign wealth fund, special economic zone, stealth mode startup, stem cell, Stephen Hawking, Steve Jobs, Steven Pinker, Stewart Brand, supercomputer in your pocket, supply-chain management, technoutopianism, Tesla Model S, Tim Cook: Apple, transaction costs, Uber and Lyft, uber lyft, unbanked and underbanked, underbanked, urban planning, Watson beat the top human players on Jeopardy!, We wanted flying cars, instead we got 140 characters, X Prize

The car/bed takes you to the local Hyperloop station, where your freshly rested self is transferred into a high-speed pod, then zipped downtown. From the roof of a Cleveland skyscraper, Uber Elevate flies you to one of Manhattan’s mega-skyports. You take the elevator down to the ground floor, where another Uber autonomous awaits to take you to your meeting on Wall Street. Total elapsed time, door-to-door: fifty-nine minutes. To borrow a term from computation, this is a future of “packet-switched humans,” where you choose your priority—speed, comfort, or cost—specify your start and end point, and the system does the rest. No fuss, no missed details, and backup options always available. Wait, wait, there’s one more thing. While the technologies we’ve discussed will decimate the traditional transportation industry, there’s something on the horizon that will disrupt travel itself. What if, to get from A to B, you didn’t have to move your body?


pages: 414 words: 123,666

Merchants' War by Stross, Charles

British Empire, dumpster diving, East Village, indoor plumbing, offshore financial centre, packet switching, peak oil, stem cell

The postal service ships high-value goods, whatever they are, either reliably-for destinations in your world, without fear of interception- or fast-for destinations in this world, by FedEx across a continent ruled by horseback." She pushed herself upright with her walking stick. "Put yourself in their shoes. They want nothing to change, because they feel threatened by change-their status is tenuous. A postal network is a packet-switched network, literally so. If world-walkers drift away from it, the bandwidth drops, and thus, its profitability. New ventures divert vital human capital. They're against exploration, because they're scrambling to stay on top of the dung heap." "Sounds like-" Mike could think of a number of people it sounded like, uncomfortably close to home- change the subject. "What about the progressives?" "We want change, simple as that.


pages: 482 words: 125,973

Competition Demystified by Bruce C. Greenwald

additive manufacturing, airline deregulation, AltaVista, asset allocation, barriers to entry, business cycle, creative destruction, cross-subsidies, deindustrialization, discounted cash flows, diversified portfolio, Everything should be made as simple as possible, fault tolerance, intangible asset, John Nash: game theory, Nash equilibrium, Network effects, new economy, oil shock, packet switching, pets.com, price discrimination, price stability, selective serotonin reuptake inhibitor (SSRI), shareholder value, Silicon Valley, six sigma, Steve Jobs, transaction costs, yield management, zero-sum game

It ran into rough going very quickly because the differences between the enterprise market and the service provider arena were much greater than Cisco had anticipated. First, there were entrenched and sophisticated competitors. Lucent, Northern Telecom, and others had been providing switching equipment to telephone companies for decades. They were large, experienced, and had established relationships with their customers. Although they did need to move their product offerings to data transmission and packet switching from their traditional analog switched equipment, they gave every indication that they were up to the task. In addition to the usual suspects, Cisco also had to face firms younger than itself, eager to exploit their own technological prowess, and backed by IPO money from an enthusiastic investing public. As a new entrant into this market, Cisco was without the critical competitive advantages it enjoyed in the enterprise market.


pages: 418 words: 128,965

The Master Switch: The Rise and Fall of Information Empires by Tim Wu

accounting loophole / creative accounting, Alfred Russel Wallace, Apple II, barriers to entry, British Empire, Burning Man, business cycle, Cass Sunstein, Clayton Christensen, commoditize, corporate raider, creative destruction, disruptive innovation, don't be evil, Douglas Engelbart, Douglas Engelbart, Howard Rheingold, Hush-A-Phone, informal economy, intermodal, Internet Archive, invention of movable type, invention of the telephone, invisible hand, Jane Jacobs, John Markoff, Joseph Schumpeter, Menlo Park, open economy, packet switching, PageRank, profit motive, road to serfdom, Robert Bork, Robert Metcalfe, Ronald Coase, sexual politics, shareholder value, Silicon Valley, Skype, Steve Jobs, Steve Wozniak, Telecommunications Act of 1996, The Chicago School, The Death and Life of Great American Cities, the market place, The Wisdom of Crowds, too big to fail, Upton Sinclair, urban planning, zero-sum game

He was interested in all forms of technologically augmented human life—what science fiction writers call cyborgs, and what Sigmund Freud meant when he described man as a “prosthetic god.”* The basic story of the Internet’s early development has been told many times; but our specific concern is to understand what was the same and what was different about this network as compared with radio, television, and the telephone system. Licklider and other early Internet founders believed that they were building an information network like none other. Some of its innovations, like packet switching, were obviously radical even in their day. Yet as we have seen time and time again, one generation’s radical innovation is the next generation’s unyielding dinosaur. In this chapter, we begin the pursuit of a central question: Was the Internet truly different, a real revolution? We don’t yet know the answer. But here, at its origins, we can gain the first inklings of what might account for that sense.


pages: 464 words: 127,283

Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia by Anthony M. Townsend

1960s counterculture, 4chan, A Pattern Language, Airbnb, Amazon Web Services, anti-communist, Apple II, Bay Area Rapid Transit, Burning Man, business process, call centre, carbon footprint, charter city, chief data officer, clean water, cleantech, cloud computing, computer age, congestion charging, connected car, crack epidemic, crowdsourcing, DARPA: Urban Challenge, data acquisition, Deng Xiaoping, digital map, Donald Davies, East Village, Edward Glaeser, game design, garden city movement, Geoffrey West, Santa Fe Institute, George Gilder, ghettoisation, global supply chain, Grace Hopper, Haight Ashbury, Hedy Lamarr / George Antheil, hive mind, Howard Rheingold, interchangeable parts, Internet Archive, Internet of things, Jacquard loom, Jane Jacobs, jitney, John Snow's cholera map, Joi Ito, Khan Academy, Kibera, Kickstarter, knowledge worker, load shedding, M-Pesa, Mark Zuckerberg, megacity, mobile money, mutually assured destruction, new economy, New Urbanism, Norbert Wiener, Occupy movement, off grid, openstreetmap, packet switching, Panopticon Jeremy Bentham, Parag Khanna, patent troll, Pearl River Delta, place-making, planetary scale, popular electronics, RFC: Request For Comment, RFID, ride hailing / ride sharing, Robert Gordon, self-driving car, sharing economy, Silicon Valley, Skype, smart cities, smart grid, smart meter, social graph, social software, social web, special economic zone, Steve Jobs, Steve Wozniak, Stuxnet, supply-chain management, technoutopianism, Ted Kaczynski, telepresence, The Death and Life of Great American Cities, too big to fail, trade route, Tyler Cowen: Great Stagnation, undersea cable, Upton Sinclair, uranium enrichment, urban decay, urban planning, urban renewal, Vannevar Bush, working poor, working-age population, X Prize, Y2K, zero day, Zipcar

Yet, possibly because they are mostly invisible, we can’t seem to figure out what to call them. None of the commonly used monikers quite capture their importance. One can only wonder how long the oddly durable anachronism “wireless” will stick around. “Cellular” (and the even worse “cellular telephony”) is a technician’s term, mostly confined to use in the United States, which describes the network’s underlying architecture of towers. It’s like calling the Internet “distributed packet-switched computer networking” instead of the “Web.” “Mobile” starts to get at the essence of why people find these technologies so utterly appealing but misses one big aspect of how we use them. Most of the time we aren’t moving, we’re sitting still. There is a more fitting adjective that captures both the technology and what it is doing to us. In the 1990s, as the US military contemplated battlefield communications in the future, it adopted the term “untethered.”


pages: 629 words: 142,393

The Future of the Internet: And How to Stop It by Jonathan Zittrain

A Declaration of the Independence of Cyberspace, Amazon Mechanical Turk, Andy Kessler, barriers to entry, book scanning, Brewster Kahle, Burning Man, c2.com, call centre, Cass Sunstein, citizen journalism, Clayton Christensen, clean water, commoditize, corporate governance, Daniel Kahneman / Amos Tversky, disruptive innovation, distributed generation, en.wikipedia.org, Firefox, game design, Hacker Ethic, Howard Rheingold, Hush-A-Phone, illegal immigration, index card, informal economy, Internet Archive, jimmy wales, John Markoff, license plate recognition, loose coupling, mail merge, national security letter, old-boy network, packet switching, peer-to-peer, post-materialism, pre–internet, price discrimination, profit maximization, Ralph Nader, RFC: Request For Comment, RFID, Richard Stallman, Richard Thaler, risk tolerance, Robert Bork, Robert X Cringely, SETI@home, Silicon Valley, Skype, slashdot, software patent, Steve Ballmer, Steve Jobs, Ted Nelson, Telecommunications Act of 1996, The Nature of the Firm, The Wisdom of Crowds, web application, wikimedia commons, zero-sum game

Griffiths, The History of the Internet, Chapter Two: From ARPANET to World Wide Web, http://www.let.leidenuniv.nl/history/ivh/chap2.htm (last visited June 1, 2007) (“It is worth remembering, at this stage, that we are still [in the mid-1970s] in a World where we are talking almost exclusively about large mainframe computers (owned only by large corporations, government institutions and universities).”). 32. See Leiner et al., A Brief History of the Internet, supra note 29 (“Internet was based on the idea that there would be multiple independent networks of rather arbitrary design, beginning with the ARPANET as the pioneering packet switching network…. In this approach, the choice of any individual network technology was not dictated by a particular network architecture but rather could be selected freely by a provider and made to interwork with the other networks through a meta-level ‘Internetworking Architecture.’”). 33. Seeid. (“Four ground rules were critical to [the early designs of the Internet]: [First, e]ach distinct network would have to stand on its own and no internal changes could be required to any such network to connect it to the Internet.


pages: 205 words: 18,208

The Transparent Society: Will Technology Force Us to Choose Between Privacy and Freedom? by David Brin

affirmative action, airport security, Ayatollah Khomeini, clean water, cognitive dissonance, corporate governance, data acquisition, death of newspapers, Extropian, Howard Rheingold, illegal immigration, informal economy, information asymmetry, Iridium satellite, Jaron Lanier, John Markoff, John von Neumann, Kevin Kelly, Marshall McLuhan, means of production, mutually assured destruction, offshore financial centre, open economy, packet switching, pattern recognition, pirate software, placebo effect, plutocrats, Plutocrats, prediction markets, Ralph Nader, RAND corporation, Robert Bork, Saturday Night Live, Search for Extraterrestrial Intelligence, Steve Jobs, Steven Levy, Stewart Brand, telepresence, trade route, Vannevar Bush, Vernor Vinge, Whole Earth Catalog, Whole Earth Review, Yogi Berra, zero-sum game, Zimmermann PGP

Under these conditions, a print journalist who wants a steady living might feel obliged to heed the advice of major clients and advertisers. 134 ... not the way living organisms do it... Recall how the Internet arose out of concern over how best to defend the United States against foreign foes. Overly rigid central command systems were seen as fatally flawed. New concepts of dispersed responsibility led to packet-switching technology, and eventually the Internetʼs magnificent chaos. 134 Criticism might be viewed as a civilizationʼs equivalent of an immune system.... In fact, mutual criticism in society has the potential of being far more effective in correcting errors than the immune system of a living organism. As John Gilmore points out, “The immune system canʼt improve on the bodyʼs pre-existing design. But criticism can.”


pages: 579 words: 160,351

Breaking News: The Remaking of Journalism and Why It Matters Now by Alan Rusbridger

accounting loophole / creative accounting, Airbnb, banking crisis, Bernie Sanders, Boris Johnson, centre right, Chelsea Manning, citizen journalism, cross-subsidies, crowdsourcing, David Attenborough, David Brooks, death of newspapers, Donald Trump, Doomsday Book, Double Irish / Dutch Sandwich, Downton Abbey, Edward Snowden, Etonian, Filter Bubble, forensic accounting, Frank Gehry, future of journalism, G4S, high net worth, invention of movable type, invention of the printing press, Jeff Bezos, jimmy wales, Julian Assange, Mark Zuckerberg, Menlo Park, natural language processing, New Journalism, offshore financial centre, oil shale / tar sands, open borders, packet switching, Panopticon Jeremy Bentham, pre–internet, ransomware, recommendation engine, Ruby on Rails, sexual politics, Silicon Valley, Skype, Snapchat, social web, Socratic dialogue, sovereign wealth fund, speech recognition, Steve Jobs, The Wisdom of Crowds, Tim Cook: Apple, traveling salesman, upwardly mobile, WikiLeaks

Back in the UK, I rejoined the Guardian and was diverted towards a route of editing – launching the paper’s Saturday magazine followed by a daily tabloid features section (named G2) and moving to be deputy editor in 1993. I had developed a love of gadgets. During my stint as diary writer in the mid-’80s I had bought a battery-powered Tandy 100 computer, which displayed a few lines of text. On assignment in Australia I learned how to unscrew a hotel phone and, with crocodile clips, squirt copy back to London using packet-switching technology in the middle of the night. It felt like landing a man on the moon. I had no idea what was to come. 3 The New World In 1993 some journalists began to be dimly aware of something clunkily referred to as ‘the information superhighway’ but few had ever had reason to see it in action. At the start of 1995 only 491 newspapers were online worldwide: by June 1997 that had grown to some 3,600.


pages: 606 words: 157,120

To Save Everything, Click Here: The Folly of Technological Solutionism by Evgeny Morozov

3D printing, algorithmic trading, Amazon Mechanical Turk, Andrew Keen, augmented reality, Automated Insights, Berlin Wall, big data - Walmart - Pop Tarts, Buckminster Fuller, call centre, carbon footprint, Cass Sunstein, choice architecture, citizen journalism, cloud computing, cognitive bias, creative destruction, crowdsourcing, data acquisition, Dava Sobel, disintermediation, East Village, en.wikipedia.org, Fall of the Berlin Wall, Filter Bubble, Firefox, Francis Fukuyama: the end of history, frictionless, future of journalism, game design, Gary Taubes, Google Glasses, illegal immigration, income inequality, invention of the printing press, Jane Jacobs, Jean Tirole, Jeff Bezos, jimmy wales, Julian Assange, Kevin Kelly, Kickstarter, license plate recognition, lifelogging, lone genius, Louis Pasteur, Mark Zuckerberg, market fundamentalism, Marshall McLuhan, moral panic, Narrative Science, Nelson Mandela, Nicholas Carr, packet switching, PageRank, Parag Khanna, Paul Graham, peer-to-peer, Peter Singer: altruism, Peter Thiel, pets.com, placebo effect, pre–internet, Ray Kurzweil, recommendation engine, Richard Thaler, Ronald Coase, Rosa Parks, self-driving car, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, Skype, Slavoj Žižek, smart meter, social graph, social web, stakhanovite, Steve Jobs, Steven Levy, Stuxnet, technoutopianism, the built environment, The Chicago School, The Death and Life of Great American Cities, the medium is the message, The Nature of the Firm, the scientific method, The Wisdom of Crowds, Thomas Kuhn: the structure of scientific revolutions, Thomas L Friedman, transaction costs, urban decay, urban planning, urban sprawl, Vannevar Bush, WikiLeaks

In the afterword to my first book, The Net Delusion, I made what I now believe to be one of its main, even if overlooked, points: the physical infrastructure we know as “the Internet” bears very little resemblance to the mythical “Internet”—the one that reportedly brought down the governments of Tunisia and Egypt and is supposedly destroying our brains—that lies at the center of our public debates. The infrastructure and design of this network of networks do play a certain role in sanctioning many of these myths—for example, the idea that “the Internet” is resistant to censorship comes from the unique qualities of its packet-switching communication mechanism—but “the Internet” that is the bane of public debates also contains many other stories and narratives—about innovation, surveillance, capitalism—that have little to do with the infrastructure per se. French philosopher Bruno Latour, writing of Louis Pasteur’s famed scientific accomplishments, distinguished between Pasteur, the actual historical figure, and “Pasteur,” the mythical almighty character who has come to represent the work of other scientists and entire social movements, like the hygienists, who, for their own pragmatic reasons, embraced Pasteur with open arms.


Turing's Cathedral by George Dyson

1919 Motor Transport Corps convoy, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, anti-communist, Benoit Mandelbrot, British Empire, Brownian motion, cellular automata, cloud computing, computer age, Danny Hillis, dark matter, double helix, fault tolerance, Fellow of the Royal Society, finite state, Georg Cantor, Henri Poincaré, housing crisis, IFF: identification friend or foe, indoor plumbing, Isaac Newton, Jacquard loom, John von Neumann, mandelbrot fractal, Menlo Park, Murray Gell-Mann, Norbert Wiener, Norman Macrae, packet switching, pattern recognition, Paul Erdős, Paul Samuelson, phenotype, planetary scale, RAND corporation, random walk, Richard Feynman, SETI@home, social graph, speech recognition, Thorstein Veblen, Turing complete, Turing machine, Von Neumann architecture

RAND began looking at how to design redundant digital communications networks for coordinating defenses both before and after nuclear attack, prompted by the game theorists’ conclusions that a survivable communications network, capable of launching even a handful of remaining missiles, was the best preventive to premeditated attack. Left unstated, but not unconsidered, was the possibility that the survivors of a nuclear attack, instead of making a final suicidal response, might want to coordinate not launching a retaliatory strike. “There was a clear but not formally stated understanding,” explained Paul Baran, a RAND colleague who helped develop the communication architecture now known as packet switching, “that a survivable communications network is needed to stop, as well as to help avoid, a war.”46 Baran’s study “On Distributed Communications” was released in 1964, and played the same role in the development of the Internet as Preliminary Discussion of the Logical Design of an Electronic Computing Instrument had played in the development of the individual machines out of which the Internet was composed.47 A similar decision was made not to patent or classify the work.


pages: 733 words: 184,118

Tesla: Inventor of the Electrical Age by W. Bernard Carlson

1960s counterculture, Albert Einstein, Clayton Christensen, creative destruction, disruptive innovation, en.wikipedia.org, Henri Poincaré, invention of radio, Isaac Newton, James Watt: steam engine, Joseph Schumpeter, Menlo Park, packet switching, popular electronics, Robert Gordon, Ronald Reagan, Steve Jobs, Steve Wozniak, undersea cable, yellow journalism

The whole earth is like a brain, as it were, and the capacity of this system is infinite, for the energy received on every few square feet of ground is sufficient to operate an instrument, and the number of devices which can be so actuated is, for all practical purposes infinite. You see, Mr. Morgan, the revolutionary character of this idea, its civilizing potency, its tremendous money-making power.18 Although Tesla was certainly not thinking about the computers, software, and packet-switching that were necessary to create the World Wide Web, his fundamental idea that all news should be collected and disseminated around the world is suggestive of the beliefs that came to underlie the World Wide Web in the 1990s. “The World-Wide Web (W3),” noted media scholars Noah Wardrip-Fruin and Nick Montfort, “was developed to be a pool of human knowledge, which would allow collaborators in remote sites to share their ideas and all aspects of a common project.”19 Tesla believed that he and Morgan would make money by manufacturing receivers, and he envisioned several versions.


pages: 826 words: 231,966

GCHQ by Richard Aldrich

belly landing, Berlin Wall, British Empire, colonial exploitation, cuban missile crisis, friendly fire, illegal immigration, index card, lateral thinking, Menlo Park, Mikhail Gorbachev, Neil Kinnock, New Journalism, packet switching, private military company, Robert Hanssen: Double agent, Ronald Reagan, social intelligence, South China Sea, undersea cable, University of East Anglia, Yom Kippur War, Zimmermann PGP

While they were there, a group of other German prisoners sent a message asking to speak to the ‘proper people’. This team had served in the OKW headquarters sigint units and now revealed that, terrified of the rapid Soviet advance, they had buried their equipment under the pavement in front of their headquarters. Called ‘OKW-Chi’, they had successfully broken what was referred to as ‘Russian Fish’. This was an encrypted Soviet military teleprinter that achieved an early version of packet switching, breaking each message into nine different parts and routing it along separate channels, before reassembling it. The Germans had already worked out that their code-breaking triumph would have post-war value, and hoped to sell themselves on as a complete team.4 They were not disappointed. By 23 May they had been encouraged to unearth and set up their equipment, allowing them to resume decrypting Soviet command traffic.


The Rise and Fall of the British Nation: A Twentieth-Century History by David Edgerton

active measures, Berlin Wall, Big bang: deregulation of the City of London, blue-collar work, British Empire, business cycle, call centre, centre right, collective bargaining, colonial exploitation, Corn Laws, corporate governance, deglobalization, deindustrialization, dematerialisation, deskilling, Donald Davies, double helix, endogenous growth, Etonian, European colonialism, feminist movement, first-past-the-post, full employment, imperial preference, James Dyson, knowledge economy, labour mobility, land reform, land value tax, manufacturing employment, means of production, Mikhail Gorbachev, Neil Kinnock, new economy, non-tariff barriers, North Sea oil, offshore financial centre, old-boy network, packet switching, Philip Mirowski, Piper Alpha, plutocrats, Plutocrats, post-industrial society, rising living standards, road to serfdom, Ronald Reagan, The inhabitant of London could order by telephone, sipping his morning tea in bed, the various products of the whole earth, trade liberalization, union organizing, very high income, wages for housework, wealth creators, Winter of Discontent, women in the workforce, working poor

It spent around £100,000 on supporting research in universities and £3 milion on development (building and operating pilot plants) and technical service, of which £1 million and 900 workers (300 graduates) were on technical service.57 In fact, the 1960s and 1970s were a moment of creativity in British state and industry in smaller-scale activities away from the glare of state policy and often connected internationally. For example, in the 1960s in the STC (the British subsidiary of ITT) laboratories in Essex Charles K. Kao, born in China, trained in Hong Kong and the United Kingdom, devised the fibre-optic cable, for which he much later won a Nobel Prize. Also in the 1960s Donald Davies of the National Physical Laboratory developed packet-switching, a key element of the internet. Sir Geoffrey Houndsfield of EMI developed the CT-scanner, introduced in the 1970s, and Sir Peter Mansfield of Nottingham University invented MRI imaging, also introduced in the 1970s. Neither had been to Oxford or Cambridge, very unusually for Nobel Prize winners, as they too became. The artificial hip can be seen as an NHS contribution to the world. It was developed in the 1960s by a surgeon, Sir John Charnley, working in a Manchester hospital, not a laboratory.


The Art of Scalability: Scalable Web Architecture, Processes, and Organizations for the Modern Enterprise by Martin L. Abbott, Michael T. Fisher

always be closing, anti-pattern, barriers to entry, Bernie Madoff, business climate, business continuity plan, business intelligence, business process, call centre, cloud computing, combinatorial explosion, commoditize, Computer Numeric Control, conceptual framework, database schema, discounted cash flows, en.wikipedia.org, fault tolerance, finite state, friendly fire, hiring and firing, Infrastructure as a Service, inventory management, new economy, packet switching, performance metric, platform as a service, Ponzi scheme, RFC: Request For Comment, risk tolerance, Rubik’s Cube, Search for Extraterrestrial Intelligence, SETI@home, shareholder value, Silicon Valley, six sigma, software as a service, the scientific method, transaction costs, Vilfredo Pareto, web application, Y2K

No one is exactly sure when it was first used in relation to technology, but it has been around at least as far back as when network diagrams came into vogue. A network diagram is a graphic representation of the physical or logic layout of a network, such as a telecommunications, routing, or neural. The cloud on network diagrams was used to represent unspecified networks. In the early 1990s, cloud became a term for ATM networks. Asynchronous Transfer Mode is a packet switching protocol that breaks data into cells and provides OSI layer 2, the data link. ATM was the core protocol used on the public switched phone network. As the World Wide Web began in 1991 as a CERN project built on top of the Internet, the cloud began to be used as a term and symbol for the underlying infrastructure. OSI Model The Open Systems Interconnection Reference Model, or OSI Model, is a descriptive abstraction of the layered model of network architecture.


pages: 956 words: 267,746

Command and Control: Nuclear Weapons, the Damascus Accident, and the Illusion ofSafety by Eric Schlosser

Albert Einstein, anti-communist, Berlin Wall, cuban missile crisis, Fall of the Berlin Wall, Haight Ashbury, impulse control, interchangeable parts, Isaac Newton, life extension, Mikhail Gorbachev, mutually assured destruction, nuclear winter, packet switching, RAND corporation, Ronald Reagan, Stanislav Petrov, Stewart Brand, too big to fail, uranium enrichment, William Langewiesche

But a series of small power surges could mimic those pulses and activate the motors. The motors might silently rotate, one notch at a time, over the course of days or even months, without the launch crews knowing. And then, when the final notch turned, fifty missiles would suddenly take off. Rubel interview. “I was scared shitless”: The engineer was Paul Baran, later one of the inventors of packet switching. Quoted in Stewart Brand, “Founding Father,” Wired, March 2001. the redesign cost about $840 million: Cited in Ball, Politics and Force Levels, p. 194. To err on the side of safety: See Dobbs, One Minute to Midnight, pp. 276–79; and “Strategic Air Command Operations in the Cuban Crisis,” pp. 72–73. “Mr. McNamara went on to describe the possibilities”: “State-Defense Meeting on Group I, II, and IV Papers,” p. 12.


Applied Cryptography: Protocols, Algorithms, and Source Code in C by Bruce Schneier

active measures, cellular automata, Claude Shannon: information theory, complexity theory, dark matter, Donald Davies, Donald Knuth, dumpster diving, Exxon Valdez, fault tolerance, finite state, invisible hand, John von Neumann, knapsack problem, MITM: man-in-the-middle, NP-complete, P = NP, packet switching, RAND corporation, RFC: Request For Comment, software patent, telemarketer, traveling salesman, Turing machine, web of trust, Zimmermann PGP

Author(s): Bruce Schneier ISBN: 0471128457 Publication Date: 01/01/96 Search this book: Go! Previous Table of Contents Next ----------- Blowfish is optimized for applications where the key does not change often, like a communications link or an automatic file encryptor. It is significantly faster than DES when implemented on 32-bit microprocessors with large data caches, such as the Pentium and the PowerPC. Blowfish is not suitable for applications, such as packet switching, with frequent key changes, or as a one-way hash function. Its large memory requirement makes it infeasible for smart card applications. Description of Blowfish Blowfish is a 64-bit block cipher with a variable-length key. The algorithm consists of two parts: key expansion and data encryption. Key expansion converts a key of up to 448 bits into several subkey arrays totaling 4168 bytes. Data encryption consists of a simple function iterated 16 times.