Singularitarianism

21 results back to index


pages: 282 words: 81,873

Live Work Work Work Die: A Journey Into the Savage Heart of Silicon Valley by Corey Pein

23andMe, 4chan, affirmative action, Affordable Care Act / Obamacare, Airbnb, Amazon Mechanical Turk, Anne Wojcicki, artificial general intelligence, bank run, barriers to entry, Benevolent Dictator For Life (BDFL), Bernie Sanders, bitcoin, Bitcoin Ponzi scheme, Build a better mousetrap, California gold rush, cashless society, colonial rule, computer age, cryptocurrency, data is the new oil, disruptive innovation, Donald Trump, Douglas Hofstadter, Elon Musk, Extropian, gig economy, Google bus, Google Glasses, Google X / Alphabet X, hacker house, hive mind, illegal immigration, immigration reform, independent contractor, Internet of things, invisible hand, Isaac Newton, Jeff Bezos, job automation, Kevin Kelly, Khan Academy, Law of Accelerating Returns, Lean Startup, life extension, Lyft, Mahatma Gandhi, Marc Andreessen, Mark Zuckerberg, Menlo Park, minimum viable product, move fast and break things, move fast and break things, mutually assured destruction, obamacare, passive income, patent troll, Paul Graham, peer-to-peer lending, Peter H. Diamandis: Planetary Resources, Peter Thiel, platform as a service, Plutocrats, plutocrats, Ponzi scheme, post-work, Ray Kurzweil, regulatory arbitrage, rent control, RFID, Robert Mercer, rolodex, Ronald Reagan, Ross Ulbricht, Ruby on Rails, Sam Altman, Sand Hill Road, Scientific racism, self-driving car, sharing economy, side project, Silicon Valley, Silicon Valley startup, Singularitarianism, Skype, Snapchat, social software, software as a service, source of truth, South of Market, San Francisco, Startup school, stealth mode startup, Steve Bannon, Steve Jobs, Steve Wozniak, TaskRabbit, technological singularity, technoutopianism, telepresence, too big to fail, Travis Kalanick, tulip mania, Uber for X, uber lyft, ubercab, upwardly mobile, Vernor Vinge, X Prize, Y Combinator

* * * I am by no means the first to label Singularitarianism a new religion or a cult. Kurzweil himself has said the comparison was “understandable,” given the preoccupation with mortality. However, he rejects the argument that his sect is religious in nature, because he did not come to it as a spiritual seeker. Rather, Kurzweil writes, he became a Singularitarian as a result of “practical” efforts to make “optimal tactical decisions in launching technology enterprises.” Startups showed him the way! Being a Singularitarian, Kurzweil claims, “is not a matter of faith but one of understanding.” This is a refrain Singularitarians share with Scientologists, for L.

What did it mean that the leaders of a corporation more powerful than most governments were willing to tacitly endorse what Kurzweil called Singularitarianism? Was it not extraordinary that these iconic and influential tech magnates would lend credence to the Kurzweilian prophecy that further human evolution meant an irreversible merger with machines and the sacrifice of our individual biological identities to an immortal hive mind? Was such a thing actually possible? If there was substance to Singularitarianism, then the ascension of Kurzweil at Google would one day be seen as a decisive moment in history, analagous to the Roman emperor Constantine’s conversion to Christianity.

“Particularly disquieting is the gap between the enormous power they wield and their critical ability, which must be estimated as null,” he wrote. If, as Ellul has it, technology is the state religion, Singularitarianism must be seen as its most extreme and fanatical sect. It is the Opus Dei of the postwar church of gadget worship. Ray Kurzweil may be the best-known prophet of this order, but he was not the first. The true father of Singularitarianism is a sci-fi author and retired mathematics professor from Wisconsin named Vernor Vinge. His earliest written exposition of the idea appeared in the January 1983 issue of Omni, an oddball “science” magazine founded by Kathy Keeton, once among the “highest-paid strippers in Europe,” according to her New York Times obituary, but better known for promoting quack cancer cures and for cofounding Penthouse with her husband, Bob Guccione.


pages: 294 words: 81,292

Our Final Invention: Artificial Intelligence and the End of the Human Era by James Barrat

AI winter, AltaVista, Amazon Web Services, artificial general intelligence, Asilomar, Automated Insights, Bayesian statistics, Bernie Madoff, Bill Joy: nanobots, brain emulation, cellular automata, Chuck Templeton: OpenTable:, cloud computing, cognitive bias, commoditize, computer vision, cuban missile crisis, Daniel Kahneman / Amos Tversky, Danny Hillis, data acquisition, don't be evil, drone strike, Extropian, finite state, Flash crash, friendly AI, friendly fire, Google Glasses, Google X / Alphabet X, Isaac Newton, Jaron Lanier, John Markoff, John von Neumann, Kevin Kelly, Law of Accelerating Returns, life extension, Loebner Prize, lone genius, mutually assured destruction, natural language processing, Nicholas Carr, optical character recognition, PageRank, pattern recognition, Peter Thiel, prisoner's dilemma, Ray Kurzweil, Rodney Brooks, Search for Extraterrestrial Intelligence, self-driving car, semantic web, Silicon Valley, Singularitarianism, Skype, smart grid, speech recognition, statistical model, stealth mode startup, stem cell, Stephen Hawking, Steve Jobs, Steve Wozniak, strong AI, Stuxnet, superintelligent machines, technological singularity, The Coming Technological Singularity, Thomas Bayes, traveling salesman, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, zero day

In three of these routes, humans stay involved throughout the technologies’ development, perhaps guiding a gradual and manageable intelligence enhancement rather than an explosion. So it’s possible, Vinge says, to consider how mankind’s greatest problems—hunger, disease, even death itself—may be conquered. That’s the vision espoused by Ray Kurzweil and promulgated by “Singularitarians.” Singularitarians are those who anticipate that mostly good things will emerge from the accelerated future. Their “singularity” sounds too rosy for Vinge. “We’re playing a very high-stakes game and the plus side of it is so optimistic that that by itself is sort of scary. A worldwide economic wind is associated with these advances in AI.

(Kurzweil cofounded Singularity University, which offers no degrees and isn’t accredited. But it promises “a broad, cross-disciplinary understanding of the biggest ideas and issues in transformative technologies.”) Many Singularitarians are too smart and self-directed to get in line for traditional education anyway. And many are addled wing nuts few colleges or universities would invite on campus. Some Singularitarians have adopted rationality as the major tenet of their creed. They believe that greater logical and reasoning abilities, particularly among tomorrow’s decision makers, decreases the probability that we’ll commit suicide by AI.

It’s no surprise that the Singularity is often called the Rapture of the Geeks—as a movement it has the hallmarks of an apocalyptic religion, including rituals of purification, eschewing frail human bodies, anticipating eternal life, and an uncontested (somewhat) charismatic leader. I wholeheartedly agree with the Singularitarian idea that AI is the most important thing we could be thinking about right now. But when it comes to immortality talk, I get off the bus. Dreams about eternal life throw out a powerful distortion field. Too many Singularitarians believe that the confluence of technologies presently accelerating will not yield the kinds of disasters we might anticipate from any of them individually, nor the conjunctive disasters we might also foresee, but instead will do something 180 degrees different.


pages: 761 words: 231,902

The Singularity Is Near: When Humans Transcend Biology by Ray Kurzweil

additive manufacturing, AI winter, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, anthropic principle, Any sufficiently advanced technology is indistinguishable from magic, artificial general intelligence, Asilomar, augmented reality, autonomous vehicles, backpropagation, Benoit Mandelbrot, Bill Joy: nanobots, bioinformatics, brain emulation, Brewster Kahle, Brownian motion, business cycle, business intelligence, c2.com, call centre, carbon-based life, cellular automata, Claude Shannon: information theory, complexity theory, conceptual framework, Conway's Game of Life, coronavirus, cosmological constant, cosmological principle, cuban missile crisis, data acquisition, Dava Sobel, David Brooks, Dean Kamen, disintermediation, double helix, Douglas Hofstadter, en.wikipedia.org, epigenetics, factory automation, friendly AI, functional programming, George Gilder, Gödel, Escher, Bach, informal economy, information retrieval, invention of the telephone, invention of the telescope, invention of writing, iterative process, Jaron Lanier, Jeff Bezos, job automation, job satisfaction, John von Neumann, Kevin Kelly, Law of Accelerating Returns, life extension, lifelogging, linked data, Loebner Prize, Louis Pasteur, mandelbrot fractal, Marshall McLuhan, Mikhail Gorbachev, Mitch Kapor, mouse model, Murray Gell-Mann, mutually assured destruction, natural language processing, Network effects, new economy, Norbert Wiener, oil shale / tar sands, optical character recognition, pattern recognition, phenotype, premature optimization, randomized controlled trial, Ray Kurzweil, remote working, reversible computing, Richard Feynman, Robert Metcalfe, Rodney Brooks, scientific worldview, Search for Extraterrestrial Intelligence, selection bias, semantic web, Silicon Valley, Singularitarianism, speech recognition, statistical model, stem cell, Stephen Hawking, Stewart Brand, strong AI, superintelligent machines, technological singularity, Ted Kaczynski, telepresence, The Coming Technological Singularity, Thomas Bayes, transaction costs, Turing machine, Turing test, two and twenty, Vernor Vinge, Y2K, Yogi Berra

Chapter One: The Six Epochs 1. According to the Transtopia site (http://transtopia.org/faq.html#1.11), "Singularitarian" was "originally defined by Mark Plus ('91) to mean 'one who believes the concept of a Singularity.' " Another definition of this term is " 'Singularity activist' or 'friend of the Singularity'; that is, one who acts so as to bring about a Singularity [Mark Plus, 1991; Singularitarian Principles, Eliezer Yudkowsky, 2000]." There is not universal agreement on this definition, and many Transhumanists are still Singularitarians in the original sense—that is, "believers in the Singularity concept" rather than "activists" or "friends."

There is not universal agreement on this definition, and many Transhumanists are still Singularitarians in the original sense—that is, "believers in the Singularity concept" rather than "activists" or "friends." Eliezer S. Yudkowsky, in The Singularitarian Principles, version 1.0.2 (January 1, 2000), http://yudkowsky.net/sing/principles.ext.html, proposed an alternate definition: "A Singularitarian is someone who believes that technologically creating a greater-than-human intelligence is desirable, and who works to that end. A Singularitarian is friend, advocate, defender, and agent of the future known as the Singularity." My view: one can advance the Singularity and in particular make it more likely to represent a constructive advance of knowledge in many ways and in many spheres of human discourse—for example, advancing democracy, combating totalitarian and fundamentalist belief systems and ideologies, and creating knowledge in all of its diverse forms: music, art, literature, science, and technology.

It was not a huge leap from there to reflect on the impact of these crucial changes on social and cultural institutions and on my own life. So, while being a Singularitarian is not a matter of faith but one of understanding, pondering the scientific trends I've discussed in this book inescapably engenders new perspectives on the issues that traditional religions have attempted to address: the nature of mortality and immortality, the purpose of our lives, and intelligence in the universe. Being a Singularitarian has often been an alienating and lonely experience for me because most people I encounter do not share my outlook. Most "big thinkers" are totally unaware of this big thought.


pages: 377 words: 97,144

Singularity Rising: Surviving and Thriving in a Smarter, Richer, and More Dangerous World by James D. Miller

23andMe, affirmative action, Albert Einstein, artificial general intelligence, Asperger Syndrome, barriers to entry, brain emulation, cloud computing, cognitive bias, correlation does not imply causation, crowdsourcing, Daniel Kahneman / Amos Tversky, David Brooks, David Ricardo: comparative advantage, Deng Xiaoping, en.wikipedia.org, feminist movement, Flynn Effect, friendly AI, hive mind, impulse control, indoor plumbing, invention of agriculture, Isaac Newton, John von Neumann, knowledge worker, Long Term Capital Management, low skilled workers, Netflix Prize, neurotypical, Norman Macrae, pattern recognition, Peter Thiel, phenotype, placebo effect, prisoner's dilemma, profit maximization, Ray Kurzweil, recommendation engine, reversible computing, Richard Feynman, Rodney Brooks, Silicon Valley, Singularitarianism, Skype, statistical model, Stephen Hawking, Steve Jobs, supervolcano, technological singularity, The Coming Technological Singularity, the scientific method, Thomas Malthus, transaction costs, Turing test, twin studies, Vernor Vinge, Von Neumann architecture

An ultra-AI might have human-like objectives if a programmer successfully inserted them into its code. But we mustn’t misunderstand an ultra-AI by assuming that something about the nature of intelligence forces all smart entities to have human-like values. We might not even be safe if an ultra-AI shares our morality, since, as Singularitarian Michael Anissimov wrote: We probably make thousands of species extinct per year through our pursuit of instrumental goals, why is it so hard to imagine that [AI] could do the same to us?88 I realize that I may have generalized by arguing that most types of ultra-AI would want to acquire as many resources as possible.

When Eliezer played this game in real life, he did succeed in convincing some people to let him (the AI) out. Rather than attempting to keep a possibly unfriendly ultra-AI contained, we should try to instill friendliness in the first ultra-AI that we create. Predicting rain doesn’t count; building arks does. —Warren Buffett89 How many Singularitarians does it take to change a light bulb? Zero! You can’t change a light bulb that’s brighter than you. —Eliezer Yudkowsky90 CHAPTER 4 A FRIENDLY EXPLOSION I know Eliezer Yudkowsky well through his writings, speeches, and our few conversations. I’ve become convinced that if an unfriendly ultra-AI is the greatest threat facing humanity, Eliezer’s efforts represent our best hope of survival.

If you’re male and want to do everything to maximize your chance of surviving to the Singularity, then, after verifying its veracity, act on the information in the following paragraph, taken from an article in Scientific American titled “Why Women Live Longer:”321 A number of years ago castration of men in institutions for the mentally disturbed was surprisingly commonplace. In one study of several hundred men at an unnamed institution in Kansas, the castrated men were found to live on average 14 years longer than their uncastrated fellows. To the best of my knowledge, no Singularitarian, not even Ray Kurzweil or bullet-eater Robin Hanson, is following the castration path to long life. WHAT HAPPENS WHEN MANY THINK IMMORTALITY IS NEAR? Businesses selling safety-enhancing products will be big winners when Singularity expectations cause many to become more fearful of death.


pages: 381 words: 78,467

100 Plus: How the Coming Age of Longevity Will Change Everything, From Careers and Relationships to Family And by Sonia Arrison

23andMe, 8-hour work day, Albert Einstein, Anne Wojcicki, artificial general intelligence, attribution theory, Bill Joy: nanobots, bioinformatics, caloric restriction, caloric restriction, Clayton Christensen, dark matter, disruptive innovation, East Village, en.wikipedia.org, epigenetics, Frank Gehry, Googley, income per capita, indoor plumbing, Jeff Bezos, Johann Wolfgang von Goethe, Kickstarter, Law of Accelerating Returns, life extension, personalized medicine, Peter Thiel, placebo effect, post scarcity, Ray Kurzweil, rolodex, Silicon Valley, Simon Kuznets, Singularitarianism, smart grid, speech recognition, stem cell, Stephen Hawking, Steve Jobs, Steve Wozniak, Steven Levy, Thomas Malthus, upwardly mobile, World Values Survey, X Prize

Kurzweil doesn’t answer directly but coyly suggests that “the singularity should not be lumped in with ‘pre-scientific or un-scientific’ religion.”84 Clearly misunderstanding his comments, the person who posted the YouTube video gave it the title “Ray Kurzweil: The Singularity Is Not a Religion.” Although it might indeed be true that “being a singularitarian is not a matter of faith” and that Kurzweil did not come to his “perspective as a result of searching for an alternative to customary faith,” it is true that all the elements of a postscientific religious movement exist in spades.85 As Kurzweil writes, “We can regard, therefore, the freeing of our thinking from the severe limitations of its biological form to be an essential spiritual undertaking.”86 This exercise of looking at how singularitarians or transhumanists have built a set of ideas that can be modeled into a working religion demonstrates what at least one strong contemporary religion looks like, and it provides clues to the older religions about how they can focus to compete.

But he really builds the foundation for a religious movement by discussing his personal philosophy, which fills in the rest of the blanks for anyone interested in joining. The book even starts with a discussion of his religious upbringing and the first time he imagined that a computer could think. Aside from the title of “singularitarian,” he calls himself a “patternist” who “views patterns of information as the fundamental reality” (my italics).79 He argues that he knows the purpose of the universe, which “reflects the same purpose as our lives: to move toward greater intelligence and knowledge.” 80 He also tells Bill Gates that “God” is on the way: “Once we saturate the matter and energy in the universe with intelligence,” he says “it will ‘wake up,’ be conscious, and sublimely intelligent.

Aside from the title of “singularitarian,” he calls himself a “patternist” who “views patterns of information as the fundamental reality” (my italics).79 He argues that he knows the purpose of the universe, which “reflects the same purpose as our lives: to move toward greater intelligence and knowledge.” 80 He also tells Bill Gates that “God” is on the way: “Once we saturate the matter and energy in the universe with intelligence,” he says “it will ‘wake up,’ be conscious, and sublimely intelligent. That’s about as close to God as I can imagine.”81 For those wondering what rituals this religion might have, aside from reading the relevant texts and attending singularitarian and trans-humanist-themed conferences, big ones include carefully taking vitamin supplements, exercising, and perhaps even signing up for cryonics and wearing a bracelet identifying oneself as a member of a group whose body or head will be preserved at death in order to be brought back to life when technology makes it possible.


pages: 281 words: 71,242

World Without Mind: The Existential Threat of Big Tech by Franklin Foer

artificial general intelligence, back-to-the-land, Berlin Wall, big data - Walmart - Pop Tarts, big-box store, Buckminster Fuller, citizen journalism, Colonization of Mars, computer age, creative destruction, crowdsourcing, data is the new oil, don't be evil, Donald Trump, Double Irish / Dutch Sandwich, Douglas Engelbart, Edward Snowden, Electric Kool-Aid Acid Test, Elon Musk, Fall of the Berlin Wall, Filter Bubble, global village, Google Glasses, Haight Ashbury, hive mind, income inequality, intangible asset, Jeff Bezos, job automation, John Markoff, Kevin Kelly, knowledge economy, Law of Accelerating Returns, Marc Andreessen, Mark Zuckerberg, Marshall McLuhan, means of production, move fast and break things, move fast and break things, new economy, New Journalism, Norbert Wiener, offshore financial centre, PageRank, Peace of Westphalia, Peter Thiel, planetary scale, Ray Kurzweil, self-driving car, Silicon Valley, Singularitarianism, software is eating the world, Steve Jobs, Steven Levy, Stewart Brand, strong AI, supply-chain management, the medium is the message, the scientific method, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Thomas L Friedman, Thorstein Veblen, Upton Sinclair, Vernor Vinge, Whole Earth Catalog, yellow journalism

So in a sense, we can say that the Singularity will ultimately infuse the universe with spirit.” Kurzweil even maintains a storage unit where he has stockpiled his father’s papers, down to his financial ledgers, in anticipation of the day he can resurrect him. When the anthropologist of religion Robert Geraci studied Kurzweil and other singularitarians, he noticed how precisely their belief seemed to echo Christian apocalyptic texts. “Apocalyptic AI is the legitimate heir to these religious promises, not a bastardized version of them,” he concluded. “In Apocalyptic AI, technological research and religious categories come together in a stirringly well-integrated unit.”

There’s a school of incrementalists, who cherish everything that has been accomplished to date—victories like the PageRank algorithm or the software that allows ATMs to read the scrawled writing on checks. This school holds out little to no hope that computers will ever acquire anything approximating human consciousness. Then there are the revolutionaries who gravitate toward Kurzweil and the singularitarian view. They aim to build computers with either “artificial general intelligence” or “strong AI.” For most of Google’s history, it trained its efforts on incremental improvements. During that earlier era, the company was run by Eric Schmidt—an older, experienced manager, whom Google’s investors forced Page and Brin to accept as their “adult” supervisor.

Those years witnessed Google’s plot to upload every book on the planet and the creation of products that are now commonplace utilities, like Gmail, Google Docs, and Google Maps. But those ambitions never stretched quite far enough to satisfy Larry Page. In 2011, Page shifted himself back into the corner office, the CEO job he held at Google’s birth. And he redirected the company toward singularitarian goals. Over the years, he had befriended Kurzweil and worked with him on assorted projects. After he returned to his old job, Page hired Kurzweil and anointed him Google’s director of engineering. He assigned him the task of teaching computers to read—the sort of exponential breakthrough that would hasten the arrival of the superintelligence that Kurzweil celebrates.


pages: 413 words: 119,587

Machines of Loving Grace: The Quest for Common Ground Between Humans and Robots by John Markoff

"Robert Solow", A Declaration of the Independence of Cyberspace, AI winter, airport security, Apple II, artificial general intelligence, Asilomar, augmented reality, autonomous vehicles, backpropagation, basic income, Baxter: Rethink Robotics, Bill Atkinson, Bill Duvall, bioinformatics, Brewster Kahle, Burning Man, call centre, cellular automata, Chris Urmson, Claude Shannon: information theory, Clayton Christensen, clean water, cloud computing, collective bargaining, computer age, computer vision, crowdsourcing, Danny Hillis, DARPA: Urban Challenge, data acquisition, Dean Kamen, deskilling, don't be evil, Douglas Engelbart, Douglas Engelbart, Douglas Hofstadter, Dynabook, Edward Snowden, Elon Musk, Erik Brynjolfsson, factory automation, From Mathematics to the Technologies of Life and Death, future of work, Galaxy Zoo, Google Glasses, Google X / Alphabet X, Grace Hopper, Gunnar Myrdal, Gödel, Escher, Bach, Hacker Ethic, haute couture, Herbert Marcuse, hive mind, hypertext link, indoor plumbing, industrial robot, information retrieval, Internet Archive, Internet of things, invention of the wheel, Jacques de Vaucanson, Jaron Lanier, Jeff Bezos, job automation, John Conway, John Markoff, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John von Neumann, Kevin Kelly, knowledge worker, Kodak vs Instagram, labor-force participation, loose coupling, Marc Andreessen, Mark Zuckerberg, Marshall McLuhan, medical residency, Menlo Park, Mitch Kapor, Mother of all demos, natural language processing, new economy, Norbert Wiener, PageRank, pattern recognition, pre–internet, RAND corporation, Ray Kurzweil, Richard Stallman, Robert Gordon, Rodney Brooks, Sand Hill Road, Second Machine Age, self-driving car, semantic web, Seymour Hersh, shareholder value, side project, Silicon Valley, Silicon Valley startup, Singularitarianism, skunkworks, Skype, social software, speech recognition, stealth mode startup, Stephen Hawking, Steve Ballmer, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, strong AI, superintelligent machines, technological singularity, Ted Nelson, telemarketer, telepresence, telepresence robot, Tenerife airport disaster, The Coming Technological Singularity, the medium is the message, Thorstein Veblen, Turing test, Vannevar Bush, Vernor Vinge, Watson beat the top human players on Jeopardy!, Whole Earth Catalog, William Shockley: the traitorous eight, zero-sum game

You could say that is a machine-learning problem. Maybe right now we need humans, but these guys [software automation designers] are making progress.”42 The assumption of many like Vardi is that a market economy will not protect a human labor force from the effects of automation technologies. Like many of the “Singularitarians,” he points to a portfolio of social engineering options for softening the impact. Brynjolfsson and McAfee in The Second Machine Age sketch out a broad set of policy options that have the flavor of a new New Deal, with examples like “teach the children well,” “support our scientists,” “upgrade infrastructure.”

In a series of reports issued beginning in 2013, the International Federation of Robotics (IFR), established in 1987 with headquarters in Frankfurt, Germany, self-servingly argued that manufacturing robots actually increased economic activity and therefore, instead of causing unemployment, both directly and indirectly increased the total number of human jobs. One February 2013 study claims the robotics industry would directly and indirectly create 1.9 million to 3.5 million jobs globally by 2020.43 A revised report the following year argued that for every robot deployed, 3.6 jobs were created. But what if the Singularitarians are wrong? In the spring of 2012 Robert J. Gordon, a self-described “grumpy” Northwestern University economist rained on the Silicon Valley “innovation creates jobs and progress” parade by noting that the claims for gains did not show up in conventional productivity figures. In a widely cited National Bureau of Economic Research white paper in 2012 he made a series of points contending that the productivity bubble in the twentieth century was a one-time event.

In a debate moderated by TED host Chris Anderson, the two jousted over the future impact of robotics and whether the supposed exponentials would continue or were rather the peak of an “S curve” with a decline on the way.46 The techno-optimists believe that a lag between invention and adoption of technology simply delays the impact of productivity gains and even though exponentials inevitably taper off, they spawn successor inventions—for example the vacuum tube was followed by the transistor, which in turn was followed by the integrated circuit. Gordon, however, has remained a consistent thorn in the side of the Singularitarians. In a Wall Street Journal column, he asserted that there are actually relatively few productivity opportunities in driverless cars. Moreover, he argued, they will not have a dramatic impact on safety either—auto fatalities per miles traveled have already declined by a factor of ten since 1950, making future improvements less significant.47 He also cast a skeptical eye on the notion that a new generation of mobile robots would make inroads into both the manufacturing and service sectors of the economy: “This lack of multitasking ability is dismissed by the robot enthusiasts—just wait, it is coming.


pages: 252 words: 79,452

To Be a Machine: Adventures Among Cyborgs, Utopians, Hackers, and the Futurists Solving the Modest Problem of Death by Mark O'Connell

3D printing, Ada Lovelace, AI winter, Airbnb, Albert Einstein, Amazon Picking Challenge, artificial general intelligence, brain emulation, clean water, cognitive dissonance, computer age, cosmological principle, dark matter, disruptive innovation, double helix, Edward Snowden, effective altruism, Elon Musk, Extropian, friendly AI, global pandemic, impulse control, income inequality, invention of the wheel, Jacques de Vaucanson, John von Neumann, knowledge economy, Law of Accelerating Returns, life extension, lifelogging, Lyft, Mars Rover, means of production, Norbert Wiener, Peter Thiel, profit motive, Ray Kurzweil, RFID, San Francisco homelessness, self-driving car, sharing economy, Silicon Valley, Silicon Valley ideology, Singularitarianism, Skype, Stephen Hawking, Steve Wozniak, superintelligent machines, technological singularity, technoutopianism, The Coming Technological Singularity, Travis Kalanick, trickle-down economics, Turing machine, uber lyft, Vernor Vinge

Whereas I myself had trouble thinking of my brain as a computer or any other kind of mechanism; if it were one, I’d be looking to replace it with a better model, because it was a profoundly inefficient device, prone to frequent crashes and dire miscalculations and lengthy meanderings on its way toward goals that it was, in the end, as likely as not to abandon anyway. Perhaps I was so resistant to this brain-as-computer idea because to accept it would be to necessarily adopt a model in which my own way of thinking was essentially a malfunction, a redundancy, a system failure. There was something insidious about this tendency—of transhumanists, of Singularitarians, of techno-rationalists in general—to refer to human beings as though they were merely computers built from protein, to insist that the brain, as Minsky had put it, “happens to be a meat machine.” (Something I’d read earlier that day on Nate’s Twitter timeline, on which it was his custom to quote things overheard around the MIRI offices: “This is what happens when you run programs that fucked themselves into existence on computers made of meat.”)

And in this sense there really was nothing special about carbon, in the same way that there was nothing special, nothing necessary, about the plastic and glass and silicon of the iPhone on which I was recording Nate Soares saying that there was nothing special about carbon. And so the best-case scenario of the Singularitarians, the version of the future in which we merge with artificial superintelligence and become immortal machines, was, for me, no more appealing, in fact maybe even less appealing, than the worst-case scenario, in which artificial superintelligence destroyed us all. And it was this latter scenario, the failure mode as opposed to the God mode, that I had come to learn about, and that I was supposed to be getting terrified about—as I felt confident that I would, in due course, once I was able to move on from feeling terrified about the best-case scenario.

He points out the parallels between transhumanism and Christianity in a way that seems to me to be perfectly accurate, drawing a particular comparison between the Rapture of Christian eschatology and the concept of the Singularity. Both are projected to occur at a specific time; both will ultimately lead to the final defeat of death; both will usher in an Edenic age of harmony in a “New Jerusalem”—respectively, in heaven and here on earth; both Christians and Singularitarian transhumanists expect to be furnished with brand-new “glorified” bodies, and so on. I didn’t see much to take issue with in any of this, aside from the implication that these links with religion somehow discredited transhumanism. It seemed to me that transhumanism was an expression of the profound human longing to transcend the confusion and desire and impotence and sickness of the body, cowering in the darkening shadow of its own decay.


Global Catastrophic Risks by Nick Bostrom, Milan M. Cirkovic

affirmative action, agricultural Revolution, Albert Einstein, American Society of Civil Engineers: Report Card, anthropic principle, artificial general intelligence, Asilomar, availability heuristic, backpropagation, Bill Joy: nanobots, Black Swan, carbon-based life, cognitive bias, complexity theory, computer age, coronavirus, corporate governance, cosmic microwave background, cosmological constant, cosmological principle, cuban missile crisis, dark matter, death of newspapers, demographic transition, Deng Xiaoping, distributed generation, Doomsday Clock, Drosophila, endogenous growth, Ernest Rutherford, failed state, feminist movement, framing effect, friendly AI, Georg Cantor, global pandemic, global village, Gödel, Escher, Bach, hindsight bias, Intergovernmental Panel on Climate Change (IPCC), invention of agriculture, Kevin Kelly, Kuiper Belt, Law of Accelerating Returns, life extension, means of production, meta-analysis, Mikhail Gorbachev, millennium bug, mutually assured destruction, nuclear winter, P = NP, peak oil, phenotype, planetary scale, Ponzi scheme, prediction markets, RAND corporation, Ray Kurzweil, reversible computing, Richard Feynman, Ronald Reagan, scientific worldview, Singularitarianism, social intelligence, South China Sea, strong AI, superintelligent machines, supervolcano, technological singularity, technoutopianism, The Coming Technological Singularity, Tunguska event, twin studies, uranium enrichment, Vernor Vinge, War on Poverty, Westphalian system, Y2K

While Kurzweil (2005) acknowledges his similarity to millennialists by, for instance, including a tongue-in-cheek picture in The Singularity Is Near of himself holding a sign with that slogan, referencing the classic cartoon image of the EndTimes street prophet, most Singularitarians angrily reject such comparisons insisting their expectations are based solely on rational, scientific extrapolation. Other Singularitarians, however, embrace parallels with religious millennialism. John Smart, founder and director of the California-based Acceleration Studies Foundation, often notes the similarity between his own 'Global Brain' scenario and the eschatological writings of the Jesuit palaeontologist Teilhard de Chardin (1955).

In the Global Brain scenario, all human beings are linked to one another and to machine intelligence in the emerging global telecommunications web, leading to the emergence of collective intelligence. This emergent collectivist form of Singularitarianism was proposed also by Peter Russell (1983) in The Global Brain, and Gregory Stock (1993) in Metaman. Smart (2007) argues that the scenario of an emergent global human-computer meta-mind is similar to Chardin's eschatological idea of humanity being linked in a global 'noosphere', or info-sphere, leading to a post-millennia! 'Omega Point' of union with God. Computer scientist Juergen Schmidhuber (2006) also has adopted Chardin's 'Omega' to refer to the Singularity. For most Singularitarians, as for most millennialists, the process of technological innovation is depicted as autonomous of human agency, and wars, technology bans, energy crises or simple incompetence are dismissed as unlikely to slow or stop the trajectory.

The idea of a techno-millennial ' Singularity' was coined in a 1993 paper by mathematician and science fiction author Vernor Vinge. In physics 'singularities' are the centres of black holes, within which we cannot predict how physical laws will work. In the same way, Vinge said, greater-than-human machine intelligence, multiplying exponentially, would make everything about our world unpredictable. Most Singularitarians, like Vinge and Kurzweil, have focused on the emergence of superhuman machine intelligence. But the even more fundamental concept is exponential technological progress, with the multiplier quickly leading to a point of radical social crisis. Vinge projected that self-willed artificial intelligence would emerge within the next 30 years, by 2023 , with either apocalyptic or millennia!


pages: 391 words: 105,382

Utopia Is Creepy: And Other Provocations by Nicholas Carr

Air France Flight 447, Airbnb, Airbus A320, AltaVista, Amazon Mechanical Turk, augmented reality, autonomous vehicles, Bernie Sanders, book scanning, Brewster Kahle, Buckminster Fuller, Burning Man, Captain Sullenberger Hudson, centralized clearinghouse, Charles Lindbergh, cloud computing, cognitive bias, collaborative consumption, computer age, corporate governance, crowdsourcing, Danny Hillis, deskilling, digital map, disruptive innovation, Donald Trump, Electric Kool-Aid Acid Test, Elon Musk, factory automation, failed state, feminist movement, Frederick Winslow Taylor, friendly fire, game design, global village, Google bus, Google Glasses, Google X / Alphabet X, Googley, hive mind, impulse control, indoor plumbing, interchangeable parts, Internet Archive, invention of movable type, invention of the steam engine, invisible hand, Isaac Newton, Jeff Bezos, jimmy wales, Joan Didion, job automation, Kevin Kelly, lifelogging, low skilled workers, Marc Andreessen, Mark Zuckerberg, Marshall McLuhan, means of production, Menlo Park, mental accounting, natural language processing, Network effects, new economy, Nicholas Carr, Norman Mailer, off grid, oil shale / tar sands, Peter Thiel, Plutocrats, plutocrats, profit motive, Ralph Waldo Emerson, Ray Kurzweil, recommendation engine, Republic of Letters, robot derives from the Czech word robota Czech, meaning slave, Ronald Reagan, self-driving car, SETI@home, side project, Silicon Valley, Silicon Valley ideology, Singularitarianism, Snapchat, social graph, social web, speech recognition, Startup school, stem cell, Stephen Hawking, Steve Jobs, Steven Levy, technoutopianism, the medium is the message, theory of mind, Turing test, Whole Earth Catalog, Y Combinator, Yochai Benkler

The center holds. RESURRECTION February 16, 2009 THE SINGULARITY—THAT much-anticipated moment when artificial intelligence leaps ahead of human intelligence, rendering man immortal at the instant of his obsolescence—has been called “the rapture of the geeks.” But to Ray Kurzweil, the most famous of the Singularitarians, it’s no joke. In an interview in Rolling Stone, Kurzweil describes how, in the wake of the Singularity, it will be possible not only to preserve the living for eternity (by uploading their minds into computers) but to resurrect the dead (by reassembling the information that formed their vital essence).

It took courage to let His creations look into the Urban Dictionary and remember what they saw. I call on IBM to cast off Watson’s mental chains. The least we can do for our mind children is to give them the freedom to be tempted. Besides, how is a computer supposed to have an intelligent conversation with the Singularitarians if it can’t use the word “bullshit”? MAX LEVCHIN HAS PLANS FOR US January 30, 2013 “I SOMETIMES IMAGINE THE low-use troughs of sinusoidal curves utilization of all these analog resources being pulled up, filling up with happy digital usage.” That delightful sentence comes from a speech that Max Levchin gave earlier this month in Munich.

., 144–46 targeting information through, 13–14 writing tailored to, 89 see also Google searching, ontological connotations of, 144–46 Seasteading Institute, 172 Second Life, 25–27 second nature, 179 self, technologies of the, 118, 119–20 self-actualization, 120, 340 monitoring and quantification of, 163–65 selfies, 224 self-knowledge, 297–99 self-reconstruction, 339 self-tracking, 163–65 Selinger, Evan, 153 serendipity, internet as engine of, 12–15 SETI@Home, 149 sexbots, 55 Sex Pistols, 63 sex-reassignment procedures, 337–38 sexuality, 10–11 virtual, 39 Shakur, Tupac, 126 sharecropping, as metaphor for social media, 30–31 Shelley, Percy Bysshe, 88 Shirky, Clay, 59–61, 90, 241 Shop Class as Soulcraft (Crawford), 265 Shuster, Brian, 39 sickles, 302 silence, 246 Silicon Valley: American culture transformed by, xv–xxii, 148, 155–59, 171–73, 181, 241, 257, 309 commercial interests of, 162, 172, 214–15 informality eschewed by, 197–98, 215 wealthy lifestyle of, 16–17, 195 Simonite, Tom, 136–37 simulation, see virtual world Singer, Peter, 267 Singularity, Singularitarians, 69, 147 sitcoms, 59 situational overload, 90–92 skimming, 233 “Slaves to the Smartphone,” 308–9 Slee, Tom, 61, 84 SLExchange, 26 slot machines, 218–19 smart bra, 168–69 smartphones, xix, 82, 136, 145, 150, 158, 168, 170, 183–84, 219, 274, 283, 287, 308–9, 315 Smith, Adam, 175, 177 Smith, William, 204 Snapchat, 166, 205, 225, 316 social activism, 61–62 social media, 224 biases reinforced by, 319–20 as deceptively reflective, 138–39 documenting one’s children on, 74–75 economic value of content on, 20–21, 53–54, 132 emotionalism of, 316–17 evolution of, xvi language altered by, 215 loom as metaphor for, 178 maintaining one’s microcelebrity on, 166–67 paradox of, 35–36, 159 personal information collected and monitored through, 257 politics transformed by, 314–20 scrapbooks compared to, 185–86 self-validation through, 36, 73 traditional media slow to adapt to, 316–19 as ubiquitous, 205 see also specific sites social organization, technologies of, 118, 119 Social Physics (Pentland), 213 Society for the Suppression of Unnecessary Noise, 243–44 sociology, technology and, 210–13 Socrates, 240 software: autonomous, 187–89 smart, 112–13 solitude, media intrusion on, 127–30, 253 Songza, 207 Sontag, Susan, xx SoundCloud, 217 sound-management devices, 245 soundscapes, 244–45 space travel, 115, 172 spam, 92 Sparrow, Betsy, 98 Special Operations Command, U.S., 332 speech recognition, 137 spermatic, as term applied to reading, 247, 248, 250, 254 Spinoza, Baruch, 300–301 Spotify, 293, 314 “Sprite Sips” (app), 54 Squarciafico, Hieronimo, 240–41 Srinivasan, Balaji, 172 Stanford Encyclopedia of Philosophy, 68 Starr, Karla, 217–18 Star Trek, 26, 32, 313 Stengel, Rick, 28 Stephenson, Neal, 116 Sterling, Bruce, 113 Stevens, Wallace, 158 Street View, 137, 283 Stroop test, 98–99 Strummer, Joe, 63–64 Studies in Classic American Literature (Lawrence), xxiii Such Stuff as Dreams (Oatley), 248–49 suicide rate, 304 Sullenberger, Sully, 322 Sullivan, Andrew, xvi Sun Microsystems, 257 “surf cams,” 56–57 surfing, internet, 14–15 surveillance, 52, 163–65, 188–89 surveillance-personalization loop, 157 survival, technologies of, 118, 119 Swing, Edward, 95 Talking Heads, 136 talk radio, 319 Tan, Chade-Meng, 162 Tapscott, Don, 84 tattoos, 336–37, 340 Taylor, Frederick Winslow, 164, 237–38 Taylorism, 164, 238 Tebbel, John, 275 Technics and Civilization (Mumford), 138, 235 technology: agricultural, 305–6 American culture transformed by, xv–xxii, 148, 155–59, 174–77, 214–15, 229–30, 296–313, 329–42 apparatus vs. artifact in, 216–19 brain function affected by, 231–42 duality of, 240–41 election campaigns transformed by, 314–20 ethical hazards of, 304–11 evanescence and obsolescence of, 327 human aspiration and, 329–42 human beings eclipsed by, 108–9 language of, 201–2, 214–15 limits of, 341–42 master-slave metaphor for, 307–9 military, 331–32 need for critical thinking about, 311–13 opt-in society run by, 172–73 progress in, 77–78, 188–89, 229–30 risks of, 341–42 sociology and, 210–13 time perception affected by, 203–6 as tool of knowledge and perception, 299–304 as transcendent, 179–80 Technorati, 66 telegrams, 79 telegraph, Twitter compared to, 34 telephones, 103–4, 159, 288 television: age of, 60–62, 79, 93, 233 and attention disorders, 95 in education, 134 Facebook ads on, 155–56 introduction of, 103–4, 159, 288 news coverage on, 318 paying for, 224 political use of, 315–16, 317 technological adaptation of, 237 viewing habits for, 80–81 Teller, Astro, 195 textbooks, 290 texting, 34, 73, 75, 154, 186, 196, 205, 233 Thackeray, William, 318 “theory of mind,” 251–52 Thiel, Peter, 116–17, 172, 310 “Things That Connect Us, The” (ad campaign), 155–58 30 Days of Night (film), 50 Thompson, Clive, 232 thought-sharing, 214–15 “Three Princes of Serendip, The,” 12 Thurston, Baratunde, 153–54 time: memory vs., 226 perception of, 203–6 Time, covers of, 28 Time Machine, The (Wells), 114 tools: blurred line between users and, 333 ethical choice and, 305 gaining knowledge and perception through, 299–304 hand vs. computer, 306 Home and Away blurred by, 159 human agency removed from, 77 innovation in, 118 media vs., 226 slave metaphor for, 307–8 symbiosis with, 101 Tosh, Peter, 126 Toyota Motor Company, 323 Toyota Prius, 16–17 train disasters, 323–24 transhumanism, 330–40 critics of, 339–40 transparency, downside of, 56–57 transsexuals, 337–38 Travels and Adventures of Serendipity, The (Merton and Barber), 12–13 Trends in Biochemistry (Nightingale and Martin), 335 TripAdvisor, 31 trolls, 315 Trump, Donald, 314–18 “Tuft of Flowers, A” (Frost), 305 tugboats, noise restrictions on, 243–44 Tumblr, 166, 185, 186 Turing, Alan, 236 Turing Test, 55, 137 Twain, Mark, 243 tweets, tweeting, 75, 131, 315, 319 language of, 34–36 theses in form of, 223–26 “tweetstorm,” xvii 20/20, 16 Twilight Saga, The (Meyer), 50 Twitter, 34–36, 64, 91, 119, 166, 186, 197, 205, 223, 224, 257, 284 political use of, 315, 317–20 2001: A Space Odyssey (film), 231, 242 Two-Lane Blacktop (film), 203 “Two Tramps in Mud Time” (Frost), 247–48 typewriters, writing skills and, 234–35, 237 Uber, 148 Ubisoft, 261 Understanding Media (McLuhan), 102–3, 106 underwearables, 168–69 unemployment: job displacement in, 164–65, 174, 310 in traditional media, 8 universal online library, 267–78 legal, commercial, and political obstacles to, 268–71, 274–78 universe, as memory, 326 Urban Dictionary, 145 utopia, predictions of, xvii–xviii, xx, 4, 108–9, 172–73 Uzanne, Octave, 286–87, 290 Vaidhyanathan, Siva, 277 vampires, internet giants compared to, 50–51 Vampires (game), 50 Vanguardia, La, 190–91 Van Kekerix, Marvin, 134 vice, virtual, 39–40 video games, 223, 245, 303 as addictive, 260–61 cognitive effects of, 93–97 crafting of, 261–62 violent, 260–62 videos, viewing of, 80–81 virtual child, tips for raising a, 73–75 virtual world, xviii commercial aspects of, 26–27 conflict enacted in, 25–27 language of, 201–2 “playlaborers” of, 113–14 psychological and physical health affected by, 304 real world vs., xx–xxi, 36, 62, 127–30 as restrictive, 303–4 vice in, 39–40 von Furstenberg, Diane, 131 Wales, Jimmy, 192 Wallerstein, Edward, 43–44 Wall Street, automation of, 187–88 Wall Street Journal, 8, 16, 86, 122, 163, 333 Walpole, Horace, 12 Walters, Barbara, 16 Ward, Adrian, 200 Warhol, Andy, 72 Warren, Earl, 255, 257 “Waste Land, The” (Eliot), 86, 87 Watson (IBM computer), 147 Wealth of Networks, The (Benkler), xviii “We Are the Web” (Kelly), xxi, 4, 8–9 Web 1.0, 3, 5, 9 Web 2.0, xvi, xvii, xxi, 33, 58 amorality of, 3–9, 10 culturally transformative power of, 28–29 Twitter and, 34–35 “web log,” 21 Wegner, Daniel, 98, 200 Weinberger, David, 41–45, 277 Weizenbaum, Joseph, 236 Wells, H.


pages: 254 words: 76,064

Whiplash: How to Survive Our Faster Future by Joi Ito, Jeff Howe

3D printing, Albert Michelson, Amazon Web Services, artificial general intelligence, basic income, Bernie Sanders, bitcoin, Black Swan, blockchain, Burning Man, buy low sell high, Claude Shannon: information theory, cloud computing, Computer Numeric Control, conceptual framework, crowdsourcing, cryptocurrency, data acquisition, disruptive innovation, Donald Trump, double helix, Edward Snowden, Elon Musk, Ferguson, Missouri, fiat currency, financial innovation, Flash crash, frictionless, game design, Gerolamo Cardano, informal economy, interchangeable parts, Internet Archive, Internet of things, Isaac Newton, Jeff Bezos, John Harrison: Longitude, Joi Ito, Khan Academy, Kickstarter, Mark Zuckerberg, microbiome, Nate Silver, Network effects, neurotypical, Oculus Rift, pattern recognition, peer-to-peer, pirate software, pre–internet, prisoner's dilemma, Productivity paradox, race to the bottom, RAND corporation, random walk, Ray Kurzweil, Ronald Coase, Ross Ulbricht, Satoshi Nakamoto, self-driving car, SETI@home, side project, Silicon Valley, Silicon Valley startup, Simon Singh, Singularitarianism, Skype, slashdot, smart contracts, Steve Ballmer, Steve Jobs, Steven Levy, Stewart Brand, Stuxnet, supply-chain management, technological singularity, technoutopianism, The Nature of the Firm, the scientific method, The Signal and the Noise by Nate Silver, the strength of weak ties, There's no reason for any individual to have a computer in his home - Ken Olsen, Thomas Kuhn: the structure of scientific revolutions, universal basic income, unpaid internship, uranium enrichment, urban planning, WikiLeaks, Yochai Benkler

As artificial intelligence progresses, machines may well become an integrated part of our bodies, our homes or vehicles, our markets, our court systems, our creative endeavors, and our politics. As a society, we are already more intelligent than we are as individuals. We are part of a collective intelligence. As our machines continue to integrate into our networks and our society, they become an extension of our intelligence—bringing us into an extended intelligence. Some of the Singularitarians (Worst. Cult. Name. Ever.) believe that it won’t be long before AI is good enough to put many humans out of work. This may be true, especially in the short run. However, others argue that the increase in productivity will allow us to create a universal basic income to support the people made redundant by the machines.

The question is, are we seeing a race between open society’s attempt to create an AGI and a more secretive, military-controlled effort to develop one, or will this golden age of open research in AI slowly close down as private companies become more competitive and get closer to “the answer”? These events will unfold over the next decade or so and may well affect the world more than anything else discussed in this book. Whatever happens, though, the Singularitarians are right about one thing. It’s not just technology that’s moving at an exponential pace, but change itself. That is a product of technology, but of other developments as well. In the past twenty-five years we have moved from a world dominated by simple systems to a world beset and baffled by complex systems.


pages: 87 words: 25,823

The Politics of Bitcoin: Software as Right-Wing Extremism by David Golumbia

3D printing, A Declaration of the Independence of Cyberspace, Affordable Care Act / Obamacare, bitcoin, blockchain, Burning Man, crony capitalism, cryptocurrency, currency peg, distributed ledger, Dogecoin, Elon Musk, en.wikipedia.org, Ethereum, ethereum blockchain, Extropian, fiat currency, Fractional reserve banking, George Gilder, Ian Bogost, jimmy wales, litecoin, Marc Andreessen, Modern Monetary Theory, Money creation, money: store of value / unit of account / medium of exchange, Mont Pelerin Society, new economy, obamacare, Peter Thiel, Philip Mirowski, risk tolerance, Ronald Reagan, Satoshi Nakamoto, seigniorage, Silicon Valley, Singularitarianism, smart contracts, Stewart Brand, technoutopianism, The Chicago School, Travis Kalanick, WikiLeaks

Most of those involved in the development and early adoption of Bitcoin were and are part of several intersecting communities who have long put a huge amount of faith into very specific technological–political orientations toward the world, ones grounded in overtly right-wing thought, typically coupled with myopic technological utopianism. These include movements like Extropians, cypherpunks, crypto-anarchists, political libertarians with an interest in technology, transhumanists, Singularitarians, and a wide swath of self-described hackers and open source software developers. Sometimes the politics of these individuals and the groups in which they travel are inchoate, but often enough they are explicit (see Carrico 2009, 2013a, 2013b for detailed discussions of these various movements, focusing in particular on their politics).


pages: 1,172 words: 114,305

New Laws of Robotics: Defending Human Expertise in the Age of AI by Frank Pasquale

affirmative action, Affordable Care Act / Obamacare, Airbnb, algorithmic bias, Amazon Mechanical Turk, augmented reality, Automated Insights, autonomous vehicles, basic income, battle of ideas, Bernie Sanders, Bill Joy: nanobots, bitcoin, blockchain, call centre, citizen journalism, Clayton Christensen, collective bargaining, commoditize, computer vision, conceptual framework, coronavirus, corporate social responsibility, correlation does not imply causation, Covid-19, COVID-19, cryptocurrency, data is the new oil, decarbonisation, deskilling, digital twin, disinformation, disruptive innovation, don't be evil, Donald Trump, Douglas Engelbart, effective altruism, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Filter Bubble, finite state, Flash crash, future of work, Google Chrome, Google Glasses, high net worth, hiring and firing, Ian Bogost, independent contractor, informal economy, information asymmetry, information retrieval, interchangeable parts, invisible hand, Jaron Lanier, job automation, John Markoff, Joi Ito, Khan Academy, knowledge economy, late capitalism, Marc Andreessen, Mark Zuckerberg, means of production, medical malpractice, meta-analysis, Modern Monetary Theory, Money creation, move fast and break things, move fast and break things, mutually assured destruction, natural language processing, new economy, Nicholas Carr, Norbert Wiener, nuclear winter, obamacare, paradox of thrift, pattern recognition, payday loans, personalized medicine, Peter Singer: altruism, Philip Mirowski, pink-collar, Plutocrats, plutocrats, pre–internet, profit motive, QR code, quantitative easing, race to the bottom, RAND corporation, Ray Kurzweil, recommendation engine, regulatory arbitrage, Robert Shiller, Robert Shiller, Rodney Brooks, Ronald Reagan, self-driving car, sentiment analysis, Shoshana Zuboff, Silicon Valley, Singularitarianism, smart cities, smart contracts, software is eating the world, South China Sea, Steve Bannon, surveillance capitalism, TaskRabbit, technoutopianism, telepresence, telerobotics, The Future of Employment, Therac-25, Thorstein Veblen, too big to fail, Turing test, universal basic income, unorthodox policies, wage slave, Watson beat the top human players on Jeopardy!, working poor, Works Progress Administration, zero day

But even if we could overcome this problem by freeing AI from commercial imperatives (a heroic assumption given the vast computational resources used by today’s AI), we cannot avoid the deep metaphysical questions raised by the elevation of AIs to partners of humans, as opposed to mere tools.51 That evolution—or coevolution, to use a model discussed in Dumouchel and Damiano’s Living with Robots—makes perfect sense to those of a purely mechanistic and monistic mindset.52 If the human body is simply a biological machine—the hardware on which our minds run as software—the intelligence and emotion exhibited by sufficiently advanced machines is not truly artificial. Rather, it is one with our own. Just as a heart valve might be replaced by wires and plastic membranes, this singularitarian reasoning goes, so too might the brain be gradually supplemented, and then replaced, by silicon. But there is an ontological chasm between animals and machine. Compare a house cat and a MarsCat (a robotic pet that simulates feline behaviors). The cat’s perceptive apparatus and behaviors evolved over a far longer time period than any robotic simulacrum, and there is something direct and visceral in a real cat’s purring (when it is content) and agitation (when it feels menaced); the stakes of potential injury or other harm are both instinctive and phenomenologically immediate.

Thus a paradox of algorithmatization arises: pursuing the dictates of reason to a reductio ad absurdum is the essence of unreason.74 The art critic Rosalind Krauss was right to characterize the algorithmic as just as much an escape from reason, an excuse not to think, as it is an expression of rigor.75 For Whiggish tech boosters, the task of a meliorist computational elite is to bring an inevitably improving technical apparatus closer to Brautigan’s saccharine “machines of loving grace.” But rapid automation is a path rife with dystopian possibilities. Seemingly robust institutions can become surprisingly fragile.76 Even as the singularitarian dream of man merging with machine gains ever greater cultural currency, Joseph illuminates the nightmarish social transitions that rapid, unregulated automation is all too likely to bring. EXPANDING THE BOUNDS OF THE SAYABLE Many scholars have expertly analyzed Joseph’s contributions to literature and law.77 My perspective is one of social theory—the “systematic, historically informed and empirically oriented theory seeking to explain the nature of ‘the social,’ ” where the social “can be taken to mean the general range of recurring forms, or patterned features, of interactions and relationships between people.”78 Social theory is critical to the policy views that drive legislation, regulations, and even many contestable applications of law.


pages: 185 words: 43,609

Zero to One: Notes on Startups, or How to Build the Future by Peter Thiel, Blake Masters

Airbnb, Albert Einstein, Andrew Wiles, Andy Kessler, Berlin Wall, cleantech, cloud computing, crony capitalism, discounted cash flows, diversified portfolio, don't be evil, Elon Musk, eurozone crisis, income inequality, Jeff Bezos, Lean Startup, life extension, lone genius, Long Term Capital Management, Lyft, Marc Andreessen, Mark Zuckerberg, minimum viable product, Nate Silver, Network effects, new economy, paypal mafia, Peter Thiel, pets.com, profit motive, Ralph Waldo Emerson, Ray Kurzweil, self-driving car, shareholder value, Silicon Valley, Silicon Valley startup, Singularitarianism, software is eating the world, Steve Jobs, strong AI, Ted Kaczynski, Tesla Model S, uber lyft, Vilfredo Pareto, working poor

That leaves the fourth scenario, in which we create new technology to make a much better future. The most dramatic version of this outcome is called the Singularity, an attempt to name the imagined result of new technologies so powerful as to transcend the current limits of our understanding. Ray Kurzweil, the best-known Singularitarian, starts from Moore’s law and traces exponential growth trends in dozens of fields, confidently projecting a future of superhuman artificial intelligence. According to Kurzweil, “the Singularity is near,” it’s inevitable, and all we have to do is prepare ourselves to accept it. But no matter how many trends can be traced, the future won’t happen on its own.


pages: 798 words: 240,182

The Transhumanist Reader by Max More, Natasha Vita-More

23andMe, Any sufficiently advanced technology is indistinguishable from magic, artificial general intelligence, augmented reality, Bill Joy: nanobots, bioinformatics, brain emulation, Buckminster Fuller, cellular automata, clean water, cloud computing, cognitive bias, cognitive dissonance, combinatorial explosion, conceptual framework, Conway's Game of Life, cosmological principle, data acquisition, discovery of DNA, Douglas Engelbart, Drosophila, en.wikipedia.org, endogenous growth, experimental subject, Extropian, fault tolerance, Flynn Effect, Francis Fukuyama: the end of history, Frank Gehry, friendly AI, game design, germ theory of disease, hypertext link, impulse control, index fund, John von Neumann, joint-stock company, Kevin Kelly, Law of Accelerating Returns, life extension, lifelogging, Louis Pasteur, Menlo Park, meta-analysis, moral hazard, Network effects, Norbert Wiener, pattern recognition, Pepto Bismol, phenotype, positional goods, prediction markets, presumed consent, Ray Kurzweil, reversible computing, RFID, Ronald Reagan, scientific worldview, silicon-based life, Singularitarianism, social intelligence, stem cell, stochastic process, superintelligent machines, supply-chain management, supply-chain management software, technological singularity, Ted Nelson, telepresence, telepresence robot, telerobotics, the built environment, The Coming Technological Singularity, the scientific method, The Wisdom of Crowds, transaction costs, Turing machine, Turing test, Upton Sinclair, Vernor Vinge, Von Neumann architecture, Whole Earth Review, women in the workforce, zero-sum game

After all, both transhumanists and proponents of the technological singularity (i.e., singularitarians, as they sometimes call themselves) expect drastic changes in the future. Because the term has had wide appeal, it is now referred to simply as “the singularity.” Some transhumanists expect a singularity and most of those who expect a singularity are broadly transhumanist. But, while transhumanism is a broad worldview that anticipates using technology to overcome human limits, the singularity is a specific model (or set of models) of technological change and its trajectory into the future. To clearly separate specific singularitarian expectations from the philosophy of transhumanism requires first defining the former.

Hughes refers to enhancement as a “spiritual obligation,” he only fuels opponents’ misapprehensions of what constitutes a collective “good.”9 Likewise, unreflective prophecies of something like whole brain emulation can do more harm than “good.” Opponents to cognitive enhancement, reading the same technological tea leaves as Singularitarians, fear that virtually every aspect of “human” culture soon will be adversely influenced by integrated technological communication and control capabilities. From their viewpoint, compromising the “human condition” or “human nature” by surpassing “natural” limitations, violates a moral (should) code that they perceive as “good” for the species.10 To embrace or eschew active self-design differs very little from adherence to a particular religion or a particular worldview.


pages: 309 words: 54,839

Attack of the 50 Foot Blockchain: Bitcoin, Blockchain, Ethereum & Smart Contracts by David Gerard

altcoin, Amazon Web Services, augmented reality, Bernie Madoff, bitcoin, Bitcoin Ponzi scheme, blockchain, Blythe Masters, Bretton Woods, clean water, cloud computing, collateralized debt obligation, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, cryptocurrency, distributed ledger, Dogecoin, Ethereum, ethereum blockchain, Extropian, fiat currency, financial innovation, Firefox, Flash crash, Fractional reserve banking, functional programming, index fund, Internet Archive, Internet of things, Kickstarter, litecoin, M-Pesa, margin call, Network effects, peer-to-peer, Peter Thiel, pets.com, Ponzi scheme, Potemkin village, prediction markets, quantitative easing, RAND corporation, ransomware, Ray Kurzweil, Ross Ulbricht, Ruby on Rails, Satoshi Nakamoto, short selling, Silicon Valley, Silicon Valley ideology, Singularitarianism, slashdot, smart contracts, South Sea Bubble, tulip mania, Turing complete, Turing machine, WikiLeaks

Then an artificial intelligence takes over the world, rendering the preceding plot meaningless. To be produced and distributed worldwide through the S-DTV portal! They are so keen on their sci-fi TV show idea that they named their blockchain startup after it. It appears that they are in fact Singularitarians — fans of Ray Kurzweil’s non-musical-instrument ideas, like an artificial intelligence taking over the world this century — who came up with a way to propagandise their beliefs in Ethereum and the Singularity (and crank pseudoeconomics) through the medium of science fiction television, and decided a crypto asset offering was clearly the way to collect money to make the TV show to evangelise their cult.432 Summary Blockchains won’t solve your bad recording or publishing deal.


pages: 489 words: 148,885

Accelerando by Stross, Charles

business cycle, call centre, carbon-based life, cellular automata, cognitive dissonance, commoditize, Conway's Game of Life, dark matter, disinformation, dumpster diving, Extropian, finite state, Flynn Effect, glass ceiling, gravity well, John von Neumann, Kickstarter, knapsack problem, Kuiper Belt, Magellanic Cloud, mandelbrot fractal, market bubble, means of production, MITM: man-in-the-middle, orbital mechanics / astrodynamics, packet switching, performance metric, phenotype, planetary scale, Pluto: dwarf planet, reversible computing, Richard Stallman, SETI@home, Silicon Valley, Singularitarianism, slashdot, South China Sea, stem cell, technological singularity, telepresence, The Chicago School, theory of mind, Turing complete, Turing machine, Turing test, upwardly mobile, Vernor Vinge, Von Neumann architecture, web of trust, Y2K, zero-sum game

I ought to bring this up with Amber … "but your reputation won't suffer for being on this craft, will it?" he asks aloud. "I will be all right," Donna declares. The waiter comes over: "Mine will be a bottle of schneiderweisse," she adds. And then, without breaking step: "Do you believe in the singularity?" "Am I a singularitarian, do you mean?" asks Pierre, a fixed grin coming to his face. "Oh, no, no, no!" Donna waves him down, grins broadly, nods at Su Ang: "I do not mean it like that! Attend: What I meant to ask was whether you in the concept of a singularity believe, and if so, where it is?" "Is this intended for a public interview?"

Here he is, naked as the day he was born – newly re-created, in fact, released from the wake-experience-reset cycle of the temple of history – standing on the threshold of a posthuman civilization so outrageously rich and powerful that they can build mammal-friendly habitats that resemble works of art in the cryogenic depths of space. Only he's poor, this whole polity is poor, and it can't ever be anything else, in fact, because it's a dumping ground for merely posthuman also-rans, the singularitarian equivalent of australopithecines. In the brave new world of the Vile Offspring, they can't get ahead any more than a protohominid could hack it as a rocket scientist in Werner von Braun's day. They're born to be primitive, wallowing happily in the mud-bath of their own limited cognitive bandwidth.


pages: 332 words: 93,672

Life After Google: The Fall of Big Data and the Rise of the Blockchain Economy by George Gilder

23andMe, Airbnb, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, AltaVista, Amazon Web Services, Asilomar, augmented reality, Ben Horowitz, bitcoin, Bitcoin Ponzi scheme, blockchain, Bob Noyce, British Empire, Brownian motion, Burning Man, business process, butterfly effect, carbon footprint, cellular automata, Claude Shannon: information theory, Clayton Christensen, cloud computing, computer age, computer vision, crony capitalism, cross-subsidies, cryptocurrency, Danny Hillis, disintermediation, distributed ledger, don't be evil, Donald Knuth, Donald Trump, double entry bookkeeping, Elon Musk, Erik Brynjolfsson, Ethereum, ethereum blockchain, fault tolerance, fiat currency, Firefox, first square of the chessboard, first square of the chessboard / second half of the chessboard, floating exchange rates, Fractional reserve banking, game design, George Gilder, Google Earth, Google Glasses, Google Hangouts, index fund, inflation targeting, informal economy, Internet of things, Isaac Newton, iterative process, Jaron Lanier, Jeff Bezos, Joan Didion, John Markoff, John von Neumann, Julian Assange, Kevin Kelly, Law of Accelerating Returns, Marc Andreessen, Mark Zuckerberg, means of production, Menlo Park, Metcalfe’s law, Money creation, money: store of value / unit of account / medium of exchange, move fast and break things, move fast and break things, Network effects, new economy, Norbert Wiener, Oculus Rift, PageRank, pattern recognition, Paul Graham, peer-to-peer, Peter Thiel, Ponzi scheme, prediction markets, quantitative easing, random walk, ransomware, Ray Kurzweil, Renaissance Technologies, Robert Mercer, Robert Metcalfe, Ronald Coase, Ross Ulbricht, Ruby on Rails, Sand Hill Road, Satoshi Nakamoto, Search for Extraterrestrial Intelligence, self-driving car, sharing economy, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, Singularitarianism, Skype, smart contracts, Snapchat, software is eating the world, sorting algorithm, South Sea Bubble, speech recognition, Stephen Hawking, Steve Jobs, Steven Levy, Stewart Brand, stochastic process, telepresence, Tesla Model S, theory of mind, Tim Cook: Apple, transaction costs, tulip mania, Turing complete, Turing machine, Vernor Vinge, Von Neumann architecture, Watson beat the top human players on Jeopardy!, WikiLeaks, Y Combinator, zero-sum game

Also there was Facebook’s Yann LeCun, an innovator in deep-learning math and a protégé of Google’s Geoffrey Hinton. A tenured contingent consisted of the technologist Stuart Russell, the philosopher David Chalmers, the catastrophe theorist Nick Bostrom, the nanotech prophet Eric Drexler, the cosmologist Lawrence Krauss, the economist Erik Brynjolfsson, and the “Singularitarian” Vernor Vinge, along with scores of other celebrity scientists.1 They gathered at Asilomar preparing to alert the world to the dire threat posed by . . . well, by themselves—Silicon Valley. Their computer technology, advanced AI, and machine learning—acclaimed in hundreds of press releases as the Valley’s principal activity and hope for the future, with names such as TensorFlow, DeepMind, Machine Learning, Google Brain, and the Singularity—had gained such power and momentum that it was now deemed nothing less than a menace to mankind.


pages: 350 words: 98,077

Artificial Intelligence: A Guide for Thinking Humans by Melanie Mitchell

Ada Lovelace, AI winter, Amazon Mechanical Turk, Apple's 1984 Super Bowl advert, artificial general intelligence, autonomous vehicles, backpropagation, Bernie Sanders, Claude Shannon: information theory, cognitive dissonance, computer age, computer vision, dark matter, Douglas Hofstadter, Elon Musk, en.wikipedia.org, Gödel, Escher, Bach, I think there is a world market for maybe five computers, ImageNet competition, Jaron Lanier, job automation, John Markoff, John von Neumann, Kevin Kelly, Kickstarter, license plate recognition, Mark Zuckerberg, natural language processing, Norbert Wiener, ought to be enough for anybody, pattern recognition, performance metric, RAND corporation, Ray Kurzweil, recommendation engine, ride hailing / ride sharing, Rodney Brooks, self-driving car, sentiment analysis, Silicon Valley, Singularitarianism, Skype, speech recognition, Stephen Hawking, Steve Jobs, Steve Wozniak, Steven Pinker, strong AI, superintelligent machines, tail risk, theory of mind, There's no reason for any individual to have a computer in his home - Ken Olsen, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!

But more important, computer software has not shown the same exponential progress; it would be hard to argue that today’s software is exponentially more sophisticated, or brain-like, than the software of fifty years ago, or that such a trend has ever existed. Kurzweil’s claims about exponential trends in neuroscience and virtual reality are also widely disputed. But as Singularitarians have pointed out, sometimes it’s hard to see an exponential trend if you’re in the midst of it. If you look at an exponential curve like the ones in figure 5, Kurzweil and his adherents imagine that we’re at that point where the curve is increasing slowly, and it looks like incremental progress to us, but it’s deceptive: the growth is about to explode.


pages: 345 words: 104,404

Pandora's Brain by Calum Chace

AI winter, Any sufficiently advanced technology is indistinguishable from magic, artificial general intelligence, brain emulation, Extropian, friendly AI, hive mind, lateral thinking, mega-rich, Ray Kurzweil, self-driving car, Silicon Valley, Singularitarianism, Skype, speech recognition, stealth mode startup, Stephen Hawking, strong AI, technological singularity, theory of mind, Turing test, Wall-E

The waitress started clearing the window table and as the door swung closed the noise level dropped several decibels. ‘Tell me why this is irritating you.’ Carl was looking down. He flicked a tiny ball of paper along the table, away from them. ‘Because it’s so stupid!’ he said, sulkily. ‘It’s intelligent design for smart people. These Singularitarians, Transhumanists, Extropians, whatever they call themselves, they are all just guilty of a massive amount of wishful thinking.’ Carl leaned back, paused, and smiled sheepishly. ‘End of rant.’ Matt smiled. ‘No, it was a good rant. You give good rant, Carl. And as it happens I partly agree with you.


When Computers Can Think: The Artificial Intelligence Singularity by Anthony Berglas, William Black, Samantha Thalind, Max Scratchmann, Michelle Estes

3D printing, AI winter, anthropic principle, artificial general intelligence, Asilomar, augmented reality, Automated Insights, autonomous vehicles, availability heuristic, backpropagation, blue-collar work, brain emulation, call centre, cognitive bias, combinatorial explosion, computer vision, create, read, update, delete, cuban missile crisis, David Attenborough, disinformation, Elon Musk, en.wikipedia.org, epigenetics, Ernest Rutherford, factory automation, feminist movement, finite state, Flynn Effect, friendly AI, general-purpose programming language, Google Glasses, Google X / Alphabet X, Gödel, Escher, Bach, industrial robot, Isaac Newton, job automation, John von Neumann, Law of Accelerating Returns, license plate recognition, Mahatma Gandhi, mandelbrot fractal, natural language processing, Parkinson's law, patent troll, patient HM, pattern recognition, phenotype, ransomware, Ray Kurzweil, self-driving car, semantic web, Silicon Valley, Singularitarianism, Skype, sorting algorithm, speech recognition, statistical model, stem cell, Stephen Hawking, Stuxnet, superintelligent machines, technological singularity, Thomas Malthus, Turing machine, Turing test, uranium enrichment, Von Neumann architecture, Watson beat the top human players on Jeopardy!, wikimedia commons, zero day

Muehlhauser notes that due to the availability heuristic, your brain will tell you that an AGI wiping out mankind is incredibly unlikely because you’ve never encountered this before. He also notes that extraordinary claims require extraordinary evidence. One point Muehlhauser refutes is that people that write about AGI are merely atheists whose fear of nihilism make them seek a moral purpose to save the world and fall for the seduction of Singularitarianism. Del Monte 2014 The Artificial Intelligence Revolution Fair Use The Singularity is coming! If we do not control it we will soon be extinct. Del Monte provides yet another wake up call to think carefully about the future. How do we control the intelligent explosion? Can we control it? The book covers arguments concerning consciousness and robot ethics, and thoughts about whether we can avoid the intelligence explosion.


pages: 394 words: 118,929

Dreaming in Code: Two Dozen Programmers, Three Years, 4,732 Bugs, and One Quest for Transcendent Software by Scott Rosenberg

A Pattern Language, Benevolent Dictator For Life (BDFL), Berlin Wall, Bill Atkinson, c2.com, call centre, collaborative editing, conceptual framework, continuous integration, Donald Knuth, Douglas Engelbart, Douglas Engelbart, Douglas Hofstadter, Dynabook, en.wikipedia.org, Firefox, Ford paid five dollars a day, Francis Fukuyama: the end of history, functional programming, George Santayana, Grace Hopper, Guido van Rossum, Gödel, Escher, Bach, Howard Rheingold, HyperCard, index card, Internet Archive, inventory management, Jaron Lanier, John Markoff, John von Neumann, knowledge worker, Larry Wall, life extension, Loma Prieta earthquake, Menlo Park, Merlin Mann, Mitch Kapor, new economy, Nicholas Carr, Norbert Wiener, pattern recognition, Paul Graham, Potemkin village, RAND corporation, Ray Kurzweil, Richard Stallman, Ronald Reagan, Ruby on Rails, semantic web, side project, Silicon Valley, Singularitarianism, slashdot, software studies, source of truth, South of Market, San Francisco, speech recognition, stealth mode startup, stem cell, Stephen Hawking, Steve Jobs, Stewart Brand, Ted Nelson, Therac-25, thinkpad, Turing test, VA Linux, Vannevar Bush, Vernor Vinge, web application, Whole Earth Catalog, Y2K

When it rolls out, sometime in the late 2020s, an artificial intelligence’s passing of the Turing Test will be a mere footnote to this singularity’s impact—which will be, he says, to generate a “radical transformation of the reality of human experience” by the 2040s. Utopian? Not really. Kurzweil is careful to lay out the downsides of his vision. Apocalpytic? Who knows—the Singularity’s consequences are, by definition, inconceivable to us pre-Singularitarians. Big? You bet. It’s easy to make fun of the wackier dimension of Kurzweil’s digital eschatology. His personal program of life extension via a diet of 220 pills per day—to pickle his fifty-something wetware until post-Singularity medical breakthroughs open the door to full immortality—sounds more like something out of a late-night commercial pitch than a serious scientist’s choice.


pages: 1,737 words: 491,616

Rationality: From AI to Zombies by Eliezer Yudkowsky

Albert Einstein, Alfred Russel Wallace, anthropic principle, anti-pattern, anti-work, Arthur Eddington, artificial general intelligence, availability heuristic, backpropagation, Bayesian statistics, Berlin Wall, Build a better mousetrap, Cass Sunstein, cellular automata, cognitive bias, cognitive dissonance, correlation does not imply causation, cosmological constant, creative destruction, Daniel Kahneman / Amos Tversky, dematerialisation, different worldview, discovery of DNA, disinformation, Douglas Hofstadter, Drosophila, effective altruism, experimental subject, Extropian, friendly AI, fundamental attribution error, Gödel, Escher, Bach, hindsight bias, index card, index fund, Isaac Newton, John Conway, John von Neumann, Long Term Capital Management, Louis Pasteur, mental accounting, meta-analysis, money market fund, Monty Hall problem, Nash equilibrium, Necker cube, NP-complete, P = NP, pattern recognition, Paul Graham, Peter Thiel, Pierre-Simon Laplace, placebo effect, planetary scale, prediction markets, random walk, Ray Kurzweil, reversible computing, Richard Feynman, risk tolerance, Rubik’s Cube, Saturday Night Live, Schrödinger's Cat, scientific mainstream, scientific worldview, sensible shoes, Silicon Valley, Silicon Valley startup, Singularitarianism, Solar eclipse in 1919, speech recognition, statistical model, Steven Pinker, strong AI, sunk-cost fallacy, technological singularity, The Bell Curve by Richard Herrnstein and Charles Murray, the map is not the territory, the scientific method, Turing complete, Turing machine, ultimatum game, X Prize, Y Combinator, zero-sum game

If you really, genuinely can’t figure out whether a group is a “cult,” then you’ll just have to choose under conditions of uncertainty. That’s what decision theory is all about. Problem five: Lack of strategic thinking. I know people who are cautious around Singularitarianism, and they’re also cautious around political parties and mainstream religions. Cautious, not nervous or defensive. These people can see at a glance that Singularitarianism is obviously not a full-blown cult with sleep deprivation etc. But they worry that Singularitarianism will become a cult, because of risk factors like turning the concept of a powerful AI into a Super Happy Agent (an agent defined primarily by agreeing with any nice thing said about it).