Thomas Bayes

61 results back to index


pages: 561 words: 120,899

The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant From Two Centuries of Controversy by Sharon Bertsch McGrayne

Bayesian statistics, bioinformatics, British Empire, Claude Shannon: information theory, Daniel Kahneman / Amos Tversky, double helix, Edmond Halley, Fellow of the Royal Society, full text search, Henri Poincaré, Isaac Newton, Johannes Kepler, John Markoff, John Nash: game theory, John von Neumann, linear programming, longitudinal study, meta analysis, meta-analysis, Nate Silver, p-value, Pierre-Simon Laplace, placebo effect, prediction markets, RAND corporation, recommendation engine, Renaissance Technologies, Richard Feynman, Richard Feynman: Challenger O-ring, Robert Mercer, Ronald Reagan, speech recognition, statistical model, stochastic process, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, traveling salesman, Turing machine, Turing test, uranium enrichment, Yom Kippur War

A letter from the late Reverend Mr. Thomas Bayes, F.R.S., to John Canton, M.A. and F.R.S. Author(s): Mr. Bayes and Mr. Price. Philosophical Transactions (1683–1775) (53) 370–418. Royal Society. The original Bayes–Price article. Bebb, ED. (1935) Nonconformity and Social and Economic Life 1660–1800. London: Epworth Press. Bellhouse, David R. (2002) On some recently discovered manuscripts of Thomas Bayes. Historia Mathematica (29) 383–94. ———. (2007a) The Reverend Thomas Bayes, FRS: A biography to celebrate the tercentenary of his birth. Statistical Science (19:1) 3–43. With Dale (2003) the main source for Bayes’ life. ———. (2007b) Lord Stanhope’s papers on the Doctrine of Chances. Historia Mathematica (34) 173–86. Bru, Bernard. (1987) Preface in Thomas Bayes. Essai en vue de résoudre un problème de la doctrine des chances, trans. and ed., J-P Cléro.

Cone, Carl B. (1952) Torchbearer of Freedom: The Influence of Richard Price on Eighteenth-Century Thought. University of Kentucky Press. Dale, Andrew I. (1988) On Bayes’ theorem and the inverse Bernoulli theorem. Historia Mathematica (15) 348–60. ———. (1991) Thomas Bayes’s work on infinite series. Historia Mathematica (18) 312–27. ———. (1999) A History of Inverse Probability from Thomas Bayes to Karl Pearson. 2d ed. Springer. One of the foundational works in the history of probability. ———. (2003) Most Honourable Remembrance: The Life and Work of Thomas Bayes. Springer. With Bellhouse, the main source for Bayes’ life. Daston, Lorraine. (1988) Classical Probability in the Enlightenment. Princeton University Press. Deming WE, ed. (1940) Facsimiles of Two Papers by Bayes, With Commentaries by W. E.

Bayes was interred on April 15, which is often called the date of his death. The degraded condition of his vault may have contributed to the confusion. Second, the often-reproduced portrait of Thomas Bayes is almost assuredly of someone else named “T. Bayes.” The sketch first appeared in 1936 in History of Life Insurance in its Formative Years by Terence O’Donnell. However, the picture’s caption on page 335 says it is of “Rev. T. Bayes, Improver of the Columnar Method developed by Barrett,” and Barrett did not develop his method until 1810, a half-century after the death of “our” Rev. Thomas Bayes. Bellhouse (2004) first noticed that the portrait’s hairstyle is anachronistic. Sharon North, curator of Textiles and Fashion at the Victoria and Albert Museum, London, agrees: “The hairstyle in this portrait looks very 20th century. . . .


pages: 283 words: 81,376

The Doomsday Calculation: How an Equation That Predicts the Future Is Transforming Everything We Know About Life and the Universe by William Poundstone

Albert Einstein, anthropic principle, Any sufficiently advanced technology is indistinguishable from magic, Arthur Eddington, Bayesian statistics, Benoit Mandelbrot, Berlin Wall, bitcoin, Black Swan, conceptual framework, cosmic microwave background, cosmological constant, cosmological principle, cuban missile crisis, dark matter, digital map, discounted cash flows, Donald Trump, Doomsday Clock, double helix, Elon Musk, Gerolamo Cardano, index fund, Isaac Newton, Jaron Lanier, Jeff Bezos, John Markoff, John von Neumann, mandelbrot fractal, Mark Zuckerberg, Mars Rover, Peter Thiel, Pierre-Simon Laplace, probability theory / Blaise Pascal / Pierre de Fermat, RAND corporation, random walk, Richard Feynman, ride hailing / ride sharing, Rodney Brooks, Ronald Reagan, Ronald Reagan: Tear down this wall, Sam Altman, Schrödinger's Cat, Search for Extraterrestrial Intelligence, self-driving car, Silicon Valley, Skype, Stanislav Petrov, Stephen Hawking, strong AI, Thomas Bayes, Thomas Malthus, time value of money, Turing test

By Forster’s time, the faded resort was being pegged as an emblem of ossified British conservatism. Since the 1940s, “Disgusted of Tunbridge Wells” has been a facetious pseudonym for letters to the editor expressing stodgy views. Tunbridge Wells was nonetheless the birthplace of one of the contemporary world’s most disruptive ideas. Not many traces remain of the town’s onetime minister Thomas Bayes (1701–1761). The Bayes family had made its fortune several generations earlier, in the cutlery business of Sheffield. Thomas Bayes studied theology and logic at the University of Edinburgh. After several years in London, he moved to Tunbridge Wells in 1733 or 1734 and became minister of Mount Sion Chapel. Bayes was a Presbyterian nonconformist, opposing the Church of England and the Book of Common Prayer on grounds vague to nearly all of today’s Presbyterians.

This bomb broke apart, and the fragments fell into a swampy area with enough water to soften the impact and spare the conventional explosives. Bomb disposal expert Lieutenant Jack ReVelle was called in to find the pieces. “Until my death,” ReVelle said, “I will never forget hearing my sergeant say, ‘Lieutenant, we found the arm/safe switch.’ And I said, ‘Great.’ He said, ‘Not great. It’s on “arm.”’” “You’re the Product” Thomas Bayes, the nonconformist minister of Tunbridge Wells, England, drew his last breath on April 17, 1761. For reasons not clear he left his life’s greatest achievement filed away, unpublished and unread. It was another mathematically inclined minister, Richard Price, who found Bayes’s manuscript after his death and recognized its importance. Price counted among his acquaintances a notorious group: the American revolutionaries Thomas Paine, Thomas Jefferson, and Benjamin Franklin, as well as Mary Wollstonecraft, the feminist who married an anarchist and gave birth to the author of Frankenstein.

.… Near them, on the sand, Half sunk a shattered visage lies, whose frown, And wrinkled lip, and sneer of cold command, Tell that its sculptor well those passions read Which yet survive, stamped on these lifeless things, The hand that mocked them and the heart that fed; And on the pedestal these words appear: My name is Ozymandias, King of Kings; Look on my Works, ye Mighty, and despair! Nothing beside remains. Round the decay Of that colossal Wreck, boundless and bare The lone and level sands stretch far away.” This is the sonnet “Ozymandias” (1818) by Romantic poet Percy Bysshe Shelley, husband of Frankenstein author Mary Shelley, daughter of feminist Mary Wollstonecraft, friend of minister Richard Price, promoter of the intellectual property of Thomas Bayes. The theme of “Ozymandias” is that glory is fleeting. Nothing lasts. In the summer of 1969, J. Richard Gott III celebrated his Harvard graduation with a tour of Europe. He visited the supreme monument of Cold War anxiety, the Berlin Wall. Standing in the shadow of the landmark, he contemplated its history and future. Would this symbol of totalitarian power one day lie in ruins? This was a matter discussed by diplomats, historians, op-ed writers, TV pundits, and spy novelists.


pages: 52 words: 16,113

The Laws of Medicine: Field Notes From an Uncertain Science by Siddhartha Mukherjee

Atul Gawande, cognitive dissonance, Johannes Kepler, medical residency, randomized controlled trial, retrograde motion, stem cell, Thomas Bayes

Thank you for downloading this TED Books eBook. * * * Join our mailing list and get updates on new releases, deals, bonus content and other great books from TED Books and Simon & Schuster. CLICK HERE TO SIGN UP or visit us online to sign up at eBookNews.SimonandSchuster.com To Thomas Bayes (1702–1761), who saw uncertainty with such certainty “Are you planning to follow a career in Magical Laws, Miss Granger?” asked Scrimgeour. “No, I’m not,” retorted Hermione. “I’m hoping to do some good in the world!” J. K. Rowling The learned men of former ages employed a great part of their time and thoughts searching out the hidden causes of distemper, were curious in imagining the secret workmanship of nature and . . . putting all these fancies together, fashioned to themselves systems and hypotheses [that] diverted their enquiries from the true and advantageous knowledge of things.

It applies not only to medicine but to any other discipline that is predicated on predictions: economics or banking, gambling or astrology. The core logic holds true whether you are trying to forecast tomorrow’s weather or seeking to predict rises and falls in the stock market. It is a universal feature of all tests. .... The man responsible for this strange and illuminating idea was neither a doctor nor a scientist by trade. Born in Hertfordshire in 1702, Thomas Bayes was a clergyman and philosopher who served as the minister at the chapel in Tunbridge Wells, near London. He published only two significant papers in his lifetime—the first, a defense of God, and the second, a defense of Newton’s theory of calculus (it was a sign of the times that in 1732, a clergyman found no cognitive dissonance between these two efforts). His best-known work—on probability theory—was not published during his lifetime and was only rediscovered decades after his death.


The Book of Why: The New Science of Cause and Effect by Judea Pearl, Dana Mackenzie

affirmative action, Albert Einstein, Asilomar, Bayesian statistics, computer age, computer vision, correlation coefficient, correlation does not imply causation, Daniel Kahneman / Amos Tversky, Edmond Halley, Elon Musk, en.wikipedia.org, experimental subject, Isaac Newton, iterative process, John Snow's cholera map, Loebner Prize, loose coupling, Louis Pasteur, Menlo Park, pattern recognition, Paul Erdős, personalized medicine, Pierre-Simon Laplace, placebo effect, prisoner's dilemma, probability theory / Blaise Pascal / Pierre de Fermat, randomized controlled trial, selection bias, self-driving car, Silicon Valley, speech recognition, statistical model, Stephen Hawking, Steve Jobs, strong AI, The Design of Experiments, the scientific method, Thomas Bayes, Turing test

However, if you ever want to ask a rung-two or rung-three query about your Bayesian network, you must draw it with scrupulous attention to causality. REVEREND BAYES AND THE PROBLEM OF INVERSE PROBABILITY Thomas Bayes, after whom I named the networks in 1985, never dreamed that a formula he derived in the 1750s would one day be used to identify disaster victims. He was concerned only with the probabilities of two events, one (the hypothesis) occurring before the other (the evidence). Nevertheless, causality was very much on his mind. In fact, causal aspirations were the driving force behind his analysis of “inverse probability.” A Presbyterian minister who lived from 1702 to 1761, the Reverend Thomas Bayes appears to have been a mathematics geek. As a dissenter from the Church of England, he could not study at Oxford or Cambridge and was educated instead at the University of Scotland, where he likely picked up quite a bit of math.

The miracle Hume had in mind was, of course, the resurrection of Christ, although he was smart enough not to say so. (Twenty years earlier, theologian Thomas Woolston had gone to prison for blasphemy for writing such things.) Hume’s main point was that inherently fallible evidence cannot overrule a proposition with the force of natural law, such as “Dead people stay dead.” FIGURE 3.1. Title page of the journal where Thomas Bayes’s posthumous article on inverse probability was published and the first page of Richard Price’s introduction. For Bayes, this assertion provoked a natural, one might say Holmesian question: How much evidence would it take to convince us that something we consider improbable has actually happened? When does a hypothesis cross the line from impossibility to improbability and even to probability or virtual certainty?

What is the probability that it will stop within x feet of the left-hand end of the table? If we know the length of the table and it is perfectly smooth and flat, this is a very easy question (Figure 3.2, top). For example, on a twelve-foot snooker table, the probability of the ball stopping within a foot of the end would be 1/12. On an eight-foot billiard table, the probability would be 1/8. FIGURE 3.2. Thomas Bayes’s pool table example. In the first version, a forward-probability question, we know the length of the table and want to calculate the probability of the ball stopping within x feet of the end. In the second, an inverse-probability question, we observe that the ball stopped x feet from the end and want to estimate the likelihood that the table’s length is L. (Source: Drawing by Maayan Harel.) Our intuitive understanding of the physics tells us that, in general, if the length of the table is L feet, the probability of the ball’s stopping within x feet of the end is x/L.


pages: 266 words: 86,324

The Drunkard's Walk: How Randomness Rules Our Lives by Leonard Mlodinow

Albert Einstein, Alfred Russel Wallace, Antoine Gombaud: Chevalier de Méré, Atul Gawande, Brownian motion, butterfly effect, correlation coefficient, Daniel Kahneman / Amos Tversky, Donald Trump, feminist movement, forensic accounting, Gerolamo Cardano, Henri Poincaré, index fund, Isaac Newton, law of one price, pattern recognition, Paul Erdős, Pepto Bismol, probability theory / Blaise Pascal / Pierre de Fermat, RAND corporation, random walk, Richard Feynman, Ronald Reagan, Stephen Hawking, Steve Jobs, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Thomas Bayes, V2 rocket, Watson beat the top human players on Jeopardy!

The experiment was still in progress, he reported, and now he was suing his former employer, who had produced a psychiatrist willing to testify that he suffered from paranoia. One of the paranoid delusions the former employer’s psychiatrist pointed to was the student’s alleged invention of a fictitious eighteenth-century minister. In particular, the psychiatrist scoffed at the student’s claim that this minister was an amateur mathematician who had created in his spare moments a bizarre theory of probability. The minister’s name, according to the student, was Thomas Bayes. His theory, the student asserted, described how to assess the chances that some event would occur if some other event also occurred. What are the chances that a particular student would be the subject of a vast secret conspiracy of experimental psychologists? Admittedly not huge. But what if one’s wife speaks one’s thoughts before one can utter them and co-workers foretell your professional fate over drinks in casual conversation?

And he presented the court with a mumbo jumbo of formulas and calculations regarding his hypothesis, concluding that the additional evidence meant that the probability was 999,999 in 1 million that he was right about the conspiracy. The enemy psychiatrist claimed that this mathematician-minister and his theory were figments of the student’s schizophrenic imagination. The student asked the professor to help him refute that claim. The professor agreed. He had good reason, for Thomas Bayes, born in London in 1701, really was a minister, with a parish at Tunbridge Wells. He died in 1761 and was buried in a park in London called Bunhill Fields, in the same grave as his father, Joshua, also a minister. And he indeed did invent a theory of “conditional probability” to show how the theory of probability can be extended from independent events to events whose outcomes are connected.

The professor supplied a deposition explaining Bayes’s existence and his theory, though not supporting the specific and dubious calculations that his former student claimed proved his sanity. The sad part of this story is not just the middle-aged schizophrenic himself, but the medical and legal team on the other side. It is unfortunate that some people suffer from schizophrenia, but even though drugs can help to mediate the illness, they cannot battle ignorance. And ignorance of the ideas of Thomas Bayes, as we shall see, resides at the heart of many serious mistakes in both medical diagnosis and legal judgment. It is an ignorance that is rarely addressed during a doctor’s or a lawyer’s professional training. We also make Bayesian judgments in our daily lives. A film tells the story of an attorney who has a great job, a charming wife, and a wonderful family. He loves his wife and daughter, but still he feels that something is missing in his life.


pages: 523 words: 143,139

Algorithms to Live By: The Computer Science of Human Decisions by Brian Christian, Tom Griffiths

4chan, Ada Lovelace, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, algorithmic trading, anthropic principle, asset allocation, autonomous vehicles, Bayesian statistics, Berlin Wall, Bill Duvall, bitcoin, Community Supported Agriculture, complexity theory, constrained optimization, cosmological principle, cryptocurrency, Danny Hillis, David Heinemeier Hansson, delayed gratification, dematerialisation, diversification, Donald Knuth, double helix, Elon Musk, fault tolerance, Fellow of the Royal Society, Firefox, first-price auction, Flash crash, Frederick Winslow Taylor, George Akerlof, global supply chain, Google Chrome, Henri Poincaré, information retrieval, Internet Archive, Jeff Bezos, Johannes Kepler, John Nash: game theory, John von Neumann, Kickstarter, knapsack problem, Lao Tzu, Leonard Kleinrock, linear programming, martingale, Nash equilibrium, natural language processing, NP-complete, P = NP, packet switching, Pierre-Simon Laplace, prediction markets, race to the bottom, RAND corporation, RFC: Request For Comment, Robert X Cringely, Sam Altman, sealed-bid auction, second-price auction, self-driving car, Silicon Valley, Skype, sorting algorithm, spectrum auction, Stanford marshmallow experiment, Steve Jobs, stochastic process, Thomas Bayes, Thomas Malthus, traveling salesman, Turing machine, urban planning, Vickrey auction, Vilfredo Pareto, Walter Mischel, Y Combinator, zero-sum game

The story begins in eighteenth-century England, in a domain of inquiry irresistible to great mathematical minds of the time, even those of the clergy: gambling. Reasoning Backward with the Reverend Bayes If we be, therefore, engaged by arguments to put trust in past experience, and make it the standard of our future judgement, these arguments must be probable only. —DAVID HUME More than 250 years ago, the question of making predictions from small data weighed heavily on the mind of the Reverend Thomas Bayes, a Presbyterian minister in the charming spa town of Tunbridge Wells, England. If we buy ten tickets for a new and unfamiliar raffle, Bayes imagined, and five of them win prizes, then it seems relatively easy to estimate the raffle’s chances of a win: 5/10, or 50%. But what if instead we buy a single ticket and it wins a prize? Do we really imagine the probability of winning to be 1/1, or 100%?

“The Unreasonable Effectiveness of Data”: The talk was derived from Halevy, Norvig, and Pereira, “The Unreasonable Effectiveness of Data.” “these arguments must be probable only”: An Enquiry Concerning Human Understanding, §IV, “Sceptical Doubts Concerning the Operations of the Understanding.” Bayes’s own history: Our brief biography draws on Dale, A History of Inverse Probability, and Bellhouse, “The Reverend Thomas Bayes.” in either 1746, ’47, ’48, or ’49: Bayes’s legendary paper, undated, had been filed between a pair of papers dated 1746 and 1749. See, e.g., McGrayne, The Theory That Would Not Die. defense of Newton’s newfangled “calculus”: An Introduction to the Doctrine of fluxions, and Defence of the Mathematicians against the Objections of the Author of the analyst, so far as they are assigned to affect their general methods of Reasoning.

Shedler. “An Anomaly in Space-Time Characteristics of Certain Programs Running in a Paging Machine.” Communications of the ACM 12, no. 6 (1969): 349–353. Belew, Richard K. Finding Out About: A Cognitive Perspective on Search Engine Technology and the WWW. Cambridge, UK: Cambridge University Press, 2000. Bell, Aubrey F. G. In Portugal. New York: John Lane, 1912. Bellhouse, David R. “The Reverend Thomas Bayes, FRS: A Biography to Celebrate the Tercentenary of His Birth.” Statistical Science 19 (2004): 3–43. Bellman, Richard. Dynamic Programming. Princeton, NJ: Princeton University Press, 1957. ______. “A Problem in the Sequential Design of Experiments.” Sankhyā: The Indian Journal of Statistics 16 (1956): 221–229. Bellows, Meghan L., and J. D. Luc Peterson. “Finding an Optimal Seating Chart.” Annals of Improbable Research (2012).


pages: 829 words: 186,976

The Signal and the Noise: Why So Many Predictions Fail-But Some Don't by Nate Silver

"Robert Solow", airport security, availability heuristic, Bayesian statistics, Benoit Mandelbrot, Berlin Wall, Bernie Madoff, big-box store, Black Swan, Broken windows theory, business cycle, buy and hold, Carmen Reinhart, Claude Shannon: information theory, Climategate, Climatic Research Unit, cognitive dissonance, collapse of Lehman Brothers, collateralized debt obligation, complexity theory, computer age, correlation does not imply causation, Credit Default Swap, credit default swaps / collateralized debt obligations, cuban missile crisis, Daniel Kahneman / Amos Tversky, diversification, Donald Trump, Edmond Halley, Edward Lorenz: Chaos theory, en.wikipedia.org, equity premium, Eugene Fama: efficient market hypothesis, everywhere but in the productivity statistics, fear of failure, Fellow of the Royal Society, Freestyle chess, fudge factor, George Akerlof, global pandemic, haute cuisine, Henri Poincaré, high batting average, housing crisis, income per capita, index fund, information asymmetry, Intergovernmental Panel on Climate Change (IPCC), Internet Archive, invention of the printing press, invisible hand, Isaac Newton, James Watt: steam engine, John Nash: game theory, John von Neumann, Kenneth Rogoff, knowledge economy, Laplace demon, locking in a profit, Loma Prieta earthquake, market bubble, Mikhail Gorbachev, Moneyball by Michael Lewis explains big data, Monroe Doctrine, mortgage debt, Nate Silver, negative equity, new economy, Norbert Wiener, PageRank, pattern recognition, pets.com, Pierre-Simon Laplace, prediction markets, Productivity paradox, random walk, Richard Thaler, Robert Shiller, Robert Shiller, Rodney Brooks, Ronald Reagan, Saturday Night Live, savings glut, security theater, short selling, Skype, statistical model, Steven Pinker, The Great Moderation, The Market for Lemons, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, too big to fail, transaction costs, transfer pricing, University of East Anglia, Watson beat the top human players on Jeopardy!, wikimedia commons

Finding patterns is easy in any kind of data-rich environment; that’s what mediocre gamblers do. The key is in determining whether the patterns represent noise or signal. But although there isn’t any one particular key to why Voulgaris might or might not bet on a given game, there is a particular type of thought process that helps govern his decisions. It is called Bayesian reasoning. The Improbable Legacy of Thomas Bayes Thomas Bayes was an English minister who was probably born in 1701—although it may have been 1702. Very little is certain about Bayes’s life, even though he lent his name to an entire branch of statistics and perhaps its most famous theorem. It is not even clear that anybody knows what Bayes looked like; the portrait of him that is commonly used in encyclopedia articles may have been misattributed.19 What is in relatively little dispute is that Bayes was born into a wealthy family, possibly in the southeastern English county of Hertfordshire.

On average, a team will go either over or under the total five games in a row about five times per season. That works out to 150 such streaks per season between the thirty NBA teams combined. 19. D. R. Bellhouse, “The Reverend Thomas Bayes FRS: A Biography to Celebrate the Tercentenary of His Birth,” Statistical Science, 19, 1, pp. 3–43; 2004. http://www2.isye.gatech.edu/~brani/isyebayes/bank/bayesbiog.pdf. 20. Bayes may also have been an Arian, meaning someone who followed the teachings of the early Christian leader Arias and who regarded Jesus Christ as the divine son of God rather than (as most Christians then and now believe) a direct manifestation of God. 21. Thomas Bayes, “Divine Benevolence: Or an Attempt to Prove That the Principal End of the Divine Providence and Government Is the Happiness of His Creatures.” http://archive.org/details/DivineBenevolenceOrAnAttemptToProveThatThe. 22.

There are many reasons for it—some having to do with our psychological biases, some having to do with common methodological errors, and some having to do with misaligned incentives. Close to the root of the problem, however, is a flawed type of statistical thinking that these researchers are applying. FIGURE 8-6: A GRAPHICAL REPRESENTATION OF FALSE POSITIVES When Statistics Backtracked from Bayes Perhaps the chief intellectual rival to Thomas Bayes—although he was born in 1890, almost 120 years after Bayes’s death—was an English statistician and biologist named Ronald Aylmer (R. A.) Fisher. Fisher was a much more colorful character than Bayes, almost in the English intellectual tradition of Christopher Hitchens. He was handsome but a slovenly dresser,42 always smoking his pipe or his cigarettes, constantly picking fights with his real and imagined rivals.


pages: 442 words: 94,734

The Art of Statistics: Learning From Data by David Spiegelhalter

Antoine Gombaud: Chevalier de Méré, Bayesian statistics, Carmen Reinhart, complexity theory, computer vision, correlation coefficient, correlation does not imply causation, dark matter, Edmond Halley, Estimating the Reproducibility of Psychological Science, Hans Rosling, Kenneth Rogoff, meta analysis, meta-analysis, Nate Silver, Netflix Prize, p-value, placebo effect, probability theory / Blaise Pascal / Pierre de Fermat, publication bias, randomized controlled trial, recommendation engine, replication crisis, self-driving car, speech recognition, statistical model, The Design of Experiments, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Malthus

We have already met the competing ideas of Fisher and Neyman–Pearson, and it is time to explore a third, Bayesian, approach to inference. This has only come to prominence in the last fifty years, but its basic principles go back somewhat further, in fact to the Reverend Thomas Bayes, a Nonconformist minister turned probability theorist and philosopher from Tunbridge Wells, who died in 1761.fn1 The good news is that the Bayesian approach opens fine new possibilities for making the most of complex data. The bad news is that it means putting aside almost everything you may have learned in this book and elsewhere about estimation, confidence intervals, P-values, hypothesis testing, and so on. What Is the Bayesian Approach? Thomas Bayes’ first great contribution was to use probability as an expression of our lack of knowledge about the world or, equivalently, our ignorance about what is currently going on.

Expected frequencies make Bayesian analysis reasonably straightforward for simple situations that involve only two hypotheses, say about whether someone does or does not have a disease, or has or has not committed an offence. However, things get trickier when we want to apply the same ideas to drawing inferences about unknown quantities that might take on a range of values, such as parameters in statistical models. The Reverend Thomas Bayes’ original paper in 1763 set out to answer a very basic question of this nature: given something has happened or not happened on a known number of similar occasions, what probability should we give to it happening next time?fn4 For example, if a thumbtack has been flipped 20 times and it has come down point-up 15 times and point-down 5 times, what is the probability of it landing point-down next time?


pages: 415 words: 125,089

Against the Gods: The Remarkable Story of Risk by Peter L. Bernstein

"Robert Solow", Albert Einstein, Alvin Roth, Andrew Wiles, Antoine Gombaud: Chevalier de Méré, Bayesian statistics, Big bang: deregulation of the City of London, Bretton Woods, business cycle, buttonwood tree, buy and hold, capital asset pricing model, cognitive dissonance, computerized trading, Daniel Kahneman / Amos Tversky, diversified portfolio, double entry bookkeeping, Edmond Halley, Edward Lloyd's coffeehouse, endowment effect, experimental economics, fear of failure, Fellow of the Royal Society, Fermat's Last Theorem, financial deregulation, financial innovation, full employment, index fund, invention of movable type, Isaac Newton, John Nash: game theory, John von Neumann, Kenneth Arrow, linear programming, loss aversion, Louis Bachelier, mental accounting, moral hazard, Myron Scholes, Nash equilibrium, Norman Macrae, Paul Samuelson, Philip Mirowski, probability theory / Blaise Pascal / Pierre de Fermat, random walk, Richard Thaler, Robert Shiller, Robert Shiller, spectrum auction, statistical model, stocks for the long run, The Bell Curve by Richard Herrnstein and Charles Murray, The Wealth of Nations by Adam Smith, Thomas Bayes, trade route, transaction costs, tulip mania, Vanguard fund, zero-sum game

With that innocent-sounding assertion, Bernoulli explained why King Midas was an unhappy man, why people tend to be risk-averse, and why prices must fall if customers are to be persuaded to buy more. Bernoulli's statement stood as the dominant paradigm of rational behavior for the next 250 years and laid the groundwork for modern principles of investment management. Almost exactly one hundred years after the collaboration between Pascal and Fermat, a dissident English minister named Thomas Bayes made a striking advance in statistics by demonstrating how to make better-informed decisions by mathematically blending new information into old information. Bayes's theorem focuses on the frequent occasions when we have sound intuitive judgments about the probability of some event and want to understand how to alter those judgments as actual events unfold. All the tools we use today in risk management and in the analysis of decisions and choice, from the strict rationality of game theory to the challenges of chaos theory, stem from the developments that took place between 1654 and 1760, with only two exceptions: In 1875, Francis Galton, an amateur mathematician who was Charles Darwin's first cousin, discovered regression to the mean, which explains why pride goeth before a fall and why clouds tend to have silver linings.

In this scenario, the data are given-10 pins, 12 pins, 1 pin-and the probability is the unknown. Questions put in this manner form the subject matter of what is known as inverse probability: with 12 defective pins out of 100,000, what is the probability that the true average ratio of defectives to the total is 0.01%? One of the most effective treatments of such questions was proposed by a minister named Thomas Bayes, who was born in 1701 and lived in Kent." Bayes was a Nonconformist; he rejected most of the ceremonial rituals that the Church of England had retained from the Catholic Church after their separation in the time of Henry VIII. Not much is known about Bayes, even though he was a Fellow of the Royal Society. One otherwise dry and impersonal textbook in statistics went so far as to characterize him as "enigmatic."16 He published nothing in mathematics while he was alive and left only two works that were published after his death but received little attention when they appeared.

The most exciting feature of all the achievements mentioned in this chapter is the daring idea that uncertainty can be measured. Uncertainty means unknown probabilities; to reverse Hacking's description of certainty, we can say that something is uncertain when our information is correct and an event fails to happen, or when our information is incorrect and an event does happen. Jacob Bernoulli, Abraham de Moivre, and Thomas Bayes showed how to infer previously unknown probabilities from the empirical facts of reality. These accomplishments are impressive for the sheer mental agility demanded, and audacious for their bold attack on the unknown. When de Moivre invoked ORIGINAL DESIGN, he made no secret of his wonderment at his own accomplishments. He liked to turn such phrases; at another point, he writes, "If we blind not ourselves with metaphysical dust we shall be led by a short and obvious way, to the acknowledgment of the great MAKER and GOUVERNOUR of all."25 We are by now well into the eighteenth century, when the Enlightenment identified the search for knowledge as the highest form of human activity.


pages: 589 words: 69,193

Mastering Pandas by Femi Anthony

Amazon Web Services, Bayesian statistics, correlation coefficient, correlation does not imply causation, Debian, en.wikipedia.org, Internet of things, natural language processing, p-value, random walk, side project, statistical model, Thomas Bayes

The various topics that will be discussed are as follows: Introduction to Bayesian statistics Mathematical framework for Bayesian statistics Probability distributions Bayesian versus Frequentist statistics Introduction to PyMC and Monte Carlo simulation Illustration of Bayesian inference – Switchpoint detection Introduction to Bayesian statistics The field of Bayesian statistics is built on the work of Reverend Thomas Bayes, an 18th century statistician, philosopher, and Presbyterian minister. His famous Bayes' theorem, which forms the theoretical underpinnings for Bayesian statistics, was published posthumously in 1763 as a solution to the problem of inverse probability. For more details on this topic, refer to http://en.wikipedia.org/wiki/Thomas_Bayes. Inverse probability problems were all the rage in the early 18th century and were often formulated as follows: Suppose you play a game with a friend. There are 10 green balls and 7 red balls in bag 1 and 4 green and 7 red balls in bag 2.


pages: 807 words: 154,435

Radical Uncertainty: Decision-Making for an Unknowable Future by Mervyn King, John Kay

"Robert Solow", Airbus A320, Albert Einstein, Albert Michelson, algorithmic trading, Antoine Gombaud: Chevalier de Méré, Arthur Eddington, autonomous vehicles, availability heuristic, banking crisis, Barry Marshall: ulcers, battle of ideas, Benoit Mandelbrot, bitcoin, Black Swan, Bonfire of the Vanities, Brownian motion, business cycle, business process, capital asset pricing model, central bank independence, collapse of Lehman Brothers, correlation does not imply causation, credit crunch, cryptocurrency, cuban missile crisis, Daniel Kahneman / Amos Tversky, David Ricardo: comparative advantage, demographic transition, discounted cash flows, disruptive innovation, diversification, diversified portfolio, Donald Trump, easy for humans, difficult for computers, Edmond Halley, Edward Lloyd's coffeehouse, Edward Thorp, Elon Musk, Ethereum, Eugene Fama: efficient market hypothesis, experimental economics, experimental subject, fear of failure, feminist movement, financial deregulation, George Akerlof, germ theory of disease, Hans Rosling, Ignaz Semmelweis: hand washing, income per capita, incomplete markets, inflation targeting, information asymmetry, invention of the wheel, invisible hand, Jeff Bezos, Johannes Kepler, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Snow's cholera map, John von Neumann, Kenneth Arrow, Long Term Capital Management, loss aversion, Louis Pasteur, mandelbrot fractal, market bubble, market fundamentalism, Moneyball by Michael Lewis explains big data, Nash equilibrium, Nate Silver, new economy, Nick Leeson, Northern Rock, oil shock, Paul Samuelson, peak oil, Peter Thiel, Philip Mirowski, Pierre-Simon Laplace, popular electronics, price mechanism, probability theory / Blaise Pascal / Pierre de Fermat, quantitative trading / quantitative finance, railway mania, RAND corporation, rent-seeking, Richard Feynman, Richard Thaler, risk tolerance, risk-adjusted returns, Robert Shiller, Robert Shiller, Ronald Coase, sealed-bid auction, shareholder value, Silicon Valley, Simon Kuznets, Socratic dialogue, South Sea Bubble, spectrum auction, Steve Ballmer, Steve Jobs, Steve Wozniak, Tacoma Narrows Bridge, Thales and the olive presses, Thales of Miletus, The Chicago School, the map is not the territory, The Market for Lemons, The Nature of the Firm, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Thomas Bayes, Thomas Davenport, Thomas Malthus, Toyota Production System, transaction costs, ultimatum game, urban planning, value at risk, World Values Survey, Yom Kippur War, zero-sum game

If the Duke and Marquis had planned to play one hundred games, then the Duke’s three to one advantage at an early stage of the evening would have counted for little. But if there were to be only five games, the Marquis would certainly have lost – the result of the fifth game would be irrelevant, and it might not even have been played. It pays to go Bayes The final step in the development of the new theory of probability was the achievement of an unlikely hero – an obscure eighteenth-century country Presbyterian clergyman in England. The Reverend Thomas Bayes is by chance buried in what is now the centre of London’s financial district. Among his papers he left a theorem that is one of the most widely taught ideas in statistics today. 13 Unknown in life Bayes may have been, but his name is known today throughout the world with branches of statistics and economics named after him. The term ‘Bayesian’, which describes not just a statistical technique but a school of thought, is the intellectual legacy of one man working in the Kent countryside.

And it is the pervasive nature of radical uncertainty which is the source of the problem. 20 THE USE AND MISUSE OF MODELS Any business craving of the leader, however foolish, will be quickly supported by detailed rates of return and strategic studies prepared by his troops. —WARREN BUFFETT 1 I n the eighteenth century there were country clergymen of exceptional intelligence who had time on their hands. They benefited from a secure reference narrative. Thomas Bayes was one; Thomas Malthus another. In 1798, Malthus set out what might be regarded as the first growth model in economics. He hypothesised that population tended to grow exponentially, as a result of what he coyly termed ‘the passions’, while food supplies could grow only linearly. The rising population would put pressure on food supplies, and then the resulting destitution would reduce that population.

The model of the lone entrepreneur – the poor boy with a brilliant business idea who rises single-handedly from poverty; the isolated scholar scribbling brilliant ideas in a garret or country vicarage – is largely mythological. There are contrary examples. Thomas Edison was fired from the only two jobs he held, the first as telegraph boy for Western Electric and the other as CEO of the corporation which is now General Electric. The Reverend Thomas Bayes died unknown. But such examples are few, and to find them we have to go some way back in history. The most common profile of the successful entrepreneur today is the individual who draws on his or her past experience in a larger organisation, and works from inception with a team of like-minded individuals. And such individuals can contribute to society only in a supportive social context. There is no shortage of entrepreneurial talent in Nigeria, but too much of it is directed to opportunistic scams and rent seeking.


pages: 411 words: 108,119

The Irrational Economist: Making Decisions in a Dangerous World by Erwann Michel-Kerjan, Paul Slovic

"Robert Solow", Andrei Shleifer, availability heuristic, bank run, Black Swan, business cycle, Cass Sunstein, clean water, cognitive dissonance, collateralized debt obligation, complexity theory, conceptual framework, corporate social responsibility, Credit Default Swap, credit default swaps / collateralized debt obligations, cross-subsidies, Daniel Kahneman / Amos Tversky, endowment effect, experimental economics, financial innovation, Fractional reserve banking, George Akerlof, hindsight bias, incomplete markets, information asymmetry, Intergovernmental Panel on Climate Change (IPCC), invisible hand, Isaac Newton, iterative process, Kenneth Arrow, Loma Prieta earthquake, London Interbank Offered Rate, market bubble, market clearing, money market fund, moral hazard, mortgage debt, Pareto efficiency, Paul Samuelson, placebo effect, price discrimination, price stability, RAND corporation, Richard Thaler, Robert Shiller, Robert Shiller, Ronald Reagan, source of truth, statistical model, stochastic process, The Wealth of Nations by Adam Smith, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, too big to fail, transaction costs, ultimatum game, University of East Anglia, urban planning, Vilfredo Pareto

This chapter explores a two-part conjecture: (1) After the occurrence of a virgin risk, people will overestimate the probability of another occurrence in the near future; (2) by contrast, after an experienced risk occurs, people will under-update their assessment of another event occurring soon. THE INABILITY TO USE BAYESIAN UPDATING IN EVERYDAY PRACTICE Risks are often posited to have an unknown true probability. The textbook model for how to proceed employs Bayes’ Rule (after eighteenth-century British mathematician Thomas Bayes), which shows mathematically how people should rationally change their existing beliefs about something in light of new evidence. Individuals use information available beforehand to form a so-called prior belief about the probability that an event will occur in a given period. New evidence about the risk is captured in something called a likelihood function, which expresses how plausible the evidence is given each possible value of the probability.

American Enterprise Institute American International Group (AIG) American Psychiatric Association, homosexuality and Americans-in-London problem Amygdala(fig.) Anxiety Arrow, Ken Arthur Andersen Assets Asteroid and Comet Impact Hazards Group (NASA) Asteroid explosions, risk of At War with the Weather (Kunreuther and Michel-Kerjan) Attention deficit disorder Awareness, behavioral change and Bali Action Plan (2007) Bargaining games. See also Game Theory; Theory of Games; Ultimatum Games Batson, Daniel Bayes, Thomas Bayes’ Rule Bayesian updating Behavior acceptable awareness and collective Behavior (continued) decision making and descriptive models of individual learned managerial market motivating myopic neuroscience and rational social uncertainty/risk and Behavioral biases Behavioral data, linking(fig.) Behavioral explanations Behavioral research Behavioral science Beliefs Benefits concentrated extreme sharing uncertain Bhopal disaster Black Death Blair, Tony Bonds catastrophe municipal Bowman, Edward Brain emotional/rational parts of Brain activity unfair offers and(fig.)


pages: 502 words: 107,657

Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die by Eric Siegel

Albert Einstein, algorithmic trading, Amazon Mechanical Turk, Apple's 1984 Super Bowl advert, backtesting, Black Swan, book scanning, bounce rate, business intelligence, business process, butter production in bangladesh, call centre, Charles Lindbergh, commoditize, computer age, conceptual framework, correlation does not imply causation, crowdsourcing, dark matter, data is the new oil, en.wikipedia.org, Erik Brynjolfsson, Everything should be made as simple as possible, experimental subject, Google Glasses, happiness index / gross national happiness, job satisfaction, Johann Wolfgang von Goethe, lifelogging, Machine translation of "The spirit is willing, but the flesh is weak." to Russian and back, mass immigration, Moneyball by Michael Lewis explains big data, Nate Silver, natural language processing, Netflix Prize, Network effects, Norbert Wiener, personalized medicine, placebo effect, prediction markets, Ray Kurzweil, recommendation engine, risk-adjusted returns, Ronald Coase, Search for Extraterrestrial Intelligence, self-driving car, sentiment analysis, Shai Danziger, software as a service, speech recognition, statistical model, Steven Levy, text mining, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Davenport, Turing test, Watson beat the top human players on Jeopardy!, X Prize, Yogi Berra, zero-sum game

To prepare for this battle, we armed PA with powerful weaponry. The predictions were generated from machine learning across 50 million learning cases, each depicting a micro-lesson from history of the form, “User Mary was shown ad A and she did click it” (a positive case) or “User John was shown ad B and he did not click it” (a negative case). The learning technology employed to pick the best ad for each user was a Naïve Bayes model. Reverend Thomas Bayes was an eighteenth-century mathematician, and the “Naïve” part means that we take a very smart man’s ideas and compromise them in a way that simplifies yet makes their application feasible, resulting in a practical method that’s often considered good enough at prediction, and scales to the task at hand. I went with this method for its relative simplicity, since in fact I needed to generate 291 such models, one for each ad.

Apple Mac Apple Siri Argonne National Laboratory Arizona Petrified Forest National Park Arizona State University artificial intelligence (AI) about Amazon.com Mechanical Turk mind-reading technology possibility of, the Watson computer and Asimov, Isaac astronomy AT&T Research BellKor Netflix Prize teams Australia Austria automobile insurance crashes, predicting credit scores and accidents driver inatentiveness, predicting fraud predictions for Averitt aviation incidents Aviva Insurance (UK) AWK computer language B backtesting. See also test data Baesens, Ben bagging (bootstrap aggregating) Bangladesh Barbie dolls Bayes, Thomas (Bayes Network) Beane, Billy Beano Beaux, Alex behavioral predictors Bella Pictures BellKor BellKor Netflix Prize teams Ben Gurion University (Israel) Bernstein, Peter Berra, Yogi Big Bang Theory, The Big Bang theory Big Brother BigChaos team “big data” movement billing errors, predicting black box trading Black Swan, The (Taleb) blogs and blogging anxiety, predicting from entries collective intelligence and data glut and content in LiveJournal mood prediction research via nature of Blue Cross Blue Shield of Tennessee BMW BNSF Railway board games, predictive play of Bohr, Niels book titles, testing Bowie, David brain activity, predicting Brandeis, Louis Brasil Telecom (Oi) breast cancer, predicting Brecht, Bertolt Breiman, Leo Brigham Young University British Broadcasting Corporation (BBC) Brobst, Stephen Brooks, Mel Brynjolfsson, Eric buildings, predicting fault in Bullard, Ben burglaries, predicting business rules, decision trees and buying behavior, predicting C Cage, Nicolas Canadian Automobile Association Canadian Tire car crashes and harm, predicting CareerBuilder Carlin, George Carlson, Gretchen Carnegie Mellon University CART decision trees Castagno, Davide causality cell phone industry consumer behavior and dropped calls, predicting GPS data and location predicting Telenor (Norway) CellTel (African telecom) Central Tables.


pages: 416 words: 112,268

Human Compatible: Artificial Intelligence and the Problem of Control by Stuart Russell

3D printing, Ada Lovelace, AI winter, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Alfred Russel Wallace, Andrew Wiles, artificial general intelligence, Asilomar, Asilomar Conference on Recombinant DNA, augmented reality, autonomous vehicles, basic income, blockchain, brain emulation, Cass Sunstein, Claude Shannon: information theory, complexity theory, computer vision, connected car, crowdsourcing, Daniel Kahneman / Amos Tversky, delayed gratification, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Ernest Rutherford, Flash crash, full employment, future of work, Gerolamo Cardano, ImageNet competition, Intergovernmental Panel on Climate Change (IPCC), Internet of things, invention of the wheel, job automation, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John Nash: game theory, John von Neumann, Kenneth Arrow, Kevin Kelly, Law of Accelerating Returns, Mark Zuckerberg, Nash equilibrium, Norbert Wiener, NP-complete, openstreetmap, P = NP, Pareto efficiency, Paul Samuelson, Pierre-Simon Laplace, positional goods, probability theory / Blaise Pascal / Pierre de Fermat, profit maximization, RAND corporation, random walk, Ray Kurzweil, recommendation engine, RFID, Richard Thaler, ride hailing / ride sharing, Robert Shiller, Robert Shiller, Rodney Brooks, Second Machine Age, self-driving car, Shoshana Zuboff, Silicon Valley, smart cities, smart contracts, social intelligence, speech recognition, Stephen Hawking, Steven Pinker, superintelligent machines, Thales of Miletus, The Future of Employment, Thomas Bayes, Thorstein Veblen, transport as a service, Turing machine, Turing test, universal basic income, uranium enrichment, Von Neumann architecture, Wall-E, Watson beat the top human players on Jeopardy!, web application, zero-sum game

Pierre-Simon Laplace, the great French mathematician, wrote in 1814, “The theory of probabilities is just common sense reduced to calculus.”57 It was not until the 1980s, however, that a practical formal language and reasoning algorithms were developed for probabilistic knowledge. This was the language of Bayesian networks,C introduced by Judea Pearl. Roughly speaking, Bayesian networks are the probabilistic cousins of propositional logic. There are also probabilistic cousins of first-order logic, including Bayesian logic58 and a wide variety of probabilistic programming languages. Bayesian networks and Bayesian logic are named after the Reverend Thomas Bayes, a British clergyman whose lasting contribution to modern thought—now known as Bayes’ theorem—was published in 1763, shortly after his death, by his friend Richard Price.59 In its modern form, as suggested by Laplace, the theorem describes in a very simple way how a prior probability—the initial degree of belief one has in a set of possible hypotheses—becomes a posterior probability as a result of observing some evidence.

An early commentary on the role of probability in human thinking: Pierre-Simon Laplace, Essai philosophique sur les probabilités (Mme. Ve. Courcier, 1814). 58. Bayesian logic described in a fairly nontechnical way: Stuart Russell, “Unifying logic and probability,” Communications of the ACM 58 (2015): 88–97. The paper draws heavily on the PhD thesis research of my former student Brian Milch. 59. The original source for Bayes’ theorem: Thomas Bayes and Richard Price, “An essay towards solving a problem in the doctrine of chances,” Philosophical Transactions of the Royal Society of London 53 (1763): 370–418. 60. Technically, Samuel’s program did not treat winning and losing as absolute rewards; by fixing the value of material to be positive; however, the program generally tended to work towards winning. 61. The application of reinforcement learning to produce a world-class backgammon program: Gerald Tesauro, “Temporal difference learning and TD-Gammon,” Communications of the ACM 38 (1995): 58–68. 62.


pages: 370 words: 107,983

Rage Inside the Machine: The Prejudice of Algorithms, and How to Stop the Internet Making Bigots of Us All by Robert Elliott Smith

Ada Lovelace, affirmative action, AI winter, Alfred Russel Wallace, Amazon Mechanical Turk, animal electricity, autonomous vehicles, Black Swan, British Empire, cellular automata, citizen journalism, Claude Shannon: information theory, combinatorial explosion, corporate personhood, correlation coefficient, crowdsourcing, Daniel Kahneman / Amos Tversky, desegregation, discovery of DNA, Douglas Hofstadter, Elon Musk, Fellow of the Royal Society, feminist movement, Filter Bubble, Flash crash, Gerolamo Cardano, gig economy, Gödel, Escher, Bach, invention of the wheel, invisible hand, Jacquard loom, Jacques de Vaucanson, John Harrison: Longitude, John von Neumann, Kenneth Arrow, low skilled workers, Mark Zuckerberg, mass immigration, meta analysis, meta-analysis, mutually assured destruction, natural language processing, new economy, On the Economy of Machinery and Manufactures, p-value, pattern recognition, Paul Samuelson, performance metric, Pierre-Simon Laplace, precariat, profit maximization, profit motive, Silicon Valley, social intelligence, statistical model, Stephen Hawking, stochastic process, telemarketer, The Bell Curve by Richard Herrnstein and Charles Murray, The Future of Employment, the scientific method, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, theory of mind, Thomas Bayes, Thomas Malthus, traveling salesman, Turing machine, Turing test, twin studies, Vilfredo Pareto, Von Neumann architecture, women in the workforce

For instance, probabilistic reasoning is at the heart of algorithms that label images, make search engine suggestions, determine ad placements in social media feeds, and suggest mates on dating sites. In each case, reasoning from big data statistics (about your online profile, and those of many other people) are determining probabilities that are used to derive a desirable outcome. But most of these algorithms also involve an additional probability rule, which was furnished in 1761 by London Presbyterian minister Thomas Bayes. Little is known about Thomas Bayes other than he was a Fellow of the Royal Society and he wrote two papers on mathematics, one of which, Essay Towards Solving a Problem in the Doctrine of Chances (published after his death in 1761), laid out the final foundational rule of modern probability theory. Bayes’ rule follows trivially from the event-size-and-zoom-based arguments presented below, as shown in Figure 3.3.


pages: 137 words: 36,231

Information: A Very Short Introduction by Luciano Floridi

agricultural Revolution, Albert Einstein, bioinformatics, carbon footprint, Claude Shannon: information theory, conceptual framework, double helix, Douglas Engelbart, Douglas Engelbart, George Akerlof, Gordon Gekko, industrial robot, information asymmetry, intangible asset, Internet of things, invention of writing, John Nash: game theory, John von Neumann, Laplace demon, moral hazard, Nash equilibrium, Nelson Mandela, Norbert Wiener, Pareto efficiency, phenotype, Pierre-Simon Laplace, prisoner's dilemma, RAND corporation, RFID, Thomas Bayes, Turing machine, Vilfredo Pareto

The question she is implicitly asking is: `what is the probability thatA (= the email was infected), given the fact that B (= the email was blocked by the antivirus and placed in the quarantine folder) when, on average, 2% of my emails are actually infected and my antivirus is successful 95% of the time?'. Jill has just identified a way of acquiring (learning) the missing piece of information that will help her to adopt the right strategy: if the chance that some emails in the quarantine folder might not be infected is very low, she will check it only occasionally. How could she obtain such a missing piece of information? The answer is by using a Bayesian approach. Thomas Bayes (1702-1761) was a Presbyterian minister and English mathematician whose investigations into probability, published posthumously, led to what is now known as Bayes' theorem and a new branch of applications of probability theory. The theorem calculates the posterior probability of an eventA given event B (that is, P(AIB) on the basis of the prior probability ofA (that is, P(A)). Basically, it tells us what sort of information can be retrodicted.


pages: 147 words: 39,910

The Great Mental Models: General Thinking Concepts by Shane Parrish

Albert Einstein, Atul Gawande, Barry Marshall: ulcers, bitcoin, Black Swan, colonial rule, correlation coefficient, correlation does not imply causation, cuban missile crisis, Daniel Kahneman / Amos Tversky, dark matter, delayed gratification, feminist movement, index fund, Isaac Newton, Jane Jacobs, mandelbrot fractal, Pierre-Simon Laplace, Ponzi scheme, Richard Feynman, statistical model, stem cell, The Death and Life of Great American Cities, the map is not the territory, the scientific method, Thomas Bayes, Torches of Freedom

Mostly, we want to make good decisions in complex social systems that were not part of the world in which our brains evolved their (quite rational) heuristics. For this, we need to consciously add in a needed layer of probability awareness. What is it and how can I use it to my advantage? There are three important aspects of probability that we need to explain so you can integrate them into your thinking to get into the ballpark and improve your chances of catching the ball: Bayesian thinking Fat-tailed curves Asymmetries Thomas Bayes and Bayesian thinking: Bayes was an English minister in the first half of the 18th century, whose most famous work, “An Essay Toward Solving a Problem in the Doctrine of Chances”, was brought to the attention of the Royal Society by his friend Richard Price in 1763—two years after his death. The essay concerned how we should adjust probabilities when we encounter new data, and provided the seeds for the great mathematician Pierre Simon Laplace to develop what we now call Bayes’s Theorem.


Statistics in a Nutshell by Sarah Boslaugh

Antoine Gombaud: Chevalier de Méré, Bayesian statistics, business climate, computer age, correlation coefficient, experimental subject, Florence Nightingale: pie chart, income per capita, iterative process, job satisfaction, labor-force participation, linear programming, longitudinal study, meta analysis, meta-analysis, p-value, pattern recognition, placebo effect, probability theory / Blaise Pascal / Pierre de Fermat, publication bias, purchasing power parity, randomized controlled trial, selection bias, six sigma, statistical model, The Design of Experiments, the scientific method, Thomas Bayes, Vilfredo Pareto

If the disease rate were 0.005 instead of 0.01, fewer of the positives would be true positives and more would be false positives, as shown in the calculations in Figure 2-14. Figure 2-14. Another example of using Bayes’ theorem to calculate the probability of disease, given a positive test; note the lower rate of true positives, due to a lower rate of disease in the population In this example, less than one third of the positives are true positives. The Reverend Thomas Bayes Bayes’ theorem was developed by a British Nonconformist minister, the Reverend Thomas Bayes (1702–1761). Bayes studied logic and theology at the University of Edinburgh and earned his livelihood as a minister in Holborn and Tunbridge Wells, England. However, his fame today rests on his theory of probability, which was developed in his essay, published after his death by the Royal Society of London. There is an entire field of study today known as Bayesian statistics, which is based on the notion of probability as a statement of strength of belief rather than as a frequency of occurrence.

A a priori, Blocking and the Latin Square, Glossary of Statistical Terms hypothesis, Glossary of Statistical Terms information, Blocking and the Latin Square absolute frequencies, Frequency Tables, Bar Charts–Bar Charts absolute value, Laws of Arithmetic–Laws of Arithmetic, Glossary of Statistical Terms abstract, Writing the Article, Evaluating the Whole Article critiquing in articles, Evaluating the Whole Article writing, Writing the Article Access, for data management, Spreadsheets and Relational Databases actions, Decision Analysis Age of Information, statistics in, Statistics in the Age of Information algebra, linear, Relationships Between Continuous Variables Alpha (α), Hypothesis Testing, Glossary of Statistical Terms definition of, Glossary of Statistical Terms probability of Type I error, Hypothesis Testing alternate form method, Reliability of a Composite Test American Standard Code for Information Interchange (ASCII), String and Numeric Data Analysis of Covariance (ANCOVA), ANCOVA–ANCOVA Analysis of Variance (ANOVA), The t-Test, Introduction to Regression and ANOVA, Analysis of Variance (ANOVA)–Analysis of Variance (ANOVA), One-Way ANOVA–One-Way ANOVA, Post Hoc Tests–Post Hoc Tests, Factorial ANOVA–Three-Way ANOVA, Two-Way ANOVA–Two-Way ANOVA, Three-Way ANOVA–Three-Way ANOVA, ANOVA about, Introduction to Regression and ANOVA, Analysis of Variance (ANOVA)–Analysis of Variance (ANOVA) factorial, Factorial ANOVA–Three-Way ANOVA one-way, One-Way ANOVA–One-Way ANOVA post hoc testing, Post Hoc Tests–Post Hoc Tests t-test and one-way, The t-Test three-way, Three-Way ANOVA–Three-Way ANOVA two-way, Two-Way ANOVA–Two-Way ANOVA violations of assumptions of, ANOVA ANCOVA (Analysis of Covariance), ANCOVA–ANCOVA Anderson–Darling test, Data Transformations ANOVA (Analysis of Variance), The t-Test, Introduction to Regression and ANOVA, Analysis of Variance (ANOVA)–Analysis of Variance (ANOVA), One-Way ANOVA–One-Way ANOVA, Post Hoc Tests–Post Hoc Tests, Factorial ANOVA–Three-Way ANOVA, Two-Way ANOVA–Two-Way ANOVA, Three-Way ANOVA–Three-Way ANOVA, ANOVA about, Introduction to Regression and ANOVA, Analysis of Variance (ANOVA)–Analysis of Variance (ANOVA) factorial, Factorial ANOVA–Three-Way ANOVA one-way, One-Way ANOVA–One-Way ANOVA post hoc testing, Post Hoc Tests–Post Hoc Tests t-test and one-way, The t-Test three-way, Three-Way ANOVA–Three-Way ANOVA two-way, Two-Way ANOVA–Two-Way ANOVA violations of assumptions of, ANOVA AR (Attributable Risk), Attributable Risk, Attributable Risk Percentage, and Number Needed to Treat–Attributable Risk, Attributable Risk Percentage, and Number Needed to Treat AR% (Attributable Risk Percentage), Attributable Risk, Attributable Risk Percentage, and Number Needed to Treat arbitrary curve-fitting, Overfitting–Overfitting arithmetic mean, About Formulas, About Formulas, Glossary of Statistical Terms (see also mean) definition of, Glossary of Statistical Terms formula for, About Formulas arithmetic, laws of, Linear regression, Laws of Arithmetic, Laws of Arithmetic–Laws of Arithmetic, Order of Operations–Properties of Real Numbers absolute value, Laws of Arithmetic–Laws of Arithmetic number line, Linear regression, Laws of Arithmetic order of operations, Order of Operations–Properties of Real Numbers articles, Writing the Article–Writing the Article, Writing the Article–Writing the Article, Evaluating the Whole Article–Evaluating the Whole Article, Common Problems–Common Problems, Quick Checklist–Quick Checklist, Issues in Research Design–The Power of Coincidence, Descriptive Statistics–Extrapolation and Trends, Extrapolation and Trends–Linear regression checklist for statistics based investigations, Quick Checklist–Quick Checklist common problems with, Common Problems–Common Problems critiquing descriptive statistics, Descriptive Statistics–Extrapolation and Trends evaluating, Evaluating the Whole Article–Evaluating the Whole Article incorrect use of tests in inferential statistics, Extrapolation and Trends–Linear regression issues in research design, Issues in Research Design–The Power of Coincidence research articles, Writing the Article–Writing the Article writing, Writing the Article–Writing the Article ASCII (American Standard Code for Information Interchange), String and Numeric Data associations, correlation, Association–Association Attributable Risk (AR), Attributable Risk, Attributable Risk Percentage, and Number Needed to Treat–Attributable Risk, Attributable Risk Percentage, and Number Needed to Treat Attributable Risk Percentage (AR%), Attributable Risk, Attributable Risk Percentage, and Number Needed to Treat attribute data, in field of quality control, Run Charts and Control Charts autocorrelated, Time Series average inter-item correlation, Reliability average item-total correlation, Reliability B background/literature review, writing, Writing the Article backward removal, in stepwise methods for building regression models, Methods for Building Regression Models, Backward removal–Backward removal balance, in research design structure, Ingredients of a Good Design bar charts, Bar Charts–Pie Charts Bartlett test, Unequal Variance t-Test, Factor Analysis baseline response variable, Specifying Response Variables Bayes, Thomas, Bayes’ Theorem Bayes’ theorem (formula), Bayes’ Theorem–Bayes’ Theorem Behrens-Fisher problem, Unequal Variance t-Test Bernoulli process, The Binomial Distribution Bernoulli trial, The Binomial Distribution Beta (β), Hypothesis Testing, Glossary of Statistical Terms definition of, Glossary of Statistical Terms probability of Type I error, Hypothesis Testing between-subjects designs, tests for, The Wilcoxon Rank Sum Test–Kruskal-Wallis H Test bias, Measurement Bias–Information Bias, Bias in Sample Selection and Retention, Bias in Sample Selection and Retention–Bias in Sample Selection and Retention, Bias in Sample Selection and Retention, Information Bias–Information Bias, Information Bias–Information Bias, Information Bias, Information Bias, Information Bias, Exercises, Ingredients of a Good Design, Identifying Experimental Units, Retrospective Adjustment, Blocking and the Latin Square, Quick Checklist, Glossary of Statistical Terms, Glossary of Statistical Terms, Glossary of Statistical Terms, Glossary of Statistical Terms, Glossary of Statistical Terms, Glossary of Statistical Terms, Glossary of Statistical Terms, Glossary of Statistical Terms definition of, Glossary of Statistical Terms detection, Information Bias, Glossary of Statistical Terms in gathering experimental data, Identifying Experimental Units information, Information Bias–Information Bias, Glossary of Statistical Terms interviewer, Information Bias–Information Bias maturation, Blocking and the Latin Square minimizing in data, Ingredients of a Good Design nonresponse, Bias in Sample Selection and Retention, Glossary of Statistical Terms predictions of presidential elections and sample, Exercises publication bias, Quick Checklist recall, Information Bias recall bias, Glossary of Statistical Terms retrospective adjustment, Retrospective Adjustment selection, Bias in Sample Selection and Retention, Glossary of Statistical Terms social desirability, Information Bias, Glossary of Statistical Terms types of, Measurement Bias–Information Bias volunteer, Bias in Sample Selection and Retention–Bias in Sample Selection and Retention, Glossary of Statistical Terms binary, Nominal Data, Glossary of Statistical Terms data, Nominal Data variables, Glossary of Statistical Terms binomial distribution, The Binomial Distribution–The Binomial Distribution, The Binomial Distribution–The Binomial Distribution biserial correlation coefficient, Item Analysis bivariate charts, Bivariate Charts, Scatterplots–Scatterplots, Line Graphs–Line Graphs about, Bivariate Charts line graphs, Line Graphs–Line Graphs scatterplots, Scatterplots–Scatterplots blinding, Blinding, Glossary of Statistical Terms blocking, Blocking and the Latin Square blocking methods for building regression models, Methods for Building Regression Models–Methods for Building Regression Models, Methods for Building Regression Models boxplot, The Boxplot–The Boxplot broken stick model, Factor Analysis Brown-Forsythe test, Unequal Variance t-Test business statistics, Index Numbers–Index Numbers, Time Series–Time Series, Decision Analysis–Decision Trees, Quality Improvement–Run Charts and Control Charts decision analysis, Decision Analysis–Decision Trees index numbers, Index Numbers–Index Numbers quality improvement, Quality Improvement–Run Charts and Control Charts time series, Time Series–Time Series C c-charts, Run Charts and Control Charts “Caffeine and cortical arousal” (Watters et al), Polynomial Regression–Polynomial Regression calculators, Power Analysis, Properties of Roots, Factorials, Permutations, and Combinations factorial key, Factorials, Permutations, and Combinations graphical power, Power Analysis scientific, Properties of Roots Campbell, Donald T., Basic Vocabulary, Quasi-Experimental Studies, Quasi-Experimental Studies Canonical Correlation Analysis (CCA), Factor Analysis canonical discriminant functions, Discriminant Function Analysis Cartesian coordinates (rectangular coordinates), Graphing Equations–Graphing Equations case control design, Observational Studies case-control studies, The Odds Ratio categorical data, Nominal Data, Nominal Data, Categorical Data–Categorical Data, Categorical Data–The Likert and Semantic Differential Scales, The R×C Table–The R×C Table, Measures of Agreement–Measures of Agreement, The Chi-Square Distribution, The Chi-Square Test, The Chi-Square Test–The Chi-Square Test, The Chi-Square Test–The Chi-Square Test, The Chi-Square Test–The Chi-Square Test, The Chi-Square Test, Fisher’s Exact Test–Fisher’s Exact Test, McNemar’s Test for Matched Pairs–McNemar’s Test for Matched Pairs, Proportions: The Large Sample Case–Proportions: The Large Sample Case, Correlation Statistics for Categorical Data–Ordinal Variables, The Likert and Semantic Differential Scales, The Likert and Semantic Differential Scales–The Likert and Semantic Differential Scales (see also nominal data) about, Categorical Data–Categorical Data chi-square distributions, The Chi-Square Distribution chi-square test, The Chi-Square Test, The Chi-Square Test–The Chi-Square Test, The Chi-Square Test–The Chi-Square Test, The Chi-Square Test–The Chi-Square Test, The Chi-Square Test, McNemar’s Test for Matched Pairs–McNemar’s Test for Matched Pairs about, The Chi-Square Test for equality of proportions, The Chi-Square Test–The Chi-Square Test for independence, The Chi-Square Test–The Chi-Square Test, The Chi-Square Test McNemar’s test, McNemar’s Test for Matched Pairs–McNemar’s Test for Matched Pairs of goodness of fit, The Chi-Square Test–The Chi-Square Test correlation statistics for, Correlation Statistics for Categorical Data–Ordinal Variables Fisher’s Exact Test, Fisher’s Exact Test–Fisher’s Exact Test Likert scale, The Likert and Semantic Differential Scales measures of agreement, Measures of Agreement–Measures of Agreement nominal data and, Nominal Data proportions, Proportions: The Large Sample Case–Proportions: The Large Sample Case R×C table, The R×C Table–The R×C Table semantic differential scale, The Likert and Semantic Differential Scales–The Likert and Semantic Differential Scales category-specific rates, Crude, Category-Specific, and Standardized Rates–Crude, Category-Specific, and Standardized Rates Cattell, James, Factor Analysis ceiling effect, The Boxplot Census (U.S.), samples and, Populations and Samples central limit theorem, The Central Limit Theorem–The Central Limit Theorem Central Moving Average (CMA), Time Series central tendency, measures of, Inferential Statistics, Measures of Central Tendency, The Mean–The Mean, The Mean–The Mean, The Median–The Median, The Mode–Comparing the Mean, Median, and Mode, Comparing the Mean, Median, and Mode–Comparing the Mean, Median, and Mode, Measures of Central Tendency–Measures of Central Tendency about, Measures of Central Tendency critiquing choice in article of, Measures of Central Tendency–Measures of Central Tendency in descriptive statistics, The Mean–The Mean mean, The Mean–The Mean mean, Inferential Statistics, The Mean–The Mean in descriptive statistics, The Mean–The Mean in inferential statistics, Inferential Statistics median, The Median–The Median, Comparing the Mean, Median, and Mode–Comparing the Mean, Median, and Mode mode in, The Mode–Comparing the Mean, Median, and Mode checklist for statistics based investigations, Quick Checklist–Quick Checklist chi-square distributions, The Chi-Square Distribution, The Chi-Square Distribution–The Chi-Square Distribution chi-square test, The Chi-Square Test, The Chi-Square Test–The Chi-Square Test, The Chi-Square Test–The Chi-Square Test, The Chi-Square Test–The Chi-Square Test, The Chi-Square Test, McNemar’s Test for Matched Pairs–McNemar’s Test for Matched Pairs about, The Chi-Square Test for equality of proportions, The Chi-Square Test–The Chi-Square Test for independence, The Chi-Square Test–The Chi-Square Test, The Chi-Square Test about, The Chi-Square Test–The Chi-Square Test Yates’s correction for continuity, in chi-square test, The Chi-Square Test McNemar’s test, McNemar’s Test for Matched Pairs–McNemar’s Test for Matched Pairs of goodness of fit, The Chi-Square Test–The Chi-Square Test CI (Cumulative Incidence), Prevalence and Incidence classic experimental design, Basic Vocabulary classical test theory, Classical Test Theory: The True Score Model–Classical Test Theory: The True Score Model Cluster analysis, Cluster Analysis–Cluster Analysis cluster samples, Probability Sampling CMA (Central Moving Average), Time Series codebooks, data management, Codebooks–Codebooks coefficient, Reliability, Reliability, The Coefficient of Determination, The General Linear Model, Reliability of a Composite Test, Reliability of a Composite Test, Split-Half Methods, Coefficient Alpha about term, The General Linear Model of determination, The Coefficient of Determination of equivalence, Reliability, Reliability of a Composite Test, Split-Half Methods of precision, Coefficient Alpha of stability, Reliability, Reliability of a Composite Test coefficient alpha, Reliability, Coefficient Alpha, Coefficient Alpha–Coefficient Alpha, Coefficient Alpha Cronbach’s alpha, Reliability, Coefficient Alpha–Coefficient Alpha Kuder-Richardson formulas, Coefficient Alpha, Coefficient Alpha Coefficient of Variation (CV), The Variance and Standard Deviation–The Variance and Standard Deviation Cohen’s kappa, Measures of Agreement–Measures of Agreement cohort, Basic Vocabulary, Glossary of Statistical Terms coins, Dice, Coins, and Playing Cards combinations, Factorials, Permutations, and Combinations combinations of elements, Combinations–Combinations common causes of variation, Run Charts and Control Charts communicating with statistics, Communicating with Statistics–Writing for Your Workplace complement of event, Complement–Complement complex random samples, Probability Sampling composite indices, Index Numbers composite test, Test Construction, Reliability of a Composite Test–Reliability of a Composite Test reliability of, Reliability of a Composite Test–Reliability of a Composite Test scores, Test Construction compound events, Events conclusions and results, critiquing in articles, Evaluating the Whole Article concurrent validity, Validity conditional probabilities, Conditional Probabilities–Conditional Probabilities confidence coefficient, Confidence Intervals confidence intervals, Confidence Intervals–Confidence Intervals, Confidence Interval for the One-Sample t-Test–Confidence Interval for the One-Sample t-Test, Confidence Interval for the Independent Samples t-Test, Confidence Interval for the Repeated Measures t-Test–Confidence Interval for the Repeated Measures t-Test, The Risk Ratio, Confidence Interval for a Proportion, Standard Error and Confidence Intervals about, Confidence Intervals–Confidence Intervals calculating for risk ratio, The Risk Ratio critiquing in articles, Standard Error and Confidence Intervals for independent samples (two-sample) t-test, Confidence Interval for the Independent Samples t-Test for one-sample t-test, Confidence Interval for the One-Sample t-Test–Confidence Interval for the One-Sample t-Test for proportions, Confidence Interval for a Proportion repeated measures (related samples) t-test, Confidence Interval for the Repeated Measures t-Test–Confidence Interval for the Repeated Measures t-Test confounding, Confounding, Stratified Analysis, and the Mantel-Haenszel Common Odds Ratio–Confounding, Stratified Analysis, and the Mantel-Haenszel Common Odds Ratio confounding variable, Glossary of Statistical Terms Conover, William, Practical Nonparametric Statistics, Nonparametric Statistics consistency measurements, Measures of Internal Consistency construct validity, Glossary of Statistical Terms Consumer Price Index (CPI), Index Numbers, Index Numbers content validity, Validity, Glossary of Statistical Terms contingency tables, The R×C Table–The R×C Table, The Risk Ratio R×C table, The R×C Table–The R×C Table two-by-two table, The Risk Ratio continuous data, Continuous and Discrete Data, Glossary of Statistical Terms definition of, Glossary of Statistical Terms vs. discrete data, Continuous and Discrete Data continuous variables, relationships between, Relationships Between Continuous Variables–Relationships Between Continuous Variables control charts and run charts, Run Charts and Control Charts control variables, Independent and Dependent Variables, Observational Studies, Glossary of Statistical Terms about, Independent and Dependent Variables definition of, Glossary of Statistical Terms in observational studies, Observational Studies controls, Identifying Treatments and Controls, Controls in experimental design, Identifying Treatments and Controls issues with, Controls convenience samples, Nonprobability Sampling Cook, Thomas D., Basic Vocabulary, Quasi-Experimental Studies, Quasi-Experimental Studies correlation statistics for categorical data, Binary Variables–Ordinal Variables correlations, The Pearson Correlation Coefficient, Association–Association, Scatterplots–Relationships Between Continuous Variables, Relationships Between Continuous Variables–Relationships Between Continuous Variables, The Pearson Correlation Coefficient–Testing Statistical Significance for the Pearson Correlation, Testing Statistical Significance for the Pearson Correlation–Testing Statistical Significance for the Pearson Correlation, The Coefficient of Determination, Methods for Building Regression Models about, The Pearson Correlation Coefficient associations, Association–Association coefficient of determination, The Coefficient of Determination correlation coefficient, The Pearson Correlation Coefficient–Testing Statistical Significance for the Pearson Correlation partial, Methods for Building Regression Models relationships between continuous variables, Relationships Between Continuous Variables–Relationships Between Continuous Variables scatterplots as visual tool, Scatterplots–Relationships Between Continuous Variables testing statistical significance for, Testing Statistical Significance for the Pearson Correlation–Testing Statistical Significance for the Pearson Correlation CPI (Consumer Price Index), Index Numbers, Index Numbers Cramer’s V, Binary Variables–Binary Variables criterion for factor retention, Factor Analysis criterion validity, Glossary of Statistical Terms criterion-referenced tests, Test Construction critiquing presentations about statistics, Evaluating the Whole Article–Evaluating the Whole Article, The Misuse of Statistics–The Misuse of Statistics, Common Problems–Common Problems, Quick Checklist–Quick Checklist, Issues in Research Design–The Power of Coincidence, Descriptive Statistics–Extrapolation and Trends, Extrapolation and Trends–Linear regression checklist for statistics based investigations, Quick Checklist–Quick Checklist common problems in presentations, Common Problems–Common Problems evaluating whole article, Evaluating the Whole Article–Evaluating the Whole Article incorrect use of tests in inferential statistics, Extrapolation and Trends–Linear regression interpretation of descriptive statistics, Descriptive Statistics–Extrapolation and Trends issues in research design, Issues in Research Design–The Power of Coincidence misusing statistics, The Misuse of Statistics–The Misuse of Statistics Cronbach’s alpha (coefficient alpha), Reliability cross-sectional design, Observational Studies–Observational Studies cross-sectional study, Glossary of Statistical Terms cross-tabulation, The Risk Ratio crude rate, Crude, Category-Specific, and Standardized Rates–Crude, Category-Specific, and Standardized Rates cubic regression model, Polynomial Regression–Polynomial Regression cumulative frequency, Frequency Tables Cumulative Incidence (CI), Prevalence and Incidence CV (Coefficient of Variation), The Variance and Standard Deviation–The Variance and Standard Deviation D data, Statistics in the Age of Information, Basic Concepts of Measurement, Basic Concepts of Measurement–Proxy Measurement, The Rectangular Data File, String and Numeric Data–Missing Data, Gathering Experimental Data–Blocking and the Latin Square, Evaluating the Whole Article converting information into, Basic Concepts of Measurement critiquing in articles, Evaluating the Whole Article gathering experimental data, Gathering Experimental Data–Blocking and the Latin Square meaning of, Statistics in the Age of Information missing data, String and Numeric Data–Missing Data types of, Basic Concepts of Measurement–Proxy Measurement unit of analysis, The Rectangular Data File data management, Data Management–Data Management, An Approach, Not a Set of Recipes–An Approach, Not a Set of Recipes, The Chain of Command, Codebooks–Codebooks, Codebooks–The Rectangular Data File, Spreadsheets and Relational Databases–Spreadsheets and Relational Databases, Spreadsheets and Relational Databases, Inspecting a New Data File–Inspecting a New Data File, Inspecting a New Data File, Inspecting a New Data File, String and Numeric Data, String and Numeric Data–Missing Data about, Data Management–Data Management approach to, An Approach, Not a Set of Recipes–An Approach, Not a Set of Recipes codebooks, Codebooks–Codebooks data entry software, Spreadsheets and Relational Databases in projects, The Chain of Command inspecting new data file, Inspecting a New Data File–Inspecting a New Data File missing data, String and Numeric Data–Missing Data spreadsheets and relational databases for, Spreadsheets and Relational Databases–Spreadsheets and Relational Databases storing data electronically in rectangular data file, Codebooks–The Rectangular Data File string and numeric data, String and Numeric Data unique identifier in, Inspecting a New Data File variable names in transfer process to software, Inspecting a New Data File data mining vs. hypothesis testing, Specifying Response Variables data transformations, Data Transformations–Data Transformations data types, Basic Vocabulary databases, for data management, Spreadsheets and Relational Databases–Spreadsheets and Relational Databases decision analysis, Decision Analysis–Decision Trees decision trees, Decision Trees decision-making, Decision Analysis, Decision Analysis, Decision Analysis under certainty, Decision Analysis under risk, Decision Analysis under uncertainty, Decision Analysis degrees of freedom, Glossary of Statistical Terms Deming, W.


pages: 660 words: 141,595

Data Science for Business: What You Need to Know About Data Mining and Data-Analytic Thinking by Foster Provost, Tom Fawcett

Albert Einstein, Amazon Mechanical Turk, big data - Walmart - Pop Tarts, bioinformatics, business process, call centre, chief data officer, Claude Shannon: information theory, computer vision, conceptual framework, correlation does not imply causation, crowdsourcing, data acquisition, David Brooks, en.wikipedia.org, Erik Brynjolfsson, Gini coefficient, information retrieval, intangible asset, iterative process, Johann Wolfgang von Goethe, Louis Pasteur, Menlo Park, Nate Silver, Netflix Prize, new economy, p-value, pattern recognition, placebo effect, price discrimination, recommendation engine, Ronald Coase, selection bias, Silicon Valley, Skype, speech recognition, Steve Jobs, supply-chain management, text mining, The Signal and the Noise by Nate Silver, Thomas Bayes, transaction costs, WikiLeaks

Bayes’ Rule Notice that in p(AB) = p(A)p(B|A) the order of A and B seems rather arbitrary—and it is. We could just as well have written: This means: And so: If we divide both sides by p(A) we get: Now, let’s consider B to be some hypothesis that we are interested in assessing the likelihood of, and A to be some evidence that we have observed. Renaming with H for hypothesis and E for evidence, we get: This is the famous Bayes’ Rule, named after the Reverend Thomas Bayes who derived a special case of the rule back in the 18th century. Bayes’ Rule says that we can compute the probability of our hypothesis H given some evidence E by instead looking at the probability of the evidence given the hypothesis, as well as the unconditional probabilities of the hypothesis and the evidence. Note: Bayesian methods Bayes’ Rule, combined with the important fundamental principle of thinking carefully about conditional independencies, are the foundation for a vast amount of more advanced data science techniques that we will not cover in this book.

., Answering Business Questions with These Techniques B bag of words approach, Bag of Words bags, Bag of Words base rates, Class Probability Estimation and Logistic “Regression”, Holdout Data and Fitting Graphs, Problems with Unbalanced Classes baseline classifiers, Advantages and Disadvantages of Naive Bayes baseline methods, of data science, Summary Basie, Count, Example: Jazz Musicians Bayes rate, Bias, Variance, and Ensemble Methods Bayes, Thomas, Bayes’ Rule Bayesian methods, Bayes’ Rule, Summary Bayes’ Rule, Bayes’ Rule–A Model of Evidence “Lift” beer and lottery example, Example: Beer and Lottery Tickets–Example: Beer and Lottery Tickets Beethoven, Ludwig van, Example: Evidence Lifts from Facebook “Likes” beginning cross-validation, From Holdout Evaluation to Cross-Validation behavior description, From Business Problems to Data Mining Tasks Being John Malkovich (film), Data Reduction, Latent Information, and Movie Recommendation Bellkors Pragmatic Chaos (Netflix Challenge team), Data Reduction, Latent Information, and Movie Recommendation benefit improvement, calculating, Costs and benefits benefits and underlying profit calculation, ROC Graphs and Curves data-driven decision-making, Data Science, Engineering, and Data-Driven Decision Making estimating, Costs and benefits in budgeting, Ranking Instead of Classifying nearest-neighbor methods, Computational efficiency bi-grams, N-gram Sequences bias errors, ensemble methods and, Bias, Variance, and Ensemble Methods–Bias, Variance, and Ensemble Methods Big Data data science and, Data Processing and “Big Data”–Data Processing and “Big Data” evolution of, From Big Data 1.0 to Big Data 2.0–From Big Data 1.0 to Big Data 2.0 on Amazon and Google, Thinking Data-Analytically, Redux big data technologies, Data Processing and “Big Data” state of, From Big Data 1.0 to Big Data 2.0 utilizing, Data Processing and “Big Data” Big Red proposal example, Example Data Mining Proposal–Flaws in the Big Red Proposal Bing, Why Text Is Important, Representation Black-Sholes model, Models, Induction, and Prediction blog postings, Why Text Is Important blog posts, Example: Targeting Online Consumers With Advertisements Borders (book retailer), Achieving Competitive Advantage with Data Science breast cancer example, Example: Logistic Regression versus Tree Induction–Example: Logistic Regression versus Tree Induction Brooks, David, What Data Can’t Do: Humans in the Loop, Revisited browser cookies, Example: Targeting Online Consumers With Advertisements Brubeck, Dave, Example: Jazz Musicians Bruichladdich single malt scotch, Understanding the Results of Clustering Brynjolfsson, Erik, Data Science, Engineering, and Data-Driven Decision Making, Data Processing and “Big Data” budget, Ranking Instead of Classifying budget constraints, Profit Curves building modeling labs, From Holdout Evaluation to Cross-Validation building models, Data Mining and Its Results, Business Understanding, From Holdout Evaluation to Cross-Validation Bunnahabhain single malt whiskey, Example: Whiskey Analytics, Hierarchical Clustering business news stories example, Example: Clustering Business News Stories–The news story clusters business problems changing definition of, to fit available data, Changing the Way We Think about Solutions to Business Problems–Changing the Way We Think about Solutions to Business Problems data exploration vs., Stepping Back: Solving a Business Problem Versus Data Exploration–Stepping Back: Solving a Business Problem Versus Data Exploration engineering problems vs., Other Data Science Tasks and Techniques evaluating in a proposal, Be Ready to Evaluate Proposals for Data Science Projects expected value framework, structuring with, The Expected Value Framework: Structuring a More Complicated Business Problem–The Expected Value Framework: Structuring a More Complicated Business Problem exploratory data mining vs., The Fundamental Concepts of Data Science unique context of, What Data Can’t Do: Humans in the Loop, Revisited using expected values to provide framework for, The Expected Value Framework: Decomposing the Business Problem and Recomposing the Solution Pieces–The Expected Value Framework: Decomposing the Business Problem and Recomposing the Solution Pieces business strategy, Data Science and Business Strategy–A Firm’s Data Science Maturity accepting creative ideas, Be Ready to Accept Creative Ideas from Any Source case studies, examining, Examine Data Science Case Studies competitive advantages, Achieving Competitive Advantage with Data Science–Achieving Competitive Advantage with Data Science, Sustaining Competitive Advantage with Data Science–Superior Data Science Management data scientists, evaluating, Superior Data Scientists–Superior Data Scientists evaluating proposals, Be Ready to Evaluate Proposals for Data Science Projects–Flaws in the Big Red Proposal historical advantages and, Formidable Historical Advantage intangible collateral assets and, Unique Intangible Collateral Assets intellectual property and, Unique Intellectual Property managing data scientists effectively, Superior Data Science Management–Superior Data Science Management maturity of the data science, A Firm’s Data Science Maturity–A Firm’s Data Science Maturity thinking data-analytically for, Thinking Data-Analytically, Redux–Thinking Data-Analytically, Redux C Caesars Entertainment, Data and Data Science Capability as a Strategic Asset call center example, Profiling: Finding Typical Behavior–Profiling: Finding Typical Behavior Capability Maturity Model, A Firm’s Data Science Maturity Capital One, Data and Data Science Capability as a Strategic Asset, From an Expected Value Decomposition to a Data Science Solution Case-Based Reasoning, How Many Neighbors and How Much Influence?


pages: 189 words: 57,632

Content: Selected Essays on Technology, Creativity, Copyright, and the Future of the Future by Cory Doctorow

AltaVista, book scanning, Brewster Kahle, Burning Man, en.wikipedia.org, informal economy, information retrieval, Internet Archive, invention of movable type, Jeff Bezos, Law of Accelerating Returns, Metcalfe's law, Mitch Kapor, moral panic, mutually assured destruction, new economy, optical character recognition, patent troll, pattern recognition, peer-to-peer, Ponzi scheme, post scarcity, QWERTY keyboard, Ray Kurzweil, RFID, Sand Hill Road, Skype, slashdot, social software, speech recognition, Steve Jobs, Thomas Bayes, Turing test, Vernor Vinge

The Future of Internet Immune Systems (Originally published on InformationWeek's Internet Evolution, November 19, 2007) Bunhill Cemetery is just down the road from my flat in London. It’s a handsome old boneyard, a former plague pit (“Bone hill” — as in, there are so many bones under there that the ground is actually kind of humped up into a hill). There are plenty of luminaries buried there — John “Pilgrim’s Progress” Bunyan, William Blake, Daniel Defoe, and assorted Cromwells. But my favorite tomb is that of Thomas Bayes, the 18th-century statistician for whom Bayesian filtering is named. Bayesian filtering is plenty useful. Here’s a simple example of how you might use a Bayesian filter. First, get a giant load of non-spam emails and feed them into a Bayesian program that counts how many times each word in their vocabulary appears, producing a statistical breakdown of the word-frequency in good emails. Then, point the filter at a giant load of spam (if you’re having a hard time getting a hold of one, I have plenty to spare), and count the words in it.


Demystifying Smart Cities by Anders Lisdorf

3D printing, artificial general intelligence, autonomous vehicles, bitcoin, business intelligence, business process, chief data officer, clean water, cloud computing, computer vision, continuous integration, crowdsourcing, data is the new oil, digital twin, distributed ledger, don't be evil, Elon Musk, en.wikipedia.org, facts on the ground, Google Glasses, income inequality, Infrastructure as a Service, Internet of things, Masdar, microservices, Minecraft, platform as a service, ransomware, RFID, ride hailing / ride sharing, risk tolerance, self-driving car, smart cities, smart meter, software as a service, speech recognition, Stephen Hawking, Steve Jobs, Steve Wozniak, Stuxnet, Thomas Bayes, Turing test, urban sprawl, zero-sum game

Use cases for cities would be in computer vision where images could be converted into counts of pedestrians, cars, and bicycles or for allowing speech interaction with city services. The resident could speak, and the input converted into text that could then be processed as questions and answers. Naïve Bayes – Based on Bayes’ theorem, the naïve Bayes algorithm aims to make probabilistic classification based on prior knowledge. Thomas Bayes was an Eighteenth-century English minister and philosopher who first proposed to use conditional probability. The basic assumption is that we can use prior knowledge to determine probabilities. If we know that smoking is an important factor in the probability of developing respiratory illnesses, it can be used to give us a probability of a given person developing a respiratory illness based on his or her smoking habits.


pages: 923 words: 163,556

Advanced Stochastic Models, Risk Assessment, and Portfolio Optimization: The Ideal Risk, Uncertainty, and Performance Measures by Frank J. Fabozzi

algorithmic trading, Benoit Mandelbrot, capital asset pricing model, collateralized debt obligation, correlation coefficient, distributed generation, diversified portfolio, fixed income, index fund, Louis Bachelier, Myron Scholes, p-value, quantitative trading / quantitative finance, random walk, risk-adjusted returns, short selling, stochastic volatility, Thomas Bayes, transaction costs, value at risk

The Law of Total Probability for More than Two Events The expression for the Law of Total Probability in equation (15.6) can easily be generalized to the case of K events: The only “catch” is that we need to take care that the events B1, B2, …, BK exhaust the sample space (their probabilities sum up to 1) and are mutually exclusive. (In the two-event case, B and Bc clearly fulfil these requirements.) BAYES’ RULE Bayes’ rule, named after the eighteenth-century British mathematician Thomas Bayes, provides a method for expressing an unknown conditional probability P(B|A) with the help of the known conditional probability P(A|B). Bayes’ rule, for events A and B, is given by the following expression Another formulation of Bayes’ rule is by using the Law of Total Probability in the denominator in place of P(A). Doing so, we obtain Generalized to K events, Bayes’ rule is written as where the subscript i denotes the i-th event and i = 1, 2, …, K. using the Law of Total Probability, we have Illustration: Application of Bayes’ Rule The hedge fund industry has experienced exceptional growth, with assets under management more than doubling between 2005 and 2007.

See Autoregressive moving average Asian currency crisis Asset returns cross-sectional collection heavy tails joint distribution modeling, log-normal distribution application Assets dependence structures, complexity duration estimate portfolio Asymmetric confidence interval, construction Asymmetric density function Asymmetry, modeling capability Atoms Augmented regression Autocorrelation. See also Negative autocorrelation; Positive autocorrelation detection presence, modeling Autoregressive conditional heteroskedasticity (ARCH) model generalization Autoregressive moving average (ARMA) models Autoregressive of order one Autoregressive process (AR) of order p Axiomatic system Bar chart usage Bayes, Thomas Bayes’ formula Bayes’ rule application Bell-shaped curve, real-life distribution Bell-shaped density Bell-shaped yield Benchmark, derivation Berkshire Hathway pie charts, comparison revenues, third quarter (pie chart) third quarter reports third quarter revenues bar charts pie chart Bermuda call option Bernoulli distributed random variable, mean/variance Bernoulli distribution generalization p parameter, estimation random variables, relationship Bernoulli population, sample mean (histogram) Bernoulli random variables distribution Bernoulli samples, examination Bernoulli trials, link Bessel function of the third kind defining Best linear unbiased estimator (BLUE) Beta distribution distribution function Beta factor Beta function Bias.


Programming Computer Vision with Python by Jan Erik Solem

augmented reality, computer vision, database schema, en.wikipedia.org, optical character recognition, pattern recognition, text mining, Thomas Bayes, web application

If you want to dive deeper into classifiers and machine learning algorithms, take a look at the scikit.learn package (http://scikit-learn.org/) and try some of the algorithms on the data in this chapter. * * * [21] Another option is to only keep a selected subset of the training set. This can, however, impact accuracy. [22] Another common name is Histogram of Oriented Gradients (HOG). [23] After Thomas Bayes, an 18th-century English mathematician and minister. [24] See http://en.wikipedia.org/wiki/Sudoku for more details if you are unfamiliar with the concept. [25] Images courtesy of Martin Byröd [4], http://www.maths.lth.se/matematiklth/personal/byrod/, collected and cropped from photos of actual Sudokus. Chapter 9. Image Segmentation Image segmentation is the process of partitioning an image into meaningful regions.


pages: 229 words: 67,599

The Logician and the Engineer: How George Boole and Claude Shannon Created the Information Age by Paul J. Nahin

Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, Any sufficiently advanced technology is indistinguishable from magic, Claude Shannon: information theory, conceptual framework, Edward Thorp, Fellow of the Royal Society, finite state, four colour theorem, Georg Cantor, Grace Hopper, Isaac Newton, John von Neumann, knapsack problem, New Journalism, Pierre-Simon Laplace, reversible computing, Richard Feynman, Schrödinger's Cat, Steve Jobs, Steve Wozniak, thinkpad, Thomas Bayes, Turing machine, Turing test, V2 rocket

The “general doctrine” does have a sort of plausibility to it: “if Y then not X” when “reversed” could be thought to imply “if X then not Y.” Boole argued that this is not so, using the ideas of the previous section, and showed that P( | X) is given by a considerably more involved expression than simply “p.” What Boole did was not really original, as conditional probability had been studied a century before by the English philosopher and minister Thomas Bayes (1701–1761), whose work was published posthumously in 1764 in the Philosophical Transactions of the Royal Society of London, where it was then promptly forgotten for twenty years until the great French mathematician Pierre-Simon Laplace (1749–1827) endorsed Bayes’s results. What Boole did, then, with the following analysis, was remind his readers what the Reverend Bayes had done a hundred years before.


pages: 250 words: 64,011

Everydata: The Misinformation Hidden in the Little Data You Consume Every Day by John H. Johnson

Affordable Care Act / Obamacare, Black Swan, business intelligence, Carmen Reinhart, cognitive bias, correlation does not imply causation, Daniel Kahneman / Amos Tversky, Donald Trump, en.wikipedia.org, Kenneth Rogoff, labor-force participation, lake wobegon effect, Long Term Capital Management, Mercator projection, Mercator projection distort size, especially Greenland and Africa, meta analysis, meta-analysis, Nate Silver, obamacare, p-value, PageRank, pattern recognition, publication bias, QR code, randomized controlled trial, risk-adjusted returns, Ronald Reagan, selection bias, statistical model, The Signal and the Noise by Nate Silver, Thomas Bayes, Tim Cook: Apple, wikimedia commons, Yogi Berra

Ellen Davis, “Committing the ‘Gambler’s Fallacy’ May Be in the Cards, New Research Shows,” Texas A&M Health Science Center website, March 9, 2015, http://news.tamhsc.edu/?post=committing-the-gamblers-fallacy-may-be-in-the-cards-new-research-shows. Thanks to Ron Friedman for the find. 27. There’s another way of looking at this, known as Bayesian probability (after the eighteenth-century English mathematician Thomas Bayes). With Bayesian probability, you use the data gathered to update your initial beliefs after the fact. It’s the opposite of the way in which the gambler’s fallacy works. As one of John’s colleagues pointed out, it’s the difference between knowing that a coin is fair and learning about the coin. So, a Bayesian might flip a coin 10 times, get heads all 10 times, and adjust his probability to say that the coin was always more likely to land heads up.


pages: 269 words: 74,955

The Crash Detectives: Investigating the World's Most Mysterious Air Disasters by Christine Negroni

Air France Flight 447, Airbus A320, Captain Sullenberger Hudson, Charles Lindbergh, Checklist Manifesto, computer age, crew resource management, crowdsourcing, low cost airline, low cost carrier, Richard Feynman, South China Sea, Tenerife airport disaster, Thomas Bayes, US Airways Flight 1549

Some of the credit for finally finding the submerged airliner goes to Metron Scientific Solutions, a company staffed with pencil-wielding mathematicians who used probability, logic, and numbers to conclude that the likely resting place of the plane was a narrow slice of ocean that had already been checked. “A lack of success tells you about where it is not, and that contributes to knowledge,” said Larry Stone, chief scientist at Metron. Talk about having a positive point of view. The Metron method is based on Bayesian probability, the theory of eighteenth-century statistician and philosopher Thomas Bayes, whose first published work, Divine Benevolence, was equally optimistic because it attempted to prove that God wants us to be happy. Using Bayesian logic to look for missing airplanes, as interpreted by Metron, involves taking all kinds of input about the missing thing (even conflicting input) and assigning levels of certainty or uncertainty to each. Everything gets a weight, and everything gets revised as things change.


pages: 296 words: 78,631

Hello World: Being Human in the Age of Algorithms by Hannah Fry

23andMe, 3D printing, Air France Flight 447, Airbnb, airport security, augmented reality, autonomous vehicles, Brixton riot, chief data officer, computer vision, crowdsourcing, DARPA: Urban Challenge, Douglas Hofstadter, Elon Musk, Firefox, Google Chrome, Gödel, Escher, Bach, Ignaz Semmelweis: hand washing, John Markoff, Mark Zuckerberg, meta analysis, meta-analysis, pattern recognition, Peter Thiel, RAND corporation, ransomware, recommendation engine, ride hailing / ride sharing, selection bias, self-driving car, Shai Danziger, Silicon Valley, Silicon Valley startup, Snapchat, speech recognition, Stanislav Petrov, statistical model, Stephen Hawking, Steven Levy, Tesla Model S, The Wisdom of Crowds, Thomas Bayes, Watson beat the top human players on Jeopardy!, web of trust, William Langewiesche

But where the Bayesian way of thinking really comes into its own is when you’re trying to consider more than one hypothesis simultaneously – for example, in attempting to diagnose what’s wrong with a patient on the basis of their symptoms,* or finding the position of a driverless car on the basis of sensor readings. In theory, any disease, any point on the map, could represent the underlying truth. All you need to do is weigh up the evidence to decide which is most likely to be right. And on that point, finding the location of a driverless car turns out to be rather similar to a problem that puzzled Thomas Bayes, the British Presbyterian minister and talented mathematician after whom the theorem is named. Back in the mid-1700s, he wrote an essay which included details of a game he’d devised to explain the problem. It went something a little like this:32 Imagine you’re sitting with your back to a square table. Without you seeing, I throw a red ball on to the table. Your job is to guess where it landed.


pages: 277 words: 87,082

Beyond Weird by Philip Ball

Albert Einstein, Bayesian statistics, cosmic microwave background, dark matter, dematerialisation, Ernest Rutherford, experimental subject, Isaac Newton, John von Neumann, Kickstarter, Murray Gell-Mann, Richard Feynman, Schrödinger's Cat, Stephen Hawking, theory of mind, Thomas Bayes

Here, all quantum mechanics refers to are beliefs about outcomes – beliefs that are individual to each observer. Those beliefs do not become realized as facts until they impinge on the consciousness of the observer – and so the facts are specific to every observer (although different observers can find themselves agreeing on the same facts). This notion takes its cue from standard Bayesian probability theory, introduced in the eighteenth century by the English mathematician and clergyman Thomas Bayes. In Bayesian statistics, probabilities are not defined with reference to some objective state of affairs in the world, but instead quantify personal degrees of belief of what might happen – which we update as we acquire new information. The QBist view, however, says something much more profound than simply that different people know different things. Rather, it asserts that there are no things that can be meaningfully spoken of beyond the self.


pages: 301 words: 85,126

AIQ: How People and Machines Are Smarter Together by Nick Polson, James Scott

Air France Flight 447, Albert Einstein, Amazon Web Services, Atul Gawande, autonomous vehicles, availability heuristic, basic income, Bayesian statistics, business cycle, Cepheid variable, Checklist Manifesto, cloud computing, combinatorial explosion, computer age, computer vision, Daniel Kahneman / Amos Tversky, Donald Trump, Douglas Hofstadter, Edward Charles Pickering, Elon Musk, epigenetics, Flash crash, Grace Hopper, Gödel, Escher, Bach, Harvard Computers: women astronomers, index fund, Isaac Newton, John von Neumann, late fees, low earth orbit, Lyft, Magellanic Cloud, mass incarceration, Moneyball by Michael Lewis explains big data, Moravec's paradox, more computing power than Apollo, natural language processing, Netflix Prize, North Sea oil, p-value, pattern recognition, Pierre-Simon Laplace, ransomware, recommendation engine, Ronald Reagan, self-driving car, sentiment analysis, side project, Silicon Valley, Skype, smart cities, speech recognition, statistical model, survivorship bias, the scientific method, Thomas Bayes, Uber for X, uber lyft, universal basic income, Watson beat the top human players on Jeopardy!, young professional

Bayes’s Rule, from Reverend to Robot Here’s the key mantra we must take away from the story of the Scorpion: all probabilities are really conditional probabilities. In other words, all probabilities are contingent upon what we know. When our knowledge changes, our probabilities must change, too—and Bayes’s rule tells us how to change them. Bayes’s rule was discovered by an obscure English clergyman named Thomas Bayes. Born in 1701 to a Presbyterian family in London, Bayes showed an early talent for mathematics, but he came of age at a time when religious dissenters were barred from universities in England. Denied the chance to study math at Oxford or Cambridge, he ended up studying theology at the University of Edinburgh instead. This must have seemed like a cruel barrier to Bayes, just as it did to so many others of his era.


pages: 246 words: 81,625

On Intelligence by Jeff Hawkins, Sandra Blakeslee

airport security, Albert Einstein, computer age, conceptual framework, Johannes Kepler, Necker cube, pattern recognition, Paul Erdős, Ray Kurzweil, Silicon Valley, Silicon Valley startup, speech recognition, superintelligent machines, the scientific method, Thomas Bayes, Turing machine, Turing test

As an example, in his 2001 book, i of the vortex, Rodolfo Llinas, at the New York University School of Medicine, wrote, "The capacity to predict the outcome of future events— critical to successful movement— is, most likely, the ultimate and most common of all global brain functions." Scientists such as David Mumford at Brown University, Rajesh Rao at the University of Washington, Stephen Grossberg at Boston University, and many more have written and theorized about the role of feedback and prediction in various ways. There is an entire subfield of mathematics devoted to Bayesian networks. Named after Thomas Bayes, an English minister born in 1702 who was a pioneer in statistics, Bayesian networks use probability theory to make predictions. What has been lacking is putting these disparate bits and pieces into a coherent theoretical framework. This, I argue, has not been done before, and it is the goal of this book. * * * Before we get into detail about how the cortex makes predictions, let's consider some additional examples.


pages: 294 words: 81,292

Our Final Invention: Artificial Intelligence and the End of the Human Era by James Barrat

AI winter, AltaVista, Amazon Web Services, artificial general intelligence, Asilomar, Automated Insights, Bayesian statistics, Bernie Madoff, Bill Joy: nanobots, brain emulation, cellular automata, Chuck Templeton: OpenTable:, cloud computing, cognitive bias, commoditize, computer vision, cuban missile crisis, Daniel Kahneman / Amos Tversky, Danny Hillis, data acquisition, don't be evil, drone strike, Extropian, finite state, Flash crash, friendly AI, friendly fire, Google Glasses, Google X / Alphabet X, Isaac Newton, Jaron Lanier, John Markoff, John von Neumann, Kevin Kelly, Law of Accelerating Returns, life extension, Loebner Prize, lone genius, mutually assured destruction, natural language processing, Nicholas Carr, optical character recognition, PageRank, pattern recognition, Peter Thiel, prisoner's dilemma, Ray Kurzweil, Rodney Brooks, Search for Extraterrestrial Intelligence, self-driving car, semantic web, Silicon Valley, Singularitarianism, Skype, smart grid, speech recognition, statistical model, stealth mode startup, stem cell, Stephen Hawking, Steve Jobs, Steve Wozniak, strong AI, Stuxnet, superintelligent machines, technological singularity, The Coming Technological Singularity, Thomas Bayes, traveling salesman, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, zero day

But by the time the tragedy unfolded, Holtzman told me, Good had retired. He was not in his office but at home, perhaps calculating the probability of God’s existence. According to Dr. Holtzman, sometime before he died, Good updated that probability from zero to point one. He did this because as a statistician, he was a long-term Bayesian. Named for the eighteenth-century mathematician and minister Thomas Bayes, Bayesian statistics’ main idea is that in calculating the probability of some statement, you can start with a personal belief. Then you update that belief as new evidence comes in that supports your statement or doesn’t. If Good’s original disbelief in God had remained 100 percent, no amount of data, not even God’s appearance, could change his mind. So, to be consistent with his Bayesian perspective, Good assigned a small positive probability to the existence of God to make sure he could learn from new data, if it arose.


pages: 761 words: 231,902

The Singularity Is Near: When Humans Transcend Biology by Ray Kurzweil

additive manufacturing, AI winter, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, anthropic principle, Any sufficiently advanced technology is indistinguishable from magic, artificial general intelligence, Asilomar, augmented reality, autonomous vehicles, Benoit Mandelbrot, Bill Joy: nanobots, bioinformatics, brain emulation, Brewster Kahle, Brownian motion, business cycle, business intelligence, c2.com, call centre, carbon-based life, cellular automata, Claude Shannon: information theory, complexity theory, conceptual framework, Conway's Game of Life, coronavirus, cosmological constant, cosmological principle, cuban missile crisis, data acquisition, Dava Sobel, David Brooks, Dean Kamen, disintermediation, double helix, Douglas Hofstadter, en.wikipedia.org, epigenetics, factory automation, friendly AI, George Gilder, Gödel, Escher, Bach, informal economy, information retrieval, invention of the telephone, invention of the telescope, invention of writing, iterative process, Jaron Lanier, Jeff Bezos, job automation, job satisfaction, John von Neumann, Kevin Kelly, Law of Accelerating Returns, life extension, lifelogging, linked data, Loebner Prize, Louis Pasteur, mandelbrot fractal, Marshall McLuhan, Mikhail Gorbachev, Mitch Kapor, mouse model, Murray Gell-Mann, mutually assured destruction, natural language processing, Network effects, new economy, Norbert Wiener, oil shale / tar sands, optical character recognition, pattern recognition, phenotype, premature optimization, randomized controlled trial, Ray Kurzweil, remote working, reversible computing, Richard Feynman, Robert Metcalfe, Rodney Brooks, scientific worldview, Search for Extraterrestrial Intelligence, selection bias, semantic web, Silicon Valley, Singularitarianism, speech recognition, statistical model, stem cell, Stephen Hawking, Stewart Brand, strong AI, superintelligent machines, technological singularity, Ted Kaczynski, telepresence, The Coming Technological Singularity, Thomas Bayes, transaction costs, Turing machine, Turing test, Vernor Vinge, Y2K, Yogi Berra

He plans to develop a system incorporating all human ideas.167 One application would be to inform policy makers of which ideas are held by which community. Bayesian Nets. Over the last decade a technique called Bayesian logic has created a robust mathematical foundation for combining thousands or even millions of such probabilistic rules in what are called "belief networks" or Bayesian nets. Originally devised by English mathematician Thomas Bayes and published posthumously in 1763, the approach is intended to determine the likelihood of future events based on similar occurrences in the past.168 Many expert systems based on Bayesian techniques gather data from experience in an ongoing fashion, thereby continually learning and improving their decision making. The most promising type of spam filters are based on this method. I personally use a spam filter called SpamBayes, which trains itself on e-mail that you have identified as either "spam" or "okay."169 You start out by presenting a folder of each to the filter.

Anthes, "Computerizing Common Sense," Computerworld, April 8, 2002, http://www.computerworld.com/news/2002/story/0,11280,69881,00.html. 167. Kristen Philipkoski, "Now Here's a Really Big Idea," Wired News, November 25, 2002, http://www.wired.com/news/technology/0,1282,56374,00.html, reporting on Darryl Macer, "The Next Challenge Is to Map the Human Mind," Nature 420 (November 14, 2002): 121; see also a description of the project at http://www.biol.tsukuba.ac.jp/~macer/index.html. 168. Thomas Bayes, "An Essay Towards Solving a Problem in the Doctrine of Chances," published in 1763, two years after his death in 1761. 169. SpamBayes spam filter, http://spambayes.sourceforge.net. 170. Lawrence R. Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition," Proceedings of the IEEE 77 (1989): 257–86. For a mathematical treatment of Markov models, see http://jedlik.phy.bme.hu/~gerjanos/HMM/node2.html. 171.


pages: 370 words: 94,968

The Most Human Human: What Talking With Computers Teaches Us About What It Means to Be Alive by Brian Christian

4chan, Ada Lovelace, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Bertrand Russell: In Praise of Idleness, carbon footprint, cellular automata, Claude Shannon: information theory, cognitive dissonance, commoditize, complexity theory, crowdsourcing, David Heinemeier Hansson, Donald Trump, Douglas Hofstadter, George Akerlof, Gödel, Escher, Bach, high net worth, Isaac Newton, Jacques de Vaucanson, Jaron Lanier, job automation, l'esprit de l'escalier, Loebner Prize, Menlo Park, Ray Kurzweil, RFID, Richard Feynman, Ronald Reagan, Skype, Social Responsibility of Business Is to Increase Its Profits, starchitect, statistical model, Stephen Hawking, Steve Jobs, Steven Pinker, Thales of Miletus, theory of mind, Thomas Bayes, Turing machine, Turing test, Von Neumann architecture, Watson beat the top human players on Jeopardy!, zero-sum game

I walk out of the Brighton Centre, to the bracing sea air for a minute, and into a small, locally owned shoe store looking for a gift to bring back home to my girlfriend; the shopkeeper notices my accent; I tell her I’m from Seattle; she is a grunge fan; I comment on the music playing in the store; she says it’s Florence + the Machine; I tell her I like it and that she would probably like Feist … I walk into a tea and scone store called the Mock Turtle and order the British equivalent of coffee and a donut, except it comes with thirteen pieces of silverware and nine pieces of flatware; I am so in England, I think; an old man, probably in his eighties, is shakily eating a pastry the likes of which I’ve never seen; I ask him what it is; “coffee meringue,” he says and remarks on my accent; an hour later he is telling me about World War II, the exponentially increasing racial diversity of Britain, that House of Cards is a pretty accurate depiction of British politics, minus the murders, but that really I should watch Spooks; do you get Spooks on cable, he is asking me … I meet my old boss for dinner; and after a couple years of being his research assistant and occasionally co-author, and after a brief thought of becoming one of his Ph.D. students, after a year of our paths not really crossing, we negotiate whether our formerly collegial and hierarchical relationship, now that its context is removed, simply dries up or flourishes into a domain-general friendship; we are ordering appetizers and saying something about Wikipedia, something about Thomas Bayes, something about vegetarian dining … Laurels are of no use. If you de-anonymized yourself in the past, great. But that was that. And now, you begin again. 1. These logs would, three years later, be put on the IBM website, albeit in incomplete form and with so little fanfare that Kasparov himself wouldn’t find out about them until 2005. Epilogue: The Unsung Beauty of the Glassware Cabinet The Most Room-Like Room: The Cornell Box The image-processing world, it turns out, has a close analogue to the Turing test, called “the Cornell box,” which is a small model of a room with one red wall and one green wall (the others are white) and two blocks sitting inside it.


Gaming the Vote: Why Elections Aren't Fair (And What We Can Do About It) by William Poundstone

affirmative action, Albert Einstein, business cycle, Debian, desegregation, Donald Trump, en.wikipedia.org, Everything should be made as simple as possible, global village, guest worker program, hiring and firing, illegal immigration, invisible hand, jimmy wales, John Nash: game theory, John von Neumann, Kenneth Arrow, manufacturing employment, Nash equilibrium, Paul Samuelson, Pierre-Simon Laplace, prisoner's dilemma, Ralph Nader, RAND corporation, Ronald Reagan, Silicon Valley, slashdot, the map is not the territory, Thomas Bayes, transcontinental railway, Unsafe at Any Speed, Y2K

Smith holds or has applied for patents covering such exotica as a computer made out of DNA, theft-proof credit cards, a 3-D vision process, and a magnetic catapult that could be used for launching satellites, In December 2000, with the Supreme Court deciding a bitterly contested presidency, Smith completed an article purporting to demonstrate the superiority of a system that no one had taken seriously, range voting. He began with an idea for comparing the merits of different voting systems, using a measure called Bayesian regret. The "Bayes" part refers to eighteenth-century English mathematician Thomas Bayes, a pioneer of probability theory, "Bayesian regret" is a statistical term that Smith defines as "expected avoidable human unhappiness." In other words, Smith tried to gauge how voting systems fail the voters by electing candidates other than the one who would have resulted in the greatest overall satisfaction, To do this, he ran a large series of computer simulations of elections. In each of his simulations, virtual voters were assigned utilities (degrees of happiness, measured numerically) for simulated candidates.


pages: 317 words: 100,414

Superforecasting: The Art and Science of Prediction by Philip Tetlock, Dan Gardner

Affordable Care Act / Obamacare, Any sufficiently advanced technology is indistinguishable from magic, availability heuristic, Black Swan, butterfly effect, buy and hold, cloud computing, cuban missile crisis, Daniel Kahneman / Amos Tversky, desegregation, drone strike, Edward Lorenz: Chaos theory, forward guidance, Freestyle chess, fundamental attribution error, germ theory of disease, hindsight bias, index fund, Jane Jacobs, Jeff Bezos, Kenneth Arrow, Laplace demon, longitudinal study, Mikhail Gorbachev, Mohammed Bouazizi, Nash equilibrium, Nate Silver, Nelson Mandela, obamacare, pattern recognition, performance metric, Pierre-Simon Laplace, place-making, placebo effect, prediction markets, quantitative easing, random walk, randomized controlled trial, Richard Feynman, Richard Thaler, Robert Shiller, Robert Shiller, Ronald Reagan, Saturday Night Live, scientific worldview, Silicon Valley, Skype, statistical model, stem cell, Steve Ballmer, Steve Jobs, Steven Pinker, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Watson beat the top human players on Jeopardy!

If he says, “It’s to the left,” the likelihood of the first ball being on the right side of the table increases a little more. Keep repeating the process and you slowly narrow the range of the possible locations, zeroing in on the truth—although you will never eliminate uncertainty entirely.16 If you’ve taken Statistics 101, you may recall a version of this thought experiment was dreamt up by Thomas Bayes. A Presbyterian minister, educated in logic, Bayes was born in 1701, so he lived at the dawn of modern probability theory, a subject to which he contributed with “An Essay Towards Solving a Problem in the Doctrine of Chances.” That essay, in combination with the work of Bayes’ friend Richard Price, who published Bayes’ essay posthumously in 1761, and the insights of the great French mathematician Pierre-Simon Laplace, ultimately produced Bayes’ theorem.


pages: 350 words: 103,270

The Devil's Derivatives: The Untold Story of the Slick Traders and Hapless Regulators Who Almost Blew Up Wall Street . . . And Are Ready to Do It Again by Nicholas Dunbar

asset-backed security, bank run, banking crisis, Basel III, Black Swan, Black-Scholes formula, bonus culture, break the buck, buy and hold, capital asset pricing model, Carmen Reinhart, Cass Sunstein, collateralized debt obligation, commoditize, Credit Default Swap, credit default swaps / collateralized debt obligations, delayed gratification, diversification, Edmond Halley, facts on the ground, financial innovation, fixed income, George Akerlof, implied volatility, index fund, interest rate derivative, interest rate swap, Isaac Newton, John Meriwether, Kenneth Rogoff, Kickstarter, Long Term Capital Management, margin call, market bubble, money market fund, Myron Scholes, Nick Leeson, Northern Rock, offshore financial centre, Paul Samuelson, price mechanism, regulatory arbitrage, rent-seeking, Richard Thaler, risk tolerance, risk/return, Ronald Reagan, shareholder value, short selling, statistical model, The Chicago School, Thomas Bayes, time value of money, too big to fail, transaction costs, value at risk, Vanguard fund, yield curve, zero-sum game

Note that we ignore any mention of time, or the time value of money, in this example, which is the equivalent of setting the risk-free interest rate to zero. 6. One might argue that since the market values the loans at $800 million, the bank ought to write down the value of the equity investment to zero. However, accounting rules for loan books don’t require such recognitions to take place. 7. After de Moivre’s death, the refinement of mortality calculations was continued in London by Richard Price, friend of Thomas Bayes and Benjamin Franklin, and founding actuary of the Equitable Life Assurance Society. 8. Arturo Cifuentes and Gerard O’Connor, “The Binomial Expansion Method Applied to CBO/CLO Analysis,” Moody’s Investors Service special report, December 13, 1996. 9. Ibid. 10. For a detailed account of the invention of BISTRO, see Gillian Tett, Fool’s Gold: How the Bold Dream of a Small Tribe at J.P. Morgan Was Corrupted by Wall Street Greed and Unleashed a Catastrophe (New York: Free Press, 2009). 11.


pages: 339 words: 105,938

The Skeptical Economist: Revealing the Ethics Inside Economics by Jonathan Aldred

airport security, Berlin Wall, carbon footprint, citizen journalism, clean water, cognitive dissonance, congestion charging, correlation does not imply causation, Diane Coyle, endogenous growth, experimental subject, Fall of the Berlin Wall, first-past-the-post, framing effect, greed is good, happiness index / gross national happiness, hedonic treadmill, Intergovernmental Panel on Climate Change (IPCC), invisible hand, job satisfaction, John Maynard Keynes: Economic Possibilities for our Grandchildren, labour market flexibility, laissez-faire capitalism, libertarian paternalism, longitudinal study, new economy, Pareto efficiency, pension reform, positional goods, Ralph Waldo Emerson, RAND corporation, risk tolerance, school choice, spectrum auction, Thomas Bayes, trade liberalization, ultimatum game

This is a very broad question, so we shall focus on just one aspect of it, namely the practice of quantifying uncertainty and ignorance — inventing probabilities when there is no basis to do so. The practice is widespread among economists because many of them believe that, no matter how extreme the uncertainty, effective probabilities always exist. This view is termed ‘subjective Bayesianism’ (hereafter Bayesianism for short), from the Reverend Thomas Bayes, an 18th-century English mathematician.35 Its implications are startling. Bayesians believe there is no such thing as pure uncertainty in the sense I have defined it. They assert that we always use probabilities, consciously or otherwise, when outcomes are not certain. The issues are more clearly depicted in simple gambling games than messy real-world choices; the Ellsberg Paradox (see box opposite) is a classic illustration.


pages: 519 words: 102,669

Programming Collective Intelligence by Toby Segaran

always be closing, correlation coefficient, Debian, en.wikipedia.org, Firefox, full text search, information retrieval, PageRank, prediction markets, recommendation engine, slashdot, Thomas Bayes, web application

In docclass.py, create a subclass of classifier called naivebayes, and create a docprob method that extracts the features (words) and multiplies all their probabilities together to get an overall probability: class naivebayes(classifier): def docprob(self,item,cat): features=self.getfeatures(item) # Multiply the probabilities of all the features together p=1 for f in features: p*=self.weightedprob(f,cat,self.fprob) return p You now know how to calculate Pr(Document | Category), but this isn't very useful by itself. In order to classify documents, you really need Pr(Category | Document). In other words, given a specific document, what's the probability that it fits into this category? Fortunately, a British mathematician named Thomas Bayes figured out how to do this about 250 years ago. A Quick Introduction to Bayes' Theorem Bayes' Theorem is a way of flipping around conditional probabilities. It's usually written as: Pr(A | B) = Pr(B | A) × Pr(A)/Pr(B) In the example, this becomes: Pr(Category | Document) = Pr(Document | Category) × Pr(Category) / Pr(Document) The previous section showed how to calculate Pr(Document | Category), but what about the other two values in the equation?


The Deep Learning Revolution (The MIT Press) by Terrence J. Sejnowski

AI winter, Albert Einstein, algorithmic trading, Amazon Web Services, Any sufficiently advanced technology is indistinguishable from magic, augmented reality, autonomous vehicles, Baxter: Rethink Robotics, bioinformatics, cellular automata, Claude Shannon: information theory, cloud computing, complexity theory, computer vision, conceptual framework, constrained optimization, Conway's Game of Life, correlation does not imply causation, crowdsourcing, Danny Hillis, delayed gratification, discovery of DNA, Donald Trump, Douglas Engelbart, Drosophila, Elon Musk, en.wikipedia.org, epigenetics, Flynn Effect, Frank Gehry, future of work, Google Glasses, Google X / Alphabet X, Guggenheim Bilbao, Gödel, Escher, Bach, haute couture, Henri Poincaré, I think there is a world market for maybe five computers, industrial robot, informal economy, Internet of things, Isaac Newton, John Conway, John Markoff, John von Neumann, Mark Zuckerberg, Minecraft, natural language processing, Netflix Prize, Norbert Wiener, orbital mechanics / astrodynamics, PageRank, pattern recognition, prediction markets, randomized controlled trial, recommendation engine, Renaissance Technologies, Rodney Brooks, self-driving car, Silicon Valley, Silicon Valley startup, Socratic dialogue, speech recognition, statistical model, Stephen Hawking, theory of mind, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, traveling salesman, Turing machine, Von Neumann architecture, Watson beat the top human players on Jeopardy!, X Prize, Yogi Berra

Tomaso Poggio had developed a hierarchical network called “HMAX” that could classify a limited number of objects.2 This suggested that performance would improve with deeper networks. In the first years of the new century, graphical models were developed that made contact with a rich vein of probabilistic models called “Bayes networks,” based on a theorem formulated by the eighteenth-century British mathematician Thomas Bayes that allows new evidence to update prior beliefs. Judea Pearl at the University of California, Los Angeles, had earlier introduced “belief networks”3 to artificial intelligence based on Bayesian analysis, which were strengthened and extended by developing methods for learning the probabilities in the networks from data. The algorithms of these and other networks built up a powerful armamentarium for machine learning researchers.


pages: 407 words: 104,622

The Man Who Solved the Market: How Jim Simons Launched the Quant Revolution by Gregory Zuckerman

affirmative action, Affordable Care Act / Obamacare, Albert Einstein, Andrew Wiles, automated trading system, backtesting, Bayesian statistics, beat the dealer, Benoit Mandelbrot, Berlin Wall, Bernie Madoff, blockchain, Brownian motion, butter production in bangladesh, buy and hold, buy low sell high, Claude Shannon: information theory, computer age, computerized trading, Credit Default Swap, Daniel Kahneman / Amos Tversky, diversified portfolio, Donald Trump, Edward Thorp, Elon Musk, Emanuel Derman, endowment effect, Flash crash, George Gilder, Gordon Gekko, illegal immigration, index card, index fund, Isaac Newton, John Meriwether, John Nash: game theory, John von Neumann, Loma Prieta earthquake, Long Term Capital Management, loss aversion, Louis Bachelier, mandelbrot fractal, margin call, Mark Zuckerberg, More Guns, Less Crime, Myron Scholes, Naomi Klein, natural language processing, obamacare, p-value, pattern recognition, Peter Thiel, Ponzi scheme, prediction markets, quantitative hedge fund, quantitative trading / quantitative finance, random walk, Renaissance Technologies, Richard Thaler, Robert Mercer, Ronald Reagan, self-driving car, Sharpe ratio, Silicon Valley, sovereign wealth fund, speech recognition, statistical arbitrage, statistical model, Steve Jobs, stochastic process, the scientific method, Thomas Bayes, transaction costs, Turing machine

To do that, the IBM researchers employed the Baum-Welch algorithm—codeveloped by Jim Simons’s early trading partner Lenny Baum—to zero in on the various language probabilities. Rather than manually programming in static knowledge about how language worked, they created a program that learned from data. Brown, Mercer, and the others relied upon Bayesian mathematics, which had emerged from the statistical rule proposed by Reverend Thomas Bayes in the eighteenth-century. Bayesians will attach a degree of probability to every guess and update their best estimates as they receive new information. The genius of Bayesian statistics is that it continuously narrows a range of possibilities. Think, for example, of a spam filter, which doesn’t know with certainty if an email is malicious, but can be effective by assigning odds to each one received by constantly learning from emails previously classified as “junk.”


pages: 354 words: 105,322

The Road to Ruin: The Global Elites' Secret Plan for the Next Financial Crisis by James Rickards

"Robert Solow", Affordable Care Act / Obamacare, Albert Einstein, asset allocation, asset-backed security, bank run, banking crisis, barriers to entry, Bayesian statistics, Ben Bernanke: helicopter money, Benoit Mandelbrot, Berlin Wall, Bernie Sanders, Big bang: deregulation of the City of London, bitcoin, Black Swan, blockchain, Bonfire of the Vanities, Bretton Woods, British Empire, business cycle, butterfly effect, buy and hold, capital controls, Capital in the Twenty-First Century by Thomas Piketty, Carmen Reinhart, cellular automata, cognitive bias, cognitive dissonance, complexity theory, Corn Laws, corporate governance, creative destruction, Credit Default Swap, cuban missile crisis, currency manipulation / currency intervention, currency peg, Daniel Kahneman / Amos Tversky, David Ricardo: comparative advantage, debt deflation, Deng Xiaoping, disintermediation, distributed ledger, diversification, diversified portfolio, Edward Lorenz: Chaos theory, Eugene Fama: efficient market hypothesis, failed state, Fall of the Berlin Wall, fiat currency, financial repression, fixed income, Flash crash, floating exchange rates, forward guidance, Fractional reserve banking, G4S, George Akerlof, global reserve currency, high net worth, Hyman Minsky, income inequality, information asymmetry, interest rate swap, Isaac Newton, jitney, John Meriwether, John von Neumann, Joseph Schumpeter, Kenneth Rogoff, labor-force participation, large denomination, liquidity trap, Long Term Capital Management, mandelbrot fractal, margin call, market bubble, Mexican peso crisis / tequila crisis, money market fund, mutually assured destruction, Myron Scholes, Naomi Klein, nuclear winter, obamacare, offshore financial centre, Paul Samuelson, Peace of Westphalia, Pierre-Simon Laplace, plutocrats, Plutocrats, prediction markets, price anchoring, price stability, quantitative easing, RAND corporation, random walk, reserve currency, RFID, risk-adjusted returns, Ronald Reagan, Silicon Valley, sovereign wealth fund, special drawing rights, stocks for the long run, The Bell Curve by Richard Herrnstein and Charles Murray, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, theory of mind, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, too big to fail, transfer pricing, value at risk, Washington Consensus, Westphalian system

Yet when you enter that data into a deficient model, you get deficient output. Investors who use complexity theory can leave mainstream analysis behind and get better forecasting results. The third tool in addition to behavioral psychology and complexity theory is Bayesian statistics, a branch of etiology also referred to as causal inference. Both terms derive from Bayes’ theorem, an equation first described by Thomas Bayes and published posthumously in 1763. A version of the theorem was elaborated independently and more formally by the French mathematician Pierre-Simon Laplace in 1774. Laplace continued work on the theorem in subsequent decades. Twentieth-century statisticians have developed more rigorous forms. Normal science including economics assembles massive data sets and uses deductive methods to derive testable hypotheses from the data.


pages: 309 words: 114,984

The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age by Robert Wachter

"Robert Solow", activist fund / activist shareholder / activist investor, Affordable Care Act / Obamacare, AI winter, Airbnb, Atul Gawande, Captain Sullenberger Hudson, Checklist Manifesto, Chuck Templeton: OpenTable:, Clayton Christensen, collapse of Lehman Brothers, computer age, creative destruction, crowdsourcing, deskilling, disruptive innovation, en.wikipedia.org, Erik Brynjolfsson, everywhere but in the productivity statistics, Firefox, Frank Levy and Richard Murnane: The New Division of Labor, Google Glasses, Ignaz Semmelweis: hand washing, Internet of things, job satisfaction, Joseph Schumpeter, Kickstarter, knowledge worker, lifelogging, medical malpractice, medical residency, Menlo Park, minimum viable product, natural language processing, Network effects, Nicholas Carr, obamacare, pattern recognition, peer-to-peer, personalized medicine, pets.com, Productivity paradox, Ralph Nader, RAND corporation, Second Machine Age, self-driving car, Silicon Valley, Silicon Valley startup, six sigma, Skype, Snapchat, software as a service, Steve Jobs, Steven Levy, the payments system, The Wisdom of Crowds, Thomas Bayes, Toyota Production System, Uber for X, US Airways Flight 1549, Watson beat the top human players on Jeopardy!, Yogi Berra

This is the part of diagnostic reasoning that beginners find most vexing, since they lack the foundational knowledge to understand why their teacher focused so intently on one nugget of information and all but ignored others that, to the novice, seemed equally crucial. How do the great diagnosticians make such choices? We now recognize this as a relatively intuitive version of Bayes’ theorem. Developed by the eighteenth-century British theologian-turned-mathematician Thomas Bayes, this theorem (often ignored by students because it is taught to them with the dryness of a Passover matzo) is the linchpin of clinical reasoning. In essence, Bayes’ theorem says that any medical test must be interpreted from two perspectives. The first: How accurate is the test—that is, how often does it give right or wrong answers? The second: How likely is it that this patient has the disease the test is looking for?


pages: 396 words: 117,149

The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World by Pedro Domingos

Albert Einstein, Amazon Mechanical Turk, Arthur Eddington, basic income, Bayesian statistics, Benoit Mandelbrot, bioinformatics, Black Swan, Brownian motion, cellular automata, Claude Shannon: information theory, combinatorial explosion, computer vision, constrained optimization, correlation does not imply causation, creative destruction, crowdsourcing, Danny Hillis, data is the new oil, double helix, Douglas Hofstadter, Erik Brynjolfsson, experimental subject, Filter Bubble, future of work, global village, Google Glasses, Gödel, Escher, Bach, information retrieval, job automation, John Markoff, John Snow's cholera map, John von Neumann, Joseph Schumpeter, Kevin Kelly, lone genius, mandelbrot fractal, Mark Zuckerberg, Moneyball by Michael Lewis explains big data, Narrative Science, Nate Silver, natural language processing, Netflix Prize, Network effects, NP-complete, off grid, P = NP, PageRank, pattern recognition, phenotype, planetary scale, pre–internet, random walk, Ray Kurzweil, recommendation engine, Richard Feynman, scientific worldview, Second Machine Age, self-driving car, Silicon Valley, social intelligence, speech recognition, Stanford marshmallow experiment, statistical model, Stephen Hawking, Steven Levy, Steven Pinker, superintelligent machines, the scientific method, The Signal and the Noise by Nate Silver, theory of mind, Thomas Bayes, transaction costs, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, white flight, zero-sum game

Once we know how to do all these things, we’ll be ready to learn the Bayesian way. For Bayesians, learning is “just” another application of Bayes’ theorem, with whole models as the hypotheses and the data as the evidence: as you see more data, some models become more likely and some less, until ideally one model stands out as the clear winner. Bayesians have invented fiendishly clever kinds of models. So let’s get started. Thomas Bayes was an eighteenth-century English clergyman who, without realizing it, became the center of a new religion. You may well ask how that could happen, until you notice that it happened to Jesus, too: Christianity as we know it was invented by Saint Paul, while Jesus saw himself as the pinnacle of the Jewish faith. Similarly, Bayesianism as we know it was invented by Pierre-Simon de Laplace, a Frenchman who was born five decades after Bayes.


pages: 336 words: 113,519

The Undoing Project: A Friendship That Changed Our Minds by Michael Lewis

Albert Einstein, availability heuristic, Cass Sunstein, choice architecture, complexity theory, Daniel Kahneman / Amos Tversky, Donald Trump, Douglas Hofstadter, endowment effect, feminist movement, framing effect, hindsight bias, John von Neumann, Kenneth Arrow, loss aversion, medical residency, Menlo Park, Murray Gell-Mann, Nate Silver, New Journalism, Paul Samuelson, Richard Thaler, Saturday Night Live, Stanford marshmallow experiment, statistical model, the new new thing, Thomas Bayes, Walter Mischel, Yom Kippur War

The subject picked one of the bags at random and, without glancing inside the bag, began to pull chips out of it, one at a time. After extracting each chip, he’d give the psychologists his best guess of the odds that the bag he was holding was filled with mostly red, or mostly white, chips. The beauty of the experiment was that there was a correct answer to the question: What is the probability that I am holding the bag of mostly red chips? It was provided by a statistical formula called Bayes’s theorem (after Thomas Bayes, who, strangely, left the formula for others to discover in his papers after his death, in 1761). Bayes’s rule allowed you to calculate the true odds, after each new chip was pulled from it, that the book bag in question was the one with majority white, or majority red, chips. Before any chips had been withdrawn, those odds were 50:50—the bag in your hands was equally likely to be either majority red or majority white.


pages: 398 words: 120,801

Little Brother by Cory Doctorow

airport security, Bayesian statistics, Berlin Wall, citizen journalism, Firefox, game design, Golden Gate Park, Haight Ashbury, Internet Archive, Isaac Newton, Jane Jacobs, Jeff Bezos, mail merge, Mitch Kapor, MITM: man-in-the-middle, RFID, Sand Hill Road, Silicon Valley, slashdot, Steve Jobs, Steve Wozniak, Thomas Bayes, web of trust, zero day

No one could tell which of the Internet's packets were Xnet and which ones were just plain old banking and e-commerce and other encrypted communication. You couldn't find out who was tying the Xnet, let alone who was using the Xnet. But what about Dad's "Bayesian statistics?" I'd played with Bayesian math before. Darryl and I once tried to write our own better spam filter and when you filter spam, you need Bayesian math. Thomas Bayes was an 18th century British mathematician that no one cared about until a couple hundred years after he died, when computer scientists realized that his technique for statistically analyzing mountains of data would be super-useful for the modern world's info-Himalayas. Here's some of how Bayesian stats work. Say you've got a bunch of spam. You take every word that's in the spam and count how many times it appears.


pages: 755 words: 121,290

Statistics hacks by Bruce Frey

Bayesian statistics, Berlin Wall, correlation coefficient, Daniel Kahneman / Amos Tversky, distributed generation, en.wikipedia.org, feminist movement, G4S, game design, Hacker Ethic, index card, Milgram experiment, p-value, place-making, reshoring, RFID, Search for Extraterrestrial Intelligence, SETI@home, Silicon Valley, statistical model, Thomas Bayes

What about the accuracy of a negative result? Of the 9,102 women who will score negative on the screening, 12 actually have cancer. This is a relatively small 1/10 of 1 percent, but the testing will miss those people altogether, and they will not receive treatment. Why It Works Medical screening accuracy uses a specific application of a generalized approach to conditional probability attributed to Thomas Bayes, a philosopher and mathematician in the 1700s. "If this, then what are the chances that..." is a conditional probability question. Bayes's approach to conditional probabilities was to look at the naturally occurring frequencies of events. The basic formula for estimating the chance that one has a disease if one has a positive test result is: Expressed as conditional probabilities, the formula is: To answer the all-important question in our breast cancer example ("If a woman scores a positive test result, how likely is she to have breast cancer?")


pages: 471 words: 124,585

The Ascent of Money: A Financial History of the World by Niall Ferguson

Admiral Zheng, Andrei Shleifer, Asian financial crisis, asset allocation, asset-backed security, Atahualpa, bank run, banking crisis, banks create money, Black Swan, Black-Scholes formula, Bonfire of the Vanities, Bretton Woods, BRICs, British Empire, business cycle, capital asset pricing model, capital controls, Carmen Reinhart, Cass Sunstein, central bank independence, collateralized debt obligation, colonial exploitation, commoditize, Corn Laws, corporate governance, creative destruction, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, currency manipulation / currency intervention, currency peg, Daniel Kahneman / Amos Tversky, deglobalization, diversification, diversified portfolio, double entry bookkeeping, Edmond Halley, Edward Glaeser, Edward Lloyd's coffeehouse, financial innovation, financial intermediation, fixed income, floating exchange rates, Fractional reserve banking, Francisco Pizarro, full employment, German hyperinflation, Hernando de Soto, high net worth, hindsight bias, Home mortgage interest deduction, Hyman Minsky, income inequality, information asymmetry, interest rate swap, Intergovernmental Panel on Climate Change (IPCC), Isaac Newton, iterative process, John Meriwether, joint-stock company, joint-stock limited liability company, Joseph Schumpeter, Kenneth Arrow, Kenneth Rogoff, knowledge economy, labour mobility, Landlord’s Game, liberal capitalism, London Interbank Offered Rate, Long Term Capital Management, market bubble, market fundamentalism, means of production, Mikhail Gorbachev, money market fund, money: store of value / unit of account / medium of exchange, moral hazard, mortgage debt, mortgage tax deduction, Myron Scholes, Naomi Klein, negative equity, Nelson Mandela, Nick Leeson, Northern Rock, Parag Khanna, pension reform, price anchoring, price stability, principal–agent problem, probability theory / Blaise Pascal / Pierre de Fermat, profit motive, quantitative hedge fund, RAND corporation, random walk, rent control, rent-seeking, reserve currency, Richard Thaler, Robert Shiller, Robert Shiller, Ronald Reagan, savings glut, seigniorage, short selling, Silicon Valley, South Sea Bubble, sovereign wealth fund, spice trade, stocks for the long run, structural adjustment programs, technology bubble, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Thomas Bayes, Thomas Malthus, Thorstein Veblen, too big to fail, transaction costs, undersea cable, value at risk, Washington Consensus, Yom Kippur War

In 1738 the Swiss mathematician Daniel Bernoulli proposed that ‘The value of an item must not be based on its price, but rather on the utility that it yields’, and that the ‘utility resulting from any small increase in wealth will be inversely proportionate to the quantity of goods previously possessed’ - in other words $100 is worth more to someone on the median income than to a hedge fund manager. 6. Inference. In his ‘Essay Towards Solving a Problem in the Doctrine of Chances’ (published posthumously in 1764), Thomas Bayes set himself the following problem: ‘Given the number of times in which an unknown event has happened and failed; Required the chance that the probability of its happening in a single trial lies somewhere between any two degrees of probability that can be named.’ His resolution of the problem - ‘The probability of any event is the ratio between the value at which an expectation depending on the happening of the event ought to be computed, and the chance of the thing expected upon it’s [sic] happening’ - anticipates the modern formulation that expected utility is the probability of an event times the payoff received in case of that event.18 In short, it was not merchants but mathematicians who were the true progenitors of modern insurance.


pages: 320 words: 33,385

Market Risk Analysis, Quantitative Methods in Finance by Carol Alexander

asset allocation, backtesting, barriers to entry, Brownian motion, capital asset pricing model, constrained optimization, credit crunch, Credit Default Swap, discounted cash flows, discrete time, diversification, diversified portfolio, en.wikipedia.org, fixed income, implied volatility, interest rate swap, market friction, market microstructure, p-value, performance metric, quantitative trading / quantitative finance, random walk, risk tolerance, risk-adjusted returns, risk/return, Sharpe ratio, statistical arbitrage, statistical model, stochastic process, stochastic volatility, Thomas Bayes, transaction costs, value at risk, volatility smile, Wiener process, yield curve, zero-sum game

For instance, if we threw a fair die 600 times we would expect to get a five 100 times. Thus, because we observe that there is 1 chance in 6 of getting a five when a fair die is thrown, we say that the probability of this event is 1/6. But long before the relative frequentist theory came to dominate our approach to probability and statistics, a more general Bayesian approach to probability and statistics had been pioneered by Thomas Bayes (1702–1761). The classical approach is based on objective information culled from experimental observations, but Bayes allowed subjective assessments of probabilities to be made, calling these assessments the prior beliefs. In fact, the classical approach is just a simple case of Bayesian probability and statistics, where there is no subjective information and so the prior distribution is uniform.


Entangled Life: How Fungi Make Our Worlds, Change Our Minds & Shape Our Futures by Merlin Sheldrake

biofilm, buy low sell high, carbon footprint, crowdsourcing, cuban missile crisis, dark matter, discovery of penicillin, experimental subject, Fellow of the Royal Society, Isaac Newton, Kickstarter, late capitalism, low earth orbit, Mason jar, meta analysis, meta-analysis, microbiome, moral panic, NP-complete, phenotype, randomized controlled trial, Ronald Reagan, the built environment, Thomas Bayes, Thomas Malthus, traveling salesman

PSK, a compound isolated from turkey tail mushrooms, extends the survival time of patients suffering from a range of cancers and is used alongside conventional cancer treatments in China and Japan (Powell [2014]). radiation-resistant biomaterials: For fungal melanins see Cordero (2017). sophistications of fungal lives: For estimates of the number of fungal species see Hawksworth (2001) and Hawksworth and Lücking (2017). when we actually look: Among neuroscientists, the involvement of our expectations in perception is known as top-down influence, or sometimes as Bayesian inference (after Thomas Bayes, a mathematician who made a founding contribution to the mathematics of probability, or “the doctrine of chances”). See Gilbert and Sigman (2007), and Mazzucato et al. (2019). “they’re cleverer than me”: Adamatzky (2016), Latty and Beekman (2011), Nakagaki et al. (2000), Bonifaci et al. (2012), Tero et al. (2010), and Oettmeier et al. (2017). In Advances in Physarum Machines (Adamatzky [2016]), researchers detail many surprising properties of slime molds.


pages: 483 words: 141,836

Red-Blooded Risk: The Secret History of Wall Street by Aaron Brown, Eric Kim

activist fund / activist shareholder / activist investor, Albert Einstein, algorithmic trading, Asian financial crisis, Atul Gawande, backtesting, Basel III, Bayesian statistics, beat the dealer, Benoit Mandelbrot, Bernie Madoff, Black Swan, business cycle, capital asset pricing model, central bank independence, Checklist Manifesto, corporate governance, creative destruction, credit crunch, Credit Default Swap, disintermediation, distributed generation, diversification, diversified portfolio, Edward Thorp, Emanuel Derman, Eugene Fama: efficient market hypothesis, experimental subject, financial innovation, illegal immigration, implied volatility, index fund, Long Term Capital Management, loss aversion, margin call, market clearing, market fundamentalism, market microstructure, money market fund, money: store of value / unit of account / medium of exchange, moral hazard, Myron Scholes, natural language processing, open economy, Pierre-Simon Laplace, pre–internet, quantitative trading / quantitative finance, random walk, Richard Thaler, risk tolerance, risk-adjusted returns, risk/return, road to serfdom, Robert Shiller, Robert Shiller, shareholder value, Sharpe ratio, special drawing rights, statistical arbitrage, stochastic volatility, stocks for the long run, The Myth of the Rational Market, Thomas Bayes, too big to fail, transaction costs, value at risk, yield curve

Second, we know nothing about the accuracy of this statement in particular; we only make a claim about the long-term accuracy of lots of statements. This is how we turn an event that has already happened—drawing nine red marbles out of 10—into a hypothetical coin-flip gambling game that can be repeated indefinitely. The main alternative to frequentist statistics today is the Bayesian view. It is named for Thomas Bayes, an eighteenth-century theorist, but it was Pierre-Simon Laplace who put forth the basic ideas. It was not until the twentieth century, however, that researchers, including Richard Cox and Bruno de Finetti, created the modern formulation. In the Bayesian view of the urn, you must have some prior belief about the number of red marbles in the urn. For example, you might believe that any number from 0 to 100 red marbles is equally likely.


pages: 478 words: 146,480

Pirate Cinema by Cory Doctorow

airport security, citation needed, Internet Archive, place-making, QR code, smart cities, Thomas Bayes

Brings up the grass a treat, as you can see." He gestured at the rolling lawns to one side of the ancient, mossy, fenced-in headstones. "Nonconformist cemetery," he went on, leading me deeper. "Unconsecrated ground. Lots of interesting folks buried here. You got your writers: like John Bunyon who wrote Pilgrims Progress. You got your philosophers, like Thomas Hardy. And some real maths geniuses, like old Thomas Bayes --" He pointed to a low, mossy tomb. "He invented a branch of statistics that got built into every spam filter, a couple hundred years after they buried him." He sat down on a bench. It was after mid-day now, and only a few people were eating lunch around us, none close enough to overhear us. "It's a grand life as a gentleman adventurer," he said. "Nothing to do all day but pluck choice morsels out of the bin and read the signboards the local historical society puts up in the graveyard."


pages: 537 words: 144,318

The Invisible Hands: Top Hedge Fund Traders on Bubbles, Crashes, and Real Money by Steven Drobny

Albert Einstein, Asian financial crisis, asset allocation, asset-backed security, backtesting, banking crisis, Bernie Madoff, Black Swan, Bretton Woods, BRICs, British Empire, business cycle, business process, buy and hold, capital asset pricing model, capital controls, central bank independence, collateralized debt obligation, commoditize, Commodity Super-Cycle, commodity trading advisor, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, currency peg, debt deflation, diversification, diversified portfolio, equity premium, family office, fiat currency, fixed income, follow your passion, full employment, George Santayana, Hyman Minsky, implied volatility, index fund, inflation targeting, interest rate swap, inventory management, invisible hand, Kickstarter, London Interbank Offered Rate, Long Term Capital Management, market bubble, market fundamentalism, market microstructure, moral hazard, Myron Scholes, North Sea oil, open economy, peak oil, pension reform, Ponzi scheme, prediction markets, price discovery process, price stability, private sector deleveraging, profit motive, purchasing power parity, quantitative easing, random walk, reserve currency, risk tolerance, risk-adjusted returns, risk/return, savings glut, selection bias, Sharpe ratio, short selling, sovereign wealth fund, special drawing rights, statistical arbitrage, stochastic volatility, stocks for the long run, stocks for the long term, survivorship bias, The Great Moderation, Thomas Bayes, time value of money, too big to fail, transaction costs, unbiased observer, value at risk, Vanguard fund, yield curve, zero-sum game

We shrink my expected returns towards zero rather than towards some equilibrium model forecast, the latter of which is more appropriate given our macro focus. We then input that adjusted expected return into the Titanic funnel, which assesses how much it will lose in a variety of cataclysms, giving me the recommended position. It is important to note that I am not putting trades on just to achieve the Titanic loss number. I am just looking for mispricings. Bayesian Methods The term Bayesian refers to the work Thomas Bayes, who proved a specific case of the now eponymous theorem, published after his death in 1761. The Bayesian interpretation of probability can be seen as a form of logic that allows for analysis of uncertain statements. To evaluate the probability of a hypothesis, Bayes’ theorem compares probabilities before and after the existence of new data. Unlike other methods for analyzing hypotheses, which attempt to reject or accept a statement, the Bayesian view seeks to assign dynamic probabilities that depend on the existence of relevant information.


Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth by Stuart Ritchie

Albert Einstein, anesthesia awareness, Bayesian statistics, Carmen Reinhart, Cass Sunstein, citation needed, Climatic Research Unit, cognitive dissonance, complexity theory, coronavirus, correlation does not imply causation, COVID-19, Covid-19, crowdsourcing, deindustrialization, Donald Trump, double helix, en.wikipedia.org, epigenetics, Estimating the Reproducibility of Psychological Science, Growth in a Time of Debt, Kenneth Rogoff, l'esprit de l'escalier, meta analysis, meta-analysis, microbiome, Milgram experiment, mouse model, New Journalism, p-value, phenotype, placebo effect, profit motive, publication bias, publish or perish, race to the bottom, randomized controlled trial, recommendation engine, rent-seeking, replication crisis, Richard Thaler, risk tolerance, Ronald Reagan, Scientific racism, selection bias, Silicon Valley, Silicon Valley startup, Stanford prison experiment, statistical model, stem cell, Steven Pinker, Thomas Bayes, twin studies, University of East Anglia

Doing away with p-values wouldn’t necessarily improve matters; in fact, by introducing another source of subjectivity, it might make the situation a lot worse.26 With tongue only partly in cheek, John Ioannidis has noted that if we remove all such objective measures we invite a situation where ‘all science will become like nutritional epidemiology’ – a scary prospect indeed.27 The same criticism is often levelled at the other main alternative to p-values: Bayesian statistics. Drawing on a probability theorem devised by the eighteenth-century statistician Thomas Bayes, this method allows researchers to take the strength of previous evidence – referred to as a ‘prior’ – into account when assessing the significance of new findings. For instance, if someone tells you their weather forecast predicts a rainy day in London in the autumn, it won’t take too much to convince you that they’re right. On the other hand, if their forecast predicts a snowstorm in the Sahara Desert in July, you’d probably want to assess that claim quite sceptically, given all the prior experience we have of scorching Saharan summers.


pages: 573 words: 157,767

From Bacteria to Bach and Back: The Evolution of Minds by Daniel C. Dennett

Ada Lovelace, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Andrew Wiles, Bayesian statistics, bioinformatics, bitcoin, Build a better mousetrap, Claude Shannon: information theory, computer age, computer vision, double entry bookkeeping, double helix, Douglas Hofstadter, Elon Musk, epigenetics, experimental subject, Fermat's Last Theorem, Gödel, Escher, Bach, information asymmetry, information retrieval, invention of writing, Isaac Newton, iterative process, John von Neumann, Menlo Park, Murray Gell-Mann, Necker cube, Norbert Wiener, pattern recognition, phenotype, Richard Feynman, Rodney Brooks, self-driving car, social intelligence, sorting algorithm, speech recognition, Stephen Hawking, Steven Pinker, strong AI, The Wealth of Nations by Adam Smith, theory of mind, Thomas Bayes, trickle-down economics, Turing machine, Turing test, Watson beat the top human players on Jeopardy!, Y2K

But part of the specs for any reverse-engineering exploration has to be the limitations and strengths of the available machinery, and by postponing questions about the brain, and underestimating the importance of the telltale performances elicited by the psycholinguists, the theoretical linguists conducted some wild goose chases of their own. With that caution in hand, we can address the idea that is now sweeping through cognitive science as a very promising answer to how the brain picks up, and uses, the available semantic information: Bayesian hierarchical predictive coding. (For excellent accounts see Hinton 2007; Clark 2013; and the commentary on Clark, Hohwy 2013.) The basic idea is delicious. The Reverend Thomas Bayes (1701–1761) developed a method of calculating probabilities based on one’s prior expectations. Each problem is couched thus: Given that your expectations based on past experience (including, we may add, the experience of your ancestors as passed down to you) are such and such (expressed as probabilities for each alternative), what effect on your future expectations should the following new data have?


pages: 547 words: 148,732

How to Change Your Mind: What the New Science of Psychedelics Teaches Us About Consciousness, Dying, Addiction, Depression, and Transcendence by Michael Pollan

1960s counterculture, Albert Einstein, Anton Chekhov, Burning Man, cognitive dissonance, conceptual framework, crowdsourcing, dark matter, Douglas Engelbart, East Village, experimental subject, Exxon Valdez, Golden Gate Park, Google Earth, Haight Ashbury, Howard Rheingold, Internet Archive, John Markoff, Kevin Kelly, Marshall McLuhan, Mason jar, Menlo Park, meta analysis, meta-analysis, moral panic, Mother of all demos, placebo effect, Ralph Waldo Emerson, randomized controlled trial, Ronald Reagan, scientific mainstream, scientific worldview, selective serotonin reuptake inhibitor (SSRI), sensible shoes, Silicon Valley, Skype, Steve Jobs, Stewart Brand, the scientific method, theory of mind, Thomas Bayes, Whole Earth Catalog

So rather than starting from scratch to build a new perception from every batch of raw data delivered by the senses, the mind jumps to the most sensible conclusion based on past experience combined with a tiny sample of that data. Our brains are prediction machines optimized by experience, and when it comes to faces, they have boatloads of experience: faces are always convex, so this hollow mask must be a prediction error to be corrected. These so-called Bayesian inferences (named for Thomas Bayes, the eighteenth-century English philosopher who developed the mathematics of probability, on which these mental predictions are based) serve us well most of the time, speeding perception while saving effort and energy, but they can also trap us in literally preconceived images of reality that are simply false, as in the case of the rotating mask. Yet it turns out that Bayesian inference breaks down in some people: schizophrenics and, according to some neuroscientists, people on high doses of psychedelics drugs, neither of whom “see” in this predictive or conventionalized manner.


pages: 654 words: 191,864

Thinking, Fast and Slow by Daniel Kahneman

Albert Einstein, Atul Gawande, availability heuristic, Bayesian statistics, Black Swan, Cass Sunstein, Checklist Manifesto, choice architecture, cognitive bias, complexity theory, correlation coefficient, correlation does not imply causation, Daniel Kahneman / Amos Tversky, delayed gratification, demand response, endowment effect, experimental economics, experimental subject, Exxon Valdez, feminist movement, framing effect, hedonic treadmill, hindsight bias, index card, information asymmetry, job satisfaction, John von Neumann, Kenneth Arrow, libertarian paternalism, loss aversion, medical residency, mental accounting, meta analysis, meta-analysis, nudge unit, pattern recognition, Paul Samuelson, pre–internet, price anchoring, quantitative trading / quantitative finance, random walk, Richard Thaler, risk tolerance, Robert Metcalfe, Ronald Reagan, Shai Danziger, Supply of New York City Cabdrivers, The Chicago School, The Wisdom of Crowds, Thomas Bayes, transaction costs, union organizing, Walter Mischel, Yom Kippur War

And if you believe that there is a 30% chance that candidate X will be elected president, and an 80% chance that he will be reelected if he wins the first time, then you must believe that the chances that he will be elected twice in a row are 24%. The relevant “rules” for cases such as the Tom W problem are provided by Bayesian statistics. This influential modern approach to statistics is named after an English minister of the eighteenth century, the Reverend Thomas Bayes, who is credited with the first major contribution to a large problem: the logic of how people should change their mind in the light of evidence. Bayes’s rule specifies how prior beliefs (in the examples of this chapter, base rates) should be combined with the diagnosticity of the evidence, the degree to which it favors the hypothesis over the alternative. For example, if you believe that 3% of graduate students are enrolled in computer science (the base rate), and you also believe that the description of Tom W is 4 times more likely for a graduate student in that field than in other fields, then Bayes’s rule says you must believe that the probability that Tom W is a computer scientist is now 11%.


pages: 651 words: 180,162

Antifragile: Things That Gain From Disorder by Nassim Nicholas Taleb

Air France Flight 447, Andrei Shleifer, banking crisis, Benoit Mandelbrot, Berlin Wall, Black Swan, business cycle, Chuck Templeton: OpenTable:, commoditize, creative destruction, credit crunch, Daniel Kahneman / Amos Tversky, David Ricardo: comparative advantage, discrete time, double entry bookkeeping, Emanuel Derman, epigenetics, financial independence, Flash crash, Gary Taubes, George Santayana, Gini coefficient, Henri Poincaré, high net worth, hygiene hypothesis, Ignaz Semmelweis: hand washing, informal economy, invention of the wheel, invisible hand, Isaac Newton, James Hargreaves, Jane Jacobs, joint-stock company, joint-stock limited liability company, Joseph Schumpeter, Kenneth Arrow, knowledge economy, Lao Tzu, Long Term Capital Management, loss aversion, Louis Pasteur, mandelbrot fractal, Marc Andreessen, meta analysis, meta-analysis, microbiome, money market fund, moral hazard, mouse model, Myron Scholes, Norbert Wiener, pattern recognition, Paul Samuelson, placebo effect, Ponzi scheme, principal–agent problem, purchasing power parity, quantitative trading / quantitative finance, Ralph Nader, random walk, Ray Kurzweil, rent control, Republic of Letters, Ronald Reagan, Rory Sutherland, selection bias, Silicon Valley, six sigma, spinning jenny, statistical model, Steve Jobs, Steven Pinker, Stewart Brand, stochastic process, stochastic volatility, Thales and the olive presses, Thales of Miletus, The Great Moderation, the new new thing, The Wealth of Nations by Adam Smith, Thomas Bayes, Thomas Malthus, too big to fail, transaction costs, urban planning, Vilfredo Pareto, Yogi Berra, Zipf's Law

There were two main sources of technical knowledge and innovation in the nineteenth and early twentieth centuries: the hobbyist and the English rector, both of whom were generally in barbell situations. An extraordinary proportion of work came out of the rector, the English parish priest with no worries, erudition, a large or at least comfortable house, domestic help, a reliable supply of tea and scones with clotted cream, and an abundance of free time. And, of course, optionality. The enlightened amateur, that is. The Reverends Thomas Bayes (as in Bayesian probability) and Thomas Malthus (Malthusian overpopulation) are the most famous. But there are many more surprises, cataloged in Bill Bryson’s Home, in which the author found ten times more vicars and clergymen leaving recorded traces for posterity than scientists, physicists, economists, and even inventors. In addition to the previous two giants, I randomly list contributions by country clergymen: Rev.


pages: 586 words: 186,548

Architects of Intelligence by Martin Ford

3D printing, agricultural Revolution, AI winter, Apple II, artificial general intelligence, Asilomar, augmented reality, autonomous vehicles, barriers to entry, basic income, Baxter: Rethink Robotics, Bayesian statistics, bitcoin, business intelligence, business process, call centre, cloud computing, cognitive bias, Colonization of Mars, computer vision, correlation does not imply causation, crowdsourcing, DARPA: Urban Challenge, deskilling, disruptive innovation, Donald Trump, Douglas Hofstadter, Elon Musk, Erik Brynjolfsson, Ernest Rutherford, Fellow of the Royal Society, Flash crash, future of work, gig economy, Google X / Alphabet X, Gödel, Escher, Bach, Hans Rosling, ImageNet competition, income inequality, industrial robot, information retrieval, job automation, John von Neumann, Law of Accelerating Returns, life extension, Loebner Prize, Mark Zuckerberg, Mars Rover, means of production, Mitch Kapor, natural language processing, new economy, optical character recognition, pattern recognition, phenotype, Productivity paradox, Ray Kurzweil, recommendation engine, Robert Gordon, Rodney Brooks, Sam Altman, self-driving car, sensor fusion, sentiment analysis, Silicon Valley, smart cities, social intelligence, speech recognition, statistical model, stealth mode startup, stem cell, Stephen Hawking, Steve Jobs, Steve Wozniak, Steven Pinker, strong AI, superintelligent machines, Ted Kaczynski, The Rise and Fall of American Growth, theory of mind, Thomas Bayes, Travis Kalanick, Turing test, universal basic income, Wall-E, Watson beat the top human players on Jeopardy!, women in the workforce, working-age population, zero-sum game, Zipcar

Nonetheless, I did ask Yann LeCun, who invented the convolutional architecture that is widely used in computer vision applications, to take a shot at explaining this concept. BAYESIAN is a term that can be generally be translated as “probabilistic” or “using the rules of probability.” You may encounter terms like Bayesian machine learning or Bayesian networks; these refer to algorithms that use the rules of probability. The term derives from the name of the Reverend Thomas Bayes (1701 to 1761) who formulated a way to update the likelihood of an event based on new evidence. Bayesian methods are very popular with both computer scientists and with scientists who attempt to model human cognition. Judea Pearl, who is interviewed in this book, received the highest honor in computer science, the Turing Award, in part for his work on Bayesian techniques. How AI Systems Learn There are several ways that machine learning systems can be trained.


pages: 685 words: 203,949

The Organized Mind: Thinking Straight in the Age of Information Overload by Daniel J. Levitin

airport security, Albert Einstein, Amazon Mechanical Turk, Anton Chekhov, Bayesian statistics, big-box store, business process, call centre, Claude Shannon: information theory, cloud computing, cognitive bias, complexity theory, computer vision, conceptual framework, correlation does not imply causation, crowdsourcing, cuban missile crisis, Daniel Kahneman / Amos Tversky, delayed gratification, Donald Trump, en.wikipedia.org, epigenetics, Eratosthenes, Exxon Valdez, framing effect, friendly fire, fundamental attribution error, Golden Gate Park, Google Glasses, haute cuisine, impulse control, index card, indoor plumbing, information retrieval, invention of writing, iterative process, jimmy wales, job satisfaction, Kickstarter, life extension, longitudinal study, meta analysis, meta-analysis, more computing power than Apollo, Network effects, new economy, Nicholas Carr, optical character recognition, Pareto efficiency, pattern recognition, phenotype, placebo effect, pre–internet, profit motive, randomized controlled trial, Rubik’s Cube, shared worldview, Skype, Snapchat, social intelligence, statistical model, Steve Jobs, supply-chain management, the scientific method, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, theory of mind, Thomas Bayes, Turing test, ultimatum game, zero-sum game

Very unlikely. So this shows we’re capable of using base rate information when events are extremely unlikely. It’s when they’re only mildly unlikely that our brains freeze up. Organizing our decisions requires that we combine the base rate information with other relevant diagnostic information. This type of reasoning was discovered in the eighteenth century by the mathematician and Presbyterian minister Thomas Bayes, and bears his name: Bayes’s rule. Bayes’s rule allows us to refine estimates. For example, we read that roughly half of marriages end in divorce. But we can refine that estimate if we have additional information, such as the age, religion, or location of the people involved, because the 50% figure holds only for the aggregate of all people. Some subpopulations of people have higher divorce rates than others.


pages: 698 words: 198,203

The Stuff of Thought: Language as a Window Into Human Nature by Steven Pinker

airport security, Albert Einstein, Bob Geldof, colonial rule, conceptual framework, correlation does not imply causation, Daniel Kahneman / Amos Tversky, David Brooks, Douglas Hofstadter, en.wikipedia.org, experimental subject, fudge factor, George Santayana, Laplace demon, loss aversion, luminiferous ether, Norman Mailer, Richard Feynman, Ronald Reagan, Sapir-Whorf hypothesis, science of happiness, social intelligence, speech recognition, stem cell, Steven Pinker, Thomas Bayes, Thorstein Veblen, traffic fines, urban renewal, Yogi Berra

The world is a tissue of causes and effects that criss and cross in tangled patterns. The embarrassments for Hume’s two theories of causation (conjunction and counterfactuals) can be diagrammed as a family of networks in which the lines fan in or out or loop around, as in the diagram on the following page. One solution to the webbiness of causation is a technique in artificial intelligence called Causal Bayes Networks.120 (They are named for Thomas Bayes, whose eponymous theorem shows how to calculate the probability of some condition from its prior plausibility and the likelihood that it led to some observed symptoms.) A modeler chooses a set of variables (amount of coffee drunk, amount of exercise, presence of heart disease, and so on), draws arrows between causes and their effects, and labels each arrow with a number representing the strength of the causal influence (the increase or decrease in the likelihood of the effect, given the presence of the cause).


Data Mining: Concepts and Techniques: Concepts and Techniques by Jiawei Han, Micheline Kamber, Jian Pei

bioinformatics, business intelligence, business process, Claude Shannon: information theory, cloud computing, computer vision, correlation coefficient, cyber-physical system, database schema, discrete time, distributed generation, finite state, information retrieval, iterative process, knowledge worker, linked data, natural language processing, Netflix Prize, Occam's razor, pattern recognition, performance metric, phenotype, random walk, recommendation engine, RFID, semantic web, sentiment analysis, speech recognition, statistical model, stochastic process, supply-chain management, text mining, thinkpad, Thomas Bayes, web application

Naïve Bayesian classifiers assume that the effect of an attribute value on a given class is independent of the values of the other attributes. This assumption is called class-conditional independence. It is made to simplify the computations involved and, in this sense, is considered “naïve." Section 8.3.1 reviews basic probability notation and Bayes’ theorem. In Section 8.3.2 you will learn how to do naïve Bayesian classification. 8.3.1. Bayes’ Theorem Bayes’ theorem is named after Thomas Bayes, a nonconformist English clergyman who did early work in probability and decision theory during the 18th century. Let X be a data tuple. In Bayesian terms, X is considered “evidence.” As usual, it is described by measurements made on a set of n attributes. Let H be some hypothesis such as that the data tuple X belongs to a specified class C. For classification problems, we want to determine , the probability that the hypothesis H holds given the “evidence” or observed data tuple X.