Estimating the Reproducibility of Psychological Science

6 results back to index

The Choice Factory: 25 Behavioural Biases That Influence What We Buy by Richard Shotton

active measures, behavioural economics, call centre, cashless society, cognitive dissonance, Daniel Kahneman / Amos Tversky, data science, David Brooks, Estimating the Reproducibility of Psychological Science, Firefox, framing effect, fundamental attribution error, Goodhart's law, Google Chrome, Kickstarter, loss aversion, nudge unit, Ocado, placebo effect, price anchoring, principal–agent problem, Ralph Waldo Emerson, replication crisis, Richard Feynman, Richard Thaler, Robert Shiller, Rory Sutherland, TED Talk, Veblen good, When a measure becomes a target, World Values Survey

, by Evan Davis, John Kay, and Jonathan Star [London Business School Review, Vol. 2, No. 3, pp. 1–23, 1991] Marketers Are from Mars, Consumers Are from New Jersey by Bob Hoffman [2015] Bias 16: The curse of knowledge Made to Stick: Why Some Ideas Survive and Others Die by Chip Heath and Dan Heath [2008] The Wiki Man by Rory Sutherland [2011] Bias 17: Goodhart’s law Long and Short of It: Balancing Short- and Long-Term Marketing Strategies by Les Binet and Peter Field [2012] Management in 10 Words by Terry Leahy [2012] Leading by Alex Ferguson and Michael Moritz [2015] Bias 18: The pratfall effect: Social Animal by Elliot Aronson [1972] The Wasp Factory by Iain Banks [1984] Bias 19: Winner’s curse The Winner’s Curse: Paradoxes and Anomalies of Economic Life by Richard Thaler [1991] Originals: How Non-Conformists Move the World by Adam Grant [2016] ‘Harnessing naturally occurring data to measure the response of spending to income’, by Michael Gelman, Shachar Kariv, Matthew Shapiro, Dan Silverman, Steven Tadelis [Science, Vol. 345, No. 6193, pp. 212–215, 2014] ‘The Psychology of Windfall Gains’, by Hal Arkes, Cynthia Joyner, Mark Pezzo, Jane Gradwohl Nash, Karen Siegel-Jacobs, Eric Stone Eric [Organizational Behaviour and Human Decision Processes, Vol. 59, No. 3, pp. 331–347, 1994] On the Fungibility of Spending and Earnings – Evidence from Rural China and Tanzania by Luc Christiaensen and Lei Pan [2012] Bias 20: The power of the group ‘Humour in Television Advertising: The Effects of Repetition and Social Setting’, by Yong Zhang and George Zinkhan [Advances In Consumer Research, Vol. 18, pp. 813–818, 1991] ‘Feeling More Together: Group Attention Intensifies Emotion’, by Garriy Shteynberg, Jacob Hirsh, Evan Apfelbaum, Jeff Larsen, Adam Galinsky, and Neal Roese [Emotion, Vol. 14, No. 6, pp. 1102–1114, 2014] Bias 21: Veblen goods ‘Commercial Features of Placebo and Therapeutic Efficacy’, by Rebecca Waber, Baba Shiv, Ziv Carmon; Dan Ariely [Journal of the American Medical Association, Vol. 299, No.9, pp. 1016–1017, 2008] Bias 22: The replicability crisis ‘Why Susie Sells Seashells by the Seashore: Implicit Egotism and Major Life Decisions’, by Brett Pelham, Matthew Mirenberg, and John Jones [Journal of Personality and Social Psychology, Vol. 82, No. 4, pp. 469–487, 2002] ‘Rich the banker? What’s not in a Name’, by Tim Harford [2016], www.timharford.com/2016/11/rich-the-banker-whats-not-in-a-name ‘Estimating the reproducibility of psychological science’, by Brian Nosek et al. [Science, Vol. 349, No. 6251, 2015] ‘Comment on “Estimating the reproducibility of psychological science”’, by Daniel Gilbert, Gary King, Stephen Pettigrew and Timothy Wilson [Science, Vol. 351, Issue 6277, p. 1037, 2016] ‘Meta-assessment of bias in science’, by Daniele Fanelli, Rodrigo Costats, and John Ioannidis [Proceedings of the National Academy of Sciences, Vol. 114, No. 14, pp. 3714–3719, 2017] ‘Evaluating replicability of laboratory experiments in economics’, by Colin F.

Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth by Stuart Ritchie

Albert Einstein, anesthesia awareness, autism spectrum disorder, Bayesian statistics, Black Lives Matter, Carmen Reinhart, Cass Sunstein, Charles Babbage, citation needed, Climatic Research Unit, cognitive dissonance, complexity theory, coronavirus, correlation does not imply causation, COVID-19, crowdsourcing, data science, deindustrialization, Donald Trump, double helix, en.wikipedia.org, epigenetics, Estimating the Reproducibility of Psychological Science, fake news, Goodhart's law, Growth in a Time of Debt, Helicobacter pylori, Higgs boson, hype cycle, Kenneth Rogoff, l'esprit de l'escalier, Large Hadron Collider, meta-analysis, microbiome, Milgram experiment, mouse model, New Journalism, ocean acidification, p-value, phenotype, placebo effect, profit motive, publication bias, publish or perish, quantum entanglement, race to the bottom, randomized controlled trial, recommendation engine, rent-seeking, replication crisis, Richard Thaler, risk tolerance, Ronald Reagan, Scientific racism, selection bias, Silicon Valley, Silicon Valley startup, social distancing, Stanford prison experiment, statistical model, stem cell, Steven Pinker, TED Talk, Thomas Bayes, twin studies, Tyler Cowen, University of East Anglia, Wayback Machine

See also Le Texier’s reply to a more recent (at the time of writing unpublished) version: Thibault Le Texier, ‘The SPE Remains Debunked: A Reply to Zimbardo and Haney (2020)’, Preprint, PsyArXiv (24 Jan. 2020); https://doi.org/10.31234/osf.io/9a2er 26.  Open Science Collaboration, ‘Estimating the Reproducibility of Psychological Science’, Science 349, no. 6251 (28 Aug. 2015): aac4716; https://doi.org/10.1126/science.aac4716 27.  77 per cent: Colin F. Camerer et al., ‘Evaluating the Replicability of Social Science Experiments in Nature and Science between 2010 and 2015’, Nature Human Behaviour 2, no. 9 (Sept. 2018): pp. 637–44; https://doi.org/10.1038/s41562-018-0399-z 28.  

…

I’ve been stressing the importance of robust results, but in making the case that there’s a replication crisis, I’m relying on multi-study replication attempts that weren’t representative samples of all the scientific literature. The conclusion of ‘only about half of published results replicate’ might not generalise to all science. This was a point made in a critique of one of the replication survey studies: D. T. Gilbert et al., ‘Comment on “Estimating the Reproducibility of Psychological Science”’, Science 351, no. 6277 (4 Mar. 2016): p. 1037; https://doi.org/10.1126/science.aad7243. Whereas I disagree with many of the arguments made in this rejoinder (for some reasons to be sceptical of it, see Daniël Lakens, ‘The Statistical Conclusions in Gilbert et al (2016) Are Completely Invalid’, The 20% Statistician, 6 March 2016; https://daniellakens.blogspot.com/2016/03/the-statistical-conclusions-in-gilbert.html), the criticism about representativeness was fair.

Thinking in Bets by Annie Duke

banking crisis, behavioural economics, Bernie Madoff, Cass Sunstein, cognitive bias, cognitive dissonance, cognitive load, Daniel Kahneman / Amos Tversky, delayed gratification, Demis Hassabis, disinformation, Donald Trump, Dr. Strangelove, en.wikipedia.org, endowment effect, Estimating the Reproducibility of Psychological Science, fake news, Filter Bubble, Herman Kahn, hindsight bias, Jean Tirole, John Nash: game theory, John von Neumann, loss aversion, market design, mutually assured destruction, Nate Silver, p-value, phenotype, prediction markets, Richard Feynman, ride hailing / ride sharing, Stanford marshmallow experiment, Stephen Hawking, Steven Pinker, systematic bias, TED Talk, the scientific method, The Signal and the Noise by Nate Silver, urban planning, Walter Mischel, Yogi Berra, zero-sum game

New York: Current, 2014. Oettingen, Gabriele, and Peter Gollwitzer. “Strategies of Setting and Implementing Goals.” In Social Psychological Foundations of Clinical Psychology, edited by James Maddox and June Price Tangney, 114–35. New York: Guilford Press, 2010. Open Science Collaboration. “Estimating the Reproducibility of Psychological Science.” Science 349, no. 6251 (August 28, 2015): 943 and aac4716-1–8. Oswald, Dan. “Learn Important Lessons from Lombardi’s Eight-Hour Session.” HR Hero (blog), March 10, 2014. http://blogs.hrhero.com/oswaldletters/2014/03/10/learn-important-lessons-from-lombardis-eight-hour-session.

Know Thyself by Stephen M Fleming

Abraham Wald, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, AlphaGo, autism spectrum disorder, autonomous vehicles, availability heuristic, backpropagation, citation needed, computer vision, confounding variable, data science, deep learning, DeepMind, Demis Hassabis, Douglas Hofstadter, Dunning–Kruger effect, Elon Musk, Estimating the Reproducibility of Psychological Science, fake news, global pandemic, higher-order functions, index card, Jeff Bezos, l'esprit de l'escalier, Lao Tzu, lifelogging, longitudinal study, meta-analysis, mutually assured destruction, Network effects, patient HM, Pierre-Simon Laplace, power law, prediction markets, QWERTY keyboard, recommendation engine, replication crisis, self-driving car, side project, Skype, Stanislav Petrov, statistical model, theory of mind, Thomas Bayes, traumatic brain injury

“Birds Have Primate-Like Numbers of Neurons in the Forebrain.” Proceedings of the National Academy of Sciences 113, no. 26 (2016): 7255–7260. Onishi, Kristine H., and Renée Baillargeon. “Do 15-Month-Old Infants Understand False Beliefs?” Science 308, no. 5719 (2005): 255–258. Open Science Collaboration. “Estimating the Reproducibility of Psychological Science.” Science 349, no. 6251 (2015). Ortoleva, Pietro, and Erik Snowberg. “Overconfidence in Political Behavior.” American Economic Review 105, no. 2 (2015): 504–535. Palser, E. R., A. Fotopoulou, and J. M. Kilner. “Altering Movement Parameters Disrupts Metacognitive Accuracy.”

Restarting the Future: How to Fix the Intangible Economy by Jonathan Haskel, Stian Westlake

"Friedman doctrine" OR "shareholder theory", activist fund / activist shareholder / activist investor, Andrei Shleifer, Big Tech, Black Lives Matter, book value, Boris Johnson, Brexit referendum, business cycle, business process, call centre, Capital in the Twenty-First Century by Thomas Piketty, central bank independence, Charles Lindbergh, charter city, cloud computing, cognitive bias, cognitive load, congestion charging, coronavirus, corporate governance, COVID-19, creative destruction, cryptocurrency, David Graeber, decarbonisation, Diane Coyle, Dominic Cummings, Donald Shoup, Donald Trump, Douglas Engelbart, Douglas Engelbart, driverless car, Edward Glaeser, equity risk premium, Erik Brynjolfsson, Estimating the Reproducibility of Psychological Science, facts on the ground, financial innovation, Francis Fukuyama: the end of history, future of work, general purpose technology, gentrification, Goodhart's law, green new deal, housing crisis, income inequality, index fund, indoor plumbing, industrial cluster, inflation targeting, intangible asset, interchangeable parts, invisible hand, job-hopping, John Maynard Keynes: Economic Possibilities for our Grandchildren, Joseph Schumpeter, Kenneth Arrow, knowledge economy, knowledge worker, lockdown, low interest rates, low skilled workers, Marc Andreessen, market design, Martin Wolf, megacity, mittelstand, new economy, Occupy movement, oil shock, patent troll, Peter Thiel, Phillips curve, postindustrial economy, pre–internet, price discrimination, quantitative easing, QWERTY keyboard, remote working, rent-seeking, replication crisis, risk/return, Robert Gordon, Robert Metcalfe, Robert Shiller, Ronald Coase, Sam Peltzman, Second Machine Age, secular stagnation, shareholder value, Silicon Valley, six sigma, skeuomorphism, social distancing, superstar cities, the built environment, The Rise and Fall of American Growth, The Spirit Level, The Wealth of Nations by Adam Smith, Thorstein Veblen, total factor productivity, transaction costs, Tyler Cowen, Tyler Cowen: Great Stagnation, Uber for X, urban planning, We wanted flying cars, instead we got 140 characters, work culture , X Prize, Y2K

utm_campaign=Matt%27s%20Thoughts%20In%20Between&utm_medium=email&utm_source=Revue%20newsletter. 18. Fukuyama 1995; Putnam 1994. REFERENCES Aarts, Alexander A., Joanna E. Anderson, Christopher J. Anderson, Peter R. Attridge, Angela Attwood, Jordan Axt, Molly Babel, et al. 2015. “Estimating the Reproducibility of Psychological Science.” Science 349 (6251). https://doi.org/10.1126/science.aac4716. Abel, A. B., A. K. Dixit, J. C. Eberly, and R. S. Pindyck. 1996. “Options, the Value of Capital, and Investment.” Quarterly Journal of Economics 111 (3): 753–77. https://doi.org/10.2307/2946671. Acemoglu, Daron. 1999.

Randomistas: How Radical Researchers Changed Our World by Andrew Leigh

Albert Einstein, Amazon Mechanical Turk, Anton Chekhov, Atul Gawande, basic income, behavioural economics, Black Swan, correlation does not imply causation, crowdsourcing, data science, David Brooks, Donald Trump, ending welfare as we know it, Estimating the Reproducibility of Psychological Science, experimental economics, Flynn Effect, germ theory of disease, Ignaz Semmelweis: hand washing, Indoor air pollution, Isaac Newton, It's morning again in America, Kickstarter, longitudinal study, loss aversion, Lyft, Marshall McLuhan, meta-analysis, microcredit, Netflix Prize, nudge unit, offshore financial centre, p-value, Paradox of Choice, placebo effect, price mechanism, publication bias, RAND corporation, randomized controlled trial, recommendation engine, Richard Feynman, ride hailing / ride sharing, Robert Metcalfe, Ronald Reagan, Sheryl Sandberg, statistical model, Steven Pinker, sugar pill, TED Talk, uber lyft, universal basic income, War on Poverty

Lalande, ‘A peculiar prevalence of p values just below .05’, Quarterly Journal of Experimental Psychology, vol. 65, no. 11, 2012, pp. 2271–9; Kewei Hou, Chen Xue & Lu Zhang, ‘Replicating anomalies’, NBER Working Paper 23394, Cambridge, MA: National Bureau of Economic Research, 2017. 46Alexander A. Aarts, Joanna E. Anderson, Christopher J. Anderson, et al., ‘Estimating the reproducibility of psychological science’, Science, vol. 349, no. 6251, 2015. 47This represented two out of eighteen papers: John P.A. Ioannidis, David B. Allison, Catherine A. Ball, et al., ‘Repeatability of published microarray gene expression analyses’, Nature Genetics, vol. 41, no. 2, 2009, pp. 149–55. 48This represented six out of fifty-three papers: C.

The Art of Statistics: Learning From Data by David Spiegelhalter

Abraham Wald, algorithmic bias, Anthropocene, Antoine Gombaud: Chevalier de Méré, Bayesian statistics, Brexit referendum, Carmen Reinhart, Charles Babbage, complexity theory, computer vision, confounding variable, correlation coefficient, correlation does not imply causation, dark matter, data science, deep learning, DeepMind, Edmond Halley, Estimating the Reproducibility of Psychological Science, government statistician, Gregor Mendel, Hans Rosling, Higgs boson, Kenneth Rogoff, meta-analysis, Nate Silver, Netflix Prize, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, p-value, placebo effect, probability theory / Blaise Pascal / Pierre de Fermat, publication bias, randomized controlled trial, recommendation engine, replication crisis, self-driving car, seminal paper, sparse data, speech recognition, statistical model, sugar pill, systematic bias, TED Talk, The Design of Experiments, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Malthus, Two Sigma

Raftery, ‘Bayes Factors’, Journal of the American Statistical Association 90 (1995), 773–95. 9. J. Cornfield, ‘Sequential Trials, Sequential Analysis and the Likelihood Principle’, American Statistician 20 (1966), 18–23. CHAPTER 12: HOW THINGS GO WRONG 1. Open Science Collaboration, ‘Estimating the Reproducibility of Psychological Science’, Science 349:6251 (28 August 2015), aac4716. 2. A. Gelman and H. Stern, ‘The Difference Between “Significant” and “Not Significant” Is Not Itself Statistically Significant’, American Statistician 60:4 (November 2006), 328–31. 3. Ronald Fisher, Presidential Address to the first Indian Statistical Congress, 1938, Sankhyā 4(1938), 14–17. 4.

The Art of Statistics: How to Learn From Data by David Spiegelhalter

Abraham Wald, algorithmic bias, Antoine Gombaud: Chevalier de Méré, Bayesian statistics, Brexit referendum, Carmen Reinhart, Charles Babbage, complexity theory, computer vision, confounding variable, correlation coefficient, correlation does not imply causation, dark matter, data science, deep learning, DeepMind, Edmond Halley, Estimating the Reproducibility of Psychological Science, government statistician, Gregor Mendel, Hans Rosling, Higgs boson, Kenneth Rogoff, meta-analysis, Nate Silver, Netflix Prize, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, p-value, placebo effect, probability theory / Blaise Pascal / Pierre de Fermat, publication bias, randomized controlled trial, recommendation engine, replication crisis, self-driving car, seminal paper, sparse data, speech recognition, statistical model, sugar pill, systematic bias, TED Talk, The Design of Experiments, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Malthus, Two Sigma

Raftery, ‘Bayes Factors’, Journal of the American Statistical Association 90 (1995), 773–95. 9. J. Cornfield, ‘Sequential Trials, Sequential Analysis and the Likelihood Principle’, American Statistician 20 (1966), 18–23. CHAPTER 12: HOW THINGS GO WRONG 1. Open Science Collaboration, ‘Estimating the Reproducibility of Psychological Science’, Science 349:6251 (28 August 2015), aac4716. 2. A. Gelman and H. Stern, ‘The Difference Between “Significant” and “Not Significant” Is Not Itself Statistically Significant’, American Statistician 60:4 (November 2006), 328–31. 3. Ronald Fisher, Presidential Address to the first Indian Statistical Congress, 1938, Sankhyā 4(1938), 14–17. 4.

Calling Bullshit: The Art of Scepticism in a Data-Driven World by Jevin D. West, Carl T. Bergstrom

airport security, algorithmic bias, AlphaGo, Amazon Mechanical Turk, Andrew Wiles, Anthropocene, autism spectrum disorder, bitcoin, Charles Babbage, cloud computing, computer vision, content marketing, correlation coefficient, correlation does not imply causation, crowdsourcing, cryptocurrency, data science, deep learning, deepfake, delayed gratification, disinformation, Dmitri Mendeleev, Donald Trump, Elon Musk, epigenetics, Estimating the Reproducibility of Psychological Science, experimental economics, fake news, Ford Model T, Goodhart's law, Helicobacter pylori, Higgs boson, invention of the printing press, John Markoff, Large Hadron Collider, longitudinal study, Lyft, machine translation, meta-analysis, new economy, nowcasting, opioid epidemic / opioid crisis, p-value, Pluto: dwarf planet, publication bias, RAND corporation, randomized controlled trial, replication crisis, ride hailing / ride sharing, Ronald Reagan, selection bias, self-driving car, Silicon Valley, Silicon Valley startup, social graph, Socratic dialogue, Stanford marshmallow experiment, statistical model, stem cell, superintelligent machines, systematic bias, tech bro, TED Talk, the long tail, the scientific method, theory of mind, Tim Cook: Apple, twin studies, Uber and Lyft, Uber for X, uber lyft, When a measure becomes a target

Press release. National Aeronautics and Space Administration. January 31, 2018. https://www.nasa.gov/​feature/​nasa-twins-study-confirms-preliminary-findings. NORC General Social Survey. 2017. Data compiled by the Pew Research Center. Open Science Collaboration. “Estimating the Reproducibility of Psychological Science.” Science 349 (2015): aac4716. Pauling, L., and R. B. Corey. “A Proposed Structure for the Nucleic Acids.” Proceedings of the National Academy of Sciences 39 (1953): 84–97. Pauling, Linus. Vitamin C and the Common Cold. 1st edition. San Francisco: W. H. Freeman, 1970.

The Data Detective: Ten Easy Rules to Make Sense of Statistics by Tim Harford

Abraham Wald, access to a mobile phone, Ada Lovelace, affirmative action, algorithmic bias, Automated Insights, banking crisis, basic income, behavioural economics, Black Lives Matter, Black Swan, Bretton Woods, British Empire, business cycle, Cambridge Analytica, Capital in the Twenty-First Century by Thomas Piketty, Cass Sunstein, Charles Babbage, clean water, collapse of Lehman Brothers, contact tracing, coronavirus, correlation does not imply causation, COVID-19, cuban missile crisis, Daniel Kahneman / Amos Tversky, data science, David Attenborough, Diane Coyle, disinformation, Donald Trump, Estimating the Reproducibility of Psychological Science, experimental subject, fake news, financial innovation, Florence Nightingale: pie chart, Gini coefficient, Great Leap Forward, Hans Rosling, high-speed rail, income inequality, Isaac Newton, Jeremy Corbyn, job automation, Kickstarter, life extension, meta-analysis, microcredit, Milgram experiment, moral panic, Netflix Prize, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, opioid epidemic / opioid crisis, Paul Samuelson, Phillips curve, publication bias, publish or perish, random walk, randomized controlled trial, recommendation engine, replication crisis, Richard Feynman, Richard Thaler, rolodex, Ronald Reagan, selection bias, sentiment analysis, Silicon Valley, sorting algorithm, sparse data, statistical model, stem cell, Stephen Hawking, Steve Bannon, Steven Pinker, survivorship bias, systematic bias, TED Talk, universal basic income, W. E. B. Du Bois, When a measure becomes a target

An alternative metric is to ask how many of the replication studies produced results that passed the standard (but rather problematic) hurdle of “statistical significance.” Only thirty-six did; ninety-seven of the original studies had cleared that hurdle. Open Science Collaboration, “Estimating the Reproducibility of Psychological Science,” Science 28, no. 6251 (August 2015), 349, DOI: 10.1126/science.aac4716. 12. Brief film on YouTube: “Derren Brown—Ten Heads in a Row,” ThinkSceptically, April 8, 2012, https://www.youtube.com/watch?v=n1SJ-Tn3bcQ. 13. Planet Money, episode 677. 14. F. J. Anscombe, “Fixed-Sample-Size Analysis of Sequential Observations,” Biometrics 10, no. 1 (1954), 89–100, JSTOR, www.jstor.org/stable/3001665; and Andrew Gelman’s blog post “Statistical Inference, Modeling and Social Science,” May 2, 2018, https://statmodeling.stat.columbia.edu/2018/05/02/continuously-increased-number-animals-statistical-significance-reached-support-conclusions-think-not-bad-actually/. 15.

Can It Happen Here?: Authoritarianism in America by Cass R. Sunstein

active measures, affirmative action, Affordable Care Act / Obamacare, airline deregulation, anti-communist, anti-globalists, availability heuristic, behavioural economics, Black Lives Matter, Brexit referendum, business cycle, Cambridge Analytica, Cass Sunstein, cognitive load, David Brooks, disinformation, Donald Trump, driverless car, Edward Snowden, Estimating the Reproducibility of Psychological Science, failed state, fake news, Filter Bubble, Francis Fukuyama: the end of history, Garrett Hardin, ghettoisation, illegal immigration, immigration reform, Isaac Newton, job automation, Joseph Schumpeter, Long Term Capital Management, microaggression, Nate Silver, Network effects, New Journalism, night-watchman state, nudge theory, obamacare, Paris climate accords, post-truth, Potemkin village, random walk, Richard Thaler, road to serfdom, Ronald Reagan, seminal paper, Steve Bannon, TED Talk, the scientific method, Tragedy of the Commons, Tyler Cowen, War on Poverty, WikiLeaks, World Values Survey

Princeton, NJ: Princeton University Press, 2017. Camerer, Colin F., and Eric J. Johnson. “The Process-Performance Paradox in Expert Judgment: How Can Experts Know So Much and Predict So Badly?” Research on Judgment and Decision Making: Currents, Connections, and Controversies 342 (1997). Collaboration, Open Science. “Estimating the Reproducibility of Psychological Science.” Science 349, no. 6251 (2015): 10.1126/science.aac4716. DiPrete, Thomas A., and Gregory M. Eirich. “Cumulative Advantage as a Mechanism for Inequality: A Review of Theoretical and Empirical Developments.” Annual Review of Sociology 32, no. 1 (2006): 271–97. Dunning, Thad.

Rationality: What It Is, Why It Seems Scarce, Why It Matters by Steven Pinker

affirmative action, Albert Einstein, autonomous vehicles, availability heuristic, Ayatollah Khomeini, backpropagation, basic income, behavioural economics, belling the cat, Black Lives Matter, butterfly effect, carbon tax, Cass Sunstein, choice architecture, classic study, clean water, Comet Ping Pong, coronavirus, correlation coefficient, correlation does not imply causation, COVID-19, critical race theory, crowdsourcing, cuban missile crisis, Daniel Kahneman / Amos Tversky, data science, David Attenborough, deep learning, defund the police, delayed gratification, disinformation, Donald Trump, Dr. Strangelove, Easter island, effective altruism, en.wikipedia.org, Erdős number, Estimating the Reproducibility of Psychological Science, fake news, feminist movement, framing effect, George Akerlof, George Floyd, germ theory of disease, high batting average, if you see hoof prints, think horses—not zebras, index card, Jeff Bezos, job automation, John Nash: game theory, John von Neumann, libertarian paternalism, Linda problem, longitudinal study, loss aversion, Mahatma Gandhi, meta-analysis, microaggression, Monty Hall problem, Nash equilibrium, New Journalism, Paul Erdős, Paul Samuelson, Peter Singer: altruism, Pierre-Simon Laplace, placebo effect, post-truth, power law, QAnon, QWERTY keyboard, Ralph Waldo Emerson, randomized controlled trial, replication crisis, Richard Thaler, scientific worldview, selection bias, social discount rate, social distancing, Social Justice Warrior, Stanford marshmallow experiment, Steve Bannon, Steven Pinker, sunk-cost fallacy, TED Talk, the scientific method, Thomas Bayes, Tragedy of the Commons, trolley problem, twin studies, universal basic income, Upton Sinclair, urban planning, Walter Mischel, yellow journalism, zero-sum game

Journal of Elections, Public Opinion and Parties, 29, 222–44. https://doi.org/10.1080/17457289.2018.1465061. O’Keefe, S. M. 2020. One in three Americans would not get COVID-19 vaccine. Gallup, Aug. 7. https://news.gallup.com/poll/317018/one-three-americans-not-covid-vaccine.aspx. Open Science Collaboration. 2015. Estimating the reproducibility of psychological science. Science, 349. https://doi.org/10.1126/science.aac4716. Paresky, P., Haidt, J., Strossen, N., & Pinker, S. 2020. The New York Times surrendered to an outrage mob. Journalism will suffer for it. Politico, May 14. https://www.politico.com/news/magazine/2020/05/14/bret-stephens-new-york-times-outrage-backlash-256494.

Model Thinker: What You Need to Know to Make Data Work for You by Scott E. Page

Airbnb, Albert Einstein, Alfred Russel Wallace, algorithmic trading, Alvin Roth, assortative mating, behavioural economics, Bernie Madoff, bitcoin, Black Swan, blockchain, business cycle, Capital in the Twenty-First Century by Thomas Piketty, Checklist Manifesto, computer age, corporate governance, correlation does not imply causation, cuban missile crisis, data science, deep learning, deliberate practice, discrete time, distributed ledger, Easter island, en.wikipedia.org, Estimating the Reproducibility of Psychological Science, Everything should be made as simple as possible, experimental economics, first-price auction, Flash crash, Ford Model T, Geoffrey West, Santa Fe Institute, germ theory of disease, Gini coefficient, Higgs boson, High speed trading, impulse control, income inequality, Isaac Newton, John von Neumann, Kenneth Rogoff, knowledge economy, knowledge worker, Long Term Capital Management, loss aversion, low skilled workers, Mark Zuckerberg, market design, meta-analysis, money market fund, multi-armed bandit, Nash equilibrium, natural language processing, Network effects, opioid epidemic / opioid crisis, p-value, Pareto efficiency, pattern recognition, Paul Erdős, Paul Samuelson, phenotype, Phillips curve, power law, pre–internet, prisoner's dilemma, race to the bottom, random walk, randomized controlled trial, Richard Feynman, Richard Thaler, Robert Solow, school choice, scientific management, sealed-bid auction, second-price auction, selection bias, six sigma, social graph, spectrum auction, statistical model, Stephen Hawking, Supply of New York City Cabdrivers, systems thinking, tacit knowledge, The Bell Curve by Richard Herrnstein and Charles Murray, The Great Moderation, the long tail, The Rise and Fall of American Growth, the rule of 72, the scientific method, The Spirit Level, the strength of weak ties, The Wisdom of Crowds, Thomas Malthus, Thorstein Veblen, Tragedy of the Commons, urban sprawl, value at risk, web application, winner-take-all economy, zero-sum game

The Logic of Collective Action: Public Goods and the Theory of Groups. Cambridge, MA: Harvard University Press. O’Neil, Cathy 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. New York, NY: Crown. Open Science Collaboration. 2015. “Estimating the Reproducibility of Psychological Science.” Science 349: 6251. Organization for Economic Co-operation and Development. 1996. The Knowledge Based Economy. Paris: OECD. Ormerod, Paul. 2012. Positive Linking: How Networks Can Revolutionise the World. London: Faber and Faber. Ostrom, Elinor. 2004. Understanding Institutional Diversity.