correlation coefficient

107 results back to index

Statistics in a Nutshell by Sarah Boslaugh

Antoine Gombaud: Chevalier de Méré, Bayesian statistics, business climate, computer age, confounding variable, correlation coefficient, experimental subject, Florence Nightingale: pie chart, income per capita, iterative process, job satisfaction, labor-force participation, linear programming, longitudinal study, meta-analysis, p-value, pattern recognition, placebo effect, probability theory / Blaise Pascal / Pierre de Fermat, publication bias, purchasing power parity, randomized controlled trial, selection bias, six sigma, sparse data, statistical model, systematic bias, The Design of Experiments, the scientific method, Thomas Bayes, Two Sigma, Vilfredo Pareto

The different t-tests and their uses Chapter 7. The Pearson Correlation Coefficient The Pearson correlation coefficient is a measure of linear association between two interval- or ratio-level variables. Although there are other types of correlation (several are discussed in Chapter 5, including the Spearman rank-order correlation coefficient), the Pearson correlation coefficient is the most common, and often the label “Pearson” is dropped, and we simply speak of “correlation” or “the correlation coefficient.” Unless otherwise specified in this book, “correlation” means the Pearson correlation coefficient. Correlations are often computed during the exploratory stage of a research project to see what kinds of relationships the different continuous variables have with each other, and often scatterplots (discussed in Chapter 4) are created to examine these relationships graphically.

…

Calculating Cramer’s V The Point-Biserial Correlation Coefficient The point-biserial correlation coefficient is a measure of association between a dichotomous variable and a continuous variable. Mathematically, it is equivalent to the Pearson correlation coefficient (discussed in detail in Chapter 7), but because one of the variables is dichotomous, a different formula can be used to calculate it. Suppose we are interested in the strength of association between gender (dichotomous) and adult height (continuous). The point-biserial correlation is symmetric, like the Pearson correlation coefficient, but for ease of notation we designate height as X and gender as Y and code Y so 0 = males and 1 = females.

…

Correlation Statistics for Categorical Data The most common measure of association for two variables, Pearson’s correlation coefficient (discussed in Chapter 7) requires variables measured on at least the interval level. However, several measures of association have been developed for categorical and ordinal data, and they are interpreted similarly to the Pearson correlation coefficient. These measures are often produced using a statistical software package or an online calculator, although they can also be calculated by hand. As with Pearson’s correlation coefficient, the correlation statistics discussed in this section are measures of association only, and statements about causality cannot be supported by a correlation coefficient alone.

Analysis of Financial Time Series by Ruey S. Tsay

Asian financial crisis, asset allocation, backpropagation, Bayesian statistics, Black-Scholes formula, Brownian motion, business cycle, capital asset pricing model, compound rate of return, correlation coefficient, data acquisition, discrete time, financial engineering, frictionless, frictionless market, implied volatility, index arbitrage, inverted yield curve, Long Term Capital Management, market microstructure, martingale, p-value, pattern recognition, random walk, risk free rate, risk tolerance, short selling, statistical model, stochastic process, stochastic volatility, telemarketer, transaction costs, value at risk, volatility smile, Wiener process, yield curve

The concurrent, or lag-zero, cross-correlation matrix of rt is defined as ρ0 ≡ [ρi j (0)] = D−1 Γ0 D−1 . More specifically, the (i, j)th element of ρ0 is Cov(rit , r jt ) i j (0) ρi j (0) = = , std(r ii (0) j j (0) it )std(r jt ) which is the correlation coefficient between rit and r jt . In a time series analysis, such a correlation coefficient is referred to as a concurrent, or contemporaneous, CROSS - CORRELATION 301 correlation coefficient because it is the correlation of the two series at time t. It is easy to see that ρi j (0) = ρ ji (0), −1 ≤ ρi j (0) ≤ 1, and ρii (0) = 1 for 1 ≤ i, j ≤ k. Thus, ρ(0) is a symmetric matrix with unit diagonal elements.

…

To overcome this difficulty, we use the simplifying notation of Tiao and Box (1981) and define a simplified cross-correlation matrix consisting of three symbols “+,” “−,” and “.,” where 1. “+” means√that the corresponding correlation coefficient is greater than or equal to 2/ T , 2. “−” means √ that the corresponding correlation coefficient is less than or equal to −2/ T , and √ 3. “.” means √ that the corresponding correlation coefficient is between −2/ T and 2/ T , √ where 1/ T is the asymptotic 5% critical value of the sample correlation under the assumption that rt is a white noise series. CROSS - CORRELATION 305 Table 8.1.

…

This bivariate GARCH(1, 1) model shows a feedback relationship between the volatilities of the two monthly log returns. 9.2.2 Time-Varying Correlation Models A major drawback of the constant-correlation volatility models is that the correlation coefficient tends to change over time in a real application. Consider the monthly log returns of IBM stock and the S&P 500 index used in Example 9.2. It is hard to justify that the S&P 500 index return, which is a weighted average, can maintain a constant correlation coefficient with IBM return over the past 70 years. Figure 9.5 shows the sample correlation coefficient between the two monthly log return series using a moving window of 120 observations (i.e., 10 years). The correlation changes over time and appears to be decreasing in recent years.

Advanced Stochastic Models, Risk Assessment, and Portfolio Optimization: The Ideal Risk, Uncertainty, and Performance Measures by Frank J. Fabozzi

algorithmic trading, Benoit Mandelbrot, Black Monday: stock market crash in 1987, capital asset pricing model, collateralized debt obligation, correlation coefficient, distributed generation, diversified portfolio, financial engineering, fixed income, global macro, index fund, junk bonds, Louis Bachelier, Myron Scholes, p-value, power law, quantitative trading / quantitative finance, random walk, risk free rate, risk-adjusted returns, short selling, stochastic volatility, subprime mortgage crisis, Thomas Bayes, transaction costs, value at risk

The resulting measure is the Pearson correlation coefficient or simply the correlation coefficient defined by(5.19) where the covariance is divided by the product of the standard deviations of x and y. By definition, rx,y ∈[−1,1] for any bivariate quantitative data. Hence, we can compare different data with respect to the correlation coefficient equation (5.19). Generally, we make the following distinctionrx,y < 0 Negative correlation rx,y = 0 No correlation rx,y > 0 Positive correlation to indicate the possible direction of joint behavior. In contrast to the covariance, the correlation coefficient is invariant with respect to linear transformation.

…

When its value is negative, we say that the random variables X and Y are negatively correlated, while they are positively correlated in the case of a positive correlation coefficient. When the correlation is zero, due to a zero covariance, we refer to X and Y as uncorrelated. We summarize this below: −1 ≤ ρX ,Y ≤ 1 −1≤ ρX,Y < 0 X and Y negatively correlated ρX,Y = 0 X and Y uncorrelated 0 < ρX ,Y ≤ 1 X and Y positively correlated As with the covariances of a k-dimensional random vector, we list the correlation coefficients of all pairwise combinations of the k components in a k-by-k matrix This matrix, referred to as the correlation coefficient matrix and denoted by Γ, is also symmetric since the correlation coefficients are symmetric.

…

That is, the correlation coefficient161 of two random variables X and Y, denoted by ρX,Y is defined as(14.22) We expressed the standard deviations as the square roots of the respective variances and Note that the correlation coefficient is equal to one, that is, ρX,X = 1, for the correlation between the random variable X with itself. This can be seen from (14.22) by inserting for the covariance in the numerator, and having , in the denominator. Moreover, the correlation coefficient is symmetric. This is due to definition (14.22) and the fact that the covariance is symmetric. The correlation coefficient given by (14.22) can take on real values in the range of -1 and 1 only. When its value is negative, we say that the random variables X and Y are negatively correlated, while they are positively correlated in the case of a positive correlation coefficient.

The Intelligent Asset Allocator: How to Build Your Portfolio to Maximize Returns and Minimize Risk by William J. Bernstein

asset allocation, backtesting, book value, buy and hold, capital asset pricing model, commoditize, computer age, correlation coefficient, currency risk, diversification, diversified portfolio, Eugene Fama: efficient market hypothesis, financial engineering, fixed income, index arbitrage, index fund, intangible asset, John Bogle, junk bonds, Long Term Capital Management, p-value, passive investing, prediction markets, random walk, Richard Thaler, risk free rate, risk tolerance, risk-adjusted returns, risk/return, South Sea Bubble, stocks for the long run, survivorship bias, the rule of 72, the scientific method, time value of money, transaction costs, Vanguard fund, Wayback Machine, Yogi Berra, zero-coupon bond

Lastly, Figure 3-5 plots two very poorly correlated assets (correlation coefficient of .068): Japanese small stocks and REITs. This plot is a “scattergram” with no discernable pattern. A good or bad result for one of these assets tells us nothing about the result for the other. Why is this so important? As already discussed the most diversification benefit is obtained from uncorrelated assets. The above Math Details: How to Calculate a Correlation Coefficient In this book’s previous versions, I included a section on the manual calculation of the correlation coefficient. In the personal computer age, this is an exercise in masochism.The easiest way to do this is with a spreadsheet.

…

A three-person office has three interpersonal relationships; a 10-person office has 45 relationships.) Real assets are almost always imperfectly correlated. In other words, an above-average return in one is somewhat more likely to be associated with an above-average return in the other. The degree of correlation is expressed by a correlation coefficient. This value ranges from ⫺1 to ⫹1. Perfectly correlated assets have a correlation coefficient of ⫹1, and uncorrelated assets have a coefficient of 0. Perfectly inversely (or negatively) correlated assets have a coefficient of ⫺1. The easiest way to understand this is to plot the returns of two assets against each other for many periods, as is done in Figures 3-3, 3-4, and 3-5.

…

Most of the points lie on nearly a straight line; a poor return for one was invariably associated with a poor return for the other. The correlation coefficient of .777 for these two assets is quite high. This graph demonstrates that adding U.S. small stocks to a portfolio of U.S. large stocks does not diminish risk very much, as a poor return for one will be very likely associated with a poor return for the other. Figure 3-4 plots two loosely correlated assets—U.S. large stocks (S&P 500) and foreign large stocks (EAFE Index). Although there does appear to be a loose relation between the two, it is far from perfect. The correlation coefficient of this pair is .483. Lastly, Figure 3-5 plots two very poorly correlated assets (correlation coefficient of .068): Japanese small stocks and REITs.

Commodity Trading Advisors: Risk, Performance Analysis, and Selection by Greg N. Gregoriou, Vassilios Karavas, François-Serge Lhabitant, Fabrice Douglas Rouah

Asian financial crisis, asset allocation, backtesting, buy and hold, capital asset pricing model, collateralized debt obligation, commodity trading advisor, compound rate of return, constrained optimization, corporate governance, correlation coefficient, Credit Default Swap, credit default swaps / collateralized debt obligations, currency risk, discrete time, distributed generation, diversification, diversified portfolio, dividend-yielding stocks, financial engineering, fixed income, global macro, high net worth, implied volatility, index arbitrage, index fund, interest rate swap, iterative process, linear programming, London Interbank Offered Rate, Long Term Capital Management, managed futures, market fundamentalism, merger arbitrage, Mexican peso crisis / tequila crisis, p-value, Pareto efficiency, Performance of Mutual Funds in the Period, Ponzi scheme, proprietary trading, quantitative trading / quantitative finance, random walk, risk free rate, risk-adjusted returns, risk/return, selection bias, Sharpe ratio, short selling, stochastic process, survivorship bias, systematic trading, tail risk, technology bubble, transaction costs, value at risk, zero-sum game

Heteroskedasticity was created by letting the values of σ be 5, 10, 15, and 20, with one-fourth of the observations using each value. This allowed us to compare the Spearman correlation coefficient calculated for data sets with and without homoskedasticity. The funds were ranked in ascending order of returns for period one (first 12 months) and period two (last 12 months). From each 24-month period of generated returns, Spearman correlation coefficients were calculated for a fund’s rank in both periods. For the distribution of Spearman correlation coefficients to be suitably approximated by a normal, at least 10 observations are needed. Because 120 pairs are used here, the normal approximation is used.

…

Table 6.3 examines the correlation coefficients between the different CTA indices as well as between the CTA indices and the first two return moments of the Russell 3000 (Russell squared). The results for the entire sample as well as the subsamples confirm our earlier findings. The correlation coefficient between the CTA index, the Financial and Metal Traders Index, the Systematic Traders Index, and the Diversified Traders Index are positive and close to 1 for all the different periods. The Currency Trader Index and the Discretionary Index have the lowest correlation coefficient with the other CTA indices.

…

Returns-enhancing diversifiers possess correlations with the same generic asset class in an up market but are relatively less correlated in a down market. 3. “Ineffective” diversifiers are assets that do not add value, even though they may possess significant correlation coefficients with the generic asset class. CTA Strategies for Returns-Enhancing Diversification 339 To illustrate, a hedge fund strategy that has a negative correlation coefficient in an up-market regime and positive correlation coefficient in a down-market regime provides diversification with no incremental returns. We classify this in the third category, that is, as an ineffective diversifier. Indeed, a strategy with such a characteristic will have the opposite effect of a good diversifier as it weakens the returns on an uptrend and exaggerates the negative returns of the portfolio.

Statistics hacks by Bruce Frey

Bayesian statistics, Berlin Wall, correlation coefficient, Daniel Kahneman / Amos Tversky, distributed generation, en.wikipedia.org, feminist movement, G4S, game design, Hacker Ethic, index card, Linda problem, Milgram experiment, Monty Hall problem, p-value, place-making, reshoring, RFID, Search for Extraterrestrial Intelligence, SETI@home, Silicon Valley, statistical model, sugar pill, systematic bias, Thomas Bayes

The Association's marketer has support for her hypothesis. Computing a Correlation Coefficient Just eyeballing two columns of numbers from a sample, though, is usually not enough to really know whether there is a relationship between two things. The marketing specialist in our example wants to use a single number to more precisely describe whatever relationship is seen. The correlation coefficient takes into account all the information we used when we looked at our two columns of numbers in Table 2-1 and decided whether there was a relationship there. The correlation coefficient is produced through a formula that does the following things: Looks at each score in a column Sees how distant that score is from the mean of that column Identifies the distance from the mean of its matching score in the other column Multiplies the paired distances together Averages the results of those multiplications If this were a statistics textbook, I'd have to present a somewhat complicated formula for calculating the correlation coefficient.

…

This is very close to 1.0, which is the strongest a positive correlation can be, so the cheese-to-cheesecake correlation represents a very strong relationship. Interpreting a Correlation Coefficient Somewhat magically, the correlation formula process produces a number, ranging in value from -1.00 to +1.00, that measures the strength of relationship between two variables. Positive signs indicate the relationship is in the same direction. As one value increases, the other value increases. Negative signs indicate the relationship is in the opposite direction. As one value increases, the other value decreases. An important point to make is that the correlation coefficient provides a standardized measure of the strength of linear relationship between two variables [Hack #12].

…

Let's imagine that a small college decides to use scores on the American College Test (ACT) as a predictor of college grade point average (GPA) at the end of students' first years. The admissions office goes back through a few years of records and gathers the ACT scores and freshman GPAs for a couple hundred students. They discover, to their delight, that there is a moderate relationship between these two variables: a correlation coefficient of .55. Correlation coefficients are a measure of the strength of linear relationships between two variables [Hack #11], and .55 indicates a fairly large relationship. This is good news because the existence of a relationship between the two makes ACT scores a good candidate as a predictor to guess GPA. Simple linear regression is the procedure that produces all the values we need to cook up the magic formula that will predict the future.

The Art of Statistics: Learning From Data by David Spiegelhalter

Abraham Wald, algorithmic bias, Anthropocene, Antoine Gombaud: Chevalier de Méré, Bayesian statistics, Brexit referendum, Carmen Reinhart, Charles Babbage, complexity theory, computer vision, confounding variable, correlation coefficient, correlation does not imply causation, dark matter, data science, deep learning, DeepMind, Edmond Halley, Estimating the Reproducibility of Psychological Science, government statistician, Gregor Mendel, Hans Rosling, Higgs boson, Kenneth Rogoff, meta-analysis, Nate Silver, Netflix Prize, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, p-value, placebo effect, probability theory / Blaise Pascal / Pierre de Fermat, publication bias, randomized controlled trial, recommendation engine, replication crisis, self-driving car, seminal paper, sparse data, speech recognition, statistical model, sugar pill, systematic bias, TED Talk, The Design of Experiments, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Malthus, Two Sigma

parameters: the unknown quantities in a statistical model, generally denoted with Greek letters. Pearson correlation coefficient: for a set of n paired numbers, (x1, y1), (x2, y2) … (xn, yn), when , sx are the sample mean and standard deviation of the xs, and , sy are the sample mean and standard deviation of the ys, the Pearson correlation coefficient is given by Suppose xs and ys have both been standardized to Z-scores given by us and vs respectively, so that ui = (xi – )/sx, and vi = (yi – )/sy. Then the Pearson correlation coefficient can be expressed as , that is the ‘cross-product’ of the Z-scores. percentile (of a population): there is, for example, a 70% chance of drawing a random observation below the 70th percentile.

…

Figure 2.6 Two sets of (fictitious) data-points for which the Pearson correlation coefficients are both 0. This clearly does not mean there is no relationship between the two variables being plotted. From Alberto Cairo’s wonderful Datasaurus Dozen4. The Pearson correlation is 0.17 for the 2012–2015 data in Figure 2.5(b), and the Spearman’s rank correlation is −0.03, suggesting that there is no longer any clear relationship between the number of cases and survival rates. However, with so few hospitals the correlation coefficient can be very sensitive to individual data-points – if we remove the smallest hospital, which has a high survival rate, the Pearson correlation jumps to 0.42.

…

‘Correlation Does Not Imply Causation’ We saw in the last chapter how Pearson’s correlation coefficient measures how close the points on a scatter-plot are to a straight line. When considering English hospitals conducting children’s heart surgery in the 1990s, and plotting the number of cases against their survival, the high correlation showed that bigger hospitals were associated with lower mortality. But we could not conclude that bigger hospitals caused the lower mortality. This cautious attitude has a long pedigree. When Karl Pearson’s newly developed correlation coefficient was being discussed in the journal Nature in 1900, a commentator warned that ‘correlation does not imply causation’.

The Art of Statistics: How to Learn From Data by David Spiegelhalter

Abraham Wald, algorithmic bias, Antoine Gombaud: Chevalier de Méré, Bayesian statistics, Brexit referendum, Carmen Reinhart, Charles Babbage, complexity theory, computer vision, confounding variable, correlation coefficient, correlation does not imply causation, dark matter, data science, deep learning, DeepMind, Edmond Halley, Estimating the Reproducibility of Psychological Science, government statistician, Gregor Mendel, Hans Rosling, Higgs boson, Kenneth Rogoff, meta-analysis, Nate Silver, Netflix Prize, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, p-value, placebo effect, probability theory / Blaise Pascal / Pierre de Fermat, publication bias, randomized controlled trial, recommendation engine, replication crisis, self-driving car, seminal paper, sparse data, speech recognition, statistical model, sugar pill, systematic bias, TED Talk, The Design of Experiments, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Malthus, Two Sigma

parameters: the unknown quantities in a statistical model, generally denoted with Greek letters. Pearson correlation coefficient: for a set of n paired numbers, (x1, y1), (x2, y2)… (xn, yn), when , sx are the sample mean and standard deviation of the xs, and , sy are the sample mean and standard deviation of the ys, the Pearson correlation coefficient is given by Suppose xs and ys have both been standardized to Z-scores given by us and vs respectively, so that ui = (xi − )/sx, and vi = (yi − )/sy. Then the Pearson correlation coefficient can be expressed as , that is the ‘cross-product’ of the Z-scores. percentile (of a population): there is, for example, a 70% chance of drawing a random observation below the 70th percentile.

…

Figure 2.6 Two sets of (fictitious) data-points for which the Pearson correlation coefficients are both 0. This clearly does not mean there is no relationship between the two variables being plotted. From Alberto Cairo’s wonderful Datasaurus Dozen4. The Pearson correlation is 0.17 for the 2012–2015 data in Figure 2.5(b), and the Spearman’s rank correlation is −0.03, suggesting that there is no longer any clear relationship between the number of cases and survival rates. However, with so few hospitals the correlation coefficient can be very sensitive to individual data-points—if we remove the smallest hospital, which has a high survival rate, the Pearson correlation jumps to 0.42.

…

‘Correlation Does Not Imply Causation’ We saw in the last chapter how Pearson’s correlation coefficient measures how close the points on a scatter-plot are to a straight line. When considering English hospitals conducting children’s heart surgery in the 1990s, and plotting the number of cases against their survival, the high correlation showed that bigger hospitals were associated with lower mortality. But we could not conclude that bigger hospitals caused the lower mortality. This cautious attitude has a long pedigree. When Karl Pearson’s newly developed correlation coefficient was being discussed in the journal Nature in 1900, a commentator warned that ‘correlation does not imply causation’.

Stocks for the Long Run, 4th Edition: The Definitive Guide to Financial Market Returns & Long Term Investment Strategies by Jeremy J. Siegel

addicted to oil, Alan Greenspan, asset allocation, backtesting, behavioural economics, Black-Scholes formula, book value, Bretton Woods, business cycle, buy and hold, buy low sell high, California gold rush, capital asset pricing model, cognitive dissonance, compound rate of return, correlation coefficient, currency risk, Daniel Kahneman / Amos Tversky, diversification, diversified portfolio, dividend-yielding stocks, dogs of the Dow, equity premium, equity risk premium, Eugene Fama: efficient market hypothesis, Everybody Ought to Be Rich, fixed income, German hyperinflation, implied volatility, index arbitrage, index fund, Isaac Newton, it's over 9,000, John Bogle, joint-stock company, Long Term Capital Management, loss aversion, machine readable, market bubble, mental accounting, Money creation, Myron Scholes, new economy, oil shock, passive investing, Paul Samuelson, popular capitalism, prediction markets, price anchoring, price stability, proprietary trading, purchasing power parity, random walk, Richard Thaler, risk free rate, risk tolerance, risk/return, Robert Shiller, Ronald Reagan, shareholder value, short selling, South Sea Bubble, stock buybacks, stocks for the long run, subprime mortgage crisis, survivorship bias, technology bubble, The Great Moderation, The Wisdom of Crowds, transaction costs, tulip mania, uptick rule, Vanguard fund, vertical integration

The diversifying strength of an asset is measured by the correlation coefficient. The correlation coefficient, which theoretically ranges between –1 and +1, measures the correlation between an asset’s return and the return of the rest of the portfolio. The lower the correlation coefficient, the better the asset serves as a portfolio diversifier. Assets with negative correlations are particularly good diversifiers. As the correlation coefficient between the asset and portfolio returns increases, the diversifying quality of the asset declines. The correlation coefficient between annual stock and bond returns for six subperiods between 1926 and 2006 is shown in Figure 2-4.

…

The correlation of returns between stocks or portfolios of stocks is measured by the correlation coefficient. A good case for investors is if there is no correlation between the stock returns of two countries, and the correlation coefficient is equal to zero. In this case, an investor who allocates his or her portfolio equally between each country can reduce his or her risk by almost one-third, compared to investing in a single country. As the correlation coefficient increases, the gains from diversification dwindle, and if there is perfect synchronization of returns, the correlation coefficient equals 1 and there is no gain (but no loss) from diversification.

…

For every percentage point increase in the expected risk of U.S. returns or decrease in the expected risk of EAFE returns, the U.S. allocation falls 6.5 percentage points. And a 0.10 increase in the correlation coefficient between EAFE and U.S. returns will lower the EAFE allocation by just over 2 percentage points.8 7 Readers who wish to understand risk-return analysis can go to Richard A. Brealey, Stewart C. Myers, and Franklin Allen, Principles of Corporate Finance,” 8th ed., New York: McGraw-Hill, 2006. 8 The impact of a change in the correlation coefficient is highly nonlinear. If the correlation rises to 0.77, the U.S. allocation will rise over 6 percentage points; if it rises to 0.87, the U.S. share will rise by over 17 percentage points.

Programming Collective Intelligence by Toby Segaran

algorithmic management, always be closing, backpropagation, correlation coefficient, Debian, en.wikipedia.org, Firefox, full text search, functional programming, information retrieval, PageRank, prediction markets, recommendation engine, slashdot, social bookmarking, sparse data, Thomas Bayes, web application

Euclidean distance A clear implementation of this formula is shown here: def euclidean(p,q): sumSq=0.0 # add up the squared differences for i in range(len(p)): sumSq+=(p[i]-q[i])**2 # take the square root return (sumSq**0.5) Euclidean distance is used in several places in this book to determine how similar two items are. Pearson Correlation Coefficient The Pearson correlation coefficient is a measure of how highly correlated two variables are. It is a value between 1 and −1, where 1 indicates that the variables are perfectly correlated, 0 indicates no correlation, and −1 means they are perfectly inversely correlated. Figure B-2 shows the Pearson correlation coefficient. Figure B-2. Pearson correlation coefficient This can be implemented with the following code: def pearson(x,y): n=len(x) vals=range(n) # Simple sums sumx=sum([float(x[i]) for i in vals]) sumy=sum([float(y[i]) for i in vals]) # Sum up the squares sumxSq=sum([x[i]**2.0 for i in vals]) sumySq=sum([y[i]**2.0 for i in vals]) # Sum up the products pSum=sum([x[i]*y[i] for i in vals]) # Calculate Pearson score num=pSum-(sumx*sumy/n) den=((sumxSq-pow(sumx,2)/n)*(sumySq-pow(sumy,2)/n))**.5 if den==0: return 1 r=num/den return r We used the Pearson correlation in Chapter 2 to calculate the level of similarity between people's preferences.

…

, Learning from Clicks, Learning from Clicks, Setting Up the Database, Feeding Forward, Training with Backpropagation, Training Test, Training Test artificial, Learning from Clicks, Learning from Clicks, Setting Up the Database, Feeding Forward, Training with Backpropagation, Training Test, Training Test backpropagation, Training with Backpropagation connecting to search engine, Training Test designing click-training network, Learning from Clicks feeding forward, Feeding Forward setting up database, Setting Up the Database training test, Training Test neural network classifier, Exercises neural networks, Neural Networks, Neural Networks, Neural Networks, Neural Networks, Training a Neural Network, Training a Neural Network, Training a Neural Network, Strengths and Weaknesses, Strengths and Weaknesses backpropagation, and, Training a Neural Network black box method, Strengths and Weaknesses combinations of words, and, Neural Networks multilayer perceptron network, Neural Networks strengths and weaknesses, Strengths and Weaknesses synapses, and, Neural Networks training, Training a Neural Network using code, Training a Neural Network news sources, A Corpus of News newsfeatures.py, Selecting Sources, Downloading Sources, Downloading Sources, Downloading Sources, Converting to a Matrix, Using NumPy, The Algorithm, Displaying the Results, Displaying the Results, Displaying by Article, Displaying by Article getarticlewords function, Downloading Sources makematrix function, Converting to a Matrix separatewords function, Downloading Sources shape function, The Algorithm showarticles function, Displaying the Results, Displaying by Article showfeatures function, Displaying the Results, Displaying by Article stripHTML function, Downloading Sources transpose function, Using NumPy nn.py, Setting Up the Database, Setting Up the Database, Setting Up the Database, Setting Up the Database searchnet class, Setting Up the Database, Setting Up the Database, Setting Up the Database, Setting Up the Database generatehiddennode function, Setting Up the Database getstrength method, Setting Up the Database setstrength method, Setting Up the Database nnmf.py, The Algorithm difcost function, The Algorithm non-negative matrix factorization (NMF), Supervised versus Unsupervised Learning, Clustering, Non-Negative Matrix Factorization, Non-Negative Matrix Factorization, Non-Negative Matrix Factorization, Using Your NMF Code factorization, Supervised versus Unsupervised Learning goal of, Non-Negative Matrix Factorization update rules, Non-Negative Matrix Factorization using code, Using Your NMF Code normalization, Normalization Function numerical predictions, Building Price Models numpredict.py, Building a Sample Dataset, Building a Sample Dataset, Defining Similarity, Defining Similarity, Defining Similarity, Defining Similarity, Subtraction Function, Subtraction Function, Weighted kNN, Weighted kNN, Cross-Validation, Cross-Validation, Cross-Validation, Heterogeneous Variables, Scaling Dimensions, Optimizing the Scale, Optimizing the Scale, Uneven Distributions, Estimating the Probability Density, Graphing the Probabilities, Graphing the Probabilities, Graphing the Probabilities createcostfunction function, Optimizing the Scale createhiddendataset function, Uneven Distributions crossvalidate function, Cross-Validation, Optimizing the Scale cumulativegraph function, Graphing the Probabilities distance function, Defining Similarity dividedata function, Cross-Validation euclidian function, Defining Similarity gaussian function, Weighted kNN getdistances function, Defining Similarity inverseweight function, Subtraction Function knnestimate function, Defining Similarity probabilitygraph function, Graphing the Probabilities probguess function, Estimating the Probability Density, Graphing the Probabilities rescale function, Scaling Dimensions subtractweight function, Subtraction Function testalgorithm function, Cross-Validation weightedknn function, Weighted kNN wineprice function, Building a Sample Dataset wineset1 function, Building a Sample Dataset wineset2 function, Heterogeneous Variables NumPy, Using NumPy, Using NumPy, Simple Usage Example, NumPy, Installation on Other Platforms, Installation on Other Platforms installation on other platforms, Installation on Other Platforms installation on Windows, Simple Usage Example usage example, Installation on Other Platforms using, Using NumPy O online technique, Strengths and Weaknesses Open Web APIs, Open APIs optimization, Optimization, Group Travel, Representing Solutions, Representing Solutions, Representing Solutions, Representing Solutions, The Cost Function, The Cost Function, The Cost Function, Random Searching, Hill Climbing, Simulated Annealing, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms, Optimizing for Preferences, Optimizing for Preferences, The Cost Function, The Cost Function, Network Visualization, Network Visualization, Counting Crossed Lines, Drawing the Network, Exercises, Exercises, Exercises, Exercises, Exercises, Exercises, Exercises, Optimizing the Scale, Exercises, Optimization, Optimization annealing starting points, Exercises cost function, The Cost Function, Optimization exercises, Exercises genetic algorithms, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms crossover or breeding, Genetic Algorithms generation, Genetic Algorithms mutation, Genetic Algorithms population, Genetic Algorithms genetic optimization stopping criteria, Exercises group travel cost function, Exercises group travel planning, Group Travel, Representing Solutions, Representing Solutions, Representing Solutions, The Cost Function, The Cost Function car rental period, The Cost Function departure time, Representing Solutions price, Representing Solutions time, Representing Solutions waiting time, The Cost Function hill climbing, Hill Climbing line angle penalization, Exercises network visualization, Network Visualization, Counting Crossed Lines, Drawing the Network counting crossed lines, Counting Crossed Lines drawing networks, Drawing the Network layout problem, Network Visualization network vizualization, Network Visualization pairing students, Exercises preferences, Optimizing for Preferences, Optimizing for Preferences, The Cost Function, The Cost Function cost function, The Cost Function running, The Cost Function student dorm, Optimizing for Preferences random searching, Random Searching representing solutions, Representing Solutions round-trip pricing, Exercises simulated annealing, Simulated Annealing where it may not work, Genetic Algorithms optimization.py, Group Travel, Representing Solutions, Representing Solutions, The Cost Function, Random Searching, Hill Climbing, Simulated Annealing, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms, Optimizing the Scale annealingoptimize function, Simulated Annealing geneticoptimize function, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms, Genetic Algorithms elite, Genetic Algorithms maxiter, Genetic Algorithms mutprob, Genetic Algorithms popsize, Genetic Algorithms getminutes function, Representing Solutions hillclimb function, Hill Climbing printschedule function, Representing Solutions randomoptimize function, Random Searching schedulecost function, The Cost Function P PageRank algorithm, Real-Life Examples, The PageRank Algorithm pairing students, Exercises Pandora, Real-Life Examples parse tree, Programs As Trees Pearson correlation, Hierarchical Clustering, Viewing Data in Two Dimensions hierarchical clustering, Hierarchical Clustering multidimensional scaling, Viewing Data in Two Dimensions Pearson correlation coefficient, Pearson Correlation Score, Pearson Correlation Coefficient, Pearson Correlation Coefficient code, Pearson Correlation Coefficient Pilgrim, Mark, Universal Feed Parser polynomial transformation, The Kernel Trick poplib, Exercises population, Genetic Algorithms, What Is Genetic Programming?, Creating the Initial Population, Genetic Algorithms diversity and, Creating the Initial Population Porter Stemmer, Finding the Words on a Page Pr(Document), Exercises prediction markets, Real-Life Examples price models, Building a Sample Dataset, Building a Sample Dataset, k-Nearest Neighbors, Exercises, Exercises, Exercises, Exercises, Exercises, Exercises, Exercises, Exercises building sample dataset, Building a Sample Dataset eliminating variables, Exercises exercises, Exercises item types, Exercises k-nearest neighbors (kNN), k-Nearest Neighbors laptop dataset, Exercises leave-one-out cross-validation, Exercises optimizing number of neighbors, Exercises search attributes, Exercises varying ss for graphing probability, Exercises probabilities, Calculating Probabilities, Starting with a Reasonable Guess, Probability of a Whole Document, A Quick Introduction to Bayes' Theorem, Combining the Probabilities, Graphing the Probabilities, Conditional Probability assumed probability, Starting with a Reasonable Guess Bayes' Theorem, A Quick Introduction to Bayes' Theorem combining, Combining the Probabilities conditional probability, Calculating Probabilities graphing, Graphing the Probabilities of entire document given classification, Probability of a Whole Document product marketing, Other Uses for Learning Algorithms public message boards, Filtering Spam pydelicious, Simple Usage Example, Simple Usage Example, pydelicious installation, Simple Usage Example usage example, Simple Usage Example pysqlite, Building the Index, Persisting the Trained Classifiers, Installation on All Platforms, Installation on All Platforms, pysqlite, Simple Usage Example importing, Persisting the Trained Classifiers installation on other platforms, Installation on All Platforms installation on Windows, Installation on All Platforms usage example, Simple Usage Example Python, Style of Examples, Python Tips advantages of, Style of Examples tips, Python Tips Python Imaging Library (PIL), Drawing the Dendrogram, Python Imaging Library, Installation on Windows, Installation on Windows, Installation on Windows installation on other platforms, Installation on Windows usage example, Installation on Windows Windows installation, Installation on Windows Python, genetic programming and, Programs As Trees, Programs As Trees, Representing Trees in Python, Building and Evaluating Trees, Displaying the Program building and evaluating trees, Building and Evaluating Trees displaying program, Displaying the Program representing trees, Representing Trees in Python traversing complete tree, Programs As Trees Q query layer, Design of a Click-Tracking Network querying, Querying, Querying query function, Querying R radial-basis function, The Kernel Trick random searching, Random Searching random-restart hill climbing, Hill Climbing ranking, What's in a Search Engine?

…

, Exercises marketing, Other Uses for Learning Algorithms mass-and-spring algorithm, The Layout Problem matchmaker dataset, Matchmaker Dataset, Difficulties with the Data, Decision Tree Classifier, Categorical Features, Creating the New Dataset, Creating the New Dataset, Applying SVM to the Matchmaker Dataset categorical features, Categorical Features creating new, Creating the New Dataset decision tree algorithm, Decision Tree Classifier difficulties with data, Difficulties with the Data LIBSVM, applying to, Applying SVM to the Matchmaker Dataset scaling data, Creating the New Dataset matchmaker.csv file, Matchmaker Dataset mathematical formulas, Euclidean Distance, Euclidean Distance, Pearson Correlation Coefficient, Weighted Mean, Tanimoto Coefficient, Conditional Probability, Gini Impurity, Entropy, Variance, Gaussian Function, Dot-Products conditional probability, Conditional Probability dot-product, Dot-Products entropy, Entropy Euclidean distance, Euclidean Distance Gaussian function, Gaussian Function Gini impurity, Gini Impurity Pearson correlation coefficient, Pearson Correlation Coefficient Tanimoto coefficient, Tanimoto Coefficient variance, Variance weighted mean, Weighted Mean matplotlib, Graphing the Probabilities, matplotlib, Installation, Simple Usage Example installation, Installation usage example, Simple Usage Example matrix math, Clustering, A Quick Introduction to Matrix Math, A Quick Introduction to Matrix Math, What Does This Have to Do with the Articles Matrix?

Finding Alphas: A Quantitative Approach to Building Trading Strategies by Igor Tulchinsky

algorithmic trading, asset allocation, automated trading system, backpropagation, backtesting, barriers to entry, behavioural economics, book value, business cycle, buy and hold, capital asset pricing model, constrained optimization, corporate governance, correlation coefficient, credit crunch, Credit Default Swap, currency risk, data science, deep learning, discounted cash flows, discrete time, diversification, diversified portfolio, Eugene Fama: efficient market hypothesis, financial engineering, financial intermediation, Flash crash, Geoffrey Hinton, implied volatility, index arbitrage, index fund, intangible asset, iterative process, Long Term Capital Management, loss aversion, low interest rates, machine readable, market design, market microstructure, merger arbitrage, natural language processing, passive investing, pattern recognition, performance metric, Performance of Mutual Funds in the Period, popular capitalism, prediction markets, price discovery process, profit motive, proprietary trading, quantitative trading / quantitative finance, random walk, Reminiscences of a Stock Operator, Renaissance Technologies, risk free rate, risk tolerance, risk-adjusted returns, risk/return, selection bias, sentiment analysis, shareholder value, Sharpe ratio, short selling, Silicon Valley, speech recognition, statistical arbitrage, statistical model, stochastic process, survivorship bias, systematic bias, systematic trading, text mining, transaction costs, Vanguard fund, yield curve

For example, the information ratio is just the average returns divided by the standard deviation of returns. Another key quality of an alpha is its uniqueness, which is evaluated by the correlation coefficient between a given alpha and other existing alphas. An alpha with a lower correlation coefficient normally is considered to be adding more value to the pool of existing alphas. If the number of alphas in the pool is small, the importance of correlation is low. As the number of alphas increases, however, different techniques to measure the correlation coefficient among them become more important in helping the investor diversify his or her portfolio. Portfolio managers will want to include relatively uncorrelated alphas in their portfolios because a diversified portfolio helps to reduce risk.

…

Published 2020 by John Wiley & Sons, Ltd. 62 Finding Alphas where Pit and Pjt denote the PnLs of i th and j th alphas on the t th day, n is the number of days used to measure correlation, and T denotes the matrix transposition. Note: tests usually select the number of days for correlation as two or four years instead of a full history, to save computational resources. Pearson Correlation Coefficient The Pearson correlation coefficient, also known as the Pearson product-­ moment correlation coefficient, has no units and can take values from 1 to 1. The mathematical formula was first developed by Karl Pearson in 1895: cov Pi , Pj r Pi where cov Pi , Pj E Pi Pj Pi (2) Pj Pj is the covariance and Pi and Pj are the standard deviations of Pi and Pj , respectively.

…

In particular, n r t 1 n t 1 Pit Pit Pi Pi 2 Pjt n t 1 Pj Pjt Pj 2 . (3) The coefficient is invariant to linear transformations of either variable. If the sign of the correlation coefficient is positive, it means that the PnLs of the two alphas tend to move in the same direction. When the return on Pi is positive (negative), the return on Pj has a tendency to be positive (negative) as well. Conversely, a negative correlation coefficient shows that the PnLs of the two alphas tend to move in opposite directions. A zero correlation implies that there is no relationship between two PnL vectors. Figure 8.1 shows the variation of maximum correlation as a function of trading signals, using two years’ worth of data.

The Mathematics of Banking and Finance by Dennis W. Cox, Michael A. A. Cox

backpropagation, barriers to entry, Brownian motion, call centre, correlation coefficient, fixed income, G4S, inventory management, iterative process, linear programming, meta-analysis, Monty Hall problem, pattern recognition, random walk, traveling salesman, value at risk

So, from above we know that the gradient of the line is estimated by n n n 1 xi yi − xi yi n i=1 i=1 i=1 â = n n 2 2 1 xi − xi n i=1 i=1 This may be written in a more compact form since the variance (section 5.6) is n n 2 2 1 xi − xi n i=1 i=1 var(x) = n−1 The covariance may be similarly written as: n cov(x, y) = (xi − x̄) (yi − ȳ) i=1 = n−1 n xi yi i=1 n n 1 − xi yi n i=1 i=1 n−1 (*) We can then write the equation for the estimated gradient in a more compact form using this new notation: â = cov(x, y) var(x) 13.3 CORRELATION COEFFICIENT A closely related term to the covariance is the correlation coefficient (r (x, y)), which is simply r (x, y) = cov(x, y) std(x) std(y) where the standard deviation of x (section 5.6) is given by n 2 n 2 1 i=1 xi − n i=1 xi std(x) = n−1 There would be a similar expression for y. The correlation coefficient is a measure of the interdependence of two variables. The coefficient ranges in value from −1 to +1, indicating perfect negative correlation at −1, absence Linear Regression 105 of correlation at zero, and perfect positive correlation at +1.

…

The coefficient ranges in value from −1 to +1, indicating perfect negative correlation at −1, absence Linear Regression 105 of correlation at zero, and perfect positive correlation at +1. Two variables are positively correlated if the correlation coefficient is greater than zero and the line that is drawn to show a relationship between the items sampled has a positive gradient. If the correlation coefficient is negative, the variables are negatively correlated and the line drawn will have a negative gradient. The final option is that the correlation coefficient vanishes, and the gradient vanishes since â = 0, in which case the variables are completely uncorrelated. This means that there is no relationship between the two variables.

…

Index a notation 103–4, 107–20, 135–47 linear regression 103–4, 107–20 slope significance test 112–20 variance 112 abscissa see horizontal axis absolute value, notation 282–4 accuracy and reliability, data 17, 47 adaptive resonance theory 275 addition, mathematical notation 279 addition of normal variables, normal distribution 70 addition rule, probability theory 24–5 additional variables, linear programming 167–70 adjusted cash flows, concepts 228–9 adjusted discount rates, concepts 228–9 Advanced Measurement Approach (AMA) 271 advertising allocation, linear programming 154–7 air-conditioning units 182–5 algorithms, neural networks 275–6 alternatives, decisions 191–4 AMA see Advanced Measurement Approach analysis data 47–52, 129–47, 271–4 Latin squares 131–2, 143–7 linear regression 110–20 projects 190–2, 219–25, 228–34 randomised block design 129–35 sampling 47–52, 129–47 scenario analysis 40, 193–4, 271–4 trends 235–47 two-way classification 135–47 variance 110–20, 121–7 anonimised databases, scenario analysis 273–4 ANOVA (analysis of variance) concepts 110–20, 121–7, 134–47 examples 110–11, 123–7, 134–40 formal background 121–2 linear regression 110–20 randomised block design 134–5, 141–3 tables 110–11, 121–3, 134–47 two-way classification 136–7 appendix 279–84 arithmetic mean, concepts 37–45, 59–60, 65–6, 67–74, 75–81 assets classes 149–57 reliability 17, 47, 215–18, 249–60 replacement of assets 215–18, 249–60 asymptotic distributions 262 ATMs 60 averages see also mean; median; mode concepts 37–9 b notation 103–4, 107–20, 132–5 linear regression 103–4, 107–20 variance 112 back propagation, neural networks 275–7 backwards recursion 179–87 balance sheets, stock 195 bank cashier problem, Monte Carlo simulation 209–12 Bank for International Settlements (BIS) 267–9, 271 banks Basel Accord 262, 267–9, 271 failures 58 loss data 267–9, 271–4 modelling 75–81, 85, 97, 267–9, 271–4 profitable loans 159–66 bar charts comparative data 10–12 concepts 7–12, 54, 56, 59, 205–6, 232–3 discrete data 7–12 examples 9–12, 205–6, 232–3 286 Index bar charts (cont.) narrative explanations 10 relative frequencies 8–12 rules 8–9 uses 7–12, 205–6, 232–3 base rates, trends 240 Basel Accord 262, 267–9, 271 bathtub curves, reliability concepts 249–51 Bayes’theorem, probability theory 27–30, 31 bell-shaped normal distribution see normal distribution bi-directional associative memory 275 bias 1, 17, 47–50, 51–2, 97, 129–35 randomised block design 129–35 sampling 17, 47–50, 51–2, 97, 129–35 skewness 41–5 binomial distribution concepts 55–8, 61–5, 71–2, 98–9, 231–2 examples 56–8, 61–5, 71–2, 98–9 net present value (NPV) 231–2 normal distribution 71–2 Pascal’s triangle 56–7 uses 55, 57, 61–5, 71–2, 98–9, 231–2 BIS see Bank for International Settlements boards of directors 240–1 break-even analysis, concepts 229–30 Brownian motion 22 see also random walks budgets 149–57 calculators, log functions 20, 61 capital Basel Accord 262, 267–9, 271 cost of capital 219–25, 229–30 cash flows adjusted cash flows 228–9 future cash flows 219–25, 227–34, 240–1 net present value (NPV) 219–22, 228–9, 231–2 standard deviation 232–4 central limit theorem concepts 70, 75 examples 70 chi-squared test concepts 83–4, 85, 89, 91–5 contingency tables 92–5 examples 83–4, 85, 89, 91–2 goodness of fit test 91–5 multi-way tables 94–5 tables 84, 91 Chu Shi-Chieh’s Ssu Yuan Y Chien 56 circles, tree diagrams 30–5 class intervals concepts 13–20, 44–5, 63–4, 241–7 histograms 13–20, 44–5 mean calculations 44–5 mid-points 44–5, 241–7 notation 13–14, 20 Sturges’s formula 20 variance calculations 44–5 classical approach, probability theory 22, 27 cluster sampling 50 coin-tossing examples, probability theory 21–3, 53–4 collection techniques, data 17, 47–52, 129–47 colours, graphical presentational approaches 9 combination, probability distribution (density) functions 54–8 common logarithm (base 10) 20 communications, decisions 189–90 comparative data, bar charts 10–12 comparative histograms see also histograms examples 14–19 completed goods 195 see also stock . . . conditional probability, concepts 25–7, 35 confidence intervals, concepts 71, 75–81, 105, 109, 116–20, 190, 262–5 constraining equations, linear programming 159–70 contingency tables, concepts 92–5 continuous approximation, stock control 200–1 continuous case, failures 251 continuous data concepts 7, 13–14, 44–5, 65–6, 251 histograms 7, 13–14 continuous uniform distribution, concepts 64–6 correlation coefficient concepts 104–20, 261–5, 268–9 critical value 105–6, 113–20 equations 104–5 examples 105–8, 115–20 costs capital 219–25, 229–30 dynamic programming 180–82 ghost costs 172–7 holding costs 182–5, 197–201, 204–8 linear programming 167–70, 171–7 sampling 47 stock control 182–5, 195–201 transport problems 171–7 trend analysis 236–47 types 167–8, 182 counting techniques, probability distribution (density) functions 54 covariance see also correlation coefficient concepts 104–20, 263–5 credit cards 159–66, 267–9 credit derivatives 97 see also derivatives Index credit risk, modelling 75, 149, 261–5 critical value, correlation coefficient 105–6, 113–20 cumulative frequency polygons concepts 13–20, 39–40, 203 examples 14–20 uses 13–14 current costs, linear programming 167–70 cyclical variations, trends 238–47 data analysis methods 47–52, 129–47, 271–4 collection techniques 17, 47–52, 129–47 continuous/discrete types 7–12, 13–14, 44–5, 53–5, 65–6, 72, 251 design/approach to analysis 129–47 errors 129–47 graphical presentational approaches 1–20, 149–57 identification 2–5, 261–5 Latin squares 131–2, 143–7 loss data 267–9, 271–4 neural networks 275–7 qualities 17, 47 randomised block design 129–35 reliability and accuracy 17, 47 sampling 17, 47–52 time series 235–47 trends 5, 10, 235–47 two-way classification analysis 135–47 data points, scatter plots 2–5 databases, loss databases 272–4 debentures 149–57 decisions alternatives 191–4 Bayes’theorem 27–30, 31 communications 189–90 concepts 21–35, 189–94, 215–25, 228–34, 249–60 courses of action 191–2 definition 21 delegation 189–90 empowerment 189–90 guesswork 191 lethargy pitfalls 189 minimax regret rule 192–4 modelling problems 189–91 Monty Hall problem 34–5, 212–13 pitfalls 189–94 probability theory 21–35, 53–66, 189–94, 215–18 problem definition 129, 190–2 project analysis guidelines 190–2, 219–25, 228–34 replacement of assets 215–18, 249–60 staff 189–90 287 steps 21 stock control 195–201, 203–8 theory 189–94 degrees of freedom 70–1, 75–89, 91–5, 110–20, 136–7 ANOVA (analysis of variance) 110–20, 121–7, 136–7 concepts 70–1, 75–89, 91–5, 110–20, 136–7 delegation, decisions 189–90 density functions see also probability distribution (density) functions concepts 65–6, 67, 83–4 dependent variables, concepts 2–5, 103–20, 235 derivatives 58, 97–8, 272 see also credit . . . ; options design/approach to analysis, data 129–47 dice-rolling examples, probability theory 21–3, 53–5 differentiation 251 discount factors adjusted discount rates 228–9 net present value (NPV) 220–1, 228–9, 231–2 discrete data bar charts 7–12, 13 concepts 7–12, 13, 44–5, 53–5, 72 discrete uniform distribution, concepts 53–5 displays see also presentational approaches data 1–5 Disraeli, Benjamin 1 division notation 280, 282 dynamic programming complex examples 184–7 concepts 179–87 costs 180–82 examples 180–87 principle of optimality 179–87 returns 179–80 schematic 179–80 ‘travelling salesman’ problem 185–7 e-mail surveys 50–1 economic order quantity see also stock control concepts 195–201 examples 196–9 empowerment, staff 189–90 error sum of the squares (SSE), concepts 122–5, 133–47 errors, data analysis 129–47 estimates mean 76–81 probability theory 22, 25–6, 31–5, 75–81 Euler, L. 131 288 Index events independent events 22–4, 35, 58, 60, 92–5 mutually exclusive events 22–4, 58 probability theory 21–35, 58–66, 92–5 scenario analysis 40, 193–4, 271–4 tree diagrams 30–5 Excel 68, 206–7 exclusive events see mutually exclusive events expected errors, sensitivity analysis 268–9 expected value, net present value (NPV) 231–2 expert systems 275 exponent notation 282–4 exponential distribution, concepts 65–6, 209–10, 252–5 external fraud 272–4 extrapolation 119 extreme value distributions, VaR 262–4 F distribution ANOVA (analysis of variance) 110–20, 127, 134–7 concepts 85–9, 110–20, 127, 134–7 examples 85–9, 110–20, 127, 137 tables 85–8 f notation 8–9, 13–20, 26, 38–9, 44–5, 65–6, 85 factorial notation 53–5, 283–4 failure probabilities see also reliability replacement of assets 215–18, 249–60 feasibility polygons 152–7, 163–4 finance selection, linear programming 164–6 fire extinguishers, ANOVA (analysis of variance) 123–7 focus groups 51 forward recursion 179–87 four by four tables 94–5 fraud 272–4, 276 Fréchet distribution 262 frequency concepts 8–9, 13–20, 37–45 cumulative frequency polygons 13–20, 39–40, 203 graphical presentational approaches 8–9, 13–20 frequentist approach, probability theory 22, 25–6 future cash flows 219–25, 227–34, 240–1 fuzzy logic 276 Garbage In, Garbage Out (GIGO) 261–2 general rules, linear programming 167–70 genetic algorithms 276 ghost costs, transport problems 172–7 goodness of fit test, chi-squared test 91–5 gradient (a notation), linear regression 103–4, 107–20 graphical method, linear programming 149–57, 163–4 graphical presentational approaches concepts 1–20, 149–57, 235–47 rules 8–9 greater-than notation 280–4 Greek alphabet 283 guesswork, modelling 191 histograms 2, 7, 13–20, 41, 73 class intervals 13–20, 44–5 comparative histograms 14–19 concepts 7, 13–20, 41, 73 continuous data 7, 13–14 examples 13–20, 73 skewness 41 uses 7, 13–20 holding costs 182–5, 197–201, 204–8 home insurance 10–12 Hopfield 275 horizontal axis bar charts 8–9 histograms 14–20 linear regression 103–4, 107–20 scatter plots 2–5, 103 hypothesis testing concepts 77–81, 85–95, 110–27 examples 78–80, 85 type I and type II errors 80–1 i notation 8–9, 13–20, 28–30, 37–8, 103–20 identification data 2–5, 261–5 trends 241–7 identity rule 282 impact assessments 21, 271–4 independent events, probability theory 22–4, 35, 58, 60, 92–5 independent variables, concepts 2–5, 70, 103–20, 235 infinity, normal distribution 67–72 information, quality needs 190–4 initial solution, linear programming 167–70 insurance industry 10–12, 29–30 integers 280–4 integration 65–6, 251 intercept (b notation), linear regression 103–4, 107–20 interest rates base rates 240 daily movements 40, 261 project evaluation 219–25, 228–9 internal rate of return (IRR) concepts 220–2, 223–5 examples 220–2 interpolation, IRR 221–2 interviews, uses 48, 51–2 inventory control see stock control Index investment strategies 149–57, 164–6, 262–5 IRR see internal rate of return iterative processes, linear programming 170 j notation 28–30, 37, 104–20, 121–2 JP Morgan 263 k notation 20, 121–7 ‘know your customer’ 272 Kohonen self-organising maps 275 Latin squares concepts 131–2, 143–7 examples 143–7 lead times, stock control 195–201 learning strategies, neural networks 275–6 less-than notation 281–4 lethargy pitfalls, decisions 189 likelihood considerations, scenario analysis 272–3 linear programming additional variables 167–70 concepts 149–70 concerns 170 constraining equations 159–70 costs 167–70, 171–7 critique 170 examples 149–57, 159–70 finance selection 164–6 general rules 167–70 graphical method 149–57, 163–4 initial solution 167–70 iterative processes 170 manual preparation 170 most profitable loans 159–66 optimal advertising allocation 154–7 optimal investment strategies 149–57, 164–6 returns 149–57, 164–6 simplex method 159–70, 171–2 standardisation 167–70 time constraints 167–70 transport problems 171–7 linear regression analysis 110–20 ANOVA (analysis of variance) 110–20 concepts 3, 103–20 equation 103–4 examples 107–20 gradient (a notation) 103–4, 107–20 intercept (b notation) 103–4, 107–20 interpretation 110–20 notation 103–4 residual sum of the squares 109–20 slope significance test 112–20 uncertainties 108–20 literature searches, surveys 48 289 loans finance selection 164–6 linear programming 159–66 risk assessments 159–60 log-normal distribution, concepts 257–8 logarithms (logs), types 20, 61 losses, banks 267–9, 271–4 lotteries 22 lower/upper quartiles, concepts 39–41 m notation 55–8 mail surveys 48, 50–1 management information, graphical presentational approaches 1–20 Mann–Whitney test see U test manual preparation, linear programming 170 margin of error, project evaluation 229–30 market prices, VaR 264–5 marketing brochures 184–7 mathematics 1, 7–8, 196–9, 219–20, 222–5, 234, 240–1, 251, 279–84 matrix plots, concepts 2, 4–5 matrix-based approach, transport problems 171–7 maximum and minimum, concepts 37–9, 40, 254–5 mean comparison of two sample means 79–81 comparisons 75–81 concepts 37–45, 59–60, 65–6, 67–74, 75–81, 97–8, 100–2, 104–27, 134–5 confidence intervals 71, 75–81, 105, 109, 116–20, 190, 262–5 continuous data 44–5, 65–6 estimates 76–81 hypothesis testing 77–81 linear regression 104–20 normal distribution 67–74, 75–81, 97–8 sampling 75–81 mean square causes (MSC), concepts 122–7, 134–47 mean square errors (MSE), ANOVA (analysis of variance) 110–20, 121–7, 134–7 median, concepts 37, 38–42, 83, 98–9 mid-points class intervals 44–5, 241–7 moving averages 241–7 minimax regret rule, concepts 192–4 minimum and maximum, concepts 37–9, 40 mode, concepts 37, 39, 41 modelling banks 75–81, 85, 97, 267–9, 271–4 concepts 75–81, 83, 91–2, 189–90, 195–201, 215–18, 261–5 decision-making pitfalls 189–91 economic order quantity 195–201 290 Index modelling (cont.) guesswork 191 neural networks 275–7 operational risk 75, 262–5, 267–9, 271–4 output reviews 191–2 replacement of assets 215–18, 249–60 VaR 261–5 moments, density functions 65–6, 83–4 money laundering 272–4 Monte Carlo simulation bank cashier problem 209–12 concepts 203–14, 234 examples 203–8 Monty Hall problem 212–13 queuing problems 208–10 random numbers 207–8 stock control 203–8 uses 203, 234 Monty Hall problem 34–5, 212–13 moving averages concepts 241–7 even numbers/observations 244–5 moving totals 245–7 MQMQM plot, concepts 40 MSC see mean square causes MSE see mean square errors multi-way tables, concepts 94–5 multiplication notation 279–80, 282 multiplication rule, probability theory 26–7 multistage sampling 50 mutually exclusive events, probability theory 22–4, 58 n notation 7, 20, 28–30, 37–45, 54–8, 103–20, 121–7, 132–47, 232–4 n!

Mathematics of the Financial Markets: Financial Instruments and Derivatives Modelling, Valuation and Risk Issues by Alain Ruttiens

algorithmic trading, asset allocation, asset-backed security, backtesting, banking crisis, Black Swan, Black-Scholes formula, Bob Litterman, book value, Brownian motion, capital asset pricing model, collateralized debt obligation, correlation coefficient, Credit Default Swap, credit default swaps / collateralized debt obligations, currency risk, delta neutral, discounted cash flows, discrete time, diversification, financial engineering, fixed income, implied volatility, interest rate derivative, interest rate swap, low interest rates, managed futures, margin call, market microstructure, martingale, p-value, passive investing, proprietary trading, quantitative trading / quantitative finance, random walk, risk free rate, risk/return, Satyajit Das, seminal paper, Sharpe ratio, short selling, statistical model, stochastic process, stochastic volatility, time value of money, transaction costs, value at risk, volatility smile, Wiener process, yield curve, zero-coupon bond

The linear correlation of x and y is 0, although they are clearly dependent: if x = 0, y can only value 1 or −1, and if x ≠ 0, y = 0. Coming back to the general case of φx(x) and φy(y) being not Gaussian, this inference cannot be made. Typically, the classic “rank correlation” coefficient of Spearman shows the way to get round the problem: this rank correlation consists in a linear correlation coefficient of the variates, 5 now transformed in a non-linear way, by a probability transformation, that is, their respective cumulative marginal distributions: with The Spearman correlation is a correlation measure that can be computed from these relationships and from the general formula for ρx, y above, but, as a step further, we can link above Φx(x), Φy(y) and Φ(x, y) relationships in a more general way that defines C – named a copula of two variables x and y – as a cumulative probability function of the marginal cumulative probabilities Φx(x), Φy(y) of x and y.6 A copula is thus a general measure of co-dependence between two variates, which is independent of their individual marginal distribution – see Figure 13.5.

…

It is indeed based on several restrictive hypotheses: Hypotheses related to financial assets: Asset returns r are modeled by a random variable, distributed as a Gaussian probabilities distribution, fully determined by its first two moments, namely its expected value E and its variance V, although instead of V, the theory makes use of the corresponding standard deviation measure STD (STD = ). Returns of different financial assets i and j are correlated by the linear correlation coefficient ρij. Markets are efficient1 – practically speaking, we observe that the more liquid a market, the more efficient it is. The theory is built on mid prices (average of the market quoted bid and offer (or ask) prices): the market bid–offer spread is thus not considered here. Various costs such as brokerage fees, taxes, and so on are not taken into account (they are too much affected by local circumstances, market features, and the investor's situation).

…

For example, in 2006, based on successive daily close prices, the return and risk of L'Oreal were 20% and 19% respectively. Figure 4.3 Example of a stock showed in a (r, σ) graph 4.3.3 The Markowitz model Markowitz's goal was to optimize the budget allocation to a portfolio P of n stocks Si(ri, σi), weighted by wi, with 0 ≤ wi ≤ 1 and ∑wi = 1, so that for P: (4.1) that is, where the ρij correlation coefficients are computed by In a (r, σ) chart, it is possible, for a given past period of data to locate by a point any Si(ri, σi), but also any possible weighted combination of up to n stocks, defining points that represent portfolios, among which the optimal ones have to be identified. Performing this graph representation shows that there is a (non-linear) “frontier” of possible portfolios presenting the highest return, for different risks.

Market Sense and Nonsense by Jack D. Schwager

3Com Palm IPO, asset allocation, Bear Stearns, Bernie Madoff, Black Monday: stock market crash in 1987, Brownian motion, buy and hold, collateralized debt obligation, commodity trading advisor, computerized trading, conceptual framework, correlation coefficient, Credit Default Swap, credit default swaps / collateralized debt obligations, diversification, diversified portfolio, fixed income, global macro, high net worth, implied volatility, index arbitrage, index fund, Jim Simons, junk bonds, London Interbank Offered Rate, Long Term Capital Management, low interest rates, managed futures, margin call, market bubble, market fundamentalism, Market Wizards by Jack D. Schwager, merger arbitrage, negative equity, pattern recognition, performance metric, pets.com, Ponzi scheme, proprietary trading, quantitative trading / quantitative finance, random walk, risk free rate, risk tolerance, risk-adjusted returns, risk/return, Robert Shiller, selection bias, Sharpe ratio, short selling, statistical arbitrage, statistical model, subprime mortgage crisis, survivorship bias, tail risk, transaction costs, two-sided market, value at risk, yield curve

Correlation Defined The correlation coefficient, typically denoted by the letter r, measures the degree of linear relationship between two variables. The correlation coefficient ranges from −1.0 to +1.0. The closer the correlation coefficient is to +1.0, the closer the relationship is between the two variables. A perfect correlation of 1.0 would occur only in artificial situations. For example, the heights of a group of people measured in inches and the heights of the same group of people measured in feet would be perfectly correlated. The closer the correlation coefficient is to −1.0, the stronger the inverse correlation is between the two variables.

…

The closer the correlation coefficient is to −1.0, the stronger the inverse correlation is between the two variables. For example, average winter temperatures in the U.S. Northeast and heating oil usage in that region would be inversely related variables (variables with a negative correlation coefficient). If two variables have a correlation coefficient near zero, it indicates that there is no significant (linear) relationship between the variables. It is important to understand that the correlation coefficient only indicates the degree of correlation between two variables and does not imply anything about cause and effect. Correlation Shows Linear Relationships Correlation reflects only linear relationships.

…

Because correlation reflects only linear relationships, and there is no linear relationship between the two variables. Figure 9.1 Strategy Returns versus S&P Returns The Coefficient of Determination (r2) The square of the correlation coefficient, which is called the coefficient of determination and is denoted as r2, has a very specific interpretation: It represents the percentage of the variability of one variable explained by the other. For example, if the correlation coefficient (r) of a fund versus the S&P is 0.7, it implies that nearly half the variability of the fund’s returns is explained by the S&P returns (r2 = 0.49). For a mutual fund that is a so-called closet benchmarker—a fund that maintains a portfolio very similar to the S&P index with only minor differences—the r2 would tend to be very high (e.g., above 0.9).

How to Read a Paper: The Basics of Evidence-Based Medicine by Trisha Greenhalgh

call centre, complexity theory, conceptual framework, confounding variable, correlation coefficient, correlation does not imply causation, deskilling, knowledge worker, longitudinal study, meta-analysis, microbiome, New Journalism, p-value, personalized medicine, placebo effect, publication bias, randomized controlled trial, selection bias, systematic bias, systems thinking, the scientific method

Ignore all withdrawals (‘drop outs’) and non-responders, so the analysis only concerns subjects who fully complied with treatment (see section ‘Were preliminary statistical questions addressed?’). 5. Always assume that you can plot one set of data against another and calculate an ‘r-value’ (Pearson correlation coefficient) (see section ‘Has correlation been distinguished from regression, and has the correlation coefficient (‘r-value’) been calculated and interpreted correctly?’), and that a ‘significant’ r-value proves causation (see section ‘Have assumptions been made about the nature and direction of causality?’). 6. If outliers (points that lie a long way from the others on your graph) are messing up your calculations, just rub them out.

…

Every r-value should be accompanied by a p-value, which expresses how likely an association of this strength would be to have arisen by chance (see section ‘Have ‘p-values’ been calculated and interpreted appropriately?’), or a confidence interval, which expresses the range within which the ‘true’ R-value is likely to lie (see section ‘Have confidence intervals been calculated, and do the authors' conclusions reflect them?’). (Note that lower case ‘r’ represents the correlation coefficient of the sample, whereas upper case ‘R’ represents the correlation coefficient of the entire population.) Remember, too, that even if the r-value is an appropriate value to calculate from a set of data, it does not tell you whether the relationship, however strong, is causal (see subsequent text). The term regression refers to a mathematical equation that allows one variable (the target variable) to be predicted from another (the independent variable).

…

As you might imagine, statistical significance is more difficult to demonstrate with rank order tests (indeed, some statisticians are cynical about the value of the latter), and this tempts researchers to use statistics such as the r-value (see section ‘Has correlation been distinguished from regression, and has the correlation coefficient (“r-value”) been calculated and interpreted correctly?’) inappropriately. Not only is the r-value (parametric) easier to calculate than an equivalent rank order statistic such as Spearman's ρ (pronounced ‘rho’) but it is also much more likely to give (apparently) significant results. Unfortunately, it will also give an entirely spurious and misleading estimate of the significance of the result, unless the data are appropriate to the test being used.

Topics in Market Microstructure by Ilija I. Zovko

Brownian motion, computerized trading, continuous double auction, correlation coefficient, financial intermediation, Gini coefficient, information asymmetry, market design, market friction, market microstructure, Murray Gell-Mann, p-value, power law, quantitative trading / quantitative finance, random walk, stochastic process, stochastic volatility, transaction costs

Later in the text we use a bootstrap approach to test the significance. Now, however, we test the significance of the correlation coefficients using a standard algorithm as in ref. (Best and Roberts, 1975). The algorithm calculates the approximate tail probabilities for Spearman’s correlation coefficient ρ. Its precision unfortunately degrades when there are ties in the data, which is the case here. With this caveat in mind, as a preliminary test, we find that, for example, for on-book trading in Vodafone for the month of May 2000, 10.3% of all correlation coefficients are significant at the 5% level. Averaging over all stocks and months, the average percentage of significant coefficients for on-book trading is 10.5% ± 0.4%, while for off-book trading it is 20.7% ± 1.7%.

…

Averaging over all stocks and months, the average percentage of significant coefficients for on-book trading is 10.5% ± 0.4%, while for off-book trading it is 20.7% ± 1.7%. Both of these averages are substantially higher than the 5% we would expect randomly with a 5% acceptance level of the test. 4.2 Significance and structure in the correlation matrices The preliminary result of the previous section that some correlation coefficients are non-random is further corroborated by testing for non-random structure in the correlation matrices. The hypothesis that there is structure in the correlation matrices contains the weaker hypothesis that some coefficients are statistically significant. The test for structure in the matrices would involve multiple joint tests for the significance of the coefficients.

…

We apply clustering techniques using a metric chosen so that two strongly correlated institutions are ’close’ and anti-correlated institutions are ’far away’. A functional form fulfilling this requirement and satisfying the properties of being a metric is (Bonanno et al., 2000) # (4.2) di,j = 2 · (1 − ρi,j ), where ρi,j is the correlation coefficient between strategies i and j. We have tried several reasonable modifications to this form but without obvious differences in the results. Ultimately the choice of this metric is influenced by the fact that it has been successfully used in other studies (Bonanno et al., 2000). We use complete linkage clustering, in which the distance between two clusters is calculated as the maximum distance between its members.

Hands-On Machine Learning With Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems by Aurelien Geron

AlphaGo, Amazon Mechanical Turk, Bayesian statistics, centre right, combinatorial explosion, constrained optimization, correlation coefficient, crowdsourcing, data science, deep learning, DeepMind, duck typing, en.wikipedia.org, Geoffrey Hinton, iterative process, Netflix Prize, NP-complete, optical character recognition, P = NP, p-value, pattern recognition, performance metric, recommendation engine, self-driving car, SpamAssassin, speech recognition, statistical model

Finally, coefficients close to zero mean that there is no linear correlation. Figure 2-14 shows various plots along with the correlation coefficient between their horizontal and vertical axes. Figure 2-14. Standard correlation coefficient of various datasets (source: Wikipedia; public domain image) Warning The correlation coefficient only measures linear correlations (“if x goes up, then y generally goes up/down”). It may completely miss out on nonlinear relationships (e.g., “if x is close to zero then y generally goes up”). Note how all the plots of the bottom row have a correlation coefficient equal to zero despite the fact that their axes are clearly not independent: these are examples of nonlinear relationships.

…

Looking for Correlations Since the dataset is not too large, you can easily compute the standard correlation coefficient (also called Pearson’s r) between every pair of attributes using the corr() method: corr_matrix = housing.corr() Now let’s look at how much each attribute correlates with the median house value: >>> corr_matrix["median_house_value"].sort_values(ascending=False) median_house_value 1.000000 median_income 0.687170 total_rooms 0.135231 housing_median_age 0.114220 households 0.064702 total_bedrooms 0.047865 population -0.026699 longitude -0.047279 latitude -0.142826 Name: median_house_value, dtype: float64 The correlation coefficient ranges from –1 to 1.

…

Note how all the plots of the bottom row have a correlation coefficient equal to zero despite the fact that their axes are clearly not independent: these are examples of nonlinear relationships. Also, the second row shows examples where the correlation coefficient is equal to 1 or –1; notice that this has nothing to do with the slope. For example, your height in inches has a correlation coefficient of 1 with your height in feet or in nanometers. Another way to check for correlation between attributes is to use Pandas’ scatter_matrix function, which plots every numerical attribute against every other numerical attribute. Since there are now 11 numerical attributes, you would get 112 = 121 plots, which would not fit on a page, so let’s just focus on a few promising attributes that seem most correlated with the median housing value (Figure 2-15): from pandas.plotting import scatter_matrix attributes = ["median_house_value", "median_income", "total_rooms", "housing_median_age"] scatter_matrix(housing[attributes], figsize=(12, 8)) Figure 2-15.

Calling Bullshit: The Art of Scepticism in a Data-Driven World by Jevin D. West, Carl T. Bergstrom

airport security, algorithmic bias, AlphaGo, Amazon Mechanical Turk, Andrew Wiles, Anthropocene, autism spectrum disorder, bitcoin, Charles Babbage, cloud computing, computer vision, content marketing, correlation coefficient, correlation does not imply causation, crowdsourcing, cryptocurrency, data science, deep learning, deepfake, delayed gratification, disinformation, Dmitri Mendeleev, Donald Trump, Elon Musk, epigenetics, Estimating the Reproducibility of Psychological Science, experimental economics, fake news, Ford Model T, Goodhart's law, Helicobacter pylori, Higgs boson, invention of the printing press, John Markoff, Large Hadron Collider, longitudinal study, Lyft, machine translation, meta-analysis, new economy, nowcasting, opioid epidemic / opioid crisis, p-value, Pluto: dwarf planet, publication bias, RAND corporation, randomized controlled trial, replication crisis, ride hailing / ride sharing, Ronald Reagan, selection bias, self-driving car, Silicon Valley, Silicon Valley startup, social graph, Socratic dialogue, Stanford marshmallow experiment, statistical model, stem cell, superintelligent machines, systematic bias, tech bro, TED Talk, the long tail, the scientific method, theory of mind, Tim Cook: Apple, twin studies, Uber and Lyft, Uber for X, uber lyft, When a measure becomes a target

Quarterbacks and kickers, for example, tend to be lighter than you would expect given their height, whereas running backs and linemen tend to be heavier. The strength of a linear correlation is measured as a correlation coefficient, which is a number between 1 and −1. A correlation of 1 means that the two measurements form a perfect line on a scatter plot, such that when one increases, the other increases as well. For example, distance in meters and distance in kilometers have a correlation coefficient of 1, because the former is just one thousand times the latter. A correlation of −1 means that two measurements form another kind of perfect line on a scatter plot, such that when one increases, the other decreases.

…

As one increases, the other decreases by the same amount. These two quantities have a correlation of −1. A correlation coefficient of 0 means that a best-fit line through the points doesn’t tell you anything.*2 In other words, one measurement tells you nothing about the other.*3 For example, psychologists sometimes use the Eysenck Personality Inventory questionnaire as a way to summarize aspects of personality known as impulsivity, sociability, and neuroticism. Across individuals, impulsivity and neuroticism are essentially uncorrelated, with a correlation coefficient of −0.07. In other words, knowing something about a person’s impulsivity tells you very little (if anything) about his neuroticism and vice versa.

…

For example, consider pairs of numbers {x, sin(x)}. If we know x, we can predict exactly what sin(x) will be, but the correlation coefficient—a measure of linear correlation—between these numbers is zero across a full-cycle sine wave. There is no linear correlation between x and sin(x) because a best-fit line through {x, sin(x)} pairs has a slope of 0 and tells us nothing about the likely value of sin(x) for any given value of x. *2 In the rare case where the data points form either a vertical or horizontal line, the correlation coefficient is undefined. In these cases, knowing one measurement also tells you nothing about the other measurement

The Rise and Fall of the Third Chimpanzee by Jared Diamond

agricultural Revolution, assortative mating, Atahualpa, Boeing 747, Columbian Exchange, correlation coefficient, double helix, Drosophila, Easter island, European colonialism, Great Leap Forward, invention of gunpowder, invention of the wheel, invention of writing, language acquisition, longitudinal study, out of africa, phenotype, planned obsolescence, Scientific racism, Search for Extraterrestrial Intelligence, the long tail, the scientific method, trade route

A simple numerical way of describing the result is by means of a statistical index called the correlation coefficient. If you line up 100 husbands in order of their ranking for some characteristic (say, their height), and if you also line up their 100 wives with respect to the same characteristic, the correlation coefficient describes whether a man tends to be at the same position in the husbands' line-up as his wife is in the line-up of wives. A correlation coefficient of plus one would mean perfect correspondence: the tallest man marries the tallest woman, the thirty-seventh tallest man marries the thirty-seventh tallest woman, and so on. A correlation coefficient of minus one would mean perfect matching by opposites: the tallest man marries the shortest woman, the thirty-seventh tallest man marries the thirty-seventh shortest woman, and so on.

…

A correlation coefficient of minus one would mean perfect matching by opposites: the tallest man marries the shortest woman, the thirty-seventh tallest man marries the thirty-seventh shortest woman, and so on. Finally, a correlation coefficient of zero would mean that husbands and wives assort completely randomly by height: a tall man is as likely to marry a short woman as a tall woman. These examples are for height, but correlation coefficients can also be calculated for anything else, such as income and IQ. If you measure enough things about enough couples, here is what you will find. Not surprisingly, the highest correlation coefficients—typically around +0.9—are for religion, ethnic background, race, socioeconomic status, age, and political views.

…

Not surprisingly, the highest correlation coefficients—typically around +0.9—are for religion, ethnic background, race, socioeconomic status, age, and political views. That is, most husbands and wives prove to be of the same religion, ethnic background, and so on. Perhaps you also will not be surprised that the next highest correlation coefficients, usually around +0.4, are for measures of personality and intelligence, such as extroversion, neatness, and IQ. Slobs tend to marry slobs, though the chances of a slob marrying a compulsively neat person are not as low as the chances of a political reactionary marrying a left-winger. What about matching of husbands and wives for physical characteristics? The answer is not one that would leap out at you immediately if you just looked at a few married couples.

Stocks for the Long Run 5/E: the Definitive Guide to Financial Market Returns & Long-Term Investment Strategies by Jeremy Siegel

Alan Greenspan, AOL-Time Warner, Asian financial crisis, asset allocation, backtesting, banking crisis, Bear Stearns, behavioural economics, Black Monday: stock market crash in 1987, Black-Scholes formula, book value, break the buck, Bretton Woods, business cycle, buy and hold, buy low sell high, California gold rush, capital asset pricing model, carried interest, central bank independence, cognitive dissonance, compound rate of return, computer age, computerized trading, corporate governance, correlation coefficient, Credit Default Swap, currency risk, Daniel Kahneman / Amos Tversky, Deng Xiaoping, discounted cash flows, diversification, diversified portfolio, dividend-yielding stocks, dogs of the Dow, equity premium, equity risk premium, Eugene Fama: efficient market hypothesis, eurozone crisis, Everybody Ought to Be Rich, Financial Instability Hypothesis, fixed income, Flash crash, forward guidance, fundamental attribution error, Glass-Steagall Act, housing crisis, Hyman Minsky, implied volatility, income inequality, index arbitrage, index fund, indoor plumbing, inflation targeting, invention of the printing press, Isaac Newton, it's over 9,000, John Bogle, joint-stock company, London Interbank Offered Rate, Long Term Capital Management, loss aversion, machine readable, market bubble, mental accounting, Minsky moment, Money creation, money market fund, mortgage debt, Myron Scholes, new economy, Northern Rock, oil shock, passive investing, Paul Samuelson, Peter Thiel, Ponzi scheme, prediction markets, price anchoring, price stability, proprietary trading, purchasing power parity, quantitative easing, random walk, Richard Thaler, risk free rate, risk tolerance, risk/return, Robert Gordon, Robert Shiller, Ronald Reagan, shareholder value, short selling, Silicon Valley, South Sea Bubble, sovereign wealth fund, stocks for the long run, survivorship bias, technology bubble, The Great Moderation, the payments system, The Wisdom of Crowds, transaction costs, tulip mania, Tyler Cowen, Tyler Cowen: Great Stagnation, uptick rule, Vanguard fund

This will be particularly true if bond and stock returns are negatively correlated, which would happen if bond and stock prices move in the opposite direction.4 The diversifying strength of an asset is measured by the correlation coefficient. The correlation coefficient ranges between -1 and +1 and measures the co-movement between an asset’s return and the return of the rest of the portfolio. The lower the correlation coefficient, the better the asset serves as a portfolio diversifier. Assets with near-zero or especially negative correlations are particularly good diversifiers. As the correlation coefficient between the asset and portfolio returns increases, the diversifying quality of the asset declines. In Chapter 3 we examined the changing correlation coefficient between the return on 10-year Treasury bonds and stocks, represented by the S&P 500 Index.

…

The standard deviation of the Magellan Fund over Lynch’s period was 21.38 percent, compared with 13.88 percent for the Wilshire 5000, while its correlation coefficient with the Wilshire was .86. 6. “The Superinvestors of Graham-and-Doddsville,” Hermes, the Columbia Business School Magazine, 1984 (reprinted 2004). 7. Money managers are assumed to expose their clients to the same risk as would the market, and the money managers have a correlation coefficient of .88 with market returns, which has been typical of equity mutual funds since 1971. 8. Darryll Hendricks, Jayendu Patel, and Richard Zeckhauser, “Hot Hands in Mutual Funds: Short-Run Persistence of Relative Performance, 1974-1988,” Journal of Finance, vol. 48, no. 1 (March 1993), pp. 93-130. 9.

…

In Chapter 3 we examined the changing correlation coefficient between the return on 10-year Treasury bonds and stocks, represented by the S&P 500 Index. Figure 6-3 displays the correlation coefficient between annual stock and bond returns for three subperiods between 1926 and 2012. From 1926 through 1965 the correlation was only slightly positive, indicating that bonds were fairly good diversifiers for stocks. Bonds were good diversifiers in this period because it contained the Great Depression, which was characterized by falling economic activity and consumer prices, a situation that was bad for stocks but good for U.S. government bonds. FIGURE 6-3 Correlation of Real Bond and Stock Returns Over Various Historical Periods However, under a paper money standard, bad economic times are more likely to be associated with inflation, not deflation.

Pure, White and Deadly: How Sugar Is Killing Us and What We Can Do to Stop It by John Yudkin

correlation coefficient, correlation does not imply causation, discovery of penicillin, disinformation

If this were so, you would say that the correlation coefficient was 1·0. Supposing on the other hand – and this is even more unlikely – that there was no relationship whatever between height and weight, so that it would be just as likely for a man weighing 150 pounds to be five feet tall or six feet tall. In this case the correlation coefficient would be 0. In fact, there is a relationship, but not a precise one; tall people tend to be heavier. If you work it out exactly, for adult men the correlation coefficient between height and weight comes to about 0·6. The correlation coefficients I have found so far for cancer and sugar consumption in different countries are as follows: Cancer of the large intestine in men: 0·60 Cancer of the large intestine in women: 0·50 Cancer of the breast: 0·63 However, such international statistics, as I have stressed repeatedly, can do no more than give a clue as to the possible role of sugar or fat in producing disease.

…

On the other hand, the lowest mortality, in ascending order, is in Japan, Yugoslavia, Portugal, Spain and Italy, with the lowest sugar consumption in Japan, Portugal, Spain, Yugoslavia and Italy. My own observations of the association between sugar and cancer of the intestine and of the breast were made several years earlier than the study I have just quoted. I calculated what are called the ‘correlation coefficients’ between these cancers and sugar consumption in all the countries for which statistics were then available. Let me explain first what correlation coefficients are, and let me take as an example the relation between people’s height and weight. On the whole, the taller people are, the more they weigh. But it is all very well to say that there is ‘on the whole’ this association between height and weight; it would be better if we could say how close this association is.

Mastering Pandas by Femi Anthony

Amazon Web Services, Bayesian statistics, correlation coefficient, correlation does not imply causation, data science, Debian, en.wikipedia.org, Internet of things, Large Hadron Collider, natural language processing, p-value, power law, random walk, side project, sparse data, statistical model, Thomas Bayes

The correlation measure, known as correlation coefficient, is a number that captures the size and direction of the relationship between the two variables. It can vary from -1 to +1 in direction and 0 to 1 in magnitude. The direction of the relationship is expressed via the sign, with a + sign expressing positive correlation and a - sign negative correlation. The higher the magnitude, the greater the correlation with a one being termed as the perfect correlation. The most popular and widely used correlation coefficient is the Pearson product-moment correlation coefficient, known as r. It measures the linear correlation or dependence between two x and y variables and takes values between -1 and +1.

…

We can also only make predictions for values within the bounds of the data. For example, we cannot predict what the chirpFrequency is at 32 degrees Fahrenheit as it is outside the bounds of the data; moreover, at 32 degrees Fahrenheit, the crickets would have frozen to death. The value of R, the correlation coefficient, is given as follows: In [38]: R=np.sqrt(result.rsquared) R Out[38]: 0.83514378678237422 Thus, our correlation coefficient is R = 0.835. This would indicate that about 84 percent of the chirp frequency can be explained by the changes in temperature. Reference of this information: The Song of Insects http://www.hup.harvard.edu/catalog.php?isbn=9780674420663 The data is sourced from http://bit.ly/1MrlJqR.

…

It measures the linear correlation or dependence between two x and y variables and takes values between -1 and +1. The sample correlation coefficient r is defined as follows: This can also be written as follows: Here, we have omitted the summation limits. Linear regression As mentioned earlier, regression focuses on using the relationship between two variables for prediction. In order to make predictions using linear regression, the best-fitting straight line must be computed. If all the points (values for the variables) lie on a straight line, then the relationship is deemed perfect. This rarely happens in practice and the points do not all fit neatly on a straight line.

The Great Mental Models: General Thinking Concepts by Shane Parrish

Albert Einstein, anti-fragile, Atul Gawande, Barry Marshall: ulcers, bitcoin, Black Swan, colonial rule, correlation coefficient, correlation does not imply causation, cuban missile crisis, Daniel Kahneman / Amos Tversky, dark matter, delayed gratification, feminist movement, Garrett Hardin, if you see hoof prints, think horses—not zebras, index fund, Isaac Newton, Jane Jacobs, John Bogle, Linda problem, mandelbrot fractal, Pepsi Challenge, Philippa Foot, Pierre-Simon Laplace, Ponzi scheme, Richard Feynman, statistical model, stem cell, The Death and Life of Great American Cities, the map is not the territory, the scientific method, Thomas Bayes, Torches of Freedom, Tragedy of the Commons, trolley problem

The problem is, without a good understanding of what is meant by these terms, these decisions fail to capitalize on real dynamics in the world and instead are successful only by luck. No Correlation The correlation coefficient between two measures, which varies between -1 and 1, is a measure of the relative weight of the factors they share. For example, two phenomena with few factors shared, such as bottled water consumption versus suicide rate, should have a correlation coefficient of close to 0. That is to say, if we looked at all countries in the world and plotted suicide rates of a specific year against per capita consumption of bottled water, the plot would show no pattern at all.

…

The only factor governing temperature—velocity of molecules—is shared by all scales. Thus each degree in Celsius will have exactly one corresponding value in Fahrenheit. Therefore temperature in Celsius and Fahrenheit will have a correlation coefficient of 1 and the plot will be a straight line. Weak to Moderate Correlation There are few phenomena in human sciences that have a correlation coefficient of 1. There are, however, plenty where the association is weak to moderate and there is some explanatory power between the two phenomena. Consider the correlation between height and weight, which would land somewhere between 0 and 1.

Risk Management in Trading by Davis Edwards

Abraham Maslow, asset allocation, asset-backed security, backtesting, Bear Stearns, Black-Scholes formula, Brownian motion, business cycle, computerized trading, correlation coefficient, Credit Default Swap, discrete time, diversified portfolio, financial engineering, fixed income, Glass-Steagall Act, global macro, implied volatility, intangible asset, interest rate swap, iterative process, John Meriwether, junk bonds, London Whale, Long Term Capital Management, low interest rates, margin call, Myron Scholes, Nick Leeson, p-value, paper trading, pattern recognition, proprietary trading, random walk, risk free rate, risk tolerance, risk/return, selection bias, shareholder value, Sharpe ratio, short selling, statistical arbitrage, statistical model, stochastic process, systematic trading, time value of money, transaction costs, value at risk, Wiener process, zero-coupon bond

(See Figure 3.10, Positive and Negative Correlation.) Some features of correlation are: ■ ■ ■ Positive Correlation. A correlation coefficient equal to +1 means that the two series have behaved identically over the testing period. Negative Correlation. A correlation coefficient of −1 indicates that the series have been inversely proportional during the testing period. In other words, when one price rises, the other price falls. Zero Correlation. A correlation coefficient of zero indicates no relationship between the two values The calculation of the correlation coefficient, ρ, is mathematically defined. (See Equation 3.10, Correlation.) Positive Correlation FIGURE 3.10 Negative Correlation Positive and Negative Correlation ∑ ρ= (x−x) (y − y ) (n − 1)σ x σ y where x Data Set.

…

If crude oil prices rise, then the stock price of the oil company should also be likely to rise. When determining if two prices are correlated, it is necessary to compare changes in price rather than prices. The most common way to measure the relationship between two assets is to calculate the correlation coefficient of their price changes. The correlation coefficient is a number between −1 and +1 that indicates the strength of 77 Financial Mathematics KEY CONCEPT: CORRELATION In the financial markets, the statement that “two assets are correlated” means “the price changes in the two assets are correlated” rather than the “prices are correlated.”

…

The relative size of the value assigned to the hedge and hedged item do not affect hedge effectiveness. A number of summary statistics are produced by a regression analysis. These statistics are commonly used to evaluate the effectiveness of the hedge. Two major statistics used for this purpose are the slope (abbreviated b above) and the correlation coefficient (usually squared and abbreviated as R2 or R‐squared). Secondary statistics include checking that there are enough observations to conduct a valid test, and that the slope and R2 tests are sufficiently stable to trust the results. For example, a hedge‐accounting memo might define five tests to determine a highly effective hedge.

A Random Walk Down Wall Street: The Time-Tested Strategy for Successful Investing (Eleventh Edition) by Burton G. Malkiel

accounting loophole / creative accounting, Alan Greenspan, Albert Einstein, asset allocation, asset-backed security, beat the dealer, Bernie Madoff, bitcoin, book value, butter production in bangladesh, buttonwood tree, buy and hold, capital asset pricing model, compound rate of return, correlation coefficient, Credit Default Swap, Daniel Kahneman / Amos Tversky, Detroit bankruptcy, diversification, diversified portfolio, dogs of the Dow, Edward Thorp, Elliott wave, equity risk premium, Eugene Fama: efficient market hypothesis, experimental subject, feminist movement, financial engineering, financial innovation, financial repression, fixed income, framing effect, George Santayana, hindsight bias, Home mortgage interest deduction, index fund, invisible hand, Isaac Newton, Japanese asset price bubble, John Bogle, junk bonds, Long Term Capital Management, loss aversion, low interest rates, margin call, market bubble, Mary Meeker, money market fund, mortgage tax deduction, new economy, Own Your Own Home, PalmPilot, passive investing, Paul Samuelson, pets.com, Ponzi scheme, price stability, profit maximization, publish or perish, purchasing power parity, RAND corporation, random walk, Richard Thaler, risk free rate, risk tolerance, risk-adjusted returns, risk/return, Robert Shiller, Salesforce, short selling, Silicon Valley, South Sea Bubble, stock buybacks, stocks for the long run, sugar pill, survivorship bias, Teledyne, the rule of 72, The Wisdom of Crowds, transaction costs, Vanguard fund, zero-coupon bond, zero-sum game

Some portfolio managers have argued that diversification has not continued to give the same degree of benefit as was previously the case. Globalization led to an increase in the correlation coefficients between the U.S. and foreign markets as well as between stocks and commodities. The following three charts indicate how correlation coefficients have risen over the first decade of the 2000s. The charts show the correlation coefficients calculated over every twenty-four-month period between U.S. stocks (as measured by the S&P 500-Stock Index) and the EAFE index of developed foreign stocks, between U.S. stocks and the broad (MSCI) index of emerging-market stocks, and between U.S. stocks and the Goldman Sachs (GSCI) index of a basket of commodities such as oil, metals, and the like.

…

The example may seem a bit strained, and most investors will realize that when the market gets clobbered, just about all stocks go down. Still, at least at certain times, some stocks and some classes of assets do move against the market; that is, they have negative covariance or (and this is the same thing) they are negatively correlated with each other. THE CORRELATION COEFFICIENT AND THE ABILITY OF DIVERSIFICATION TO REDUCE RISK Correlation Coefficient Effect of Diversification on Risk +1.0 No risk reduction is possible. +0.5 Moderate risk reduction is possible. 0 Considerable risk reduction is possible. –0.5 Most risk can be eliminated. –1.0 All risk can be eliminated. Now comes the real kicker; negative correlation is not necessary to achieve the risk reduction benefits from diversification.

…

Now comes the real kicker; negative correlation is not necessary to achieve the risk reduction benefits from diversification. Markowitz’s great contribution to investors’ wallets was his demonstration that anything less than perfect positive correlation can potentially reduce risk. His research led to the results presented in the preceding table. As shown, it demonstrates the crucial role of the correlation coefficient in determining whether adding a security or an asset class can reduce risk. DIVERSIFICATION IN PRACTICE To paraphrase Shakespeare, can there be too much of a good thing? In other words, is there a point at which diversification is no longer a magic wand safeguarding returns? Numerous studies have demonstrated that the answer is yes.

Hands-On Machine Learning With Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems by Aurélien Géron

AlphaGo, Amazon Mechanical Turk, Anton Chekhov, backpropagation, combinatorial explosion, computer vision, constrained optimization, correlation coefficient, crowdsourcing, data science, deep learning, DeepMind, don't repeat yourself, duck typing, Elon Musk, en.wikipedia.org, friendly AI, Geoffrey Hinton, ImageNet competition, information retrieval, iterative process, John von Neumann, Kickstarter, machine translation, natural language processing, Netflix Prize, NP-complete, OpenAI, optical character recognition, P = NP, p-value, pattern recognition, pull request, recommendation engine, self-driving car, sentiment analysis, SpamAssassin, speech recognition, stochastic process

Finally, coefficients close to zero mean that there is no linear correlation. Figure 2-14 shows various plots along with the correlation coefficient between their horizontal and vertical axes. Figure 2-14. Standard correlation coefficient of various datasets (source: Wikipedia; public domain image) Warning The correlation coefficient only measures linear correlations (“if x goes up, then y generally goes up/down”). It may completely miss out on nonlinear relationships (e.g., “if x is close to zero then y generally goes up”). Note how all the plots of the bottom row have a correlation coefficient equal to zero despite the fact that their axes are clearly not independent: these are examples of nonlinear relationships.

…

Looking for Correlations Since the dataset is not too large, you can easily compute the standard correlation coefficient (also called Pearson’s r) between every pair of attributes using the corr() method: corr_matrix = housing.corr() Now let’s look at how much each attribute correlates with the median house value: >>> corr_matrix["median_house_value"].sort_values(ascending=False) median_house_value 1.000000 median_income 0.687170 total_rooms 0.135231 housing_median_age 0.114220 households 0.064702 total_bedrooms 0.047865 population -0.026699 longitude -0.047279 latitude -0.142826 Name: median_house_value, dtype: float64 The correlation coefficient ranges from –1 to 1.

…

Note how all the plots of the bottom row have a correlation coefficient equal to zero despite the fact that their axes are clearly not independent: these are examples of nonlinear relationships. Also, the second row shows examples where the correlation coefficient is equal to 1 or –1; notice that this has nothing to do with the slope. For example, your height in inches has a correlation coefficient of 1 with your height in feet or in nanometers. Another way to check for correlation between attributes is to use Pandas’ scatter_matrix function, which plots every numerical attribute against every other numerical attribute. Since there are now 11 numerical attributes, you would get 112 = 121 plots, which would not fit on a page, so let’s just focus on a few promising attributes that seem most correlated with the median housing value (Figure 2-15): from pandas.tools.plotting import scatter_matrix attributes = ["median_house_value", "median_income", "total_rooms", "housing_median_age"] scatter_matrix(housing[attributes], figsize=(12, 8)) Figure 2-15.

Human Diversity: The Biology of Gender, Race, and Class by Charles Murray

23andMe, affirmative action, Albert Einstein, Alfred Russel Wallace, Asperger Syndrome, assortative mating, autism spectrum disorder, basic income, behavioural economics, bioinformatics, Cass Sunstein, correlation coefficient, CRISPR, Daniel Kahneman / Amos Tversky, dark triade / dark tetrad, domesticated silver fox, double helix, Drosophila, emotional labour, epigenetics, equal pay for equal work, European colonialism, feminist movement, glass ceiling, Gregor Mendel, Gunnar Myrdal, income inequality, Kenneth Arrow, labor-force participation, longitudinal study, meritocracy, meta-analysis, nudge theory, out of africa, p-value, phenotype, public intellectual, publication bias, quantitative hedge fund, randomized controlled trial, Recombinant DNA, replication crisis, Richard Thaler, risk tolerance, school vouchers, Scientific racism, selective serotonin reuptake inhibitor (SSRI), Silicon Valley, Skinner box, social intelligence, Social Justice Warrior, statistical model, Steven Pinker, The Bell Curve by Richard Herrnstein and Charles Murray, the scientific method, The Wealth of Nations by Adam Smith, theory of mind, Thomas Kuhn: the structure of scientific revolutions, twin studies, universal basic income, working-age population

Take a pencil (literally or imaginarily) and draw a straight, sloping line through the dots in a way that seems to you to best reflect this upward-sloping trend. Now continue to read and see how well you have intuitively produced the basis for a correlation coefficient and a regression coefficient. The Correlation Coefficient Modern statistics provide more than one method for measuring correlation, but we confine ourselves to the one that is most important in both use and generality: the Pearson product-moment correlation coefficient (named after Karl Pearson, the English mathematician and biometrician). To get at this coefficient, let us first replot the graph of the class, replacing inches and pounds with standard scores.

…

We focus on the slope of the best-fitting line because it is the correlation coefficient—in this case, equal to .50, which is quite large by the standards of variables used by social scientists. The closer it gets to ±1.0, the stronger is the linear relationship between the standardized variables (the variables expressed as standard scores). When the two variables are mutually independent, the best-fitting line is horizontal; hence its slope is 0. Anything other than 0 signifies a relationship, albeit possibly a very weak one. Whatever the correlation coefficient of a pair of variables is, squaring it yields another notable number.

…

“Multiply the effect identified with this correlation by the number of people in a department store the week before Christmas,” the authors wrote, “and it becomes obvious why merchandisers should care deeply about the personalities of their customers.”9 They offered a new set of guidelines based on the correlation coefficient (r). In the summary that follows, I have replaced the value of r with the equivalent value of Cohen’s d. The authors argued that an effect size of .10 “is ‘very small’ for the explanations of single events but potentially consequential in the not-very long run,” while an effect size of .20 “is still ‘small’ at the level of single events but potentially more ultimately consequential.”10 Other scholars have advocated similar guidelines for interpreting small values of d.11 But their treatment of “small” collides with the position taken by the most influential work arguing for small sex differences in cognitive repertoires—the “gender similarities hypothesis” originated by psychologist Janet Shibley Hyde in the September 1985 issue of American Psychologist, the flagship journal of the American Psychological Association.

Python for Finance by Yuxing Yan

asset-backed security, book value, business cycle, business intelligence, capital asset pricing model, constrained optimization, correlation coefficient, data science, distributed generation, diversified portfolio, financial engineering, functional programming, implied volatility, market microstructure, P = NP, p-value, quantitative trading / quantitative finance, risk free rate, Sharpe ratio, tail risk, time value of money, value at risk, volatility smile, zero-sum game

What is the market risk (beta) for IBM in 2010? (Hint: the source of data could be from Yahoo! Finance.) 19. What is wrong with the following lines of code? >>>c=20 >>>npv=np.npv(0.1,c) [ 121 ] Introduction to NumPy and SciPy 20. The correlation coefficient function from NumPy is np.corrcoef(). Find more about this function. Estimate the correlation coefficient between IBM, DELL, and W-Mart. 21. Why is it claimed that the sn.npv() function from SciPY() is really a Present Value (PV) function? 22. Design a true NPV function using all cash flows, including today's cash flow. 23. The Sharpe ratio is used to measure the trade-off between risk and return: Sharpe = R − Rf σ Here, R is the expected returns for an individual security, and R f is the expected risk-free rate. σ is the volatility, that is, standard deviation of the return on the underlying security.

…

For more detail about the function, just type help(plt.annotate) after issuing import matplotlib.pyplot as plt. From the preceding graph, we see that the fluctuation, uncertainty, or risk of our equal-weighted portfolio is much smaller than those of individual stocks in its portfolio. We can also estimate their means, standard deviation, and correlation coefficient. The correlation coefficient between those two stocks is -0.75, and this is the reason why we could diversify away firm-specific risk by forming an even equal-weighted portfolio as shown in the following code: >>>import scipy as sp >>>sp.corrcoef(A,B) array([[ 1. , -0.74583429], [-0.74583429, 1. ]]) In the preceding example, we use hypothetical numbers (returns) for two stocks.

…

First, let us look at a hypothetical case by assuming that we have 5 years' annual returns of two stocks as follows: Year Stock A Stock B 2009 0.102 0.1062 2010 -0.02 0.23 2011 0.213 0.045 2012 0.12 0.234 2013 0.13 0.113 We form an equal-weighted portfolio using those two stocks. Using the mean() and std() functions contained in NumPy, we can estimate their means, standard deviations, and correlation coefficients as follows: >>>import numpy as np >>>A=[0.102,-0.02, 0.213,0.12,0.13] >>>B=[0.1062,0.23, 0.045,0.234,0.113] >>>port_EW=(np.array(ret_A)+np.array(ret_B))/2. >>>round(np.mean(A),3),round(np.mean(B),3),round(np.mean(port_EW),3) (0.109, 0.146, 0.127) >>>round(np.std(A),3),round(np.std(B),3),round(np.std(port_EW),3) (0.075, 0.074, 0.027) In the preceding code, we estimate mean returns, their standard deviations for individual stocks, and an equal-weighted portfolio.

A Random Walk Down Wall Street: The Time-Tested Strategy for Successful Investing by Burton G. Malkiel

accounting loophole / creative accounting, Alan Greenspan, Albert Einstein, asset allocation, asset-backed security, backtesting, Bear Stearns, beat the dealer, Bernie Madoff, book value, BRICs, butter production in bangladesh, buy and hold, capital asset pricing model, compound rate of return, correlation coefficient, Credit Default Swap, Daniel Kahneman / Amos Tversky, diversification, diversified portfolio, dogs of the Dow, Edward Thorp, Elliott wave, Eugene Fama: efficient market hypothesis, experimental subject, feminist movement, financial engineering, financial innovation, fixed income, framing effect, hindsight bias, Home mortgage interest deduction, index fund, invisible hand, Isaac Newton, Japanese asset price bubble, John Bogle, junk bonds, Long Term Capital Management, loss aversion, low interest rates, margin call, market bubble, Mary Meeker, money market fund, mortgage tax deduction, new economy, Own Your Own Home, PalmPilot, passive investing, Paul Samuelson, pets.com, Ponzi scheme, price stability, profit maximization, publish or perish, purchasing power parity, RAND corporation, random walk, Richard Thaler, risk free rate, risk tolerance, risk-adjusted returns, risk/return, Robert Shiller, short selling, Silicon Valley, South Sea Bubble, stock buybacks, stocks for the long run, sugar pill, survivorship bias, The Myth of the Rational Market, the rule of 72, The Wisdom of Crowds, transaction costs, Vanguard fund, zero-coupon bond

Some portfolio managers have argued that diversification has not continued to give the same degree of benefit as was previously the case. Globalization led to an increase in the correlation coefficients between the U.S. and foreign markets as well as between stocks and commodities. The following three charts indicate how correlation coefficients have risen over the first decade of the 2000s. The charts show the correlation coefficients calculated over every twenty-four-month period between U.S. stocks (as measured by the S&P 500-Stock Index) and the EAFE index of developed foreign stocks, between U.S. stocks and the broad (MSCI) index of emerging-market stocks, and between U.S. stocks and the Goldman Sachs (GSCI) index of a basket of commodities such as oil, metals, and the like.

…

The example may seem a bit strained, and most investors will realize that when the market gets clobbered, just about all stocks go down. Still, at least at certain times, some stocks and some classes of assets do move against the market; that is, they have negative covariance or (and this is the same thing) they are negatively correlated with each other. THE CORRELATION COEFFICIENT AND THE ABILITY OF DIVERSIFICATION TO REDUCE RISK Correlation Coefficient Effect of Diversification on Risk +1.0 No risk reduction is possible. +0.5 Moderate risk reduction is possible. 0 Considerable risk reduction is possible. –0.5 Most risk can be eliminated. –1.0 All risk can be eliminated. Now comes the real kicker; negative correlation is not necessary to achieve the risk reduction benefits from diversification.

…

Now comes the real kicker; negative correlation is not necessary to achieve the risk reduction benefits from diversification. Markowitz’s great contribution to investors’ wallets was his demonstration that anything less than perfect positive correlation can potentially reduce risk. His research led to the results presented in the preceding table. As shown, it demonstrates the crucial role of the correlation coefficient in determining whether adding a security or an asset class can reduce risk. DIVERSIFICATION IN PRACTICE To paraphrase Shakespeare, can there be too much of a good thing? In other words, is there a point at which diversification is no longer a magic wand safeguarding returns? Numerous studies have demonstrated that the answer is yes.

Beyond the Random Walk: A Guide to Stock Market Anomalies and Low Risk Investing by Vijay Singal

3Com Palm IPO, Andrei Shleifer, AOL-Time Warner, asset allocation, book value, buy and hold, capital asset pricing model, correlation coefficient, cross-subsidies, currency risk, Daniel Kahneman / Amos Tversky, diversified portfolio, endowment effect, fixed income, index arbitrage, index fund, information asymmetry, information security, junk bonds, liberal capitalism, locking in a profit, Long Term Capital Management, loss aversion, low interest rates, margin call, market friction, market microstructure, mental accounting, merger arbitrage, Myron Scholes, new economy, prediction markets, price stability, profit motive, random walk, Richard Thaler, risk free rate, risk-adjusted returns, risk/return, selection bias, Sharpe ratio, short selling, short squeeze, survivorship bias, Tax Reform Act of 1986, transaction costs, uptick rule, Vanguard fund

Notice that the returns from beachwear and video rental always go in the opposite direction. If one of them does well, the other does not. Therefore, adding stocks that do not behave like other stocks in your portfolio is good and can reduce risk. The correlation is measured by what is called a correlation coefficient. The correlation coefficient varies between –1 and +1. The two stocks in the above example have a correlation of –1. Unfortunately, most stocks have a positive correlation, and many of them have a correlation with the market portfolio that is close to +1. The challenge in diversifying risk is to find stocks that have a correlation of less than +1.

…

Therefore, it is not always necessary to ensure that the return is preserved. A general rule to evaluate whether a new asset should be included in an existing portfolio is based on the risk-return trade-off relationship: E(Rn ) = R f + σ n ρn, p σp × E(Rp ) − R f  where E(R) is the return from an asset, s is the standard deviation, r is the correlation coefficient, and the subscripts n and p refer to the new stock and existing portfolio. Rf is the return on the risk-free asset. If the new asset’s return is greater than the right-hand side in the above equation, then the asset should be included in the existing portfolio, otherwise not. That condition can be rewritten as below: E( Rn ) − R f σn > E(Rp ) − R f σp × ρn, p Evidence Before looking at the evidence, consider the potential benefits from international investing and the source of those benefits.

…

That condition can be rewritten as below: E( Rn ) − R f σn > E(Rp ) − R f σp × ρn, p Evidence Before looking at the evidence, consider the potential benefits from international investing and the source of those benefits. Assume that the dollar return on U.S. stocks is 12 percent with a standard deviation of 18 percent, and the dollar return on non-U.S. stocks is also 12 percent with a standard deviation of 18 percent. Since the U.S. markets and foreign markets are not well correlated, let the correlation coefficient be 0.60. Putting the U.S. stocks and the non-U.S. stocks in a 50-50 combination would generate a new world portfolio with the following characteristics: Rw = w1RUS + w2 Rnon −US = 0.50 × 12% + 0.50 × 12% = 12% σ w = w12σ 12 + w22σ 22 + 2w1w2 ρσ 1σ 2 = 0.50 2 × 0.18 2 + 0.50 2 × 0.18 2 + 2 × 0.50 × 0.50 × 0.60 × 0.18 × 0.18 = 0.16 236 Beyond the Random Walk The new world portfolio has a return of 12 percent and a risk of 16 percent.

Trading Risk: Enhanced Profitability Through Risk Control by Kenneth L. Grant

backtesting, business cycle, buy and hold, commodity trading advisor, correlation coefficient, correlation does not imply causation, delta neutral, diversification, diversified portfolio, financial engineering, fixed income, frictionless, frictionless market, George Santayana, global macro, implied volatility, interest rate swap, invisible hand, Isaac Newton, John Meriwether, Long Term Capital Management, managed futures, market design, Myron Scholes, performance metric, price mechanism, price stability, proprietary trading, risk free rate, risk tolerance, risk-adjusted returns, Sharpe ratio, short selling, South Sea Bubble, Stephen Hawking, the scientific method, The Wealth of Nations by Adam Smith, transaction costs, two-sided market, uptick rule, value at risk, volatility arbitrage, yield curve, zero-coupon bond

For example, 170 TRADING RISK there’s no reason to believe that there are any statistical commonalities between, say, the Swedish rate of inflation and the price of silkworms in Malaysia; and over time we would expect a correlation between these two variables to be roughly zero. In terms of magnitudes, the correlation coefficient has a maximum value of 1.0, or 100%, indicating perfect correlation (e.g., the temperature in Toronto as measured in Fahrenheit and Celsius), and a minimum value of 1.0, or 100%, indicating perfect negative correlation (e.g., the price of a zero-coupon bond and its yield). All values in between are valid, and the process lends itself to all the subjectivity that the human mind can muster. However, you may find the following (admittedly simplistic) rules of thumb to be useful: Value of Correlation Coefficient Less than 50% Between 50% and 10% Between 10% and 10% Between 10% and 50% Greater than 50% Interpretation High negative correlation—merits full investigation.

…

Correlations The final core element of our introductory statistical tool kit is correlation analysis. You ought to be at least nominally familiar with this concept, which involves identifying the extent to which two or more data series dynamically exhibit similar characteristics, most notably, for our purposes, across time. Correlation coefficients can range from 100% to 100% but (unless data series are simply disguised representations of a single concept, for example, the yield on a given bond and its price) typically fall somewhere in between. By performing correlation analysis on the time series of portfolio returns, traders stand to gain unique and specific insights into underlying portfolio economics.

…

Following is a summary of some of the standard categories of correlation analysis that you may find useful in identifying the drivers of relative performance for your portfolio. Correlation against Market Benchmarks. This is the general case associated with the example provided prior, under which you might calculate “correlation coefficients” between your returns and the performance 74 TRADING RISK of various market indexes. Here, I recommend that you begin the process by simply identifying, in an anecdotal sense, the market indexes that might best capture the essence of your trading and then running some introductory correlations there.

Data Mining: Concepts, Models, Methods, and Algorithms by Mehmed Kantardzić

Albert Einstein, algorithmic bias, backpropagation, bioinformatics, business cycle, business intelligence, business process, butter production in bangladesh, combinatorial explosion, computer vision, conceptual framework, correlation coefficient, correlation does not imply causation, data acquisition, discrete time, El Camino Real, fault tolerance, finite state, Gini coefficient, information retrieval, Internet Archive, inventory management, iterative process, knowledge worker, linked data, loose coupling, Menlo Park, natural language processing, Netflix Prize, NP-complete, PageRank, pattern recognition, peer-to-peer, phenotype, random walk, RFID, semantic web, speech recognition, statistical model, Telecommunications Act of 1996, telemarketer, text mining, traveling salesman, web application

It is wrong to conclude that r = 0.6 indicates a linear relationship twice as strong as that indicated by the value r = 0.3. For our simple example of linear regression given at the beginning of this section, the model obtained was B = 0.8 + 0.92A. We may estimate the quality of the model using the correlation coefficient r as a measure. Based on the available data in Figure 4.3, we obtained intermediate results and the final correlation coefficient: A correlation coefficient r = 0.85 indicates a good linear relationship between two variables. Additional interpretation is possible. Because r2 = 0.72, we can say that approximately 72% of the variations in the values of B is accounted for by a linear relationship with A. 5.5 ANOVA Often the problem of analyzing the quality of the estimated regression line and the influence of the independent variables on the final regression is handled through an ANOVA approach.

…

Given a data set with two dimensions X and Y: X Y 1 5 4 2.75 3 3 5 2.5 (a) Use a linear regression method to calculate the parameters α and β where y = α + β x. (b) Estimate the quality of the model obtained in (a) using the correlation coefficient r. (c) Use an appropriate nonlinear transformation (one of those represented in Table 5.3) to improve regression results. What is the equation for a new, improved, and nonlinear model? Discuss a reduction of the correlation coefficient value. 5. A logit function, obtained through logistic regression, has the form: Find the probability of output values 0 and 1 for the following samples: (a) { 1, −1, −1 } (b) { −1, 1, 0 } (c) { 0, 0, 0 } 6.

…

On the other hand, the test for feature Y is significantly above the threshold value; this feature is not a candidate for reduction because it has the potential to be a distinguishing feature between two classes. A similar idea for feature ranking is shown in the algorithm that is based on correlation criteria. Let us consider first the prediction of a continuous outcome y. The Pearson correlation coefficient is defined as: where cov designates the covariance and var the variance. The estimate of R(i) for the given data set with samples’ inputs xk,,j and outputs yk is defined by: where the bar notation stands for an average over the index k (set of all samples). Using R(i)2 as a variable-ranking criterion enforces a ranking according to goodness of linear fit of individual variables.

Corporate Finance: Theory and Practice by Pierre Vernimmen, Pascal Quiry, Maurizio Dallocchio, Yann le Fur, Antonio Salvi

"Friedman doctrine" OR "shareholder theory", accelerated depreciation, accounting loophole / creative accounting, active measures, activist fund / activist shareholder / activist investor, AOL-Time Warner, ASML, asset light, bank run, barriers to entry, Basel III, Bear Stearns, Benoit Mandelbrot, bitcoin, Black Swan, Black-Scholes formula, blockchain, book value, business climate, business cycle, buy and hold, buy low sell high, capital asset pricing model, carried interest, collective bargaining, conceptual framework, corporate governance, correlation coefficient, credit crunch, Credit Default Swap, currency risk, delta neutral, dematerialisation, discounted cash flows, discrete time, disintermediation, diversification, diversified portfolio, Dutch auction, electricity market, equity premium, equity risk premium, Eugene Fama: efficient market hypothesis, eurozone crisis, financial engineering, financial innovation, fixed income, Flash crash, foreign exchange controls, German hyperinflation, Glass-Steagall Act, high net worth, impact investing, implied volatility, information asymmetry, intangible asset, interest rate swap, Internet of things, inventory management, invisible hand, joint-stock company, joint-stock limited liability company, junk bonds, Kickstarter, lateral thinking, London Interbank Offered Rate, low interest rates, mandelbrot fractal, margin call, means of production, money market fund, moral hazard, Myron Scholes, new economy, New Journalism, Northern Rock, performance metric, Potemkin village, quantitative trading / quantitative finance, random walk, Right to Buy, risk free rate, risk/return, shareholder value, short selling, Social Responsibility of Business Is to Increase Its Profits, sovereign wealth fund, Steve Jobs, stocks for the long run, supply-chain management, survivorship bias, The Myth of the Rational Market, time value of money, too big to fail, transaction costs, value at risk, vertical integration, volatility arbitrage, volatility smile, yield curve, zero-coupon bond, zero-sum game

Depending on the proportion of Criteo shares in the portfolio (XC), the portfolio would look like this: XC (%) 0 25 33.3 50 66.7 75 100 E(rH,C) (%) 6   7.8  8.3   9.5 10.7 11.3   13 The portfolio’s variance is determined as follows: where Cov(rH, rC) is the covariance. It measures the degree to which Heineken and Criteo fluctuate together. It is equal to: Here, pi,j is the probability of joint occurrence and rH,C is the correlation coefficient of returns offered by Heineken and Criteo. The correlation coefficient is a number between −1 (returns 100% inversely proportional to each other) and 1 (returns 100% proportional to each other). Correlation coefficients are usually positive, as most stocks rise together in a bullish market and fall together in a bearish market. By plugging the variables back into our variance equation above, we obtain: Given that: it is therefore possible to say: or: As the above calculations show, the overall risk of a portfolio consisting of Criteo and Heineken shares is less than the weighted average of the risks of the two stocks.

…

By varying ρH,C between −1 and +1, we obtain: Proportion of C shares in portfolio (XC) (%) 0 25 33.3 50 66.7 75 100 Return on the portfolio: E(rH,C) (%) 6.0 7.8 8.3 9.5 10.7 11.3 13.0 Portfolio risk σ(rH,C) (%) ρH,C = −1 10.0 3.3 1.0 3.5 8.0 10.3 17.0 ρH,C = −0.5 10.0 6.5 6.2 7.4 10.1 11.7 17.0 ρH,C = 0 10.0 8.6 8.7 9.9 11.8 13.0 17.0 ρH,C = 0.3 10.0 9.7 10.0 11.1 12.7 13.7 17.0 ρH,C = 0.5 10.0 10.3 10.7 11.8 13.3 14.2 17.0 ρH,C = 1 10.0 11.8 12.3 13.5 14.7 15.3 17.0 Note the following caveats: If Criteo and Heineken were perfectly correlated (i.e. the correlation coefficient was 1), then diversification would have no effect. All possible portfolios would lie on a line linking the risk/return point of Criteo with that of Heineken. Risk would increase in direct proportion to Criteo’s stock added. If the two stocks were perfectly inversely correlated (correlation coefficient −1), then diversification would be total. However, there is little chance of this occurring, as both companies are exposed to the same economic conditions.

…

Generally speaking, Criteo and Heineken are positively, but imperfectly, correlated and diversification is based on the desired amount of risk. With a fixed correlation coefficient of 0.3, there are portfolios that offer different returns at the same level of risk. Thus, a portfolio consisting of two-thirds Heineken and one-third Criteo shows the same risk (10%) as a portfolio consisting of just Heineken, but returns 8.3% vs. only 6% for Heineken. As long as the correlation coefficient is below 1, diversification will be efficient. There is no reason for an investor to choose a given combination if another offers a better (efficient) return at the same level of risk.

Optimization Methods in Finance by Gerard Cornuejols, Reha Tutuncu

asset allocation, call centre, constrained optimization, correlation coefficient, diversification, financial engineering, finite state, fixed income, frictionless, frictionless market, index fund, linear programming, Long Term Capital Management, passive investing, Sharpe ratio, transaction costs, value at risk

Consider an investor who has a certain amount of money to be invested in a number of different securities (stocks, bonds, etc.) with random returns. For each security i, i = 1, . . . , n, estimates of its expected return, µi , and variance, σi2 , are given. Furthermore, for any two securities i and j, their correlation coefficient ρij is also assumed to be known. If we represent the proportion of the total funds invested in security i by xi , one can compute the expected return and the variance of the resulting portfolio x = (x1 , . . . , xn ) as follows: E[x] = x1 µ1 + . . . + xn µn = µT x, and V ar[x] = X ρij σi σj xi xj = xT Qx i,j where ρii ≡ 1, Qij = ρij σi σj for i 6= j, Qii = σi2 , and µ = (µ1 , . . . , µn ).

…

Chapter 5 QP Models and Tools in Finance 5.1 Mean-Variance Optimization In the introductory chapter, we have discussed Markowitz’ theory of mean-variance optimization (MVO) for the selection of portfolios of securities (or asset classes) in a manner that trades off the expected returns and the perceived risk of potential portfolios. Consider assets S1 , S2 , . . . , Sn (n ≥ 2) with random returns. Let µi and σi denote the expected return and the standard deviation of the return of asset Si . For i 6= j, ρij denotes the correlation coefficient of the returns of assets Si and Sj . Let µ = [µ1 , . . . , µn ]T , and Q be the n × n symmetric covariance matrix with Qii = σi2 and Qij = ρij σi σj for i 6= j. Denoting the proportion of the total funds invested in security i by xi , one can represent the expected return and the variance of the resulting portfolio x = (x1 , . . . , xn ) as follows: E[x] = x1 µ1 + . . . + xn µn = µT x, and V ar[x] = X ρij σi σj xi xj = xT Qx, i,j where ρii ≡ 1.

…

The standard deviation of a random variable is the square-root of its variance. 108 APPENDIX C. A PROBABILITY PRIMER For two jointly distributed random variables X1 and X2 , their covariance is defined to be Cov(X1 , X2 ) = E [(X1 − E[X1 ])(X2 − E[X2 ])] = E[X1 X2 ] − E[X1 ]E[X2 ] The correlation coefficient of two random variables is the ratio of their covariance to the product of their standard deviations. For a collection of random variables X1 , . . . , Xn , the expected value of the sum of these random variables is equal to the sum of their expected values: " E n X i=1 # Xi = n X E[Xi ].

The Art of Computer Programming by Donald Ervin Knuth

Abraham Wald, Brownian motion, Charles Babbage, complexity theory, correlation coefficient, Donald Knuth, Eratosthenes, G4S, Georg Cantor, information retrieval, Isaac Newton, iterative process, John von Neumann, Louis Pasteur, mandelbrot fractal, Menlo Park, NP-complete, P = NP, Paul Erdős, probability theory / Blaise Pascal / Pierre de Fermat, RAND corporation, random walk, sorting algorithm, Turing machine, Y2K

We may also compute the following statistic: This is the "serial correlation coefficient," a measure of the extent to which Uj+i depends on Uj. Correlation coefficients appear frequently in statistical work. If we have n quantities Uo, Ui, ..., t/n-i and n others Vo, Vi, ..., Vn_i, the correlation coefficient between them is defined to be c = All summations in this formula are to be taken over the range 0 < j < n; Eq. B3) is the special case Vj = C/(j+i) mod n- The denominator of B4) is zero when JJo = U\ = ¦ ¦ ¦ = Un-\ or V$ = V\ = ¦ ¦ ¦ = Vn-\\ we exclude that case from discussion. 3.3.2 EMPIRICAL TESTS 73 A correlation coefficient always lies between —1 and +1.

…

Show that the correlation coefficient C given in Eq. B4) is equal to ? u'kvL 0<k<n b) Let C = N/D, where N and D denote the numerator and denominator of the expression in part (a). Show that N2 < D2, hence — 1 < C < 1; and obtain a formula for the difference D2 - N2. [Hint: See exercise 1.2.3-30.] c) If C = ±1, show that aUk + CVk = t, 0 < k < n, for some constants a, C, and r, not all zero. 18. [M20] (a) Show that if n = 2, the serial correlation coefficient B3) is always equal to —1 (unless the denominator is zero), (b) Similarly, show that when n = 3, the serial correlation coefficient always equals — \.

…

In actual fact, since U0U1 is not completely independent of U1U2, the serial correlation coefficient is not expected to be exactly zero. (See exercise 18.) A "good" value of C will be between \xn — 2an and \xn + 2an, where l" ">2' B5) "-=;rrr °l=(n-1) (n - 2)' ">2' We expect C to be between these limits about 95 percent of the time. The formula for a\ in B5) is an upper bound, valid for serial correlations between independent random variables from an arbitrary distribution. When the C/'s are uniformly distributed, the true variance is obtained by subtracting %r-n~2 + O(n~7/3 logn). (See exercise 20.) Instead of simply computing the correlation coefficient between the obser- observations (Uo, U\, ..., Un-\) and their immediate successors (U\,..., Un-i,Uo), we can also compute it between (Uo, U\,..., Un-\) and any cyclically shifted sequence (Uq,...

Triumph of the Optimists: 101 Years of Global Investment Returns by Elroy Dimson, Paul Marsh, Mike Staunton

asset allocation, banking crisis, Berlin Wall, Black Monday: stock market crash in 1987, book value, Bretton Woods, British Empire, buy and hold, capital asset pricing model, capital controls, central bank independence, classic study, colonial rule, corporate governance, correlation coefficient, cuban missile crisis, currency risk, discounted cash flows, diversification, diversified portfolio, dividend-yielding stocks, equity premium, equity risk premium, Eugene Fama: efficient market hypothesis, European colonialism, fixed income, floating exchange rates, German hyperinflation, index fund, information asymmetry, joint-stock company, junk bonds, negative equity, new economy, oil shock, passive investing, purchasing power parity, random walk, risk free rate, risk tolerance, risk/return, selection bias, shareholder value, Sharpe ratio, stocks for the long run, survivorship bias, Tax Reform Act of 1986, technology bubble, transaction costs, yield curve

The inter-war period, with its post-war boom, hyperinflation in Germany, the Wall Street Crash, and the Great Depression, was unique. Correlations were quite high due to common factors such as the crash and Depression, but the correlation structure differed from all other periods. Figure 8-6: Correlation coefficients between four core countries over seven successive sub-periods 0.7 0.6 0.5 0.4 Correlation coefficients US:UK US:Fra UK:Fra Average US:Ger UK:Ger Ger:Fra .40 .26 0.3 0.2 0.1 .09 .14 .15 .01 0.0 -0.1 -.07 -0.2 -0.3 -0.4 -0.5 1872–1889 1889–1914 Source: Goetzmann, Li, and Rouwenhorst, 2001 1915–1918 1919–1939 1940–1945 1946–1971 1972–2000 Chapter 8: International investment 117 Longin and Solnik (1995) provide further evidence of high correlations during periods of poor performance.

…

France’s highest correlations were with Belgium, The Netherlands, Italy, Ireland, Spain, and Switzerland; Italy’s were with France and Switzerland; The Netherlands was most highly correlated with Belgium, followed by France, Denmark, and Switzerland; and Sweden was highly correlated with Denmark, Canada (natural resources), and Switzerland (neutral countries). Australia’s highest correlations were with the United Kingdom and Ireland (historical and trade links), and Canada and South Africa (gold, mining, and the British Empire). 115 Chapter 8: International investment Table 8-3: Correlation coefficients between world equity markets* Wld Wld US UK .93 Swi Swe Spa SAf Neth Jap Ita Ire Ger Fra Den Can Bel Aus .77 .59 .62 .67 .54 .73 .68 .52 .69 .69 .73 .57 .82 .54 .69 .67 .44 .46 .53 .46 .57 .49 .40 .66 .56 .56 .46 .78 .45 .57 US .85 UK .70 .55 Swi .68 .50 .62 Swe .62 .44 .42 .54 Spa .41 .25 .25 .36 .37 SAf .55 .43 .49 .39 .34 .26 Neth .57 .39 .42 .51 .43 .28 .58 .44 .63 .31 .71 .42 .39 .73 .58 .59 .57 .57 .59 .56 .39 .60 .19 .72 .36 .45 .57 .53 .64 .58 .35 .63 .37 .63 .38 .63 .34 .49 .27 .76 .76 .44 .61 .29 .44 .35 .63 .32 .64 .50 .64 .75 .56 .51 .55 .54 .30 .29 .44 .24 .31 .42 .37 .25 .62 .10 .66 .39 .59 .63 .74 .77 .64 .55 .70 .46 Jap .45 .21 .33 .29 .39 .40 .31 .25 Ita .54 .37 .43 .52 .39 .41 .41 .32 Ire .58 .38 .73 .70 .42 .35 .42 .46 .29 .43 Ger .30 .12 -.01 .22 .09 -.03 .05 .27 .06 .16 .18 .34 .33 .25 .36 .24 .50 .17 .59 .33 .55 .71 .50 .40 .51 .38 .42 .03 .45 .49 .54 .57 .50 .83 .61 .57 .59 .46 Fra .62 .36 .45 .54 .44 .47 .38 .48 .25 .52 .53 .19 Den .57 .38 .40 .51 .56 .34 .31 .50 .46 .38 .55 .22 .45 Can .80 .80 .55 .48 .53 .27 .54 .34 .30 .37 .41 .13 .35 .46 Bel .58 .38 .40 .57 .43 .40 .29 .60 .25 .47 .49 .26 .68 .42 .35 Aus .66 .47 .66 .51 .50 .28 .56 .41 .28 .43 .62 .04 .47 .42 .62 .63 .60 .66 .48 .55 .54 .30 .30 .65 .30 .35 * Correlations in bold (lower left-hand triangle) are based on 101 years of real dollar returns, 1900–2000.

…

Correlations have nevertheless shifted significantly over time, and Table 8-4 provides evidence of this. The table presents the results of estimating correlations over a prediction period from information over an earlier historical period. The top panel shows that when the full set of pairwise correlation coefficients between equity markets are estimated separately for the first and second halves of the twentieth century, there was no discernable relationship between the two. It would not have been possible to predict correlations for 1950–2000 from those estimated from annual data over the first half-century.

Principles of Corporate Finance by Richard A. Brealey, Stewart C. Myers, Franklin Allen

3Com Palm IPO, accelerated depreciation, accounting loophole / creative accounting, Airbus A320, Alan Greenspan, AOL-Time Warner, Asian financial crisis, asset allocation, asset-backed security, banking crisis, Bear Stearns, Bernie Madoff, big-box store, Black Monday: stock market crash in 1987, Black-Scholes formula, Boeing 747, book value, break the buck, Brownian motion, business cycle, buy and hold, buy low sell high, California energy crisis, capital asset pricing model, capital controls, Carl Icahn, Carmen Reinhart, carried interest, collateralized debt obligation, compound rate of return, computerized trading, conceptual framework, corporate governance, correlation coefficient, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, cross-border payments, cross-subsidies, currency risk, discounted cash flows, disintermediation, diversified portfolio, Dutch auction, equity premium, equity risk premium, eurozone crisis, fear index, financial engineering, financial innovation, financial intermediation, fixed income, frictionless, fudge factor, German hyperinflation, implied volatility, index fund, information asymmetry, intangible asset, interest rate swap, inventory management, Iridium satellite, James Webb Space Telescope, junk bonds, Kenneth Rogoff, Larry Ellison, law of one price, linear programming, Livingstone, I presume, London Interbank Offered Rate, Long Term Capital Management, loss aversion, Louis Bachelier, low interest rates, market bubble, market friction, money market fund, moral hazard, Myron Scholes, new economy, Nick Leeson, Northern Rock, offshore financial centre, PalmPilot, Ponzi scheme, prediction markets, price discrimination, principal–agent problem, profit maximization, purchasing power parity, QR code, quantitative trading / quantitative finance, random walk, Real Time Gross Settlement, risk free rate, risk tolerance, risk/return, Robert Shiller, Scaled Composites, shareholder value, Sharpe ratio, short selling, short squeeze, Silicon Valley, Skype, SpaceShipOne, Steve Jobs, subprime mortgage crisis, sunk-cost fallacy, systematic bias, Tax Reform Act of 1986, The Nature of the Firm, the payments system, the rule of 72, time value of money, too big to fail, transaction costs, University of East Anglia, urban renewal, VA Linux, value at risk, Vanguard fund, vertical integration, yield curve, zero-coupon bond, zero-sum game, Zipcar

As you might guess, the covariance is a measure of the degree to which the two stocks “covary.” The covariance can be expressed as the product of the correlation coefficient ρ12 and the two standard deviations:28 For the most part stocks tend to move together. In this case the correlation coefficient ρ12 is positive, and therefore the covariance σ12 is also positive. If the prospects of the stocks were wholly unrelated, both the correlation coefficient and the covariance would be zero; and if the stocks tended to move in opposite directions, the correlation coefficient and the covariance would be negative. Just as you weighted the variances by the square of the proportion invested, so you must weight the covariance by the product of the two proportionate holdings x1 and x2.

…

Recalculate the expected portfolio return and standard deviation for different values of x1 and x2, assuming the correlation coefficient ρ12 = 0. Plot the range of possible combinations of expected return and standard deviation as in Figure 8.3. Repeat the problem for ρ12 = +.25. 11. Portfolio risk and return Mark Harrywitz proposes to invest in two shares, X and Y. He expects a return of 12% from X and 8% from Y. The standard deviation of returns is 8% for X and 5% for Y. The correlation coefficient between the returns is .2. a. Compute the expected return and standard deviation of the following portfolios: b.

…

Calculate the variance and standard deviation of the returns on a portfolio that has equal investments in 2 shares, 3 shares, and so on, up to 10 shares. b. Use your estimates to draw a graph like Figure 7.11. How large is the underlying market risk that cannot be diversified away? c. Now repeat the problem, assuming that the correlation between each pair of stocks is zero. 17. Portfolio risk Table 7.9 shows standard deviations and correlation coefficients for eight stocks from different countries. Calculate the variance of a portfolio with equal investments in each stock. 18. Portfolio risk Your eccentric Aunt Claudia has left you $50,000 in BP shares plus $50,000 cash. Unfortunately her will requires that the BP stock not be sold for one year and the $50,000 cash must be entirely invested in one of the stocks shown in Table 7.9.

Is God a Mathematician? by Mario Livio

Albert Einstein, Alvin Toffler, Antoine Gombaud: Chevalier de Méré, Brownian motion, cellular automata, correlation coefficient, correlation does not imply causation, cosmological constant, Dava Sobel, double helix, Edmond Halley, Eratosthenes, Future Shock, Georg Cantor, Gerolamo Cardano, Gregor Mendel, Gödel, Escher, Bach, Henri Poincaré, Isaac Newton, Johannes Kepler, John von Neumann, music of the spheres, Myron Scholes, Plato's cave, probability theory / Blaise Pascal / Pierre de Fermat, Russell's paradox, seminal paper, Thales of Miletus, The Design of Experiments, the scientific method, traveling salesman

For a given value of the temperature, one cannot predict precisely the number of forest fires that will break out, since the latter depends on other variables such as the humidity and the number of fires started by people. In other words, for any value of the temperature, there could be many corresponding numbers of forest fires and vice versa. Still, the mathematical concept known as the correlation coefficient allows us to measure quantitatively the strength of the relationship between two such variables. The person who first introduced the tool of the correlation coefficient was the Victorian geographer, meteorologist, anthropologist, and statistician Sir Francis Galton (1822–1911). Galton—who was, by the way, the half-cousin of Charles Darwin—was not a professional mathematician.

…

If the correlation between them is very close, a very long cubit would usually imply a very tall stature, but if it were not very close, a very long cubit would be on the average associated with only a tall stature, and not a very tall one; while, if it were nil, a very long cubit would be associated with no especial stature, and therefore, on the average, with mediocrity. Pearson eventually gave a precise mathematical definition of the correlation coefficient. The coefficient is defined in such a way that when the correlation is very high—that is, when one variable closely follows the up-and-down trends of the other—the coefficient takes the value of 1. When two quantities are anticorrelated, meaning that when one increases the other decreases and vice versa, the coefficient is equal to–1. Two variables that each behave as if the other didn’t even exist have a correlation coefficient of 0. (For instance, the behavior of some governments unfortunately shows almost zero correlation with the wishes of the people whom they supposedly represent.)

Data Mining: Concepts and Techniques: Concepts and Techniques by Jiawei Han, Micheline Kamber, Jian Pei

backpropagation, bioinformatics, business intelligence, business process, Claude Shannon: information theory, cloud computing, computer vision, correlation coefficient, cyber-physical system, database schema, discrete time, disinformation, distributed generation, finite state, industrial research laboratory, information retrieval, information security, iterative process, knowledge worker, linked data, machine readable, natural language processing, Netflix Prize, Occam's razor, pattern recognition, performance metric, phenotype, power law, random walk, recommendation engine, RFID, search costs, semantic web, seminal paper, sentiment analysis, sparse data, speech recognition, statistical model, stochastic process, supply-chain management, text mining, thinkpad, Thomas Bayes, web application

Since our computed value is above this, we can reject the hypothesis that gender and preferred_reading are independent and conclude that the two attributes are (strongly) correlated for the given group of people. Correlation Coefficient for Numeric Data For numeric attributes, we can evaluate the correlation between two attributes, A and B, by computing the correlation coefficient (also known as Pearson's product moment coefficient, named after its inventer, Karl Pearson). This is(3.3) where n is the number of tuples, ai and bi are the respective values of A and B in tuple i, Ā and are the respective mean values of A and B, σA and σB are the respective standard deviations of A and B (as defined in Section 2.2.2), and Σ(aibi) is the sum of the AB cross-product (i.e., for each tuple, the value for A is multiplied by the value for B in that tuple).

…

Inconsistencies in attribute or dimension naming can also cause redundancies in the resulting data set. Some redundancies can be detected by correlation analysis. Given two attributes, such analysis can measure how strongly one attribute implies the other, based on the available data. For nominal data, we use the χ2 (chi-square) test. For numeric attributes, we can use the correlation coefficient and covariance, both of which access how one attribute's values vary from those of another. χ2 Correlation Test for Nominal Data For nominal data, a correlation relationship between two attributes, A and B, can be discovered by a χ2 (chi-square) test. Suppose A has c distinct values, namely a1, a2, … ac.

…

Consider two numeric attributes A and B, and a set of n observations {(a1, b1), …, (an, bn)}. The mean values of A and B, respectively, are also known as the expected values on A and B, that is, and The covariance between A and B is defined as(3.4) If we compare Eq. (3.3) for rA, B (correlation coefficient) with Eq. (3.4) for covariance, we see that(3.5) where σA and σB are the standard deviations of A and B, respectively. It can also be shown that(3.6) This equation may simplify calculations. For two attributes A and B that tend to change together, if A is larger than Ā (the expected value of A), then B is likely to be larger than (the expected value of B).

The Concepts and Practice of Mathematical Finance by Mark S. Joshi

Black-Scholes formula, Brownian motion, correlation coefficient, Credit Default Swap, currency risk, delta neutral, discrete time, Emanuel Derman, financial engineering, fixed income, implied volatility, incomplete markets, interest rate derivative, interest rate swap, London Interbank Offered Rate, martingale, millennium bug, power law, quantitative trading / quantitative finance, risk free rate, short selling, stochastic process, stochastic volatility, the market place, time value of money, transaction costs, value at risk, volatility smile, yield curve, zero-coupon bond

In particular, we have that E((Yt - YS)(Xil) - X(1))) = PE((Xtl) - Xs 1))2) + 1 - p2]E((Xi2)- X(Z))(Xt`1 Xs1))). (11.4) Since V) and X(2) are independent, the second expectation is zero and so 1E((Yt - (Xrl) YS) - Xs 1))) = p(t - s). (11.5) As Yt - YS and Xtl) - XSl) both have variance t - s, this means that the correlation coefficient is p. Thus we have constructed a Brownian motion whose increments are correlated to those of X(1) with correlation p. More generally, we could construct a Brownian motion from any vector a = (al,ak) with a? = 1, by taking Ek=l ajX(J). 11.3 The higher-dimensional Ito calculus 263 The existence of such correlated Brownian motions will be crucial in pricing multi-asset options.

…

(11.12) The second term has mean zero and variance of order At2 so we can discard it as small, whereas the first term has mean pjk At and variance of order At2 and therefore contributes. This gives us a new rule for the multi-dimensional Ito calculus: dWrj)dW(k) = pjkdt. To summarize, we have Theorem 11.1 (Multi-dimensional Ito lemma) Let Wtj) be correlated Brownian motions with correlation coefficient pjk between the Brownian motions WU) and Wtk). Let Xj be an Ito process with respect to Wt W. Let f be a smooth function; we then have that af of 11 at j=1 11 1 T a2f + 2 j,k=1 ax axi` (t, X1, ax j (t,X1,...,X,1)dXj ..., X7,)dX jdXk, (11.14) with dWtj)dWtk) = pjkdt. When collecting terms, the final double sum will be absorbed into the dt term.

…

Perfect correlation means the vectors point the same way, perfect negative correlation means they point the opposite way, and zero correlation means they are orthogonal. In (11.20), the first vector has length O'1, and the second length oa2. When we add two vectors, v1, V2, the square of the length of the resultant vector is IIv1112 + 2 cos(9)IIv111. i1v211 + IIv2112, where 9 is the angle between the vectors. If we interpret the correlation coefficient as being the cosine of the angle between the two Brownian motions, then this means that the new volatility is just the length of the vector obtained by summing the vectors for each Brownian motion. 266 Multiple sources of risk More generally, we could construct a Brownian motion from any vector a=(a1,...

The Book of Why: The New Science of Cause and Effect by Judea Pearl, Dana Mackenzie

affirmative action, Albert Einstein, AlphaGo, Asilomar, Bayesian statistics, computer age, computer vision, Computing Machinery and Intelligence, confounding variable, correlation coefficient, correlation does not imply causation, Daniel Kahneman / Amos Tversky, data science, deep learning, DeepMind, driverless car, Edmond Halley, Elon Musk, en.wikipedia.org, experimental subject, Great Leap Forward, Gregor Mendel, Isaac Newton, iterative process, John Snow's cholera map, Loebner Prize, loose coupling, Louis Pasteur, Menlo Park, Monty Hall problem, pattern recognition, Paul Erdős, personalized medicine, Pierre-Simon Laplace, placebo effect, Plato's cave, prisoner's dilemma, probability theory / Blaise Pascal / Pierre de Fermat, randomized controlled trial, Recombinant DNA, selection bias, self-driving car, seminal paper, Silicon Valley, speech recognition, statistical model, Stephen Hawking, Steve Jobs, strong AI, The Design of Experiments, the scientific method, Thomas Bayes, Turing test

Galton’s disciple Karl Pearson later derived a formula for the slope of the (properly rescaled) regression line and called it the correlation coefficient. This is still the first number that statisticians all over the world compute when they want to know how strongly two different variables in a data set are related. Galton and Pearson must have been thrilled to find such a universal way of describing the relationships between random variables. For Pearson, especially, the slippery old concepts of cause and effect seemed outdated and unscientific, compared to the mathematically clear and precise concept of a correlation coefficient. GALTON AND THE ABANDONED QUEST It is an irony of history that Galton started out in search of causation and ended up discovering correlation, a relationship that is oblivious of causation.

…

It was Galton who first freed me from the prejudice that sound mathematics could only be applied to natural phenomena under the category of causation.” In Pearson’s eyes, Galton had enlarged the vocabulary of science. Causation was reduced to nothing more than a special case of correlation (namely, the case where the correlation coefficient is 1 or –1 and the relationship between x and y is deterministic). He expresses his view of causation with great clarity in The Grammar of Science (1892): “That a certain sequence has occurred and reoccurred in the past is a matter of experience to which we give expression in the concept causation.… Science in no case can demonstrate any inherent necessity in a sequence, nor prove with absolute certainty that it must be repeated.”

…

The parameter a (often denoted by rYX, the regression coefficient of Y on X) tells us the average observed trend: a one-unit increase of X will, on average, produce an a-unit increase in Y. If there are no confounders of Y and X, then we can use this as our estimate of an intervention to increase X by one unit. But what if there is a confounder, Z? In this case, the correlation coefficient rYX will not give us the average causal effect; it only gives us the average observed trend. That was the case in Wright’s problem of the guinea pig birth weights, discussed in Chapter 2, where the apparent benefit (5.66 grams) of an extra day’s gestation was biased because it was confounded with the effect of a smaller litter size.

The Rise of the Quants: Marschak, Sharpe, Black, Scholes and Merton by Colin Read

Abraham Wald, Albert Einstein, Bayesian statistics, Bear Stearns, Black-Scholes formula, Bretton Woods, Brownian motion, business cycle, capital asset pricing model, collateralized debt obligation, correlation coefficient, Credit Default Swap, credit default swaps / collateralized debt obligations, David Ricardo: comparative advantage, discovery of penicillin, discrete time, Emanuel Derman, en.wikipedia.org, Eugene Fama: efficient market hypothesis, financial engineering, financial innovation, fixed income, floating exchange rates, full employment, Henri Poincaré, implied volatility, index fund, Isaac Newton, John Meriwether, John von Neumann, Joseph Schumpeter, Kenneth Arrow, Long Term Capital Management, Louis Bachelier, margin call, market clearing, martingale, means of production, moral hazard, Myron Scholes, Paul Samuelson, price stability, principal–agent problem, quantitative trading / quantitative finance, RAND corporation, random walk, risk free rate, risk tolerance, risk/return, Robert Solow, Ronald Reagan, shareholder value, Sharpe ratio, short selling, stochastic process, Thales and the olive presses, Thales of Miletus, The Chicago School, the scientific method, too big to fail, transaction costs, tulip mania, Works Progress Administration, yield curve

These assumptions allow us to calculate the expected return of the portfolio Rp by summing across all securities: E ( Rp ) = ∑ wi E (Ri ) i and the portfolio variance: s 2p = ∑ wi2 s i2 + ∑ ∑ wi w j si sj r ij i i j ≠i Notice that all of the coefficients on the right-hand side of the portfolio variance expression are necessarily positive, except for the correlation coefficient. We can readily see that portfolio variance is minimized if the correlation coefficient ij 1. We can generalize this risk minimization procedure through the matrix algebra for which Markowitz developed efficient solution algorithms that were more easily computable. This matrix algebra approach that minimizes variance for a given return R and wealth w becomes: min s 2 = min XVX T ∋ r = (W − X.1)rf + XR X X where the wealth constraint r = ( W − X.1) rf + XR affirms that wealth is invested in a risky portfolio R that returns R and a risk-free asset that returns rf.

…

Most significantly for financial pricing theory, he went on: [W]e reinterpret [the decision variables] to mean not future yields but parameters [e.g., moments and joint moments] of the jointfrequency distribution of future yields. Thus, x may be interpreted as the mathematical expectation of first year’s meat consumption, y may be its standard deviation, z may be the correlation coefficient between meat and salt consumption … etc. … It is sufficiently realistic, however, to confine ourselves, for each [return] to two parameters only: the mathematical expectation … and the coefficient of variation [“risk”].8 Marschak proposed a simple approach to the consideration of the interplay between return and risk by confining its description to first moments, known as means, and second moments of returns, labeled variances and covariances.

The Practice of Cloud System Administration: DevOps and SRE Practices for Web Services, Volume 2 by Thomas A. Limoncelli, Strata R. Chalup, Christina J. Hogan

active measures, Amazon Web Services, anti-pattern, barriers to entry, business process, cloud computing, commoditize, continuous integration, correlation coefficient, database schema, Debian, defense in depth, delayed gratification, DevOps, domain-specific language, en.wikipedia.org, fault tolerance, finite state, Firefox, functional programming, Google Glasses, information asymmetry, Infrastructure as a Service, intermodal, Internet of things, job automation, job satisfaction, Ken Thompson, Kickstarter, level 1 cache, load shedding, longitudinal study, loose coupling, machine readable, Malcom McLean invented shipping containers, Marc Andreessen, place-making, platform as a service, premature optimization, recommendation engine, revision control, risk tolerance, Salesforce, scientific management, seminal paper, side project, Silicon Valley, software as a service, sorting algorithm, standardized shipping container, statistical model, Steven Levy, supply-chain management, systems thinking, The future is already here, Toyota Production System, vertical integration, web application, Yogi Berra

Headroom is usually specified as a percentage of current capacity. • Timetable: For each component, what is the lead time from ordering to delivery, and from delivery until it is in service? Are there specific constraints for bringing new capacity into service, such as change windows? * * * Math Terms Correlation Coefficient: Describes how strongly measurements for different data sources resemble each other. Moving Average: A series of averages, each of which is taken across a short time interval (window), rather than across the whole data set. Regression Analysis: A statistical method for analyzing relationships between different data sources to determine how well they correlate, and to predict changes in one based on changes in another.

…

The number of data samples in that time period is n. If your core driver metric is x and your primary resource metric is y, you first calculate the sum of the last n values for x, x2, y, y2, and x times y, giving Σx, Σx2, Σy, Σy2, and Σxy. Then calculate SSxy, SSxx, SSyy, and R as follows: Regression analysis results in a correlation coefficient R, which is a number between –1 and 1. Squaring this number and then multiplying by 100 gives the percentage match between the two data sources. For example, for the MAU and network utilization figures shown in Figure 18.2, this calculation gives a very high correlation, between 96 percent and 100 percent, as shown in Figure 18.3, where R2 is graphed.

…

The large fluctuations in b for the length of the correlation window are due to significant changes in the moving averages from day to day, as the moving average has both pre- and post-upgrade data. When sufficient time has passed so that only post-upgrade data is used in the moving average, b becomes stable and the correlation coefficient returns to its previous high levels. The value of b corresponds to the slope of the line, or the multiplier in the equation linking the core driver and the usage of the primary resource. When correlation returns to normal, b is at a higher level. This result indicates that the primary resource will be consumed more rapidly with this software release than with the previous one.

Wall Street: How It Works And for Whom by Doug Henwood

accounting loophole / creative accounting, activist fund / activist shareholder / activist investor, affirmative action, Alan Greenspan, Andrei Shleifer, asset allocation, asset-backed security, bank run, banking crisis, barriers to entry, bond market vigilante , book value, borderless world, Bretton Woods, British Empire, business cycle, buy the rumour, sell the news, capital asset pricing model, capital controls, Carl Icahn, central bank independence, computerized trading, corporate governance, corporate raider, correlation coefficient, correlation does not imply causation, credit crunch, currency manipulation / currency intervention, currency risk, David Ricardo: comparative advantage, debt deflation, declining real wages, deindustrialization, dematerialisation, disinformation, diversification, diversified portfolio, Donald Trump, equity premium, Eugene Fama: efficient market hypothesis, experimental subject, facts on the ground, financial deregulation, financial engineering, financial innovation, Financial Instability Hypothesis, floating exchange rates, full employment, George Akerlof, George Gilder, Glass-Steagall Act, hiring and firing, Hyman Minsky, implied volatility, index arbitrage, index fund, information asymmetry, interest rate swap, Internet Archive, invisible hand, Irwin Jacobs, Isaac Newton, joint-stock company, Joseph Schumpeter, junk bonds, kremlinology, labor-force participation, late capitalism, law of one price, liberal capitalism, liquidationism / Banker’s doctrine / the Treasury view, London Interbank Offered Rate, long and variable lags, Louis Bachelier, low interest rates, market bubble, Mexican peso crisis / tequila crisis, Michael Milken, microcredit, minimum wage unemployment, money market fund, moral hazard, mortgage debt, mortgage tax deduction, Myron Scholes, oil shock, Paul Samuelson, payday loans, pension reform, planned obsolescence, plutocrats, Post-Keynesian economics, price mechanism, price stability, prisoner's dilemma, profit maximization, proprietary trading, publication bias, Ralph Nader, random walk, reserve currency, Richard Thaler, risk tolerance, Robert Gordon, Robert Shiller, Savings and loan crisis, selection bias, shareholder value, short selling, Slavoj Žižek, South Sea Bubble, stock buybacks, The inhabitant of London could order by telephone, sipping his morning tea in bed, the various products of the whole earth, The Market for Lemons, The Nature of the Firm, The Predators' Ball, The Wealth of Nations by Adam Smith, transaction costs, transcontinental railway, women in the workforce, yield curve, zero-coupon bond

Sometimes it doesn't matter whether the bad news is true; if the short can take a position and undertake a successful disinformation campaign, he or she can profitably cover the short. 35. For the "real" sector, however, borders still matter, and the "global assembly line" is a bit of an exaggeration. 36. The correlation coefficient is a measure of how tightly two sets of numbers are related to each other, ranging from -1 (a perfect mirror image) through 0 (no relation at all) to PLAYERS +1 (perfect lockstep). A correlation coefficient under 0.2 marks a fairly cacophanous relation, but figures over 0.9 signify great intimacy. 37. In fact, many foreign investments made in the U.S. during the 1980s have had apparently dismal rates of return.

…

Most of the sovereign defaulters, by the way, had good ratings from Moody's (Cantor and Packer 1995). But now those defaults are a distant memory, and today's capital markets look seamless. Statistics confirm the decreasing importance of borders for the financial markets."*^ In the 1970s, the correlation coefficient between interest rates on 10-year U.S. government bonds and German bonds of similar maturity was 0.191, but from 1990 to 1994, it was 0.934; Japan and the U.S., 0.182 and 0.965, respectively; and the U.S. and the U.K., 0.590 and 0.949 (Bank of England data, reported in Goldstein et al. 1994, p. 5).-^'' While it would be an exaggeration to say that there's now a single global credit market, we're definitely moving in that direction.

…

One doesn't want to get too carried away naturalizing temperament and values, but the model seems particularly to drive away women and nonwhites, at least in America, because of its chilly irreality. It may just be that sex and race are simply convenient markers for hierarchy —that economics is an ideology of privilege, and the already privileged, or those who wish to become apologists for the privileged, are drawn to its study. WALL STREET 8. Correlation coefficients for the various versions of q suggested in the text are all well over .92. The correlation for the simple equity q (market value of stock divided by tangible assets, as shown in the charts and used in the text) and the values for 1960-74 reported in Tobin and Brainard C1977) is .97. 9- It's interesting that investment rose during what are usually considered the bad years of the 1970s.

Trend Following: How Great Traders Make Millions in Up or Down Markets by Michael W. Covel

Albert Einstein, Alvin Toffler, Atul Gawande, backtesting, Bear Stearns, beat the dealer, Bernie Madoff, Black Swan, buy and hold, buy low sell high, California energy crisis, capital asset pricing model, Carl Icahn, Clayton Christensen, commodity trading advisor, computerized trading, correlation coefficient, Daniel Kahneman / Amos Tversky, delayed gratification, deliberate practice, diversification, diversified portfolio, Edward Thorp, Elliott wave, Emanuel Derman, Eugene Fama: efficient market hypothesis, Everything should be made as simple as possible, fiat currency, fixed income, Future Shock, game design, global macro, hindsight bias, housing crisis, index fund, Isaac Newton, Jim Simons, John Bogle, John Meriwether, John Nash: game theory, linear programming, Long Term Capital Management, managed futures, mandelbrot fractal, margin call, market bubble, market fundamentalism, market microstructure, Market Wizards by Jack D. Schwager, mental accounting, money market fund, Myron Scholes, Nash equilibrium, new economy, Nick Leeson, Ponzi scheme, prediction markets, random walk, Reminiscences of a Stock Operator, Renaissance Technologies, Richard Feynman, risk tolerance, risk-adjusted returns, risk/return, Robert Shiller, shareholder value, Sharpe ratio, short selling, South Sea Bubble, Stephen Hawking, survivorship bias, systematic trading, Teledyne, the scientific method, Thomas L Friedman, too big to fail, transaction costs, upwardly mobile, value at risk, Vanguard fund, William of Occam, zero-sum game

They traded with their gut and treated drawdowns as a cancer, rather than the natural ebb and flow of trading.” Interestingly, there is another perspective on drawdowns that few people consider. When you look at trend following performance data—for example, Dunn’s track record—you can’t help but notice that certain times are better than others to invest with Dunn. Correlation coefficient: A statistical measure of the interdependence of two or more random variables. Fundamentally, the value indicates how much of a change in one variable is explained by a change in another.25 Smart clients of Dunn look at his performance chart and buy in when his fund is experiencing a drawdown.

…

Unlike misguided comparisons, such as using standard deviation, I find correlation comparisons of performance data useful. In a research paper titled “Learning to Love Non-Correlation,” correlation is defined as “a statistical term giving the strength of linear relationship between two random variables. It is the historical tendency of one thing to move in tandem with another.” The correlation coefficient is a number from –1 to +1, with –1 being the perfectly opposite behavior of two investments (for example, up 5 percent every time the other is down 5 percent). The +1 reflects identical investment results (up or down the same amount each period). The further away from +1 one gets (and thus closer to –1), the better a diversifier one investment is for the other.

…

The further away from +1 one gets (and thus closer to –1), the better a diversifier one investment is for the other. But because his firm is keenly aware of keeping things simple, it also provides another description of correlation: the tendency for one investment to “zig” while another “zags.”27 I took the monthly performance numbers of trend followers and computed their correlation coefficients. Comparing correlations provided evidence that trend followers trade typically the same markets in the same way at the same time. Look at the correlation chart (see Chart 3.5) and ask yourself: “Why do two trend followers who don’t work in the same office, who are on opposite sides of the continent, have the same three losing months in a row with similar percentage losses?”

The Handbook of Personal Wealth Management by Reuvid, Jonathan.

asset allocation, banking crisis, BRICs, business cycle, buy and hold, carbon credits, collapse of Lehman Brothers, correlation coefficient, credit crunch, cross-subsidies, currency risk, diversification, diversified portfolio, estate planning, financial deregulation, fixed income, global macro, high net worth, income per capita, index fund, interest rate swap, laissez-faire capitalism, land tenure, low interest rates, managed futures, market bubble, merger arbitrage, negative equity, new economy, Northern Rock, pattern recognition, Ponzi scheme, prediction markets, proprietary trading, Right to Buy, risk tolerance, risk-adjusted returns, risk/return, short selling, side project, sovereign wealth fund, statistical arbitrage, systematic trading, transaction costs, yield curve

Forestry in the United States has been repeatedly shown to have a negative correlation coefficient with, among other financial assets, common stocks, corporate and government bonds, and the S&P 500 (see Table 2.3.1), and in certain studies reduced real portfolio risk by an average of 5 per cent.1 We observe that forestry generally forms a minor element of an overall investment portfolio, perhaps no more than 5–10 per cent as a maximum. Table 2.3.1 Timberland correlation coefficients, 1959–78 Investment correlation coefficient Timberland Residential housing Farm real estate S&P 500 index OTC stocks Preferred stock average No-load mutual fund average Municipal bonds Treasury Bills Long-term corporate bonds Commodity futures average 1.0000 –0.0905 0.5612 –0.4889 –0.4917 –0.3533 –0.6351 –0.0900 0.3118 –0.2704 0.8988 Source: Zinkhan, FC, Sizemore, WR, Mason, GH and Ebner, TJ (1992) Timberland Investments, Portland, Oregon, Timber Press. _______________________________________ CURRENT OPPORTUNITIES IN FORESTRY 81 ឣ Liquidity Certain forestry investments such as COEIC funds and exchange-traded funds (see below) are traded daily on markets such as the London Stock Exchange and Alternative Investment Market (AIM).

Facing Reality: Two Truths About Race in America by Charles Murray

2021 United States Capitol attack, 23andMe, affirmative action, Black Lives Matter, centre right, correlation coefficient, critical race theory, Donald Trump, feminist movement, gentrification, George Floyd, Gunnar Myrdal, income inequality, invention of agriculture, longitudinal study, low skilled workers, medical malpractice, meta-analysis, publication bias, school vouchers, Silicon Valley, The Bell Curve by Richard Herrnstein and Charles Murray, War on Poverty

You will find brilliant performers of every race in any occupation. That doesn’t negate the relevance of these considerations to group means. The magnitude of the relationship of cognitive ability to job performance varies. Magnitude in this case is usually expressed as the correlation coefficient between IQ and a measure of job performance. A correlation coefficient goes from –1 (a perfectly inverse relationship) to +1 (a perfectly direct relationship). The square of a correlation represents the percentage of the variance it “explains.” Rules of thumb are that the correlations between IQ scores and job productivity for low-complexity jobs are seldom below .2; for medium-complexity jobs, they are seldom below .4; for high-complexity jobs, they are seldom below .5.

The Drunkard's Walk: How Randomness Rules Our Lives by Leonard Mlodinow

Albert Einstein, Alfred Russel Wallace, Antoine Gombaud: Chevalier de Méré, Atul Gawande, behavioural economics, Brownian motion, butterfly effect, correlation coefficient, Daniel Kahneman / Amos Tversky, data science, Donald Trump, feminist movement, forensic accounting, Gary Kildall, Gerolamo Cardano, Henri Poincaré, index fund, Isaac Newton, law of one price, Monty Hall problem, pattern recognition, Paul Erdős, Pepto Bismol, probability theory / Blaise Pascal / Pierre de Fermat, RAND corporation, random walk, Richard Feynman, Ronald Reagan, Stephen Hawking, Steve Jobs, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Thomas Bayes, V2 rocket, Watson beat the top human players on Jeopardy!

For example, if data revealed that by eating the latest McDonald’s 1,000-calorie meal once a week, people gained 10 pounds a year and by eating it twice a week they gained 20 pounds, and so on, the correlation coefficient would be 1. If for some reason everyone were to instead lose those amounts of weight, the correlation coefficient would be -1. And if the weight gain and loss were all over the map and didn’t depend on meal consumption, the coefficient would be 0. Today correlation coefficients are among the most widely employed concepts in statistics. They are used to assess such relationships as those between the number of cigarettes smoked and the incidence of cancer, the distance of stars from Earth and the speed with which they are moving away from our planet, and the scores students achieve on standardized tests and the income of the students’ families.

Portfolio Design: A Modern Approach to Asset Allocation by R. Marston

asset allocation, Bob Litterman, book value, Bretton Woods, business cycle, capital asset pricing model, capital controls, carried interest, commodity trading advisor, correlation coefficient, currency risk, diversification, diversified portfolio, equity premium, equity risk premium, Eugene Fama: efficient market hypothesis, family office, financial engineering, financial innovation, fixed income, German hyperinflation, global macro, high net worth, hiring and firing, housing crisis, income per capita, index fund, inventory management, junk bonds, Long Term Capital Management, low interest rates, managed futures, mortgage debt, Nixon triggered the end of the Bretton Woods system, passive investing, purchasing power parity, risk free rate, risk-adjusted returns, Robert Shiller, Ronald Reagan, Sharpe ratio, Silicon Valley, stocks for the long run, superstar cities, survivorship bias, transaction costs, Vanguard fund

But for the last 10 years alone ending in 2009, the correlation between EAFE and the S&P rises to 0.87. There are correspondingly large increases in correlations between the S&P and the regional MSCI indexes. When did this increase in correlations occur? Consider Figure 5.9 which shows five- and 10-year correlation coefficients between the EAFE and S&P 500 indexes. Since the EAFE index starts only in 1970, the graph begins in 1975 for the five-year correlation and in 1980 for the 10-year correlation. The figure is noteworthy in several respects. First, the correlations vary widely over time whether they are measured over five- or ten-year intervals.

…

The Russell 1000 Growth Index is dominated by the Russell 1000 Value Index. This result should not be surprising given the analysis in Chapter 4. Russell 1000 Value has a higher return and a lower standard deviation than Russell 1000 Growth. What’s more, the two indexes are highly correlated with a correlation coefficient of 0.82. The optimizer finds that one series is totally dominated by the other. So the optimizer rejects one whole asset class. The optimizer is also not fond of small-cap stocks. The Russell 2000 Index has a small weighting in the lowest risk portfolios, and its role disappears in portfolios with larger allocations to stocks.

…

The overlap among the three largest databases was analyzed after eliminating the funds that only appeared in MSCI. The percentages were rounded to the nearest decimal. 12. As explained in Chapter 5, the underperformance of EAFE relative to U.S. stocks is almost entirely due to Japan. 13. Recall that beta is equal to the correlation coefficient times the ratio of the standard deviation of the asset relative to the standard deviation of the benchmark. The beta is 0.36 = 0.77 ∗ (0.071/0.152). 14. Since the average return on the risk-free Treasury bill is 3.8 percent and the average return on the Russell 3000 is 9.2 percent, the alpha = 11.8 percent – [3.8% + 0.36∗(9.2% – 3.8%)] = 6.1%. 15.

How Markets Fail: The Logic of Economic Calamities by John Cassidy

Abraham Wald, Alan Greenspan, Albert Einstein, An Inconvenient Truth, Andrei Shleifer, anti-communist, AOL-Time Warner, asset allocation, asset-backed security, availability heuristic, bank run, banking crisis, Bear Stearns, behavioural economics, Benoit Mandelbrot, Berlin Wall, Bernie Madoff, Black Monday: stock market crash in 1987, Black-Scholes formula, Blythe Masters, book value, Bretton Woods, British Empire, business cycle, capital asset pricing model, carbon tax, Carl Icahn, centralized clearinghouse, collateralized debt obligation, Columbine, conceptual framework, Corn Laws, corporate raider, correlation coefficient, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, crony capitalism, Daniel Kahneman / Amos Tversky, debt deflation, different worldview, diversification, Elliott wave, Eugene Fama: efficient market hypothesis, financial deregulation, financial engineering, financial innovation, Financial Instability Hypothesis, financial intermediation, full employment, Garrett Hardin, George Akerlof, Glass-Steagall Act, global supply chain, Gunnar Myrdal, Haight Ashbury, hiring and firing, Hyman Minsky, income per capita, incomplete markets, index fund, information asymmetry, Intergovernmental Panel on Climate Change (IPCC), invisible hand, John Nash: game theory, John von Neumann, Joseph Schumpeter, junk bonds, Kenneth Arrow, Kickstarter, laissez-faire capitalism, Landlord’s Game, liquidity trap, London Interbank Offered Rate, Long Term Capital Management, Louis Bachelier, low interest rates, mandelbrot fractal, margin call, market bubble, market clearing, mental accounting, Mikhail Gorbachev, military-industrial complex, Minsky moment, money market fund, Mont Pelerin Society, moral hazard, mortgage debt, Myron Scholes, Naomi Klein, negative equity, Network effects, Nick Leeson, Nixon triggered the end of the Bretton Woods system, Northern Rock, paradox of thrift, Pareto efficiency, Paul Samuelson, Phillips curve, Ponzi scheme, precautionary principle, price discrimination, price stability, principal–agent problem, profit maximization, proprietary trading, quantitative trading / quantitative finance, race to the bottom, Ralph Nader, RAND corporation, random walk, Renaissance Technologies, rent control, Richard Thaler, risk tolerance, risk-adjusted returns, road to serfdom, Robert Shiller, Robert Solow, Ronald Coase, Ronald Reagan, Savings and loan crisis, shareholder value, short selling, Silicon Valley, South Sea Bubble, sovereign wealth fund, statistical model, subprime mortgage crisis, tail risk, Tax Reform Act of 1986, technology bubble, The Chicago School, The Great Moderation, The Market for Lemons, The Wealth of Nations by Adam Smith, too big to fail, Tragedy of the Commons, transaction costs, Two Sigma, unorthodox policies, value at risk, Vanguard fund, Vilfredo Pareto, wealth creators, zero-sum game

As the big hedge fund Long-Term Capital Management discovered to its cost during the international financial crisis of 1998, many assets that seem to have little or nothing in common suddenly move in the same direction. Prior to the blowup, for example, the correlation coefficient between certain bonds issued by the governments of the Philippines and Bulgaria was just 0.04: as the crisis unfolded, their correlation coefficient rose to 0.84. (A correlation coefficient of zero means two assets have no relationship; a coefficient of one means they move in perfect unison.) During a period of market upheaval, as a Wall Street saying has it, “all correlations go to one.”

…

. : Philippe Jorion, Value at Risk: The New Benchmark for Managing Financial Risk, 2nd ed. (New York: McGraw-Hill, 2000), 107. 274 “In contrast with traditional . . .”: Ibid., xxii. 275 “It helps you understand . . .”: Quoted in Joe Nocera, “Risk Mismanagement,” New York Times Magazine, January 2, 2009. 277 the correlation coefficient . . . : Linda Allen, Jacob Boudoukh, and Anthony Saunders, Understanding Market, Credit, and Operational Risk: The Value at Risk Approach (Hoboken, N.J.: Wiley-Blackwell, 2004), 103. 278 “We remind our readers . . .”: “CreditMetrics Technical Document,” RiskMetrics, April 1997, available at www.riskmetrics.com/publications/techdocs/cmtdovv.html. 278 “The relative prevalence of . . .”: Allen et al., Understanding Market, 35. 278 “I believe that . . .”: “Against Value at Risk: Nassim Taleb Replies to Philippe Jorion,” 1997, available at www.fooledbyrandomness.com/jorion.html. 279 “business planning relied on . . .”: UBS, “Shareholder Report on UBS’s Write-Downs,” 34. 279 “even though delinquency . . .”: Ibid., 38–39. 280 “would overturn . . .”: Gillian Tett, Fool’s Gold: How the Bold Dream of a Small Tribe at J.P.

The Bogleheads' Guide to Investing by Taylor Larimore, Michael Leboeuf, Mel Lindauer

asset allocation, behavioural economics, book value, buy and hold, buy low sell high, corporate governance, correlation coefficient, Daniel Kahneman / Amos Tversky, diversification, diversified portfolio, Donald Trump, endowment effect, estate planning, financial engineering, financial independence, financial innovation, high net worth, index fund, John Bogle, junk bonds, late fees, Long Term Capital Management, loss aversion, Louis Bachelier, low interest rates, margin call, market bubble, mental accounting, money market fund, passive investing, Paul Samuelson, random walk, risk tolerance, risk/return, Sharpe ratio, statistical model, stocks for the long run, survivorship bias, the rule of 72, transaction costs, Vanguard fund, yield curve, zero-sum game

While some bond funds invest in highly rated investment-grade bonds, still others invest in lower-rated junk bonds. For more information on the various types of bonds, see Chapter 3. When investments (like stocks and bonds) don't always move together, they're said to have a low correlation coefficient. Understanding the correlation coefficient principal isn't really that difficult. The correlation numbers for any two investments can range from +1.0 (perfect correlation) to -1.0 (negative correlation) Basically, if two stocks (or funds) normally move together at the same rate, they're said to be highly correlated, and when two investments move in the opposite directions, they're said to be negatively correlated.

…

When two investments each randomly go their separate ways, independent of the movement of the other one, there is said to be no correlation between them, and their correlation figure would be shown as 0. Finally, when two investments always move in the opposite direction, they would have a negative correlation, which would be represented by a rating of -1.0. In actual practice, you'll find that most investment choices available to you will have a correlation coefficient somewhere between 1.0 (perfect correlation) and 0 (noncorrelated). It's very difficult to find negatively correlated asset classes that have similar expected returns. The closer the number is to 1.0, the higher the correlation between the two assets, and the lower the number, the less correlation there is between the two investments.

Thinking, Fast and Slow by Daniel Kahneman

Albert Einstein, Atul Gawande, availability heuristic, Bayesian statistics, behavioural economics, Black Swan, book value, Cass Sunstein, Checklist Manifesto, choice architecture, classic study, cognitive bias, cognitive load, complexity theory, correlation coefficient, correlation does not imply causation, Daniel Kahneman / Amos Tversky, delayed gratification, demand response, endowment effect, experimental economics, experimental subject, Exxon Valdez, feminist movement, framing effect, hedonic treadmill, hindsight bias, index card, information asymmetry, job satisfaction, John Bogle, John von Neumann, Kenneth Arrow, libertarian paternalism, Linda problem, loss aversion, medical residency, mental accounting, meta-analysis, nudge unit, pattern recognition, Paul Samuelson, peak-end rule, precautionary principle, pre–internet, price anchoring, quantitative trading / quantitative finance, random walk, Richard Thaler, risk tolerance, Robert Metcalfe, Ronald Reagan, Shai Danziger, sunk-cost fallacy, Supply of New York City Cabdrivers, systematic bias, TED Talk, The Chicago School, The Wisdom of Crowds, Thomas Bayes, transaction costs, union organizing, Walter Mischel, Yom Kippur War

It was a simple matter to rank the advisers by their performance in each year and to determine whether there were persistent differences in skill among them and whether the same advisers consistently achieved better returns for their clients year after year. To answer the question, I computed correlation coefficients between the rankings in each pair of years: year 1 with year 2, year 1 with year 3, and so on up through year 7 with year 8. That yielded 28 correlation coefficients, one for each pair of years. I knew the theory and was prepared to find weak evidence of persistence of skill. Still, I was surprised to find that the average of the 28 correlations was .01. In other words, zero.

…

If all you know about Tom is that he ranks twelfth in weight (well above average), you can infer (statistically) that he is probably older than average and also that he probably consumes more ice cream than other children. If all you know about Barbara is that she is eighty-fifth in piano (far below the average of the group), you can infer that she is likely to be young and that she is likely to practice less than most other children. The correlation coefficient between two measures, which varies between 0 and 1, is a measure of the relative weight of the factors they share. For example, we all share half our genes with each of our parents, and for traits in which environmental factors have relatively little influence, such as height, the correlation between parent and child is not far from .50.

…

In one study, the CEOs were characterized by the strategy of the companies they had led before their current appointment, as well as by management rules and procedures adopted after their appointment. CEOs do influence performance, but the effects are much smaller than a reading of the business press suggests. Researchers measure the strength of relationships by a correlation coefficient, which varies between 0 and 1. The coefficient was defined earlier (in relation to regression to the mean) by the extent to which two measures are determined by shared factors. A very generous estimate of the correlation between the success of the firm and the quality of its CEO might be as high as .30, indicating 30% overlap.

The Flat White Economy by Douglas McWilliams

access to a mobile phone, banking crisis, Big bang: deregulation of the City of London, bonus culture, Boris Johnson, Chuck Templeton: OpenTable:, clean tech, cloud computing, computer age, correlation coefficient, Crossrail, Edward Glaeser, en.wikipedia.org, Erik Brynjolfsson, eurozone crisis, George Gilder, hiring and firing, income inequality, informal economy, Kickstarter, knowledge economy, loadsamoney, low skilled workers, mass immigration, Metcalfe’s law, military-industrial complex, Network effects, new economy, offshore financial centre, Pareto efficiency, Peter Thiel, Productivity paradox, Robert Metcalfe, Robert Solow, Shenzhen special economic zone , Silicon Valley, smart cities, special economic zone, Steve Jobs, vertical integration, working-age population, zero-sum game

b) Growing Together: London and the UK Economy 2005 This report looks at a range of links between the London economy and those of the rest of the UK.18 It concludes that London’s growth is not at the expense of the rest of the UK, but that London and other UK regions and countries are interdependent. Table 6.1: Correlation between economic growth in London and the rest of the UK, 1983–2004 Regions and countries of Great Britain Correlation coefficient South East 0.80 East England 0.81 South West 0.64 East Midlands 0.45 West Midlands 0.73 North West 0.73 Yorkshire and Humberside 0.56 North East 0.22 Wales 0.55 Scotland 0.27 Northern Ireland 0.36 Table 6.2: Percentage change in employment, 1989–2001 Regions and countries of Great Britain % change South East 23.7 South West 21.2 East of England 18.8 Scotland 17.4 London 15.3 East Midlands 12.5 Wales 11.7 West Midlands 10.8 Yorkshire and Humberside 10.2 North West 9.9 North East 7.2 A unique feature of this analysis is research into the correlations between GVA growth in London and other regions and countries.

…

It is also interesting that the three emerging economies for which we have data in this sample – China, Mexico and South Korea – have much lower labour shares of income than in the advanced economies (which might be a bit of evidence to support Marx’s contention that capitalism in the long term would ultimately bid profits down to a level that is too low to permit economic growth). However, these emerging economies currently have much faster rates of economic growth. What is the evidence that higher profits boost economic growth? A crude statistical analysis for the OECD economies for which data is available shows a negative correlation coefficient of -0.31 between the labour income share average from 2000 to 2006 and the rate of economic growth from 2001 to 2008. What this says is that there is a statistically significant negative correlation between the labour income share and GDP growth. In other words, the higher the share of profits, the faster the rate of growth.

All About Asset Allocation, Second Edition by Richard Ferri

activist fund / activist shareholder / activist investor, Alan Greenspan, asset allocation, asset-backed security, barriers to entry, Bear Stearns, Bernie Madoff, Black Monday: stock market crash in 1987, book value, buy and hold, capital controls, commoditize, commodity trading advisor, correlation coefficient, currency risk, Daniel Kahneman / Amos Tversky, diversification, diversified portfolio, equity premium, equity risk premium, estate planning, financial independence, fixed income, full employment, high net worth, Home mortgage interest deduction, implied volatility, index fund, intangible asset, inverted yield curve, John Bogle, junk bonds, Long Term Capital Management, low interest rates, managed futures, Mason jar, money market fund, mortgage tax deduction, passive income, pattern recognition, random walk, Richard Thaler, risk free rate, risk tolerance, risk-adjusted returns, risk/return, Robert Shiller, selection bias, Sharpe ratio, stock buybacks, stocks for the long run, survivorship bias, too big to fail, transaction costs, Vanguard fund, yield curve

These relationships are dynamic, and they can and do change without warning. Selecting investments that do not go up and down at the same time (or most of the time) can be made easier with correlation analysis. This is a mathematical measure of the tendency of one investment to move in relation to another. The correlation coefficient is a mathematically derived number that measures this tendency toward comovement relative to the investments’ average return. If two investments each move in the same direction at the same time above their average returns, they have a positive correlation. If they each move in opposite directions below their average returns, they have a negative correlation.

…

These pairs of investments just do not exist. Correlation is measured using a range between ⫹1 and ⫺1. Two investments that have a correlation of ⫹0.3 or greater are considered positively correlated. When two investments have a correlation of ⫺0.3 or less, this is considered negative correlation. A correlation coefficient between ⫺0.3 and ⫹0.3 is considered noncorrelated. When two investments are noncorrelated, either the movement of one does not track the movement of the other or the tracking is inconsistent and shifts between positive and negative. Figure 3-4 represents two investments that are noncorrelated; sometimes they move together, and sometimes they do not.

…

Closed-End Fund A mutual fund that has a fixed number of shares, usually listed on a major stock exchange. Commodities Unprocessed goods, such as grains, metals, and minerals, traded in large amounts on a commodities exchange. Consumer Price Index (CPI) A measure of the price change in consumer goods and services. The CPI is used to track the pace of inflation. Correlation Coefficient A number between ⫺1 and 1 that measures the degree to which two variables are linearly related. Cost Basis The original cost of an investment. For tax purposes, the cost basis is subtracted from the sale price to determine any capital gain or loss. Glossary 323 Country Risk The possibility that political events (e.g., a war, national elections), financial problems (e.g., rising inflation, government default), or natural disasters (e.g., an earthquake, a poor harvest) will weaken a country’s economy and cause investments in that country to decline.

The Elements of Statistical Learning (Springer Series in Statistics) by Trevor Hastie, Robert Tibshirani, Jerome Friedman

algorithmic bias, backpropagation, Bayesian statistics, bioinformatics, computer age, conceptual framework, correlation coefficient, data science, G4S, Geoffrey Hinton, greed is good, higher-order functions, linear programming, p-value, pattern recognition, random walk, selection bias, sparse data, speech recognition, statistical model, stochastic process, The Wisdom of Crowds

This is the covariance matrix between these two variables, after linear adjustment for all the rest. In the Gaussian distribution, this is the covariance matrix of the conditional distribution of Xa |Xb . The partial correlation coefficient ρjk|rest between the pair Xa conditional on the rest Xb , is simply computed from this partial covariance. Define Θ = Σ−1 . 1. Show that Σa.b = Θ−1 aa . 2. Show that if any off-diagonal element of Θ is zero, then the partial correlation coefficient between the corresponding variables is zero. 3. Show that if we treat Θ as if it were a covariance matrix, and compute the corresponding “correlation” matrix R = diag(Θ)−1/2 · Θ · diag(Θ)−1/2 , then rjk = −ρjk|rest Ex. 17.4 Denote by f (X1 |X2 , X3 , . . . , Xp ) the conditional density of X1 given X2 , . . . , Xp .

…

Their proposal has the form (compare (3.69)) B̂c+w = B̂UΛU−1 , (3.72) where Λ is a diagonal shrinkage matrix (the “c+w” stands for “Curds and Whey,” the name they gave to their procedure). Based on optimal prediction in the population setting, they show that Λ has diagonal entries λm = c2m + c2m p N (1 − c2m ) , m = 1, . . . , M, (3.73) where cm is the mth canonical correlation coefficient. Note that as the ratio of the number of input variables to sample size p/N gets small, the shrinkage factors approach 1. Breiman and Friedman (1997) proposed modified versions of Λ based on training data and cross-validation, but the general form is the same. Here the fitted response has the form Ŷc+w = HYSc+w , (3.74) 86 3.

…

The shrinkage strategy (10.41) tends to eliminate the problem of overfitting, especially for larger data sets. The value of AAE after 800 iterations is 0.31. This can be compared to that of the optimal constant predictor median{yi } which is 0.89. In terms of more familiar quantities, the squared multiple correlation coefficient of this model is R2 = 0.84. Pace and Barry (1997) use a sophisticated spatial autoregression procedure, where prediction for each neighborhood is based on median house values in nearby neighborhoods, using the other predictors as covariates. Experimenting with transformations they achieved R2 = 0.85, predicting log Y .

A Mathematician Plays the Stock Market by John Allen Paulos

Alan Greenspan, AOL-Time Warner, Benoit Mandelbrot, Black-Scholes formula, book value, Brownian motion, business climate, business cycle, butter production in bangladesh, butterfly effect, capital asset pricing model, confounding variable, correlation coefficient, correlation does not imply causation, Daniel Kahneman / Amos Tversky, diversified portfolio, dogs of the Dow, Donald Trump, double entry bookkeeping, Elliott wave, endowment effect, equity risk premium, Erdős number, Eugene Fama: efficient market hypothesis, four colour theorem, George Gilder, global village, greed is good, index fund, intangible asset, invisible hand, Isaac Newton, it's over 9,000, John Bogle, John Nash: game theory, Larry Ellison, Long Term Capital Management, loss aversion, Louis Bachelier, mandelbrot fractal, margin call, mental accounting, Myron Scholes, Nash equilibrium, Network effects, passive investing, Paul Erdős, Paul Samuelson, Plato's cave, Ponzi scheme, power law, price anchoring, Ralph Nelson Elliott, random walk, Reminiscences of a Stock Operator, Richard Thaler, risk free rate, Robert Shiller, short selling, six sigma, Stephen Hawking, stocks for the long run, survivorship bias, transaction costs, two and twenty, ultimatum game, UUNET, Vanguard fund, Yogi Berra

Even a portfolio of stocks from the same sector will be less volatile than the individual stocks in it, while a portfolio consisting of Wal-Mart, Pfizer, General Electric, Exxon, and Citigroup, the biggest stocks in their respective sectors, will provide considerably more protection against volatility. To find the volatility of a portfolio in general, we need what is called the “covariance” (closely related to the correlation coefficient) between any pair of stocks X and Y in the portfolio. The covariance between two stocks is roughly the degree to which they vary together—the degree, that is, to which a change in one is proportional to a change in the other. Note that unlike many other contexts in which the distinction between covariance (or, more familiarly, correlation) and causation is underlined, the market generally doesn’t care much about it.

…

Brian auditors Aumann, Robert availability error average values compared with distribution of incomes risk as variance from averages average return compared with median return average value compared with distribution of incomes buy-sell rules and outguessing average guess risk as variance from average value averaging down Bachelier, Louis Bak, Per Barabasi, Albert-Lazló Bartiromo, Maria bear markets investor self-descriptions and shorting and distorting strategy in Benford, Frank Benford’s Law applying to corporate fraud background of frequent occurrence of numbers governed by Bernoulli, Daniel Beta (B) values causes of variations in comparing market against individual stocks or funds strengths and weaknesses of technique for finding volatility and Big Bang billiards, as example of nonlinear system binary system biorhythm theory Black, Fischer Black-Scholes option formula blackjack strategies Blackledge, Todd “blow up,” investor blue chip companies, P/E ratio of Bogle, John bonds Greenspan’s impact on bond market history of stocks outperforming will not necessarily continue to be outperformed by stocks Bonds, Barry bookkeeping. see accounting practices bottom-line investing Brock, William brokers. see stock brokers Buffett, Warren bull markets investor self-descriptions and pump and dump strategy in Butterfly Economics (Ormerod) “butterfly effect,” of nonlinear systems buy-sell rules buying on the margin. see also margin investments calendar effects call options. see also stock options covering how they work selling strategies valuation tools campaign contributions Capital Asset Pricing Model capital gains vs. dividends Central Limit Theorem CEOs arrogance of benefits in manipulating stock prices remuneration compared with that of average employee volatility due to malfeasance of chain letters Chaitin, Gregory chance. see also whim trading strategies and as undeniable factor in market chaos theory. see also nonlinear systems charity Clayman, Michelle cognitive illusions availability error confirmation bias heuristics rules of thumb for saving time mental accounts status quo bias Cohen, Abby Joseph coin flipping common knowledge accounting scandals and definition and importance to investors dynamic with private knowledge insider trading and parable illustrating private information becoming companies/corporations adjusting results to meet expectations applying Benford’s Law to corporate fraud comparing corporate and personal accounting financial health and P/E ratio of blue chips competition vs. cooperation, prisoner’s dilemma complexity changing over time horizon of sequences (mathematics) of trading strategies compound interest as basis of wealth doubling time and formulas for future value and present value and confirmation bias definition of investments reflecting stock-picking and connectedness. see also networks European market causing reaction on Wall Street interactions based on whim interactions between technical traders and value traders irrational interactions between traders Wolfram model of interactions between traders Consumer Confidence Index (CCI) contrarian investing dogs of the Dow measures of excellence and rate of return and cooperation vs. competition, prisoner’s dilemma correlation coefficient. see also statistical correlations counter-intuitive investment counterproductive behavior, psychology of covariance calculation of portfolio diversification based on portfolio volatility and stock selection and Cramer, James crowd following or not herd-like nature of price movements dart throwing, stock-picking contest in the Wall Street Journal data mining illustrated by online chatrooms moving averages and survivorship bias and trading strategies and DeBondt, Werner Deciding What’s News (Gans) decimalization reforms decision making minimizing regret selling WCOM depression of derivatives trading, Enron despair and guilt over market losses deviation from the mean. see also mean value covariance standard deviation (d) variance dice, probability and Digex discounting process, present value of future money distribution of incomes distribution of wealth dynamic of concentration UN report on diversified portfolios. see stock portfolios, diversifying dividends earnings and proposals benefitting returns from Dodd, David dogs of the Dow strategy “dominance” principle, game theory dot com IPOs, as a pyramid scheme double-bottom trend reversal “double-dip” recession double entry bookkeeping doubling time, compound interest and Dow dogs of the Dow strategy percentages of gains and losses e (exponential growth) compound interest and higher mathematics and earnings anchoring effect and complications with determination of inflating (WCOM) P/E ratio and stock valuation and East, Steven H.

Why Airplanes Crash: Aviation Safety in a Changing World by Clinton V. Oster, John S. Strong, C. Kurt Zorn

air traffic controllers' union, airline deregulation, airport security, correlation coefficient, flag carrier, operational security, Tenerife airport disaster, trickle-down economics

the risk that appears to be indicated by the incidents and the appropriate type of accident. Air traffic control operational errors, pilot deviations, and near midair collisions all appear to be indicators of risk of midair collision. Operational errors do not seem closely correlated with midair collisions. Terminal airspace operational errors have essentially no correlation (a correlation coefficient of -0.03) based on the eight years of available data. ARTCC operational errors have been influenced by the introduction of the snitch patch and the controllers' adjustment to it. In the four years of the post-snitch patch era, the correlation with midair collisions is only 0.20. Over the same period, operational errors are actually negatively correlated with the FAA's count of total near midair collisions (-0.84) and critical near midair collisions (-0.83).

…

Five years of data are simply too little upon which to base a conclusion. The Margin of Safety 119 Near midair collisions are also not strongly correlated with midair collisions. Indeed, near midair collisions as reported to the ASRS show no correlation (-0.08) over the seven years of available data. Midair collisions reported to the FAA have a correlation coefficient of only 0.51, although critical NMACs are somewhat higher at 0.76. The FAA correlation is based on only seven years of data. The poor correlation between potential midair collision incidents and accidents is disappointing to those seeking nonaccident leading indicators of aviation safety, but may not be surprising in light of the characteristics of the incident data.

Numpy Beginner's Guide - Third Edition by Ivan Idris

algorithmic trading, business intelligence, Conway's Game of Life, correlation coefficient, data science, Debian, discrete time, en.wikipedia.org, functional programming, general-purpose programming language, Khan Academy, p-value, random walk, reversible computing, time value of money

The correlaton coefcient takes values between -1 and 1 . The correlaton of a set of values with itself is 1 by defniton. This would be the ideal value; however, we will also be happy with a slightly lower value. Calculate the correlaton coefcient (or, more accurately, the correlaton matrix) with the corrcoef() functon: print("Correlation coefficient", np.corrcoef(bhp_returns, vale_ returns)) The coefcients are as follows: [[ 1. 0.67841747] [ 0.67841747 1. ]] The values on the diagonal are just the correlatons of the BHP and VALE with themselves and are, therefore, equal to 1. In all likelihood, no real calculaton takes place. The other two values are equal to each other since correlaton is symmetrical, meaning that the correlaton of BHP with VALE is equal to the correlaton of VALE with BHP.

…

For the source code, see the correlation.py fle in this book's code bundle: from __future__ import print_function import numpy as np import matplotlib.pyplot as plt bhp = np.loadtxt('BHP.csv', delimiter=',', usecols=(6,), unpack=True) bhp_returns = np.diff(bhp) / bhp[ : -1] vale = np.loadtxt('VALE.csv', delimiter=',', usecols=(6,), unpack=True) vale_returns = np.diff(vale) / vale[ : -1] covariance = np.cov(bhp_returns, vale_returns) print("Covariance", covariance) print("Covariance diagonal", covariance.diagonal()) print("Covariance trace", covariance.trace()) print(covariance/ (bhp_returns.std() * vale_returns.std())) print("Correlation coefficient", np.corrcoef(bhp_returns, vale_ returns)) difference = bhp - vale avg = np.mean(difference) dev = np.std(difference) print("Out of sync", np.abs(difference[-1] - avg) > 2 * dev) t = np.arange(len(bhp_returns)) plt.plot(t, bhp_returns, lw=1, label='BHP returns') plt.plot(t, vale_returns, '--', lw=2, label='VALE returns') plt.title('Correlating arrays') plt.xlabel('Days') plt.ylabel('Returns') plt.grid() plt.legend(loc='best') plt.show() Q1.

Quantitative Value: A Practitioner's Guide to Automating Intelligent Investment and Eliminating Behavioral Errors by Wesley R. Gray, Tobias E. Carlisle

activist fund / activist shareholder / activist investor, Alan Greenspan, Albert Einstein, Andrei Shleifer, asset allocation, Atul Gawande, backtesting, beat the dealer, Black Swan, book value, business cycle, butter production in bangladesh, buy and hold, capital asset pricing model, Checklist Manifesto, cognitive bias, compound rate of return, corporate governance, correlation coefficient, credit crunch, Daniel Kahneman / Amos Tversky, discounted cash flows, Edward Thorp, Eugene Fama: efficient market hypothesis, financial engineering, forensic accounting, Henry Singleton, hindsight bias, intangible asset, Jim Simons, Louis Bachelier, p-value, passive investing, performance metric, quantitative hedge fund, random walk, Richard Thaler, risk free rate, risk-adjusted returns, Robert Shiller, shareholder value, Sharpe ratio, short selling, statistical model, stock buybacks, survivorship bias, systematic trading, Teledyne, The Myth of the Rational Market, time value of money, transaction costs

Rolling 5-Year Wins Proportion of rolling 5-year periods that a designated strategy beats the identified benchmarks. Rolling 5-Year Wins Proportion of rolling 10-year periods that a designated strategy beats identified benchmarks. Cumulative Drawdown Sum of the rolling 5-year period worst drawdowns for the designated strategy. Correlation Correlation coefficient for a designated strategy and the identified benchmarks, which demonstrates the extent to which a designated strategy and the identified benchmarks move together. RISK AND RETURN Table 12.2 sets out the standard statistical analyses of the Quantitative Value strategy's performance and risk profile, comparing it to the Magic Formula, the Standard & Poor's (S&P) 500 and the MW Index, the market capitalization–weighted index of the universe from which we draw the stocks in the model portfolios.

…

Rolling 5-Year Wins Proportion of rolling 5-year periods that a designated strategy beats the identified benchmarks. Rolling 10-Year Wins Proportion of rolling 10-year periods that a designated strategy beats identified benchmarks. Cumulative Drawdown Sum of the rolling 5-year period worst drawdowns for the designated strategy. Correlation Correlation coefficient for a designated strategy and the identified benchmarks, which demonstrates the extent to which a designated strategy and the identified benchmarks move together. About the Authors Wesley R. Gray, PhD, is the founder and executive managing member of Empiritrage, LLC, an SEC-Registered Investment Advisor, and Turnkey Analyst, LLC, a firm dedicated to educating and sharing quantitative investment techniques to the general public.

Democratizing innovation by Eric von Hippel

additive manufacturing, correlation coefficient, Debian, disruptive innovation, Free Software Foundation, hacker house, informal economy, information asymmetry, inventory management, iterative process, James Watt: steam engine, knowledge economy, longitudinal study, machine readable, meta-analysis, Network effects, placebo effect, principal–agent problem, Richard Stallman, software patent, systematic bias, the Cathedral and the Bazaar, tragedy of the anticommons, transaction costs, vertical integration, Vickrey auction

Controlling for profit expectations, he found that increases in the stickiness of user information were associated with a significant increase in the amount of need-related design undertaken by the user (Kendall correlation coefficient = 0.5784, P < 0.01). Conversely he found that increased stickiness of technology-related information was associated in a significant reduction in the amount of technology design done by the user (Kendall correlation coefficients = 0.4789, P < 0.05). In other words, need-intensive tasks within product-development projects will tend to be done by users, while solutionintensive ones will tend to be done by manufacturers.

No One Would Listen: A True Financial Thriller by Harry Markopolos

Alan Greenspan, backtesting, barriers to entry, Bernie Madoff, buy and hold, call centre, centralized clearinghouse, correlation coefficient, diversified portfolio, Edward Thorp, Emanuel Derman, Eugene Fama: efficient market hypothesis, family office, financial engineering, financial thriller, fixed income, forensic accounting, high net worth, index card, Long Term Capital Management, Louis Bachelier, low interest rates, Market Wizards by Jack D. Schwager, offshore financial centre, payment for order flow, Ponzi scheme, price mechanism, proprietary trading, quantitative trading / quantitative finance, regulatory arbitrage, Renaissance Technologies, risk-adjusted returns, risk/return, rolodex, Sharpe ratio, statistical arbitrage, too big to fail, transaction costs, two and twenty, your tax dollars at work

We found out later that several hedge funds believed he was doing this. I created hypothetical baskets using the best-performing stocks and followed his split-strike strategy, selling the call option to generate income and buying the put option for protection. The following week I’d pick another basket. I expected the correlation coefficient—the relationship between Bernie’s returns and the movement of the entire S&P 100—legitimately to be around 50 percent, but it could have been anywhere between 30 percent and 80 percent and I would have accepted it naively. Instead Madoff was coming in at about 6 percent. Six percent! That was impossible.

…

Pointing to the 6 percent correlation and the 45-degree return line, he said, “That doesn’t look like it came from a finance distribution. We don’t have those kinds of charts in finance.” I was right, he agreed. Madoof’s strategy description claimed his returns were market-driven, yet his correlation coefficient was only 6 percent to the market and his performance line certainly wasn’t coming from the stock market. Volatility is a natural part of the market. It moves up and down—and does it every day. Any graphic representation of the market has to reflect that. Yet Madoff’s 45-degree rise represented a market without that volatility.

…

Broyhill meets Scott Franzblau and mobsters on Ponzi scheme on Ponzi scheme vs. front-running post Bernie Madoff arrest public acknowledgment of role and Rene-Thierry de la Villehuchet on reporting to SEC on reverse engineering role of sailing disaster on SEC failure Wall Street Journal warns individual investors Casey, Judy Cattle trading scam Charles, Prince Chelo, Neil: business education of careers at Rampart continued activities of early career of impact of Bernie Madoff case on information gathering leaves Rampart OPRA tapes on payment for order flow on Ponzi scheme vs. front-running post Bernie Madoff arrest public acknowledgment of role on quants on reporting to SEC reviews strategy analysis role of talks to Amit Vijayvergiya Wall Street Journal warns individual investors Cheung, Meaghan Chicago Art Museum Chicago Board of Options Exchange (CBOE) Chinese vitamin suppliers scandal Citigroup Client redemptions Clinton, Hillary Cohen, Steve Collars Commodities straddle Commodity Futures Trading Commission Congress: Chuck Schumer call to SEC Harry Markopolos testimony investigation by SEC established by SEC in hearings by Congressional Record Contacts and relationships Cook, Boyd Correlation coefficient Corruption: as business as usual drug cartels incompetence vs. municipal bonds organized crime regulatory reporting on Russian mafia vs. stupidity Wall Street crimes See also Taxpayers Against Fraud; whistleblowers Court, Andy Covered call writing program Cox, Christopher Criminal investigation CSPAN3 Cuomo, Andrew Danger, concerns about Darien Capital Management Data analysis Data collection DeBello, Nicole de la Villehuchet, Bertrand de la Villehuchet, Claudine de la Villehuchet, Rene-Thierry: on Bernie Madoff and Frank Casey and Harry Markopolos meets Frank Casey suicide Department of Justice Derivative experts Devoe, George diBartolomeo, Dan Dickens, Charles Direct accounts Discrepancies Documentation Documentation and literature Dominelli, David Donnelly, Joe Drosos, Elaine Drug cartels Due diligence Dumb equity Ebbers, Bernie Efficient markets hypothesis Electronic security Electronic trading European banks European investors Excuses for investing with Bernie Madoff Fairfield Emerald Fairfield Greenwich Group Fairfield Greenwich Sentry Fund False Claims Act cases Fax machines Federal Bureau of Investigation (FBI) Feeder funds Fielder, David Financial frauds.

The Global Minotaur by Yanis Varoufakis, Paul Mason

active measures, Alan Greenspan, AOL-Time Warner, banking crisis, Bear Stearns, Berlin Wall, Big bang: deregulation of the City of London, Bretton Woods, business climate, business cycle, capital controls, Carmen Reinhart, central bank independence, collapse of Lehman Brothers, collateralized debt obligation, colonial rule, corporate governance, correlation coefficient, creative destruction, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, debt deflation, declining real wages, deindustrialization, Easter island, endogenous growth, eurozone crisis, financial engineering, financial innovation, first-past-the-post, full employment, Glass-Steagall Act, Great Leap Forward, guns versus butter model, Hyman Minsky, industrial robot, Joseph Schumpeter, Kenneth Rogoff, Kickstarter, labour market flexibility, light touch regulation, liquidity trap, London Interbank Offered Rate, Long Term Capital Management, low interest rates, market fundamentalism, Mexican peso crisis / tequila crisis, military-industrial complex, Money creation, money market fund, mortgage debt, Myron Scholes, negative equity, new economy, Nixon triggered the end of the Bretton Woods system, Northern Rock, paper trading, Paul Samuelson, planetary scale, post-oil, price stability, quantitative easing, reserve currency, rising living standards, Ronald Reagan, special economic zone, Steve Jobs, structural adjustment programs, Suez crisis 1956, systematic trading, too big to fail, trickle-down economics, urban renewal, War on Poverty, WikiLeaks, Yom Kippur War

Theocarakis (2011) Modern Political Economics: Making sense of the post-2008 world, London and New York: Routledge. 11. In more technical language, the formulae used to assemble the CDOs assumed that the correlation coefficient between the probability of default across a CDO’s different tranches or slices was constant, small and knowable. 12. Doubt about the constancy of the correlation coefficient (see previous footnote) would have cost them their jobs, particularly as their supervisors did not really understand the formula but were receiving huge bonuses while it was being used. 13. See George Soros (2009) The Crash of 2008 and What It Means: The new paradigm for financial markets, New York: Public Affairs.

Beginning R: The Statistical Programming Language by Mark Gardener

correlation coefficient, distributed generation, natural language processing, New Urbanism, p-value, statistical model

The commands summarized in Table 6-3 enable you to carry out a range of correlation tasks. In the following sections you see a few of these options illustrated, and you can then try some correlations yourself in the activity that follows. Simple Correlation Simple correlations are between two continuous variables and you can use the cor() command to obtain a correlation coefficient like so: > count = c(9, 25, 15, 2, 14, 25, 24, 47) > speed = c(2, 3, 5, 9, 14, 24, 29, 34) > cor(count, speed) [1] 0.7237206 The default for R is to carry out the Pearson product moment, but you can specify other correlations using the method = instruction, like so: > cor(count, speed, method = 'spearman') [1] 0.5269556 This example used the Spearman rho correlation but you can also apply Kendall’s tau by specifying method = “kendall”.

…

You need to use attach() or with() commands to allow R to “read inside” the data frame and access the variables within. You could also use the $ syntax so that the command can access the variables as the following example shows: > cor(women$height, women$weight) [1] 0.9954948 In this example the cor() command has calculated the Pearson correlation coefficient between the height and weight variables contained in the women data frame. You can also use the cor() command directly on a data frame (or matrix). If you use the data frame women that you just looked at, for example, you get the following: > cor(women) height weight height 1.0000000 0.9954948 weight 0.9954948 1.0000000 Now you have a correlation matrix that shows you all combinations of the variables in the data frame.

…

In regression you are taking the analysis further and assuming a mathematical, and therefore predictable, relationship between the variables. The results of regression analysis show the slope and intercept values that describe this relationship. The R squared value that you obtain from the regression is the square of the correlation coefficient from the Pearson correlation, which demonstrates the similarities between the methods. The result shows you the coefficients for the regression, that is, the intercept and the slope. To see more details you should save your regression as a named object; then you can use the summary() command like so: > fw.lm = lm(count ~ speed, data = fw) > summary(fw.lm) Call: lm(formula = count ~ speed, data = fw) Residuals: Min 1Q Median 3Q Max -13.377 -5.801 -1.542 5.051 14.371 Coefficients: Estimate Std.

Upheaval: Turning Points for Nations in Crisis by Jared Diamond

anti-communist, Asian financial crisis, Berlin Wall, British Empire, California gold rush, carbon tax, clean water, correlation coefficient, cuban missile crisis, Dissolution of the Soviet Union, Gini coefficient, high-speed rail, illegal immigration, interchangeable parts, invention of writing, Jeff Bezos, low interest rates, medical malpractice, mutually assured destruction, Nelson Mandela, Nick Bostrom, nuclear winter, oil shale / tar sands, peak oil, post-work, purchasing power parity, rising living standards, risk tolerance, Ronald Reagan, Suez canal 1869, Suez crisis 1956, The Spirit Level, Timothy McVeigh, traffic fines, transcontinental railway, women in the workforce, World Values Survey

One method by which social scientists have tested this belief is to compare, among different countries, the correlation coefficients between incomes (or income ranks within people of their generation) of adults and the incomes of their parents. A correlation coefficient of 1.0 would mean that relative incomes of parents and of their adult children are perfectly correlated: all high-income people are children of high-income parents, all low-income people are children of low-income parents, kids from low-income families have zero chance of achieving high incomes, and socio-economic mobility is zero. At the opposite extreme, if the correlation coefficient were zero, it would mean that children of low-income parents have as good a chance of achieving high incomes as do children of high-income parents, and socio-economic mobility is high.

Unhealthy societies: the afflictions of inequality by Richard G. Wilkinson

attribution theory, business cycle, clean water, correlation coefficient, experimental subject, full employment, fundamental attribution error, Gini coefficient, income inequality, income per capita, Indoor air pollution, invisible hand, land reform, longitudinal study, means of production, purchasing power parity, rising living standards, twin studies, upwardly mobile

If increases in GNPpc over time were simply understated, it might be thought that this would not mask a statistical relationship between health and GNPpc: the extent to which societies benefited from qualitative changes in output would be a constant function of their growth rates. If this were so, then understated growth would change the units rather than weaken the correlation between the two. It would tend to make any given increase in income appear more health effective. In technical terms: rather than weakening the correlation coefficient it would increase the size of the regression coefficient. However, it could be argued that the spread of better products does not depend simply on the expenditure which results from the few per cent of income growth. Much nearer to the truth is that, as earlier forms of goods are made obsolete and replaced in the shops by new models and lines, the whole flow of expenditure is applied to the current range of goods, including new goods and ones in which the quality has changed.

…

Since then van Doorslaer et al. have reported that differences in self-reported illness were greatest in countries whose income differences were greatest (van Doorslaer Income distribution and health 89 et al. 1996). The relationship between measures of inequality in income and in illness was very close: across the USA and the eight European countries for which they had data, the correlation coefficient was 0.87. The methods used in each study were quite different. Kunst and Mackenbach classified people according to occupation in one of their studies and by education in another, and they concluded that occupational and educational differences in mortality were greater in countries where income differences were greater.

The Investopedia Guide to Wall Speak: The Terms You Need to Know to Talk Like Cramer, Think Like Soros, and Buy Like Buffett by Jack (edited By) Guinan

Albert Einstein, asset allocation, asset-backed security, book value, Brownian motion, business cycle, business process, buy and hold, capital asset pricing model, clean water, collateralized debt obligation, computerized markets, correlation coefficient, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, currency risk, discounted cash flows, diversification, diversified portfolio, dividend-yielding stocks, dogs of the Dow, equity premium, equity risk premium, fear index, financial engineering, fixed income, Glass-Steagall Act, implied volatility, index fund, intangible asset, interest rate swap, inventory management, inverted yield curve, junk bonds, London Interbank Offered Rate, low interest rates, margin call, money market fund, mortgage debt, Myron Scholes, passive investing, performance metric, risk free rate, risk tolerance, risk-adjusted returns, risk/return, shareholder value, Sharpe ratio, short selling, short squeeze, statistical model, time value of money, transaction costs, yield curve, zero-coupon bond

Related Terms: • Bond • Municipal Bond • Yield to maturity—YTM • Debt Financing • Yield Correlation What Does Correlation Mean? In the investment world, correlation is a statistical measure of how two securities move in relation to each other. Correlations are used in advanced portfolio management. Investopedia explains Correlation Correlation is expressed as the correlation coefficient, which ranges between –1 and +1. Perfect positive correlation (a correlation coefficient of +1) means that as one security moves up or down, the other security will move lockstep in the same direction. Perfect negative correlation means that when one security moves in one direction, the other security will move by an equal amount in the opposite direction.

Guide to business modelling by John Tennent, Graham Friend, Economist Group

book value, business cycle, correlation coefficient, discounted cash flows, double entry bookkeeping, G4S, Herman Kahn, intangible asset, iterative process, low interest rates, price elasticity of demand, purchasing power parity, RAND corporation, risk free rate, shareholder value, the market place, time value of money

The coefficient of determination R2 (which is calculated automatically by most spreadsheet packages) indicates how much of the variation in Y is explained by the explanatory variables. The greater the value of R2, the more the variation in the dependent variable is explained by the selected independent variables. The square root of the coefficient of determination is the product moment correlation coefficient in the case of linear regression of a straight line. The product moment correlation is a number between 1 and ⫺1. If r ⫽ 1 then there is a perfect, positive relationship between the dependent and explanatory variable. A perfect relation implies that every data point lies on a straight line.

…

Chart 10.14 Regression equation for monthly gross connections against time The R2 value is very low at 0.315. This implies that only 31.5% of the variation in gross connections is explained by time, so any forecast based on this regression equation will be liable to considerable error. The correlation coefficient is the square root of R2⫽SQR(0.315)⫽0.561. Although this value is low it is possible to show, using significance testing, that time is still a significant determinant of gross connections. Regression techniques 99 This procedure is quick and simple to use. However, to develop a forecast it is necessary to use the equation of the straight line.

Peer-to-Peer by Andy Oram

AltaVista, big-box store, c2.com, combinatorial explosion, commoditize, complexity theory, correlation coefficient, dark matter, Dennis Ritchie, fault tolerance, Free Software Foundation, Garrett Hardin, independent contractor, information retrieval, Kickstarter, Larry Wall, Marc Andreessen, moral hazard, Network effects, P = NP, P vs NP, p-value, packet switching, PalmPilot, peer-to-peer, peer-to-peer model, Ponzi scheme, power law, radical decentralization, rolodex, Ronald Coase, Search for Extraterrestrial Intelligence, semantic web, SETI@home, Silicon Valley, slashdot, statistical model, Tragedy of the Commons, UUNET, Vernor Vinge, web application, web of trust, Zimmermann PGP

We might declare that this category is 0.05-related to the first two. With some experience, an algorithm might be developed to tweak the correlation coefficients on the fly, based on how effective the current values have been at predicting the results of future transactions. Similarly, we might be able to reduce the discrete categories into a single continuous function that converts “distance” between file size and expiration date into a correlation coefficient. Reputation Technologies is not so lucky. Within a given exchange, buyers and sellers might barter thousands of different types of goods, each with different qualities and prices; the correlation between any pair of categories might be entirely unclear.

…

Separating reputations into categories can act as a defense against some of the subtle shilling attacks described above, such as when a vendor develops a good reputation at selling yo-yos and has a side business fraudulently selling used cars. The category idea raises very difficult questions. How do we pick categories? How do we know which categories are related to which other categories, and how related they are? Can this be automated somehow, or do the correlation coefficients have to be estimated manually? In the case of Free Haven, where there is only one real commodity—a document—and servers either behave or they don’t, it might be feasible to develop a set of categories manually and allow each server to manually configure the numbers that specify how closely related the categories are.

The Rise and Decline of Nations: Economic Growth, Stagflation, and Social Rigidities by Mancur Olson

barriers to entry, British Empire, business cycle, California gold rush, collective bargaining, correlation coefficient, David Ricardo: comparative advantage, full employment, income per capita, Kenneth Arrow, market clearing, Norman Macrae, Pareto efficiency, Phillips curve, price discrimination, profit maximization, rent-seeking, Robert Solow, Sam Peltzman, search costs, selection bias, Simon Kuznets, The Wealth of Nations by Adam Smith, trade liberalization, transaction costs, urban decay, working poor

Kwang Choi, "A Study of Comparative Rates of Economic Growth" (forthcoming, Iowa State University Press) and Kwang Choi, "A Statistical Test of the Political Economy of Comparative Growth Rates Model," in Mueller, The Political Economy of Growth. 34. Spearman rank correlation coefficients between years since statehood and LPI, PN, and per capita LP/, PN were respectively -.52, -.67, -.52, and -.52, and the correlation coefficients were in every case significant. 35. Farm organization membership need not be correlated with union membership, but farm groups focus almost exclusively on the farm policies of the federal government, and any losses in output due to them must fall mainly on consumers throughout the United States, rather than in the state in which the farmers are organized, so farm organization membership probably should not be included in tests on the forty-eight contiguous states.

Cryptoassets: The Innovative Investor's Guide to Bitcoin and Beyond: The Innovative Investor's Guide to Bitcoin and Beyond by Chris Burniske, Jack Tatar

Airbnb, Alan Greenspan, altcoin, Alvin Toffler, asset allocation, asset-backed security, autonomous vehicles, Bear Stearns, bitcoin, Bitcoin Ponzi scheme, blockchain, Blythe Masters, book value, business cycle, business process, buy and hold, capital controls, carbon tax, Carmen Reinhart, Clayton Christensen, clean water, cloud computing, collateralized debt obligation, commoditize, correlation coefficient, creative destruction, Credit Default Swap, credit default swaps / collateralized debt obligations, cryptocurrency, disintermediation, distributed ledger, diversification, diversified portfolio, Dogecoin, Donald Trump, Elon Musk, en.wikipedia.org, Ethereum, ethereum blockchain, fiat currency, financial engineering, financial innovation, fixed income, Future Shock, general purpose technology, George Gilder, Google Hangouts, high net worth, hype cycle, information security, initial coin offering, it's over 9,000, Jeff Bezos, Kenneth Rogoff, Kickstarter, Leonard Kleinrock, litecoin, low interest rates, Marc Andreessen, Mark Zuckerberg, market bubble, money market fund, money: store of value / unit of account / medium of exchange, moral hazard, Network effects, packet switching, passive investing, peer-to-peer, peer-to-peer lending, Peter Thiel, pets.com, Ponzi scheme, prediction markets, quantitative easing, quantum cryptography, RAND corporation, random walk, Renaissance Technologies, risk free rate, risk tolerance, risk-adjusted returns, Robert Shiller, Ross Ulbricht, Salesforce, Satoshi Nakamoto, seminal paper, Sharpe ratio, Silicon Valley, Simon Singh, Skype, smart contracts, social web, South Sea Bubble, Steve Jobs, transaction costs, tulip mania, Turing complete, two and twenty, Uber for X, Vanguard fund, Vitalik Buterin, WikiLeaks, Y2K

Cryptoassets have near-zero correlation to other capital market assets. The best explanation for this is that cryptoassets are so new that many capital market investors don’t play in the same asset pools. Therefore, cryptoassets aren’t dancing to the same rhythm of information as traditional capital market assets, at least not yet. Figure 7.19 The correlation coefficient and effects of diversification on risk Source: A Random Walk Down Wall Street, Burton G. Malkiel, 2015 Figure 7.19 clearly shows that if an asset is zero correlated to other assets in a portfolio, then “considerable risk reduction is possible.” In quantitative terms, reducing risk can be seen by a decrease in the volatility of the portfolio.

…

See Bitcoin Tracker One Cold storage, 221–222 Collaboration, 111 community and, 56 platforms for, 159 Collateralized mortgage obligations (CMOs), 4–5 Colored coins, 53 Commodities, 80, 172, 276–277 Commodities Futures Trading Commission (CFTC), 107, 112, 224, 276 Communication, 14 Communication Nets, xxiii Community, 57, 62 collaboration and, 56 of computers, 18 developers and, 182 Companies, 28, 63, 118 as incumbents, 264–273 interface services by, 113 OTC by, 216 as peer-to-peer, 13 perspective of, 249–250 risk and, 75 support and, 198–200 technology and, 264–265 value of, 152 venture capitalism for, 248 Competition, 16, 214 Compound annual growth rate (CAGR), 118–119 Compound annual returns, 87, 88, 103–104 Computer scientists, 60 Computers blockchain technology and, 26, 186 community of, 18 as miners, 16 for mining, 212 private keys on, 226 supercomputers as, 59 Consortium, 272–273 Consumable/Transformable (C/T) Assets, 109–110 Content, 174 Corbin, Abel, 164–165 Cornering, 163–166 cryptoassets and, 166–168 Correlation coefficient, 101 Correlation of returns, 74–76 Correlations, 122 assets and, 74 Bitcoin and, 133 cryptoassets and, 101–102 market behavior and, 132–135 Counterparty, 53–54 CPUs. See Central processing units Credit, 153 assets and, 143 issuers quality of, 239 Credulity, 141 The Crowd: A Study of the Popular Mind (Le Bon), 140 Crowd theory, 141 Crowdfunding, 60 Internet and, 250–254, 256 for investors, 250–252 for projects, 254 regulations and, 250 Crowds, 137–153 Crowdsale, 257 Cryptoassets.

Mindware: Tools for Smart Thinking by Richard E. Nisbett

affirmative action, Albert Einstein, availability heuristic, behavioural economics, big-box store, Cass Sunstein, choice architecture, cognitive dissonance, confounding variable, correlation coefficient, correlation does not imply causation, cosmological constant, Daniel Kahneman / Amos Tversky, dark matter, do well by doing good, Edward Jenner, endowment effect, experimental subject, feminist movement, fixed income, fundamental attribution error, Garrett Hardin, glass ceiling, Henri Poincaré, if you see hoof prints, think horses—not zebras, Intergovernmental Panel on Climate Change (IPCC), Isaac Newton, job satisfaction, Kickstarter, lake wobegon effect, libertarian paternalism, longitudinal study, loss aversion, low skilled workers, Menlo Park, meta-analysis, Neil Armstrong, quantitative easing, Richard Thaler, Ronald Reagan, selection bias, Shai Danziger, Socratic dialogue, Steve Jobs, Steven Levy, tacit knowledge, the scientific method, The Wealth of Nations by Adam Smith, Thomas Kuhn: the structure of scientific revolutions, Tragedy of the Commons, William of Occam, Yitang Zhang, Zipcar

A correlation of .8 corresponds to the degree of association you find between scores on the math portion of the Scholastic Aptitude Test (SAT) at one testing and scores on that test a year later—quite high but still plenty of room for difference between the two scores on average. Correlation Does Not Establish Causality Correlation coefficients are one step in assessing causal relations. If there is no correlation between variable A and variable B, there (probably) is no causal relation between A and B. (An exception would be when there is a third variable C that masks the correlation between A and B when there is in fact a causal relation between A and B.)

…

Coding Is the Key to Thinking Statistically I’m going to ask you some questions concerning your beliefs about what you think the correlation is between a number of pairs of variables. The way I’ll do that is to ask you how likely it is that A would be greater than B on one occasion given that A was greater than B on another occasion. Your answers in probability terms can be converted to correlation coefficients by a mathematical formula. Note that if you say “50 percent” for a question below, you’re saying that you think there’s no relationship between behavior on one occasion and behavior on another. If you say “90 percent,” you’re saying that there is an extremely strong relationship between behavior on one occasion and behavior on another.

Unweaving the Rainbow by Richard Dawkins

Any sufficiently advanced technology is indistinguishable from magic, Arthur Eddington, Boeing 747, complexity theory, correlation coefficient, David Attenborough, discovery of DNA, double helix, Douglas Engelbart, Douglas Engelbart, Eddington experiment, I think there is a world market for maybe five computers, Isaac Newton, Jaron Lanier, Mahatma Gandhi, music of the spheres, Necker cube, p-value, phenotype, Ralph Waldo Emerson, Richard Feynman, Ronald Reagan, Skinner box, Steven Pinker, Stuart Kauffman, world market for maybe five computers, Zipf's Law

Even if their verdicts are wrong, you'd think their methods would be systematic enough at least to agree in producing the same wrong verdicts! Alas, as shown in a study by G. Dean and colleagues, they don't even achieve this minimal and easy benchmark. For comparison, when different assessors judged people on their performance in structured interviews, the correlation coefficient was greater than 0.8 (a correlation coefficient of 1.0 would represent perfect agreement, –1.0 would represent perfect disagreement, 0.0 would represent complete randomness or lack of association; 0.8 is pretty good). Against this, in the same study, the reliability coefficient for astrology was a pitiable 0.1, comparable to the figure for palmistry (0.11), and indicating near total randomness.

The Big Fat Surprise: Why Butter, Meat and Cheese Belong in a Healthy Diet by Nina Teicholz

Albert Einstein, caloric restriction, caloric restriction, confounding variable, correlation coefficient, correlation does not imply causation, Gary Taubes, Indoor air pollution, meta-analysis, phenotype, placebo effect, randomized controlled trial, Robert Gordon, selection bias, TED Talk, the scientific method, Upton Sinclair

In 1999, when the Seven Countries study’s lead Italian researcher, Alessandro Menotti, went back twenty-five years later and looked at data from the study’s 12,770 subjects, he noticed an interesting fact: the category of foods that best correlated with coronary mortality was sweets. By “sweets,” he meant sugar products and pastries, which had a correlation coefficient with coronary mortality of 0.821 (a perfect correlation is 1.0). Possibly this number would have been higher had Menotti included chocolate, ice cream, and soft drinks in his “sweets” category, but those fell under a different category and, he explained, would have been “too troublesome” to recode. By contrast, “animal food” (butter, meat, eggs, margarine, lard, milk, and cheese) had a correlation coefficient of 0.798, and this number likely would have been lower had Menotti excluded margarine.

…

Public Health Nutrition 8, no. 6 (2005): 666. “we should not” . . . “the ideal thing all the time”: Daan Kromhout, interview with author, October 4, 2007. he knew it would go unnoticed: Keys, Aravanis, and Sdrin, “Diets of Middle-Aged Men in Two Rural Areas of Greece,” 577. category of foods . . . which had a correlation coefficient: Alessandro Menotti et al., “Food Intake Patterns and 25-Year Mortality from Coronary Heart Disease: Cross-Cultural Correlations in the Seven Countries Study,” European Journal of Epidemiology 15, no. 6 (1999): 507–515. “too troublesome” to recode: Alessandro Menotti, interview with author, July 24, 2008.

Rationality: What It Is, Why It Seems Scarce, Why It Matters by Steven Pinker

affirmative action, Albert Einstein, autonomous vehicles, availability heuristic, Ayatollah Khomeini, backpropagation, basic income, behavioural economics, belling the cat, Black Lives Matter, butterfly effect, carbon tax, Cass Sunstein, choice architecture, classic study, clean water, Comet Ping Pong, coronavirus, correlation coefficient, correlation does not imply causation, COVID-19, critical race theory, crowdsourcing, cuban missile crisis, Daniel Kahneman / Amos Tversky, data science, David Attenborough, deep learning, defund the police, delayed gratification, disinformation, Donald Trump, Dr. Strangelove, Easter island, effective altruism, en.wikipedia.org, Erdős number, Estimating the Reproducibility of Psychological Science, fake news, feminist movement, framing effect, George Akerlof, George Floyd, germ theory of disease, high batting average, if you see hoof prints, think horses—not zebras, index card, Jeff Bezos, job automation, John Nash: game theory, John von Neumann, libertarian paternalism, Linda problem, longitudinal study, loss aversion, Mahatma Gandhi, meta-analysis, microaggression, Monty Hall problem, Nash equilibrium, New Journalism, Paul Erdős, Paul Samuelson, Peter Singer: altruism, Pierre-Simon Laplace, placebo effect, post-truth, power law, QAnon, QWERTY keyboard, Ralph Waldo Emerson, randomized controlled trial, replication crisis, Richard Thaler, scientific worldview, selection bias, social discount rate, social distancing, Social Justice Warrior, Stanford marshmallow experiment, Steve Bannon, Steven Pinker, sunk-cost fallacy, TED Talk, the scientific method, Thomas Bayes, Tragedy of the Commons, trolley problem, twin studies, universal basic income, Upton Sinclair, urban planning, Walter Mischel, yellow journalism, zero-sum game

The residuals also allow us to quantify how correlated the two variables are: the shorter the bands, as a proportion of how splayed out the entire cluster is left to right and up and down, the closer the dots are to the line, and the higher the correlation. With a bit of algebra this can be converted into a number, r, the correlation coefficient, which ranges from –1 (not shown), where the dots fall in lockstep along a diagonal from northwest to southeast; through a range of negative values where they splatter diagonally along that axis; through 0, when they are an uncorrelated swarm of gnats; through positive values where they splatter southwest to northeast; to 1, where they lie perfectly along the diagonal.

…

See moral progress —recommendations to strengthen accountability for lying and disinformation, 313, 314, 316–17 avoidance of sectarian symbolism, 312 educational curricula, 314–15 evidence-based evaluation, 312, 317 incentive structures, 315–17 in journalism, 314, 316, 317 norms valorizing, 311–13, 315 in punditry, 317 “Republican party of stupid,” 312–13, 357n73 scientists in legislatures, 312 in social media, 313, 316–17 viewpoint diversity in higher ed, 313–14 Rationality Community, 149–50, 312 Rationality Quotient, 311 Rawls, John, 69 realism, universal, 300–301 reality, motivating rationality, 41–42, 288, 298, 309, 320 reality mindset Active Open-Mindedness/Openness to Evidence and, 310–11, 324, 356–57n67 definition, 299–300 mythology mindset, border with, 303 as unnatural, 300–301 reason, in definition of “rationality,” 36 Rebel Without a Cause, 59, 236, 344n29 reciprocity, norms of, 5 recursion, 71, 108 reflectiveness Cognitive Reflection Test, 8–11, 50 definition, 311 intelligence correlating with, 311 openness to evidence correlating with, 311 and reasoning competence, 323, 324 resistance to cognitive illusions and, 311 unreflective thinking, 8–10, 311 of weird beliefs, 299 regression correlation coefficient (r), 250–51 definition, 248, 252 equation for, 272, 278–81 general linear model, 272 human vs., accuracy of, 278–80 instrumental variable regression, 267–68 multiple regression, 270–72, 271 regression line, 248–49, 253–54, 270 residuals, 249–50, 270–71 regression discontinuity, 266–67 regression line, 248–49, 253–54, 270 regression to the mean and bell curve distribution, 253 definition, 252–53 imperfect correlation as producing, 254 regression line, 253–54 scatterplots and, 253–54 as statistical phenomenon, 253 unawareness of, 254–56, 320, 353n13 regret avoidance, 17, 190 relativism and relativists argument against rationality, 39–40 as hypocritical, 42 morality and, 42, 66–67 religion argument from authority, 90 the cluster illusion and, 147 forbidden base rates and, 163–66 the Golden Rule in, 68–69 heretical counterfactuals, 64–65 monotheism, 40 the mythology mindset and, 301–2, 307 persecution of, progress against, 330–31 See also God Rendezvous game, 233–34 replicability crisis in science Bayesian reasoning failures and, 159–61 preregistration as remedy, 145–46 questionable research practices and, 145–46, 160, 353n13 science journalism and, 161–62 statistical significance and, 225 Texas sharpshooter fallacy, 144–46, 160 Winner’s Curse, 256 representativeness heuristic, 27, 155–56 Republican Party and Republicans calling Democrats socialists, 83–84 expressive rationality and, 298 Fox News and, 267, 268, 296 pizza classified as vegetable by, 101 politically motivated numeracy, 292–94 rehabilitating, 312–13, 357n73 See also left and right (political); politics reputation, 47, 237, 242, 308, 313 resistance to evidence.

Debunking Economics - Revised, Expanded and Integrated Edition: The Naked Emperor Dethroned? by Steve Keen

accounting loophole / creative accounting, Alan Greenspan, banking crisis, banks create money, barriers to entry, behavioural economics, Benoit Mandelbrot, Big bang: deregulation of the City of London, Black Swan, Bonfire of the Vanities, book value, business cycle, butterfly effect, capital asset pricing model, cellular automata, central bank independence, citizen journalism, clockwork universe, collective bargaining, complexity theory, correlation coefficient, creative destruction, credit crunch, David Ricardo: comparative advantage, debt deflation, diversification, double entry bookkeeping, en.wikipedia.org, equity risk premium, Eugene Fama: efficient market hypothesis, experimental subject, Financial Instability Hypothesis, fixed income, Fractional reserve banking, full employment, Glass-Steagall Act, Greenspan put, Henri Poincaré, housing crisis, Hyman Minsky, income inequality, information asymmetry, invisible hand, iterative process, John von Neumann, Kickstarter, laissez-faire capitalism, liquidity trap, Long Term Capital Management, low interest rates, mandelbrot fractal, margin call, market bubble, market clearing, market microstructure, means of production, minimum wage unemployment, Money creation, money market fund, open economy, Pareto efficiency, Paul Samuelson, Phillips curve, place-making, Ponzi scheme, Post-Keynesian economics, power law, profit maximization, quantitative easing, RAND corporation, random walk, risk free rate, risk tolerance, risk/return, Robert Shiller, Robert Solow, Ronald Coase, Savings and loan crisis, Schrödinger's Cat, scientific mainstream, seigniorage, six sigma, South Sea Bubble, stochastic process, The Great Moderation, The Wealth of Nations by Adam Smith, Thorstein Veblen, time value of money, total factor productivity, tulip mania, wage slave, zero-sum game

Though the huge fiscal and monetary stimulus packages also played a role, changes in debt-financed demand dominate economic performance. One statistical indicator of the importance of debt dynamics in causing both the Great Depression and the Great Recession and the booms that preceded them is the correlation coefficient between changes in debt and the level of unemployment. Over the whole period from 1921 till 1940, the correlation coefficient was minus 0.83, while over the period from 1990 till 2011, it was minus 0.91 (versus the maximum value it could have taken of minus one). A correlation of that scale, over time periods of that length, when economic circumstances varied from bust to boom and back again, is staggering. 13.31 Debt-financed demand and unemployment, 1990–2011 The Credit Impulse confirms the dominant role of private debt.

…

In Sharpe’s words: In order to derive conditions for equilibrium in the capital market we invoke two assumptions. First, we assume a common pure rate of interest, with all investors able to borrow or lend funds on equal terms. Second, we assume homogeneity of investor expectations: investors are assumed to agree on the prospects of various investments – the expected values, standard deviations and correlation coefficients described in Part II. Needless to say, these are highly restrictive and undoubtedly unrealistic assumptions. However, since the proper test of a theory is not the realism of its assumptions but the acceptability of its implications, and since these assumptions imply equilibrium conditions which form a major part of classical financial doctrine, it is far from clear that this formulation should be rejected – especially in view of the dearth of alternative models leading to similar results.

The Age of Turbulence: Adventures in a New World (Hardback) - Common by Alan Greenspan

addicted to oil, air freight, airline deregulation, Alan Greenspan, Albert Einstein, asset-backed security, bank run, Berlin Wall, Black Monday: stock market crash in 1987, Bretton Woods, business cycle, business process, buy and hold, call centre, capital controls, carbon tax, central bank independence, collateralized debt obligation, collective bargaining, compensation consultant, conceptual framework, Corn Laws, corporate governance, corporate raider, correlation coefficient, cotton gin, creative destruction, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, crony capitalism, cuban missile crisis, currency peg, currency risk, Deng Xiaoping, Dissolution of the Soviet Union, Doha Development Round, double entry bookkeeping, equity premium, everywhere but in the productivity statistics, Fall of the Berlin Wall, fiat currency, financial innovation, financial intermediation, full employment, Gini coefficient, Glass-Steagall Act, Hernando de Soto, income inequality, income per capita, information security, invisible hand, Joseph Schumpeter, junk bonds, labor-force participation, laissez-faire capitalism, land reform, Long Term Capital Management, low interest rates, Mahatma Gandhi, manufacturing employment, market bubble, means of production, Mikhail Gorbachev, moral hazard, mortgage debt, Myron Scholes, Nelson Mandela, new economy, North Sea oil, oil shock, open economy, open immigration, Pearl River Delta, pets.com, Potemkin village, price mechanism, price stability, Productivity paradox, profit maximization, purchasing power parity, random walk, Reminiscences of a Stock Operator, reserve currency, Right to Buy, risk tolerance, Robert Solow, Ronald Reagan, Savings and loan crisis, shareholder value, short selling, Silicon Valley, special economic zone, stock buybacks, stocks for the long run, Suez crisis 1956, the payments system, The Theory of the Leisure Class by Thorstein Veblen, The Wealth of Nations by Adam Smith, Thorstein Veblen, Tipper Gore, too big to fail, total factor productivity, trade liberalization, trade route, transaction costs, transcontinental railway, urban renewal, We are all Keynesians now, working-age population, Y2K, zero-sum game

THE M O D E S OF C A P I T A L I S M at the same time, Germany ranks among the highest in terms of the freedom of its people to open and close businesses, property-rights protection, and the overall rule of law. France (number forty-five) and Italy (number sixty) have profiles that are similarly mixed. The ultimate test of the usefulness of such a scoring process is whether it correlates with economic performance. And it does. The correlation coefficient of 157 countries between their "Economic Freedom Score" and the log of their per capita incomes is 0.65, impressive for such a motley body of data.* Thus, we are left with a critical question: Granted that open competitive markets foster economic growth, is there an optimum trade-off between economic performance and the competitive stress it imposes on the one hand, and the civility that, for example, the continental Europeans and many others espouse?

…

Accordingly the weighted correlation between national saving rates and domestic investment rates for countries or regions representing virtually all of the world's gross domestic product, a measure of the degree of home bias, declined from a coefficient of around 0.95 in 1992, where it had hovered since 1970, to an estimated 0.74 in 2005. (If in every country saving equaled investment—that is, if there were 100 percent home bias—the correlation coefficient would be 1.0. On the other hand, if there were no home bias, and the amount of domestic saving bore no relationship to the amount and location of investments, the coefficient would be 0.)* Only in the past decade has expanding trade been associated with the emergence of ever-larger U.S. trade and current account deficits, matched by a corresponding widening of the aggregate external surpluses of many of our trading partners, most recently including China.

…

Although foreign investors *The persistent divergence subsequent to t h e creation of t h e euro of m a n y prices of identical goods a m o n g m e m b e r countries of t h e euro area is analyzed in John H. Rogers (2002). For t h e case of U.S. and Canadian prices, see Charles Engel and John H. Rogers ( 1 9 9 6 ) . t T h e correlation coefficient measures of h o m e bias have flattened o u t since 2 0 0 0 . So have t h e measures of dispersion. This is consistent w i t h t h e United States' accounting for a rising share of deficits. 361 More ebooks visit: http://www.ccebook.cn ccebook-orginal english ebooks This file was collected by ccebook.cn form the internet, the author keeps the copyright.

Mastering Machine Learning With Scikit-Learn by Gavin Hackeling

backpropagation, computer vision, constrained optimization, correlation coefficient, data science, Debian, deep learning, distributed generation, iterative process, natural language processing, Occam's razor, optical character recognition, performance metric, recommendation engine

An r-squared score of one indicates that the response variable can be predicted without any error using the model. An r-squared score of one half indicates that half of the variance in the response variable can be predicted using the model. There are several methods to calculate r-squared. In the case of simple linear regression, r-squared is equal to the square of the Pearson product moment correlation coefficient, or Pearson's r. [ 29 ] www.it-ebooks.info Linear Regression Using this method, r-squared must be a positive number between zero and one. This method is intuitive; if r-squared describes the proportion of variance in the response variable explained by the model, it cannot be greater than one or less than zero.

Be Your Own Financial Adviser: The Comprehensive Guide to Wealth and Financial Planning by Jonquil Lowe

AltaVista, asset allocation, banking crisis, BRICs, buy and hold, correlation coefficient, cross-subsidies, diversification, diversified portfolio, estate planning, fixed income, high net worth, money market fund, mortgage debt, mortgage tax deduction, negative equity, offshore financial centre, Own Your Own Home, passive investing, place-making, Right to Buy, risk/return, short selling, zero-coupon bond

For example, equities and property tend to do well in times of inflation, unlike cash and bonds. But cash and bonds may produce solid returns in an economic downturn, while property and equities tend to suffer. The extent to which different investments or assets are correlated can be measured and represented by a statistic called a ‘correlation coefficient’. A coefficient of 1 would mean that two asset classes moved in exactly the same way (so there would not be any point combining the assets). A coefficient of zero would mean the asset classes were completely uncorrelated. Most coefficients lie between these two extremes. A negative coefficient means that positive performance for one asset class is associated with negative performance from the other.

…

A negative coefficient means that positive performance for one asset class is associated with negative performance from the other. (A coefficient of –1 would mean the assets were perfectly negatively correlated with the risk of losses on one asset being completely offset by the chance of gains on the other, so eliminating risk altogether for a portfolio made up of these two assets.) Table 10.1 on p. 301 shows the correlation coefficient for different pairs of asset class using data for the period from 1997 to 2009. The ideal for investors is to combine asset classes that have low or negative correlations, say, less than 0.5. The global financial crisis that hit the world in 2007 demonstrated that, in some situations, the relative independence of the asset classes can break down, with all delivering poor performance simultaneously.

Trend Commandments: Trading for Exceptional Returns by Michael W. Covel

Alan Greenspan, Albert Einstein, Alvin Toffler, behavioural economics, Bernie Madoff, Black Swan, business cycle, buy and hold, commodity trading advisor, correlation coefficient, delayed gratification, disinformation, diversified portfolio, en.wikipedia.org, Eugene Fama: efficient market hypothesis, family office, full employment, global macro, Jim Simons, Lao Tzu, Long Term Capital Management, managed futures, market bubble, market microstructure, Market Wizards by Jack D. Schwager, Mikhail Gorbachev, moral hazard, Myron Scholes, Nick Leeson, oil shock, Ponzi scheme, prediction markets, quantitative trading / quantitative finance, random walk, Reminiscences of a Stock Operator, Sharpe ratio, systematic trading, the scientific method, three-martini lunch, transaction costs, tulip mania, upwardly mobile, Y2K, zero-sum game

Consider this nearly 40-year track record of trend trading wealth building: Chart 1: Bill Dunn Unit Value Log Scale DUNN Composite Performance: $500,000 $570,490 10 Drawdowns Greater Than -25% October 1974 through January 2011 Average Major Drawdown: 38% $100,000 -40% Compound Annual Rate of Return -27% -60% -34% $62,375 DUNN Composite: 19.09% S&P 500 (Total return) : 11.85% -29% -34% -45% -35% -51% -43% $10,000 -30% Correlation Coefficient -0.05 Past Performance is Not Necessarily Indicative of Future Results -28% -45% Includes Notional and Proprietary Funds All Net of Pro Forma Fees and Expenses -52% ' 70 ' 71 ' 72 ' 73 ' 74 ' 75 ' 76 ' 77 ' 78 ' 79 ' 80 ' 81 ' 82 ' 83 ' 84 ' 85 ' 86 ' 87 ' 88 ' 89 ' 90 ' 91 ' 92 ' 93 ' 94 ' 95 ' 96 ' 97 ' 98 ' 99 ' 00 ' 01 ' 02 ' 03 ' 04 ' 05 ' 06 ' 07 ' 08 ' 09 ' 10 ' 11 $1,000 That picture is worth a thousand words.

Keeping Up With the Quants: Your Guide to Understanding and Using Analytics by Thomas H. Davenport, Jinho Kim

behavioural economics, Black-Scholes formula, business intelligence, business process, call centre, computer age, correlation coefficient, correlation does not imply causation, Credit Default Swap, data science, en.wikipedia.org, feminist movement, Florence Nightingale: pie chart, forensic accounting, global supply chain, Gregor Mendel, Hans Rosling, hypertext link, invention of the telescope, inventory management, Jeff Bezos, Johannes Kepler, longitudinal study, margin call, Moneyball by Michael Lewis explains big data, Myron Scholes, Netflix Prize, p-value, performance metric, publish or perish, quantitative hedge fund, random walk, Renaissance Technologies, Robert Shiller, self-driving car, sentiment analysis, six sigma, Skype, statistical model, supply-chain management, TED Talk, text mining, the scientific method, Thomas Davenport

., records in a database) into groups (called clusters) so that objects within clusters are similar in some manner while objects across clusters are dissimilar to each other. Clustering is a main task of exploratory data mining, and a common technique for statistical data analysis used in many fields. Correlation: The extent to which two or more variables are related to one another. The degree of relatedness is expressed as a correlation coefficient, which ranges from −1.0 to +1.0. Correlation = +1 (Perfect positive correlation, meaning that both variables always move in the same direction together) Correlation = 0 (No relationship between the variables) Correlation = −1 (Perfect negative correlation, meaning that as one variable goes up, the other always trends downward) Correlation does not imply causation.

Capital in the Twenty-First Century by Thomas Piketty

accounting loophole / creative accounting, Asian financial crisis, banking crisis, banks create money, Berlin Wall, book value, Branko Milanovic, British Empire, business cycle, capital controls, Capital in the Twenty-First Century by Thomas Piketty, carbon footprint, carbon tax, central bank independence, centre right, circulation of elites, collapse of Lehman Brothers, conceptual framework, corporate governance, correlation coefficient, David Ricardo: comparative advantage, demographic transition, distributed generation, diversification, diversified portfolio, European colonialism, eurozone crisis, Fall of the Berlin Wall, financial intermediation, full employment, Future Shock, German hyperinflation, Gini coefficient, Great Leap Forward, high net worth, Honoré de Balzac, immigration reform, income inequality, income per capita, index card, inflation targeting, informal economy, invention of the steam engine, invisible hand, joint-stock company, Joseph Schumpeter, Kenneth Arrow, low interest rates, market bubble, means of production, meritocracy, Money creation, mortgage debt, mortgage tax deduction, new economy, New Urbanism, offshore financial centre, open economy, Paul Samuelson, pension reform, power law, purchasing power parity, race to the bottom, randomized controlled trial, refrigerator car, regulatory arbitrage, rent control, rent-seeking, Robert Gordon, Robert Solow, Ronald Reagan, Simon Kuznets, sovereign wealth fund, Steve Jobs, Suez canal 1869, Suez crisis 1956, The Nature of the Firm, the payments system, The Theory of the Leisure Class by Thorstein Veblen, The Wealth of Nations by Adam Smith, Thomas Malthus, Thorstein Veblen, trade liberalization, twin studies, very high income, Vilfredo Pareto, We are the 99%, zero-sum game

But this is a different issue from skill and earned income mobility, which is what is of interest here and is the focal point of these measurements of intergenerational mobility. The data used in these works do not allow us to isolate mobility of capital income. 28. The correlation coefficient ranges from 0.2–0.3 in Sweden and Finland to 0.5–0.6 in the United States. Britain (0.4–0.5) is closer to the United States but not so far from Germany or France (0.4). Concerning international comparisons of intergenerational correlation coefficients of earned income (which are also confirmed by twin studies), see the work of Markus Jantti. See the online technical appendix. 29. The cost of an undergraduate year at Harvard in 2012–2013 was $54,000, including room and board and various other fees (tuition in the strict sense was $38,000).

…

According to the available data, the answer seems to be no: the intergenerational correlation of education and earned incomes, which measures the reproduction of the skill hierarchy over time, shows no trend toward greater mobility over the long run, and in recent years mobility may even have decreased.26 Note, however, that it is much more difficult to measure mobility across generations than it is to measure inequality at a given point in time, and the sources available for estimating the historical evolution of mobility are highly imperfect.27 The most firmly established result in this area of research is that intergenerational reproduction is lowest in the Nordic countries and highest in the United States (with a correlation coefficient two-thirds higher than in Sweden). France, Germany, and Britain occupy a middle ground, less mobile than northern Europe but more mobile than the United States.28 These findings stand in sharp contrast to the belief in “American exceptionalism” that once dominated US sociology, according to which social mobility in the United States was exceptionally high compared with the class-bound societies of Europe.

Once the American Dream: Inner-Ring Suburbs of the Metropolitan United States by Bernadette Hanlon

big-box store, classic study, company town, correlation coefficient, deindustrialization, desegregation, edge city, feminist movement, gentrification, housing crisis, illegal immigration, informal economy, longitudinal study, low skilled workers, low-wage service sector, manufacturing employment, McMansion, New Urbanism, Silicon Valley, statistical model, streetcar suburb, The Chicago School, transit-oriented development, urban sprawl, white flight, working-age population, zero-sum game

Table A.7 174 / Appendix TABLE A.7 RESULTS OF PEARSON CORRELATION BETWEEN INDEX SCORE AND CHANGE IN THE MEDIAN HOUSEHOLD INCOME RATIO FROM 1980 TO 2000 Variables Index score Change in median household income ratio from 1980 to 2000 Change in median household income ratio from 1980 to 2000 1 −0.801a −0.801a 1 3,428 3,428 N a Correlation Index score is significant at the 0.01 level (2-tailed). shows a Pearson’s Correlation between these two variables of −0.801. Pearson’s Correlation is a measure of correlation between two variables— that is, a measure of the tendency of variables to increase or decrease together. The correlation coefficient of −0.801 indicates that 80 percent of the variance in income is explained by variance in index score. The index score and the change in median household income ratio are highly negatively correlated. As the index score increases, the median household income ratio increases less over time.

Hubris: Why Economists Failed to Predict the Crisis and How to Avoid the Next One by Meghnad Desai

3D printing, Alan Greenspan, bank run, banking crisis, Bear Stearns, Berlin Wall, Big bang: deregulation of the City of London, Bretton Woods, BRICs, British Empire, business cycle, Capital in the Twenty-First Century by Thomas Piketty, Carmen Reinhart, central bank independence, collapse of Lehman Brothers, collateralized debt obligation, correlation coefficient, correlation does not imply causation, creative destruction, Credit Default Swap, credit default swaps / collateralized debt obligations, David Ricardo: comparative advantage, deindustrialization, demographic dividend, Eugene Fama: efficient market hypothesis, eurozone crisis, experimental economics, Fall of the Berlin Wall, financial innovation, Financial Instability Hypothesis, floating exchange rates, full employment, German hyperinflation, Glass-Steagall Act, Gunnar Myrdal, Home mortgage interest deduction, imperial preference, income inequality, inflation targeting, invisible hand, Isaac Newton, Joseph Schumpeter, Kenneth Arrow, Kenneth Rogoff, laissez-faire capitalism, liquidity trap, Long Term Capital Management, low interest rates, market bubble, market clearing, means of production, Meghnad Desai, Mexican peso crisis / tequila crisis, mortgage debt, Myron Scholes, negative equity, Northern Rock, oil shale / tar sands, oil shock, open economy, Paul Samuelson, Phillips curve, Post-Keynesian economics, price stability, purchasing power parity, pushing on a string, quantitative easing, reserve currency, rising living standards, risk/return, Robert Shiller, Robert Solow, Ronald Reagan, savings glut, secular stagnation, seigniorage, Silicon Valley, Simon Kuznets, subprime mortgage crisis, The Chicago School, The Great Moderation, The inhabitant of London could order by telephone, sipping his morning tea in bed, the various products of the whole earth, The Wealth of Nations by Adam Smith, Tobin tax, too big to fail, women in the workforce

This is to allow for the basic uncertainty of all economic events, as well as to allow for many other variables which have to be omitted to keep the relationship simple. If the basic equation is sound, then it will explain a large part of the variation in the variable we are interested in, in our case y, the amount bought of a commodity. A measure of the “goodness of fit” is the correlation coefficient r or its square R2 (R squared). Many equations together constitute a model and there are more sophisticated measures of the explanatory powers of a model. The use of econometric techniques is widespread now in public and private sector decision-making. Increasingly numbers have become an indispensable part of the toolkit of economists.

After the New Economy: The Binge . . . And the Hangover That Won't Go Away by Doug Henwood

"World Economic Forum" Davos, accounting loophole / creative accounting, affirmative action, Alan Greenspan, AOL-Time Warner, Asian financial crisis, barriers to entry, Benchmark Capital, book value, borderless world, Branko Milanovic, Bretton Woods, business cycle, California energy crisis, capital controls, corporate governance, corporate raider, correlation coefficient, credit crunch, deindustrialization, dematerialisation, deskilling, digital divide, electricity market, emotional labour, ending welfare as we know it, feminist movement, fulfillment center, full employment, gender pay gap, George Gilder, glass ceiling, Glass-Steagall Act, Gordon Gekko, government statistician, greed is good, half of the world's population has never made a phone call, income inequality, indoor plumbing, intangible asset, Internet Archive, job satisfaction, joint-stock company, Kevin Kelly, labor-force participation, Larry Ellison, liquidationism / Banker’s doctrine / the Treasury view, low interest rates, manufacturing employment, Mary Meeker, means of production, Michael Milken, minimum wage unemployment, Naomi Klein, new economy, occupational segregation, PalmPilot, pets.com, post-work, profit maximization, purchasing power parity, race to the bottom, Ralph Nader, rewilding, Robert Gordon, Robert Shiller, Robert Solow, rolling blackouts, Ronald Reagan, shareholder value, Silicon Valley, Simon Kuznets, statistical model, stock buybacks, structural adjustment programs, tech worker, Telecommunications Act of 1996, telemarketer, The Bell Curve by Richard Herrnstein and Charles Murray, The Wealth of Nations by Adam Smith, total factor productivity, union organizing, War on Poverty, warehouse automation, women in the workforce, working poor, zero-sum game

Hours worked was estimated by multiplying the BL5 figure for total employment by average weekly hours in each of the component industries, adding them together, and subtracting the result from a similar estimate of total manufacturing hours worked. While not exact, the approximation is pretty good; an estimate of total manufacturing productivity using this technique had a correlation coefficient of .86 with the official index. See text for discussion. the way we live and work, sometimes to the good, sometimes not. Do they make a 28% annual contribution to the growth of human happiness? Closely related to the productivity argument is a claim about innovation: that we Hve in a time of new product development without any his- torical precedent.

Empirical Market Microstructure: The Institutions, Economics and Econometrics of Securities Trading by Joel Hasbrouck

Alvin Roth, barriers to entry, business cycle, conceptual framework, correlation coefficient, discrete time, disintermediation, distributed generation, experimental economics, financial intermediation, index arbitrage, information asymmetry, interest rate swap, inventory management, market clearing, market design, market friction, market microstructure, martingale, payment for order flow, power law, price discovery process, price discrimination, quantitative trading / quantitative finance, random walk, Richard Thaler, second-price auction, selection bias, short selling, statistical model, stochastic process, stochastic volatility, transaction costs, two-sided market, ultimatum game, zero-sum game

This conforms to the usual intuition that positive correlation aggravates total portfolio risk. On the other hand, if we assume (as before) that the dealer is starting at his optimum, then B1 = P1 − ασ12 /2. Surprisingly, this is the same result as in the one-security case. In particular, the correlation coefficient drops out. This is a consequence of offsetting effects. The optimal n1 and n2 in equation (11.3) depends negatively on ρ, leaving the bracketed term invariant to changes in ρ. (Although this offset is a general feature of the problem, the complete disappearance of ρ in the final expression for the bid is a consequence of CARA utility.) 11.3 Empirical Analysis of Dealer Inventories 11.3.1 A First Look at the Data Changes in the dealer’s position reveal the dealer’s trades, which may disclose strategy and profitability.

Who's Your City?: How the Creative Economy Is Making Where to Live the Most Important Decision of Your Life by Richard Florida

Abraham Maslow, active measures, assortative mating, back-to-the-city movement, barriers to entry, big-box store, blue-collar work, borderless world, BRICs, business climate, Celebration, Florida, correlation coefficient, creative destruction, dark matter, David Brooks, David Ricardo: comparative advantage, deindustrialization, demographic transition, edge city, Edward Glaeser, epigenetics, extreme commuting, financial engineering, gentrification, Geoffrey West, Santa Fe Institute, happiness index / gross national happiness, high net worth, income inequality, industrial cluster, invention of the telegraph, Jane Jacobs, job satisfaction, Joseph Schumpeter, knowledge economy, knowledge worker, low skilled workers, megacity, new economy, New Urbanism, Peter Calthorpe, place-making, post-work, power law, Richard Florida, risk tolerance, Robert Gordon, Robert Shiller, Seaside, Florida, Silicon Valley, Silicon Valley startup, superstar cities, The Death and Life of Great American Cities, the strength of weak ties, The Wealth of Nations by Adam Smith, Thomas L Friedman, Tyler Cowen, urban planning, World Values Survey, young professional

Seligman, “Beyond Money: Toward an Economy of Well-Being,” Psychological Science in the Public Interest 5, 1, 2004, pp. 1-31.Betsy Stevenson and Justin Wolfers, “Economic Growth and Subjective Wellbeing: reassessing the Easterlin Paradox,” Wharton School, University of Pennsylvania, May 9, 2008, http://bpp.wharton.upenn.edu/jwolfers/Papers/EasterlinParadox.pdf. 3 Also see Angus Deaton, “Income, Aging, Health, and Wellbeing Around the World: Evidence from the Gallup World Poll,” Center for Health and Wellbeing, Research Program in Development Studies, Princeton University, August 2007. 4 Nick Paumgarten, “There and Back Again,” New Yorker, April 16, 2007. 5 Robert Manchin, “The Emotional Capital and Desirability of European Cities,” Gallup Europe, presented at the European Week of Cities and Regions, Brussels, October 2007. 6 The correlation coefficients between overall happiness and various factors are as follows: financial satisfaction (.369), job satisfaction (.367), place satisfaction (.303). Compare with income (.153), homeownership (.126), and age (.06). The regression coefficients (from an ordered probit regression) are as follows: financial satisfaction (.342), place satisfaction (.254), job satisfaction (.254).

The Behavioral Investor by Daniel Crosby

affirmative action, Asian financial crisis, asset allocation, availability heuristic, backtesting, bank run, behavioural economics, Black Monday: stock market crash in 1987, Black Swan, book value, buy and hold, cognitive dissonance, colonial rule, compound rate of return, correlation coefficient, correlation does not imply causation, Daniel Kahneman / Amos Tversky, disinformation, diversification, diversified portfolio, Donald Trump, Dunning–Kruger effect, endowment effect, equity risk premium, fake news, feminist movement, Flash crash, haute cuisine, hedonic treadmill, housing crisis, IKEA effect, impact investing, impulse control, index fund, Isaac Newton, Japanese asset price bubble, job automation, longitudinal study, loss aversion, market bubble, market fundamentalism, mental accounting, meta-analysis, Milgram experiment, moral panic, Murray Gell-Mann, Nate Silver, neurotypical, Nick Bostrom, passive investing, pattern recognition, Pepsi Challenge, Ponzi scheme, prediction markets, random walk, Reminiscences of a Stock Operator, Richard Feynman, Richard Thaler, risk tolerance, Robert Shiller, science of happiness, Shai Danziger, short selling, South Sea Bubble, Stanford prison experiment, Stephen Hawking, Steve Jobs, stocks for the long run, sunk-cost fallacy, systems thinking, TED Talk, Thales of Miletus, The Signal and the Noise by Nate Silver, Tragedy of the Commons, trolley problem, tulip mania, Vanguard fund, When a measure becomes a target

The authors of ‘Positive Illusions and Forecasting Errors in Mutual Fund Investment Decisions’ discovered that most participants had consistently overestimated both the future and past performance of their investments.70 One-third of those who believed that they had outperformed the market had actually lagged by at least 5% and another quarter of people lagged by 15% or greater. Even more damning research is found by Glaser and Weber who discovered that, “Investors are unable to give a correct estimate of their own past portfolio performance. The correlation coefficient between return estimates and realized returns was not distinguishable from zero.”71 The finding that investors would misstate their returns is not entirely surprising, but the size and scope of the problem is. Only 30% of those surveyed considered themselves to be “average” investors and the average overestimation of returns was 11.5% per year!

Ever Since Darwin: Reflections in Natural History by Stephen Jay Gould

Alfred Russel Wallace, British Empire, correlation coefficient, Drosophila, European colonialism, invisible hand, Isaac Newton, Monroe Doctrine, Paul Samuelson, planned obsolescence, Scientific racism, sexual politics, the scientific method, twin studies

He has found an astonishing number of inconsistencies and downright inaccuracies. For example, the late Sir Cyril Burt, who generated the largest body of data on identical twins reared apart, pursued his studies of intelligence for more than forty years. Although he increased his sample sizes in a variety of “improved” versions, some of his correlation coefficients remain unchanged to the third decimal place—a statistically impossible situation.5 IQ depends in part upon sex and age; and other studies did not standardize properly for them. An improper correction may produce higher values between twins not because they hold genes for intelligence in common, but simply because they share the same sex and age.

The Crisis of Crowding: Quant Copycats, Ugly Models, and the New Crash Normal by Ludwig B. Chincarini

affirmative action, Alan Greenspan, asset-backed security, automated trading system, bank run, banking crisis, Basel III, Bear Stearns, Bernie Madoff, Black-Scholes formula, Bob Litterman, business cycle, buttonwood tree, Carmen Reinhart, central bank independence, collapse of Lehman Brothers, collateralized debt obligation, collective bargaining, corporate governance, correlation coefficient, Credit Default Swap, credit default swaps / collateralized debt obligations, currency risk, delta neutral, discounted cash flows, diversification, diversified portfolio, family office, financial engineering, financial innovation, financial intermediation, fixed income, Flash crash, full employment, Gini coefficient, Glass-Steagall Act, global macro, high net worth, hindsight bias, housing crisis, implied volatility, income inequality, interest rate derivative, interest rate swap, John Meriwether, Kickstarter, liquidity trap, London Interbank Offered Rate, Long Term Capital Management, low interest rates, low skilled workers, managed futures, margin call, market design, market fundamentalism, merger arbitrage, Mexican peso crisis / tequila crisis, Mitch Kapor, money market fund, moral hazard, mortgage debt, Myron Scholes, National best bid and offer, negative equity, Northern Rock, Occupy movement, oil shock, price stability, proprietary trading, quantitative easing, quantitative hedge fund, quantitative trading / quantitative finance, Ralph Waldo Emerson, regulatory arbitrage, Renaissance Technologies, risk free rate, risk tolerance, risk-adjusted returns, Robert Shiller, Ronald Reagan, Sam Peltzman, Savings and loan crisis, Sharpe ratio, short selling, sovereign wealth fund, speech recognition, statistical arbitrage, statistical model, survivorship bias, systematic trading, tail risk, The Great Moderation, too big to fail, transaction costs, value at risk, yield curve, zero-coupon bond

Unfortunately, it is impossible to have an average correlation of −1 with more than two strategies in a portfolio. As you add more strategies, however, the conditions necessary for a very low-risk portfolio grow less stringent. Consider a portfolio of n trading strategies. Assuming a similar variance for each strategy, , and that any pair of strategies has the same correlation coefficient, . If all strategies are equally weighted (that is, ) and individual strategy returns are positive, the portfolio variance is: (A.4) Construct a portfolio with a large number of positions that have an average correlation of zero, and the portfolio risk decreases toward zero. For the purposes of this analysis, assume that this was LTCM’s driving concept.

…

., k = 1.96 for a 97.5% confidence interval).9 Table A.1 presents the potential VaR calculations at a 99% confidence level for a normal distribution (k = 2.33) and a capital base of $4.8B (the amount that LTCM had at the beginning of 1998). The VaR numbers are presented as monthly numbers. Given the correlation coefficient, this represents what might have been expected to occur in any given month at LTCM. TABLE A.1 Sensitivity of VaR to Strategy Correlations Table A.1 shows that an unlevered fund’s standard deviation was 0.0951% per month and 0.6723% per month with a correlation of 0 and 1 respectively.

Big Business: A Love Letter to an American Anti-Hero by Tyler Cowen

"Friedman doctrine" OR "shareholder theory", 23andMe, Affordable Care Act / Obamacare, augmented reality, barriers to entry, Bernie Sanders, Big Tech, bitcoin, blockchain, Bretton Woods, cloud computing, cognitive dissonance, company town, compensation consultant, corporate governance, corporate social responsibility, correlation coefficient, creative destruction, crony capitalism, cryptocurrency, dark matter, David Brooks, David Graeber, don't be evil, Donald Trump, driverless car, Elon Musk, employer provided health coverage, experimental economics, Fairchild Semiconductor, fake news, Filter Bubble, financial innovation, financial intermediation, gentrification, Glass-Steagall Act, global reserve currency, global supply chain, Google Glasses, income inequality, Internet of things, invisible hand, Jeff Bezos, junk bonds, late fees, Mark Zuckerberg, mobile money, money market fund, mortgage debt, Network effects, new economy, Nicholas Carr, obamacare, offshore financial centre, passive investing, payday loans, peer-to-peer lending, Peter Thiel, pre–internet, price discrimination, profit maximization, profit motive, RAND corporation, rent-seeking, reserve currency, ride hailing / ride sharing, risk tolerance, Ronald Coase, shareholder value, Silicon Valley, Silicon Valley startup, Skype, Snapchat, Social Responsibility of Business Is to Increase Its Profits, Steve Jobs, The Nature of the Firm, Tim Cook: Apple, too big to fail, transaction costs, Tyler Cowen, Tyler Cowen: Great Stagnation, ultimatum game, WikiLeaks, women in the workforce, World Values Survey, Y Combinator

Here too we should be cautious about how grand a conclusion we draw from a single study, but this is suggestive evidence that the workplace often serves a significant protective and equalizing function when it comes to personal stress. Furthermore, the Kahneman and Krueger research generates a broadly similar result. The positive affect associated with the workday is not closely related to the features we usually associate with a “good” job. (For instance, the correlation coefficient of positive affect in the workplace with “excellent benefits” is only about 0.10.) People with lower-quality jobs still get a lot of the benefits from the positive affect associated with work. Here’s a simple and probably familiar story from Elizabeth Bernstein, writing in the Wall Street Journal.

Heads I Win, Tails I Win by Spencer Jakab

Alan Greenspan, Asian financial crisis, asset allocation, backtesting, Bear Stearns, behavioural economics, Black Monday: stock market crash in 1987, book value, business cycle, buy and hold, collapse of Lehman Brothers, correlation coefficient, crowdsourcing, Daniel Kahneman / Amos Tversky, diversification, dividend-yielding stocks, dogs of the Dow, Elliott wave, equity risk premium, estate planning, Eugene Fama: efficient market hypothesis, eurozone crisis, Everybody Ought to Be Rich, fear index, fixed income, geopolitical risk, government statistician, index fund, Isaac Newton, John Bogle, John Meriwether, Long Term Capital Management, low interest rates, Market Wizards by Jack D. Schwager, Mexican peso crisis / tequila crisis, money market fund, Myron Scholes, PalmPilot, passive investing, Paul Samuelson, pets.com, price anchoring, proprietary trading, Ralph Nelson Elliott, random walk, Reminiscences of a Stock Operator, risk tolerance, risk-adjusted returns, Robert Shiller, robo advisor, Savings and loan crisis, Sharpe ratio, short selling, Silicon Valley, South Sea Bubble, statistical model, Steve Jobs, subprime mortgage crisis, survivorship bias, technology bubble, transaction costs, two and twenty, VA Linux, Vanguard fund, zero-coupon bond, zero-sum game

Studies, among them one by German finance professors Markus Glaser and Martin Weber, show that there’s absolutely no connection between how well an investor actually has done and how well they think they have done. None, nada, zip. Or, as the Herren Professors put it in more scientific language (ideally recited in a thick Teutonic accent), “The correlation coefficient between return estimates and realized returns is not distinguishable from zero.”1 That’s pretty amazing. But if I wanted to do better than just break even—much better, in fact—I’d bet you instead on whether or not you keep up with the market return. In Glaser and Weber’s study the investors surveyed estimated that on average they had made nearly 15 percent on an annualized basis over the four years in question.

The Power of Passive Investing: More Wealth With Less Work by Richard A. Ferri

Alan Greenspan, asset allocation, backtesting, Benchmark Capital, Bernie Madoff, book value, buy and hold, capital asset pricing model, cognitive dissonance, correlation coefficient, currency risk, Daniel Kahneman / Amos Tversky, diversification, diversified portfolio, endowment effect, estate planning, Eugene Fama: efficient market hypothesis, fixed income, implied volatility, index fund, intangible asset, John Bogle, junk bonds, Long Term Capital Management, money market fund, passive investing, Paul Samuelson, Performance of Mutual Funds in the Period, Ponzi scheme, prediction markets, proprietary trading, prudent man rule, random walk, Richard Thaler, risk free rate, risk tolerance, risk-adjusted returns, risk/return, Sharpe ratio, survivorship bias, Tax Reform Act of 1986, too big to fail, transaction costs, Vanguard fund, yield curve, zero-sum game

closed-end fund A mutual fund that has a fixed number of shares, usually listed on a major stock exchange. commodities Unprocessed goods, such as grains, metals, and minerals, traded in large amounts on a commodities exchange. consumer price index (CPI) A measure of the price change in consumer goods and services. The CPI is used to track the pace of inflation. correlation coefficient A number between −1 and 1 that measures the degree to which two variables are linearly related. cost basis The original cost of an investment. For tax purposes, the cost basis is subtracted from the sale price to determine any capital gain or loss. country risk The possibility that political events (e.g., a war, national elections); financial problems (e.g., rising inflation, government default); or natural disasters (e.g., an earthquake, a poor harvest) will weaken a country’s economy and cause investments in that country to decline.

The Blank Slate: The Modern Denial of Human Nature by Steven Pinker

affirmative action, Albert Einstein, Alfred Russel Wallace, anti-communist, behavioural economics, belling the cat, British Empire, clean water, cognitive dissonance, Columbine, conceptual framework, correlation coefficient, correlation does not imply causation, cuban missile crisis, Daniel Kahneman / Amos Tversky, Defenestration of Prague, desegregation, disinformation, Dutch auction, epigenetics, Exxon Valdez, George Akerlof, germ theory of disease, ghettoisation, glass ceiling, Gregor Mendel, Hobbesian trap, income inequality, invention of agriculture, invisible hand, Joan Didion, language acquisition, long peace, meta-analysis, More Guns, Less Crime, Murray Gell-Mann, mutually assured destruction, Norman Mailer, Oklahoma City bombing, PalmPilot, Peter Singer: altruism, phenotype, plutocrats, Potemkin village, prisoner's dilemma, profit motive, public intellectual, QWERTY keyboard, Richard Feynman, Richard Thaler, risk tolerance, Robert Bork, Rodney Brooks, Saturday Night Live, Skinner box, social intelligence, speech recognition, Stanford prison experiment, stem cell, Steven Pinker, tacit knowledge, The Bell Curve by Richard Herrnstein and Charles Murray, the new new thing, theory of mind, Thomas Malthus, Thorstein Veblen, Timothy McVeigh, twin studies, Tyler Cowen, ultimatum game, urban renewal, War on Poverty, women in the workforce, Yogi Berra, zero-sum game

Chapter 16 Politics I often think it’s comical How nature always does contrive That every boy and every gal, That’s born into the world alive, Is either a little Liberal, Or else a little Conservative!1 GILBERT AND SULLIVAN got it mostly right in 1882: liberal and conservative political attitudes are largely, though far from completely, heritable. When identical twins who were separated at birth are tested in adulthood, their political attitudes turn out to be similar, with a correlation coefficient of. 62 (on a scale from-1 to +1).2 Liberal and conservative attitudes are heritable not, of course, because attitudes are synthesized directly from DNA but because they come naturally to people with different temperaments. Conservatives, for example, tend to be more authoritarian, conscientious, traditional, and rule-bound.

…

It is mathematically meaningful to say that a certain percentage of the variance in a group overlaps with one factor (perhaps, though not necessarily, its cause), another percentage overlaps with a second factor, and so on, the percentages adding up to 100. The degree of overlap may be measured as a correlation coefficient, a number between-1 and +1 that captures the degree to which people who are high on one measurement are also high on another measurement. It is used in behavioral genetic research as an estimate of the proportion of variance accounted for by some factor.3 Heritability is the proportion of variance in a trait that correlates with genetic differences.

The New Science of Asset Allocation: Risk Management in a Multi-Asset World by Thomas Schneeweis, Garry B. Crowder, Hossein Kazemi

asset allocation, backtesting, Bear Stearns, behavioural economics, Bernie Madoff, Black Swan, book value, business cycle, buy and hold, capital asset pricing model, collateralized debt obligation, commodity trading advisor, correlation coefficient, Credit Default Swap, credit default swaps / collateralized debt obligations, currency risk, diversification, diversified portfolio, financial engineering, fixed income, global macro, high net worth, implied volatility, index fund, interest rate swap, invisible hand, managed futures, market microstructure, merger arbitrage, moral hazard, Myron Scholes, passive investing, Richard Feynman, Richard Feynman: Challenger O-ring, risk free rate, risk tolerance, risk-adjusted returns, risk/return, search costs, selection bias, Sharpe ratio, short selling, statistical model, stocks for the long run, survivorship bias, systematic trading, technology bubble, the market place, Thomas Kuhn: the structure of scientific revolutions, transaction costs, value at risk, yield curve, zero-sum game

The break-even (Rc) and excess break-even rate of return (EBK) is often computed as follows: ⎛ E (Rp ) − Rf ⎞ E ( Rc ) = ⎜ ⎟⎠ ( ρcp ) σ c + Rf ⎝ σp ⎤ ⎡⎛ E (Rp ) − Rf ⎞ EBK = Rc − ⎢⎜ ⎟⎠ ( ρcp ) σ c + Rf ⎥ ⎝ σ p ⎦ ⎣ where E(Rc) = Break-even rate of return required for the asset to improve the Sharpe Ratio of alternative index p Rc = Rate of return on asset c Rf = Riskless rate of return E(Rp) = Rate of return on alternative index p ρcp = Correlation coefficient between asset c and alternative benchmark p σc = Standard deviation of asset c σp = Standard deviation of alternative index p First, it is important to realize that the above expression is based on the assumption that only mean and variance matter in evaluating the risk-return profile of a portfolio.

Advances in Financial Machine Learning by Marcos Lopez de Prado

algorithmic trading, Amazon Web Services, asset allocation, backtesting, behavioural economics, bioinformatics, Brownian motion, business process, Claude Shannon: information theory, cloud computing, complexity theory, correlation coefficient, correlation does not imply causation, data science, diversification, diversified portfolio, en.wikipedia.org, financial engineering, fixed income, Flash crash, G4S, Higgs boson, implied volatility, information asymmetry, latency arbitrage, margin call, market fragmentation, market microstructure, martingale, NP-complete, P = NP, p-value, paper trading, pattern recognition, performance metric, profit maximization, quantitative trading / quantitative finance, RAND corporation, random walk, risk free rate, risk-adjusted returns, risk/return, selection bias, Sharpe ratio, short selling, Silicon Valley, smart cities, smart meter, statistical arbitrage, statistical model, stochastic process, survivorship bias, transaction costs, traveling salesman

Tamhane, and F. Bretz (2010): Multiple Testing Problems in Pharmaceutical Statistics, 1st ed. CRC Press. Dudoit, S. and M.J. van der Laan (2008): Multiple Testing Procedures with Applications to Genomics, 1st ed. Springer. Fisher, R.A. (1915): “Frequency distribution of the values of the correlation coefficient in samples of an indefinitely large population.” Biometrika (Biometrika Trust), Vol. 10, No. 4, pp. 507–521. Hand, D. J. (2014): The Improbability Principle, 1st ed. Scientific American/Farrar, Straus and Giroux. Harvey, C., Y. Liu, and H. Zhu (2013): “. . . And the cross-section of expected returns.”

The Intelligence Trap: Revolutionise Your Thinking and Make Wiser Decisions by David Robson

active measures, Affordable Care Act / Obamacare, Albert Einstein, Alfred Russel Wallace, Atul Gawande, autism spectrum disorder, availability heuristic, behavioural economics, classic study, cognitive bias, corporate governance, correlation coefficient, cuban missile crisis, Daniel Kahneman / Amos Tversky, dark matter, deep learning, deliberate practice, dematerialisation, Donald Trump, Dunning–Kruger effect, fake news, Flynn Effect, framing effect, fundamental attribution error, illegal immigration, Isaac Newton, job satisfaction, knowledge economy, Large Hadron Collider, lone genius, meta-analysis, Nelson Mandela, obamacare, Parler "social media", pattern recognition, post-truth, price anchoring, reality distortion field, Richard Feynman, risk tolerance, Silicon Valley, social intelligence, Steve Jobs, sunk-cost fallacy, tacit knowledge, TED Talk, the scientific method, theory of mind, traveling salesman, ultimatum game, Y2K, Yom Kippur War

Stanovich has now spent more than two decades building on those foundations with a series of carefully controlled experiments. To understand his results, we need some basic statistical theory. In psychology and other sciences, the relationship between two variables is usually expressed as a correlation coefficient between 0 and 1. A perfect correlation would have a value of 1 – the two parameters would essentially be measuring the same thing; this is unrealistic for most studies of human health and behaviour (which are determined by so many variables), but many scientists would consider a ‘moderate’ correlation to lie between 0.4 and 0.59.11 Using these measures, Stanovich found that the relationships between rationality and intelligence were generally very weak.

The Genetic Lottery: Why DNA Matters for Social Equality by Kathryn Paige Harden

23andMe, Affordable Care Act / Obamacare, assortative mating, autism spectrum disorder, Bayesian statistics, Berlin Wall, Black Lives Matter, classic study, clean water, combinatorial explosion, coronavirus, correlation coefficient, correlation does not imply causation, COVID-19, CRISPR, crowdsourcing, delayed gratification, deliberate practice, desegregation, double helix, epigenetics, game design, George Floyd, Gregor Mendel, impulse control, income inequality, Jeff Bezos, longitudinal study, low skilled workers, Mark Zuckerberg, meritocracy, meta-analysis, Monkeys Reject Unequal Pay, phenotype, randomized controlled trial, replication crisis, Scientific racism, stochastic process, surveillance capitalism, TED Talk, The Bell Curve by Richard Herrnstein and Charles Murray, twin studies, War on Poverty, zero-sum game

Galton, however, wasn’t content merely to document familial resemblance in the form of pedigree tables; he wanted to quantify—put a number on—that resemblance. Indeed, quantification was his most enduring enthusiasm; “whenever you can, count” was his slogan.30 In seeking a mathematical representation of familial resemblance, Galton invented foundational statistical concepts, like the correlation coefficient. But alongside his statistical developments, he also speculated about how heredity could and should be manipulated in humans. In a footnote published in 1883, Galton introduced the new word “eugenics” to “express the science of improving stock,” the aim of which was “to give to more suitable races or strains of blood a better chance of prevailing speedily over the less suitable.”31 From the very beginning, then, the nascent science of statistics, and the application of statistics to study patterns of familial resemblance, were entangled with beliefs about racial superiority and with proposals to intervene in human reproduction for the goal of species betterment.

Fortune's Formula: The Untold Story of the Scientific Betting System That Beat the Casinos and Wall Street by William Poundstone

"RICO laws" OR "Racketeer Influenced and Corrupt Organizations", Albert Einstein, anti-communist, asset allocation, Bear Stearns, beat the dealer, Benoit Mandelbrot, Black Monday: stock market crash in 1987, Black-Scholes formula, Bletchley Park, Brownian motion, buy and hold, buy low sell high, capital asset pricing model, Claude Shannon: information theory, computer age, correlation coefficient, diversified portfolio, Edward Thorp, en.wikipedia.org, Eugene Fama: efficient market hypothesis, financial engineering, Henry Singleton, high net worth, index fund, interest rate swap, Isaac Newton, Johann Wolfgang von Goethe, John Meriwether, John von Neumann, junk bonds, Kenneth Arrow, Long Term Capital Management, Louis Bachelier, margin call, market bubble, market fundamentalism, Marshall McLuhan, Michael Milken, Myron Scholes, New Journalism, Norbert Wiener, offshore financial centre, Paul Samuelson, publish or perish, quantitative trading / quantitative finance, random walk, risk free rate, risk tolerance, risk-adjusted returns, Robert Shiller, Ronald Reagan, Rubik’s Cube, short selling, speech recognition, statistical arbitrage, Teledyne, The Predators' Ball, The Wealth of Nations by Adam Smith, transaction costs, traveling salesman, value at risk, zero-coupon bond, zero-sum game

For reasons mathematical, psychological, and sociological, it is a good idea to use a money management system that is relatively forgiving of estimation errors. Fat Tails and Leverage Suppose you’re betting on a simultaneous toss of coins believed to have a 55 percent chance of coming up heads, as depicted on the previous page. But on this toss, only 45 percent of the coins are heads. Call it a “fat tail” event, or a failure of correlation coefficients, or a big dumb mistake in somebody’s computer model. What then? The Kelly bettor cannot be ruined in a single toss. (He is prepared to survive the worst-case scenario, of zero heads.) In this situation, with many coins, the Kelly bettor will stake just short of his full bankroll. He wins only 45 percent of the wagers, doubling the amount bet on each coin that comes up heads.

Rage Inside the Machine: The Prejudice of Algorithms, and How to Stop the Internet Making Bigots of Us All by Robert Elliott Smith

"World Economic Forum" Davos, Ada Lovelace, adjacent possible, affirmative action, AI winter, Alfred Russel Wallace, algorithmic bias, algorithmic management, AlphaGo, Amazon Mechanical Turk, animal electricity, autonomous vehicles, behavioural economics, Black Swan, Brexit referendum, British Empire, Cambridge Analytica, cellular automata, Charles Babbage, citizen journalism, Claude Shannon: information theory, combinatorial explosion, Computing Machinery and Intelligence, corporate personhood, correlation coefficient, crowdsourcing, Daniel Kahneman / Amos Tversky, data science, deep learning, DeepMind, desegregation, discovery of DNA, disinformation, Douglas Hofstadter, Elon Musk, fake news, Fellow of the Royal Society, feminist movement, Filter Bubble, Flash crash, Geoffrey Hinton, Gerolamo Cardano, gig economy, Gödel, Escher, Bach, invention of the wheel, invisible hand, Jacquard loom, Jacques de Vaucanson, John Harrison: Longitude, John von Neumann, Kenneth Arrow, Linda problem, low skilled workers, Mark Zuckerberg, mass immigration, meta-analysis, mutually assured destruction, natural language processing, new economy, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, On the Economy of Machinery and Manufactures, p-value, pattern recognition, Paul Samuelson, performance metric, Pierre-Simon Laplace, post-truth, precariat, profit maximization, profit motive, Silicon Valley, social intelligence, statistical model, Stephen Hawking, stochastic process, Stuart Kauffman, telemarketer, The Bell Curve by Richard Herrnstein and Charles Murray, The Future of Employment, the scientific method, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, theory of mind, Thomas Bayes, Thomas Malthus, traveling salesman, Turing machine, Turing test, twin studies, Vilfredo Pareto, Von Neumann architecture, warehouse robotics, women in the workforce, Yochai Benkler

Processing the lab’s big data required statistical mathematics, so in 1911 Pearson (who already held a chair in Applied Mathematics at UCL) merged the biometric and eugenics laboratories to form the Department of Applied Statistics, the first university statistics department in the world. Pearson went on to create the Pearson correlation coefficient, one of the most fundamental calculations in statistics. In fact, his work is so foundational to statistics that he was offered a knighthood (which he declined based on his personal commitment to socialism). The UCL building which once housed the Department of Statistics bears his name. Pearson also founded The Annals of Eugenics journal (which now exists as the prominent Annals of Genetics), the masthead of which originally included the famous (mis)quote from Charles Darwin.

Deaths of Despair and the Future of Capitalism by Anne Case, Angus Deaton

Affordable Care Act / Obamacare, basic income, Bertrand Russell: In Praise of Idleness, Boeing 737 MAX, business cycle, call centre, collapse of Lehman Brothers, collective bargaining, company town, Corn Laws, corporate governance, correlation coefficient, crack epidemic, creative destruction, crony capitalism, declining real wages, deindustrialization, demographic transition, Dissolution of the Soviet Union, Donald Trump, Downton Abbey, Edward Glaeser, Elon Musk, falling living standards, Fellow of the Royal Society, financial engineering, fulfillment center, germ theory of disease, income inequality, Jeff Bezos, Joseph Schumpeter, Ken Thompson, Kenneth Arrow, labor-force participation, Les Trente Glorieuses, low skilled workers, Martin Wolf, meritocracy, Mikhail Gorbachev, obamacare, opioid epidemic / opioid crisis, pensions crisis, pill mill, randomized controlled trial, refrigerator car, rent-seeking, risk tolerance, shareholder value, Silicon Valley, The Spirit Level, The Wealth of Nations by Adam Smith, Tim Cook: Apple, trade liberalization, Tyler Cowen, universal basic income, working-age population, zero-sum game

National Research Council, 2005, “Firearms and suicide,” in Firearms and violence: A critical review, National Academies Press, 152–200. 12. Robert D. Putnam, 2000, Bowling alone: The collapse and revival of American community, Simon and Schuster. 13. CDC Wonder, average suicide rates over the period 2008–17. 14. Across the fifty US states, the correlation coefficient is .4. 15. Anne Case and Angus Deaton, 2017, “Suicide, age, and well-being: An empirical investigation,” in David A. Wise, ed., Insights in the economics of aging, National Bureau of Economic Research Conference Report, University of Chicago Press for NBER, 307–34. 16. The fractions of the birth cohorts of 1945 and 1970 who finish a four-year degree are not very different, so these results are unlikely to be attributable to changing compositions of those with and without a bachelor’s degree between the cohorts. 17.

Succeeding With AI: How to Make AI Work for Your Business by Veljko Krunic

AI winter, Albert Einstein, algorithmic trading, AlphaGo, Amazon Web Services, anti-fragile, anti-pattern, artificial general intelligence, autonomous vehicles, Bayesian statistics, bioinformatics, Black Swan, Boeing 737 MAX, business process, cloud computing, commoditize, computer vision, correlation coefficient, data is the new oil, data science, deep learning, DeepMind, en.wikipedia.org, fail fast, Gini coefficient, high net worth, information retrieval, Internet of things, iterative process, job automation, Lean Startup, license plate recognition, minimum viable product, natural language processing, recommendation engine, self-driving car, sentiment analysis, Silicon Valley, six sigma, smart cities, speech recognition, statistical model, strong AI, tail risk, The Design of Experiments, the scientific method, web application, zero-sum game

In 2008, a widely quoted article [152] advanced the argument that if you have all the data so that you can predict what’s going to happen, does it matter why? As a corollary of that approach, theory (and theories about causality) was declared to matter much less. 3 Here, I use the word correlation to mean that there’s some form of relationship (or co-occurrence) between events. I’m not implying that AI would necessarily calculate a correlation coefficient, only that AI would learn that these events occur together. 204 CHAPTER 8 AI trends that may affect you You never have all the data Unless you possess special skills such as clairvoyance, you’ll never have all the data about all possible outcomes! To start with, you’re missing data about results that have yet to occur.

Luxury Fever: Why Money Fails to Satisfy in an Era of Excess by Robert H. Frank

Alan Greenspan, business cycle, clean water, company town, compensation consultant, Cornelius Vanderbilt, correlation coefficient, Daniel Kahneman / Amos Tversky, full employment, Garrett Hardin, germ theory of disease, global village, haute couture, hedonic treadmill, impulse control, income inequality, invisible hand, job satisfaction, Kenneth Arrow, lake wobegon effect, loss aversion, market clearing, McMansion, means of production, mega-rich, mortgage debt, New Urbanism, Pareto efficiency, Post-Keynesian economics, RAND corporation, rent control, Richard Thaler, rising living standards, Ronald Reagan, Silicon Valley, Tax Reform Act of 1986, telemarketer, The Theory of the Leisure Class by Thorstein Veblen, The Wealth of Nations by Adam Smith, Thomas Malthus, Thorstein Veblen, Tragedy of the Commons, trickle-down economics, ultimatum game, winner-take-all economy, working poor

Note the striking contrast between this relationship and the one we saw earlier in which average satisfaction levels remained constant as average per-capita incomes rose manyfold over time within a country. Notwithstanding the consistency of findings like the one summarized in the diagram, however, the actual numerical correlation between income and subjective well-being across individuals is relatively small. Thus, according to one typical study, the partial correlation coefficient between income and subjective well-being for Americans at a single moment in time was only 0.13.2 (This means that, after controlling for the influence of other factors, variations in income explain less than 2 percent of the observed individual variance in subjective well-being.) Numbers like these have inspired many psychologists to assert that income is not, in fact, an important determinant of individual differences in subjective well-being.

Work Rules!: Insights From Inside Google That Will Transform How You Live and Lead by Laszlo Bock

Abraham Maslow, Abraham Wald, Airbnb, Albert Einstein, AltaVista, Atul Gawande, behavioural economics, Black Swan, book scanning, Burning Man, call centre, Cass Sunstein, Checklist Manifesto, choice architecture, citizen journalism, clean water, cognitive load, company town, correlation coefficient, crowdsourcing, Daniel Kahneman / Amos Tversky, deliberate practice, en.wikipedia.org, experimental subject, Fairchild Semiconductor, Frederick Winslow Taylor, future of work, Google Earth, Google Glasses, Google Hangouts, Google X / Alphabet X, Googley, helicopter parent, immigration reform, Internet Archive, Kevin Roose, longitudinal study, Menlo Park, mental accounting, meta-analysis, Moneyball by Michael Lewis explains big data, nudge unit, PageRank, Paul Buchheit, power law, Ralph Waldo Emerson, Rana Plaza, random walk, Richard Thaler, Rubik’s Cube, self-driving car, shareholder value, Sheryl Sandberg, side project, Silicon Valley, six sigma, statistical model, Steve Ballmer, Steve Jobs, Steven Levy, Steven Pinker, survivorship bias, Susan Wojcicki, TaskRabbit, The Wisdom of Crowds, Tony Hsieh, Turing machine, Wayback Machine, winner-take-all economy, Y2K

Hunter, “The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings,” Psychological Bulletin 124, no. 2 (1998): 262–274. The r2 values presented in this chapter are calculated based on the reported corrected correlation coefficients (r). 86. Phyllis Rosser, The SAT Gender Gap: Identifying the Causes (Washington, DC: Center for Women Policy Studies, 1989). 87. Subsequent studies have validated the gender gap on the SAT and demonstrated racial bias as well. See, for example, Christianne Corbett, Catherine Hill, and Andresse St.

Career and Family: Women’s Century-Long Journey Toward Equity by Claudia Goldin

coronavirus, correlation coefficient, COVID-19, en.wikipedia.org, equal pay for equal work, estate planning, financial independence, gender pay gap, glass ceiling, global pandemic, Grace Hopper, hiring and firing, income inequality, Internet Archive, job automation, labor-force participation, longitudinal study, occupational segregation, old-boy network, principal–agent problem, profit maximization, remote working, Silicon Valley, Simon Kuznets, social distancing, women in the workforce, Works Progress Administration, young professional

Note that the CTS excludes physicians who do not have their own patients and is not a random sample of all physicians. 210  More than 55 percent … among the younger group of doctors    Current American Medical Association data on the specialties of recent graduates have a higher fraction of women in various specialties than does the CTS data, in part because the AMA data are more recent. 211  the higher the hours for male physicians in a specialty, the fewer female physicians    I use male physician hours as the reference so that the statement is more causal. 211  strong negative relationship between average hours … and the fraction female    The relationship between fraction female and mean hours worked for males less than forty-five years old is strong among the nineteen specialties with enough women to analyze. There are two outliers: OB-GYN and pediatrics. These specialties have a higher fraction female than would be predicted on the basis of the relationship between hours and fraction female. Without the two outlier specialties, the correlation coefficient between fraction female and weekly hours for males is around 0.8. For the entire sample it is 0.66. 211  a smaller fraction than for dermatologists    The fraction female in each specialty is from the American Medical Association (2013), but the hours data are from the CTS data. The AMA 2013 data are used to be more consistent with the older CTS data. 211  younger women doctors work fewer hours than do younger men doctors    See Online Appendix figure 1A (Ch10), “Physician Hours by Specialty, Sex, and Age,” which gives the relationship between the hours of male and female physicians by specialty and age group. 212  67 to 82 cents on the male physician dollar    The CTS does not have actual job experience, and these estimates use information on time since receiving the MD.

Inside the House of Money: Top Hedge Fund Traders on Profiting in a Global Market by Steven Drobny

Abraham Maslow, Alan Greenspan, Albert Einstein, asset allocation, Berlin Wall, Bonfire of the Vanities, Bretton Woods, business cycle, buy and hold, buy low sell high, capital controls, central bank independence, commoditize, commodity trading advisor, corporate governance, correlation coefficient, Credit Default Swap, currency risk, diversification, diversified portfolio, family office, financial engineering, fixed income, glass ceiling, Glass-Steagall Act, global macro, Greenspan put, high batting average, implied volatility, index fund, inflation targeting, interest rate derivative, inventory management, inverted yield curve, John Meriwether, junk bonds, land bank, Long Term Capital Management, low interest rates, managed futures, margin call, market bubble, Market Wizards by Jack D. Schwager, Maui Hawaii, Mexican peso crisis / tequila crisis, moral hazard, Myron Scholes, new economy, Nick Leeson, Nixon triggered the end of the Bretton Woods system, oil shale / tar sands, oil shock, out of africa, panic early, paper trading, Paul Samuelson, Peter Thiel, price anchoring, proprietary trading, purchasing power parity, Reminiscences of a Stock Operator, reserve currency, risk free rate, risk tolerance, risk-adjusted returns, risk/return, rolodex, Sharpe ratio, short selling, Silicon Valley, tail risk, The Wisdom of Crowds, too big to fail, transaction costs, value at risk, Vision Fund, yield curve, zero-coupon bond, zero-sum game

Risk management systems based on historical prices are one way to look at risk but are in no way faultless. Financial market history is filled with theoretically low probability or fat tail events. In LTCM’s case, its risk systems calculated roughly a 1-in-6-billion chance of a major blowup. Ironically, however, one correlation the brilliant minds of LTCM neglected to consider was the correlation coefficient of positions that were linked for no other reason than the fact 2.50 AAA Spread BAA Spread Yield (%) 2.00 Spreads Blow Out 1.50 1.00 0.50 4 Ju l-9 4 Oc t-9 4 Ja n95 Ap r-9 5 Ju l-9 5 Oc t-9 5 Ja n96 Ap r-9 6 Ju l-9 6 Oc t-9 6 Ja n97 Ap r-9 7 Ju l-9 7 Oc t-9 7 Ja n98 Ap r-9 8 Ju l-9 8 Oc t-9 8 Ja n99 Ap r-9 9 Ju l-9 9 Oc t-9 9 -9 Ap r Ja n- 94 0.00 FIGURE 2.13 Corporate Spreads to Treasuries, 1994–1999 Source: Bloomberg. 26 INSIDE THE HOUSE OF MONEY GREENSPAN ON LTCM How much dependence should be placed on financial modeling, which, for all its sophistication, can get too far ahead of human judgment?

How I Became a Quant: Insights From 25 of Wall Street's Elite by Richard R. Lindsey, Barry Schachter

Albert Einstein, algorithmic trading, Andrew Wiles, Antoine Gombaud: Chevalier de Méré, asset allocation, asset-backed security, backtesting, bank run, banking crisis, Bear Stearns, Black-Scholes formula, Bob Litterman, Bonfire of the Vanities, book value, Bretton Woods, Brownian motion, business cycle, business process, butter production in bangladesh, buy and hold, buy low sell high, capital asset pricing model, centre right, collateralized debt obligation, commoditize, computerized markets, corporate governance, correlation coefficient, creative destruction, Credit Default Swap, credit default swaps / collateralized debt obligations, currency manipulation / currency intervention, currency risk, discounted cash flows, disintermediation, diversification, Donald Knuth, Edward Thorp, Emanuel Derman, en.wikipedia.org, Eugene Fama: efficient market hypothesis, financial engineering, financial innovation, fixed income, full employment, George Akerlof, global macro, Gordon Gekko, hiring and firing, implied volatility, index fund, interest rate derivative, interest rate swap, Ivan Sutherland, John Bogle, John von Neumann, junk bonds, linear programming, Loma Prieta earthquake, Long Term Capital Management, machine readable, margin call, market friction, market microstructure, martingale, merger arbitrage, Michael Milken, Myron Scholes, Nick Leeson, P = NP, pattern recognition, Paul Samuelson, pensions crisis, performance metric, prediction markets, profit maximization, proprietary trading, purchasing power parity, quantitative trading / quantitative finance, QWERTY keyboard, RAND corporation, random walk, Ray Kurzweil, Reminiscences of a Stock Operator, Richard Feynman, Richard Stallman, risk free rate, risk-adjusted returns, risk/return, seminal paper, shareholder value, Sharpe ratio, short selling, Silicon Valley, six sigma, sorting algorithm, statistical arbitrage, statistical model, stem cell, Steven Levy, stochastic process, subscription business, systematic trading, technology bubble, The Great Moderation, the scientific method, too big to fail, trade route, transaction costs, transfer pricing, value at risk, volatility smile, Wiener process, yield curve, young professional

Our paper examined a corollary of their result: In the presence of money illusion, the correlation between stock and bond returns will be abnormally high during periods of high inflation. For the United States, it was shown that inflation had exactly this effect on stock/bond correlations during the postwar era. As a result, asset allocation strategies that are based on the high correlation coefficients calculated using data from the 1970s and early 1980s can be expected to generate inefficient portfolios in regimes of low inflation. JWPR007-Lindsey 82 May 7, 2007 16:44 h ow i b e cam e quant Ray LeClair and I wrote a paper, “Revenue Recognition Certificates: A New Security,” in which we explored the concept and potential benefits of a new type of security.14 This security provides returns as a specified function of a firm’s sales or gross revenues over a defined period of time, say 10 years, and then expires worthless.

Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms: Proceedings of the Agi Workshop 2006 by Ben Goertzel, Pei Wang

AI winter, artificial general intelligence, backpropagation, bioinformatics, brain emulation, classic study, combinatorial explosion, complexity theory, computer vision, Computing Machinery and Intelligence, conceptual framework, correlation coefficient, epigenetics, friendly AI, functional programming, G4S, higher-order functions, information retrieval, Isaac Newton, Jeff Hawkins, John Conway, Loebner Prize, Menlo Park, natural language processing, Nick Bostrom, Occam's razor, p-value, pattern recognition, performance metric, precautionary principle, Ray Kurzweil, Rodney Brooks, semantic web, statistical model, strong AI, theory of mind, traveling salesman, Turing machine, Turing test, Von Neumann architecture, Y2K

The ANEW list contains 1,034 words, 479 of which were found in the English core. The scatter plot of our PC #1 versus the first dimension of ANEW, which is the mean value of pleasure, is represented in Figure 7. The plot shows strong correlation, with similar bimodal distributions in both PC #1 and the ANEW-pleasure dimensions. Pearson correlation coefficient r = 0.70. Figure 7. Scatter plot demonstrating strong correlation of PC #1 with the first dimension of ANEW: pleasure. The dashed line is a linear fit. The two clusters (“positive” and “negative”) are separated in each dimension. How can we match PCs with ANEW dimensions? Our correlation analysis shows that PC #1 is the best match (i.e., most highly correlated among all PCs) for ANEWpleasure, and vice versa (r = 0.70, p = 10-70).

Mastering Private Equity by Zeisberger, Claudia,Prahl, Michael,White, Bowen, Michael Prahl, Bowen White

Alan Greenspan, asset allocation, backtesting, barriers to entry, Basel III, Bear Stearns, book value, business process, buy low sell high, capital controls, carbon credits, carried interest, clean tech, commoditize, corporate governance, corporate raider, correlation coefficient, creative destruction, currency risk, deal flow, discounted cash flows, disintermediation, disruptive innovation, distributed generation, diversification, diversified portfolio, family office, fixed income, high net worth, impact investing, information asymmetry, intangible asset, junk bonds, Lean Startup, low interest rates, market clearing, Michael Milken, passive investing, pattern recognition, performance metric, price mechanism, profit maximization, proprietary trading, risk tolerance, risk-adjusted returns, risk/return, Savings and loan crisis, shareholder value, Sharpe ratio, Silicon Valley, sovereign wealth fund, statistical arbitrage, time value of money, transaction costs, two and twenty

We spent a lot of time doing educational presentations for trustees and their consultants at offsite retreats, board meetings and pension conferences. During the 1980s, our hard work finally began to pay off. As we had actual data going back to 1972, we became pension funds’ source of information on expected returns, standard deviations and correlation coefficients for the private equity “asset class.” The new term “asset class” implied a transition from a niche activity to something that was becoming institutional. We took the lead in establishing the first industry performance benchmarks, chaired the committee that established the private equity valuation guidelines, and worked with the CFA Institute to establish the guidelines for private equity performance reporting.

Beautiful Data: The Stories Behind Elegant Data Solutions by Toby Segaran, Jeff Hammerbacher

23andMe, airport security, Amazon Mechanical Turk, bioinformatics, Black Swan, business intelligence, card file, cloud computing, computer vision, correlation coefficient, correlation does not imply causation, crowdsourcing, Daniel Kahneman / Amos Tversky, DARPA: Urban Challenge, data acquisition, data science, database schema, double helix, en.wikipedia.org, epigenetics, fault tolerance, Firefox, Gregor Mendel, Hans Rosling, housing crisis, information retrieval, lake wobegon effect, Large Hadron Collider, longitudinal study, machine readable, machine translation, Mars Rover, natural language processing, openstreetmap, Paradox of Choice, power law, prediction markets, profit motive, semantic web, sentiment analysis, Simon Singh, social bookmarking, social graph, SPARQL, sparse data, speech recognition, statistical model, supply-chain management, systematic bias, TED Talk, text mining, the long tail, Vernor Vinge, web application

(This relationship holds even if one controls for other predictors of roll call voting, such as nominee quality and ideological distance between the senator and the nominee.) The beauty of this graph is that it combines raw data with a simple inferential model in a single plot. Typically, bivariate relationships are presented in tabular form; in this example, doing so would require either nine correlation coefficients or regression coefficients and standard errors from nine regression models, which would be ungainly, make it difficult to visualize the relationship between opinion and voting for each nominee, and create difficulties in making comparisons across nominees. The only actual numbers we include BEAUTIFUL POLITICAL DATA Download at Boykma.Com 329 Pr(Voting Yes) Bork Rehnquist 1 1 .75 .75 .75 .5 .5 .5 .25 .25 .25 42−58 0 65−33 0 40 45 50 55 60 65 45 50 55 60 65 70 Pr(Voting Yes) 55 1 .75 .75 .75 .5 .5 .25 0 78−22 0 70 75 .25 65 Ginsburg 70 75 80 85 Breyer 70 75 80 85 90 O'Connor 1 1 .75 .75 .75 .5 .5 .5 .25 .25 .25 87−9 0 70 75 80 85 State Support for Nominee 80 90−9 65 1 96−3 75 0 60 0 70 .5 .25 52−48 65 Souter 1 65 60 Roberts 1 60 58−42 0 Thomas Pr(Voting Yes) Alito 1 99−0 0 70 75 80 85 90 88 90 92 State Support for Nominee All Nominees Pr(Voting Yes) 1 .75 .5 .25 0 40 50 60 70 80 90 State Support for Nominee F I G U R E 1 9 - 4 .

Good Money: Birmingham Button Makers, the Royal Mint, and the Beginnings of Modern Coinage, 1775-1821 by George Anthony Selgin

British Empire, correlation coefficient, flying shuttle, George Gilder, invention of the steam engine, Isaac Newton, James Watt: steam engine, large denomination, lone genius, profit motive, RAND corporation, school choice, seigniorage, The Wealth of Nations by Adam Smith, Tyler Cowen

Manufactured Copper Prices and Halfpenny Token Weights, 1787-1800 Year 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 Price of copper (d/lb) (Grenfell) (Tooke) 11 11 11 11 11 12 13 13 13 13 1.4 14 15 17 9.480 9.600 9.600 10.080 10.400 11.460 13.230 13.152 13.152 13.776 14.400 14.400 15.600 18.000 Average weight (ounces) Average "intrinsic value" (pence) (Tooke) (Grenfell) 0.499 0.499 0.452 0.448 0.450 0.419 0.372 0.346 0.339 0.334 0.351 0.370 0.361 0.275 0.296 0.299 0.271 0.282 0.293 0.300 0.307 0.285 0.279 0.287 0.316 0.333 0.352 0.309 0.343 0.343 0.311 0.308 0.309 0.314 0.302 0.281 0.275 0.271 0.307 0.324 0.338 0.292 12.595 0.394 Average 12.786 Correlation coefficients: -0.879 (Grenfell) -0.928 (Tooke) 0.301 0.309 SOUT(:es: Token weight~: Elks 2005. Copper prices: Thomas Tooke 1838,400 (average of reported quarterly prices); Grenfel1 1814. 146 GOOD MONEY well) have devoted so much effort and time to distinguishing specious tokens and mules from authentic issues and to documenting variants of authentic issues.

Data Wrangling With Python: Tips and Tools to Make Your Life Easier by Jacqueline Kazil

Amazon Web Services, bash_history, business logic, cloud computing, correlation coefficient, crowdsourcing, data acquisition, data science, database schema, Debian, en.wikipedia.org, Fairphone, Firefox, Global Witness, Google Chrome, Hacker News, job automation, machine readable, Nate Silver, natural language processing, pull request, Ronald Reagan, Ruby on Rails, selection bias, social web, statistical model, web application, WikiLeaks

We want to determine whether perceived government corruption and child labor rates are related. The first tool we’ll use is a simple Pearson’s correlation. agate is at this point in time working on building this correlation into the agate-stats library. Until then, you can correlate using numpy. Correlation coefficients (like Pearson’s) tell us if data is related and whether one variable has any effect on another. If you haven’t already installed numpy, you can do so by running pip install numpy. Then, calculate the correlation between child labor rates and perceived government corruption by running the following line of code: import numpy numpy.corrcoef(cpi_and_cl.columns['Total (%)'].values(), cpi_and_cl.columns['CPI 2013 Score'].values())[0, 1] We first get an error which looks similar to the CastError we saw before.

The Origins of the Urban Crisis by Sugrue, Thomas J.

affirmative action, business climate, classic study, collective bargaining, correlation coefficient, creative destruction, Credit Default Swap, deindustrialization, desegregation, Detroit bankruptcy, Ford paid five dollars a day, gentrification, George Gilder, ghettoisation, Gunnar Myrdal, hiring and firing, housing crisis, income inequality, indoor plumbing, informal economy, invisible hand, job automation, jobless men, Joseph Schumpeter, labor-force participation, low-wage service sector, manufacturing employment, mass incarceration, military-industrial complex, New Urbanism, oil shock, pink-collar, postindustrial economy, Quicken Loans, rent control, restrictive zoning, Richard Florida, Ronald Reagan, side project, Silicon Valley, strikebreaker, technological determinism, The Bell Curve by Richard Herrnstein and Charles Murray, The Chicago School, union organizing, upwardly mobile, urban planning, urban renewal, War on Poverty, white flight, working-age population, Works Progress Administration

“Ghetto” tracts, in contrast, were poorer—only five out of twenty-six tracts with a majority-black population in 1940 had incomes above the average for all blacks. “Infill” tracts (containing second-wave black newcomers) were split evenly between above- and below-average income. To offer a more precise statistical measure of impressionistic evidence about black residential stratification, the correlation coefficient (Pearson) was calculated for tract of percentage black population in 1940 and percentage change of black population with income in 1950. The results demonstrate a clear negative correlation between the income and percentage black in 1940 and income and increase in black population. Both correlations underscore the fact that transitional tracts—those that had smaller black populations in 1940 than in 1950, and those that gained a large number of blacks between 1940 and 1950—were those tracts that had the highest median incomes. 36.

The Aristocracy of Talent: How Meritocracy Made the Modern World by Adrian Wooldridge

"World Economic Forum" Davos, Ada Lovelace, affirmative action, Alan Greenspan, Albert Einstein, assortative mating, barriers to entry, Bernie Sanders, Black Lives Matter, Bletchley Park, borderless world, Boris Johnson, Brexit referendum, business intelligence, central bank independence, circulation of elites, Clayton Christensen, cognitive bias, Corn Laws, coronavirus, corporate governance, correlation coefficient, COVID-19, creative destruction, critical race theory, David Brooks, Dominic Cummings, Donald Trump, Double Irish / Dutch Sandwich, Etonian, European colonialism, fake news, feminist movement, George Floyd, George Gilder, Gini coefficient, glass ceiling, helicopter parent, Home mortgage interest deduction, income inequality, intangible asset, invention of gunpowder, invention of the printing press, Isaac Newton, Jeff Bezos, Jeremy Corbyn, Jim Simons, joint-stock company, Joseph Schumpeter, knowledge economy, knowledge worker, land tenure, London Interbank Offered Rate, Long Term Capital Management, Louis Pasteur, Mahatma Gandhi, Mark Zuckerberg, means of production, meritocracy, meta-analysis, microaggression, mortgage tax deduction, Myron Scholes, offshore financial centre, opioid epidemic / opioid crisis, Panopticon Jeremy Bentham, Peter Thiel, plutocrats, post-industrial society, post-oil, pre–internet, public intellectual, publish or perish, Ralph Waldo Emerson, RAND corporation, rent-seeking, Richard Florida, Ronald Reagan, scientific management, sexual politics, shareholder value, Sheryl Sandberg, Silicon Valley, spinning jenny, Steve Bannon, Steven Pinker, supply-chain management, surveillance capitalism, tech bro, The Bell Curve by Richard Herrnstein and Charles Murray, The Wealth of Nations by Adam Smith, Thorstein Veblen, three-martini lunch, Tim Cook: Apple, transfer pricing, Tyler Cowen, unit 8200, upwardly mobile, Vilfredo Pareto, W. E. B. Du Bois, wealth creators, women in the workforce

One of his first scientific papers, published in 1909, concluded with the triumphant statement that: ‘Parental intelligence … may be inherited, individual intelligence measured, and general intelligence analysed; and they can be analysed, measured and inherited to a degree which few psychologists have hitherto legitimately ventured to maintain.’36 Sixty years later, his last, and posthumously published paper, was devoted to proving that ‘the hypothesis of a general factor entering into every type of cognitive process’ and ‘the contention that differences in this general factor depend largely on the individual’s genetic constitution’ were ‘wholly consistent with the empirical facts’ and thus ‘beyond all question’.37 ‘Beyond all question’? In reality, Burt’s arguments were subjected to probing criticism throughout his life. His most perceptive critic was a fellow English psychologist, Godfrey Thomson. In 1916, Thomson demonstrated that the hierarchical arrangement of correlation coefficients could be explained by the laws of chance.38 He managed to produce a hierarchy from the results of a set of imitation ‘mental tests’ which can’t have had a common factor, since they were the throws of dice. The implication was that tests measured statistical abstractions rather than concrete entities.39 Leading American psychologists were equally sceptical about the idea of ‘g’.

Derivatives Markets by David Goldenberg

Black-Scholes formula, Brownian motion, capital asset pricing model, commodity trading advisor, compound rate of return, conceptual framework, correlation coefficient, Credit Default Swap, discounted cash flows, discrete time, diversification, diversified portfolio, en.wikipedia.org, financial engineering, financial innovation, fudge factor, implied volatility, incomplete markets, interest rate derivative, interest rate swap, law of one price, locking in a profit, London Interbank Offered Rate, Louis Bachelier, margin call, market microstructure, martingale, Myron Scholes, Norbert Wiener, Paul Samuelson, price mechanism, random walk, reserve currency, risk free rate, risk/return, riskless arbitrage, Sharpe ratio, short selling, stochastic process, stochastic volatility, time value of money, transaction costs, volatility smile, Wiener process, yield curve, zero-coupon bond, zero-sum game

We are assured by replication (and no-arbitrage, of course) that, H1=C1–Δ*S1=rB and therefore, H0=C0–Δ*S0=B. 17.4.1 Volatility of the Hedge Portfolio We now want to look at the volatility of the hedged portfolio, H, in another way in terms of its components. We write it generically as H=C–Δ*S and calculate its variance using our rule from portfolio analysis that says, where and ρX,Y is the correlation coefficient between X and Y defined by ρX,Y≡Cov(X,Y)/(σX*σY). Applying this rule to our hedge portfolio H we obtain, The interpretation of is the variance of the dollar returns on the option. Similarly, is the variance of the dollar returns on the underlying stock. Why dollar returns? We will demonstrate this shortly and also formulate the analysis in terms of percentage returns to the option, the stock, and the hedge.

The Art of UNIX Programming by Eric S. Raymond

A Pattern Language, Albert Einstein, Apple Newton, barriers to entry, bioinformatics, Boeing 747, Clayton Christensen, combinatorial explosion, commoditize, Compatible Time-Sharing System, correlation coefficient, David Brooks, Debian, Dennis Ritchie, domain-specific language, don't repeat yourself, Donald Knuth, end-to-end encryption, Everything should be made as simple as possible, facts on the ground, finite state, Free Software Foundation, general-purpose programming language, George Santayana, history of Unix, Innovator's Dilemma, job automation, Ken Thompson, Larry Wall, level 1 cache, machine readable, macro virus, Multics, MVC pattern, Neal Stephenson, no silver bullet, OSI model, pattern recognition, Paul Graham, peer-to-peer, premature optimization, pre–internet, publish or perish, revision control, RFC: Request For Comment, Richard Stallman, Robert Metcalfe, Steven Levy, the Cathedral and the Bazaar, transaction costs, Turing complete, Valgrind, wage slave, web application

In his paper, Graham noted accurately that computer programmers like the idea of pattern-matching filters, and sometimes have difficulty seeing past that approach, because it offers them so many opportunities to be clever. Statistical spam filters, on the other hand, work by collecting feedback about what the user judges to be spam versus nonspam. That feedback is processed into databases of statistical correlation coefficients or weights connecting words or phrases to the user's spam/nonspam classification. The most popular algorithms use minor variants of Bayes's Theorem on conditional probabilities, but other techniques (including various sorts of polynomial hashing) are also employed. In all these programs, the correlation check is a relatively trivial mathematical formula.

The Impact of Early Life Trauma on Health and Disease by Lanius, Ruth A.; Vermetten, Eric; Pain, Clare

autism spectrum disorder, classic study, cognitive load, conceptual framework, correlation coefficient, delayed gratification, epigenetics, false memory syndrome, Helicobacter pylori, impulse control, intermodal, longitudinal study, meta-analysis, Mount Scopus, Nelson Mandela, p-value, phenotype, randomized controlled trial, selective serotonin reuptake inhibitor (SSRI), social intelligence, Socratic dialogue, theory of mind, traumatic brain injury, twin studies, yellow journalism