Karhunen-Loeve Directions and Regression

Students sometimes ask about the difference between the regression line and the Karhunen–Loeve direction. The obvious answer a professor might give is that they are different animals! The object of this Demonstration is to give a more interesting answer.

The case of variables, and , is shown. It is assumed that and are bivariate normal with means zero, variances one, and covariance . For , the expected values for the regression lines on and on are shown together with the theoretical Karhunen–Loeve directions. For the estimated regressions and directions are shown along with a scatter plot of the data.