Regression

Recommended Literature:
If you want to read up further on this topic, the following material is recommended. Bishop. Pattern recognition and machine learning. 2006. Sections 3.1 Linear Basis Function Models and 3.2. Bias-Variance Decomposition

Friedman, Hastie, and Tibshirani. The elements of statistical learning. 2001 Sections 2.6 Statistical Models, Supervised Learning and Function Approximations and Section 2.9 Model Selection and Bias-Variance Tradeoff

Section 3 Linear Methods for Regression (in particular Subsections 3.1 and 3.2)