Equations For Quadratic Regression

For $y =\beta_1 +\beta_2 x +\beta_3 x^2$, let's define $$x_1 = x$$ and $$x_2= x^2.$$ Now we can use the equations of multiple linear regression: $$S_{11}= \sum_{n=1}^N x_1^2- \frac{(\sum_{n=1}^N x_1)^2}{N}$$ $$S_{12}= \sum_{n=1}^N x_1x_2- \frac{(\sum_{n=1}^N x_1\sum_{n=1}^N x_2)}{N}$$ $$S_{22}= \sum_{n=1}^N x_2^2- \frac{(\sum_{n=1}^N x_2)^2}{N}$$ $$S_{y1}= \sum_{n=1}^N y x_1- \frac{(\sum_{n=1}^N y\sum_{n=1}^N x_1)}{N}$$ $$S_{y2}= \sum_{n=1}^N y x_2- \frac{(\sum_{n=1}^N y\sum_{n=1}^N x_2)}{N}$$ $$\overline{x}_1 = \frac{(\sum_{n=1}^N x_1)}{N}$$ $$\overline{x}_2 = \frac{(\sum_{n=1}^N x_2)}{N}$$ $$\overline{y} = \frac{(\sum_{n=1}^N y)}{N}$$ $$\beta_2=\frac{S_{y1}S_{22}-S_{y2}S_{12}}{S_{22}S_{11}-S_{12}^2}$$ $$\beta_3=\frac{S_{y2}S_{11}-S_{y1}S_{12}}{S_{22}S_{11}-S_{12}^2}$$ $$\beta_1=\overline{y}-\beta_2\overline{x}_1-\beta_3\overline{x}_2$$

Assuming $y_j =\beta_1 +\beta_2 x_j +\beta_3 x_j^2 + \epsilon_j$ for $j = 1\dots n$, we can write the matrix equation

$$\mathbf{Y}= \mathbf{XB} + \mathbf{E}$$

where

$$\mathbf Y = [y_1 \ y_2 \ \dots \ y_n]^T,$$

$$ \mathbf X = \left[ \begin{array}{c} 1 & x_1 & x_1^2 \\ 1 & x_2 & x_2^2 \\ \vdots & \vdots & \vdots \\ 1 & x_n & x_n^2 \\ \end{array} \right], $$

$$\mathbf B = [\beta_1 \ \beta_2 \ \beta_3]^T,$$

and

$$\mathbf E = [e_1 \ e_2 \ \dots \ e_n]^T$$

We seek the vector $\mathbf B$ that minimizes

$$e_1^2 + 3_2^2 + \cdots + e_n^2 = \mathbf E^T \mathbf E$$

Assuming that $\mathbf X^T \mathbf X$ is nonsingular (usually a safe assumption), we find that

$$\mathbf B = (\mathbf X^T \mathbf X)^{-1} \mathbf X^T \mathbf Y$$