Why is linear regression called 'linear'?

Linear regression is called linear because you model your output variable (lets call it f(x)) as a linear combination of inputs and weights (lets call them x and w respectively). Namely

f(x) = < w, x > + b = SUM w_i x_i + b

From strict mathematical perspective these models should be called affine models, due to the existance of bias (b term, which should not occur in linear transformation), but due to historical reasons we use the name linear instead.

You can consider linear regression on top of non-linear transformation of your data, but this is just a semantic trick, as efficiently this is then linear regression of transformed data, not data itself (in terms of data the fitted regression is not linear). From mathematical perspective it is important that linearity is preserved wrt. to whatever you optimize, so in terms of w. So if you consider

f(x) = <w, g(x)> + b 

You can still call it linear regression (as it is linear wrt. to w) but it is no longer linear regression applied to x, but applied to g(x).


Referring this answer I think it is with respect to the parameters theta. Even though the the wiki link on Linear Regression starts with the following statement:

Given a data set of n statistical units, a linear regression model assumes that the relationship between the dependent variable yi and the p-vector of regressors xi is linear.

However after a couple of lines they mention:

Sometimes one of the regressors can be a non-linear function of another regressor or of the data, as in polynomial regression and segmented regression. The model remains linear as long as it is linear in the parameter vector β.

So I guess the linearity is not in terms of x but the parameters.