Cannot understand with sklearn's PolynomialFeatures
If you have features [a, b, c]
the default polynomial features(in sklearn
the degree is 2) should be [1, a, b, c, a^2, b^2, c^2, ab, bc, ca]
.
2.61576000e+03
is 37.8x62.2=2615,76
(2615,76 = 2.61576000 x 10^3
)
In a simple way with the PolynomialFeatures
you can create new features. There is a good reference here. Of course there are and disadvantages("Overfitting") of using PolynomialFeatures
(see here).
Edit:
We have to be careful when using the polynomial features. The formula for calculating the number of the polynomial features is N(n,d)=C(n+d,d)
where n
is the number of the features, d
is the degree of the polynomial, C
is binomial coefficient(combination). In our case the number is C(3+2,2)=5!/(5-2)!2!=10
but when the number of features or the degree is height the polynomial features becomes too many. For example:
N(100,2)=5151
N(100,5)=96560646
So in this case you may need to apply regularization to penalize some of the weights. It is quite possible that the algorithm will start to suffer from curse of dimensionality (here is also a very nice discussion).
PolynomialFeatures generates a new matrix with all polynomial combinations of features with given degree.
Like [a] will be converted into [1,a,a^2] for degree 2.
You can visualize input being transformed into matrix generated by PolynomialFeatures.
from sklearn.preprocessing import PolynomialFeatures
a = np.array([1,2,3,4,5])
a = a[:,np.newaxis]
poly = PolynomialFeatures(degree=2)
a_poly = poly.fit_transform(a)
print(a_poly)
Output:
[[ 1. 1. 1.]
[ 1. 2. 4.]
[ 1. 3. 9.]
[ 1. 4. 16.]
[ 1. 5. 25.]]
You can see matrix generated in form of [1,a,a^2]
To observe polynomial features on scatter plot, let's use number 1-100.
import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.preprocessing import PolynomialFeatures
#Making 1-100 numbers
a = np.arange(1,100,1)
a = a[:,np.newaxis]
#Scaling data with 0 mean and 1 standard Deviation, so it can be observed easily
scaler = StandardScaler()
a = scaler.fit_transform(a)
#Applying PolynomialFeatures
poly = PolynomialFeatures(degree=2)
a_poly = poly.fit_transform(a)
#Flattening Polynomial feature matrix (Creating 1D array), so it can be plotted.
a_poly = a_poly.flatten()
#Creating array of size a_poly with number series. (For plotting)
xarr = np.arange(1,a_poly.size+1,1)
#Plotting
plt.scatter(xarr,a_poly)
plt.title("Degree 2 Polynomial")
plt.show()
Output:
Changing degree=3 ,we get:
You have 3-dimensional data and the following code generates all poly features of degree 2:
X=np.array([[230.1,37.8,69.2]])
from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures()
X_poly=poly.fit_transform(X)
X_poly
#array([[ 1.00000000e+00, 2.30100000e+02, 3.78000000e+01,
# 6.92000000e+01, 5.29460100e+04, 8.69778000e+03,
# 1.59229200e+04, 1.42884000e+03, 2.61576000e+03,
# 4.78864000e+03]])
This can also be generated with the following code:
a, b, c = 230.1, 37.8, 69.2 # 3-dimensional data
np.array([[1,a,b,c,a**2,a*b,c*a,b**2,b*c,c**2]]) # all possible degree-2 polynomial features
# array([[ 1.00000000e+00, 2.30100000e+02, 3.78000000e+01,
6.92000000e+01, 5.29460100e+04, 8.69778000e+03,
1.59229200e+04, 1.42884000e+03, 2.61576000e+03,
4.78864000e+03]])