ALS model - how to generate full_u * v^t * v?
This new user is not the the matrix U, so you don't have its latent representation in 'k' factors, you only know its full representation, i.e., all its ratings. full_u here means all of the new user ratings in a dense format (not the sparse format ratings
are) e.g.:
[0 2 0 0 0 1 0]
if user u has rated item 2 with a 2 and item 6 with a 1.
then you can get v
pretty much like you did and turning it to a matrix in numpy for instance:
pf = model.productFeatures()
Vt = np.matrix(np.asarray(pf.values().collect()))
then is is just a matter of multiplying:
full_u*Vt*Vt.T
Vt
and V
are transposed compared to the other answer but that's just a matter of convention.
Note that the Vt*Vt.T
product is fixed, so if you are going to use this for multiple new users it will be computationally more efficient to pre-compute it. Actually for more than one user it would be better to put all their ratings in bigU (in the same format as my one new user example) and just do the matrix product:
bigU*Vt*Vt.T
to get all the ratings for all the new users. Might still be worth checking that the product is done in the most efficient way in terms of number of operations.
Just a word of warning. People talk about the user and product matrices like they are left and right singular vectors. But as far as I understand, the method used to find U and V is an optimization of a straight squared error cost function, which makes none of the orthogonality guarantees of SVD.
In other words, think algebraically about what the above answer claims. We have a full ratings matrix R, an n by p matrix of ratings for n users over p products. We decompose it with k latent factors to approximate R = UV, where the rows of U, an n by k matrix, are the latent user representations, and the columns of V, a k by p matrix, are the latent product representations. In order to find latent user representations for a matrix R of entirely new users without refitting the model, we need:
R = U V
R V^{-1} = U V V^{-1}
R V^{-1} = U I_{k}
R V^{-1} = U
where I_{k} is the k dimensional identity matrix and V^{-1} is the p by k right inverse of V. The tip above assumes that V^{T} = V^{-1}. This is not guaranteed. And in general there is no guarantee that assuming this is true will give you anything but nonsense answers.
Let me know if I'm missing something in the optimization method behind MLLib's CF implementation. Is there a trick in the ALS model that guarantees orthogonality that I'm missing?