How do you make a matrix out of vectors in eigen?

You can also append them using the comma initializer syntax:

m << v1, v2, v3, v4;

The matrix m must have been properly resized first.

A quick check on the docs:

Vector4f v1(1,0,0,0);
Vector4f v2(0,1,0,0);
Vector4f v3(0,0,1,0);
Vector4f v4(0,0,0,1);
Matrix4f m;

m.row(0) = v1;
m.row(1) = v2;
m.row(2) = v3;
m.row(3) = v4;

std::cout << m << std::endl;

output:

1,0,0,0
0,1,0,0
0,0,1,0
0,0,0,1