Rotation and Transformation of a plane to XY plane and origin in PointCloud

Do the translation first, then do the rotation.

Check the sign of theta

Eigen::Vector3f rotation_vector = xy_plane_normal_vector.cross(floor_plane_normal_vector);
float theta = -atan2(rotation_vector.norm(), xy_plane_normal_vector.dot(floor_plane_normal_vector));

In case anyone is interested

There were 2 problems in the code

  1. Rotation vector need to be normalized (just call rotation_vector.normalized())
  2. angle need to be negated (as suggested by previous answer).

Thank you for posting the code, it helped me to finish this quickly.