Polar transform in CSS3?
I never worked with CSS transform matrices, but I think what you want is not possible. Using transform matrices you do a linear transformation. Linear transformations ALWAYS map a straight line to a straight line or to 0. Take a look at Wikipedia for more information.
So it's impossible to map a straight line to a circle using matrices.