How to convert spherical coordinates to equirectangular projection coordinates?

hmm, i think maybe you should go one step back. Consider your camera angle (70mm ? or so). but your background image is a 360 degree in horizontal (but also vertical). Consider the perspective distortions on both type of pictures. For the background pict, in a vertical sense only the horizon is not vertically distorted. Sadly its only a thin line. As the distortion increases the more you get to the top or bottom.

Its not constant as in barrel distortion, but it depends on vertical distance of horizon.

I think the best way to realize the difference is to take a side view of both type of camera's and the target they supposed to project upon, from there its trigonometry, math.

Note that for the 70mm picture you need to know the angle it was shot. (or estimate it)

If photos are taken from a fixed point, and the camera can only rotate its yaw and pitch around that point. Then we can consider a sphere of any radius (for the math, it is highly recommended to use a radius of 1). The photo will be a rectangular shape on this sphere (from perspective of the camera).

Horizon-case

If you are looking at the horizon (equator), then vertical pixels account for latitude, and horizontal pixels account for longitude. For a simple panorama photo of the horizon there is not much of a problem:

Here we look at roughly the horizon of our world. That is, the camera has angle va = ~0. Then this is pretty straightforward, because if we know that the photo is 70 degrees wide and 40 degrees high, then we also know that the longitude range will be approximately 70 degrees and latitude range 40 degrees.

If we don't care about a slight distortion, then the formula to calculate the (longitude,latitude) from any pixel (x,y) from the photo would be easy:

photo_width_deg = 70
photo_height_deg = 30
photo_width_px = 1280
photo_height_px = 720
ha = 0
va = 0
longitude = photo_width_deg * (x - photo_width_px/2) / photo_width_px + ha
latitude = photo_height_deg * (y - photo_height_px/2) / photo_height_px + va

Problem

But this approximation does not work at all when we move the camera much more vertically:

So how do we transform a pixel from the picture at (x, y) to a (longitude, latitude) coordinate given a vertical/horizontal angle at which the photo was taken (va,ha)?

Solution

The important idea that solved the problem for me is this: you basically have two spheres:

• The photo-sphere with the camera in the center.
• The geo-sphere (equirectangular projection sphere), with longitude/latitude coordinates.

You know the spherical coordinate of a point on the photo-sphere, and you want to know where this point is on the geo-sphere with the different camera-angle.

The real problem

We have to realize that it is difficult to do any calculations between the two spheres using just spherical coordinates. The math for the cartesian coordinate system is much simpler. In the cartesian coordinate system we can easily rotate around any axis using rotation matrices that are multiplied with the coordinate vector [x,y,z] to get the rotated coordinate back.

Warning: Here it is very important to know that there are different conventions with regard to the meaning of x-axis, y-axis, and z-axis. It is uncertain which axis is the vertical one, and which one points where to. You just have to make a drawing for yourself and decide on this. If the result is wrong, it's probably because these are mixed up. The same goes for the theta and phi for spherical coordinates.

The real solution

So the trick is to transform from photo-sphere to cartesian, then apply the rotations, and then go back to spherical coordinates:

1. Take any pixel on the photo, and calculate the relative degrees it is away from the center of the photo both horizontally and vertically.
2. Transform the photo-sphere spherical coordinates into cartesian coordinates ([x,y,z] vectors).
3. Apply rotation matrices to the coordinate just like the camera was rotated (ha,va).
4. Transform the cartesian coordinates back to spherical coordinates, and these will be your longitude and latitude.

Example code

// Photo resolution
double img_w_px = 1280;
double img_h_px = 720;
// Camera field-of-view angles
double img_ha_deg = 70;
double img_va_deg = 40;
// Camera rotation angles
double hcam_deg = 230;
double vcam_deg = 60;
// Camera rotation angles in radians
double hcam_rad = hcam_deg/180.0*PI;
double vcam_rad = vcam_rad/180.0*PI;
// Rotation around y-axis for vertical rotation of camera
Matrix rot_y = {
    cos(vcam_rad), 0, sin(vcam_rad),
    0, 1, 0,
    -sin(vcam_rad), 0, cos(vcam_rad)
};
// Rotation around z-axis for horizontal rotation of camera
Matrix rot_z = {
    cos(hcam_rad), -sin(hcam_rad), 0,
    sin(hcam_rad), cos(hcam_rad), 0,
    0, 0, 1
};

Image img = load('something.png');
for(int i=0;i<img_h_px;++i)
{
    for(int j=0;j<img_w_px;++j)
    {
        Pixel p = img.getPixelAt(i, j);

        // Calculate relative position to center in degrees
        double p_theta = (j - img_w_px / 2.0) / img_w_px * img_w_deg / 180.0 * PI;
        double p_phi = -(i - img_h_px / 2.0) / img_h_px * img_h_deg / 180.0 * PI;

        // Transform into cartesian coordinates
        double p_x = cos(p_phi) * cos(p_theta);
        double p_y = cos(p_phi) * sin(p_theta);
        double p_z = sin(p_phi);
        Vector p0 = {p_x, p_y, p_z};

        // Apply rotation matrices (note, z-axis is the vertical one)
        // First vertically
        Vector p1 = rot_y * p0;
        Vector p2 = rot_z * p1;

        // Transform back into spherical coordinates
        double theta = atan2(p2[1], p2[0]);
        double phi = asin(p2[2]);

        // Retrieve longitude,latitude
        double longitude = theta / PI * 180.0;
        double latitude = phi / PI * 180.0;

        // Now we can use longitude,latitude coordinates in many different projections, such as:
        // Polar projection
        {
            int polar_x_px = (0.5*PI + phi)*0.5 * cos(theta) /PI*180.0 * polar_w;
            int polar_y_px = (0.5*PI + phi)*0.5 * sin(theta) /PI*180.0 * polar_h;
            polar.setPixel(polar_x_px, polar_y_px, p.getRGB());
        }
        // Geographical (=equirectangular) projection
        {
            int geo_x_px = (longitude + 180) * geo_w;
            int geo_y_px = (latitude + 90) * geo_h;
            geo.setPixel(geo_x_px, geo_y_px, p.getRGB());
        }
        // ...
    }
}

Note, this is just some kind of pseudo-code. It is advised to use a matrix-library that handles your multiplications and rotations of matrices and vectors.