Stitch multiple images using OpenCV (Python)

Hacky approach

The easiest way (though not super efficient) given the functions you've written, is to just grow the panorama image by stitching it with each successive image. Something like this pseudocode:

panorama = images[0]
for i in 1:len(images)-1
    panorama = stitch(panorama,images[i])

This method basically attempts to match the next image to any part of the current panorama. It should work decently well, assuming each new image is somewhere on the border of the current panorama, and there isn't too much perspective distortion.

Mathematical approach

The other option, if you know the order that you want to stitch, is to find the Homography from one image to the next, and then multiply them. The result is the Homography from that image to image 0.

For example: the H that transforms image 3 to line up with image 0 is H_03 = H_01 * H_12 * H_23. Where H_01 is the H that transforms image 1 to line up with image 0. (Depending on the way their code defines H, you might need to reverse the above multiplication order.) So you would multiply to obtain H_0i and then use it to transform image i to line up with image 0.

For background on why you multiply the transformations, see: Transformations and Matrix Multiplication specifically the "Composition of tranformations" part.

I had the similar problem with gaps between images. The first thing you should do is to init your accumulated homography matrix to identity at first frame. Then, with every new frame you should multiply it by homography matrix between current and next frame. Be aware to use numpy matrices and not numpy arrays. IDK why but they have different multiplication routines.

Here is my code:

def addFramePair(self, images, ratio=0.75, reprojThresh=4.0, showMatches=False):        
    (imageA, imageB) = images
    (kpsA, featuresA) = self.detectAndDescribe(imageA)
    (kpsB, featuresB) = self.detectAndDescribe(imageB)

    H = self.matchKeypoints(kpsA, kpsB, featuresA, featuresB, ratio, reprojThresh)
    self.accHomography *= np.asmatrix(H)
    result = cv2.warpPerspective(imageA, np.linalg.inv(self.accHomography), (1600, 900))
    return result

imageA is current, imageB is the next one.

Hope this helps.

Step by step, assuming you want to stitch four images I0, I1, I2, I3, your goal is to compute homographies H_0, H_1, H_2, H_3;

1. Compute all pairwise homographies H_01, H_02, H_03, H_12, H_13, H_23 where homography H_01 warps image I0 into I1, etc...
2. Select one anchor image e.g. I1 which position will remain fixed i.e H_1 = Identity
3. Find image that better align with I1 based on maximum number of consistent matches e.g. I3
4. Update H_3 = H_1 * inv(H_13) = inv(H_13) = H_31
5. Find image that better matches I1 or I3 e.g I2 matching I3
6. Update H_2 = H_3 * H_23
7. Same as above for image I0
8. Do bundle adjustment to globally optimize alignment

See section 4 of this seminal paper Automatic Panoramic Image Stitching using Invariant Features for an in depth explanation.