Fastest way to detect the non/least-changing pixels of successive images

An approach is to compare each frame-by-frame using cv2.bitwise_and(). The idea is that pixels in the previous frame must be present in the current frame to be a non-changing pixel. By iterating through the list of frames, all features in the scene must be present in the previous and current frame to be considered a non-moving item. So if we sequentially iterate through each frame, the last iteration will have shared features from all previous frames.

Using this set of frames captured once per second

We convert each frame to grayscale then cv2.bitwise_and() with the previous and current frame. The non-changing pixels of each successive iteration are highlighted in gray while changing pixels are black. The very last iteration should showcase pixels shared between all frames.

If instead you also thresholded each frame, you get a more pronounced result

import cv2
import glob

images = [cv2.imread(image, 0) for image in glob.glob("*.png")]

result = cv2.bitwise_and(images[0], images[1])
for image in images[2:]:
    result = cv2.bitwise_and(result, image)

cv2.imshow('result', result)
cv2.waitKey(0)

It is possible to compute variance and standard deviation from sum and sum of squares.

VAR X = EX^2 - (EX)^2

See link https://en.wikipedia.org/wiki/Variance#Definition

Sum and Sum of squares can be updates sequentially by adding a new image and subtracting an image captures n_of_frames ago. Next compute a variance and take a square root to get standard deviation. Note that computation time does not depend on number of frames.

See the code

import math
import cv2
import numpy as np


video = cv2.VideoCapture(0)
previous = []
n_of_frames = 200

sum_of_frames = 0
sumsq_of_frames = 0

while True:
   ret, frame = video.read()
   if ret:
      cropped_img = frame[0:150, 0:500]
      gray = cv2.cvtColor(cropped_img, cv2.COLOR_BGR2GRAY)
      gray = gray.astype('f4')
      if len(previous) == n_of_frames:
         stdev_gray = np.sqrt(sumsq_of_frames / n_of_frames - np.square(sum_of_frames / n_of_frames))
         cv2.imshow('stdev_gray', stdev_gray * (1/255))
         sum_of_frames -= previous[0]
         sumsq_of_frames -=np.square(previous[0])
         previous.pop(0)
      previous.append(gray)
      sum_of_frames = sum_of_frames + gray
      sumsq_of_frames = sumsq_of_frames + np.square(gray)

      #cv2.imshow('frame', frame)

      key = cv2.waitKey(1)
      if key == ord('q'):
         break

video.release()
cv2.destroyAllWindows()

Result looks pretty awesome.