How does copyright work on screen captures?

You need legal advice. Talk to an attorney.

Perhaps your institution has one that you can take advantage of. Everything from here on out is rank speculation, and even your attorney's advice may not protect your from suit.

If you can, get permission from each source you want to take a screen capture of. There may be copyright protectable elements in those screen shots. There may also be trademarked elements.

If you cannot get permission for whatever reason, you may have to rely on fair use. Fair use is a defense to an infringement suit not a get out of jail free card. By using a copyright or trademark protected work without a license, you run the risk of lawsuit. Your institution may not be willing to rely on a fair use defense. Also, your publisher may not be willing to rely on a fair use defense (since they will be the lawsuit target).

I personally think your use is without question a classic and canonical example of fair use, but that doesn't mean that you will prevail in publishing without permission. Many venues require that your sign a form saying that you have the copyright or permission to use all your images, so be careful.