What is the purpose of the 'figure' tag in html?

The reason to use a tag is because it is HTML5 semantic. It does not have class traits assigned to it. As the other answers have noted, you can remove it or replace with a div and it really wont change anything. However if you are writing good 'semantic' code, which defines meaning in its tags then this is one of the standard tags you would include.
It is similar to or and other semantic tags which define your code in readable way to the browser and to the developer.

Good question.

It all goes down to a few things - semantic meaning, SEO, accessibility (and maybe more…)

What is it?

The element is a semantic tag that just came out in HTML5. This tells the browser that is a container that holds elements that have a relation to each other. The plays a role to this and it will be explained below.

Also, note that it does not have to contain just element, it can contain other elements such as tables, video clip, audio clips, etc

Why should we use it?

Before HTML5, there was no element for captions for images, diagram, video and so on. It will have a semantic meaning now with element. The will help the search engine to find these images.

And with semantics, your SEO improves and the bots will find your image faster on search engine with it and show it in results.

Accessbility

With figure and figcaption tags, the user (visually-impaired user) will be able to understand the structure (with the help of the screen reader). The screen reader bot will inform the user that this is a caption content, so if it is placed right after or before the related images, the user will be able to understand it.

Answer inspired by: https://forum.freecodecamp.org/t/why-use-the-figure-element/315116/3

The <figure> element is intended to be used in conjunction with the element to mark up diagrams, illustrations, photos, and code examples (among other things). The spec says this about <figure>:

W3C

The figure element represents a unit of content, optionally with a caption, that is self-contained, that is typically referenced as a single unit from the main flow of the document, and that can be moved away from the main flow of the document without affecting the document’s meaning.

MDN

The HTML <figure> element represents self-contained content, frequently with a caption <figcaption>, and is typically referenced as a single unit. While it is related to the main flow, its position is independent of the main flow. Usually this is an image, an illustration, a diagram, a code snippet, or a schema that is referenced in the main text, but that can be moved to another page or to an appendix without affecting the main flow.