What exactly is prepreg and core in a PCB?

The important difference is this.

Core is a layer of FR4 with copper either side, that's made in a core factory. The layer of FR4 is formed between two smooth foils of copper, to a specified thickness.

Pre-preg is a layer of uncured FR4, that's used by PCB manufacturers to glue together etched cores, or a copper foil to an etched core. This means the thickness of the prepreg varies with the height of the etched boards either side of it.

For applications where the dielectric's physical properties are important (as in high-frequency transmission lines and antennae) you get much better repeatability with signal and ground either side of a core, than if the fields go across pre-preg.

Choosing which layers are made in which way can affect the processing steps and so costs if you are building a board with buried vias. It's easy to drill holes through cores to get buried vias, but this restricts which layers can connect to which.

A core is a thick, more rigid layer of glass fiber while a prepreg is a thin layer of glass fiber/copper laminated onto a core. In the past, there only ever was one thick core, so the distinction made a lot more sense than today, when they are roughly the same thickness.

There's still a difference in how vias are handled but you better refer to the complete stackup in question instead of making assumptions about how ''core'' and ''prepreg'' vias differ.

From this link:

Prepreg, which is an abbreviation for pre impregnated, is a fibre weave impregnated with a resin bonding agent. It is used to stick the core layers together. The core layers being FR4 with copper traces. The layer stack is pressed together at temperature to the required board finish thickness. Prepreg comes in different thicknesses.