How does the CPU know its instruction set?

The instruction set is hardwired into the CPU... It's actually a result of the way the circuit is built, there isn't an actual space where the instructions are kept.

When a computer starts, there's a small program on a ROM that knows how to initialize all the components and boot the OS. Because of the way a computer is built, this program is the first to run.

It sounds like you're conflating two different concepts:

Instruction set

An instruction set is the set of machine code instructions that the CPU is built to handle. The CPU doesn't have to load it from anywhere, it's built-in to the CPU itself. Examples of instruction sets are: Intel x86, MIPS.

Boot loader

The boot loader is a small program that tells the CPU how to get the computer started. Obviously it's written using the CPU instruction set, and usually resides in ROM or EEPROM or something. It tells the CPU how to interface with the disk to load a boot sector (which then loads more of the operating system). On a PC, the boot loader is part of the BIOS.

Short answer: The actual circuitry in the processor of the computer is what "determines" the instruction set.

Relatively short answer: The software that runs on the processor is physically just patterns of electrical signals. The transistors in the computer switch on and off many, many, many times a second, modifying these patterns of signals based on other signals. For example, consider the mov instruction found on just about every processor out there:

mov dest, src

This is encoded by a certain electrical signal pattern "stored" in memory. That signal pattern activates certain transistors on and off in such a way that the signals stored at dest matches that at src, because of how the circuitry is wired.

Long answer: Take a class in computer architecture. :-)

When we say a computer has a given instruction set, it refers to the set of instructions (operation codes) the processor can understand. A machine is digitally designed to understand the instructions it will be able to receive. When you design a CPU, one of the components is the decoder, which is the part of the CPU that decodes the electrical signals; and performs computations it was instructed to compute by those signals (the instruction). If you're interested in how computers work at the core, I suggest reading CODE or The Elements of a Computing Systems