Why does ` (backtick) not appear until after I press another key?

This is either your keyboard or the software you are using (we can't see that from here). So it is not specific to StackExchange sites.

This is done because the software supports another feature: pressing the apostroph or the backtick followed by another letter (most common a vowel) is an easy way to write diacritical characters like é and è.

Because of this feature the software cannot immediately display the apostroph or backtick, it has to wait for the next character to determine what your intention is. Only when you press the next backtick it knows that you intended to enter a backtick.

This is not bad in itself: just remember to type the backtick twice, then press cursor left, then type what comes between them. That way you won't forget the closing backtick.

This is akin to typing the { } brackets in the C programming language first before entering the code that goes in between. There, having a closing bracket at the correct location is much more important.

This is not only done with apostroph and backtick, but e.g. also with double quotation marks (" and e gives ë) and even with consonants (, and C gives Ç).

This is called a dead key. https://en.wikipedia.org/wiki/Dead_key

It depends on your local settings (i.e. Keyboard layout).

Other answers have mentioned that this can be a keyboard layout thing, but I felt that a concrete example might be useful, so:

On Windows I generally use the "English (UK) Extended" keyboard layout, which has better support for typing accented characters, but means I have to press backtick twice to get the ` character. If I switch to the basic "English (UK)" keyboard layout, I only have to press backtick once. So the latter keyboard is perhaps better for some uses (such as coding), even though the extended version is better for others (such as writing).