Is a crash always exploitable for RCE?

Many crashes aren't exploitable for anything except denial-of-service (DoS). The most common example would be a NULL pointer read; attempting to dereference a pointer to (or anywhere near) 0 will fail, and unless the exception/signal is caught will cause the program to crash. However, just because the program tries to read from zero when given invalid input doesn't mean that it'll do anything more exciting if given a different invalid input.

Determining the exploitability of a crash is tricky. There's a few heuristics you can use (writes are usually worse than reads, if the instruction pointer is pointing at non-executable code - a NX/DEP violation - that's almost certainly exploitable, if it's a null pointer read then that's pretty unlikely to be exploitable) but they're not perfectly accurate. There are tools that will analyze a crash dump and try to guess for you (for example, Microsoft published a "!exploitable" command for the Windows debugger windbg that will perform this analysis; there are similar tools for Linux debuggers). To know for sure, you need to analyze the program flow that caused the memory corruption and see how much influence you have over it, and whether you can make it do anything actually dangerous.

Consider a segfault caused by trying to access memory location 0x0 (ie a null pointer de-reference).

Maybe one part of the code thought it was done with a variable and set the pointer to null, and maybe under normal operations it really is done with it, but fuzz testing found some edge-case where something tries to read that variable again after it was set to null.

That will certainly cause a crash, because your application is trying to read kernel memory, and the kernel gets angry about that, but won't allow you to do anything RCE-like.