How do I include multiple paragraphs in the "remarks" section using Doxygen?

Your final example code, that is

/// @remarks
///     Note that this function is reentrant, but not thread-safe!
/// @remarks
///     If thread safety is required, the user must handle locking and unlocking
///     the region manually.

is exactly the expected use of \remarks for multi-paragraph comment blocks. From the doxygen manual (emphasis mine):

\remark { remark text }

Starts a paragraph where one or more remarks may be entered. The paragraph will be indented. The text of the paragraph has no special internal structure. All visual enhancement commands may be used inside the paragraph. Multiple adjacent \remark commands will be joined into a single paragraph. Each remark will start on a new line. Alternatively, one \remark command may mention several remarks. The \remark command ends when a blank line or some other sectioning command is encountered.

So \remark (and \remarks, which is just the same as \remark) ends at the end of a paragraph, but adjacent \remarks will be stitched together to form one \remark block.

You are right to say that this behaviour is not limited to \remarks and \remark. The same applies to any command that takes a paragraph of text as an argument, see, for example, \bug, \todo, \warning etc.

If you don't like having multiple @remarks lines in your source for your multi-paragraph remarks, I think you can use @parblock & @endparblock to include multiple paragraphs for a single @remark.

One solution that seems to work that I haven't seen mentioned here, is to end your lines with \n. This will keep everything grouped together, while putting in the white space you may want to see.

If you want a blank line, make sure the line above has \n, and you have a blank line with \n on it.

In your example, you would want:

/// @remarks
///     Note that this function is reentrant, but not thread-safe!\n
///     \n
///     If thread safety is required, the user must handle locking and unlocking
///     the region manually.

And that should make it appear as you want it to appear.