UTF-8 encoded html pages show � (questions marks) instead of characters

When [dropping] the encoding settings mentioned above all characters [are rendered] correctly but the encoding that is detected shows either windows-1252 or ISO-8859-1 depending on the browser.

Then that's what you're really sending. None of the encoding settings in your bullet list will actually modify your output in any way; all they do is tell the browser what encoding to assume when interpreting what you send. That's why you're getting those �s - you're telling the browser that what you're sending is UTF-8, but it's really ISO-8859-1.

In my case, database returned latin1, when my browser expected utf8.

So for MySQLi I did:

 mysqli_set_charset($dblink, "utf8");    

See http://php.net/manual/en/mysqli.set-charset.php for more info