MySQL utf8mb4 on Amazon RDS: global variables set correctly but variables not set

To clarify Aaron's answer, this was (at least for us) caused by the character set/collation of our connection to the database; the database itself is set up correctly. When you use any client to connect to the db -- whether it be SQLyog, MySQL Workbench, built in client or any other -- there is a character set and collation associated with that connection. Thus you need to need to change this connection charset/collation to utf8mb4 and utf8mb4_unicode_ci, and the values of

SHOW VARIABLES WHERE Variable_name LIKE 'character\_set\_%' OR Variable_name LIKE 'collation%';

will then all show correctly. If you take a close look at the "problem" values in the original question you'll notice they are _client, _connection etc. which should have given me an obvious clue that the problem was with my mysql client and not the database itself.