Is it better to encrypt a message and then compress it or the other way around? Which provides more security?
There is no difference in the security provided, but because of the way compression algorithms work, you are probably going to get better compression if you compress first then encrypt.
Compression algorithms exploit statistical redundancies (such as those that exist in natural language or in many file formats) in the data which should be eliminated when you encrypt it, therefore an encrypted message shouldn't be able to be compressed all that well.
From the wikipedia article:
However, lossless data compression algorithms will always fail to compress some files; indeed, any compression algorithm will necessarily fail to compress any data containing no discernible patterns. Attempts to compress data that has been compressed already will therefore usually result in an expansion, as will attempts to compress all but the most trivially encrypted data.
Warning: if an attacker controls part of the plaintext that gets compressed, and can observe the size of the resulting encrypted ciphertext, they may be able to deduce the rest of the plaintext, by adjusting the part that they control until the length of the ciphertext decreases (which implies that there was some repetition between the part of the plaintext they control and the secret part of the plaintext).
See https://en.wikipedia.org/wiki/CRIME for example.
Encryption works better on short messages, with a uniform distribution of symbols. Compression replaces a message with a non-uniform distribution of symbols by another, shorter sequence of symbols that are more uniformly distributed.
Therefore, it's mathemathically safer to compress before encryption. Compression after encryption doesn't affect the encryption, which remains relatively weak due to the non-uniform distribution of plaintext.
Of course, if you use anything like AES256, and the NSA isn't after you, this is all theory.