Is it okay for me to show some other publication's data for data comparison reasons without prior permission if I properly reference/cite their work?

Utilization of data sets (acquired from other academic resources) is fairly common; specially when you need to compare your analytical method with the other one's in view of performance, precision, etc. In such case, people often try to use a pre-assessed data set of their own or some other study. All you need to do is referencing the used data set, carefully.

Answer to question #11, here, says:

Data cannot be copyrighted, so you are free to use data to create any figure you like *[or any analysis you need]*. The source of the data must be properly acknowledged