Java heap dump and the heap size after the heap analysis differs

I have experienced similar situation. The difference (HPROF file size - Size of the heap indicated by MAT) is effectively garbage (unreachable objects). Unreachable object Histogram in MAT should help here.

jmap -F -dump:live,format=b,file=<file_name.hprof> <process_id> will only dump live objects and NOT garbages.

top and other OS level tools show how much system memory does your JVM process consume. Java heap, defined by -Xmx command line option, is only a part of that memory. Apart from heap JVM needs some memory for itself. Then there are java threads, each requiring a certain amount of memory. And Metaspace/Permanent Generation. And several others. You can read this blog post and this SO answer for more information.

About the size of the dump file and the actual heap size the answer of @arnab-biswas is certainly true. MAT reports the size of actually used heap, consumed by live objects. But heap dump contains the whole of the heap, including garbage.