How do I analyze a .mdmp file?

The .mdmp file is a Windows MiniDump file that you can only read with a debugger (like WinDbg). Typically you need the sources of the crashed application to really get some information out of the dump. So in your case you can't do much but contacting JRockit support.

Here a link to the Orace JRockit information about JVM crahes.

.mdmp files are the Windows equivalent of unix/linux core dumps. You can analyse them with WinDBG but if it's a Java process that has crashed most likely you'll want to use Java's own tools to analyse the crashed process.

If you want to look at the heap of the crashed Java process you can use a tool that ships with the JDK called jmap to extract a HPROF file from a .core or .mdmp and then load this into a memory analyser. Note also that some memory analyzers can load core dumps and Windows minidumps directly.

Related issue and the jmap docs

If you want to see the state of the threads then you can use a tool called jstack to print stack traces for every thread at the point the dump was created. jstack docs.