How can a JVM be written in Java

Your assumption that Java requires a virtual machine is incorrect to begin with. Check out the project GCJ: The GNU Compiler for the Java Programming Language.

See bootstrapping.

You are asking about the chicken and the egg.

Read: http://en.wikipedia.org/wiki/Bootstrapping_%28compilers%29

The JVM that you need to bootstrap a JVM written in Java probably does not need a lot of features (such as garbage collection and JIT), could be very simple. All the more advanced features could then be implemented in Java (which seems to be exactly the point of Maxine, to experiment with new ideas in JVM technology).

Also, Maxine does contain C code, which I guess makes up a minimal runtime environment that is used to get the rest of Maxine going. I take it that the interesting bits (JIT compiler, garbage collection) are then completely implemented in Java.