Measuring Java execution time, memory usage and CPU load for a code segment
With the ThreadMXBean you can get CPU usage of individual threads and cpu time consumed rather than elapse time which may be useful.
However, its often simpler to use a profiler as this process often generates a lot of data and you need a good visualisation tool to see what is going on.
I use Yourkit as I find it easier to solve problems that other profilers I have used. I also use the builtin hprof as this can give you a different view on the profile of your application (but not as useful)
Using a Java Profiler is the best option and it will give you all the insight that you need into the code. viz Response Times, Thread CallTraces, Memory Utilisations, etc
I will suggest you JENSOR, an open source Java Profiler, for its ease-of-use and low overheads on CPU. You can download it, instrument the code and will get all the info you need about your code.
You can download it from: http://jensor.sourceforge.net/
Profiling may be an easier option since you don't require in-production stats. Profiling also doesn't require code modification. VisualVM (which ships w/ the JDK 1.6.06+) is a simple tool. If you want something more in-depth I'd go with Eclipse TPTP, Netbeans profiler, or JProfiler(pay).
If you want to write you own, consider the following:
Simple measurments like execution time can be done by "clocking" the section you're interested in:
long start = System.nanoTime(); // requires java 1.5
// Segment to monitor
double elapsedTimeInSec = (System.nanoTime() - start) * 1.0e-9;
You can use a similar technique to monitor memory via Runtime.getRuntime().*memory() methods. Keep in mind that tracking memory usage in a garbage collected environment is trickier than simple subtraction.
CPU load is hard to measure in Java, I typically stick with execution time and optimize the longer / repetitive sections