During execution, how can a java program tell how much memory it is using?

This won't be exact, but for a rough estimate, just subtract Runtime.getRuntime.freeMemory() from Runtime.getRuntime.totalMemory(). Do that at the beginning of the program to get an idea of the JVM's overhead memory usage and at intervals latter on in the execution.

VonC's answer is an interactive solution - if you want to know programatically, you can use Runtime.totalMemory() to find out the total amount used by the JVM, and Runtime.freeMemory() to find out how much of that is still available (i.e. it's allocated to the JVM, but not allocated within the JVM - new objects can use this memory).

These are instance methods - use Runtime.getRuntime() to first get the singleton instance.

If you are have a java1.6 somewhere and your program is running in java 1.4.2, 1.5 or 1.6, you can launch a visualVM session, connect to your application, and follow the memory (and much more)

(The images are not displayed at the moment, so please refers to Monitoring an application for an illustration.)