Get the file name without file extension in Vim function.

:help expand() should give you the answer, see expand().

You should use the r modifier for %, with %:r instead of % to get the file name without extension.

If you want to write functions to build and execute files, you should also have a look at the documentation for shellescape, in order to prevent problems with spaces in file name or path.

If you want to expand a filename (other than % etc) take a look at fnamemodify()

fnamemodify({fname}, {mods})                *fnamemodify()*
        Modify file name {fname} according to {mods}.  {mods} is a
        string of characters like it is used for file names on the
        command line.  See |filename-modifiers|.

fnamemodify("main.java", ":r") returns main.

I literally just read a similar question to this (in that someone else seemed to be trying to configure vim to build automagically for them with the F-key), and wrote an answer about how you can leverage the power of vim's :make command without even needing to write a Makefile. In your case, it's less directly related to the question, but I thought I'd mention it in case you were interested.

Furthermore, someone seems to have written something on Vim Tips Wiki about how to set up vim's :make command to specifically work with Java projects built with ant. I haven't worked with Java in a while myself, but in your case specifically it might be a good place to get started.