Can I redirect output to a log file and background a process at the same time?

One problem with your first command is that you redirect stderr to where stdout is (if you changed the $ to a & as suggested in the comment) and then, you redirected stdout to some log file, but that does not pull along the redirected stderr. You must do it in the other order, first send stdout to where you want it to go, and then send stderr to the address stdout is at

some_cmd > some_file 2>&1 &

and then you could throw the & on to send it to the background. Jobs can be accessed with the jobs command. jobs will show you the running jobs, and number them. You could then talk about the jobs using a % followed by the number like kill %1 or so.

Also, without the & on the end you can suspend the command with Ctrlz, use the bg command to put it in the background and fg to bring it back to the foreground. In combination with the jobs command, this is powerful.

to clarify the above part about the order you write the commands. Suppose stderr is address 1002, stdout is address 1001, and the file is 1008. The command reads left to right, so the first thing it sees in yours is 2>&1 which moves stderr to the address 1001, it then sees > file which moves stdout to 1008, but keeps stderr at 1001. It does not pull everything pointing at 1001 and move it to 1008, but simply references stdout and moves it to the file.
The other way around, it moves stdout to 1008, and then moves stderr to the point that stdout is pointing to, 1008 as well. This way both can point to the single file.

java -jar myProgram.jar &> output.log &

Note that the &> directs both stdout and stderr to output.log

Stopping with <Ctrl+Z> and continuing in the background with bg is equivalent to execute with & at the end of the command.

So, for run in the background and redirect output:

java -jar myProgram.jar 2> errorOutput.log > output.log &

If you also need that this command does not die when you leave the terminal, then you should use nohup