Will Java app slow down by presence of -Xdebug or only when stepping through code?

First, to strictly answer your question - at least as stated in its title - -Xdebug only enables debugging support in the VM using JVMDI in JVMs prior to 5.0. So in itself, it doesn't do much. Moreover, JVMDI is deprecated since 5.0 in favor of JVMTI:

-Xdebug
Start with support for JVMDI enabled. JVMDI has been deprecated and is not used for debugging in J2SE 5.0, so this option isn't needed for debugging in J2SE 5.0.

So -Xdebug doesn't do anything anymore and the important part is:

-Xrunjdwp:<name1>[=<value1>],<name2>[=<value2>]...

or, starting with Java 5.0, the newer (that you should prefer as the JDWP agent in 5.0 uses the JVM TI interface to the VM rather than the older JVMDI interface):

--agentlib:jdwp=<name1>[=<value1>],<name2>[=<value2>]...

Now, to my knowledge, just loading the jwdp agent and/or configuring the JVM to listen for a socket connection on a given port don't have any noticeable performance impact. But connecting a debugger does.

Performance testing results at AMD indicate that simply enabling the debug agent via the JVM commandline does cause performance degradation regardless of having a debugger connected to it, and that the degradation can be quite large depending on the workload:

Note that we weren’t actually attaching a debugger when we were measuring performance, so we had assumed this agentlib option would be performance-neutral in this usage scenario. When we removed this option, both the CPU utilization and the performance on this workload (measured in requests handled per second) improved dramatically.

See their report:

http://developer.amd.com/resources/documentation-articles/articles-whitepapers/java-performance-when-debugging-is-enabled/

Above link is dead, here's a web archive link of it: https://web.archive.org/web/20160316201129/http://developer.amd.com/resources/documentation-articles/articles-whitepapers/java-performance-when-debugging-is-enabled/