Java Heap Memory Holding Large byte Array

Get a dump before running the stress test and a dump after running it. Eclipse MAT lets you compare the histogram between two dumps, so you'll know that a memory leak occurs.

I recommend using Java Mission Control (JMC) to monitor the JVM, where you can take a closer look at memory consumption (heap and not heap). Note the difference between memory in use and committed memory.

Newer versions of Java have more sophisticated algorithms where the JVM returns memory to the OS. For Java 8 one option is to use the Eclipse Open J9 JVM.

Recommended reading:

https://openjdk.java.net/jeps/346

https://jelastic.com/blog/elastic-jvm-vertical-scaling/

Tomcat caches a number of objects to make it go faster. With the setting server.max-http-header-size=2097152 you made one of those cached objects claim 2 MB of memory and keep it. In this case it is the Http11OutputBuffer and you can see here that it claims (in your case) the 2 MB of memory. The Http11OutputBuffer is used by the Http11Processor which you can see here.

The documentation has the following to say about processorCache:

The protocol handler caches Processor objects to speed up performance. This setting dictates how many of these objects get cached. -1 means unlimited, default is 200. If not using Servlet 3.0 asynchronous processing, a good default is to use the same as the maxThreads setting. If using Servlet 3.0 asynchronous processing, a good default is to use the larger of maxThreads and the maximum number of expected concurrent requests (synchronous and asynchronous).

So my suggestion is to set the server.max-http-header-size to something more reasonable, e.g. 8KB (the default) and slowly double that when testing shows you really need that (related: Tomcat throws "400 Bad request" when total header size is larger than server.max-http-header-size).