Should I use VNC or X protocol to control a Linux application from Windows?

Based on the request for making it work as is you were runing as an App in windows, it sounds like X is the best choice. VNC is a nice simple remote protocol, but it always seems to choke on some condition that makes it less than ideal for serious use. Another comment, is that many anti-virus packages like to complain about some components in VNC, typically reporting the view capture software as trojans. They'r not, but if you are in a strict enterprise environmennt, X is more tolerated than VNC.

From my experience and what I learned, X11 requires many more back and forth packets through internet, while each is small. VNC is the opposite: less packets but larger size.

Given this people really need to choose which protocol to use based on the latency/bandwidth of their connection. I'd say in general VNC appears more responsive, and in your case (<1ms ping gigabit network), both would work well!