Why compile OpenCV with QT?
LE: misunderstood the question
WITH_QT option is used by highgui module to create windows using qt, so the QImage to cv::Mat conversion and vice-versa will work no matter how WITH_QT option is set.
First some clarifications: you are not compiling OpenCV with Qt, Qt is not a compiler so you can't compile anything with it.
Qt is a C++ library (it's called a framework because it imposes some design some rules to your application source code, but basically it's a C++ library, just like OpenCV).
Now, in C++ world* to use some libraries together you need to build those libraries with the same compiler (and in some cases even the same compiler settings), so must decide which C++ compiler you want to use and get both Qt and OpenCV built with the same compiler not necessarily build by you, binaries can be obtained from their websites.
If you want to use MinGW you will need to build OpenCV with MinGW compiler, because OpenCV (at least, version 2410) comes build only with Visual C++ version 10, 11, 12 - that means Visual Studio 2010, 2012 and 2013.
So if you decide to use some Visual Studio version, depending on version you choose, you might be able to use Qt with OpenCV without having to build neither yourself, but if you want MinGW compiler you need to build OpenCV with MinGW.
*you can get away with it if your libraries only export a C interface, but that is not the case with neither Qt nor latest OpenCV versions.
//if you want more details about this use your favorite internet search engine to search for: c++ binary compatibility and or c++ abi