Random crashes on Windows 10 64bit with ATL subclassing

This is the bug inside atlthunk.dll. When it loads itself second time and further this happens manually via MapViewOfFile call. In this case not every address relative to the module base is properly changed (when DLL loaded by LoadLibarary/LoadLibraryEx calls system loader does this automatically). Then if the first time DLL was loaded on preferred base address everything works fine as unchanged addresses point to the similar code or data. But if not you got crash when 257th subclassed window handles messages.

Since Vista we have "address space layout randomization" feature this explains why your code crashes randomly. To have crash every time you have to discover atlthunk.dll base address on your OS (it differs on different OS versions) and do one memory page address space reservation at this address using VirtualAlloc call before the first subclass. To find the base address you can use dumpbin /headers atlthunk.dll command or parse PE headers manually.

My test shows that on Windows 10 build 14393.693 x32 version is affected but x64 is not. On Server 2012R2 with latest updates both (x32 and x64) versions are affected.

BTW, atlthunk.dll code has around 10 times more CPU instructions per thunk call as previous implementation. It may be not very significant but it slows down the message processing.