Detect when IE11, with its built-in memory leak, runs out of memory (1.5GB recyclable pool)
As far as my experience with this goes, your best bet to mitigate this is to reduce re-creation of iframes and reduce total memory used so the leak is small (for example by code-splitting unused code, removing large libraries, etc). You can then do a test to approximate how much memory you expect to leak per page load (or whatever relevant metric in your case) and come up with a heuristic to reload the app (for example, after 50 page loads).
Keep in mind that sometimes reloading the app will not help IE reclaim memory. There is a weird workaround for this - you can open a new tab, navigate to your app, switch to that tab, and close the old tab. Obviously not an ideal user experience, but it is more reliable than a page refresh.
In other cases memory is leaking because you have a reference to the iframe from somewhere in the main window, or did not clean up event handlers within the iframe before removing it - I assume you have checked for this from the description, but I thought it wouldn't hurt to mention it :).