Transform not working in AVMutableVideoComposition while exporting

Ok figured it out and posting here to help other people not waste the time that I did.

The issue is that if you use AVAssetExportPresetPassthrough on an AVExportSession, then the exporter will ignore the video composition instructions. I was expecting it to at least honor the video composition instructions while passing through the format etc. but apparently that isn't how it works. After I have filled a documentation bug, you can find it in Technical Q&A.