MySQL LOAD DATA INFILE: Better Server, Worse Performance
Here's what seems to have been causing this behavior:
Per the Azure documentation, the Basic tier server on Azure comes with "variable" IOPS whereas the Memory Optimized server comes with a fixed IOPS which is based on the amount of storage assigned to the database server.
I had 100GB assigned to the Memory Optimized server. This resulted in it having 300 IOPS, in accordance with Azure's 3 IOPS / GB ratio.
Presumably the "variable" IOPS on the Basic server ended up being significantly more than the 300 IOPS that the Memory Optimized server had.
Lesson learned: to get fast storage access on Azure Database, you need to assign plenty of storage capacity to your server (even if you don't need that much storage!).