Apple - How to use keynote magic move with Connection Lines?
Turns out, according to several sources you can't do it. You would need to separate the lines (i.e. not use connection lines).
I found that if I grouped the connection with something else (in my case, an image) then the connection doesn't fade out and in between slides.