How to blame previous versions of a specific line of code in VS 2015?

The "Annotate This Version" found in the context-menu of the left gutter works just as expected. In my case, the history didn't follow a file rename which was confusing me.