Distances between rankings

You're looking for Normalized Discounted Cumulative Gain (NDGC). It's a metric commonly used in search engine rankings to test the quality of the result ranking.

The idea is that you test your ranking (in your case the two methods) against user feedback through clicks (in your cast the true rank). NDGC will tell you the quality of your ranking relative to the truth.

Python has RankEval based module that implements this metric (and some others if you want to try them). The repo is here and there is a nice IPython NB with examples


The scikit-learn library also seems to have a NDCG (and DCG) metric implemented now.

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.ndcg_score.html#sklearn.metrics.ndcg_score