What to do when a referee asks me to do something impossible/unreasonable?

I believe the key is that you can include the information in your paper about this direction being "impossible", and improve it significantly by doing so.

Rather than rejecting the suggestion outright, add a discussion to your paper of that future direction and some notes on its difficulty (i.e., although we show in this paper that 10+10=20 through novel use of the podal digits, the corollary of 20-10=10 cannot currently be proven empirically due to limitations of the primary technique of positive incremental dactylonomy, and the lack of willing test subjects for the secondary approach).

Although it might seem obvious to you why that improvement is "impossible", your reviewers may read your paper even more carefully than your broader audience. If something was unclear (or at least, non-obvious) to this reviewer, you should expect that at least some non-negligible proportion of your broader readership will come to the same conclusion. I think you improve your paper substantially if you can explain this concisely.

Even better, if you can suggest some approach that is possible but currently difficult to implement or outside the scope of your current paper, your work could be the priority reference for that approach.

I am not sure about your field, but when I have made a revision to a manuscript, it is required to have a letter detailing your changes and your response to the reviewers' comments. In your case, make the changes that you can accomplish and detail your reasons for not doing X in this manuscript. Be detailed and specific. You could even address the fact that you did not do X in your manuscript as well, if you think it could be useful to a reader.