Should we reply to the editor after a paper has been rejected and we don't plan to revise/resubmit?

I don't recall ever having received such an e-mail as an editor, so it's not standard (let alone mandatory), but I wouldn't be unhappy to receive one. I doubt anyone would care enough to make it worth the effort, but it could be a nice gesture if you feel inclined.

I believe that if you feel the rejection was unjustified and you do not plan to submit to that journal again, you do not need to reply.

But if you feel the rejection was justified, you plan to submit to that journal again, or you were helped by the comments and suggestions of the referees, then I would recommend that you write an e-mail thanking the editor and the referees for their time and effort.

I think it is reasonable to thank editors that keep you up to date, solicit reviews in a timely manner, and provide an understandable overview of the reviews. Unfortunately, the number of good editors is small, so I would caution against thanking editors for merely acting as editor. The benefits are small while the potential cost is huge. It is so atypical that you run the risk of the editor thinking you are whining.