Should I use my university's letterhead for a cover letter for a job application?

No. It is you as a person who is applying, not you as a representative for your university. Univ. Y would probably look very negatively on an application on such lerterhead, as would Univ. X. Letterheads are intended for your official business of your position at X as teacher, researcher etc. Applying for a job or for example writing somewhere for your private business is just that, private. You can, however, create your own private letterhead to use for such instances but this should reflect your private standing and show only private address, phone and e-mail.

So keep the official business separate from your private. As alluded to this also applies to e-mail, something most people forget about. It is good to have your own private e-mail for instances such as this.

I personally side with Peter Jansson on this one (do not use an institutional letterhead if you're not conducting business on behalf of your institution), but you will have to note that this position is not universally shared. I cite only one example, of somewhat high-profile blogger/consultant, who says:

Your letter must be on letterhead if you have a current academic affiliation of any kind. This is not negotiable.

I think it may be a field-specific and/or generation issue: some people, and some fields (humanities/law/medicine) have more attachement to older traditions and think a letterhead is a crucial part of correspondence etiquette.

I would ask a senior person in your field. In my experience in mathematics in the US, no one gives a flying flip how you format your cover letter; in all likelihood, no one will read it. Using department stationary is common, though far from universal and I don't think affects anyone's thinking one way or the other.

I don't think this advice is universally applicable; I know in many other disciplines, cover letters are read carefully, and thus their professionalism will have some salutary effect. My personal feeling is that using your current institution's letterhead, your office address, etc. looks more professional, but obviously this isn't a universal feeling, so all the more reason to check about your field specifically.