Which way to test if an element is checked is better? .is(':checked') or .prop('checked')

They both end up checking the same thing.

If you're using 1.6.0 or higher, prop('checked') is the most direct jQuery way. jQuery doesn't have to parse and process a selector to figure out what to do.^{[Note below]}

You can also (as of 1.6.1) use attr('checked') again as with 1.5.x and earlier.

Or you can go directly to the element. Assuming x is the thing to be tested, if you know that at least one element matched, then:

if (x[0].checked) { /* ... */ }

If you're not sure and want to hedge your bets:

if (x[0] && x[0].checked) { /* ... */ }

But unless you're in a really tight loop, use whatever you find easiest to read, as the performance differences aren't going to matter. I find the last ones quite easy to read and I know they're very fast, so I use them. But if you find them awkward, use whatever you like best. No harm in using is(':checked') if you like it and you're not seeing an actual performance hit from it (which is unlikely barring, again, some kind of tight loop).

---

Note: The degree to which prop is more direct than is varies by browser. prop isn't just a direct check of a property, it does go through a couple of levels of indirection first; and is isn't necessarily hugely complex: On WebKit browsers, for instance, is can be fairly direct as WebKit provides a function to test whether an element matches a selector, and supports :checked natively; on Firefox (and, I suspect, IE), is results in a huge number of function calls as this seemingly-simple selector works its way through the guts of Sizzle.

I would use prop('checked') myself (so long as I didn't have to support older jQuery versions) as it is accessing the checked property directly of the object and is easy enough to read.

is(':checked') runs a bit of extra overhead with string parsing, etc. I generally reserve :checked for when selecting elements.