How do defragmenting programs work?

I'm not profound in defragmentation but I would like to share some information regarding your question. For the first part of your question, MSDN mentions about the defragmentation algorithms in their blog:

In Windows XP, any file that is split into more than one piece is considered fragmented. Not so in Windows Vista if the fragments are large enough – the defragmentation algorithm was changed (from Windows XP) to ignore pieces of a file that are larger than 64MB. As a result, defrag in XP and defrag in Vista will report different amounts of fragmentation on a volume. So, which one is correct? Well, before the question can be answered we must understand why defrag in Vista was changed. In Vista, we analyzed the impact of defragmentation and determined that the most significant performance gains from defrag are when pieces of files are combined into sufficiently large chunks such that the impact of disk-seek latency is not significant relative to the latency associated with sequentially reading the file. This means that there is a point after which combining fragmented pieces of files has no discernible benefit.

You can read the whole article from here: Engineering Windows 7 - Disk Defragmentation.

For the second part of your question, Piriform Docs has an answer:

Defraggler has two different modes for defragmentation: regular defrag and Quick Defrag. Quick Defrag is faster, but the results aren't as optimal as with a regular defrag.

Quick Defrag works faster by skipping over files that have certain properties. These are specified in the Quick Defrag tab of the Options dialog box.

The default Quick Defrag rule is to skip files with fragments larger than 50MB.

For better comprehension of defragmentation, Wikipedia has a good page.

One important thing to note: Microsoft provides an API that programs can use for safely defragmenting files. (See Defragmenting Files on MSDN) Any defragger should be using those APIs, and most do.

All the "magic" of defragmenting comes in determining how to lay the files out, and this is what the various programs use to differentiate each other. If you think you're really smart, you can use MyDefrag to write your own layout script.