How do I clean up this space-town?

As of version 0.20, the tracking screen will let you select debris and "end flight" to clear it out.

Not nearly as much fun as some of the other methods on the page, but simpler.

Another option for removing debris is to limit how much of it you want to keep around. One of the settings allows you to choose anywhere from no debris, to unlimited debris.

This option is available under Settings in the main menu (the menu you see when first starting the game, not the in-game menu), and is called Max Persistent Debris.

Keep in mind this setting only handles debris from not controlled flights; it won't automatically kill everything that gets detached from your current launch. That only happens when you end the flight, or switch to the Space Center.

Not quite what you were asking for, but the only other option, yes, is to intercept each and every piece, and manually de-orbit it.

Unfortunately, I was unable to make the debris my currently active flight using the techniques @MBraedley suggested. A solution that worked ended was to back up my ship files (in the main Kerbal Space Program folder, under /saves/%save file name%/Ships) and create a new save, though that also wiped out my space base. No matter! Science carries on!

In the future I will probably just ignore the space junk, as collisions are either hilariously unlikely or just hilarious.