What should a "unit" be when unit testing?
It may seem trivial to quote Wikipedia, but I think it's very succinct and accurate in this case:
A unit is the smallest testable part of an application.
This seems to agree with the comment in your question that a unit is "the smallest piece of the code that can be usefully tested". In other words, make the unit as small as you possibly can such that it still makes sense to the developer/tester by itself.
Often you will want to test parts of a project in isolation, and then test how they interact in combination. Having various tiers (levels) of unit testing is often a wise thing to do, as it helps insure that your code is working as it should on all levels, from individual functions up to entire self-contained tasks. I personally do not believe that it is wrong, or even unhelpful, to test individual functions, so long as they are doing something useful in themselves, which can often be the case.
To be quite honest, there is no definite or rigorous definition of a "unit" in "unit testing", which is precisely why the vague term "unit" is used! Learning what needs to be tested and at what level is a matter of experience, and quite often, simply trial and error. It may sound like a slightly unsatisfying answer, but I believe it is a sound rule to follow.
I've always tested at the function level, and that's worked just fine. The big point for me is that unit testing tests the contract between two pieces of code. The unit test just acts as a caller, and ensures that the code being tested (no matter how large - a single function or a huge, nested library that takes 30 minutes to test) returns results in a predictable way.
The whole point of a unit test it to ensure compatibility (by not breaking intereactions) and ensure predictable results. Testing at any place there's an exchange of information will help stabilize your application.
UPDATE: I've also always added a unit test whenever a customer or user reports an edge case that breaks an interaction in my application. Fixing it isn't sufficient for me - adding a unit test to ensure that the fix holds prevents regression, and helps keep things stable down the line.
A "unit" should be a single atomic unit for your definition. That is, precisely what defines a "unit" for unit testing is rather subjective; it can depend rather strongly on precisely what your architecture is, and how your application chooses to break down functional units. What I'm saying is that there is no clearly defined "unit", except for what you define. A "unit" can be a single method which has a single side effect, or it can be a related set of methods which together define a single, coherent set of functionality.
Rationality is rather important here; it's entirely possible to write unit tests for each accessor of each class that you have; however, that's clearly overkill. Similarly, it's possible but silly to define your entire application as a unit, and expect to have a single test to test it.
Generally, you want a "unit" for testing to describe one clearly defined, clearly distinct chunk of functionality. At each level of viewing your application, you should be able to see clear "atoms" of functionality (if your design is good); these can be as simple as "retrieve a record from the database" for a low-level view, and as complicated as "create, edit, and delete a user" at a high-level view. They're not mutually exclusive, either; you can easily have both as unit tests.
The point to be taken here is that the unit you want to test is the unit that makes sense. You want unit testing to validate and verify functionality; so what a unit test tests is what you define as functionality. What is a unit of functionality? That's up to your individual situation to define.