How can I limit the number of IMAP emails Outlook 2010 syncs?
There is no way I know of to do that. You can only choose to download headers, or full e-mails (HIGHLY recommended in my opinion). I would also recommend that you teach users to not store too many items in one folder, since that seems to affect performance. In general, for IMAP, it is just slow with larger mailboxes.
Ultimately, if you want excellent performance with large mailboxes, and quick synchronization times, I can't recommend hosted Exchange enough. I have multiple clients using it, including a lawyer's office where they must keep everything, and it performs extremely well. Some of the users have as much as 12GB of mail, and it synchronizes header changes in seconds from a server 3000 miles away....only after that does it pull the bodies and attachments.
If you do choose to go this route, come back and I can offer some concrete advice on how to make the move, because I encountered numerous pitfalls the first time I went from IMAP to hosted Exchange, including who I would go with and why.
I dealt a bit differently with the same issue, where a user wants to use Outlook with Google Apps for Business. You can just go to Gmail settings and under "Forwarding and POP/IMAP" tab, change option "Limit IMAP folders to contain no more than this many messages" to 1000 (which is the lowest number). This will make Outlook download only 1000 latest emails. It works without a problem.
Since you're using Gmail, you might consider the Google Apps Sync for Microsoft Outlook. It lets you set a hard mailbox size limit for Outlook. For example, if you set the limit to 1 GB, the Sync tool will begin removing older messages when you hit the limit (the emails are still available in Gmail though!)
More on this: https://support.google.com/a/bin/answer.py?hl=en&answer=184126
Download here: www.google.com/apps/intl/en/business/outlook_sync.html