How do you get Outlook to show a preview of a new message near the taskbar?

For displaying alerts on new email arriving in other folders than the default Inbox, the solution would be to use Rules. See this link for detailed step-by-step guide to do that. Most important is step 9 to make sure this runs ahead of all other rules. This bolg post has a brief version of the same with screenshots if that is easier to follow.

Also an article from Microsoft Office online help on same topic for more info.