Force Facebook to use large images when sharing my page
You can add a meta tag in your page's <head> area with the og:image property to point to the image you want Facebook to use when people Like your page:
<meta property="og:image" content="http://yoursite.com/path/to/image.jpg"/>
If you don't use this tag, Facebook's page scraper will pick an image for you. It's very likely that Facebook is finding 100px wide images elsewhere on the page and using those instead of your larger images. (The other sites you've seen are probably specifying their 320px images using the og:image meta tag.)
How you add the meta tag depends on the publishing software you're using. If you're using WordPress, for example, the Open Graph Pro plugin allows you to pick a featured image in your post editor for Facebook to use; it then inserts the og:image meta tag pointing to that image.
Update: following your comments below, the reason some of the photos on your wall appear larger than the linked photo is that they have been reposted from one of your friend's photos that was initially uploaded to Facebook. i.e. They are not shared links to external websites.
Facebook shows your friend's photos with a large thumbnail, but external sites with a small thumbnail. There's no change you can make to your site to get it to use larger thumbnails. It's just how Facebook works at present.
If you wanted the images to appear bigger in other people's streams, you could upload them as images to your Facebook page (or to an official Facebook page for your site), and get people to like the photos you've uploaded to Facebook directly instead of liking the link to the page containing the photo on your site.