Why does AWS recommend against public S3 buckets?
Solution 1:
Yes, if you know what you're doing (edit: and everyone else with access to it does, too...), you can ignore this warning.
It exists because even large organizations who should know better have accidentally placed private data into public buckets. Amazon will also send you heads-up emails if you leave buckets public in addition to the in-console warnings.
Accenture, Verizon, Viacom, Illinois voter information and military information has all been found inadvertently left open to everyone online due to IT bods misconfiguring their S3 silos.
If you are absolutely, 100% certain that everything in the bucket should be public and that no one's going to accidentally put private data in it - a static HTML site's a good example - then by all means, leave it public.
Solution 2:
The privacy issue featured in ceejayoz's answer is not the only problem.
Reading objects from an S3 bucket has a price. You will be billed by AWS for each download from this bucket. And if you have a lot of traffic (or if someone who wants to hurt your business starts to heavily download files all day long) it will quickly become expensive.
If you want files from your bucket to be publicly accessible, you should create a Cloudfront Distribution that points to and is granted access to the S3 bucket.
Now, you can use the Cloudfront Distribution's domain name to serve your files without granting any S3 access to the public.
In this configuration, you pay for Cloudfront's data usage instead of S3's. And at higher volumes it's a lot cheaper.