Amazon S3 Hosting Streaming Video

The answer to the first part of your question is, yes, it is really that simple. There is a how-to about it and a working demo at the end of the article that you can see as a proof of concept.

This is Brandon from Zencoder. What you're looking for is probably something like Video JS (videojs.com) for video playback. You can just upload an MP4 to S3 and reference it in a player (or the video tag directly, but that has additional issues). Our service is actually used for transcoding the video itself, not delivery. We actually created Video JS to help our customers (and the web at large) with easy, compatible HTML5 playback. If you have any other questions just ask. Thanks.

I just had the same question as you and was happy to see that uploading an mp4 file to S3, setting "public" permissions on it, and then using this simple HTML worked great without any additional tools. And CloudFront wasn't even necessary.

<video id="clip" controls preload=auto width=640 height=264 data-setup="{}">
    <source src="https://s3.amazonaws.com/xxxxbucketnamexxxxx/xxxxfilenamexxxx.mp4" type='video/mp4'/>        
</video>