How do I set expiration headers for CSS, JS, and Images?

Update your Apache configuration to include the directives below as part of your core configuration:

# 
# associate .js with "text/javascript" type (if not present in mime.conf)
# 
AddType text/javascript .js

# 
# configure mod_expires
# 
# URL: http://httpd.apache.org/docs/2.2/mod/mod_expires.html
# 
<IfModule mod_expires.c>
    ExpiresActive On
    ExpiresDefault "access plus 1 seconds"
    ExpiresByType image/x-icon "access plus 2692000 seconds"
    ExpiresByType image/jpeg "access plus 2692000 seconds"
    ExpiresByType image/png "access plus 2692000 seconds"
    ExpiresByType image/gif "access plus 2692000 seconds"
    ExpiresByType application/x-shockwave-flash "access plus 2692000 seconds"
    ExpiresByType text/css "access plus 2692000 seconds"
    ExpiresByType text/javascript "access plus 2692000 seconds"
    ExpiresByType application/x-javascript "access plus 2692000 seconds"
    ExpiresByType text/html "access plus 600 seconds"
    ExpiresByType application/xhtml+xml "access plus 600 seconds"
</IfModule>

# 
# configure mod_headers
# 
# URL: http://httpd.apache.org/docs/2.2/mod/mod_headers.html
# 
<IfModule mod_headers.c>
    <FilesMatch "\\.(ico|jpe?g|png|gif|swf|css|js)$">
        Header set Cache-Control "max-age=2692000, public"
    </FilesMatch>
    <FilesMatch "\\.(x?html?|php)$">
        Header set Cache-Control "max-age=600, private, must-revalidate"
    </FilesMatch>
    Header unset ETag
    Header unset Last-Modified
</IfModule>

You can put this in your htaccess:

<FilesMatch "(?i)^.*\.(ico|flv|jpg|jpeg|png|gif|js|css)$">
ExpiresActive On
ExpiresDefault A2592000
</FilesMatch>

It will target files with those extensions (ico, flv, jpg and so on) and set the Expires header to be access time (A) plus 30 days (2592000 seconds). You can also add this at the server level if you have access to that.

It depend from the host and the way you server these things. Option 1) if you control the server make the apache to add expiration headers in the response Option 2) if you do not control the web server, or you server the images/js/css/etc you can set these headers from the script that server them

Have in mind that these hints are recommendable but not absolute truth. They are more to save you a bandwidth than to speed up your website . So if you have a low traffic to your site do not worry too much about this.