File Caching: Query string vs Last-Modified?
TL;DR
Why do so many websites use the "query string" method, instead of just letting the last-modified header do its work?
Changing the query string changes the url, ensuring content is "fresh".
Should I unset the Last-modified header and just work with query strings?
No. Though that's almost the right answer.
There are three basic caching strategies used on the web:
- No caching, or caching disabled
- Using validation/conditional requests
- Caching forever
To illustrate all three, consider the following scenario:
A user accesses a website for the first time, loads ten pages and leaves. Each page loads the same css file. For each of the above caching strategies how many requests would be made?
No caching: 10 requests
In this scenario, it should be clear that there isn't anything else influencing the result, 10 requests for the css file would result in it being sent to the client (browser) 10 times.
Advantages
- Content always fresh
- No effort/management required
Disadvantages
- Least efficient, content always transferred
Validation requests: 10 requests
If Last-Modified or Etag are used, there will also be 10 requests. However 9 of them will only be the headers, and no body is transferred. Clients use conditional requests to avoid re-downloading something it already has. Take for example the css file for this site.
The very first time the file is requested, the following happens:
$ curl -i http://cdn.sstatic.net/stackoverflow/all.css
HTTP/1.1 200 OK
Server: cloudflare-nginx
Date: Mon, 12 May 2014 07:38:31 GMT
Content-Type: text/css
Connection: keep-alive
Set-Cookie: __cfduid=d3fa9eddf76d614f83603a42f3e552f961399880311549; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.sstatic.net; HttpOnly
Cache-Control: public, max-age=604800
Last-Modified: Wed, 30 Apr 2014 22:09:37 GMT
ETag: "8026e7dfc064cf1:0"
Vary: Accept-Encoding
CF-Cache-Status: HIT
Expires: Mon, 19 May 2014 07:38:31 GMT
CF-RAY: 1294f50b2d6b08de-CDG
.avatar-change:hover{backgro.....Some KB of content
A subsequent request for the same url would look like this:
$ curl -i -H "If-Modified-Since:Wed, 30 Apr 2014 22:09:37 GMT" http://cdn.sstatic.net/stackoverflow/all.css
HTTP/1.1 304 Not Modified
Server: cloudflare-nginx
Date: Mon, 12 May 2014 07:40:11 GMT
Content-Type: text/css
Connection: keep-alive
Set-Cookie: __cfduid=d0cc5afd385060dd8ba26265f0ebf40f81399880411024; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.sstatic.net; HttpOnly
Cache-Control: public, max-age=604800
Last-Modified: Wed, 30 Apr 2014 22:09:37 GMT
ETag: "8026e7dfc064cf1:0"
Vary: Accept-Encoding
CF-Cache-Status: HIT
Expires: Mon, 19 May 2014 07:40:11 GMT
CF-RAY: 1294f778e75d04a3-CDG
Note there is no body, and the response is a 304 Not Modified. This is telling the client that the content it already has (in local cache) for that url is still fresh.
That's not to say this is the optimal scenario. Using tools such as the network tab of chrome developer tools allows you to see exactly how long, and doing what, a request takes:
Because the response has no body, the response time will be much less because there's less data to transfer. But there is still a response. and there is still all of the overhead of connecting to the remote server.
Advantages
- Content always fresh
- Only one "Full" request sent
- Nine requests are much slimmer only containing headers
- More efficient
Disadvantages
- Still issues the maximum number of requests
- Still incurs DNS lookups
- Still needs to establish a connection to the remote server
- Doesn't work offline
- May require server configuration
Caching forever: 1 request
If there are no etags, no last modified header and only an expires header set far in the future - only the very first access to a url will result in any communication with the remote server. This is a well-known? best practice for better frontend performance. If this is the case, for subsequent requests a client will read the content from it's own cache and not communicate with the remote server at all.
This has clear performance advantages, which are especially significant on mobile devices where latency can be significant (to put it mildly).
Advantages
- Most efficient, content only transferred once
Disadvantages
- The url must change to prevent existing visitors loading stale cached versions
- Most effort to setup/manage
Don't use query strings for cache busting
It is to circumvent a client's cache that sites use a query argument. When the content changes (or if a new version of the site is published) the query argument is modified, and therefore a new version of that file will be requested as the url has changed. This is less work/more convenient than renaming the file every time it changes, it is not however without its problems,
Using query strings prevents proxy caching, in the below quote the author is demonstating that a request from browser<->proxy cache server<->website does not use the proxy cache:
Loading mylogo.gif?v=1.2 twice (clearing the cache in between) results in these headers:
>> GET http://stevesouders.com/mylogo.gif?v=1.2 HTTP/1.1 << HTTP/1.0 200 OK << Date: Sat, 23 Aug 2008 00:19:34 GMT << Expires: Tue, 21 Aug 2018 00:19:34 GMT << X-Cache: MISS from someserver.com << X-Cache-Lookup: MISS from someserver.com >> GET http://stevesouders.com/mylogo.gif?v=1.2 HTTP/1.1 << HTTP/1.0 200 OK << Date: Sat, 23 Aug 2008 00:19:47 GMT << Expires: Tue, 21 Aug 2018 00:19:47 GMT << X-Cache: MISS from someserver.com << X-Cache-Lookup: MISS from someserver.comHere it’s clear the second response was not served by the proxy: the caching response headers say MISS, the Date and Expires values change, and tailing the stevesouders.com access log shows two hits.
This shouldn't be taken lightly - when accessing a website physically located on the other side of the world response times can be very slow. Getting an answer from a proxy server located along the route can mean the difference between a website being usable or not - in the case of cached-forever resources it means the first load of a url is slow, in the case of using validation requests it means the whole site will be sluggish.
Instead version-control assets
The "best" solution is to version control files such that whenever the content changes so does the url. Normally that would be automated as part of the build process.
However a near-compromise to that is to implement a rewrite rule such as
# ------------------------------------------------------------------------------
# | Filename-based cache busting |
# ------------------------------------------------------------------------------
# If you're not using a build process to manage your filename version revving,
# you might want to consider enabling the following directives to route all
# requests such as `/css/style.12345.css` to `/css/style.css`.
# To understand why this is important and a better idea than `*.css?v231`, read:
# http://stevesouders.com/blog/2008/08/23/revving-filenames-dont-use-querystring
<IfModule mod_rewrite.c>
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.+)\.(\d+)\.(js|css|png|jpe?g|gif)$ $1.$3 [L]
</IfModule>
In this way a request for foo.123.css is processed by the server as foo.css - this has all the advantages of using a query parameter for cache busting, but without the problem of disabling proxy caching.
The Last-Modified header is applied differently across browsers, but generally the browser will issue a conditional GET request that the server must respond to if the cache needs to be updated. For example, in Firefox...
The "Last-Modified" response header can be used as a weak validator. It is considered weak because it only has 1-second resolution. If the "Last-Modified" header is present in a response, then the client can issue an "If-Modified-Since" request header to validate the cached document.
When a validation request is made, the server can either ignore the validation request and response with a normal 200 OK, or it can return 304 Not Modified to instruct the browser to use its cached copy. The latter response can also include headers that update the expiration time of the cached document.
By setting a timestamp (or fingerprint), you explicitly let the browser know when it needs to update its cache and you can then set very long expiration times.
It may be worth noting that the documentation on the rails asset pipeline (http://guides.rubyonrails.org/asset_pipeline.html) cites 3 advantages for fingerprinting over a querystring timestamp:
- Not all caches will reliably cache content where the filename only differs by query parameters
- The file name can change between nodes in multi-server environments.
- Too much cache invalidation
For more detail and best practices on caching: https://developers.google.com/speed/docs/best-practices/caching