How do I capture the HTTP error code from a download.file request?

Probably the best option is to use cURL library directly rather than via the download.file wrapper which does not expose the full functionality of cURL. We can do this, for example, using the RCurl package (although other packages such as httr, or system calls can also achieve the same thing). Using cURL directly will allow you to access the cURL Info, including response code. For example:

library(RCurl)
curl = getCurlHandle()
x = getURL("https://en.wikipedia.org/asdfasdfasdf", curl = curl)
write(x, 'output.html')
getCurlInfo(curl)$response.code
# [1] 404

Although the first option above is much cleaner, if you really want to use download.file instead, one possible way would be to capture the warning using withCallingHandlers

try(withCallingHandlers( 
  download.file(url, destfile = "output.html", method = "libcurl"),
  warning = function(w) {
    my.warning <<- sub(".+HTTP status was ", "", w)
    }),
  silent = TRUE)

cat(my.warning)
'404 Not Found'

If you don't mind using a different method you can try GET from the httr package:

url_200 <- "https://en.wikipedia.org/wiki/R_(programming_language)"
url_404 <- "https://en.wikipedia.org/asdfasdfasdf"

# OK
raw_200 <- httr::GET(url_200)
raw_200$status_code
#> [1] 200

# Not found
raw_404 <- httr::GET(url_404)
raw_404$status_code
#> [1] 404

^{Created on 2019-01-02 by the reprex package (v0.2.1)}