Use CDN with carrierwave + fog in s3 + cloudfront with rails 3.1

seems that amazon cdn doesn't work with config.fog_public = false, so private files are accessible only from s3, not from cdn


After some searching and struggling with this for a long time I found a page that says that CarrierWave doesn't support CloudFront signed urls. CloudFront signed urls are different than S3 signed urls, which caused me some confusion. Once I figured that out, it was a lot easier to know what to do.

If you configure CarrierWave with config.fog_public = false then it will automatically begin signing S3 urls, but it can't be configured to work with Fog and CloudFront private content in the version of CarrierWave I'm using (1.0.0). I even tried using the carrierwave-aws gem and that didn't help either.

So what would happen is that CarrierWave would sign the URL and the host would look something like this:

https://my_bucket_name.s3-us-west-2.amazonaws.com/uploads/...?signature...

That points directly to the S3 bucket, but I needed it to point to CloudFront. I needed the host to look like this:

https://s3.cloudfront_domain_name.com/uploads/...

And what would happen if I set config.asset_host equal to my CloudFront location is I'd get this, (with double slashes before "uploads"):

https://s3.cloudfront_domain_name.com//uploads/...

That, too, made it clear CarrierWave wasn't yet designed to be used with CloudFront. Hopefully they'll improve it. This was my work-around. It's ugly, but it worked to get done what I needed without needing to modify CarrierWave itself, as I hope CarrierWave will at some point add support for CloudFront.

  1. First I did a regex find/replace on my url and removed the S3 host portion and put on my CloudFront host portion. cf_url = s3_url.gsub("my_bucket_name.s3-us-west-2.amazonaws.com", "s3.cloudfront_domain_name.com")
  2. Next I did another regex find/replace to remove the S3 signed url at the end of the string: non_signed_cf_url = cf_url.gsub(/\?.+/, '') This is because the signature will be incorrect because it was using the API for S3 and not for CloudFront for signing the URL.
  3. Now I re-sign the URL myself, using the cloudfront-signer gem: signed_cf_url = Aws::CF::Signer.sign_url(non_signed_cf_url, :expires => 1.day.from_now)

There are a few other things you need to be aware of when serving private content on CloudFront:

  • In the Cache Behavior Settings for your path pattern (not necessarily the default one), set: "Restrict Viewer Access (Use Signed URLs or Signed Cookies)" to "Yes"
  • Set "Trusted Signers" to "self"
  • Set "Query String Forwarding and Caching" to "Forward all, cache based on all" if you want to use other query strings more than the CloudFront signature in your url, such as response-content-disposition and response-content-type (I was able to get these to work successfully, but they have to be url_encoded properly.)
  • In your CloudFront Origin Settings, set your access-identity and set "Grant Read Permissions on Bucket" to "Yes, Update Bucket Policy"
  • In your General Distribution Settings, make sure "Distribution State" is "Enabled" and that you've added a CNAME to "Alternate Domain Names (CNAMEs)" if you're using one.
  • If using a CNAME, make sure your DNS is correctly configured to point it to your CloudFront distribution's name.
  • Lastly, once you set the configurations there is a long wait while AWS updates the distribution, so you won't see your changes happen right away. It may seem like your app/website is still broken until the changes propagate through CloudFront. This can make configuring it difficult because if you get it wrong you have to wait a long time before you can see your changes take effect and you may not be sure what happened. But with these settings I was able to get it working for me.
  • You can also create more than one caching path pattern so that some content is private and requires a CloudFront signed url, and other content isn't. For example, I set a path pattern of *.mp4 that requires a signature for all mp4 files and placed that above the default behavior. And then I have the default cache behavior set to NOT require signed urls, which allows all other files - such as images - to be publicly accessible through the CloudFront distribution.

CarrierWave won't work when you set config.fog_public = false and point config.asset_host to a CloudFront distribution. This has been documented multiple times:

https://github.com/carrierwaveuploader/carrierwave/issues/1158 https://github.com/carrierwaveuploader/carrierwave/issues/1215

In a recent project I was happy using CarrierWave to handle uploads to S3, but wanted it to return a signed CloudFront URL when using Model.attribute_url. I came up with the following (admittedly ugly) workaround that I hope others can benefit from or improve upon:

Add the 'cloudfront-signer' gem to your project and configure it per the instructions. Then add the following override of /lib/carrierwave/uploader/url.rb in a new file in config/initializers (note the multiple insertions of AWS::CF::Signer.sign_url):

module CarrierWave
      module Uploader
        module Url
          extend ActiveSupport::Concern
          include CarrierWave::Uploader::Configuration
          include CarrierWave::Utilities::Uri

          ##
          # === Parameters
          #
          # [Hash] optional, the query params (only AWS)
          #
          # === Returns
          #
          # [String] the location where this file is accessible via a url
          #
          def url(options = {})
            if file.respond_to?(:url) and not file.url.blank?
              file.method(:url).arity == 0 ? AWS::CF::Signer.sign_url(file.url) : AWS::CF::Signer.sign_url(file.url(options))
            elsif file.respond_to?(:path)
              path = encode_path(file.path.gsub(File.expand_path(root), ''))

              if host = asset_host
                if host.respond_to? :call
                  AWS::CF::Signer.sign_url("#{host.call(file)}#{path}")
                else
                  AWS::CF::Signer.sign_url("#{host}#{path}")
                end
              else
                AWS::CF::Signer.sign_url((base_path || "") + path)
              end
            end
          end

        end # Url
     end # Uploader
end # CarrierWave

Then override /lib/carrierwave/storage/fog.rb by adding the following to the bottom of the same file:

require "fog"

module CarrierWave
  module Storage
    class Fog < Abstract
       class File
          include CarrierWave::Utilities::Uri
          def url
             # Delete 'if statement' related to fog_public
             public_url
          end
       end
    end
  end
end

Lastly, in config/initializers/carrierwave.rb:

config.asset_host = "http://d12345678.cloudfront.net"

config.fog_public = false

That's it. You can now use Model.attribute_url and it will return a signed CloudFront URL to a private file uploaded by CarrierWave to your S3 bucket.


It looks like you haven't added the line below to your config. You will need to replace the sample address below with your cloudfront address from Amazon.

From the github README: https://github.com/jnicklas/carrierwave

"You can optionally include your CDN host name in the configuration. This is highly recommended, as without it every request requires a lookup of this information"

config.asset_host = "http://c000000.cdn.rackspacecloud.com"

Tags:

Ruby On Rails

Amazon S3

Cdn

Carrierwave

Recent Posts