How to set amazon S3 bucket policy to private to everyone except admin?

Ok, so as far as I understand if I only set allow to me then it should be deny to everyone else by default.

I tried to set that bucket policy but my files are still downloadable : just remove the access key id from the URL, like : http://s3.amazonaws.com/myBucket/myFile.pdf

My files ACL are still set as :public-Read so it seems that indeed I have a conflict between bucket policy and individual files ACL.

I gave up making all files private through bucket policy, if anybody ever needs to make a large number of files private inside a S3 bucket, here is the rake task I finally wrote :

  desc "Make all objects in S3 private"
  task :make_private  => :environment do
    require 'aws/s3'

    bucket_name = 'yourBucket'
    marker = ""

    AWS::S3::Base.establish_connection!(
      :access_key_id => "yourKey",
      :secret_access_key => "yourSecret"
    )

    #create the read-only by me policy
    owner_grant = ACL::Grant.new
    grantee = ACL::Grantee.new
    owner_grant.grantee = grantee
    owner_grant.permission = 'READ'
    grantee.type = "CanonicalUser"
    grantee.id = 'yourID'
    grantee.display_name = "yourName"


    # Iterate over all files inside bucket and apply the policy to each files
    loop do
      objects = Bucket.objects(bucket_name, :marker=>marker, :max_keys=>1000)

      marker = objects.last.key
      puts "new marker is \"#{marker}\""

      objects.each do |obj|
          policy = S3Object.acl(obj.key, bucket_name)
          policy.grants = [owner_grant]
          S3Object.acl(obj.key, bucket_name, policy)
      end
    end
  end

P.S: For info I have tried to change all files ACL using Firefox S3 Organizer or bucket explorer, none of them works if you have several hundreds of thousands of files, they just freeze.

This is caused by the respective Evaluation Logic of the The Access Policy Language used with Bucket Policies:

The goal at evaluation time is to decide whether a given request should be allowed or denied. The evaluation logic follows several basic rules:

• By default, all requests to use your resource coming from anyone but you are denied

• An allow overrides any default denies

• An explicit deny overrides any allows

• The order in which the policies are evaluated is not important

[emphasis mine]

The page also provides an instructive flow chart and discussion [which] describe in more detail how the decision is made.

So your Deny "*" overrides your Allow "arn:aws:iam::6527...3775:root". As illustrated in the flow chart linked above, you can work around this by removing the explicit deny in favor of the default deny (please be aware of the potential subtleties when Using ACLs and Bucket Policies Together, which don't seem to apply to your use case though).

The policy you were using does not work because the deny takes precedence over the allow, so all users are denied access. The correct way to do this is using the NotPrincipal policy element. It allows you to apply a policy to all principles except a specific list. Your policy should then be:

{
    "Version": "2008-10-17",
    "Id": "Policy1331136935471",
    "Statement": [
        {
            "Sid": "Stmt1331136294179",
            "Effect": "Deny",
            "NotPrincipal": {
                "AWS": "arn:aws:iam::6527...3775:root"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::myBucket/*"
        },
        {
            "Sid": "Stmt1331136364169",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::6527...3775:root"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::myBucket/*"
        }
    ]
}

Note that I don't think the allow is actually necessary because your account should have access to the files because it is the bucket/object owner who is granted access by default. Though that depends on the ACLs of your objects.