Googlebot causes an invalid Cross Origin Request (COR) on Rails 4.1
As per "CSRF protection from remote tags " from the rails guide:
In the case of tests, where you also doing the client, change from:
get :index, format: :js
To:
xhr :get, :index, format: :js
http://edgeguides.rubyonrails.org/upgrading_ruby_on_rails.html#csrf-protection-from-remote-script-tags
In the case you want to make this route skip csrf check, white list the route using something like:
protect_from_forgery :except => :create
Googlebot is using the format "*/*" (http://apidock.com/rails/Mime) and the application renders the js since it's the only thing available. Since it's remote, it correctly causes an Invalid COR.
This was reproducible using:
curl -H "Accept: */*" https://www.example.com/users/123/flag
The fix is to have an html fallback resource for the spider to crawl:
respond_to do |format|
format.html { render template: 'users/flag' }
format.js { render template: 'users/flag', layout: "some_layout" }
end