How to use curl to get a GET request exactly same as using Chrome?
If you need to set the user header string in the curl request, you can use the -H option to set user agent like:
curl -H "user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36" http://stackoverflow.com/questions/28760694/how-to-use-curl-to-get-a-get-request-exactly-same-as-using-chrome
Updated user-agent form newest Chrome at 02-22-2021
Using a proxy tool like Charles Proxy really helps make short work of something like what you are asking. Here is what I do, using this SO page as an example (as of July 2015 using Charles version 3.10):
- Get Charles Proxy running
- Make web request using browser
- Find desired request in Charles Proxy
- Right click on request in Charles Proxy
- Select 'Copy cURL Request'
You now have a cURL request you can run in a terminal that will mirror the request your browser made. Here is what my request to this page looked like (with the cookie header removed):
curl -H "Host: stackoverflow.com" -H "Cache-Control: max-age=0" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36" -H "HTTPS: 1" -H "DNT: 1" -H "Referer: https://www.google.com/" -H "Accept-Language: en-US,en;q=0.8,en-GB;q=0.6,es;q=0.4" -H "If-Modified-Since: Thu, 23 Jul 2015 20:31:28 GMT" --compressed http://stackoverflow.com/questions/28760694/how-to-use-curl-to-get-a-get-request-exactly-same-as-using-chrome
Check the HTTP headers that chrome is sending with the request (Using browser extension or proxy) then try sending the same headers with CURL - Possibly one at a time till you figure out which header(s) makes the request work.
curl -A [user-agent] -H [headers] "http://something.com/api"
Open Chrome Developer Tools, go to Network tab, make your request (you may need to check "Preserve Log" if the page refreshes). Find the request on the left, right-click, "Copy as cURL".