True difference between HttpRequest and XMLHttpRequest
As I discussed in the comments, I can reproduce: the XMLHttpRequest example from the first "Edit" works (HTTP status = 200), while the "copy as cURL" version of it returns 403 from Cloudfare.
Adding --cert-status
to curl makes it work for me, so it seems that Cloudfare analyzes TLS-level communication when deciding to deny a request.
Your curl command from the first Edit has a few other differences from the version I get when using "Copy as cURL":
curl 'URL'
instead ofhttps://haapi.ankama.com/json/Ankama/v2/Api/CreateApiKey
obviously fails, please don't make it harder to reproduce your results.-H 'origin: null'
vs-H 'Origin: https://localhost:4443' -H 'Referer: https://localhost:4443/test_http.html'
- this doesn't make a difference.- I have a few additional headers
-H 'DNT: 1' -H 'Connection: keep-alive' -H 'Cookie: __cfduid=dcf1b80eef19562054c9b64f79139509e1566138746'
that don't make a difference either. - Varying
-H 'user-agent:
- doesn't affect Cloudfare either - You have an extra
-H 'authority: URL.com'
(with placeholder in place of the real domain), and this doesn't make a difference either. - Whether the POST data is correct
--data-binary 'login=123&password=def'
only affects the API results; doesn't affect the 403. - The missing
-H 'Accept-Language:
header causes the 403 from Cloudfare.
So you could try adding the missing Accept-Language
to the Node version to see if it helps.
My version of Node doesn't send Extension: status_request
in the TLS Client Hello (which seems to be the difference between curl invocations with or without --cert-status
), and I don't see how you would enable it. At this point I'd try contacting support if possible or falling back to calling curl from node.
P.S. while debugging it I attempted to compare the Wireshark captures of curl vs browser (node doesn't support SSLKEYLOGFILE
, forcing you to jump through hoops, so I didn't even try checking how its capture looks). There are so many minor differences, that trying to reverse engineer the rules that Cloudfare uses would be very time-consuming. --cert-status
was a lucky guess.
The SSL Client Hello across Firefox/curl/node are very different: Firefox curl node11