Get CSRF token using python requests

See the following code example. You can use it directly to login into a website that only uses cookies to store login information.

import requests

LOGIN_URL = 'https://examplenotarealpage.com'
headers = {
    'accept': 'text/html,application/xhtml+xml,application/xml',
    'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'
}

response = requests.get(LOGIN_URL, headers=headers, verify=False)

headers['cookie'] = '; '.join([x.name + '=' + x.value for x in response.cookies])
headers['content-type'] = 'application/x-www-form-urlencoded'
payload = {
    'username': 'user_name',
    'password': 'randompass123'
}

response = requests.post(LOGIN_URL, data=payload, headers=headers, verify=False)
headers['cookie'] = '; '.join([x.name + '=' + x.value for x in response.cookies])

There are a few possible locations of the CSRF token. Different websites use different ways to pass it to browser. Here are some of them:

• It can come with response headers, in that case getting it is easy.

• Sometimes page meta holds the CSRF token. You have to parse the html content of the page to get it. Find the proper CSS selector for it. See an example:

  from bs4 import BeautifulSoup
  soup = BeautifulSoup(response.text, 'lxml')
  csrf_token = soup.select_one('meta[name="csrf-token"]')['content']
  

• It can be inside of a script tag with JavaScript code. Getting it will be tricky. But, you can always use regex to isolate it.