How to grab headers in python selenium-webdriver

You could try Mobilenium, a python package (still in development) that binds BrowserMob Proxy and Selenium.

An usage example:

>>> from mobilenium import mobidriver
>>>
>>> browsermob_path = 'path/to/browsermob-proxy'
>>> mob = mobidriver.Firefox(browsermob_binary=browsermob_path)
>>> mob.get('http://python-requests.org')
301
>>> mob.response['redirectURL']
'http://docs.python-requests.org'
>>> mob.headers['Content-Type']
'application/json; charset=utf8'
>>> mob.title
'Requests: HTTP for Humans \u2014 Requests 2.13.0 documentation'
>>> mob.find_elements_by_tag_name('strong')[1].text
'Behold, the power of Requests'

You can get the header via the log (source from Mma's answer)

from selenium import webdriver
import json
driver = webdriver.PhantomJS(executable_path=r"your_path")
har = json.loads(driver.get_log('har')[0]['message']) # get the log
print('headers: ', har['log']['entries'][0]['request']['headers'])

Unfortunately, you cannot get this information from the Selenium webdriver, nor will you be able to any time in the near future it seems. An excerpt from a very long conversation on the subject:

This feature isn't going to happen.

The gist of the main reason being, from what I gather from the discussion, that the webdriver is meant for "driving the browser", and extending the API beyond that primary goal will, in the opinion of the developers, cause the overall quality and reliability of the API to suffer.

One potential workaround that I have seen suggested in a number of places, including the conversation linked above, is to use BrowserMob Proxy, which can be used to capture HTTP content, and can be used with selenium - though the linked example does not use the Python selenium API. It does seem that there is a Python wrapper for BrowserMob Proxy, but I cannot vouch for it's efficacy since I have never used it.


Now, it is very easy i suppose https://pypi.org/project/selenium-wire/ it is an extension of selenium. use from seleniumwire import webdriver and proceed as usual.