Unable to locate elements on webpage with headless chrome

I had the same problem. You could take screenshots to understand whats wrong.

driver.get_screenshot_as_file("screenshot.png")

A few reasons why selenium works when run normally but stops working in headless mode -

1)It might have switched to a mobile template. Can be fixed by changing the window size.

chrome_options.add_argument("--window-size=1920,1080")

2)If its a blank page (screenshot), it might be due to an invalid SSL certificate.(see @Marcel_Wilson post) It should be fixed by -

chrome_options.add_argument('--ignore-certificate-errors')
chrome_options.add_argument('--allow-running-insecure-content')

3)Its possible that the website blocks 'headless' mode. (Your screenshots might show errors which you cannot recreate in normal mode) You could try this-

user_agent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.50 Safari/537.36'
options.add_argument(f'user-agent={user_agent}')

However, the above code won't work if the website has a more robust blocking method. You can find more about this here https://intoli.com/blog/making-chrome-headless-undetectable/.

I met the same situation. After studying, the following is correct：

self.chrome_options = webdriver.ChromeOptions()
self.chrome_options.add_argument("--window-size=1920,1080")
self.chrome_options.add_argument("--disable-extensions")
self.chrome_options.add_argument("--proxy-server='direct://'")
self.chrome_options.add_argument("--proxy-bypass-list=*")
self.chrome_options.add_argument("--start-maximized")
self.chrome_options.add_argument('--headless')
self.chrome_options.add_argument('--disable-gpu')
self.chrome_options.add_argument('--disable-dev-shm-usage')
self.chrome_options.add_argument('--no-sandbox')
self.chrome_options.add_argument('--ignore-certificate-errors')
self.browser = webdriver.Chrome(options=self.chrome_options)