Karma Chrome Headless not working on Jenkins

For me, below worked, seems version and conf issue:

In package.json I am using:

"karma": "1.3.0",
"karma-chrome-launcher": "2.2.0"

In karma conf add this:

    browsers: ['ChromeHeadlessNoSandbox'],
    customLaunchers: {
        ChromeHeadlessNoSandbox: {
            base: 'ChromeHeadless',
            flags: ['--no-sandbox']
        }
    }

It shall work.

Based on issue Karma 1.6 breaks Headless support for Chrome created on github, it is related to the slower machine and happens, because it took > 60 seconds before test bundle was parsed and executed by Chrome and therefore test run was started and communicated back to Karma server. Reasons why it may take long vary.

There are 2 ways to handle timeout:

Investigate why your test bundle loads >60 seconds and make sure it loads faster.

1. Increase browserNoActivityTimeout to highter value, so test bundle has enough time to load.
2. This particular appearance of timeout does not seem to be a Karma issue, but rather problem in the project or misconfiguration.

Based on the Derek's comment

There was a connection that was disconnecting too soon.

He found that in /static/karma.js, when the socket was created, there was a timeout value that is hardcoded to 2 seconds (see below). He just added another 0 to make it 20 seconds and the connection stayed open long enough for the server to respond to the initial request. karma/client/main.js

Lines 14 to 20 in e79463b

var socket = io(location.host, { 
   reconnectionDelay: 500, 
   reconnectionDelayMax: Infinity, 
   timeout: 2000, 
   path: KARMA_PROXY_PATH + KARMA_URL_ROOT.substr(1) + 'socket.io', 
   'sync disconnect on unload': true 
 }) 

The next problem he faced was that Karma thought there was no activity even though there was traffic going back and forth on the socket. To fix that he just added browserNoActivityTimeout: 60000 to the Karma configuration.

You need to change the timeout configuration more then that is in the configuration file.

had the same issue "ChromHeadless have not captured in 60000 ms" (failing after 3 attempts), on Jenkins running on RHEL 7.5. Tried several configurations, and eventually adding the --proxy-bypass-list, and --proxy-server made it work.

minimal working configuration

 browsers: ['HeadlessChrome'],
    customLaunchers:{
      HeadlessChrome:{
        base: 'ChromeHeadless',
        flags: [
          '--no-sandbox',
          '--proxy-bypass-list=*',
          '--proxy-server=\'http://<my org proxy server>:8080\''
       ]
      }
    },

Below you can see a few more options in the config like the one I used. We have two browser configuration, Chrome for day to day dev work where we do want to see the browser open, and a headless chrome for CI/CD tests when building our solution on a Jenkins server.

Command line to run it in Jenkins:

npm run test -- -cc -sr --browser HeadlessChrome

In package.json we added a couple of lines to the scripts section:

 "test": "ng test",
    "test-dev": "ng test --browser Chrome",

karma.conf.js

 browsers: ['Chrome', 'HeadlessChrome'],
    customLaunchers:{
      HeadlessChrome:{
        base: 'ChromeHeadless',
        flags: [
          '--no-sandbox',
       //   '--remote-debugging-port=9222',
       //   '--enable-logging',
       //   '--user-data-dir=./karma-chrome',
       //   '--v=1',
       //   '--disable-background-timer-throttling',
       //   '--disable-renderer-backgrounding',
          '--proxy-bypass-list=*',
          '--proxy-server=\'http://<my org proxy server>:8080\''
       ]
      }
    },

After the above steps it worked from a shell on the Jenkins machine. However it failed when running as a Jenkins job with Cannot start ChromeHeadless Trying to start ChromeHeadless again (1/2). printed to the console.

I compared the env variables, and after a few trial and errors found that XDG_DATA_DIRS environment variable exists when logged in a bash shell (where headless chrome tests succeeds), and this variable was not defined in the failing Jenkins job env. So adding it (copied from the shell env | grep XDG_DATA_DIRS) finally solved it. I guess I should check what is the minimal configuration/dirs I should put there, and what is the root cause, but its working now :-)

Added the below to the jenkins job just before running the test

export XDG_DATA_DIRS=/users/<jenkins user e.g. jk1003>/.local/share/flatpak/exports/share/:/var/lib/flatpak/exports/share/:/usr/local/share/:/usr/share/

Another possible solution

A friend told me that he solved such an issue long ago using Xvfb

package.json

"karma": "~1.7.1",
"karma-chrome-launcher": "~2.2.0",

karma.conf.js

browsers: ['ChromeHeadlessNoSandbox'],
customLaunchers: {
    ChromeHeadlessNoSandbox: {
        base: 'ChromeHeadless',
        flags: ['--no-sandbox']
    }
},

and need to use the below command with --source-map=false

"qa-test": "ng test --watch=false --progress=false --browsers=ChromeHeadless --code-coverage --source-map=false"

my build got succeeded after this change.