Karma vs testing framework Jasmine, Mocha, QUnit

The thesis of the guy who designed Karma was very informative in describing existing solutions and comparing them, and of course describing Karma itself

https://github.com/karma-runner/karma/blob/master/thesis.pdf

Summary: Karma is a test runner. It can be used by QUnit, Jasmine, Mocha, ... Karma has advantages to other test runners to improve your TDD/BDD development cycle. It "watches" files, so when you save a change, Karma runs tests and reports back instantly, no switching context to Web Browser to run the test.

In short, perhaps question should be Karma AND Jasmine or Mocha or QUnit?

One shorter way to understand the difference:

People testing with plain Jasmine / Mocha most likely are running all the code with the Node virtual machine.

Adding Karma to the mix (on top of your existing framework of choice) will run your test suite with the engine of other browsers.

By doing this you get the little extras you get with a browser environment. It will be easier to test DOM related code, but you will also be giving up to the extra resources given by the Node engine (like filesystem / shell access)

Karma is a browser test runner.

The idea is that browsers do not have natively a concept of loading tests files, running them, and reporting results. What karma does is (roughly) :

• starting a small web server to serve "client-side" javascript files to be tested (1)
• also serve the "client-side" javascript files with the tests (or Specs, as they're often called) (2)
• serve a custom web page that will run javascript code for the tests (3)
• start a browser to load this page (4)
• report the results of the test to the server (5)
• karma can then again report the results to text files, the console, anything your CI server likes, etc...

Looking at each part :

(1) Those files will be your actual js files ; you will tell karma how to load them. If you use requirejs, there is a karma plugin, and some config is needed.

(2) Those tests can be written in a variety of Javascript testing framework (Jasmine, QUnit, Mocha) ; this is JS code that is run in the browser.

(3) The custom web page will be a bit different for each testing framework ; this is why karma has plugins for different frameworks.

(4) Karma can launch the page in many browsers (FF, Chrome, or headless browsers like PhantomJs.)

(5) Reporting to karma is, again, framework-dependant, and dealt with karma plugins.

So to answer your questions :

• in Java, most people use JUnit which is both a framework to write tests and run them, but does not have the problem of differentiating the environment in which tests are run and the one in which test reports are aggregated ; karma would be the missing piece between a JUnit Suite and a JUnit TestRunner
• Yes, you can do everything that karma does "by hand" - pick one framework (jasmine, qunit, mocha) and follow instructions. The advantage of karma is that it provide a solution out-of-the-box, if you're in a standard setup.
• Karma can be used for both unit test (with jasmine / qunit / whatever) and integration tests (which will use another API, like webdriver, to drive the browser)