Is JUnit the right tool to write performance tests?

There may be better approaches, but there are also some frameworks that help implement benchmarking with JUnit. Some useful practices are warmup runs, statistical evaluations. Take a look at JUnitBenchmarks and JUnitPerf

EDIT looks like JUnitBenchmarks has been deprecated since the question has been stated. The project's maintainers recommend moving on to JMH. Thank you, Barry NL for the heads-up.

Your tests should be better defined as benchmark tests. Yes, JUnit can be used this way although it is not the best choice. But you can for example define max. evaluation time for test, so if algorithm is changed that caused performance degradation the test fails. Use @Test(timeout=12345) for configuration.

If you need real performance test think about JMeter.

No. JUnit is designed for unit testing. There are lot of things that need to be taken care while writing performance tests in Java. Use Google Caliper which is specifically designed for writing micro-benchmark tests.

Look at How do I write a correct micro-benchmark in Java?