is calling libgdx SpriteBatch begin and end method multiple times expensive?

It is not expensive. Do not worry. Just remember you don't have to, if you are using different textures you don't have to call begin and end for each. it automatically switches the texture for the region you are drawing. Of course, this means you have to group the drawing calls of the regions.

This will explain it better, suppose we have an atlas with all the TextureRegions of Hero and Enemies, and another one with all the TextureRegions of Coins and Platforms:

//You only need to call batch.begin once:

batch.begin();
//then draw
//-Hero
//-Enemies
//(Automatic texture switching here)
//-Coins
//-Platforms
batch.end();

As a little plus, there is a special case: Usually you have a background that fills all the screen, totally opaque. So disabling blending for that draw definitely improves performance. So you batch once for it, and then again for the other regions, like this:

batch.disableBlending();
batch.begin();
//draw background
batch.end();

batch.enableBlending();
batch.begin();
//etc

Going from 2 calls to 5 calls isn't going to be too big of a deal.

However, in general, the end call causes a batch to flush out all the state it has accumulated and invoke the underlying OpenGL draw call. (It may flush earlier if it fills up, or you switch textures, but generally the flush happens at end time.) Reducing invocations of OpenGL draw calls and texture uploads is the major motivation for the SpriteBatch class.

There are some counters on the SpriteBatch you can use to see how its actually doing in practice that will help point out if you're doing it too much. See https://code.google.com/p/libgdx/wiki/SpriteBatch#Performance_tuning