What is the right data type for calculations in Java

For general floating point calculations, you should use double. If you are absolutely sure that you really do need arbitrary precision arithmetic (most applications don't), then you can consider BigDecimal.

You will find that double will significantly outperform BigDecimal (not to mention being easier to work with) for any application where double is sufficient precision.

Update: You commented on another answer that you want to use this for a finance related application. This is one of the areas where you actually should consider using BigDecimal, otherwise you may get unexpected rounding effects from double calculations. Also, double values have limited precision, and you won't be able to accurately keep track of pennies at the same time as millions of dollars.

For a serious financial application BigDecimal is a must.

Depends on how many digits you need you can go with a long and a decimal factor for visualization.

How much is the overhead in terms of performance for BigDecimal as compared to double?

A lot. For example, a multiplication of two doubles is a single machine instruction. Multiplying two BigDecimals is probably a minimum of 50 machine instructions, and has complexity of O(N * M) where M and N are the number of bytes used to represent the two numbers.

However, if your application requires the calculation to be "decimally correct", then you need to accept the overhead.

However (#2) ... even BigDecimal can't do this calculation with real number accuracy:

       1/3 + 1/3 + 1/3  ->  ?

To do that computation precisely you would need to implement a Rational type; i.e. a pair of BigInteger values ... and some thing to reduce the common factors.

However (#3) ... even a hypothetical Rational type won't give you a precise numeric representation for (say) Pi.