data type of results of java arithmetic calculation

a. These rules are called numeric promotion rules and are specified in Java Language Specification, §5.6.2 (currently).

b. There are two generally accepted method for dealing with overflows.

The first method, a post-check, where you do an operation, say addition and then check that the result is greater than either of the operands. For example:

int c = a + b;

if( c<a) {  // assuming a>=0 and b>=0
   // overflow happened
}

The second method, is a pre-check, where you basically try to avoid the overflow from happening in the first place. Example:

if( a > Integer.MAX_INTERGER - b ) {
   // overflow happened
}

The specific section of the Java Language Specification that deals with these rules is section 4.

If you don't want values to overflow at all, use a BigInteger or some other arbitrary-precision arithmetic type.

For avoiding overflows in the general case, Guava (which I contribute to) provides methods like IntMath.checkedAdd(int, int) and LongMath.checkedMultiply(long, long), which throw exceptions on overflow. (Some of those are nontrivial to implement yourself, but these are all very exhaustively tested.) You can look at the source to see how they work, but most of them rely on cute bit-twiddling tricks to check for overflow efficiently.