Iterating through the regex find

Try the following:

String formula = "Input!RC + Calc!R[1]C[1]";
Matcher m = Pattern.compile("\\W(R(\\[(.+?)\\])?C(\\[(.+?)\\])?)").matcher(formula);
while (m.find()) {
    System.out.println(m.group(1));
}

Output:

RC
R[1]C[1]

The main change here is how the loop works, what I have above is the typical way for iterating over matches of a pattern. I am also printing m.group(1) instead of m.group(3) because it is the first group that will contain the entire match except for the !, which I think is what you want.

The only change to your regex here was adding the ? after the second (\\[(.+?)\\]) group to make it optional.

If you look at this piece of code:

for (; m.find(); m.reset(formula)) {
    System.out.println(m.group(3));
}

For every loop, it calls successively m.reset(formula), resetting the matcher so that it starts at 0, then m.find(), which look for the next match in the string. Since you called reset() before, the next match is the first match. Therefore you have an infinite loop.

If you want to find all matches in the string, simply write:

while(m.find()) {
    System.out.println(m.group(1));
}