$a,b$ in $G$ has finite order. Then is the order of $ab, ba, a^{-1}b^{-1}$ with conditions finite?

You idea is a good way to approach things.

Let's look at each of the cases in turn. You know that $a$ and $b$ have finite order, so there exist some $m,n\in\mathbb{N}$ such that $a^n=b^m=e$ (where we are assuminng the group is written multiplicatively and that it has identity element $e$).

1) If you assume that $ab=ba$, then what do you know about $(ab)^2=abab$.? Can you find a way of writing this in the form $a^rb^s$ for some $r,s$.? Can you do something similar for $(ab)^i$ for an arbitrary value of $i$.? You should be able to, and then you can use your assumption that $a^n=b^m=e$ to find an appropriate $i$ such that $(ab)^{i}=e$.

2) If $ab$ has finite order, then you know that there exists some $j$ such that $(ab)^j=e$. Can you use this to determine anything about $(ba)^{j+1}$. Try working this out and writing it in the form $b^ra^s$ for some $r,s$. You should then be able to prove part (2).

3) Assuming you have proved part (2), what do you know about $(ba)^{-1}$. In particular, what do you know about it if $(ab)$ (and hence by part (2) $(ba)$) has finite order?