Tikz-Euclid, Incircle, and Calculated Coordinates
IMHO there is a numerical instability in the computations done by the macros of the tkz-euclide package. This has nothing to do with the coordinates set via the calc syntax. To see this, consider
\documentclass[tikz, border=1cm]{standalone}
\usepackage{tkz-euclide}
\begin{document}
\begin{tikzpicture}
\coordinate (A) at (0,0);
\coordinate (B) at (4,0);
% \coordinate (C) at ($(A) + (2,-3)$); %({-60}:4)
\coordinate (C) at ({-60.01}:4); %({-60}:4)
% \coordinate (C) at (2,-3);
\draw (A)--(B)--(C)--cycle;
% circumcircle
\tkzCircumCenter(A,B,C)\tkzGetPoint{O}
% \tkzDrawPoint(O)
\tkzDrawCircle(O,A)
% incircle
\tkzDefCircle[in](A,B,C)\tkzGetPoint{I}\tkzGetLength{rIN}
\tkzDrawPoint(I)
\tkzDrawCircle[R](I,\rIN pt)
\end{tikzpicture}
\end{document}
Everything is cool (except for the bounding box, which could be computed correctly if tkz-euclide were to put overlay when it is computing the auxiliary paths, but this is off-topic here).
However, if I remove the last digit of the angle that defines C, i.e. do
\coordinate (C) at ({-60}:4);
I get what you got
I stress that I have even made an attempt to figure out what's going on internally, simply because I don't speak French and don't understand many things in the manual or code.
Bottom-line: To me it seems that you have discovered an issue in tkz-euclide.
ADDENDUM: Why does that happen here? To understand this, let us recall how incircles are computed. To this end, I follow this nice answer, which uses Heron's formula to compute the radius \inrad of the incircle. This tells us that the incircle is where the lines that are parallel to the edges with distance \incircle intersect. However, for a given edge there are two parallel lines with that distance.
\documentclass[tikz, border=1cm]{standalone}
\usepackage{tkz-euclide}
\begin{document}
% from https://tex.stackexchange.com/a/299451/121799
% calculate semiperimeter
% \pgfmathsetmacro{\semip}{(\sideA+\sideB+\sideC)/2}
% calculate radius of incircle: area (given by Heron's formula) divided by semiperimeter
% \pgfmathsetmacro{\inrad}{sqrt(\semip*(\semip-\sideA)*(\semip-\sideB)*(\semip-\sideC))/\semip}
\begin{tikzpicture}[globalize/.code n args={2}{\xdef#2{#1}}]
\coordinate (A) at (0,0);
\coordinate (B) at (4,0);
% \coordinate (C) at ($(A) + (2,-3)$); %({-60}:4)
\coordinate (C) at ({-60.01}:4); %({-60}:4)
% \coordinate (C) at (2,-3);
\path let \p1=($(A)-(B)$), \p2=($(B)-(C)$),\p3=($(C)-(A)$),
\n1={0.5*(veclen(\x1,\y1)+veclen(\x2,\y2)+veclen(\x3,\y3))},
\n2={sqrt(((\n1-veclen(\x1,\y1))/\n1))*sqrt((\n1-veclen(\x2,\y2))*(\n1-veclen(\x3,\y3)))}
in [globalize={\n2}{\inrad}];
\foreach \X/\Y in {A/B,B/C,C/A}
{\draw[gray,thin] ($(\X)!\inrad!90:(\Y)$) -- ($(\Y)!\inrad!-90:(\X)$);
\draw[gray,thin] ($(\X)!\inrad!-90:(\Y)$) -- ($(\Y)!\inrad!90:(\X)$);}
\draw (A)--(B)--(C)--cycle;
% circumcircle
\tkzCircumCenter(A,B,C)\tkzGetPoint{O}
% \tkzDrawPoint(O)
\tkzDrawCircle(O,A)
% incircle
\tkzDefCircle[in](A,B,C)\tkzGetPoint{I}\tkzGetLength{rIN}
\tkzDrawPoint(I)
\tkzDrawCircle[R](I,\rIN pt)
\end{tikzpicture}
\end{document}
Looking at this picture, I think that tkz-euclide computes the intersection of the wrong (i.e. outer) parallels. This picture also shows that the incircle center is not precisely at the intersection of the parallel lines.
This suggest that, at least as long tkz-euclide requires proficiency in the French language, to compute the incircle radius and center without that package, but following this nice answer.
\documentclass[tikz,border=3.14mm]{standalone}
\usetikzlibrary{calc,intersections}
\begin{document}
\begin{tikzpicture}[globalize/.code n args={2}{\xdef#2{#1}}]
\coordinate (A) at (0,0);
\coordinate (B) at (4,0);
\coordinate (C) at ($(A)+({-60}:4)$);
\draw (A) -- (B) -- (C) -- cycle;
%
\path let \p1=($(A)-(B)$), \p2=($(B)-(C)$),\p3=($(C)-(A)$),
\n1={0.5*(veclen(\x1,\y1)+veclen(\x2,\y2)+veclen(\x3,\y3))},
\n2={sqrt(((\n1-veclen(\x1,\y1))/\n1))*sqrt((\n1-veclen(\x2,\y2))*(\n1-veclen(\x3,\y3)))}
in [globalize={\n2}{\inrad}];
\foreach \X/\Y in {A/B,B/C,C/A}
{\path[name path global=path-\X-\Y-1] ($(\X)!\inrad!90:(\Y)$) -- ($(\Y)!\inrad!-90:(\X)$);
\path[name path global=path-\X-\Y-2] ($(\X)!\inrad!-90:(\Y)$) -- ($(\Y)!\inrad!90:(\X)$);}
\foreach \X in {1,2}
{\foreach \Y in {1,2}
{\path[name intersections={of={path-A-B-\X} and {path-B-C-\Y},total=\t}]
\ifnum\t=1
(intersection-1) coordinate (incenter)
\else
\fi
;}
}
\draw (incenter) circle (\inrad);
\end{tikzpicture}
\end{document}
One can even go a step further and get rid of the intersections library, and make things more TikZy by using elementary trigonometry the determination of the center based on Cartesian coordinates.
\documentclass[tikz,border=3.14mm]{standalone}
\usetikzlibrary{calc}
\begin{document}
\begin{tikzpicture}[incircle/.style n args={3}{%
insert path={
let \p1=($(#2)-(#1)$), \p2=($(#3)-(#1)$),\p3=($(#2)-(#3)$),
\n1={0.5*(veclen(\x1,\y1)+veclen(\x2,\y2)+veclen(\x3,\y3))},
\n2={sqrt(((\n1-veclen(\x1,\y1))/\n1))*sqrt((\n1-veclen(\x2,\y2))*(\n1-veclen(\x3,\y3)))},
\n3={veclen(\x1,\y1)}, % length #1 -- #2
\n4={veclen(\x2,\y2)}, % length #1 -- #3
\n5={veclen(\x3,\y3)}, % length #2 -- #3
\n6={\n3+\n4+\n5}
in % \pgfextra{\typeout{\n1,\n2,\n3,\n4,\n5,\n6}}
(${(\n5/\n6)}*(#1)+{(\n4/\n6)}*(#2)+{(\n3/\n6)}*(#3)$) circle (\n2)
}}]
\coordinate (A) at (0,0);
\coordinate (B) at (4,0);
\coordinate (C) at ($(A)+({-60}:4)$);
\draw (A) -- (B) -- (C) -- cycle;
%
\draw[incircle={A}{B}{C}];
\end{tikzpicture}
\end{document}
This leads the same output as above. (When writing this, I found it is not at all trivial to grasp all cases, and there is no guarantee that I did. That is, I am really deeply impressed by Alain Matthes' achievements. In other words, even though I do think that there might be an issue with tkz-euclide, one should also highlight that it overall works great.)
You found a bug ... The macro \tkzDefCircle[in] is incorrect.
THe problem disappears with the new version of tkz-base and tkz-euclide.
You can find them Archive euclide
Now the correct code with tkz-euclide is
\documentclass[tikz, border=1cm]{standalone}
\usepackage{tkz-euclide}
\begin{document}
\begin{tikzpicture}
\tkzDefPoint(0,0){A}
\tkzDefPoint(4,0){B}
\tkzDefTriangle[equilateral](B,A)
\tkzGetPoint{D}
\tkzDrawPolygon(B,A,D)
% circumcircle
\tkzCircumCenter(A,B,D)\tkzGetPoint{O}
\tkzDrawCircle(O,A)
% incircle
\tkzDefCircle[in](A,B,D)\tkzGetPoint{I}\tkzGetLength{rIN}
\tkzDrawPoint(I)
\tkzDrawCircle[R](I,\rIN pt)
\end{tikzpicture}
\end{document}
Now you can avoid \usetkzobj{all}. The main modules are charged only very specific modules can be called with this macro.
Now how, why, where it's necessary or not to use calc , tikz etc.
It's preferable to use \tkzDefPoint instead of \coordinate when you use tkz-euclide because if you need some computations to get a point, tkz-euclide uses the package fp. Tikz and TeX are very fine tools but they are limited to calculate and quickly errors occur if one connects calculations. For example, you can look in the manual how to draw the heights of a triangle. The intersection of heights is defined very badly.
The best way to do calculations is to use lua. Perhaps the next version of euclide will be in two parts. The first one will use lua to define the points and then the second part will use tikz to draw the different objects.
Now you can use the calc syntax to get the coordinates but you have other ways
\documentclass[tikz, border=1cm]{standalone}
\usepackage{tkz-euclide}
\begin{document}
\begin{tikzpicture}
\tkzDefPoint(0,0){A}
\tkzDefPoint(4,0){B}
\coordinate (C) at ($(A) + ({60}:4)$);
% Better is \tkzDefTriangle[equilateral](B,A)
% or \tkzDefShiftPoint[A](-60:4){C}
% possible
% \begin{scope}[shift={(A)}]
% \tkzDefPoint(-60:4){C}
% \end{scope}
\tkzDrawPolygon(B,A,C)
% circumcircle
\tkzCircumCenter(A,B,C)\tkzGetPoint{O}
\tkzDrawPoint(O)
\tkzDrawCircle(O,A)
% incircle
\tkzDefCircle[in](A,B,C)\tkzGetPoint{I}\tkzGetLength{rIN}
\tkzDrawCircle[R,color=red](I,\rIN pt)
\end{tikzpicture}
\end{document}