To inscribe a circle in a given triangle

version 10

pA = {0, 0}; pB = {1.1, 1}; pC = {1.5, 0};
lr = MeshRegion[{pA, pB, pC}, Triangle[Range@3]];
r1 = RegionDistance[InfiniteLine[{pA, pB}], {x, y}];
r2 = RegionDistance[InfiniteLine[{pB, pC}], {x, y}];
r3 = RegionDistance[InfiniteLine[{pC, pA}], {x, y}];
centerC = {x, y} /. NSolve[{r1 == r2, r2 == r3}, {x, y}, Reals];
centerC = Select[centerC, RegionMember[lr, #] &][[1]]
raioC = RegionDistance[Line[{pA, pB}], centerC]
Graphics[{
  EdgeForm[{Thick, Blue}], White, Triangle[{pA, pB, pC}],
  PointSize[Large], Red, Point@centerC,
  Red, Circle[centerC, raioC]}
 ]

{0.954787, 0.369127}

0.369127

As KennyColnago has observed your equations seem to be off but there is another way of determining the incircle. I'd already typed it up with sources that may be helpful so let's post it, even if it's the same thing.

{a, b, c} = {Norm[pC - pB], Norm[pA - pC], Norm[pA - pB]};
area = With[{s = (a + b + c)/2}, Sqrt[s (s - a) (s - b) (s - c)]];
r = 2 area/(a + b + c);
incenter = (a pA + b pB + c pC)/(a + b + c);

Graphics[{
  EdgeForm[Black], White, Triangle[{pA, pB, pC}],
  Black, Point[incenter],
  Circle[incenter, r]
  }]

The formula for the coordinates of the incenter can be found here, the relationship between the radius and the area is discussed here and the area is calculated with Heron's formula found here.

It's also worth noting that we can "cheat" with Mathematica 10:

incenter = {a, b, c}.{pA, pB, pC}/ArcLength[Line[{pA, pB, pC, pA}]];
r = 2 Area[Triangle[{pA, pB, pC}]]/ArcLength[Line[{pA, pB, pC, pA}]];

Although this sort of cheating is not as outrageous as that by Junho Lee of course ;)

The incircle of a triangle may be calculated as follows.

InCircle[{x1_, y1_}, {x2_, y2_}, {x3_, y3_}] := 
   With[{
      a = Norm[{x2,y2} - {x3,y3}], 
      b = Norm[{x3,y3} - {x1,y1}], 
      c = Norm[{x1,y1} - {x2,y2}]}, 
   Circle[(a {x1, y1} + b {x2, y2} + c {x3, y3})/(a + b + c), 
          1/2 Sqrt[-(((a - b - c) (a + b - c) (a - b + c))/(a + b + c))]]]

For your example points, InCircle[{0,0},{1.1,1},{1.5,0}] produces Circle[{0.954787, 0.369127}, 0.369127], which may be plotted with

Graphics[{
   Line[{{0, 0}, {1.1, 1}, {1.5, 0}, {0, 0}}],
   Red, Thick, InCircle[{0, 0}, {1.1, 1}, {1.5, 0}]}]