Waggle dance behavior when `Arrowheads` is used to label an arrow

I wonder if this is the programmer breaking an unwritten rule about the numerics of graphics. I mean I would expect numerics issues in graphics, just like in other floating-point computing. For example, Cos[Pi/2.] != 0 returns True.

Some evidence for @Wjx's comment and the preceding cosine example:

Graphics[{
  Line[{{-1, 0}, {1, 0}}],
  Text[
   Framed[Style["wanndance?", Medium], Background -> LightGray, 
     FrameStyle -> Directive[Thickness[0], Opacity[0]]],
   {0, 0},
   {0, 0},
   Through[{Cos, Sin}[ArcTan[$MachineEpsilon, Cos[Pi/2.]]]]  (* direction vector *)
  }], 
 ImageSize -> Tiny]

Looks like the same angle:

I'm not sure it is fair to call this a bug in Mathematica.

There is no reason to specify $MachineEpsilon in the definition of annotatedArrow. I believe a correct specification would be would Automatic. The following code which uses Automatic is entirely well-behaved.

annotatedArrow[p_, q_, label_] :=
  {Arrowheads[
     {{-0.05, 0},
      {Automatic, 0.5, 
         Graphics[
           Inset[Framed[Style[label, Medium], 
           Background -> LightGray, 
           FrameStyle -> Directive[Thickness[0], Opacity[0]]]]]},
      {0.05, 1}}],
   Arrow[{p, q}]}

I assert the bug demonstrated in the question is not a Mathematica bug, but a programming error on the part of the OP.