Flutter: What is the correct way to detect touch enter, move and exit on CustomPainter objects

That's because you are using one Listener per CustomPainter, you should use just one Listener for all your Stack.

And if you want to know if the current touch event is inside each Circle , you could use GlobalKeys to get the RenderBox for each Circle, then you have the renderBox, and the PointerEvent, you can easily check the HitTest, check the code:

class _MyHomePageState extends State<MyHomePage> {
  GlobalKey _keyYellow = GlobalKey();
  GlobalKey _keyRed = GlobalKey();

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        // Here we take the value from the MyHomePage object that was created by
        // the App.build method, and use it to set our appbar title.
        title: Text("title"),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Listener(
              onPointerMove: (PointerEvent details) {
                final RenderBox box = _keyRed.currentContext.findRenderObject();
                final RenderBox boxYellow =
                    _keyYellow.currentContext.findRenderObject();
                final result = BoxHitTestResult();
                Offset localRed = box.globalToLocal(details.position);
                Offset localYellow = boxYellow.globalToLocal(details.position);
                if (box.hitTest(result, position: localRed)) {
                  print("HIT...RED ");
                } else if (boxYellow.hitTest(result, position: localYellow)) {
                  print("HIT...YELLOW ");
                }
              },
              child: Stack(
                children: <Widget>[
                  CustomPaint(
                    key: _keyYellow,
                    painter: ShapesPainter(),
                    child: Container(
                      height: 400,
                      width: 400,
                    ),
                  ),
                  CustomPaint(
                    key: _keyRed,
                    painter: ShapesPainter1(),
                    child: Container(
                      height: 200,
                      width: 200,
                    ),
                  ),
                ],
              ),
            ),
          ],
        ),
      ),
    );
  }
}

Also I modified the hitTest method of your CustomPainters to ignore the touchs outside the circle.

class ShapesPainter extends CustomPainter {
  @override
  void paint(Canvas canvas, Size size) {
    final paint = Paint();
    // set the color property of the paint
    paint.color = Colors.yellow;
    // center of the canvas is (x,y) => (width/2, height/2)
    final center = Offset(size.width / 2, size.height / 2);

    // draw the circle on centre of canvas having radius 75.0
    canvas.drawCircle(center, size.width / 2, paint);
  }

  @override
  bool shouldRepaint(CustomPainter oldDelegate) {
    return true;
  }

  @override
  bool hitTest(Offset position) {
    final Offset center = Offset(200, 200);
    Path path = Path();
    path.addRRect(RRect.fromRectAndRadius(
        Rect.fromCenter(center: center, width: 400, height: 400),
        Radius.circular(center.dx)));
    path.close();
    return path.contains(position);
  }
}

class ShapesPainter1 extends CustomPainter {
  @override
  void paint(Canvas canvas, Size size) {
    final paint = Paint();
    // set the color property of the paint
    paint.color = Colors.red;
    // center of the canvas is (x,y) => (width/2, height/2)
    var center = Offset(size.width / 2, size.height / 2);

    // draw the circle on centre of canvas having radius 75.0
    canvas.drawCircle(center, size.width / 2, paint);
  }

  @override
  bool shouldRepaint(CustomPainter oldDelegate) {
    // TODO: implement shouldRepaint
    return true;
  }

  @override
  bool hitTest(Offset position) {
    final Offset center = Offset(100, 100);
    Path path = Path();
    path.addRRect(RRect.fromRectAndRadius(
        Rect.fromCenter(center: center, width: 200, height: 200),
        Radius.circular(center.dx)));
    path.close();
    return path.contains(position);
  }
}

I have developed a library called touchable for the purpose of adding gesture callbacks to each individual shape you draw on the canvas.

Here's what you can do to detect touch and drag on your circle.

Just Wrap your CustomPaint widget with CanvasTouchDetector. It takes a builder function as argument that expects your CustomPaint widget as shown below.

import 'package:touchable/touchable.dart';


CanvasTouchDetector(
    builder: (context) => 
        CustomPaint(
            painter: MyPainter(context)
        )
)

Inside your CustomPainter class's paint method , create and use the TouchyCanvas object (using the context obtained from the CanvasTouchDetector and canvas) to draw your shape and you can give gesture callbacks like onPanUpdate , onTapDown here to detect your drag events.

var myCanvas = TouchyCanvas(context,canvas);
myCanvas.drawRect( rect , Paint() , onPanUpdate: (detail){
    //This callback runs when you drag this rectangle. Details of the location can be got from the detail object.
    //Do stuff here. Probably change your state and animate
});