RemoveView not working

I had that same issue when trying to create a custom banner. I believe it's because of animation during layout, that's why a delay could work. In my case, I made a custom viewgroup class to eliminate the animation delay:

private class BannerLayout extends LinearLayout {
  public BannerLayout(Context context) {
     super(context);
  }

  @Override
  protected void removeDetachedView(View child, boolean animate) {
     super.removeDetachedView(child, false);
  }
}

Once I did this, everything worked as expected.

Hope it helps!