Why >>24 causes -Wconversion but >>23 doesn't?

As mentioned by Shafik Yaghmour, this appears to be a bug in GCC:

GCC Bug 40752: -Wconversion generates false warnings for operands not larger than target type

It appears to have been present since version 4.4.0, first reported on 2009-07-14, and has 5 duplicates. Based on the comments in the bug report, there seems to be some debate on how to handle it.