Why does Postgres ORDER BY seem to halfway ignore leading underscores?

As you haven't defined a different collation for your column in question, it uses the database-wide one, which is en_US.UTF8 - just like on my test box. I observe the exact same behaviour, take it as a consolation :)

What we see is apparently a case of the variable collation elements. Depending on the character and the collation, a number of different behaviours is possible. Here the underscore (and the hyphen and some others, too) are used only for breaking ties - 'a' and '_a' are equivalent in the first round, then the tie between them is resolved by taking the underscore into account.

If you want to sort with ignoring the underscores (and hyphens, question marks and exclamation marks in my example), you can define an ordering on an expression:

SELECT * 
FROM (VALUES ('a'), 
             ('b1'), 
             ('_a'), 
             ('-a'), 
             ('?a'), 
             ('!a1'), 
             ('a2')
     ) t (val) 
ORDER BY translate(val, '_-?!', '');

In my experiments adding a new value to the list often changes the order between otherwise equal items, showing they are treated really equal.