String.CompareTo with case

This is the expected behavior. String.CompareTo(string) does a culture sensitive comparison, using its sort order. In fact it calls CultureInfo to do the job as we can see in the source code:

public int CompareTo(String strB) {
    if (strB==null) {
        return 1;
    }

    return CultureInfo.CurrentCulture.CompareInfo.Compare(this, strB, 0);
}

Your current culture puts 'A' after 'a' in the sort order, since it would be a tie, but not after 'ab' since clearly 'ab' comes after either 'a' or 'A' in most sort orders I know. It's just the tie breaking mechanism doing its work: when the sort order would be the same, use the ordinal value!