C#: better way than to combine StartsWith and two ToUpperInvariant calls

You can use the overloaded StartsWith method taking a StringComparison enum value:

keyAttributeValue.StartsWith(STR_ConnectionString, StringComparison.OrdinalIgnoreCase) // or use StringComparison.InvariantCultureIgnoreCase here

There is a StartsWith overload which supports case-insensitive matching:

if (keyAttributeValue.StartsWith(STR_ConnectionString, 
                                 StringComparison.InvariantCultureIgnoreCase)
{
    ...
}

It also makes your code more readable, because it expresses your intention: What you really want is a case-insensitive comparison, and that's what's written here. You don't really want "a case-sensitive comparison of values converted to upper-case"... that's just the workaround you use to achieve the goal.

If it smells bad because you're doing the ToUpper then the string compare, those can be combined using an overload of startswith:

STR_ConnectionString..StartsWith(upperCaseConnectionString, StringComparison.CurrentCultureIgnoreCase);

However, it looks like you're rolling your own way to handle application configuration, which you shouldn't do. See http://msdn.microsoft.com/en-us/library/system.configuration.configurationmanager.aspx