WPF Binding with StringFormat
Binding in WPF does not consider StringFormat while falling back to FallbackValue in case it fails.
You can use what leon suggested or go with PriorityBinding.
--EDIT--
This should work:
<TextBlock DataContext="{Binding Fail, FallbackValue=DEFAULT}" Text="{Binding StringFormat=VALUE IS {0}}" />