In XAML I want to use StringFormat to display a numerical value without any rounding happening. Is it possible?

I do not think you can do this simply with using StringFormat. Is there a reason why you do not want to use a converter? (see below).

Converter:

public class TruncateConverter : IValueConverter
{
    public object Convert(object value, Type targetType, object parameter, CultureInfo culture)
    {
        if (value != null)
        {
            double truncated;
            if (Double.TryParse(value, out truncated))
            {
                return ((double)((int)(truncated * 1000.0))) / 1000.0;
            }
        }
        return string.Empty;
    }

    public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture)
    {
        return DependencyProperty.UnsetValue;
    }
}

Usage:

<TextBlock Text="{Binding MyDecimalProperty, Converter={StaticResource TruncateConverter}}"/>

If you do not want to use a converter (I am not sure why, but you will have your reasons) you can consider changing the returned type of your binding source and then implementing IFormattable interface.

Take a look to this sample. First of all your MyDecimalProperty should not return a decimal anymore. You should change it in order to return a TruncatedDecimal. Here its code:

public class TruncatedDecimal : IFormattable
{
    private decimal value;

    public TruncatedDecimal(decimal doubleValue)
    {
        value = doubleValue;
    }

    public decimal Value
    {
        get
        {
            return value;
        }
    }

    public string ToString(string format, IFormatProvider formatProvider)
    {
        int decimalDigits = Int32.Parse(format.Substring(1));
        decimal mult = (decimal)Math.Pow(10, decimalDigits);
        decimal trucatedValue = Math.Truncate(value * mult) / mult;

        return trucatedValue.ToString(format, formatProvider);
    }
}

Now in your XAML your can use the format string that you need:

<TextBlock Text="{Binding StringFormat={}N5}" Margin="5" FontSize="20" />

So, for example, if the DataContext is set in this way:

DataContext = new TruncatedDecimal(new Decimal(.4997888869));

You will see 0.49978. I hope this solution will be suitable for your needs and can help you.