How to use custom properties in a ControlTemplate trigger
Of course it only takes me 30 minutes after posting my question to find the answer when I spent two days looking for it first. Oh well, the solution is to use DataTriggers.
<ControlTemplate.Triggers>
<DataTrigger Binding="{Binding RelativeSource={RelativeSource Self}, Path=ThumbIsVisible}" Value="False">
<Setter TargetName="m_Thumb" Property="Visibility" Value="Hidden" />
</DataTrigger>
</ControlTemplate.Triggers>
The key is to use the RelativeSource={RelativeSource Self} to find the custom property. After that it works exactly as expected.
It looks like there is an even simpler way to solve this problem.
<ControlTemplate.Triggers>
<Trigger Property=local:CustomSlider.ThumbIsVisible" Value="False">
<Setter TargetName="m_Thumb" Property="Visibility" Value="Hidden" />
</Trigger>
</ControlTemplate.Triggers>
where local is the namespace of the CustomSlider class.