How can I stop my installer from triggering Windows 10's "This app has been blocked for your protection" error?
You are almost surely talking about SmartScreen, covered by this superuser.com Q+A. Not new in Windows 10, it has been around for quite a while already. Originally started in IE8, integrated into the operating system at Windows 8. A version that was skipped by many users so easy to think it is a new malady in Win10.
You won't like that Q+A, nor what is behind this feature. But a very basic truth is that a certificate just isn't enough anymore to gain trust. It proves so very little, just that the author has been willing to keep up the payments to a certificate authority. A low one these days, compared to what it used to be, with companies like GoDaddy seeing good business in selling a number for hundreds of dollars. There is no connection whatsoever between having a certificate and an installer that's trustworthy enough to not mess up a machine. And no way for a user to complain.
SmartScreen is reputation based, not unlike the way StackOverflow works. SO users trust Jon Skeet answers, SmartScreen trusts installers that don't cause problems. Windows machines send telemetry back to Redmond about installed programs and how much trouble they cause. If you get enough thumbs-up then SmartScreen stops blocking your installer automatically. This takes time and lots of installs to get sufficient thumbs. There is no way to find out how far along you got.
It is almost certainly no coincidence that SmartScreen got integrated into the OS at the exact same time that the Windows Store opened for business. Which, for small software resellers, is the backdoor to get users to trust them again. Somebody has installed and reviewed the software and gave it the thumbs-up with a certificate. Double-up are the limitations imposed by the sandbox in which it must run, very hard to damage a machine. Microsoft has very little incentive to make SmartScreen less draconian.
Bad news, no doubt, but that's the way it rolls today.
You can improve your reputation by signing with a trusted code-signing certificate. Established businesses resolve this chicken-and-egg problem by having previously used code-signing (i.e., back when Microsoft was less draconian). The expensive way to resolve this chicken-and-egg problem is to buy an EV code-signing certificate. Assuming this blog post applies to Windows 10:
Programs signed by an EV code signing certificate can immediately establish reputation with SmartScreen reputation services even if no prior reputation exists for that file or publisher. Other factors are considered when generating reputation and determining product experiences and EV-signed programs will be closely monitored over time. We think the improvements in the vetting and security of these certificates are a great development for both users and developers.
...
The presence of an EV code signing certificate is a strong indicator that the file was signed by an entity that has passed a rigorous validation process and was signed with hardware which allows our systems to establish reputation for that entity more quickly than unsigned or non-EV code signed programs.
This is not guaranteed to work.