For which Content-Types should I set security related HTTP response headers?

Strict-Transport-Security

In the deployment recommendations of "HSTS Preload List" it is stated:

Add the Strict-Transport-Security header to all HTTPS responses

In apache this would look like (note I did not include the preload directive, developers should read the HSTS Preload List's deployment recommendations first before adding that):

<IfModule mod_headers.c>
    Header always set Strict-Transport-Security "max-age=16070400; includeSubDomains" "expr=%{HTTPS} == 'on'"
</IfModule>

X-XSS-Protection

If you are using CSP (without allowing 'unsafe-inline') then you probably don't need to worry about X-XSS-Protection anymore:

• Chrome has an "Intent to Deprecate and Remove the XSS Auditor".
• Firefox have not, and will not implement X-XSS-Protection.
• Edge retired their XSS filter

Content-Security-Policy (and security-related headers in general)

As a general approach, you'd at least want to add security headers to all (common) MIME-Types that are able to execute scripts:

• HTML
• XML
• JS (Javascript is only executed in a "browsing context", however this applicable due to JS's ability to create Workers, see https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy#CSP_in_workers)
• PDF - Yes really, PDF files can also execute javascript.

Also, IMO consider setting a strict Referrer-Policy for ALL responses. I hope this helps :)

Theoretically, only 'active' documents should need it much like the X-XSS-Protection header (related answer here from Info Security). As long as the policy is set on the main document (even through a Meta tag), external resources should be blocked based on that policy, not the policy on the external resource (easy to see when loading CDN files which almost certainly do not have your CSP, or any CSP, set).

So I would say your estimate is correct; text/HTML and XML absolutely should have it, anything that can execute Javascript. It shouldn't matter for static resources. They'll be blocked or allowed based on the main Document's CSP.

I will admit that personally I simply send them on all resources served directly from my server as I'd rather be paranoid than screw something up and the few dozen bytes per request don't appear to be a big impact especially on a site that doesn't serve a great deal of requests. And if your site does serve an extreme amount of requests...usually best to cut down on requests before trying to shrink your headers.

As with anything like this I'd be sure to test your specific implementation and try loading some resources the CSP should block. You never know when a browser implementation may be flawed (or more frequently, a typo or over/under eager application of your own rules).