Is it ok to put line-breaks in add_header in nginx configuration?

Unfortunately, nginx treats the white space between the quotes literally, so as long as you begin each new line with a space or tab character, the header will remain valid.

However, it is possible to create an invalid header. For example, this produces an invalid header:

add_header Content-Security-Policy "default-src 'self' *.google-analytics.com;
object-src 'none';
report-uri /csp-report;";

The support for splitting header lines is deprecated in RFC 7230:

From RFC 7230 section 3.2.4

Historically, HTTP header field values could be extended over
multiple lines by preceding each extra line with at least one space
or horizontal tab (obs-fold). This specification deprecates such
line folding except within the message/http media type

The safest solution would be to accept that some lines in your configuration file may be very much longer than you would prefer.

You can use variable nesting like this, which still in the end creates a one liner:

set $SCRIPT "script-src 'self'";
set $SCRIPT "${SCRIPT} https://www.a.com"; # comment each line if you like
set $SCRIPT "${SCRIPT} https://b.com";
set $STYLE "style-src 'self'";
set $STYLE "${STYLE} https://a.com";
set $IMG "img-src 'self' data:";
set $IMG "${IMG} https://a.com";
set $IMG "${IMG} https://www.b.com";
set $FONT "font-src 'self' data:";
set $FONT "${FONT} https://a.com";
set $DEFAULT "default-src 'self'";
set $CONNECT "connect-src 'self'";
set $CONNECT "${CONNECT} https://www.a.com";
set $CONNECT "${CONNECT} https://www.b.com";
set $FRAME "frame-src 'self'";
set $FRAME "${FRAME} https://a.com";
set $FRAME "${FRAME} https://b.com";
add_header Content-Security-Policy "${SCRIPT}; ${STYLE}; ${IMG}; ${FONT}; ${DEFAULT}; ${CONNECT}; ${FRAME}";