Why does this ldapadd command quit with an "Invalid syntax" error?

Your problem is undoubtedly that you need to load the nis schema into your LDAP server. How to do this depends on whether your are using the legacy slapd.conf configuration file or the newer dynamic configuration hosted in cn=config and backed by a slapd.d directory.

Using slapd.conf

You will need to include the schema definition in your slapd.conf by adding a line along the lines of:

include /usr/local/etc/openldap/schema/nis.schema

This assumes that the nis.schema file is located at that path; if not, modify the path appropriately.

You will need to restart slapd to activate the new schema.

Using slapd.d

(I'm including this for completeness, although it's not directly relevant to your current configfuration).

To load a schema into slapd if you're using the dynamic cn=config configuration, you would use ldapadd. Depending on how your ACLs are configured, the command might look like this:

ldapadd -Y EXTERNAL -H ldapi:// -f /usr/local/etc/openldap/schema/nis.ldif

This assumes that your running slapd has an ACL permitting "peer credentials" authentication to root. If that doesn't work, you would need to provide an appropriate bind DN and password using -D and -W.

There is no restart required in this case.