Is "NT AUTHORITY\SYSTEM" a user or a group?

First, access token contains much more than the security identifier (SID). One only has to "Run as administrator" a program to see in the Task Manager that its user is oneself and not Administrator, and this miracle is achieved just by the modification of the access token, not by replacing the SID.

Second, NT-AUTHORITY and SYSTEM are neither accounts nor groups, in spite of what say various other sources (even inside Microsoft). An SID usually has a name that is displayed whenever required. A user account will contribute its SID as principal SID to the access token, which will also determine the name displayed by various utilities. But the access token may contain additional SIDs, for example for all the groups to which belongs that user account. When checking permissions, Windows will look for any SID in the access token that has that permission.

Some well-known Windows SIDs will have names reported by Windows, although they do not really belong to any account.

A Security Identifier is defined by Wikipedia as :

a unique, immutable identifier of a user, user group, or other security principal.

The SID does not need to even define a user account or a group. It just defines a set of permissions. The above Wikipedia article adds:

Windows grants or denies access and privileges to resources based on access control lists (ACLs), which use SIDs to uniquely identify users and their group memberships. When a user logs into a computer, an access token is generated that contains user and group SIDs and user privilege level. When a user requests access to a resource, the access token is checked against the ACL to permit or deny particular action on a particular object.

The SID of NT-AUTHORITY\SYSTEM can be added to other accounts. For example, this is said about the LocalSystem Account:

The LocalSystem account is a predefined local account used by the service control manager. [...] Its token includes the NT AUTHORITY\SYSTEM and BUILTIN\Administrators SIDs; these accounts have access to most system objects.

One can already see in the above text the confusion that reigns even in Microsoft documentation as regarding system SIDs, which are not exactly accounts nor groups - which are just a set of permissions. This confusion further extends to other utilities and articles, so any returned information should be carefully examined.

The Microsoft article Well-known security identifiers in Windows operating systems details all system SIDs, some of whom I include below:

Conclusion: NT-AUTHORITY\SYSTEM is the name of a Security ID, which is neither a group nor an account. It is displayed in Task Manager as SYSTEM when it is the principal SID of a program. The most I would call it is "a pseudo account".

IMHO your observation is correct. NT-AUTHORITY\SYSTEM is a group, so you could refer to it as the system group. This group exists since Windows NT 4 at least and has already been a group there:

Special Groups

[...]

System - The operating system.

There is also an account called LocalSystem which

[...] includes the NT AUTHORITY\SYSTEM [...]

so you could call this the system user who is member of the SYSTEM group.

SysInternals PsGetSid supports the group theory for SYSTEM:

C:\>PsGetsid.exe S-1-5-18

PsGetSid v1.44 - Translates SIDs to names and vice versa
Copyright (C) 1999-2008 Mark Russinovich
Sysinternals - www.sysinternals.com

Account for YOURPCNAMEHERE\S-1-5-18:
Well Known Group: NT-AUTHORITY\SYSTEM

Regarding the start of a process as a group:

To manage the security, a process gets an access token. The access token contains SIDs only. I'm not sure if there's a check whether the user's SID is really a user or a group. In principle it wouldn't matter: the SID defines what can be accessed. Perhaps the CodeProject article can help with the implementation