How do I use command arguments with Cmnd_Alias in sudoers?

You haven't used any wildcards, but have provided two arguments. Therefore sudo looks for commands exactly as written (excepting path-lookup) (from man 5 sudoers):

 If a Cmnd has associated command line arguments, then the arguments in
 the Cmnd must match exactly those given by the user on the command line
 (or match the wildcards if there are any).

Try something like:

Cmnd_Alias AWSS3_CMD = /usr/local/bin/aws s3 cp *, /usr/local/aws/bin/aws s3 cp *

Note that:

 Wildcards in command line arguments should be used with care.  Because
 command line arguments are matched as a single, concatenated string, a
 wildcard such as ‘?’ or ‘*’ can match multiple words. 

So, only one wildcard is needed per command.

The same as @muru, but for those who like full working example:

# Cmnd alias specification
Cmnd_Alias ECHO_CMD=/bin/echo A *,/bin/echo B *

# Make USER run specific commands on given HOSTNAME
# USER_NAME 

#userx
userx ALL=(root) NOPASSWD: /sbin/cmd1,/sbin/cmd2,ECHO_CMD

While /sbin/cmd1,/sbin/cmd2 can be any other commands.

The purpose of ECHO_CMD is to present that sudo echo X A will ask for passphrase, while sudo echo A X not, and to allow you to gain confidence through such simple experiment.

(It was assumed that echo sits in /bin/echo, to check where it is on your system try whereis echo)