How can I list the contents of a makeself archive without running it?

The generated archive has a --list option, which you can use to list its contents.

For reference, I'm talking about this version in Debian:

ii  makeself       2.2.0-1      all          utility to generate self-extractables

which generates this chunk in the script:

MS_Help()
{   
    cat << EOH >&2 
Makeself version 2.2.0 
 1) Getting help or info about $0 : 
  $0 --help   Print this message 
  $0 --info   Print embedded info : title, default target directory, embedded script ... 
  $0 --lsm    Print embedded lsm entry (or no LSM) 
  $0 --list   Print the list of files in the archive 
  $0 --check  Checks integrity of the archive 

 2) Running $0 : 
  $0 [options] [--] [additional arguments to embedded script] 
  with following options (in that order) 
  --confirm             Ask before running embedded script 
  --quiet               Do not print anything except error messages 
  --noexec              Do not run embedded script 
  --keep                Do not erase target directory after running 
                        the embedded script 
  --noprogress          Do not show the progress during the decompression 
  --nox11               Do not spawn an xterm 
  --nochown             Do not give the extracted files to the current user 
  --target dir          Extract directly to a target directory 
                        directory path can be either absolute or relative 
  --tar arg1 [arg2 ...] Access the contents of the archive through the tar command 
  --                    Following arguments will be passed to the embedded script 
EOH
}

Its manual page needs some work, but the script is easy enough to read - see git repository

Further reading: makeself - Make self-extractable archives on Unix