How can I safely create and access temp files from shell scripts?

Use the mktemp utility to create a temporary file with an unpredictable name. It isn't standardized by POSIX, but it's available on *BSD as well as Linux.

> /tmp/predictable.$RANDOM is not a good choice because it's mostly predictable¹, which opens your script to an attack where the attacker can trick your script into overwriting a file you have write access to, or giving them access to the temporary file. This is an insecure temporary file vulnerability. mktemp doesn't have this vulnerability because creates the file safely (it won't overwrite an existing file, even if symbolic links are involved) and uses a sufficiently unpredictable name to avoid a denial of service.

If creating one temporary file and working with it is not good enough, create a temporary directory with mktemp -d, and work in there.

mktemp also takes care to use $TMPDIR if the variable is set, falling back to /tmp if it's unset.

More and more distributions set up TMPDIR to be a private directory, e.g. /run/1234/tmp where 1234 is your UID. This eliminates the risk of temporary file vulnerabilities, at the cost of no longer being able to share temporary files between users (which is occasionally useful, but not very often; /tmp is still available, just not TMPDIR).

If you need a reproducible file name, then create a file with a well-defined name (with no random component) under the user's home directory. The modern convention is the XDG user directory specification. If the file could be removed without causing data loss, use the XDG_CACHE_HOME environment variable, defaulting to ~/.cache. You should probably create a subdirectory named after your application and work in there.

CACHE_DIR="${XDG_CACHE_HOME:-"$HOME/.cache"}"/Wildcard-scripts
[ -d "$CACHE_DIR" ] || mkdir -p -- "$CACHE_DIR"
CACHE_FILE="$CACHE_DIR/tmpfileformyscript"

¹ _{Not only does $RANDOM only takes 32767 possible values, but it's easy to predict without even trying many values. Bash's random number generator is a LCG seeded by the PID and time of first use. Zsh's is the platform's rand seeded by startup time. ATT Ksh's is the platform's rand seeded by PID. Mksh's is an LCG with a more complex, but still not security-quality seed. All of them can be predicted by another process with a fairly large chance of success.}

mktemp was designed for this. From the man page:

TMPFILE=`mktemp /tmp/example.XXXXXXXXXX` || exit 1
echo "program output" >> $TMPFILE

mktemp will create the file or exit with a non-zero exit status. The logical or (||) ensures that the script will exit if mktemp is unable to create the file. After this command, you can be sure that the file is available. There is no need to check it again. The only thing you may need to add is the cleanup of the file at the end of your script.

And possibly also when the script is terminated by a signal. Whether that is necessary or not is something you have to decide.

Both can be done using the trap command.