Which folder should I write to for my intermediate files, as a dev?

I find the Filesystem Hierarchy Standard document invaluable when looking for this stuff.

There are a few options,

• /tmp - 'non-permanent' temporary files
• /var/tmp - 'permanent' temporary files
• /var/cache - 'application' transient data files

It really depends on the kind of data you're storing.

If you need to write some temporary files that only last as long as your script or application is running, use the directory indicated by the TMPDIR environment variable, or if that variable is not defined, /tmp.

/tmp is cleared at boot time on some systems (sometimes it's even in RAM, e.g. by default on Solaris, and on some Linux installations), so it cannot be used for files that must survive a power failure or a reboot. /var/tmp can be used for files that must survive a reboot, but that may be cleaned by the system administrator from time to time. If your application needs to save files on a permanent basis, write them somewhere in the user's home directory (in ~/.programmingnoobsapp or ~/.cache/programmingnoobsapp) or under /var/lib/programmingnoobsapp or /var/cache/programmingnoobsapp.

Note that /tmp is shared between all users, so you need to take precautions when creating a file there. You need to pick a file name that doesn't exist yet, and you need to be careful not to allow a race condition where another process creates the file ahead of you with different permissions, which could be a security hole (if the other process is running as a different user, it could then access and modify your process's data). Use the mktemp command to create a file in /tmp or /var/tmp. By default, mktemp creates a file in $TMPDIR, or /tmp if TMPDIR is unset, which is usually the right place. If you need to use multiple temporary files, or even if you need a single one, I recommend creating a directory for all your temporary files with mktemp -d and removing it at the end of your script.

#!/bin/sh
tmp_root=
trap 'rm -rf "$tmp_root"' EXIT INT TERM HUP
tmp_root=$(mktemp -d)
tmpfile1=$tmp_root/file1
tmpfile2=$tmp_root/file2
…

Just to complement the answers that have already been posted thus far.

There is also /dev/shm on some Linux distributions which can be use for temporary storage. This storage should only be used when performance of the file I/O is a critical factor given /dev/shm makes use of the tmpfs filesystem. Also it should be used for reasonably sized files & data. The tmpfs filesystem makes use of a system RAM as storage so it isn't persistent from boot to boot.

There's a good overview of all the choices mentioned here on StackOverflow as well in this Q&A titled: /tmp vs. /dev/shm for temp file storage on Linux?. It's covered well in this SuperUser Q&A titled: When should I use /dev/shm/ and when should I use /tmp?.

References

• Wikipedia - Shared Memory
• Wikipedia - Filesystem Hierarchy Standard
• Wikipedia - Tmpfs
• Tmpfs - kernel.org