tmpfs does not overflow to swap

I get the impression you have a few misconceptions regarding tmpfs. You might find it useful to read the kernel documentation on the topic; I’ll attempt to clarify things for you here.

Your question’s title “tmpfs does not overflow to swap” doesn’t seem to reflect the actual contents of your question, but in any case tmpfs does use swap, although arguably it doesn’t overflow to swap. tmpfs is fundamentally a (virtual) memory-based file system; its contents live in memory only, but since they’re swappable the kernel can store them in swap instead of physical memory if necessary. Nevertheless tmpfs file systems can’t be larger than the total amount of virtual memory available, i.e. physical RAM and swap, as indicated e.g. by free -h.

By default tmpfs file systems have a maximum size equal to half the amount of physical memory available. You can increase this using the size parameter, but again it can’t ever be more than the available physical memory and swap (although that limit isn’t enforced at mount time). Once the file system reaches its maximum size (or rather, contains files occupying that much space), it reports that it’s run out of space, as you found out. tmpfs itself doesn’t support overflowing anywhere when it runs out of space.

If you need temporary storage space for large files, you should use /var/tmp rather than /tmp. You really don’t want a very large tmpfs file system, that’s a recipe for disaster when it fills up (the kernel’s usual ways of recovering memory don’t work in a tmpfs).

(If you have lots of RAM of course, a large tmpfs can work. I run a few systems with build tmpfs file systems sized at 75% of RAM, out of 32GiB, 64GiB or even more.)