FreeNAS, Do I need 1GB per TB of usable storage, or 1GB of memory per TB of physical disc?

Solution 1:

With ZFS, it's 1 GB per TB of actual disk (since you lose some to parity). See this post about how ZFS works for details. For example, if you have 16 TB in physical disks, you need 16 GB of RAM.

Depending on usage requirements, you need 8 GB minimum for ZFS. UFS requires much less.

Solution 2:

This topic is controversal, and still ongoing debate. I think the best answer is: "It depends". If you are going to need deduplication, you probably want to have huge amount of rams. There the 1 GB per TB rule seems to underestimate the empirical observation, that suggest more something like 5 GB RAM per TB storage (ZFS Tuning Guide).

Some ZFS developers seem to have a different view: Reddit-Comment 1 Reddit-Comment 2.

Quote from Comment 2:

That is wrong. 1GB is fine for ZFS. You can use the same amount for data deduplication, although writes will slow down from 3 random seeks being done on DDT misses after a certain amount of unique records have been stored. Each unique record takes 320 bytes of space in the DDT, it counts as metadata and ZFS ARC on ZoL will only allow 1/8 of RAM to be used for metadata by default. You can do the math.

As for the amount of RAM, 1GB or more. Performance tends to be better with more RAM for more cache though. As I said elsewhere, the amount of storage does not determine how much RAM you need.

Addendum:

Apparently it's the deduplication feature that requires lots of RAM See Oracle documentation. ZFS only support block-level deduplication, and if dedup is turned on, you will need approximately 320 Bytes per block per core. This makes the resulting RAM consumption a bit tricky. Conclusively you can only tell, that if you are not gonna need deduplication, ZFS will not require the often-stated 1 GB per TB rule.

TL;DR

1 GB RAM is fine, if you are not going to use deduplication (default is off)