Is ZFS on Ubuntu 20.04 using a ton of memory?

ZFS will cache data and metadata so given a lot of free memory this will be used by ZFS. When memory pressure starts to occur (for example, loading programs that require lots of pages) the cached data will be evicted. If you have lots of free memory it will be used as cache until it is required.

You can use the arc_summary tool to see the resources used by the ZFS ARC (adaptive replacement cache)