How can I determine the current size of the ARC in ZFS, and how does the ARC relate to free or cache memory?
The ZFS code reports various statistics through procfs. To determine the size of the ARC, look at /proc/spl/kstat/zfs/arcstats
(assuming procfs mounted on /proc, as customary), specifically the values for c
, c_max
and size
. (See also this post on the Oracle Community forum. Alternative Internet Archive copy in case the Oracle site becomes unavailable.)
c
is the target size of the ARC in bytesc_max
is the maximum size of the ARC in bytessize
is the current size of the ARC in bytes
The maximum size of the ARC can be adjusted either by passing a zfs_arc_max=N
parameter to the zfs
module (through modprobe), where N
is the maximum ARC size in bytes, or on the fly by writing the new maximum size in bytes to /sys/module/zfs/parameters/zfs_arc_max
.
Because of how ZFS on Linux is implemented, the ARC memory behaves like cache memory (for example, it is evicted if the system comes under memory pressure), but is aggregated by the kernel as ordinary memory allocations. This can lead to confusion as the system appears to have far less free memory than would be expected given the current system workload, but is normal.
To get the ARC size in megabytes, you can use something like awk '/^size/ { print $1 " " $3 / 1048576 }' < /proc/spl/kstat/zfs/arcstats
. (1,048,576 is the number of bytes to the megabyte.)
For example, my system (which uses ZFS almost exclusively) might report
$ free -m
total used free shared buffers cached
Mem: 32194 17948 14245 0 158 982
-/+ buffers/cache: 16808 15386
Swap: 49152 0 49152
$ awk '/^size/ { print $1 " " $3 / 1048576 }' < /proc/spl/kstat/zfs/arcstats
size 8138.73
$
which means that the actual memory usage by currently resident processes is approximately 8,669 MB (16,808 MB reduced by 8,139 MB).
To build on Michael Kjörling's answer, you can also use arc_summary.py.
Here you can see how the ARC is using half of my desktop's memory:
root@host:~# free -g total used free shared buffers cached Mem: 62 56 6 1 1 5 -/+ buffers/cache: 49 13 Swap: 7 0 7 root@host:~# arc_summary.py ------------------------------------------------------------------------ ZFS Subsystem Report Fri Feb 24 19:44:20 2017 ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 1.33m Mutex Misses: 99 Evict Skips: 99 ARC Size: 98.13% 30.80 GiB Target Size: (Adaptive) 100.00% 31.39 GiB Min Size (Hard Limit): 0.10% 32.00 MiB Max Size (High Water): 1004:1 31.39 GiB ARC Size Breakdown: Recently Used Cache Size: 84.25% 26.45 GiB Frequently Used Cache Size: 15.75% 4.95 GiB ARC Hash Breakdown: Elements Max: 1.11m Elements Current: 53.48% 592.56k Collisions: 763.42k Chain Max: 4 Chains: 19.62k ARC Total accesses: 36.34m Cache Hit Ratio: 87.02% 31.62m Cache Miss Ratio: 12.98% 4.72m Actual Hit Ratio: 84.78% 30.81m Data Demand Efficiency: 93.49% 24.22m Data Prefetch Efficiency: 2.57% 819.12k CACHE HITS BY CACHE LIST: Anonymously Used: 2.27% 716.60k Most Recently Used: 17.26% 5.46m Most Frequently Used: 80.17% 25.35m Most Recently Used Ghost: 0.19% 60.25k Most Frequently Used Ghost: 0.11% 35.37k CACHE HITS BY DATA TYPE: Demand Data: 71.60% 22.64m Prefetch Data: 0.07% 21.04k Demand Metadata: 25.82% 8.16m Prefetch Metadata: 2.51% 794.39k CACHE MISSES BY DATA TYPE: Demand Data: 33.44% 1.58m Prefetch Data: 16.92% 798.09k Demand Metadata: 48.90% 2.31m Prefetch Metadata: 0.75% 35.27k DMU Prefetch Efficiency: 173.06m Hit Ratio: 86.14% 149.07m Miss Ratio: 13.86% 23.99m ZFS Tunable: metaslab_debug_load 0 zfs_arc_min_prefetch_lifespan 0 zfetch_max_streams 8 zfs_nopwrite_enabled 1 zfetch_min_sec_reap 2 zfs_dbgmsg_enable 0 zfs_dirty_data_max_max_percent 25 zfs_arc_p_aggressive_disable 1 spa_load_verify_data 1 zfs_zevent_cols 80 zfs_dirty_data_max_percent 10 zfs_sync_pass_dont_compress 5 l2arc_write_max 8388608 zfs_vdev_scrub_max_active 2 zfs_vdev_sync_write_min_active 10 zvol_prefetch_bytes 131072 metaslab_aliquot 524288 zfs_no_scrub_prefetch 0 zfs_arc_shrink_shift 0 zfetch_block_cap 256 zfs_txg_history 0 zfs_delay_scale 500000 zfs_vdev_async_write_active_min_dirty_percent 30 metaslab_debug_unload 0 zfs_read_history 0 zvol_max_discard_blocks 16384 zfs_recover 0 l2arc_headroom 2 zfs_deadman_synctime_ms 1000000 zfs_scan_idle 50 zfs_free_min_time_ms 1000 zfs_dirty_data_max 6741298790 zfs_vdev_async_read_min_active 1 zfs_mg_noalloc_threshold 0 zfs_dedup_prefetch 0 zfs_vdev_max_active 1000 l2arc_write_boost 8388608 zfs_resilver_min_time_ms 3000 zfs_vdev_async_write_max_active 10 zil_slog_limit 1048576 zfs_prefetch_disable 0 zfs_resilver_delay 2 metaslab_lba_weighting_enabled 1 zfs_mg_fragmentation_threshold 85 l2arc_feed_again 1 zfs_zevent_console 0 zfs_immediate_write_sz 32768 zfs_dbgmsg_maxsize 4194304 zfs_free_leak_on_eio 0 zfs_deadman_enabled 1 metaslab_bias_enabled 1 zfs_arc_p_dampener_disable 1 zfs_object_mutex_size 64 zfs_metaslab_fragmentation_threshold 70 zfs_no_scrub_io 0 metaslabs_per_vdev 200 zfs_dbuf_state_index 0 zfs_vdev_sync_read_min_active 10 metaslab_fragmentation_factor_enabled 1 zvol_inhibit_dev 0 zfs_vdev_async_write_active_max_dirty_percent 60 zfs_vdev_cache_size 0 zfs_vdev_mirror_switch_us 10000 zfs_dirty_data_sync 67108864 spa_config_path /etc/zfs/zpool.cache zfs_dirty_data_max_max 16853246976 zfs_arc_lotsfree_percent 10 zfs_zevent_len_max 128 zfs_scan_min_time_ms 1000 zfs_arc_sys_free 0 zfs_arc_meta_strategy 1 zfs_vdev_cache_bshift 16 zfs_arc_meta_adjust_restarts 4096 zfs_max_recordsize 1048576 zfs_vdev_scrub_min_active 1 zfs_vdev_read_gap_limit 32768 zfs_arc_meta_limit 0 zfs_vdev_sync_write_max_active 10 l2arc_norw 0 zfs_arc_meta_prune 10000 metaslab_preload_enabled 1 l2arc_nocompress 0 zvol_major 230 zfs_vdev_aggregation_limit 131072 zfs_flags 0 spa_asize_inflation 24 zfs_admin_snapshot 0 l2arc_feed_secs 1 zio_taskq_batch_pct 75 zfs_sync_pass_deferred_free 2 zfs_disable_dup_eviction 0 zfs_arc_grow_retry 0 zfs_read_history_hits 0 zfs_vdev_async_write_min_active 1 zfs_vdev_async_read_max_active 3 zfs_scrub_delay 4 zfs_delay_min_dirty_percent 60 zfs_free_max_blocks 100000 zfs_vdev_cache_max 16384 zio_delay_max 30000 zfs_top_maxinflight 32 ignore_hole_birth 1 spa_slop_shift 5 zfs_vdev_write_gap_limit 4096 spa_load_verify_metadata 1 spa_load_verify_maxinflight 10000 l2arc_noprefetch 1 zfs_vdev_scheduler noop zfs_expire_snapshot 300 zfs_sync_pass_rewrite 2 zil_replay_disable 0 zfs_nocacheflush 0 zfs_arc_max 0 zfs_arc_min 0 zfs_read_chunk_size 1048576 zfs_txg_timeout 5 zfs_pd_bytes_max 52428800 l2arc_headroom_boost 200 zfs_send_corrupt_data 0 l2arc_feed_min_ms 200 zfs_arc_meta_min 0 zfs_arc_average_blocksize 8192 zfetch_array_rd_sz 1048576 zfs_autoimport_disable 1 zfs_arc_p_min_shift 0 zio_requeue_io_start_cut_in_line 1 zfs_vdev_sync_read_max_active 10 zfs_mdcomp_disable 0 zfs_arc_num_sublists_per_state 8