Does NTFS performance degrade significantly in volumes larger than five or six TB?

I've made 30TB volumes before. They were holding large files, so that greatly assists in avoiding performance degradation. No problems there versus smaller volumes.

Where problems might begin to occur is if that large filesystem builds up enough files and directories to get to the insanely big ranges. I'm talking 20 million files and directories or more levels. At that point the MFT is likely to be pretty fragmented and on RAM constrained servers that might start to introduce some directory-display performance issues. Actual access to files should not be affected, just fileOpen() and directory scanning operations.

Is it a real possibility? Yes, but the effect also holds true for similarly large filesystems on other platforms and for the same reason. Also, the operations most impacted by the performance drop may not even be the ones you're worried about. Since this sounds like a new setup, it doesn't sound like you'll be using this 10TB on a server with only 1GB of RAM so you shouldn't have to worry.

Where the worry about size came from is likely WinNT and Win2K era, when MBR limitations made large volumes tricky to create. Such needed some trickery to address all of that, and in the era the trickery had a cost. This magnified the lots of little files performance penalty. GPT doesn't have this problem. GPT volumes were first in Windows 2003, but their newness meant conservative sysadmins didn't use them much at first.

Old attitudes die hard, sometimes. Especially if there is trauma reinforcing them.


We have some much bigger than that (using W2K8) and we've not seen any noticable performance drop offs.


You're more likely to hit hardware performance issues long before software related ones.

Consider this,

  • Disk: 10x 1tb sata green disks might outperform 5x 2tb disks. One reasons is that you will have more disk queues (DiskQueueLength) to use with the samller disks. However, keep your OS on a seperate faster volume. The RAID card and it's cache also makes for some serious performance considerations.
  • RAM: also affects the number and size of files that can be opened (PagedPoolSize).
  • NIC: will affect how much data can be written to/fom the server. Consider having your backups run off a seperate NIC. And use adapter teaming to combine the two primary NICs to act as one.

Currently I have two servers. The volume is 16TB and 6TB of data 5million files/directories. At the end of the day, 200GB of data has been changed or added. There are 100 users connecting to the server. And of course these servers replicate the data real time using DoubleTake. No one complains about the server performance.

That said, it also depends on what you want to implement. MS DFS-R on Win2008R2 for example does have limits such as 8million files and 10TB per server/volume as an offical answer. But I've seen reports of 20+TB.

You can also have a read through this if you want... http://www.microsoft.com/whdc/system/sysperf/perf_tun_srv.mspx

Tags:

Windows

Ntfs