Which filesystem for large LVM of disks (8 TB)?
Solution 1:
It's not file system problem, it's disks' physical limitations. Here's some data:
SATA drives are commonly specified with an unrecoverable read error rate (URE) of 10^14. That means that 1 byte per 12TB will be unrecoverably lost even if disks work fine.
This means that with no RAID you will lose data even if no drive fails - RAID is your only option.
If you choose RAID5 (total capacity n-1, where n = number of disks) it's still not enough. With 10TB RAID5 consisting of 6 x 2TB HDD you will have a 20% chance of one drive failure per year and with a single disk failing, due to URE you'll have 50% chance of successfully rebuilding RAID5 and recovering 100% of your data.
Basically with the high capacity of disks and relatively high URE you need RAID6 to be secure even again single disk failure.
Read this: http://www.zdnet.com/blog/storage/why-raid-5-stops-working-in-2009/162
Solution 2:
Do yourself a favor and use a RAID for your disks, could even be software RAID with mdadm. Also think about why you "often get errors on your disks" - this is not normal except when you use cheap desktop class SATA drives instead of RAID grade disks.
After that, the filesystem is not that important anymore - ext4, xfs are both fine choices.
Solution 3:
I've had good luck with ZFS, you could check to see if it's available on whatever distro you use. Fair warning, it'll probably mean rebuilding your whole system, but it gives really good performance and fault-tolerance.
Solution 4:
I add new disks of greater sizes progressively
Since you are interesting in using LVM, and you want to handle multiple drives, the simple answer would be to just use the mirror feature that is part of LVM. Simply add all the physical volumes into your LVM. When you are creating a logical volume pass the --mirrors option. This duplicates your data.
Another option might be to just setup several RAID1 pairs. Then add all the RAID1 volumes as PVs to your VG. Then whenever you want to expand your storage, just buy a pair of disks.
Solution 5:
You should really be using a RAID 5, 6, 10, 50, or 60. Here's some resources to get you started:
background info about RAIDs
- http://en.wikipedia.org/wiki/RAID
- http://www.techrepublic.com/blog/datacenter/choose-a-raid-level-that-works-for-you/3237
howto's & setup
- http://www.dedoimedo.com/computers/linux-raid.html
- http://www.linuxplanet.com/linuxplanet/tutorials/6514/1
- http://dtbaker.com.au/random-bits/ubuntu---howto-easily-setup-raid-5-with-lvm.html
Check out my delicious links for additional RAID links: http://delicious.com/slmingol/raid