mkfs Operation Takes Very Long on Linux Software Raid 5

Solution 1:

I suspect you're running into the typical RAID5 small write issue. For writes under the size of a stripe size, it has to do a read-modify-write for both the data and the parity. If the write is the same size as the stripe, it can simply overwrite the parity, since it knows what the value is, and doesn't have to recalculate it.

Solution 2:

I agree, that it may be related to stripe alignment. From my experience creation of unaligned XFS on 3*2TB RAID-0 takes ~5 minutes but if it is aligned to stripe size it is ~10-15 seconds. Here is a command for aligning XFS to 256KB stripe size:

mkfs.xfs -l internal,lazy-count=1,sunit=512 -d agsize=64g,sunit=512,swidth=1536 -b size=4096 /dev/vg10/lv00

BTW, stripe width in my case is 3 units, which will be the same for you with 4 drives but in raid-5.

Obviously, this also improves FS performance, so you better keep it aligned.

---

Solution 3:

Your mkfs and subsequent filesystem performance might improve if you specify the stride and stripe width when creating the filesystem. If you are using the default 4k blocks, your stride is 16 (RAID stripe of 64k divided by filesystem block of 4k) and your stripe width is 48 (filesystem stride of 16 multiplied by the 3 data disks in your array).

mkfs.ext3 -E stride=16 stripe-width=48 /dev/your_raid_device