Recommendations for replacing a GFS cluster?
Ceph and GlusterFS is where the clustered FS technology is currently going. Since I'm not familiar with GlusterFS I'll speak on the features of Ceph.
Ceph scales horizontally; the more low-end nodes you add to it the better the performance. Unlike GlusterFS this is a primary benefit to Ceph as it shards the data.
However, Ceph is in active development (it's production ready except for Ceph FS) and requires a modern kernel (as I write this not even CentOS 6.5 default kernel can take advantage of RBD/CephFS features). To get around this I installed ELRepo kernel-lt
.
To break it down for you:
- Cephs RBD is a clustered SAN replacement; you can create "virtual" devices that are in the cluster and can be mounted on servers. Note: Only one server should have an RBD image mounted at one time (you don't want multiple OS's mounting a SATA drive do you?). You would then format the RBD disk,
mount
it like normal, and then have NFS/CIFS make it available. If the server providing NFS/CIFS goes down no data is lost. - Ceph FS is a clustered NAS replacement (although not production ready); it provides file locking capabilities that are needed for a clustered FS that is shared between servers (such as a web server for example).
RBD runs in kernel space; so there is no fuse performance hits. Ceph FS also runs in kernel space but can run with FUSE.
Ceph is also pretty easy to deploy:
pip install ceph-deploy
on an admin node (your desktop/workstation).- Add Inktank RPM repos and
ceph-deploy install node1 node2 ... nodeN
to install ceph on all of the nodes.