What is Ceph?

Ceph is a software-defined storage solution designed to address the object, block, and file storage needs for data centers. Ceph is an open-source storage platform available for CentOS 8, CentOS7 (partially from Octopus release), Ubuntu, Debian Buster, Fedora, openSUSE and also in Container image (based on CC8). It is designed to be a self-healing and self-managed cluster and reduce administration and budget costs, while it deals with outages on its own. Inside the Ceph Cluster, all components are scalable and there is no single point of failure, which can also easily scale to exabyte level.

Our Ceph Instance

Our Ceph Cluster running the latest CentOS 7 and latest Ceph (as of the time of writing it runs v14.2.16 Nautilus release) and on this cluster we are mainly using only CephFS and testing different Erasure codings. We decided to do an upgrade to the latest Octopus release and will cover most details in this blog post.

Hardware:

  • 11 Nodes (For Monitors, Metadata, Managers, OSDs):
    • 32 cores, AMD Opteron(tm) Processor 6378
    • 94 GB RAM;
    • 2 OSDs in each (3.7TB)
    • Network: 1Gb/s (All connected to the same switch)
  • 1 Node (For testing):
    • 32 cores, AMD Opteron(tm) Processor 6378
    • 94 GB RAM;
    • 2 OSDs (3.7TB)
    • Network: 10Gb/s (Connected to the same switch)

 Octopus vs Nautilus

You can find most of the notable changes here [1] and the most problematic for our upgrade is the following statement:

Note that the dashboard, Prometheus, and restful manager modules will not work on the CentOS 7 build due to Python 3 module dependencies that are missing in CentOS 7.

Because of our environment and huge dependency on CentOS7 (all cluster is only CentOS 7) – we will look into more details what we are missing from these. We really depend a lot on customized monitoring based on zabbix, telegraf and custom scripts.

For this case, we also need to install Python 3.6 on our CentOS 7 cluster. It is already included in epel-testing repository, so you can do this with the following command:

yum –enablerepo=epel-testing update
yum install python36

https://docs.ceph.com/en/latest/releases/octopus/#major-changes-from-nautilus

Written by jbalcas

Leave a Comment