Pacemaker (Admin Guide)
Pacemaker is a mature software for setting up a high availability cluster on Linux. For CentOS 7, the packages are in the default repository but for CentOS 8 were moved to the High Availablity section.
Pacemaker consists of the services
corosync (synchronisation service),
pcsd (PCS GUI and remote configuration interface) and
pacemaker (Pacemaker High Availability Cluster Manager). Setup of pacemaker is described on many websites, very detailed is RedHat Documentation.
- Resource: A Resource can be a service, a mount point, basically anything for which a script exists that describs how this resource works. These scripts are usually save under
/usr/lib/ocf/resource.d/and referenced according to their path (e.g.
ocf::heartbeat:Filesystemfor the file
- Clone: The default is for a resource to run only once in a HA cluster. If a resource should be run on all nodes of the HA cluster, for example
ntpd, then a clone is used.
- Group: A Collection of Resources that are ordered (in the order they are added to a group). If a resource in a group fails or is disabled, all dependent resources are stopped.
- Meta Properties: Properties of a Resource,etc that are only evaluated by pacemaker, not by the resource.
- Constraints: Pacemaker starts resources according to the constraints set by the administrator. A constraint can be based upon hosts, time, other resources (colocation), etc.
- Stickiness: Resources have meta property called “stickiness” which is just a number. This number sets a limit so that a resource is (not) moved to a node. Its purpose is to not have resource hop around nodes just because some minor detail changes in the cluster.
- Fail Counts: If a resource fails, a counter is increased and a resource is usually forbidden on a node where it failed in the past.
Configuration of pacemaker is done using
pcs or the
crm (cluster resource manager) tools.
pcs is just a
python frontend for the
crm commands, but it is definitely easier to use. For example,
pcs status is basically only
crm_info -1otf, but a much clearer command. The
crm tools are not known for being admin friendly, so
pcs is definitely recommended.
pcs has a very nice help and man page.
Resources and their dependencies should be put into groups together. Do not group resources according to type.
Resource Group: group1 group1-fs (ocf::heartbeat:Filesystem): Started group1-ip (ocf::heartbeat:IPaddr2): Started group1-server (systemd:group1-service): Started Resource Group: group2 group2-fs (ocf::heartbeat:Filesystem): Started group2-ip (ocf::heartbeat:IPaddr2): Started group2-server (systemd:group1-service): Started
Resource Group: group-fs group1-fs (ocf::heartbeat:Filesystem): Started group2-fs (ocf::heartbeat:Filesystem): Started Resource Group: group1 group1-ip (ocf::heartbeat:IPaddr2): Started group1-server (systemd:group1-service): Started Resource Group: group2 group2-ip (ocf::heartbeat:IPaddr2): Started group2-server (systemd:group1-service): Started Colocation Constraints: group1 with group-fs (score:INFINITY) group2 with group-fs (score:INFINITY) Ordering Constraints: start group-fs then start group1 (kind:Mandatory) start group-fs then start group2 (kind:Mandatory)
- Try to use constraints as little as possible. There are valid cases for them (for example, when several resources depend on one but there is no order among the resources), but a group is usually preferable to colocation and ordering constraints.
systemdServices are automatically available as resources. In case this is not sufficient, writing your own scripts is very easy. The only important detail to look out for is this:
# Monitor _MUST!_ differentiate correctly between running # (SUCCESS), failed (ERROR) or _cleanly_ stopped (NOT RUNNING). # That is THREE states, not just yes/no.
For an example, see generic-script on github