Easiest , dirtiest and quickest way I found to delete a freshly configured GPFS cluster (DON’T DO THIS ON A PRODUCTION SERVER !!! IT IS FOR TESTING/DEV PURPOSES ONLY !!! ) :

GPFS V4 : change to CCR mode

I have to change the type of cluster to CCR mode (only available with GPFS V4), in order to be able to remove all the nodes in the cluster , because the cluster is not server-based anymore and does not need a configure node as a server to hold the cluster configuration.

–> from the mmcrcluster page :

–ccr-enable : Enables the configuration server repository (CCR), which stores redundant copies of configuration data files on all quorum nodes. All GPFS administration commands, as well as file system mounts and daemon startups, work normally as long as a majority of quorum nodes are accessible.

This is the default.

–ccr-disable : Indicates that the traditional primary/backup server-based configuration repository (non-CCR, earlier than GPFS 4.1) is to be used.

When using this option you must also specify a primary configuration server (-p option). It is suggested that you also specify a secondary GPFS cluster configuration server (-s option) to prevent the loss of configuration data in the event your primary GPFS cluster configuration server goes down. When the GPFS daemon starts up, at least one of the two GPFS cluster configuration servers must be accessible.

If your primary GPFS cluster configuration server fails and you have not designated a secondary server, the GPFS cluster configuration files are inaccessible, and any GPFS administration commands that are issued fail. File system mounts or daemon startups also fail if no GPFS cluster configuration server is available.

You are strongly advised to designate the cluster configuration servers as quorum nodes.

In the old fashioned server-based mode (GPFS <V4), you cannot do that (or I didn’t find how, in this case , please tell me ūüôā )

So you have to manipulate conf files in /var/mmfs directory, but I find it to be very annoying and ugly.


Now the ugly stuff

To sum up, you have to :

  • unmount the filesystems on both nodes
  • delete the filesystems one by one
  • delete the freed NSD ( if one is not free, of course, you cannot go on with the procedure, you have to delete it prior to the mmshutdown)
  • stop the cluster
  • change config to enable CCR mode
  • delete node 2
  • delete node 1
  • Optional : delete /var/mmfs directory on both nodes, IF YOU KNOW WHAT YOU’RE DOING (or back it up before).
  • check with mmlscluster or any GPFS command

As you can see, I put a lot of sleep commands , because GPFS sends asynchronous commands, so the jobs seem to be finished , but in fact they’re not … yet.

And it can be pretty messy if you execute all commands one after another, without waiting a bit between each one of them. I’ve tried many ways, this is not the cutest, but the safest IMHO.

mmumount all -a
sleep 2
mmlsconfig | awk 'gsub("/dev/","") {print "mmdelfs "$1";sleep 2"}'|sh
mmchconfig tiebreakerDisks=""
mmlsnsd -m | awk '/hdisk/{print "mmdelnsd "$3";sleep 1"}'|sh
mmshutdown -a
sleep 5 
mmchcluster --ccr-enable
sleep 5 
mmdelnode NODE2 
sleep 2 
mmdelnode NODE1 



Share Button
GPFS : delete a configured cluster you just created (Testing only)

Laisser un commentaire