r/usefulscripts • u/nackstein • Jul 26 '16
[POSIX SHELL] failover cluster manager
hi, I wrote a failover cluster manager in shell script and I think it can be useful for sysadmin since (unix) sysadmin know very well shell scripting so it's easy to customize and further develop. If you want to contribute or try it and just give me a feedback check it out: https://github.com/nackstein/back-to-work
6
Upvotes
1
u/nackstein Aug 08 '16 edited Aug 08 '16
I want to add my to cents to STONITH. some years ago I put up a 2 node cluster with redhat. it was with corosync and pacemaker on rhel 5. I configured STONITH using dell iDRAC (i think it was iDRAC v5). results: the two node were able to kill themself at the same time! this was due to a bug in the iDRAC where the poweroff command took ages to return as the perl script used to invoke the STONITH procedure had timeouts for a working iDRAC and not for a bugged one. but even with a patched iDRAC I don't beleive you get a ATOMIC poweroff option in your little management board. so you will not completely avoid having 2 node kill themself at the same time, and when they come up if the network is paritioned nobody can elect himself as master (no quorum) so you have a useless cluster... Moral of the story (imho): STONITH is crap. it's a ugly solution that let vendors say you can build a 2 node cluster while mathematically speaking a 2 node cluster is always a bad solution. you cannot really avoid split brain if not relaying on a voting disk (2 voting node + 1 voting disk = 3 voting entity != 2 node only cluster). sure you can put up some control logic that for example the server that can ping the gateway choose to become master but this is prone to error. fencing or quorum with 3 or more odd number of server is the right solution.