Who is the Master Node in my Oracle Cluster?

I got this question during my present course Oracle Grid Infrastructure 11g: Manage Clusterware and ASM while we discussed backup of the OCR. That backup is done by the OCR Master, which can be any node in the cluster. It is therefore recommended to configure the backup location to a shared folder that is accessible from all cluster nodes. But back to the question – here is how to find the OCR Master of the Oracle Cluster:

[grid@host01 ~]$ cat /u01/app/11.2.0/grid/log/host01/crsd/crsd.log | grep -i 'ocr master'
2013-09-17 07:49:44.237: [  OCRMAS][3014282128]th_master:12: I AM THE NEW OCR MASTER at incar 1. Node Number 1
2013-09-17 07:53:00.305: [  OCRMAS][3009534864]th_master:12: I AM THE NEW OCR MASTER at incar 1. Node Number 1
2013-09-17 12:19:21.414: [  OCRMAS][3009604496]th_master: NEW OCR MASTER IS 2
[grid@host01 ~]$ olsnodes -n
host01  1
host02  2
host03  3

(GRID_HOME and local hostname) The CRSD (Cluster Registry Service Demon) is the one who deals with the OCR, which is why I search through this log file.
The Master Node of the Cluster is the one who will become the only surviving node if the interconnect fails completely. It is also the node who will pass the time to the other nodes in the absence of NTP via CTSSD (Cluster Time Synchronization Service Demon). I suppose it is always the same as the OCR Master, but just to be on the safe side, you can check that this way:

[grid@host01 ~]$ cat /u01/app/11.2.0/grid/log/host01/cssd/ocssd.log | grep -i 'master node'
2013-09-17 07:48:53.541: [    CSSD][3004672912]clssgmCMReconfig: reconfiguration successful, incarnation 274866533 with 1 nodes, local node number 1, master node number 1
2013-09-17 07:49:47.427: [    CSSD][2990480272]clssgmCMReconfig: reconfiguration successful, incarnation 274866534 with 1 nodes, local node number 1, master node number 1
2013-09-17 07:52:27.595: [    CSSD][2989472656]clssgmCMReconfig: reconfiguration successful, incarnation 274866536 with 1 nodes, local node number 1, master node number 1
2013-09-17 07:59:20.783: [    CSSD][2989472656]clssgmCMReconfig: reconfiguration successful, incarnation 274866537 with 2 nodes, local node number 1, master node number 1
2013-09-17 11:34:59.157: [    CSSD][2989472656]clssgmCMReconfig: reconfiguration successful, incarnation 274866538 with 3 nodes, local node number 1, master node number 1
2013-09-17 12:18:48.885: [    CSSD][2992602000]clssgmCMReconfig: reconfiguration successful, incarnation 274866540 with 3 nodes, local node number 1, master node number 2
2013-09-17 12:22:52.660: [    CSSD][2992602000]clssgmCMReconfig: reconfiguration successful, incarnation 274866541 with 2 nodes, local node number 1, master node number 2
2013-09-17 12:23:32.836: [    CSSD][2992602000]clssgmCMReconfig: reconfiguration successful, incarnation 274866542 with 3 nodes, local node number 1, master node number 2
2013-09-17 12:26:29.474: [    CSSD][3016432528]clssgmCMReconfig: reconfiguration successful, incarnation 274866543 with 3 nodes, local node number 1, master node number 2
2013-09-17 12:28:42.960: [    CSSD][2987871120]clssgmCMReconfig: reconfiguration successful, incarnation 274866544 with 3 nodes, local node number 1, master node number 2

The CSSD (Cluster Synchronization Service Demon) is the one who deals with the Voting File that is used to determine which nodes must reboot and which nodes will survive in case of a problem with the Interconnect. Therefore, I search through its log file to determine the Master of the Cluster.

The question comes up quite often, so this little post will be handy to point to in the future. Hope you find it useful as well :-)

,

  1. #1 by Anju Garg on September 17, 2013 - 17:11

    Here is a related post :

    http://oracleinaction.com/find-master-node/

    Regards
    Anju Garg

  2. #2 by Pavan Kumara on September 18, 2013 - 13:26

    That’s an nice post with good information… !!

  3. #3 by Nitin Vengurlekar on September 20, 2013 - 20:53

    Nice article. I’ll just add that, the term Master Node is very ambiguous, and not always meaningful unless you talk about the context of the code layer. You kinda of alluded that w/ CTSS. For example several CRS subcomponents have “master nodes” ; such as CRS Policy Engine, CRSPE, (which is the core layer of CRS) and it has a master. If you search for CRSPE in crsd.log (like OCRMAS) you’ll see it tries to locate its master. Note, most subcomponents will affine to the oldest node, but I know this is going to change in future to distribute the work across nodes. Anyways, knowing the master node is only really useful during any coordination or convergence activity (reconfiguration, batch writes to OCR, etc.), but all this is transparent to the admin…so only cool to know for us geeks (or debugging efforts)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 2,959 other followers

%d bloggers like this: