Oracle HA Architecture

A very popular topic in many of my courses is Oracle Database Architecture regarding High Availability (HA). This page is supposed to address this topic with a high level overview, covering “Ordinary” Single Instance Databases, Data Guard, Real Application Clusters (RAC) and Extended RAC (sometime called “Stretched Cluster”). The combination of RAC and Data Guard is advertised by Oracle Corp. under the label Maximum Availability Architecture (MAA). In addition to these Oracle HA Solutions, I will briefly cover also one Third Party HA Solution: Remote Mirroring. I don’t intend to dive deep into technical details of all these solutions but instead just want to differentiate them and talk briefly about the various advantages and maybe drawbacks of each of them.

At first, we look at the still most common Oracle Database Architecture: Single Instance. An Oracle RDBMS consists always of one Database - made up by Datafiles, Online Redo Logfiles and Controlfile(s) and at least one Instance – made up by Memory Structures (like a Database Buffer Cache) and Background Processes (like a Database Writer). If we have one Database and multiple Instances accessing it – that’s a RAC. If only one Instance accesses the Database – that’s Single Instance. Small Installations have stored all the components inside one Server like this:

Also common these days is the placement of the Database on a Storage Area Network (SAN) like this:

From a HA perspective this Architecture is vulnerable: Server A and Server B are Single Point of Failures (SPOFs) as well as Database A and Database B are. Also, the Site where these Servers are placed is a SPOF. Should one of the SPOFs fail, the whole Database is unavailable. An “ordinary” RAC addresses the Server SPOFs like this:

Should one of the two Servers fail, the Database C is still available. See here for a more detailed picture about 11gR2 RAC Architecture. HA is not the only reason to use RAC, of course. Amongst others, a further valid reason to use RAC is Scalability: If our requirements increase in the future, we can add another Server (Node) to the cluster. Also, we have options like Service-Management and Load Balancing with RAC. In short words: RAC is not just for HA, but it is out of the scope of this article to address the other reasons in detail. Drawback from a HA perspective of the above architecture is: The Database C resp. the Site C is a SPOF. Should i.e a fire destroy Site C, the Database C is unavailable. Therefore, we have the option to stretch the Database across two Sites, which is usually called Extended RAC:

The Sites are no longer SPOFs here. The Database D is mirrored across the two sites. Drawback of this architecture is the cost of the Network Connection between the two Sites, if long distances are desired. That is critical, because large Data Volume has to be mirrored. In effect, this leads to distances that usually do not extend a couple of kilometers – which may conflict with the goal to get Disaster Protection. Here, Data Guard kicks in: We can reach long distances for Disaster Protection with Data Guard easier, because we do not transmit the whole Data Volume but instead just the (relatively small) Redo Protocol. In the following picture, the Servers hold  Single Instances like Server A and Server B above:

The Redo Protocol from the Primary Database is used to actualize the Standby Database. Should the Primary fail, we can failover to the Standby and continue to work productively. This failover can be done automatically by an Observer (which is called Fast-Start Failover). The distance between the two Servers can reach thousands of kilometers – depending on the kind of redo transmission and the protection level. If we combine RAC and Data Guard, we get MAA. Obviously, MAA is an expensive solution, but it also combines the advantages of RAC and Data Guard.

A popular Third Party HA Solution is Remote Mirroring. On a high level, it looks like that:

The Site is no SPOF here also, like with Extended RAC. Drawbacks of this solution: The distance is usually very limited for the same reason as with Extended RAC. Also, the Secondary Site cannot be used productively while the mirroring is in progress – contrary to the above Oracle HA solutions. With RAC all Servers resp. Sites are in use productively. Even with Data Guard, the Standby is not merely waiting for the Primary to fail. It can also be used for Read Only Access – effectively reducing the load on the Primary:

Above illustrates the 11g New Feature Real-Time Query for Physical Standby Databases. The Standby is accessed Read Only while the actualization continues. Additionally, it is possible to offload Backups to the Physical Standby (also before 11g).

  1. #1 by jametong on March 19, 2010 - 06:37

    Very good summary. Thanks a lot.

  2. #2 by jametong on March 19, 2010 - 08:53

    I have translated this page into Chinese,and have put the translation onto my Blog. If there was some problem , please let me know.

    Chinese edit of this page

  3. #3 by Uwe Hesse on March 19, 2010 - 09:30

    You’re welcome! Feel free to translate it into any language you like and link to it as much as you can :-)

  4. #4 by Shadab Shaukat on June 10, 2010 - 06:48

    An Amazing Article Uwe…..Inspite of being a seasoned Oracle professional. It is informative. I will post this on my blog and link it back to yours as well.I hope you don’t mind. Please visit my blog if you like :) Though I’m nowhere close to your expertise. We lesser mortals can try :)

  5. #5 by Uwe Hesse on June 10, 2010 - 07:38

    Shadab, thank you for the nice feedback! It is of course OK, if you post & link about this article – the more the better :-) Just visited your Blog and it looked nice to me. Tell you what: You’re an expert – no need to hide your light under a bushel :-)

  6. #6 by shadabshaukat on June 10, 2010 - 07:47

    Thank you Sir, its an honor just to be mentioned by you: an Oracle veteran. I gain immensely by reading your blog. Continue the excellent work. Spread the light like an Oracle :)

    Regards

    -Shadab

  7. #7 by Ulfat on June 20, 2010 - 08:46

    Very nice article. I read, read and read. It was excellent explained.

  8. #8 by gheorghe ciortan on July 10, 2010 - 06:39

    A good summary and very interesting.Thanks a lot.

  9. #9 by Uwe Hesse on July 12, 2010 - 17:33

    You’re welcome :-)

  10. #10 by yusuf on August 17, 2010 - 12:25

    thanks for clean explanation of maa.

  11. #11 by Henrik on September 16, 2010 - 15:15

    Uwe,
    you write:
    In effect, this leads to distances that usually do not extend a couple of kilometers – which may conflict with the goal to get Disaster Protection.

    What is the maximum length for that connection? Is there an Oracle recommendation?

    Thanks,
    Henrik

  12. #12 by Uwe Hesse on September 20, 2010 - 09:36

    Henrik,
    technically, there is no “maximum length”. The latency gets too high for large distances to keep up a good performance, though.

    I give you two quotes from the Oracle Whitepaper
    PETERSON, ERIK (2006): Oracle Real Application Clusters on Extended Distance Cluster:

    “This architecture fits best where 2 datacenters are located relatively close (< ~100km) and where the extremely expensive costs of setting up direct cables with dedicated channels between the sites has already been taken." (Page 3) and

    "I am generally comfortable with a RAC on Extended Distance Clusters at distances under 25km, concerned about performance at 50km, and skeptical at 100km or more." (Page 14)

  13. #13 by mdinh on October 21, 2010 - 23:47

    I would like permission to user your diagrams on HA and RAC. I will provide credit as to where the information was obtained.

  14. #14 by Uwe Hesse on October 22, 2010 - 06:00

    Yes, you may do so. Thank you for linking to my site :-)

  15. #15 by macalga on January 20, 2011 - 16:27

    I just discovered your blog after receiving an email from “Oracle University: Expert Corner “and I thank you for this summary about the HA architectures and MAA.

    The mentioned distances are also limited with a proprietary network of fiber optics in which are distributed datacenters?

    Thank you for sharing your knowledge and experience.
    Good luck.

    Best Regards

  16. #16 by Uwe Hesse on January 20, 2011 - 17:02

    Yes, even with Dark Fiber, the network latency probably gets to high to go beyond MAN distances with Extended RAC without significant performance impact. That is why we offer Data Guard resp. Maximum Availabilty Architecture.

  17. #17 by Shyfur on March 25, 2011 - 06:45

    Sir, This is a very very helpful documentation. I was searching which will give me clear picture about HA on Oracle. I really like to thank you from the core of my heart…Sir, I have question,

    Sir, do I need to be a good on Oracle DBA for Implementing HA ? Or its a matter of system level expert to Implement Oracle HA.

    I dont know I asked a you a right question or not. Actually I am interested on HA rather then working with creating databases.

    Good Luck

    rgds,
    Shyfur

  18. #18 by Uwe Hesse on March 25, 2011 - 10:39

    Shyfur,
    you’re welcome :-)
    To your question: In order to implement serious Oracle Database HA, you need to have deep DBA knowledge as a foundation. There are relations to System/Network Administration also, of course. So as an Oracle DBA, you will cooperate at least with Sys-Admins or Network-Admins.

    An Oracle Database is not the only IT-part, relevant for HA, on the other hand. Many other applications and infrastructure need to be taken care of for HA also.

    But my focus is exclusively on Oracle Database HA, and it is a little beyond the scope of this Blog to go into details regarding the mentioned other areas, although I am well aware of them.

  19. #19 by Halim on July 13, 2011 - 12:32

    very nice , Many Many Thanks Uwe Hesse . I love your writing style. its make me very clear.
    again thanks and please keep it up .

    Regards
    Halim

  20. #20 by Uwe Hesse on July 13, 2011 - 15:33

    Halim, you’re welcome :-)

  21. #21 by neerajvasudeva on September 11, 2011 - 22:07

    Presented in very plain & simple language. Nicely presented.

  22. #22 by gouthamk10 on February 7, 2012 - 00:50

    Thanks for excellent document on HA RAC and DATA GUARD.

  23. #23 by Uwe Hesse on February 7, 2012 - 13:27

    Thank you for the nice feedback :-)

  24. #24 by Zee on March 1, 2012 - 18:40

    A very interesting article. We are currently working on moving an existing solution to the Amazon Cloud and implementing a DR capability in two different zones (ie, Primary database in Zone 1 – “eu-zone1″ & Standby database in Zone 2 “eu-zone2″) with Data Guard between the two using the above MAA concept. This would then make it a less expensive MAA solution to what has been outlined above and certainly a more affordable one.

  25. #25 by Uwe Hesse on March 3, 2012 - 12:17

    Thank you, Zee, for stopping by and sharing this information! Sounds interesting but also a little scary :-)
    Seriously, I don’t know enough about Amazon Cloud to judge that setup. Would appreciate it, if you could share also how that worked out for you!

  26. #26 by karthik on March 3, 2012 - 16:11

    Hi Uwe,
    This is the first time i have gone through the blog … it was simple and understandable. With regard to HA in oracle database i want to know little bit on how it works internally , like how the ohad invoke the css/crs for running HA in oracle.
    Please correct me if i am wrong .

  27. #27 by Uwe Hesse on March 5, 2012 - 10:48

    Hi Karthik,
    when I find some time, I may do a little sketch that highlights some Data Guard key components in more detail.

  28. #28 by Zee on March 5, 2012 - 12:24

    Hi Uwe, many thanks for your comments. I am under NDA but once we have actually performed the transition to EC2. I will be happy to share the architecture and lessons learned. I may have some questions regarding the data guard and maa so will keep you posted. Many thanks for a very interesting forum for discussions. :-)

  29. #29 by Fareedh on March 20, 2012 - 22:04

    Very good article explaining the fundamentals of HA. A quick question / comment , In your Extended RAC diagram, in Site D2, Sever D2 has instance D1, is that a typo ? I see that Server D1 also has instance D1. Could you have a same oracle instance shared between two servers D1 and D2?

  30. #30 by Uwe Hesse on March 20, 2012 - 23:35

    Funny that I never did spot that although the pictures is there for years :) You are right: It should be instance D2 of course. I copied the red box and forgot to change that on the other site. Thanks for mentioning!

  31. #31 by Ozgur Umut Vurgun on November 7, 2012 - 10:08

    Hi Uwe,
    Really very good article on HA. All posts are helpful for Oracle Users….
    Thanks for all articles ,

    Best Regards..

    Özgür Umut Vurgun

  32. #32 by Uwe Hesse on November 12, 2012 - 09:36

    Thanks for leaving this nice feedback! You’re welcome :)

  33. #33 by vivien on November 14, 2012 - 17:15

    Always Great explanation !!
    Once again thanks a lot !!!!!!

    Vivien
    DBA Oracle
    SENIOR Oracle instructor

  34. #34 by Uwe Hesse on November 15, 2012 - 11:10

    Thank YOU, Vivien, for stopping by and leaving such a nice feedback! Very much appreciated :-)

  35. #35 by Marc Handelman on November 17, 2012 - 22:43

    Absolutely outstanding post on HA.
    Thanks Uwe and very much appreciated!

  36. #36 by Uwe Hesse on November 28, 2012 - 20:51

    You’re welcome :)

  37. #37 by shinuna on August 29, 2013 - 08:58

    It is usefull thanks a lot sir.

  38. #38 by preethi on May 1, 2014 - 07:28

    Hi Uwe,

    Thanks for the sharing very good informantion about HA and MAA

  39. #39 by YR on May 20, 2014 - 22:28

    very good high level information on HA & MAA specially for a person with very limited oracle background. Thanks!

  40. #40 by oraclerac on July 24, 2014 - 22:26

    Hi Uwe.
    I am from IRAN-TABRIZ.
    Thanks for the sharing very good informantion about High Availability.

    Best regards,
    Arsalan Dehghani

  1. Oracle高可用架构 | a db thinker's home
  2. Alibaba DBA Team » Oracle高可用架构
  3. Oracle HA Architecture Article published in OU EMEA Newsletter « The Oracle Instructor
  4. Oracle Database HA Architecture « Easy Oracle DBA – Shadab Shaukat
  5. Qifeng's life and notes » Oracle高可用架构(转)
  6. Merry Christmas & A Happy New Year 2012! « The Oracle Instructor
  7. Merry Christmas & A Happy New Year 2012! | Oracle Administrators Blog - by Aman Sood

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 2,959 other followers

%d bloggers like this: