Appliance? How #Exadata will impact your IT Organization

The impact, Exadata will have on your IT’ s organizational structure can range from ‘None at all’ to ‘Significantly’. I’ll try to explain which kind of impact will likely be seen under which circumstances. The topic seems to be very important, as it is discussed often in my courses and also internally. First, it is probably useful to be clear about the often used term ‘Appliance’ in relation to Exadata: I think that term is misleading in so far as Exadata requires ongoing Maintenance & Administration, similar to an ordinary Real Application Cluster. It is not like you deploy it once and then it takes care of itself.

Due to its nature as an Engineered System, it is partly much easier to deploy and to manage than a self-cooked solution, but there are still administrative tasks to do with Exadata, as the following picture shows:

The administrative tasks on ExadataWithout pretending precision (therefore no percentages mentioned), you can see that the major task on Exadata is by far Database Administration, while Storage Administration and System/Network-Administration are the smaller portions. The question is now how this maintenance is done. You could of course decide to let Oracle do it partly or completely – very comfortable, but not the focus of this article. Instead, let’s assume your IT Organization is supposed to manage Exadata.

1. The siloed approach

I think it’s important to emphasize that your internal organization does not have to change because of Exadata. You can stay as you are! Many customers have dedicated and separated teams for Database Administration, Storage Administration and System/Network Administration. It is perfectly valid to decide that the different components of your Exadata are managed by those different teams as on this picture visualized:

Differend separate Teams manage ExadataThe responsiveness and agility towards business requirements will be the same with the siloed approach as it is presently with other Oracle-related deployments at your site. There is obviously some internal overhead involved, because the different teams need to be coordinated for the Exadata maintenance tasks.

This approach is likely to be seen if Exadata is deployed more for tactical reasons (we put this one critical and customer-facing OLTP system on an Exadata eight-rack, e.g.) respectively if your internal organization is very static and difficult to change. I have seen this from time to time (SysAdmins in the course who have never seen Oracle Databases before), but I would say it is more an exception than the rule.

In short: You will get the technical benefits out of Exadata, but you will leave benefits that come from increased administrative efficiency and agility on the table.

2. The Exadata Database Administration (EDBA) Team approach

Here you give the administrative ownership of Exadata to a team, built largely or exclusively from your DBAs and give them named contacts from the other IT groups, so they can get their expertise on demand, like this picture here shows:

The EDBA TeamWhy is the DBA team supposed to own Exadata? Because as shown on Pic 1 above, they are doing the major task on Exadata anyways. And it is relatively easy to train these DBAs for Oracle Database Administration on Exadata, because they know already most of it:

Exadata-specific Database Administration tasksI simply cannot emphasize enough how important this point is: The know-how of your Oracle DBAs remains completely valid & useful on Exadata! The often huge investment into that know-how keeps paying back! I am still surprised that the true quote ‘Exadata is still Oracle!’ is from a competitor and not from our Marketing :-)

For DBAs, it is similar as moving from a Single-Instance system to RAC. Some additional things to learn, like Smart Scan and Storage Indexes. Over time, the DBAs on that team may incorporate the know-how they gather at first from their named contacts of the other groups. The pragmatic EDBA approach is likely if Exadata is seen as a strategic platform, but the effort to build a DBMA team is regarded to high respectively the internal organizational structure is not flexible enough to start with the third approach. Administrative responsiveness and agility are already higher here as with the first approach, though.

3. The Database Machine Administration (DBMA) Team

Either the EDBA team evolves over time into a DBMA team or you start straight with this approach, typically by assigning some Admins from the Storage/System/Network groups to the DBMA team and let all team members cross-train in the three areas:

The DBMA TeamThis gives you the optimal administrative responsiveness and agility for your business requirements related to Exadata, which is what the majority of customers will probably want to achieve – at least in the long term. You will see this approach most likely if Exadata is supposed to be a strategic platform at your site. The good news (in my opinion) for the team members here is that they all will enlarge their mental horizon and gather new and exciting skills!

How to manage Exadata?

Regardless of your particular approach, the most important single administrative tool will be the Enterprise Manager 12C:

Enterprise Manager 12C for ExadataYou can see that you can manage all Exadata components with a single, centralized tool. The picture is from our excellent Whitepaper Oracle Enterprise Manager 12c: Oracle Exadata Discovery Cookbook

Much of the above, especially the terms EDBA and DBMA can be found in our Whitepaper Operational Impact of Deploying an Oracle Engineered System (Exadata) – for some reason it is presently only available through our Oracle Partner Network. It’s kind of our official ‘party line’ about the topic, though. I’d also like to recommend Arup Nanda’s posting Who Manages the Exadata Machine? that contains related information as well.

Finally, let me highlight the offerings of Oracle University for Exadata – you will see that there is training available for any of the three approaches :-)

About these ads

  1. #1 by coskan on February 25, 2013 - 01:01

    Uwe, Great post, but I think it needs more detailed information for real life of a DBA who is managing Exadata on consolidated environment which consist of different product groups as EDBA.

    Consolidated environment Situation needs extreme amount of time governing people and applications

    Our exadata DBAs (3 of them in follow the sun model) are currently having many(really many) meetings (conference calls) just to convince business groups to upgrade to a certain version/BP patch, flash cache upgrades, battery replacements, disk replacements and on site engineer visits etc.

    Once you convince them, it is just a beginning, it continues to start with scheduling and planning, raising changes chasing people for approval spending endless hours on mail threads and telephone.

    This ends up spending too much time on political activities rather than technical.

    Worst is that if your business groups are early adapters where so called “online” changes caused interruption, there is no way you can convince them to have weekday changes so everything needs to be scheduled for the weekends.

    In my humble opinion Exadata especially EDBA approach is really a pain as single team needs to manage/plan/implement everything in whole stack.

  2. #2 by Uwe Hesse on February 25, 2013 - 08:01

    Hi Coskan,
    thank you for sharing your experience here, I really appreciate it! What you say makes of course sense and is not so surprising – I think it was kind of predicted in the article even.
    It’s often for ‘political’ reasons that the DBMA team is not the first choice, although it offers the most administrative efficiency for a strategic Exadata deployment.
    So maybe your EDBA team will evolve into a DBMA team then :-)

  3. #3 by oraclebychoice on March 11, 2013 - 12:15

    Dear Uhesse,

    Thanks for sharing this nice article.

    I have a question on one of your old post – Tablespace Point in time recovery.

    I am trying to test TSPITR recovery as per the steps given by you.

    I followed all the steps but getting below error message at last (pls check last 2 lines):

    ……………………………………………
    ……………………………………………
    Finished recover at 2013-03-11:13:02:01

    database opened

    contents of Memory Script:
    {
    # make read only the tablespace that will be exported
    sql clone ‘alter tablespace TEST_TBS read only’;
    # create directory for datapump import
    sql “create or replace directory TSPITR_DIROBJ_DPDIR as ”
    /home/oracle/aux_dir””;
    # create directory for datapump export
    sql clone “create or replace directory TSPITR_DIROBJ_DPDIR as ”
    /home/oracle/aux_dir””;
    }
    executing Memory Script

    sql statement: alter tablespace TEST_TBS read only

    sql statement: create or replace directory TSPITR_DIROBJ_DPDIR as ”/home/oracle/aux_dir”

    sql statement: create or replace directory TSPITR_DIROBJ_DPDIR as ”/home/oracle/aux_dir”

    Performing export of metadata…
    EXPDP> ORA-39006: internal errorORA-39065: unexpected master process exception in DISPATCHORA-25153: Temporary Tablespace is Empty
    ORA-39097: Data Pump job encountered unexpected error -25153
    EXPDP> ORA-39004: invalid stateORA-39053: parameter or attribute TABLESPACE_EXPR must be defined for a TRANSPORTABLE job

    I checked on Oracle support and re-run the catdph.sql,prvtdtde.plb, catdpb.sql, dbmspump.sql, utlrp.sql and bounced the DB but this issue still persists.

    Kindly let me know if I am missing some thing here.

    Thanks in advance.

    Regards
    Saurabh

  4. #4 by Uwe Hesse on March 14, 2013 - 16:45

    Saurabh, your question has no relation to the posting. Please engage Oracle Support to resolve your technical problems. Or try an open forum like http://forums.oracle.com/forums/forum.jspa?forumID=61&start=0

  1. Exadata: Storage Cells IO performance metrics and IO distribution with DB servers « bdt's oracle blog
  2. The Oracle Platform Administrator | Julian Dontcheff's Database Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 2,430 other followers

%d bloggers like this: