Dell Compares RAC and Non-RAC Performance and Cost

May 30, 2007. BLOG UPDATE: Note, the author of the papers I discussed in this blog entry has visited and commented. If nothing else, I recommend reading my follow up regarding the fact that these papers don’t even have the word Oracle in them.

It isn’t very often that you get a tier one hardware vendor directly comparing RAC with non-RAC. When it happens, it is generally by accident. That doesn’t stop me from learning from the information. I hope you will find it interesting too.

So, Dell didn’t exactly set out to compare RAC to non-RAC, but they inadvertently did. In October 2006, they released a series of whitepapers that compare Dell with Oracle to Sun with Oracle. I personally think such comparisons are a complete waste of time since Sun shops are going to run Sun and Windows shops are going to run Windows.

The whitepapers take two swipes at the Sun V490 with 8 UltraSPARC IV+ processors. The first is a cluster of Dell 2950s each with 2 dual-core Xeon 5160 (Woodcrest) processors running Red Hat Enterprise Linux 4. The second was a single Dell 6850 with 4 dual-core Xeon 7140 (Clovertown) processors running Windows 2003. Oh if only they both would have been Linux. No matter though, the comparison is still very interesting. The papers are available at the following URLs:

Even though the paper was intended to cast stones at the Sun V490, there was one particularly interesting aspect of the testing that makes the results helpful in so many other ways. See, Dell did all this testing with the same SAN. In fact, a good portion of these papers are identical. The description of the SAN used for the V490, Clustered 2950s and the 6850 appears in both papers as follows:

Storage for both the Dell and Sun servers was provided by a Storage Area Network (SAN) attached Dell/EMC CX3-80 fibre channel storage array. Each server was attached to the SAN via two QLogic Host Bus Adapters.

There we have it, 3 configurations with the same application code, the same database schema and the same database size. How tasty!

The Workload
They used Dell’s DVD Store Test Application test suite that has been available since about 2005 at http://linux.dell.com/dvdstore/. I have used this workload quite a bit myself actually. It exhibits a lot of the same characteristics as TPC-C—for what it is worth. By the way, the link I provided works, the one in the whitepapers is faulty. I guess that will be my value add.

The Numbers
Like I said, forget about the comparison to Sun. I say look at the comparison of clustered versus non-clustered Oracle. I’ll let you read the papers for the full nitty-gritty, but the summary is worth a lengthy discussion:

Configuration Cost Throughput (Orders/Minute)

Dell 6850 $185,747 32,264
Dell 2950 Cluster $266,852 22,169

Remarkable, and remember, all the important aspects of this sort of test were constant between the two. By important I mean the application, database schema and database size and storage.

Highly Available
Yes, the Dell 2950 cluster theoretically offers more availability. That is important in the event of a failure, sure, but it performs at 31% less throughput than the 6850 solution when it is fully healthy. The important comparison, I believe, is the 6850 to the “brown-out” effect of running an application on a single surviving node of the 2950 cluster. With only one node surviving in the event of a failure, the 2950 cluster solution would be capable of 11,084 orders per minute—about 67% less throughput than the 6850. I think it breaks down like this; the clustered 2950 solution costs 44% more and performs 31% less but in the event of a failure, a surviving 2950 will offer about 1/3rd the throughput of a 6850.

26 Responses to “Dell Compares RAC and Non-RAC Performance and Cost”


  1. 1 Alex Gorbachev January 31, 2007 at 2:26 am

    Kevin, very nice catch. Thanks for this post.

    One of your assumption might be wrong when estimating throughput for a single survived node of 2850 cluster. You assumed that a single node will be able to process only 50% of what two nodes RAC cluster can (22,169/2). Which is probably not true in practice.

    Put it another way around – you don’t usually double your throughput by going from single-node RAC (i.e. practically non-RAC) to two-nodes RAC. If you are lucky you can get 50% increase providing you are CPU bound. Dell’s DVD Store application can be different but my experienced with real life applications suggests that moving to two node RAC requires additional 70% of CPU capacity to sustain the same throughput.

  2. 2 kevinclosson January 31, 2007 at 5:17 am

    Alex,

    You know and I know that RAC doesn’t usually scale 100%…I didn’t want to get into that…I get in enough political trouble as it is :-) Concentrate on the delta between the cluster and the 6850…that is the real gotcha…

    As always, thanks for stopping by

  3. 3 Anjo Kolk January 31, 2007 at 8:01 am

    The question I have is why would Oracle Corp. allow this white paper to be published? It doesn’t really say anything good RAC. May be that is exactly what DELL wanted. Isn’t DELL using Oracle Apps internally? May be they have some problems and they think that Oracle Corp. is not really supporting them. This paper may be a way to put pressure on Oracle.

    Oracle has been really hard on people that say bad (and often true) things about RAC. Why haven’t they succeeded in convincing DELL so that they remove the papers (well may be that is the reason the links didin’t work :))

  4. 4 kevinclosson January 31, 2007 at 3:58 pm

    Anjo,

    The what links don’t work? I just walked through this blog entry and tested each URL and they all seem to work.

  5. 5 yfeefy February 6, 2007 at 6:17 pm

    Isn’t it also possible that Dell is always looking for a way to make linux look bad wrt windows? Is it just coincidence that didn’t they make the windows system the clustered one? Dell has always been one of the most loyal remoras that hangs around the various orifices of the Shark Of Redmond, after all..

  6. 6 kevinclosson February 6, 2007 at 7:06 pm

    yfeefy,

    No. List price is not subjective. Also, their goal was to make Sun look bad so they put their best into both the Linux and Windows test.

    Why wasn’t the Windows test setup clustered? Well, think about it. They used a non-clustered 6850 to stomp both a clustered 2950 and a Sun v90. The only thing a clustered 6850 setup would show is an even more cost (MUCH more cost) and a heavy-handed slam of the other two configurations.

    I don’t think there is any slight of hand here on Dell’s part. I just think their Technical Marketing guys didn’t realize that they were actually releasing an apples comparison of RAC versus non-RAC. Oh, and that sticky bit about using Solaris in violation of EULA…but, what the heck, Sun doesn’t care.

  7. 7 tiff312 February 17, 2007 at 9:56 am

    From the Production experience that we have : (figures are approximative, but they are giving some trends).

    15 months running in production
    Red Hat 3.0 – 10.0.1.4 in RAC 2 nodes – IBM HS40 2×2 procs 2x4Go RAM and a FastT600 Disk.

    The workload is about 250 users (up to 1000 oracle connexions, due to a mix of Windows applications).

    With the two nodes running : 50 – 60% cpu. Users are not fully happy with response times. Few internode data cache transferts(15 years of experience in Oracle single instance, but i can consider that we are still rookies for Rac, with 1 year old.

    Many commands are different from a single instance, take into consideration the effort of training, beginning by the installation.

    Running a failover system in production (this is the target) means running something :
    – robust (failing over and recovering alone)
    – simple to understand
    – simple to manage, because in production, many people are involved, during years…

    I hope, that, in the future, Oracle will be able to meet these goals, the technology which is not new (bought to Rdb in the 1990…) was probably better on VMS 10 years ago.

    Regards.

  8. 8 Chris March 12, 2007 at 1:05 pm

    Kevin,

    is there a little typo in your story?
    You start by mentioning the Dell 2950 cluster but later you refer to a 2850 cluster… My guess is it’s a minor typo.

  9. 9 Bryon March 13, 2007 at 2:28 pm

    Your apples to apples comparison is not even close.

    The RAC setup has 3.0 GHz the non-RAC is 3.4 (10%).

    The RAC setup has only 2MB L3 cache the non-RAC has 16MB. Difficult to quantitize this delte without a better understanding of the workload.

    Regards,
    Bryon

  10. 10 kevinclosson March 13, 2007 at 4:26 pm

    Byron,

    The workload is fully available for you to better understand. I provided the URL to it. This is an Oracle workload and if you could increase the L3 cache in a 2950 to 16MB (to match the 6850) it would not bring the performance up 45% (to match the single node 6850). Cache does not benefit Oracle to that degree since the code and data footprint blows away the cache anyway. Oracle is a load and store workload.

  11. 11 Bryon March 13, 2007 at 5:32 pm

    However, I’m just pointing out that there are several differences between the two system which can be quantified. I don’t have all of the data to do that quantifying but I can fill in some holes if you’d like.
    It has been my experience that installing RAC software on a system will give you about a 10-15% hit in performance due to the overhead of the additional RAC code path.
    Giving simple estimates:
    10% for cpu speed differences (3.0GHz vs 3.4GHz)
    20% FSB memory speed differences (667MHz vs 800 MHz, FBDs vs DDR2)
    15% RAC vs. NonRac
    All in all the numbers are indeed interesting.
    Thanks,
    Bryon

  12. 12 Larry March 13, 2007 at 6:54 pm

    I have to agree with Byron. IMHO, the size of level 2 and 3 cache IS extremely important in Oracle performance. I have only anecdotal evidence:
    When Oracle, Intel, and Dell were working on the “Mega Grid” project at Oracle’s Austin data center, they performed comparison tests on Linux using Intel’s 64-bit Itanium processors and Intel’s 32-bit Xeon processors. The performance for the Itanium processor was twice as high. After analysis, Intel concluded that this was largely not due to the difference in architecture or the 232-bit vs. 64-bit comparison, but due to the difference of cache on the processors. The Itanium had over twice as much level 2 and level 3 cache as the Xeon processor
    Another bit of anecdotal evidence comes from Dell. I have talked with Dell engineers, who tell me that the performance advantage of Itanium processors over EM64T began to erode as the EM64T offered cache equivalent to the Itanium. Now, you can get up to 16MB of cache on the EM64T processor, and Dell no longer sells the Itanium processor. Draw your own conclusions.

  13. 13 kevinclosson March 13, 2007 at 7:32 pm

    Larry,

    Good input. I believe nothing I hear and only half of what I see however. So I would have to see with my own eyes how anyone could finger processor cache when accounting for the performance delta between such two entirely different beasts–32bit Xeon and IA64. The only thing that would make the it stranger would be to hear that it was Oracle9i verus Oracle10g. I’m not saying that you are wrong, but that just smells fishy. The only way to compare processor cache effectiveness is to change NOTHING other than the cache size–same processor, memory, OS, Oracle bits. I had the luxury of doing a lot of that with the Sequent port of Oracle (a long time ago, I know).

    Did the Intel guys bother to mention the difference in memory latency on the Itanium versus the Xeon in that particular case? Oracle’s footprint mashes the cache so it all boils down to memory latency.

    And yes, Dell no longer sells Itanium. Isn’t it odd that about the only thing Oracle uses for TPCC is Itanium. I need to go dig, but I can’t recall the last x86 Oracle TPCC.

  14. 14 PJ April 27, 2007 at 9:55 pm

    Kevin,
    I’ve never bought into the idea that RAC is a good option for scalability. Your observations from the Dell papers seem intuitive and obvious. IMO, aggressive censorship, by Oracle, seems to be the only believable explanation as to why there are not more examples of this.
    In 2003, the company I work for accepted a good sales pitch and bought RAC. The reason was purely for the (advertised) simplicity of recovery from a failure. For us, owning RAC means owning database servers that are always underutilized. Since then, the extra costs and complexities of Vx SFRAC (or any clustering software), combined with the extra demands on sa and dba expertise has been quite expensive.
    Why put such a high load on a set of servers, when
    – there is no room to sustain that load in the event of failure and
    – a smaller number of bigger servers seems to be easier and cheaper?
    What is your take on using RAC primarily for availability?
    -paul

  15. 15 kevinclosson April 30, 2007 at 9:33 pm

    What is your take on using RAC primarily for availability?
    -paul

    …sounds like a good idea to me.

  16. 16 PJ May 1, 2007 at 6:34 pm

    Ok, so, using RAC for scalability is:
    – actively advertised by Oracle, for whatever reason.
    – probably not the best (or even a good) choice
    AND
    using RAC for availability (assuming there is human expertise and unutilized hardware resources to cover losses) is a good idea.
    Maybe, that’s a bit altruistic and naive, but it is my opinion and I get the idea that you think so, too. Yes?
    -paul

  17. 17 PJ May 1, 2007 at 6:41 pm

    ps: I’ve been enjoying the reading on your blog. It jogs memories of the days of when we had two Symmetry 2000’s, named “right” and “left”, because the operators could easily figure out where to put the tape. From there we went through 3 numa-q’s and now we are on Sun. Apparently, before my time, you came here (American TV) to help with problems with OPS v7.0 (!!). 7.0 OPS, wow, talk about iron men and wooden ships. top2, nice job.
    -paul

  18. 18 kevinclosson May 1, 2007 at 7:12 pm

    Hello PJ,

    Yes, I did come to Madison (American TV) to troubleshoot OPS problems on the right/left cluster…and, yes, it was 7.0.12 in production :)

    Wow, that was a long time ago.

    I’m glad you are enjoying the blog.

    I think RAC for scalability depends on the application. If you go to the papers section on my blog you’ll see what sort of testing I’ve done to prove (at least to myself if nobody else) that RAC does scale OLTP. Further, RAC scales parallel query nearly for free (that is, it pretty much just works). The problem is that I know of folks who cobble 2 way servers into, say, 2 node RAC clusters (total of 4 CPUs) and say they are doing so for scalability. Such a config would be more for availability because you’d have to work long and hard to find a workload that scaled to 4 CPUs separated by a RAC interconnect as well as a 4 way SMP.

  19. 19 PJ May 1, 2007 at 9:35 pm

    Kevin,

    Thanks for clarifying your rationale for the commentary about the Dell tests. Your answer makes perfect sense.

    Amtv has grown quite a bit since the “days of old”. Having survived 9 Black Fridays, biggest OLTP load for the whole year, I can attest that RAC has not caused any performance problems.

    Lately, turnover in our sys admin function has reaffirmed (in spades) the importance of:
    a) in depth knowledge of the environment. You are right, “dba’s who don’t know clustering, don’t know RAC”.
    b) choosing rac for the right reasons. “Fools rush in where angels fear to tread”.

    -paul

  20. 20 Dave Jaffe May 29, 2007 at 9:49 pm

    Kevin-

    I recently learned of your blog and your interest in these two papers, of which I am a co-author, along with Todd Muirhead.

    First of all, thanks for picking up the bad URL for our DVD Store database test workload. The papers show http://www.linux.dell.com/dvdstore but the correct URL as you point out doesn’t contain the www. We have no idea how that bug got past our quality control team. (Just kidding, Todd and I don’t have a quality control team, just a few internal reviewers and, of course, our Dell Legal team).

    I think enough people have already responded to your main point, that these two papers constitute an “apples to apples” test of RAC vs single node Oracle, but, just to summarize, we purposely chose these two configurations to have different processors, OSes, cache sizes and memory speeds exactly to prevent such a comparison. In our view we don’t care if customers run Oracle on Windows or Linux, single node or RAC, 2 socket or 4 socket, as long as they run it on Dell.

    Finally, to counter some of the comments that our papers show a bias against Linux or RAC, let me point out that the Dell Information Technology department runs much of our factories and online business on Oracle RAC on Linux.

    In fact, Todd and I have recently published another paper, with two Dell IT guys, on exactly that topic: how Dell IT moved from a mission critical supply chain management application from large Sun systems running single node Oracle to RAC clusters running on PowerEdge servers. That paper and others are available on our new enterprise wiki at delltechcenter.com. We invite you and your readers to come check out the (growing) collection of Oracle (and other) articles, pointers, blogs, etc.

    Dave Jaffe
    dave_jaffe@dell.com

  21. 21 kevinclosson May 30, 2007 at 7:31 pm

    Dave,

    Thanks for stopping by. I’d like to follow-up a bit. First, I’m surprised that although this blog entry has been viewed thousands of times (and now once more since you’ve visited :-)) nobody–not one soul–has pointed out that neither of these papers (pe2950_vs_sunv490.pdf, pe6850_vs_sunv490.pdf) even have the word Oracle in them. Not a single mention. I have waited patiently for someone else to notice that and when I saw your comment in my moderation queue, I thought for sure the time had come!

    You state:

    “In our view we don’t care if customers run Oracle on Windows or Linux, single node or RAC, 2 socket or 4 socket, as long as they run it on Dell.”

    An excellent position to take! I do see a lot of “religion” wrapped up in all this. At the end of the day, it so happens that Oracle works well enough on both Linux and Windows to make a solid solution. Couple that with such tools as RAC and DataGuard and you have enough on hand to tackle just about any problem. You continued with:

    “Finally, to counter some of the comments that our papers show a bias against Linux or RAC, let me point out that the Dell Information Technology department runs much of our factories and online business on Oracle RAC on Linux.”

    I was indeed aware that Dell migrated off the Sun servers some time back. As for the comparison in the papers, I was only pointing out that using the same SAN and same workload (DVD) is sufficient for comparing the solution architecture (clustered versus unclustered). After all, does anyone really think that when Oracle is running full-bore there is really that much difference between Windows and Linux overhead? I should hope not.

  22. 22 Hameed, Amir October 22, 2013 at 4:37 pm

    Hi Kevin,
    A little background on our direction for systems running Oracle. As a corporate directive, we are going to get off of SPARC and move to RHEL/x86_64. I am in the process of migrating one of our mission critical Oracle ERP systems from Solaris/SPARC over to RHEL 6. On the HW side, we are standardizing on Cisco’s UCS bade servers. In one of your past blog posts, you had mentioned that roughly “x” number of CPU cycles are spent on doing an IO. From capacity planning standpoint, I am trying to figure out, if my application go from 40k IOPS to 60k IOPS in the next two years, ,how many x86 CPUs would I need in my RAC farm or how many nodes would I need.

    Thank you for always responding and providing guidance.
    Amir

    • 23 kevinclosson October 22, 2013 at 8:52 pm

      @Amir : So nice of you to stop by. I’m still digging for that post. However, these calculations for CPU/IOP are very easy to do with SLOB and I’m quite sure today’s servers are much more powerful than most folks know. Much more powerful.

      Consider that 128 Sandy Bridge E5-2690 CPUs (8-node cluster) in a VCE vBlock Specialty Solution for High Performance Database delivers 4M random 8K IOPS to the SGAs at the rate of nearly 25,000 IOPS/ DB CPU second. Consider:

      http://wp.me/a21zc-17e
      http://wp.me/a21zc-17f

  23. 24 David Aucoin October 22, 2013 at 4:49 pm

    I think you made a mistake here. Your number for orders/min made no sense

    Dell 6850 $185,747 32,264
    Dell 2950 Cluster $266,85 222,169
    In the paper its 22,169

    [cid:image001.png@01CECF46.ACDA96B0]


  1. 1 Jengates Blog » Oracle RAC vs. Non-Rac Trackback on February 3, 2007 at 2:52 am

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




EMC Employee Disclaimer

The opinions and interests expressed on EMC employee blogs are the employees' own and do not necessarily represent EMC's positions, strategies or views. EMC makes no representation or warranties about employee blogs or the accuracy or reliability of such blogs. When you access employee blogs, even though they may contain the EMC logo and content regarding EMC products and services, employee blogs are independent of EMC and EMC does not control their content or operation. In addition, a link to a blog does not mean that EMC endorses that blog or has responsibility for its content or use.

This disclaimer was put into place on March 23, 2011.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 2,050 other followers

Oracle ACE Program Status

Click It

website metrics

Fond Memories

Copyright

All content is © Kevin Closson and "Kevin Closson's Blog: Platforms, Databases, and Storage", 2006-2013. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Kevin Closson and Kevin Closson's Blog: Platforms, Databases, and Storage with appropriate and specific direction to the original content.

Follow

Get every new post delivered to your Inbox.

Join 2,050 other followers

%d bloggers like this: