Kevin Closson's Blog: Platforms, Databases and Storage

12 Responses to “Attempted Murder of a 4-Socket AMD Opteron Server with RHEL4. Oracle Can’t Kill It.”

1. 1 CSAB March 24, 2008 at 8:53 pm

   We have a 6 way RAC cluster running 16 core AMD servers in production running RHAS4 x64 across SilverStorm IB. While we had memory errors from two of the servers (thus demonstrating the fail over capabilities), the original three have been real troopers for us.

   16:43:20 up 288 days, 16:46, 3 users, load average: 11.49, 10.45, 8.63

   The load average does not even give a taste of how well they run during our peak times (load over 30 for hours at a time). Glad to hear of others having the same success we are with this platform.

   Reply
2. 2 kevinclosson March 24, 2008 at 9:23 pm

   Wow, CSAB, 16 core? That must be the Sun 4600 ? And just so I’m straight, you mean a 6-node RAC cluster?

   Reply
3. 3 CSAB March 24, 2008 at 10:26 pm

   Correct. 6 @ x4600 each w/ 8 boards AMD dual core and 64GB of RAM. All running RAC. And yes, this is production and not a concept or benchmark system.

   CPU’s of the first ones purchased.

   processor : 15
   vendor_id : AuthenticAMD
   cpu family : 15
   model : 65
   model name : Dual-Core AMD Opteron(tm) Processor 8218
   stepping : 2
   cpu MHz : 2600.027
   cache size : 1024 KB
   physical id : 7
   siblings : 2
   core id : 1
   cpu cores : 2
   fpu : yes
   fpu_exception : yes
   cpuid level : 1
   wp : yes
   flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy
   bogomips : 5199.32
   TLB size : 1088 4K pages
   clflush size : 64
   cache_alignment : 64
   address sizes : 40 bits physical, 48 bits virtual
   power management: ts fid vid ttp tm stc

   CPU’s of the second servers purchased:

   processor : 15
   vendor_id : AuthenticAMD
   cpu family : 15
   model : 65
   model name : Dual-Core AMD Opteron(tm) Processor 8220
   stepping : 3
   cpu MHz : 2800.053
   cache size : 1024 KB
   physical id : 7
   siblings : 2
   core id : 1
   cpu cores : 2
   fpu : yes
   fpu_exception : yes
   cpuid level : 1
   wp : yes
   flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy
   bogomips : 5599.25
   TLB size : 1088 4K pages
   clflush size : 64
   cache_alignment : 64
   address sizes : 40 bits physical, 48 bits virtual
   power management: ts fid vid ttp tm stc

   Reply
4. 4 kevinclosson March 25, 2008 at 4:24 am

   CSAB,

   Cool. I’ve never “talked” to anyone that has these 4600s in production. I always presumed they’d work just fine–depending on workload of course.

   Reply
5. 5 no_treble March 27, 2008 at 2:41 am

   I’m anxiously awaiting the 32 core DL785… we’re using HP gear (currently DL585’s) for our RAC clusters, so it’s only a matter of time… 🙂

   Reply
6. 6 kevinclosson March 27, 2008 at 2:47 pm

   no_treble,

   Do you intend to condense down? That is, do you intend to reduce the degree of horizontal scalability or will you tend to stay with the node count you are at and “fatten” the nodes?

   Reply
7. 7 no_treble March 28, 2008 at 12:00 am

   Currently our group only has a 2-node cluster in production, with four other 2-nodes in testing and development, and one 3-node in testing. So we’d probably just leverage the extra hardware and stay 2 or 3 node.

   In my SA/SE career I’ve had much greater success with many smaller systems making up a cluster (non-RAC, OS-level clusters), not only for greater overall uptime, but for the “invisibility-factor” of one node dropping out if there’s trouble or for maintenance. But the way our people want to use RAC here, the focus seems to be more on the HA benefits than HPC. So if we’re only going to be rolling out two nodes for HA, it seems like it would be better to fatten them up.

   Reply
8. 8 Amir Hameed April 2, 2008 at 7:35 pm

   We are currently running a mission critical 11i system with database running on a 20 CPU dual-core/1.5 GHz SunFire 20k server. This is a pretty costly solution for us. I am looking at the possibility of using a few dual-core 8-way AMD-based 4600 servers via RAC. It will definitely help us reduce the cost without sacrificing the performance. Would someone like to comment on this.

   Reply
9. 9 jeff needham April 17, 2008 at 4:51 am

   When choosing to upgrade RevF CPUS, the 2222s (3.0/1000) have the best write performance of any processor in the family (including the 3.2Ghz parts). With the core being an integral number of the HT baseband, bus cyles are used efficiently.

   Reply
10. 10 Krishna Manoharan June 9, 2008 at 9:38 pm

    Hi Kevin,

    Can you please let me know what kind of load were you running on the 4 socket AMD?

    We have been testing a RAC cluster with Dell 2950’s (2 Quad Core) and these systems were unable to sustain a run-queue of 15+. While oracle does not crash, nor does Linux, the system is so painfully slow that you cannot use it for any practical purposes. The loads we were running were primarily complex Datawarehouse extracts.

    Also I would like your opinion on parallel_instance_groups and splitting loads across nodes. All from oracle and dell recommend to not split loads across nodes.

    Thanks
    Krishna

    Reply
11. 11 kevinclosson June 11, 2008 at 12:37 am

    The workload was software development and test….

    What OS are you running on this slow 2950? Is your DW extraction split across the cluster?

    Reply
12. 12 Krishna Manoharan June 11, 2008 at 3:04 am

    Hi Kevin,

    The OS is Redhat AS 4.0 with Veritas Cluster Filesystem. Splitting the extracts across nodes become a rather contentious topic as Oracle and Dell – both have recommended to not split across nodes.

    We are going to be testing it anyway.

    Thanks
    Krishna

    Reply