I want to make these two points right out of the gate:
- I do not question Oracle’s IOPS claims in Exadata datasheets
- Everyone makes mistakes
Everyone Makes Mistakes
Like me. On January 21, 2015, Oracle announced the X5 generation of Exadata. I spent some time studying the datasheets from this product family and also compared the information to prior generations of Exadata namely the X3 and X4. Yesterday I graphed some of the datasheet numbers from these Exadata products and tweeted the graphs. I’m sorry to report that two of the graphs were faulty–the result of hasty cut and paste. This post will clear up the mistakes but I owe an apology to Oracle for incorrectly graphing their datasheet information. Everyone makes mistakes. I fess up when I do. I am posting the fixed slides but will link to the deprecated slides at the end of this post.
We’re Only Human
Wouldn’t IT be a more enjoyable industry if certain IT vendors stepped up and admitted when they’ve made little, tiny mistakes like the one I’m blogging about here? In fact, wouldn’t it be wonderful if some of the exceedingly gruesome mistakes certain IT vendors make would result in a little soul-searching and confession? Yes. It would be really nice! But it’ll never happen–well, not for certain IT companies anyway. Enough of that. I’ll move on to the meat of this post. The rest of this article covers:
- Three Generations of Exadata IOPS Capability
- Exadata IOPS Per Host CPU
- Exadata IOPS Per Flash SSD
- IOPS Per Exadata Storage Server License Cost
Three Generations of Exadata IOPS Capability
The following chart shows how Oracle has evolved Exadata from the X3 to the X5 EF model with regard to IOPS capability. As per Oracle’s datasheets on the matter these are, of course, SQL-driven IOPS. Oracle would likely show you this chart and nothing else. Why? Because it shows favorable, generational progress in IOPS capability. A quick glance shows that read IOPS improved just shy of 3x and write IOPS capability improved over 4x from the X3 to X5 product releases. These are good numbers. I should point out that the X3 and X4 numbers are the datasheet citations for 100% cached data in Exadata Smart Flash Cache. These models had 4 Exadata Smart Flash Cache PCIe cards in each storage server (aka, cell). The X5 numbers I’m focused on reflect the performance of the all-new Extreme Flash (EF) X5 model. It seems Oracle has started to investigate the value of all-flash technology and, indeed, the X5 EF is the top-dog in the Exadata line-up. For this reason I choose to graph X5 EF data as opposed to the more pedestrian High Capacity model which has 12 4TB SATA drives fronted with PCI Flash cards (4 per storage server). 
The tweets I hastily posted yesterday with the faulty data points aimed to normalize these performance numbers to important factors such as host CPU, SSD count and Exadata Storage Server Software licensing costs. The following set of charts are the error-free versions of the tweeted charts.
Exadata IOPS Per Host CPU
Oracle’s IOPS performance citations are based on SQL-driven workloads. This can be seen in every Exadata datasheet. All Exadata datasheets for generations prior to X4 clearly stated that Exadata IOPS are limited by host CPU. That is a very important fact to understand because SQL-driven IOPS is a host metric no matter what your storage is.
Indeed, anyone who studies Oracle Database with SLOB knows how all of that works. SQL-driven IOPS requires host CPU. Sadly, however, Oracle ceased stating the fact that IOPS are host-CPU bound in Exadata as of the advent of the X4 product family. I presume Oracle stopped correctly stating the factual correlation between host CPU and SQL-driven IOPS for only the most honorable of reasons with the best of customers’ intentions in mind.
In case anyone should doubt my assertion that Oracle historically associated Exadata IOPS limitations with host CPU I submit the following screen shot of the pertinent section of the X3 datasheet: 
Now that the established relationship between SQL-driven IOPS and host CPU has been demystified, I’ll offer the following chart which normalizes IOPS to host CPU core count: 
I think the data speaks for itself but I’ll add some commentary. Where Exadata is concerned, Oracle gives no choice of host CPU to customers. If you adopt Exadata you will be forced to take the top-bin Xeon SKU with the most cores offered in the respective Intel CPU family. For example, the X3 product used 8-core Sandy Bridge Xeons. The X4 used 12-core Ivy Bridge Xeons and finally the X5 uses 18-core Haswell Xeons. In each of these CPU families there are other processors of varying core counts at the same TDP. For example, the Exadata X5 processor is the E5-2699v3 which is a 145w 18-core part. In the same line of Xeons there is also a 145w 14c part (E5-2697v3) but that is not an option to Exadata customers.
All of this is important since Oracle customers must license Oracle Database software by the host CPU core. The chart shows us that read IOPS per core from X3 to X4 improved 18% but from X4 to X5 we see only a 3.6% increase. The chart also shows that write IOPS/core peaked at X4 and has actually dropped some 9% in the X5 product. These important trends suggest Oracle’s balance between storage plumbing and I/O bandwidth in the Storage Servers is not keeping up with the rate at which Intel is packing cores into the Xeon EP family of CPUs. The nugget of truth that is missing here is whether the 145w 14-core E5-2697v3 might in fact be able to improve this IOPS/core ratio. While such information would be quite beneficial to Exadata-minded customers, the 22% drop in expensive Oracle Database software in such an 18c versus 14c scenario is not beneficial to Oracle–especially not while Oracle is struggling to subsidize its languishing hardware business with gains from traditional software.
Exadata IOPS Per Flash SSD
Oracle uses their own branded Flash cards in all of the X3 through X5 products. While it may seem like an implementation detail, some technicians consider it important to scrutinize how well Oracle leverages their own components in their Engineered Systems. In fact, some customers expect that adding significant amounts of important performance components, like Flash cards, should pay commensurate dividends. So, before you let your eyes drift to the following graph please be reminded that X3 and X4 products came with 4 Gen3 PCI Flash Cards per Exadata Storage Server whereas X5 is fit with 8 NVMe flash cards. And now, feel free to take a gander at how well Exadata architecture leverages a 100% increase in Flash componentry: 
This chart helps us visualize the facts sort of hidden in the datasheet information. From Exadata X3 to Exadata X4 Oracle improved IOPS per Flash device by just shy of 100% for both read and write IOPS. On the other hand, Exadata X5 exhibits nearly flat (5%) write IOPS and a troubling drop in read IOPS per SSD device of 22%. Now, all I can do is share the facts. I cannot change people’s belief system–this I know. That said, I can’t imagine how anyone can spin a per-SSD drop of 22%–especially considering the NVMe SSD product is so significantly faster than the X4 PCIe Flash card. By significant I mean the NVMe SSD used in the X5 model is rated at 260,000 random 8KB IOPS whereas the X4 PCIe Flash card was only rated at 160,000 8KB read IOPS. So X5 has double the SSDs–each of which is rated at 63% more IOPS capacity–than the X4 yet IOPS per SSD dropped 22% from the X4 to the X5. That means an architectural imbalance–somewhere. However, since Exadata is a completely closed system you are on your own to find out why doubling resources doesn’t double your performance. All of that might sound like taking shots at implementation details. If that seems like the case then the next section of this article might be of interest.
IOPS Per Exadata Storage Server License Cost
As I wrote earlier in this article, both Exadata X3 and Exadata X4 used PCIe Flash cards for accelerating IOPS. Each X3 and X4 Exadata Storage Server came with 12 hard disk drives and 4 PCIe Flash cards. Oracle licenses Exadata Storage Server Software by the hard drive in X3/X4 and by the NVMe SSD in the X5 EF model. To that end the license “basis” is 12 units for X3/X5 and 8 for X5. Already readers are breathing a sigh of relief because less license basis must surely mean less total license cost. Surely Not! Exadata X3 and X4 list price for Exadata Storage Server software was $10,000 per disk drive for an extended price of $120,000 per storage server. The X5 EF model, on the other hand, prices Exadata Storage Server Software at $20,000 per NVMe SSD for an extended price of $160,000 per Exadata Storage Server. With these values in mind feel free to direct your attention to the following chart which graphs the IOPS per Exadata Storage Server Software list price (IOPS/license$$). 
The trend in the X3 to X4 timeframe was a doubling of write IOPS/license$$ and just short of a 100% improvement in read IOPS/license$$. In stark contrast, however, the X5 EF product delivers only a 57% increase in write IOPS/license$$ and a troubling, tiny, 17% increase in read IOPS/license$$. Remember, X5 has 100% more SSD componentry when compared to the X3 and X4 products.
Summary
No summary needed. At least I don’t think so.
About Those Faulty Tweeted Graphs
As promised, I’ve left links to the faulty graphs I tweeted here: Faulty / Deleted Tweet Graph of Exadata IOPS/SSD: http://wp.me/a21zc-1ek Faulty / Deleted Tweet Graph of Exadata IOPS/license$$: http://wp.me/a21zc-1ej
References
Exadata X3-2 datasheet: http://www.oracle.com/technetwork/server-storage/engineered-systems/exadata/exadata-dbmachine-x3-2-ds-1855384.pdf Exadata X4-2 datasheet: http://www.oracle.com/technetwork/database/exadata/exadata-dbmachine-x4-2-ds-2076448.pdf Exadata X5-2 datasheet: http://www.oracle.com/technetwork/database/exadata/exadata-x5-2-ds-2406241.pdf X4 SSD info: http://www.oracle.com/us/products/servers-storage/storage/flash-storage/f80/overview/index.html X5 SSD info: http://docs.oracle.com/cd/E54943_01/html/E54944/gokdw.html#scrolltoc Engineered Systems Price List: http://www.oracle.com/us/corporate/pricing/exadata-pricelist-070598.pdf , http://www.ogs.state.ny.us/purchase/prices/7600020944pl_oracle.pdf
Kevin Closson's Blog: Platforms, Databases and Storage 
These graphs seem to indicate that the recent Oracle Exadata X5 announcements around “lowest price and highest performance”
are suspect or their Oracle Exadata X5 benchmark data is flawed.
Maybe their engineers made a benchmarking mistake or published the wrong data? Better take a backup of it before they take it down and republish the numbers or everyone begins receiving a 404 error on the published benchmarks … laughing, chuckle, chuckle, grin, grin …
way back machine has my back
Hi Kevin
very insightful post which will no doubt help demystify the true benefits-TCO of ExaData further!
Point of clarification: where you refer to Oracle not giving customers choice of host CPU and “…All of this is important since Oracle customers must license Database software by the host CPU core” – this is not such as issue providing you make sure that you following Oracle HARD Partition rules and only allocate the number of cores you actually need on the guests. I grant you that you would then potentially have a bunch of cores left over doing next to nothing though.
e.g. HARD partition CPU rules setup on OVM 3 https://docs.oracle.com/cd/E26996_01/E18549/html/vm_hardpart.html
Surely the bigger TCO concern is the Exa Storage capacity base support fees? Oracle fees tied into data growth is a new cost paradigm that could catch many out after they have purchased Exadata when its to late to rip it out and the addiction has set in?
@Gaffer: Thanks for the info. I know the recent Exadata products support OVM on ACFS. I just fail to believe anyone would do that.
First, thanks for giving more advertising to [… comment truncated by publisher for brevity …]
@Phil: I can’t let your comment through moderation since you changed your tweet handle from the more recognizable, official Voice of Oracle tweet handle we all formerly recognized (@Phil_Oracle).
Thanks Kevin, another enjoyable post demystifying the Oracle Marketing smokescreen.
It may be worth adding to your collection of reference links the datasheet for the Exadata X5 NVMe flash cards, which are in fact Intel DC P3600 Series SSDs:
http://www.intel.com/content/www/us/en/solid-state-drives/ssd-dc-p3600-spec.html
We know this because the Exadata X5 press kit contained pictures of the cards complete with Intel logo and model numbers. Yet intriguingly, the X5-2 datasheet talks about the “Flash Accelerator F160 PCIe Card”, a model which at the time of writing doesn’t appear to exist on Oracle’s website…
Hi flash,
The body of my post has a ref to Oracle’s F160 literature:http://docs.oracle.com/cd/E54943_01/html/E54944/gokdw.html#scrolltoc
According to folks inside Oracle this particular SSD is supposedly an Oracle-special from Intel. I’ll take the word of the individual who told me that even though I think it is astounding that a) Intel would do anything special for an outfit that has zero volume to speak of and b) also has a hardware platform (albeit dying) in SPARC that Oracle says is superior to Intel’s products. In other words, I’m skeptical but cannot shoot down the claim that this SSD is a one-off for Oracle’s gain.
Hi Kevin,
Database licensing on Exadata is a lot more versatile than stated in this post. With Capacity on Demand and Trusted Partitions, customers have a lot options at their disposal as to how they deploy their Oracle assets on Exadata Database Machine.
This post discusses all these options in some detail https://blogs.oracle.com/dr/entry/capacity_on_demand_and_trusted
@ExadataPM : I’ll let your comment through and my readers can see for themselves whether these optional deployment methods are palatable. So I take it you’ve replaced Ron Weiss? Or maybe Mahesh’s role?
I took over from Mahesh nearly a year ago, Ron is no longer with Oracle.
@ExadataPM : Yeah I saw that Ron had retired. I’ll have to ping Mahesh to see what he’s up to these days. If you see him ask him if he’s finally healed up from the time we got stranded in Berlin.
Pleased to meet you. I do have a follow up to your comment that I hope you’ll read and respond to. I’ll get to that later.
Oracle seems to support a feature on Exadata they don’t allow on traditional x86 servers by disabling cores to reduce Oracle licensing. This is in addition to Oracle not allowing VMware or Hyper-V customers to reduce licensing by VM but Exadata does allow it with OVM. Amazing ‘lock-in’ and that customers would even consider it.
Further, your blog states pretty definitely the problem with Oracle on Intel; it costs a lot to license Oracle on the servers thus Exadata jumps through hoops to reduce it.
If customers want the most flexibility on the compute platform, at the highest levels of performance while providing the maximum license/maintenance savings they should run it on IBM’s Power8. Secondly, they should do themselves another favor and look at DB2 on Power8 to get more features for a lower cost. Even PostgreSQL on Linux is a better option than being enslaved to the Oracle Savings and Loan.
@powertheenterprise : I don’t want to go into the topic of migrations form Oracle Database. In this series I am specifically focused on getting bang for the Oracle Database buck without lockin and sub-optimal platform choices.
@powertheenterprise: Juan Loaiza discusses the next generation innovations of OVM on Oracle Exadata X5. Evidently, OVM bypasses the OS and hypervisor with Direct to Wire Infiniband protocols that supposedly achieve 3X OLTP performance. This is a major virtualization differentiator apart from the whole Trusted Partition for OVM to limit software licenses.
However, I’m somewhat surprised that the Oracle Exadata X5 published that they only achieved 4.14 M 8K rIOPS on random I/O (8K OLTP) from the Exadata Smart Flash Cache on the latest Intel Haswell offerings. Speaking from personal experience, achieving beyond 5.2 M 8K rIOPS on random I/O (8K OLTP) from PCIe flash cache on Intel Haswell is achievable without having to license high redundancy ASM disks on Oracle Exadata X5 Storage Servers. It sounds like the Oracle Exadata X5 Smart Flash Cache’s data destaging capabilities that were designed around maintaining Oracle MAA (high, normal redundancies) has become an OLTP IO design limitation on the latest/faster Intel Haswell generation.
Maybe a more truthful marketing lingo would be “Higher costs with Extreme Flash Storage Server and Above Average Performance …”
You imply that OVM improves performance but one analyst which @kevinclosson posted on twitter today said this “Oracle has also added OVM…Although this isolation adds to the overhead with system resources” https://twitter.com/kevinclosson/status/563774939405508608
@ExadataPM : May I ask if the must-license minimum of 40% is a per-host or a per-configuration C.O.D. Minumum? In other words, can one license 100% of one host and only 40% of various others?
40% minimum is at a per server level, you can have one X5-2 database server at 60%, other at 100% and another at 40%. We recommend users to have a consistent distribution if the servers are a part of a RAC cluster, but at end the of the day the choice remains with the user.
Thank you @ExadataPM.
Please allow me to speak on behalf of Oracle’s entire customer base on this matter. What has been implemented as an Exadata-only Capacity On Demand feature is a feature that in no way requires Exadata storage. It is an entirely host-based solution to help customers save money on extremely expensive Oracle database software. This feature first appeared in the Oracle Database Appliance and it should have been supported on any Linux x64 machine because that is *exactly* what both the Oracle Database Appliance and the hosts in Exadata Database Machine are.
Oracle continues to damage their reputation with users when they implement host-level features such as this Capacity On Demand (and, in fact, generic database features like Hybrid Columnar Compression) but instead tie the features to the purchase of some other unwanted Oracle technology like Exadata or ZFS Storage Appliance or Pillar Data (FS1). Customers realize that it’s their RDBMS maintenance fees that fund these features.
@kevinclosson – I am standing up applauding. Thank you for saying this as you are spot on. As a IBM Power seller this helps me as it shows vendor lock-in, lack of trust but loss of competitive leverage for platform choice. They think they are being clever but instead customers view it with resentment. Best thing for any technology is competition as it drives innovation. If Oracle owns the entire stack will there continue to be innovation?
Well Kevin I’m afraid your assessment is not accurate.
CoD type functionality is available on generic x86 servers, please refer to the Oracle Partitioning Policy and the following white paper.
Click to access ovm-hardpart-168217.pdf
“CoD type functionality is available on generic x86 servers”
@ExadataPM : Gurmeet, thanks for staying with the thread. What you refer to now is based on OVM. We fully dismiss this as mitigating information. Please allow me to explain.
When running Oracle Database on industry-standard X64 servers housed in an Exadata rack–or, indeed, in a little Oracle Database Appliance broom-closet-system, one does not need OVM to enjoy Capacity on Demand. What you folks have done is to make customers running the same Database code on the same hardware choose your market-lagging OVM as a way to get a form of Capacity on Demand.
Oracle constantly uses market dominance to shill up the market-lagging portions of their portfolio and not everyone is keen on the situation.
The database host servers in Oracle Database Appliance and Exadata Database Machine are Sun Server X5-2 systems running Oracle Linux. Consider a scenario. Let’s say a customer buys a Sun Server X5-2 and loads Oracle Linux on it. Why, in the name of customer trust, would they not be able to implement the same sort of Capacity on Demand available to them when the same Sun Server X5-2 is cobbled into either Oracle Database Appliance or in an Exadata Database Machine rack?
See, this is the sort of thing that sends customer ire, aimed at Oracle, through the stratosphere. Yes, customers still seem willing to take this whipping. Nothing lasts forever.
@ExadataPM
The document you are referencing to has the following text in its footer:
“This document is provided for information purposes only, and the contents hereof are subject to change without notice.”
Do you have an official license document that describes the use of cpu pinning in OVM to limit the number of processors that needs to be licensed?
Dumb question: I’m currently evaluating Exadata as a solution for a customer and met the X4->X5 upgrade warmly. For the same hardware costs, i’m getting 33% more CPU cores in my DB and storage nodes, doubled the amount of flash and (per the first graph) am more than doubling my WIOPS and nearly doubling my RIOPS.
Do I really care if the IOPS per SSD went down slightly? The IOPS for the *solution* went up, for the same hardware cost.
What’s the real concern here? Yes they’ve increased the cores in the solution, which will mean more Oracle s/w licensing costs … but you’re … also getting more horsepower for your buck.
@Todd
Thanks for your comment. I’ll answer from the bottom because your last sentence has the answer. What’s the real concern here? The fact that core packing and license cost are not scaling with the hardware.
Since Oracle’s own cited datasheet IOPS/core nor IOPS/SSD are not scaling I can’t see how one can say there is more horsepower for the buck. The license dollars are based on these added components. There are more components (cores, SSDs), yes, but I pointed out the added cost that comes with that. I’ve detailed how the Storage Server Software increases from $120K to $160K per Storage Server when comparing X4 HP to X5 EF (the HP model is discontinued).
As a field practitioner helping out a customer I really appreciate your visit here. I hope you’ll answer these questions:
1. X4 is a product Oracle shipped for only 13 months (was announced in Dec 2013). Are you saying you have a customer that is considering going from a systems that is less than 13 months old (an X4) an X5? I guess what I’m asking is, who “met the X4-X5 upgrade warmly?”
2. In the sentence where you said 33% more cores, double flash and double WIOPS, that would be a citation of the differences in the datasheets, correct? In other words, you don’t have a customer workload that exhibits double RIOPS and WIOPS when comparing an application running on the X4 to running on the X5. Right?
3. Since you are evaluating Exadata as a solution for a customer would you be so kind as to say what hardware is currently in place? For example, what type of CPUs, how many CPUs, what type of storage, how many paths of what type of storage plumbing (e.g., 4GFC, etc). I’ve always been keen to know what people leave behind when they adopt Exadata.
4. Since you are evaluating Exadata as a solution for a customer can you please also say what other solutions are in consideration? Perhaps an alternative converged infrastructure play?
We would be most grateful for answers to these questions but understand if you are too busy.
We started doing evaluations of Exadata in Sept 2014. We got a quote for a hardware proposal at $X dollars … then in Dec 2014 the X5 rev was released. From the customer’s perspective, suddenly we’re getting more CPU, more Flash for the same dollar amount. That’s how a customer looks at it.
What’s really “new” in X5? A new chip? Is the Intel 18-core chip so “new” that you’re worried about being on the bleeding edge? To me, it just seems like an incremental increase in storage and capacity.
I’m an existing Oracle customer; i’m just going to transition existing Oracle licenses over to the new machine. I already license more than enough EE and hundreds of named users, so the increase in CPU licensing cost is immaterial. Understood if this was a *new* customer suddenly looking at a huge jump in EE and other licensing … of course that being said, you can virtualize the new Exadata to lower some licensing costs (an option we’re considering to lower RAC licensing).
We’re looking at updating a database infrastructure built partially on stand alone Dell/Linux, part VMWare, using NAS. Identifying a handful of core databases that total about 75 cores, 750gb SGA and about 12TB of database space. With the new X5, I can stick the entire database into the deployed flash for all 10 dbs in a quarter rack of Exadata, quite a selling point before even talking about the 40gb internal networking or the columnar storage or the dedicated storage CPUs … all of which seem to be features that nobody else can match.
Other solutions in play: considered cloud but in our opinion RDS doesn’t come close to offering what we need. We looked at storage-only stuff from existing NAS vendor but the requirements were not solely about increasing performance. We also need clustering (RAC) and failover (Data Guard) and DB security improvements (Data Vault, ASO/TDE) that require a fully engineered solution, not just band-aids on the storage side. We issued an RFP and forcibly sent it to multiple cloud vendors and none of them bothered to respond.
We also sent the RFP to EMC: Exadata’s hardware cost for two quarter racks and everything that comes with it was only just a little more than EMC’s 20tb XtremIO proposal. No offense, since It seems you’re an EMC guy … but what I get for my dollar in Exadata solution seems so much better. What am I missing here?
@Todd : Thanks for the reply.
>”Identifying a handful of core databases that total about 75 cores, 750gb SGA and about 12TB of database space. With the new X5, I can stick the entire database into the deployed flash for all 10 dbs in a quarter rack ”
Did I understand you correctly? You’re taking databases that busy 75 cores currently and considering housing that in a quarter rack X5? What CPUs are these 75 cores in (model)? A quarter rack X5 has 72 host cores.
I wish I could see these proposals side by side:
>”We also sent the RFP to EMC: Exadata’s hardware cost for two quarter racks and everything that comes with it was only just a little more than EMC’s 20tb XtremIO proposal.”
The usage patterns here are very feast or famine; some databases are only really heavily used for a month out of the year, then sit mostly idle for the rest of the year. So, we thought there was also some room for “rightsizing” of CPU allocation. There’s no real over-lap of these busy periods too, so the though is that (through Oracle Resource Manager) we can size up/size down the various engines to match these busy periods and really over-allocate CPU during the known crush times.
Obviously cannot share RFP responses. But I am accurately depicting the solution and dollar amounts. If EMC’s solution for a 20Tb with a number of “blades” but no specs and no implementation detail was $X, then Oracle’s solution for two entire quarter racks of exadata came in at just 10% more.
@Todd:
> But I am accurately depicting the solution and dollar amounts.
Thanks for sharing. I’m not questioning your accuracy. I only said I’d like to see the RFP responses so I could line-item compare. I am confused by the following though:
“20Tb with a number of “blades” but no specs and no implementation detail”
In the intro paragraph they said they were proposing “Eight B200 M4 blades” but then never described what was in these blades. What chips, what memory, how much on-board storage (if any?). In fact, the word ‘blade” did not appear in the technical proposal. The pricing had one line item that just said it was $ figure for the VSPEX solution. No accompanying specs.
@Todd :
OK, so this is starting to make more sense. For a moment there you had us to believe that a single 20TB XtremIO was 10% less than 2 Quarter Rack X5 systems which would list price (EF model) at 2 x ( $330,000 + (8 x 3 x $20,000)) == $1,620,000 – 10% == $1,458,000.
$1,458,000 would be one really expense single brick XtremIO array.
So now it looks like there’s a proposal, with EMC storage in it, that includes 8 host systems (“Eight B200 M4 blades”). Since 2 Quarter-Rack Exadata configs would have only 4 hosts total in it I think maybe someone read the RFP wrong? Given the response has 100% more hosts I don’t think there’s a comparison.
This (and other reasons) is why I said I wish I could see the proposals side by side.
Yeah, too bad you can’t see the responses, else you’d understand where i’m coming from.
If those 8 blades had single dual core CPUs in them … well then that’s not nearly as powerful as Exadata irrespective of the number of hosts. We had no way of knowing what was in those blade servers because …. EMC didn’t offer any technical specs. If you’re a customer and you’re googling things like “default cisco blade configuration” or “what is vspex” … it doesn’t bode well for your offered solution to be picked. I’m not an EMC expert so I don’t automatically know what VSPEX is.
Besides; what’s “better” anyway; a prod solution that depends on two DB nodes clustered together with 18-core chips or four blade servers rac’d together with lesser chips? Are they quad-core hyper threaded chips? that’d be closer. Are they 12-core chips? 12*4 =48 cores. But then how is the storage attached? Is it Network attached? Is Network attached XtremIO a better solution than directly-connected storage utilizing an infiniband 40gb pipe?
To say nothing of this: at least Oracle’s proposal costed out the required software. EMC’s did not, also leaving that as an exercise to the reader. So take your cost estimate above and add in oracle software licensing and you’re pretty close to what we figured their cost was.
@Todd : The last dual-core Xeon was a very boutique WSM-EP processor (circa 2010) that none of the majors put into servers. The B200 M4 hosts you’re talking about are Haswell-EP which is googling “B200 M4”–not that you should have had to do that. No matter which CPU SKU they configured into a 8 host B200 M4 config it is way over-powered compared to 2 quarter racks X5 systems. It’s an apple/orange and I’m sorry you didn’t get a quote you could understand.
May I ask if you asked for a little more detail when you got the response showed up lacking clarity? Care to ping me in email with the name of the rep? This sounds like a broken workflow and I’d like to know more about how it ended up this way.
Suffice it to say that, based on what I now know from your comments, there is no meaningful comparison here.
For other readers I’d like to capitalize on a quote from your last comment:
“Is Network attached XtremIO a better solution than directly-connected storage utilizing an infiniband 40gb pipe?”
Folks, Exadata Storage Servers are not “directly-connected” to hosts in the Exadata Database Machine. Hosts in the Exadata Database Machine are connected via 40Gb QDR IB running a network protocol with the marketing name iDB which leverages OFED RDS (things one can read about in this book: http://wp.me/a21zc-1a4 ).
@Todd: Since I can’t see the RFP response I can’t answer the question about what the networking was in the proposal. I can guess, however, that it is a NEXUS unified fabric switch which would accommodate 8x8GFC plumbing per host to equal data flow to that of 2 active 40Gb QDR IB ports found on the hosts of the Exadata Database Machine.
I don’t see the word “Nexus” but I did find one line searching for “fabric” in the EMC response. “Two 6296 Fabric Interconnects” are apparently included in the response.
Thanks for the information. It has been quite informative for someone who doesn’t generally delve this deeply into technical specs like this.
@Todd:
You started the comment thread with “Dumb question.” I actually don’t think there is any such thing as a dumb question. My readers know I therefore never pull the old RTFM on visitors. That said, you’ve ended the thread with saying you are “someone who doesn’t generally delve this deeply into technical specs like this” but that came only after a lot of questioning of the deep technical analysis I’ve put into my post and the assertion that you have comparable RFP responses that guide your opinions of Exadata X5.
So what I’d like to know is a) do you feel I treated you fairly on this comment thread and b) does your customer have anyone involved in their project who does “delve this deeply?”
With regards to COD only working on oracle branded boxes my guess how they can do this is because on the oracle servers they can control the bios and ensure no one can turn on additional cpus… I am sure there is no generic way to assure the same, so how could they provide the same on generic hardware. If you look at the way IBM or SPARC used to do it I believe it was similar. COD was something that got controlled at the bios level and hence could only be done on proprietary boxes.Note that the boxes have to be connected to Oracles my oracle support site to generate the key files for ODA.
“they can do this is because on the oracle servers they can control the bios a”
@robinsc: Please re-read my comment thread. I posed the question to one of Oracle’s PMs. The pertinent question is why can’t one buy a Sun Server X5-2 (stand alone SKU) and attach it to, say, a JBOD storage device from Fry’s Electronics and implement CoD using the BIOS in the Sun Server X5-2? It has to be a simple, silly, Sun X5-2 stuffed into an Engineered System to qualify for this special licensing scheme and if you are a paying Oracle customer you *should* be outraged.
CoD should be supported on any system that lets you disable CPUs in the BIOS. Period. No more excuses and suppositions about Oracle’s deeper, mysterious ways of looking out for customer’s best interest. The adage has it that god works in mysterious ways. Let’s limit it to that.
Furthermore @kevinclosson, that is what Oracle LMS is for. There to help those customers who “mistakenly” use more cores and thus licenses than entitled to. “Like a good neighbor, LMS is there” 🙂
Seems to me there is some confusion all around. the extreme flash servers have two types of flash, 8 NVMe ssds which are 2.5 inch and 4 PCIe flash cards and of these are of capacity 1.6 TB. the press kit referring to the Flash Accelerator F160 PCIe Card pCIe flash cards is probably the flash cache not the 2.5 inch SSDs. however it does look like all the flash is from Intel… That being said probably the extreme flash storage serves can’t provide as much of a boost because there is a flash cache in between the database nodes and the permanent flash storage any way….
“Some confusion all around”
“That being said probably the extreme flash storage serves can’t provide as much of a boost because there is a flash cache in between the database nodes and the permanent flash storage any way….”
@robinsc: Some confusion indeed! Thanks for stopping by, but since there is *no cache in the EF models* your hypothesis is off base. Lot’s of folks presume I’m wrong because for the price Oracle must be right.
Reblogged this on Kevin Closson's Blog: Platforms, Databases and Storage and commented:
Scrutinizing Exadata X5 Datasheet IOPS Claims…and Correcting Mistakes
This discussion has been very insightful. Kevin, I wonder if you have managed to assess the X6 series released about a year ago. I would love to see if Oracle is really evolving the platform for performance or is it just a new name.