According to this techweb article, one of Oracle’s presidents “knows CIOs.” The article didn’t spend much time substantiating that notion but the title roped me in. I did read the entire article which I feel entitles me to blog a bit.
First, I’ll simply quote a quote that the article attributes to Oracle’ Mark Hurd:
Hurd reiterated Oracle’s claim that the highly tuned Exadata hardware-software combo yields 70x performance improvements–reports that took 70 minutes now take one minute
Sure, Exadata can easily be 70x faster than some other system. For instance, the “70-minute system” might have been a 2-socket Harpertown Xeon-based server. That would be about 1/70th Exadata’s database grid–from a CPU perspective. Or, perhaps, the Exadata 70x edge on these “reports” came from improved I/O. In that case, perhaps the 70-minute system was attached to storage that provided about 1GB/s (e.g., a low-end storage array that suffered controller saturation at 1GB/s). That would be about 1/70th the storage bandwidth of a full-rack Exadata configuration. But that all seems unlikely. It is much more likely that someone took the time to tune the query plans used by the “report” in which case the storage and CPU doesn’t really factor as heavily.
Certainly the I/O power of Exadata was not the 70x ingredient. Allow me to explain. If the “report” actually touched the same amount of data in the Exadata case the total data visited would have been about 5 terabytes and nobody runs “reports” that perform nearly 5 TB of disk I/O. We are talking about Oracle database after all and therefore the 70-minute system would have had indexes, partitioning and a other such I/O-elimination features available to it. Visiting nearly 5TB of data after I/O elimination (e.g., partition elimination, indexes, etc) is unlikely. Unless the query plan was non-optimal (likely). But I’m not blogging about that.
The article continues to quote Mark Hurd:
The customer who says it cost me $7 million to do that job before, you can literally take 70x off that and it costs him $100,000
That’s weird math and I’m simply not going to blog about that.
Finally, the article quotes Mark Hurd’s take on Big Data:
Well, it’s a tough world, man. When I grew up in this industry, there were IBM 360s, DEC VAXs, Data Generals–all that kind of stuff. And this [pointing to his iPhone] is a VAX. The power in this thing is like a VAX.
Alright, so that is the quote I’m blogging about. The VAX family of products spanned many generations. However, if one mentions IBM 360 and VAX in the same sentence we can safely presume the VAX in mind is of the printed circuit board (PCB) era. While I’m personally not quoted in press articles as “knowing CIOs”, I do know trivia. DEC VAX products of the PCB era were 1MIPS machines. I cannot impress upon you how terribly disappointed you’d be just waiting for an iPhone application to start up on a 1MIPS system.
No, I can’t go about saying I “know CIOs” but I do know that the processor in my smart phone—a Qualcomm Snapdragon—is a 2100 MIPS processor.
Yes, sadly, 2100x is all I’m blogging about.
Kevin Closson's Blog: Platforms, Databases and Storage 
I’m always in awe that in 1990 I was working with a 500MB Oracle DB that everyone considered “very large”… On a VAX, as well.
Now, my ipod has 160GB capacity and I consider a 4TB Oracle db doing 8TB of I/O per day a “small” db.
Unreal!
Hi Noons,
I agree. I remember my first terabyte Oracle (mirrored) database had over 2,000 drives and 30 CPUs to whack it silly. The reason I bothered with this blog post is I recently upgraded to a smart phone with a Snapdragon processor and was doing a little head-math to realize it is about 25x more powerful than the that 2,000 drive system I just spoke of. Now a real parlor trick would be to watch someone attempt the plumbing of 2,000 winchester drives into my smartphone 🙂
In the last 4 years, I’ve upgraded from carrying a USB disk of 100GB in my bag to a 2TB USB3 monster that cost me less than the first one!
In another 10 years the bottleneck is sooooo going to be the piping between the storage and the processing units…
Going back to my “no Moore” series, I still reckon the solution is going to have to be to move the filtering and grouping logic to the storage itself: there is no way we can physically afford to continue to move enormous amounts of disk block data to memory just so that we can total one column or count the rows!
Microsoft’s Jim Gray was spot-on in 2003 when he presented the “Personal Petabyte” talk.
Big data indeed! If only the CIOs knew what they talking about…
Hi Noons,
I don’t think payload reduction is a sufficiently generic approach to the problem of I/O to CPU disparity. In fact, I see the opposite. I see a future with more I/O-related componentry on processor die (i.e., PCI on Sandy Bridge LGA-2011 die) and gargantuan RAM solutions tiered down to solid state. And when I say solid state I’m talking about scalable, modular, redundant and huge solid state.
I think the specific topic you called out (block->column) is already solved with column orientation (Greenplum has that) but, sure, there are certainly opportunities for improved efficiency in dealing with columnar orientation. When the orientation is rotated from horizontal to vertical there is efficiency lost if the code still treats the data as a long stream of narrow tuples.
I guess what I’m saying is that I think the next 10 years will still have us swamping the CPUs with whatever the state of the art I/O plumbing so happens to be. This is even more likely when we consider the cost of licensing processors to run enterprise RDBMS software. For example, it costs a very, very little money to plumb a 16GFC (1.6GB/s) data flow into top-bin Xeon CPUs. However, it costs huge piles of money to license enough cores to deal with 1.6GB/s data flow.
Let it go Kevin.
Hey Luis,
Long time no speak… hard to let go of fond memories (PCB systems) 🙂 I have fond memories of working on systems like CDC Cyber 180 (NOS) and Honeywell DPS (GECOS) but they are certainly not as powerful as my smart phone. Now I just need to find a CIO that cares about such things 🙂
Yes, I see your point there: licensing per cpu is a big problem at the moment. I’d still like to see what would happen if someone figured out how to slap a CPU on every disk with enough memory to run a micro-RDBMS interpreter, Hardware-wise, it wouldn’t be difficult in this day and age of computer-in-USB-key. It’d mean a lot of changes to the RDBMS itself, but such a thing would be sending only final results to the central CPU/mem for further processing. Instead of heaps of extra clutter to be filtered off. But as you pointed out: with the current Oracle licensing model, it’d likely send any small to medium sized country bankrupt! Maybe the free software guys will have a say in that…
Thanks for the interesting chat, I appreciate bouncing ideas off you every once in a while! 😉
The sandpit system is nearly ready. I’m testing it with orion first, but once you have the sql stuff ready, let me know. The email is the same as before.
Hi Noons,
You’ll be bouncing ideas off me as my only AIX beta tester of YAIK (Yet another IOPS Kit) 🙂 I’m on vacation at the moment but aim to get the kit at you asap upon my return.
As for computers close to disk, uh, now you’re talking… I’m very excited about SoC (Calxeda,etc). Although RAC is ridiculously scalable for DSS it is unlikely you’d ever see it running on hundreds of nodes given the licesning model. I do know for a fact that hundreds of nodes is not an impossibility (the opposite in fact) so I hope readers don’t bother telling me what I already know about Oracle Parallel Server. Did you see my recent post about that :
https://kevinclosson.wordpress.com/2011/11/01/very-cool-yet-very-dense-low-wattage-hp-technology-in-project-moonshot-by-the-way-a-large-cpu-count-requires-a-symmetrical-architecture/
Yup, saw that one. And it stumped me: waaay out there for my level of knowledge at the moment! I can see the shared nothing vs shared everything point in a CPU + separate storage perspective. But I’m still trying to get my head around all the consequences of something like cpu-on-disk ! Anyways: have a great hol. I won’t be taking time off during Xmas so YAIK can come anytime.
“I can see the shared nothing vs shared everything point in a CPU + separate storage perspective.”
…actually, I don’t fester about with the shared-disk versus shared nothing as I really don’t think it matters. It’s true that Real Application Clusters requires shared disk but that is not a scalability hindrance–so long as one works out the storage bandwidth requirements–a task that is not all that difficult with modern storage networking options. So long as ample I/O flow is plumbed into RAC it scales DW/BI workloads. It is as simple as that. On the other hand, what doesn’t scale is asymmetry. Asymmetry has never scaled as would be obvious to even the casual observer. As long as all code can run on all CPUs (symmetry) scalability is within reach. What I’m saying is that RAC actually has better scalability characteristics when running with conventional storage than with Exadata! That’s a preposterous statement to the folks who don’t actually know the technology, as well as those who are dishonest about the technology, but obvious to the rest of us. It’s simple computer science. One cannot take the code path of query processing, chop it off at the knees (filtration/projection) and offload that to some arbitrary percentage of your CPU assets and pigeon-hole all the rest of the code to the remaining CPUs and cross fingers.
A query cannot be equally CPU-intensive in all query code all the time. There is natural ebb and tide. If the query plan is at the point of intensive join processing it is not beneficial to have over fifty percent of the CPUs in the rack unable to process join code (as is the case with Exadata).
To address this sort of ebb/tide imbalance Oracle has “released” a “feature” referred to as “passthrough” where Exadata cells stop doing their value-add (filtration and HCC decompression) for up to about 40% of the data flowing off storage when cells get too busy (CPU-wise). At that point they just send unfiltered, compressed data to the RAC grid. The RAC grid, unfortunately, has less CPU cores than the storage grid and has brutally CPU-intensive work of its own to do (table join, sort, agg). “Passthrough” is discussed in the Expert Oracle Exadata (Apress) book.
This passthrough feature does allow water to find its level, as it were. When Exadata falls back to passthrough mode the whole configuration does indeed utilize all CPU and since idle CPU doesn’t do well to increase query processing performance this is a good thing. However, if Exadata cells stop doing the “Secret Sauce” (a.k.a., Offload Processing) when they get busy then why not just build a really large database grid (e.g., with the CPU count of all servers in an Exadata rack) and feed it with conventional storage? That way all CPU power is “in the right place” all the time. Well, the answer to that is clearly RAC licensing. Very few folks can afford to license enough cores to run a large enough RAC grid to make any of this matter. Instead they divert some monies that could go for a bigger database grid into “intelligent storage” and hope for the best.
It is beneficial from the licensing perspective to run on Exadata as the processing is moved from high cost grid core (47k+23k) down to the storage. So, why would I not want to do this again? As you indicate in your last paragraph above that very few folks can afford this. May be I missed something here.
“run on Exadata as the processing is moved from high cost grid core (47k+23k) down to the storage”
Your statement leads me to believe one of two things. Either 1) you don’t have an Exadata system or 2) you do have an Exadata system but you don’t monitor the processor utilization on the Storage CPUS. I understand the principle of what you (actually Oracle marketing) suggest but in practice the principle falls short. The most processer-intensive RDBMS code is **not** offloaded to storage CPUs (table join/sort/agg, compression data, converting ASCII to internal data types when loading data). The cost of the Exadata Storage Servers makes it difficult to amortize out the value of offloading physical I/O, filtration and projection (all minuscule CPU cost).
Oracle customers would do much better to negotiate more aggressively to get down the cost of Oracle Database (and RAC if needed) and fund those price-reduced licenses with the monies that would have gone to very expensive Exadata storage. Use the remainder to buy modern, convention storage and storage connectivity and take what left (yes, there’s still be more) and hand out big bonuses to the DBA staff.
May I ask if whether my perception is right? Is it true that you don’t monitor active Exadata Storage Server cells’ CPU utilization during such processing as complex queries? The RAC grid ***always*** bottlenecks long before the CPUs in the storage grid (of Exadata) even start to warm up. That is the weakness of an asymmetrical MPP.
“The RAC grid ***always*** bottlenecks long before the CPUs in the storage grid”
There is “Never say never” principle… I think it applies well to “always”. Eh? 😉
Yeah, you’re probably right, Alex. In fact, you are right. If data is deeply compressed and the SQL is moderately selective the cost of inflating (decompression) in Exadata does often saturate storage at which point cells start shipping up to 40% of their HCC compression units to the database grid in compressed form. Of course that’s a problem since high selectivity (thus the higher cell payload) means more intense RDBMS processing and there is less CPU in the database grid of Exadata than in storage…. ’round and ’round it goes.