Kevin Closson's Blog: Platforms, Databases and Storage

A recent thread on the oracle-l email list was about the AIX 5L KPROC process called “wait”. The email that started the thread reads:

We are reviewing processes on our P690 machine and get the following.

I’ve googled a little bit but can’t find anything of interest. Are these processes that I should be concerned with – should we kill them? A normal ps -ef | grep 45078 does not return the process, so I really can’t figure out what these are.

$ ps auxw | head -10

USER PID %CPU %MEM SZ RSS TTY STAT STIME TIME COMMAND
root 45078 9.3 0.0 48 36 – A Oct 13 120026:37 wait
root 40980 9.0 0.0 48 36 – A Oct 13 116428:47 wait
root 36882 8.9 0.0 48 36 – A Oct 13 114010:26 wait
root 32784 8.8 0.0 48 40 – A Oct 13 113205:56 wait
[…output truncated…]

Another participant in the thread followed up with:

you will find the answer in:

http://www-304.ibm.com/jct09002c/isv/tech/faq/individual.jsp?oid=1:89156

And yet another good member of the list added:

Also, the reason you don’t see it with “ps -ef” is that ps doesn’t show kernel processes by default – you have to specify the “-k” flag, e.g.:

/opt/oracle ->ps -efk|grep wait

root 8196 0 0 Nov 11 – 720:31 wait
root 53274 0 0 Nov 11 – 3628:35 wait
root 57372 0 0 Nov 11 – 554:40 wait
root 61470 0 0 Nov 11 – 1883:24 wait
[…output truncated…]

So What Do I Have To Add?
So why am I blogging about this if the mystery has been explained? Well, I think having a kernel process attributed with time when the processor is in the idle loop is just strange. Microprocessors only have two states; running and idle. On a Unix system, the running state is attributed to either user or kernel mode. Attributing the idle state to anything is like charging nothing to something.

Yes, I suppose I’m nit-picking. There is something about the running state that I find so many people do not know and it has to do with processor efficiency. Regardless of which mode—user or kernel—the processor monitoring tools can only report that the processor was idle or not. That’s all. Processor monitoring tools (e.g., vmstat, sar, etc) cannot report processor efficiency. Remember that a processor is not always getting work done efficiently. Not that there is anything you can do about it, but a processor running in either mode accessing heavily contended memory is getting very little work done per cycle. The term CPI (cycles per instruction)  is used to represent this efficiency. Think of it this way, if a CPU accesses a memory location in cache, the instruction completes in a couple of CPU cycles. If the processor is accessing a word in a memory line that is being completely hammered by other processors (shared memory), that single instruction will stall the processor until it completes. As such, the workload is said to execute with a high CPI.

There you have it, some trivial pursuit.

What Does This Have To Do With Oracle?
Well, I’ll give you an example. A process spinning on a latch is executing the test loop in cache. The loop executes at a very, very low CPI. So if you have a lot of processes routinely spinning on latches, you have a low CPI—but that doesn’t mean you are getting any throughput. Latch contention is just tax if you will. When the latch is released, the processors that are spinning get a cacheline invalidation. They immediately read the line again. The loading of that line brings the CPI way up for a moment as the line is installed into cache, and on and on it goes. The “ownership” of the memory line with the latch structure just ping-pongs around the box. Envision a bunch of one-armed people standing around passing around a hot potato. Yep, that about covers it. No, not actually. Somewhere there has to be a copy of the potato and a race to get back to the original. Hmmm, I’ll have to work on that analogy—or take an interest in hierarchical locking. <smiley>

Therein lies the reason that just a few contended memory lines with really popular Oracle latches (e.g., redo allocation, hot chains latches, etc) can account for reasonable percentages of the work that gets done on an Oracle system. On the other hand, systems with really balanced processor/memory capabilities (e.g., System p, Opteron on Hypertransport, etc), and systems with very few processors don’t have much trouble with this stuff. And, of course, Oracle is always working to eliminate singleton latches as well.

In the blog entry about the Solaris 10.2.0.3 patchset not functioning on VxFS, I reported that Metalink says the patchset does not work on VxFS. That is true. Since the Metalink notes have not been updated, I’ll blog a bit about what I’ve found out. Note, the Metalink note says not to use the patchset because of this bug. I am not here to fight Oracle support.

It turns out that what is happening is the Solaris porting group is now using an ioctl() that is not supported on VxFS files—but not calling the ioctl(2) directly. The bug results in an error stack a bit like this:

ORA-01501: CREATE DATABASE failed
ORA-00200: control file could not be created
ORA-00202: control file: ‘/some/path/control01.ctl’
ORA-27037: unable to obtain file status
SVR4 Error: 25: Inappropriate ioctl for device

The text in bug number 5747918 is nice enough to include the output of truss when the problem happens. The ioctl() is _ION. This is the ioctl(2) that is implemented within the directio(3C) library routine. No, don’t believe this developers.sun.com webpage when it refers to directio(3C) as a system call. It isn’t. However, they do provide an example of using the directio(3C) call in this small directio(3C) test program.

The Solaris directio(3C) call is used to push direct I/O onto a file. In the demonstration of the bug (5747918), the 10.2.0.3 patchset is trying to push direct I/O onto the file descriptor held on the control file stored in VxFS. That isn’t how you get direct I/O on VxFS. I wonder if this call to directio(3C) only happens if you have filesystemio_options=DirectIO|setall. That would make sense.

Workaround
If you use ODM on VxFS, this call to directio(3C) does not occur so you wont see the problem. Thanks to a reader comment on my blog and my age old friend still at Veritas (I mean Symantec) for verification that ODM works around the problem.

A Test Program

If you create a file in a VxFS mount called “foo”, like this:

$ dd if=/dev/zero of=foo bs=4096 count=16

And then compile and run the following small program, you will see the same problem Oracle 10.2.0.3 is exhibiting. The same program on UFS should work fine.

$ cat t.c
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/fcntl.h>

main ()
{

int ret, handle;
handle = open (“./foo”, O_RDONLY);
if ((ret = directio (handle, DIRECTIO_ON)) < 0)
{
printf (“Failure : return code is: %d\n”, ret);
}
else
{
printf
(“The ioctl embedded in the directio(3C) call functions on the file.\n”);
}
} /* End */

Another Potential Workaround
If you want to test the rest of what 10.2.0.3 has to offer without ODM—even with a VxFS database—I think you should be able to explicitly set filesystemio_options=none and get around the problem. Be aware I have not tested that however. The worst thing that could happen is that setting filesystemio_options in this manner is indeed a workaround that would allow you to test the many other reasons you actually need 10.2.0.3!

If you find otherwise, please comment on the blog.

A Faulty Patchset
One of the members of the oracle-l email list just posted up that 10.2.0.3 is a non-functional patchset for Solaris SPARC 64-bit if you are using Veritas VxFS for datafiles. I just checked Metalink and found that this is Oracle Bug 5747918 covered in Metalink Note: 405825.1.

According to the email on oracle-l:

Applies to:

Oracle Server – Enterprise Edition – Version: 10.2.0.3 to 10.2.0.3
Oracle Server – Standard Edition – Version: 10.2.0.3 to 10.2.0.3
Solaris Operating System (SPARC 64-bit)
Oracle Database Server 10.2.0.3
Sun Solaris Operating System
Veritas Filesystem

Solution

Until a patch for this bug is available, or 10.2.0.3 for Solaris is re-released, you should restore from backups made before the attempted upgrade. Do not attempt to upgrade again until a fix for this issue is available. Bug:5747918 is published on Metalink if you wish to follow the progress.

I’d stay with 10.2.0.2 I think.

This is just a quick announcement to point out that I have done a little format clean-up on the blog. The blog is about clustering and other platform topics related to Oracle, but the fact of the matter is that most IT shops that care about Oracle platform (especially clustering) topics also likely deploy non-Oracle databases.

So, I am going to start posting some stuff here along those lines. Yes, the first thing I’ve just posted there is about SQL Server 2005 consolidation and scale-out (shared database concurrent) reporting with SQL Server 2005.  This page has a whitepaper that covers new SQL Server 2005 functionality which allows databases in the PolyServe Database Utility for SQL Server to scale-out for concurrent reporting on up to 16 servers in the cluster. Switching from normal OLTP mode to scale-out reporting mode does not require any replication or structural changes to the data and the mode change occurs in less than 1 minute. Switching back to normal OLTP mode is just the opposite operation and also takes less than one minute. There are no physical storage manipulations (e.g., filesystem remounting) and no server reboots involved.

Upgrading CRS to 10.2.0.3 on RHEL4 x86_64
It is quite likely I’m the last person to get around to updating my 10gR2 CRS—er, clusterware—with the 10.2.0.3 patchset. Why? Well, upgrades always break something and since 10.2.0.1 CRS was really quite stable for the specific task of node membership services (libskgxn.so), I was happy to stay with it and skip 10.2.0.2. Compared to the offal we referred to as 10.1 CRS, I have been very happy with 10gR2 CRS for the main job of CRS (which is monitoring node health). Fencing is another topic as I’ve blogged about before.

Oh, Great, He’s Blogging Screen Shots of Stuff Working Fine
Well, I can’t think of anything more boring to look at than a screen shot of a successful execution of an upgrade script. With the 10.2.0.3 upgrade it is root102.sh—the root script that OUI instructs you to execute in $ORA_CRS_HOME after it finishes such activities as copying pre-10.2.0.3 files over to ${ORA_CRS_HOME}/install/prepatch10203 and so on. So why am I blogging on a successful application of this patchset?

Knowing How Bad Something Has Failed—and Where
Often times when RAC installation and patch applications go awry—a very frequent ordeal—it is often nice to see what you should have seen at the point where it went wrong. Such clues can sometimes be helpful. It is for this reason that when I—and others in my group—write install guides for Oracle products on our Database Utility for Oracle clustering package I often include a lot of boring screen shots.

Testing a Rolling Application of 10.2.0.3 CRS
As described later in this post it is fully supported to implement a shared ORA_CRS_HOME—as it is on OCFS2 and Red Hat GFS. In fact, there are several permutations of supported configurations to choose from:

• Local CRS HOME, raw disk OCR/CSS
• Local CRS HOME, CFS OCR/CSS
• Local CRS HOME, NFS OCR/CSS
• Shared CRS HOME, raw disk OCR/CSS
• Shared CRS HOME, CFS OCR/CSS
• Shared CRS HOME, NFS OCR/CSS

As a normal part of my testing, I wanted to make sure the storing the OCR and CSS disks on the PolyServe CFS in no way impacts the ability to perform a 10.2.0.3 rolling upgrade of local ORA_CRS_HOME installations. It doesn’t. First, OUI determined is was OK for me to do so because ORA_CRS_HOME on all three nodes of this puny little cluster were installed under /opt on the internal drives. The CRS files (e.g., OCR/CSS), on the other hand, were on PolyServe:

tmr6s15:/opt/oracle/crs/install # grep u02 *
paramfile.crs:CRS_OCR_LOCATIONS=/u02/crs/ocr.dbf
paramfile.crs:CRS_VOTING_DISKS=/u02/crs/css1.dbf,/u02/crs/css2.dbf,/u02/crs/css3.dbf
rootconfig:CRS_OCR_LOCATIONS=/u02/crs/ocr.dbf
rootconfig:CRS_VOTING_DISKS=/u02/crs/css1.dbf,/u02/crs/css2.dbf,/u02/crs/css3.dbf
tmr6s15:/opt/oracle/crs/install # mount | grep u02
/dev/psd/psd1p3 on /u02 type psfs (rw,dboptimize,shared,data=ordered)

The first screen shot shows what to expect when OUI determines a rolling application of this patch is allowed:

NOTE: You may have to right click->view the image (e.g., with firefox I believe)

Next, OUI instructs you to stop CRS on a node and then execute the root102.sh script:

If all that goes well, you’ll see the following sort of feedback as root102.sh does its work:

I was able to move along to the other two nodes and get the same feedback from root102.sh there as well.

To Share or Not to Share ORA_CRS_HOME
Oracle and PolyServe fully support the installation of CRS in either shared or unshared filesystems. The choice is up to the administrator. There are important factors to consider when making this decision. Using a shared ORA_CRS_HOME facilitates a single, central location for maintenance and operations such as log monitoring and so on. Some administrators consider this a crucial factor on larger clusters; it eliminates the need to monitor large numbers of ORA_CRS_HOME locations, each requiring logging into a different server. When ORA_CRS_HOME is shared in the PolyServe cluster filesystem, administrators can access the files from any node in the cluster.

A shared ORA_CRS_HOME does have one important disadvantage—rolling patch application is not supported. However, a patch that manipulates the Oracle Cluster Repository cannot be applied in a rolling fashion anyway. Although 10.2.0.3 is not that such a patch, it is not inconceivable that other upgrades could make format changes to the OCR that that would be incompatible with the prior-versions executing on other nodes. Oracle would, of course, inform you that such a release was not a candidate for rolling upgrade just as they do with a good number of the Critical Patch Updates (CPU).
The parallel to shared ORACLE_HOME is apparent. Many Oracle patches for the database require updates to the data dictionary, so a lot of administrators ignore the exaggerated messaging from Oracle Corporation regarding “Rolling Upgrades” of ORACLE_HOME and deploy a shared ORACLE_HOME, eliminating the need to patch several ORACLE_HOME locations whenever a patch is required. This concern is only obvious to large IT shops where there is not just one RAC database, but perhaps 10 or more. These same administrators generally apply this logic to ORA_CRS_HOME. Indeed, having only one location to patch in either the ORA_CRS_HOME or ORACLE_HOME case significantly reduces the time it takes to apply a patch. To that end, planning a very brief outage to apply patches to shared ORA_CRS_HOME and/or ORACLE_HOME for up to 16 nodes in a cluster is an acceptable situation for many applications. For those cases where downtime cannot be tolerated, Oracle Data Guard is required anyway and again the question of shared or unshared ORACLE_HOME and ORA_CRS_HOME arises. The question can only be answered on a per-application basis and the choice is yours. PolyServe finds that, in general, when an application is migrated from a single large UNIX platform to RAC on Linux, administrators do not have sufficient time to deal with the increased amount of software maintenance. These IT shops generally opt for the “single system feel” that shared software installs for ORACLE_HOME and ORA_CRS_HOME offers. In fact, PolyServe customers have used Share Oracle Home since 2002 with Oracle9i and then with Oracle10g—it has always been a staple feature of the Database Utility for Oracle. With Oracle10g the choice is yours.

Not to be confused with the x86 ISA CPUID instruction (which serialized the CPU by the way), I found a nice little tool for in-depth CPU information called cpuid(1).I’ve snipped a bit of the manpage and pasted it below. The RPM for the cpuid(1) tool can be found here.

Let’s take a quick look at the contrast between what this tool reports and what is generically available if you can /proc/cpuinfo. Once again, I’ll go over to my favorite lab cluster of DL585s fit with Opteron 850s running the PolyServe Database Utility for Oracle. I’ll use more(1) to get one processor worth of information:

$ cat /proc/cpuinfo | more
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 33
model name : AMD Opteron ™ Processor 850
stepping : 0
cpu MHz : 1800.005
cache size : 1024 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse
sse2 ht syscall nx mmxext lm 3dnowext 3dnow pni
bogomips : 3599.35
TLB size : 1088 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

On the other hand, the cpuid(1) command shows:

$ cpuid | more

CPU 0:
vendor_id = “AuthenticAMD”
version information (1/eax):
processor type = primary processor (0)
family = Intel Pentium 4/Pentium D/Pentium Extreme Edition/Celeron/Xeon/Xeon MP/Itanium2, AMD At
hlon 64/Athlon XP-M/Opteron/Sempron/Turion (15)
model = 0x1 (1)
stepping id = 0x0 (0)
extended family = 0x0 (0)
extended model = 0x2 (2)
(simple synth) = AMD Dual Core Opteron (Italy/Egypt JH-E1), 940-pin, 90nm
miscellaneous (1/ebx):
process local APIC physical ID = 0x0 (0)
cpu count = 0x2 (2)
CLFLUSH line size = 0x8 (8)
brand index = 0x0 (0)
brand id = 0x00 (0): unknown
feature information (1/edx):
x87 FPU on chip = true
virtual-8086 mode enhancement = true
debugging extensions = true
page size extensions = true
time stamp counter = true
RDMSR and WRMSR support = true
physical address extensions = true
machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSENTER and SYSEXIT = true
memory type range registers = true
PTE global bit = true
machine check architecture = true
conditional move/compare instruction = true
page attribute table = true
page size extension = true
processor serial number = false
CLFLUSH instruction = true
debug store = false
thermal monitor and clock ctrl = false
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
SSE2 extensions = true
self snoop = false
hyper-threading / multi-core supported = true
therm. monitor = false
IA64 = false
pending break event = false
feature information (1/ecx):
PNI/SSE3: Prescott New Instructions = true
MONITOR/MWAIT = false
CPL-qualified debug store = false
VMX: virtual machine extensions = false
Enhanced Intel SpeedStep Technology = false
thermal monitor 2 = false
context ID: adaptive or shared L1 data = false
cmpxchg16b available = false
xTPR disable = false
extended processor signature (0x80000001/eax):
generation = AMD Athlon 64/Opteron/Sempron/Turion (15)
model = 0x1 (1)
stepping = 0x0 (0)
(simple synth) = AMD Dual Core Opteron (Italy/Egypt JH-E1), 940-pin, 90nm
extended feature flags (0x80000001/edx):
x87 FPU on chip = true
virtual-8086 mode enhancement = true
debugging extensions = true
page size extensions = true
time stamp counter = true
RDMSR and WRMSR support = true
physical address extensions = true
machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSCALL and SYSRET instructions = true
memory type range registers = true
global paging extension = true
machine check architecture = true
conditional move/compare instruction = true
page attribute table = true
page size extension = true
multiprocessing capable = false
no-execute page protection = true
AMD multimedia instruction extensions = true
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
RDTSCP = false
long mode (AA-64) = true
3DNow! instruction extensions = true
3DNow! instructions = true
extended brand id = 0xe86 (3718):
MSB = reserved (0b111010)
NN = 0x6 (6)
AMD feature flags (0x80000001/ecx):
LAHF/SAHF supported in 64-bit mode = false
CMP Legacy = true
SVM: secure virtual machine = false
AltMovCr8 = false
brand = “AMD Opteron ™ Processor 850”
L1 TLB/cache information: 2M/4M pages & L1 TLB (0x80000005/eax):
instruction # entries = 0x8 (8)
instruction associativity = 0xff (255)
data # entries = 0x8 (8)
data associativity = 0xff (255)
L1 TLB/cache information: 4K pages & L1 TLB (0x80000005/ebx):
instruction # entries = 0x20 (32)
instruction associativity = 0xff (255)
data # entries = 0x20 (32)
data associativity = 0xff (255)
L1 data cache information (0x80000005/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x2 (2)
size (Kb) = 0x40 (64)
L1 instruction cache information (0x80000005/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x2 (2)
size (Kb) = 0x40 (64)
L2 TLB/cache information: 2M/4M pages & L2 TLB (0x80000006/eax):
instruction # entries = 0x0 (0)
instruction associativity = L2 off (0)
data # entries = 0x0 (0)
data associativity = L2 off (0)
L2 TLB/cache information: 4K pages & L2 TLB (0x80000006/ebx):
instruction # entries = 0x200 (512)
instruction associativity = 4-way (4)
data # entries = 0x200 (512)
data associativity = 4-way (4)
L2 unified cache information (0x80000006/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 16-way (8)
size (Kb) = 0x400 (1024)
Advanced Power Management Features (0x80000007/edx):
temperature sensing diode = 0x1 (1)
frequency ID (FID) control = 0x1 (1)
voltage ID (VID) control = 0x1 (1)
thermal trip (TTP) = 0x1 (1)
thermal monitor (TM) = 0x0 (0)
software thermal control (STC) = 0x0 (0)
TscInvariant = 0x0 (0)
Physical Address and Linear Address Size (0x80000008/eax):
maximum physical address = 0x28 (40)
maximum linear address = 0x30 (48)
Logical CPU cores (0x80000008/ecx):
number of logical CPU cores – 1 = 0x1 (1)
ApicIdCoreIdSize = 0x0 (0)
SVM Secure Virtual Machine (0x8000000a/eax):
SvmRev: SVM revision = 0x0 (0)
SVM Secure Virtual Machine (0x8000000a/edx):
LBR virtualization = false
NASID: number of address space identifiers = 0x0 (0):
(multi-processing synth): multi-core (c=2)
(synth) = AMD Dual Core Opteron (Italy/Egypt JH-E1), 940-pin, 90nm Processor 875

And the manpage:

CPUID(1)
NAME

cpuid – Dump CPUID information for each CPU

SYNOPSIS

cpuid [options…]

DESCRIPTION

cpuiddumpsdetailedinformationabouttheCPU(s) gathered from the CPUID instruction, and also determines the exact model of CPU(s) from that information.

It dumps all information available from the CPUID instruction.The exact collectionofinformation availablevariesbetweenmanufacturers and even between different CPUs from a single manufacturer.

The following information is available consistently on all modern CPUs:

vendor_id

version information (1/eax)

miscellaneous (1/ebx)

feature information (1/ecx)

When it comes to storage protocols, the big storage vendors are sending a clear message: Some is good, more must be better!

NAS, CAS, What a Mess (That Almost Rhymes)
Yes, Oracle cares about Oracle over NFS, and clustered storage is taking off, but the clustered storage offerings are fracturing into structured versus unstructured data optimization and that is a bad choice to have to make.

Back in June 2006, Tony Asaro of the Enterprise Storage Group covered clustered storage in this SearchStorage.com article. He said:

Clustered storage is gaining ground with an increasing number of vendors and systems in today’s market. Over time, clustered storage will be a requisite architectural design element for all storage systems.

The article covers a lot of different clustered storage offerings running the gamut from products like Isilon to CAS technology such as the EMC Centera and mention of PolyServe oddly listed alongside 3Par who partners with PolyServe for scalable NAS. One particular quote in the article stands out:

The Isilon IQ NAS storage system is one of the best examples of a true storage cluster.

Special Purpose Storage
While this may be true, I want to blog about a very important issue that I see arising out of the clustered storage wars. You see, so many of these interesting technologies are very special purpose. Some do streaming media well, others do seismic, others do RDBMS, but few—if not only one—do it all. Deploying special purpose storage technology means you are certain to have more than one kind of storage. For instance, if you adopt EMC Centera for unstructured data, you are going to need some other solution for your structured data—and since this is an Oracle blog we’ll presume Oracle is charged with your structured data.

Centera Storage is Optimized for Databases
“Hold it, Kevin”, you say. I can hear it already. I know, you read this EMC solution brief covering Centera posted on Oracle’s website! It says (emphasis added by me):

[…] Centera’s unique storage capabilities, you can centralize and manage massive volumes of information generated by all aspects of your organization […]

A document on Oracle’s website states Centera handles information generated by all aspects of your organization. Certainly that must also include the things you cram into your ERP database! No, CAS is an EMC term for write-once, or in their terminology “fixed content.” In short they implement WORM on ordinary magnetic media. Centera is not for databases.

So Oracle and EMC both recommend Centera for some of your data. How many different types of storage presentation do you want? What do you do with your database then? Oh, of course, I know, ASM. Centera is a network attached storage device so if you are settling on IP, wouldn’t life be simpler with NAS for the database too? But as I pointed out in this blog entry about ASM over NFS, EMC specifically recommends against combining ASM and NFS. So how many different connectivity models do you want? See, what I don’t get is how the market tolerates having products marketed to them in a way that doesn’t have their best interests in mind. It suits EMC quite well to sell you some Centera and some Celerra for NFS or even a mix of Centera and DMX via FCP (FCP is expensive). Any storage vendor that pushes Content DB will get a head nod from Oracle, but in the end, Content DB runs on all major platforms. So who are the forces behind this drive towards such special purpose and fractured storage management architectures?

Unlike Isilon and EMC file serving, with PolyServe you can buy any commodity hardware. And unlike Isilon and EMC, you can choose Windows Server or Linux—no proprietary embedded operating system. And most importantly—unlike Isilon and EMC—with PolyServe you get general purpose network attached clustered storage. So, sure, do your Content DB and Oracle Database (RAC included) all in one management infrastructure. Makes sense to me, but of course I’m biased.

Isilon: The Best Example of a True Storage Cluster
Yes, Isilon is a true clustered storage, but the product doesn’t support the Oracle database. Yet another special purpose offering. But, as I said here, I wish Isilon well. We are, after all, kindred spirits in this clustered storage wave. 

OK, there, I shamelessly plugged the outfit I work for <smiley face>.

The Browser Wars Rage On
While reading the latest Time Magazine about how you are the person of the year, I stumbled across some interesting stuff. In this Time Magazine article, we learn that Blake Ross is “Outfoxing Microsoft” with the Firefox web brower. Are there really any living human beings left that care about the “browser wars”? I thought it was all about content now. Oh, Well.

Near the beginning of the article, we get this jewel regarding how most software is developed:

Most software is developed exactly the way you think it is: you pay a bunch of geeks in cubicles to write it

Lovely. On the contrary when referring to some of the people that write open source software, the article quotes Blake:

[open source developers] aren’t necessarily professionals

But no worries, when it comes to the commitment level of open source developers, the article quotes Blake as follows:

It also means the people are truly dedicated because there’s no payday

Uh, OK, that’s really nifty. I don’t know about you, but I’m a lot happier with software developed by people that do it because they need to meet their financial obligations. The thought of my local 911 service running on software written by ueber-dedicated, unpaid not-necessarily-professionals makes me restless. Think about it, they might actually have to attend to their day job at some point, or is that where they are getting the best of “their ideas?”

Oh the Hypocrisy!
I used Firefox to post this blog entry. You know what I would have used of Firefox wasn’t free? IE6—I wouldn’t pay for Firefox. When I installed Firefox, there is a welcome to Firefox page that reads:

Experience the difference. Firefox is developed and supported by Mozilla, a global community working together to make the Web a better place for everyone.

I don’t think whoever wins the nonexistent browser wars can make the Web a better place for everyone. It’s not the browser, it’s the content.

What’s This Have to do with Oracle
Oracle is not open source. I’m glad there are those “geeks in cubicles” developing and maintaining the database server. I know a lot of them, and they deserve a lot of respect.

Yes, this is an Oracle-related blog, but most Oracle sites have file serving requirements and the majority have Windows infrastructure as well. This is just an invitation to you readers that might be interested:

PolyServe Windows Scalable File Serving Web Demo Announcement