Since the release of Oracle Database 11g I have made a few posts about Oracle NUMA awareness and the _enable NUMA_support parameter. There is an index of most of those posts here.
This is a really short blog entry about a little-known fact about the Oracle Database 11g Release 2 (188.8.131.52) default value for the _enable_NUMA_support initialization parameter in the Linux x86_64 port. The following is the if-then-else logic. There aren’t many initialization parameters (that I know of) that have so much logic around the default assignment.
At instance boot time, the booting foreground process performs “discovery” to see if there are Exadata Storage Servers available. If you strace instance startup you’ll see the following:
open(“/etc/oracle/cell/network-config/cellinit.ora”, O_RDONLY) = -1 ENOENT (No such file or directory)
Also at boot time the numa libraries are dynamically linked and API calls are used to determine how many NUMA nodes there are. If there are more than 4 NUMA nodes and Exadata Storage is discovered the _enable_NUMA_support parameter is set to TRUE.
I have systems that attach to both Exadata and NFS storage at the same time. I have databases that reside entirely in both storage types as well. For maintenance reasons I needed to sever away the Exadata storage. That’s why the above discovery call suffered ENOENT. That changed my default setting for _enable_NUMA_support and in doing so my performance numbers changed dramatically because I was not explicitly setting _enable_NUMA_support = TRUE on the Sun x4800 system I was testing. The Sun x4800 is 8-socket Nehalem EX and 8-socket EX is something you don’t want to do without Oracle NUMA awareness. Well, at least not if the instance will be running on all processors.
I doubt any of you would ever run into this, but I thought it was worth a blog entry.