Manly Men Only Deploy Oracle with Fibre Channel – Part VII. A Very Helpful Step-by-Step RAC Install Guide for NFS

Tim Hall has stepped up to the plate to document a step-by-step recipe for setting up Oracle10g RAC on NFS mounts. In Tim’s blog entry, he points out that for testing and training purposes it is true that you can simply export some Ext3 filesystem from a Linux server and use it for all things Oracle. Tim only had 2 systems, so what he did was use one of the servers as the NFS server. The NFS server exported a filesystem and both the servers mounted the filesystem. In this model, you have 2 NFS clients and one is acting as both an NFS client and an NFS server.

This is the link to Tim’s excellent step-by-step guide.

How Simple

If you’ve ever had a difficult time getting RAC going, I think you’d be more than happy with how simple it is with NFS and using Tim’s guide and a couple of low-end test servers would prove that out.

Recently I blogged about the fact that most RAC difficulties are in fact storage difficulties. That is not the case with NFS/NAS.

Thanks Tim!

22 Responses to “Manly Men Only Deploy Oracle with Fibre Channel – Part VII. A Very Helpful Step-by-Step RAC Install Guide for NFS”


  1. 1 Rob Johnson July 12, 2007 at 6:38 pm

    I have a Manly Man question for you. This Manly Man Wanna Be (MMWB) runs a 2-node 10g RAC on Dell 2850s with 2 dual-core Xeon CPUs (total of 4 CPUs). Each server has 16 GB of memory. While MMWB was installing this last year, he struggled mightily with 64-bit RAC on 64-bit Red Hat Linux 4.0. MMWB finally got it working after learning a lot of things about RPMs and such.

    However, Boss Of Manly Man Wanna Be (BOMMWB) was nervous about 64-bit being “new,” and all of the difficulties that MMWB had with it, so we reinstalled with 32-bit RAC running on 32-bit Red Hat Linux 4.0.

    BOMMWB was happy, but this leaves MMWB wondering, is he truly a Manly Man if he’s running x86 Oracle on hardware that’s x86-64 capable? Is 16 GB of memory manly enough? We do use a SAN, does that make me a Manly Man?

    Seriously, what are your thoughts about x86 vs. x86-64 Linux, in relation to Oracle and RAC? I’d appreciate a blog entry if you could.

  2. 2 kevinclosson July 12, 2007 at 7:03 pm

    Hi Rob, or should I say MMWB? I love the tongue-in-cheekedness! Odd you should start this thread since I was just about to make a 32bit/64bit post…

  3. 3 Pedro Alvarez July 13, 2007 at 3:45 am

    Kevin,

    How can NFS substitute the functionality that a cluster file system offers? I am confused about this whole nfs with rac. If two instances are trying to write a file, how can nfs protect?

    Thanks.

  4. 4 kevinclosson July 13, 2007 at 5:40 am

    Pedro,

    A CFS would be in as much of the way as NFS if Oracle were to use either without Direct I/O. However, RAC uses Direct I/O with both. It is a non-issue.

  5. 5 uday shet July 13, 2007 at 6:16 am

    Kevin,
    First – let me thank you for maintaining this excellent blog.
    I was thinking about NFSv4, now that that’s pretty much supported by almost all OS vendors. Have you had an opportunity to explore NFSv4 and Oracle RAC? If so then what impact do you think it’ll have on O10g RAC customers wanting to migrate to V4?

    thanks,
    Uday Shet

  6. 6 Tim Hall July 13, 2007 at 8:47 am

    Rob Johnson: I would have thought it was obvious. The number 64 is twice the size of 32, so it must be twice as good, making you twice as manly! 😉

    Cheers

    Tim…

    PS. I think politically correct names for this series of blog entries are:

    Personally Persons only deploy…
    Humanly Humans only deploy…

    🙂

  7. 7 kevinclosson July 13, 2007 at 3:14 pm

    I’m surprised nobody has brought up “That Manly Feeling” that Irish Spring brings on! But then who would have seen those commercials? Time to blog it.

  8. 8 Philip Papadopoulos July 13, 2007 at 5:48 pm

    Does Manly Man drive an old landrover? Is it a requirement?

    Keep up the great blogging about RAC and NFS. I will be trying it out in August.

  9. 9 kevinclosson July 13, 2007 at 5:55 pm

    Philip! If you are talking about my avatar, that is actually a 1940s Willey Jeep!

  10. 10 kevinclosson July 13, 2007 at 10:55 pm

    Uday,

    I have not explored NFSV4. Since Oracle Direct NFS is here, I’m not sure there would be much value…but I don’t know.

  11. 11 Tim Hall July 17, 2007 at 7:49 am

    Hi.

    What do you set your filesystemio_options init.ora parameter to when using Oracle over NFS?

    Based on what you’ve written before I know NFS supports direct I/O, but I’m struggling to find a specific statement about NFS and asynchronous I/O. So should I use:

    filesystemio_options=directIO

    or

    filesystemio_options=setall

    I guess “setall” is fine because Oracle only uses direct and/or synchronous IO if it can, but it would be nice to get some confirmation.

    Cheers

    Tim…

  12. 13 cristiancudizio July 23, 2007 at 12:55 pm

    i’m confused as Pedro Alvarez in is previous question about CFS and NFS, if Oracle uses direct i/o RAC can work also with shared datafiles on a non-cluster filesystem if this supports direct i/o? Is Oracle RAC that coordinates writes by different instances to same datafiles?

    Thanks,
    Cristian

  13. 14 kevinclosson July 23, 2007 at 2:59 pm

    Pedro, Christian,

    NFS is not full filesystem so it is neither a cluster filesystem nor a non-cluster filesystem. I is cluster-able in that you can mount the same NFS export on more than one system. Try not to make it more complex than it is, OK.

  14. 15 Mike Smith December 2, 2007 at 6:01 pm

    Hi Kevin, great blog!

    I have been combing over Tim’s Blog as well as some of the the details you provide here. Very informative.

    While Tim’s guide is very good, I am trying to extrapolate a bit and apply his solution to a single instance to RAC migration (of an existing DB whose data must remain pssst…it’s a secret E-Business suite thing for a development project I am working on).

    I believe that I am running into an issue related to the data file storage location of the NFS mount and it being an EXT3 file system and NOT ASM or OCFS but I am uncertain.

    rconfig pukes everytime I try to convert 10gR2 (ORA-1034 & ORA-1078). I logged a bug with an am awaiting a reply – 6631213).

    I have tried manual conversion as well which fails with ORA-29702: error occurred in Cluster Group Service operation after I relinked the oracle with rac_on.

    It’s an interesting project I am working on and I would really like to make this work. Time frame is an issue and after trying one final time to see if my NAS mount can handle a standard new RAC install and config via dbca…if it can’t even handle that, I know something is wrong with my config.

    CRS runs and OCR and voting disks are good. But the RAC db migration fails at every turn.

    Any thoughts?

    Thanks,

    Mike

  15. 16 kevinclosson December 2, 2007 at 9:14 pm

    Hi Mike,

    Thanks for stopping by…

    You are using 10gR2 but no more specifics than that. With 10gR2 you’ll need to inform us of what the NAS device is, and what your mount options are. Actually, it is not clear what the NFS client (DB server) platform is either. Also, what is this type of migration? RMAN? I’ve heard rumors of 10gR2 RMAN NFS peculiarities so I need to know that …

  16. 17 Mike Smith December 2, 2007 at 9:32 pm

    Hey Kevin, thanks for the prompt reply.

    Sorry for being vague…wasn’t intentional. My bad.

    Openfiler 2.2 is the NAS.

    Oracle’s EL 4U5 is the OS for the two “clients” or rac nodes if you prefer.

    The mount options I am using are as follows:

    openfiler:/mnt/nfsrac_data/crs_ocr_voting/crs_configuration
    /u01/crs_configuration nfs
    rw,bg,hard,nointr,tcp,vers=3,timeo=300,rsize=32768,wsize=32768,actimeo=0
    0 0
    openfiler:/mnt/nfsrac_data/crs_ocr_voting/voting_disk
    /u01/voting_disk nfs
    rw,bg,hard,nointr,tcp,vers=3,timeo=300,rsize=32768,wsize=32768,actimeo=0
    0 0
    openfiler:/mnt/nfsrac_data/nfs_db_data/data_files /u02/data_files
    nfs
    rw,bg,hard,nointr,tcp,vers=3,timeo=300,rsize=32768,wsize=32768,actimeo=0
    0 0

    As to 10gR2..

    10.2.0.2 for the single instance as well as the software only new RAC enabled $ORACLE_HOME.

    The migration is using the newer utility rconfig which I believe calls rman in the background as archive logging is enabled on the source DB (single instance originating DB that is).

    Manual conversion attempts were done by dumping the dbf’s to the NFS shared storage area and recreating the control via a ouicli front end that apps uses called RapidClone. Once the single instance had the dbf’s relocated, I relinked the oracle kernel with rac_on and attempted to add another UNDO_TS…that’s when I got the ORA-29702.

    Any thoughts or suggestions are welcomed.

    Thanks again,

    Mike

  17. 18 Mike Smith December 5, 2007 at 12:59 pm

    Hey Kevin,

    I wanted to close the loop and update you and your readers on my quest seeking NFS Apps RAC nirvana. 🙂

    As mentioned above, I filed a bug related to the rconfig utility and it’s usage when CFS was the chosen file system versus ASM. Even though CFS isn’t realy a good definition for the EXT3 file system defined for an NFS RAC data file storage…this was the most sensible choice of the two options for the target file system type. Metalink note 388577.1 discusses how one might use the utility rconfig to convert single instance to RAC however the note falls short in the exact details required when NOT using ASM. In this particular case, when not using ASM, there is an ASMInfo stanza in the ConvertToTac.xml file used by the utility. This entire stanza must be commented out. This is not documented anywhere clearly.

    A second issue in my specific case, was that while I had setup rsh host equivalence, which was enough for CRS to install and work fine over NFS between the rac nodes, rconfig requires ssh equivalence. Without ssh host and user equivalence, this rconfig utility spits and error saying you don’t have a defined shared storage area in the xml format log and terminal message during execution.

    After working directly with the developer who owns the Metalink document 388577.1 for 2 days regarding my conversion, I am happy to say that Apps RAC with NFS shared storage is in fact working or rather, mostly so. I have several small issues to resolve, but nothing specific to the RAC on NFS deployment (apps related issues really)!!

    My thanks and appreciation go out to you and Tim for your informative and detailed blog postings.

    Best Regards,

    Mike

  18. 19 Kanishka August 21, 2009 at 12:35 pm

    Dear Friend..

    While CRS installation ( as I am following Tim’s procedure) .. after running root.sh I get stuck as … on either node on error comes “Failed to initialize crs…” in error log it is asking that nfs not mounted with correct option (rsize=0,wsize=0)… while it is properly mounted and many times rebooted also …

    Any help will be appreciated .

    Kanishka


  1. 1 Manly Men Deploy Oracle with 64 Bit Linux - Part I. What About an x86 Port on EM64t/AMD64 Hardware? « Kevin Closson’s Oracle Blog: Platform, Storage & Clustering Topics Related to Oracle Databases Trackback on July 13, 2007 at 9:58 pm
  2. 2 Manly Men Only Deploy Oracle with Fibre Channel - Part VIII. After All, Oracle Doesn’t Support Async I/O on NFS « Kevin Closson’s Oracle Blog: Platform, Storage & Clustering Topics Related to Oracle Databases Trackback on July 17, 2007 at 4:50 pm
  3. 3 Direct I/O e cluster filesystem: un dubbio « Oracle and other Trackback on July 23, 2007 at 12:48 pm

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




DISCLAIMER

I work for Amazon Web Services. The opinions I share in this blog are my own. I'm *not* communicating as a spokesperson for Amazon. In other words, I work at Amazon, but this is my own opinion.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 2,986 other followers

Oracle ACE Program Status

Click It

website metrics

Fond Memories

Copyright

All content is © Kevin Closson and "Kevin Closson's Blog: Platforms, Databases, and Storage", 2006-2015. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Kevin Closson and Kevin Closson's Blog: Platforms, Databases, and Storage with appropriate and specific direction to the original content.

%d bloggers like this: