Pages

Tuesday 24 July 2012

11g R2 RAC : PROBLEMS DISCOVERED AND SOLUTIONS



                11G R2 RAC PROBLEMS AND SOLUTIONS

I discovered some issues while working with 11g R2 RAC. I would like to share them with u all.

- During c/w installation, If kernel parameters are not set to
  appropriate values and runfixup.sh is run, kernel.sem parameter
  is set for running machine only. It's value is reset on next
  reboot and will have to be set manually to

  kernel.sem = 250 32000  100 128

-------------------------------------------------------------

- Whenver there is any problemwith ASM, it could be due to
  following reasons

 - parameter kernel.sem not set to appropriate value
 - Inappropriate permissions on /u01/app/11.2.0/grid/bin/oracle
 - Inappropriate permissions on /u01/app/oracle/product/11.2.0/dbhome_1/bin


------------------------------------------------------
  - parameter kernel.sem not set to appropriate value
-------------------------------------------------------

  soln:

  - add following line to /etc/sysctl.conf

    kernel.sem = 250 32000  100 128

  - #sysctl -p

-------------------------------------------------------
 - Inappropriate permissions on /u01/app/11.2.0/grid/bin/oracle
-------------------------------------------------------

- Check  that Permission on file /u01/app/11.2.0/grid/bin/oracle
  shd be

# ls -lr /u01/app/11.2.0/grid/bin/oracle

-rwsr-s--x 1 grid oinstall 152462728 Apr  4 08:45 oracle

  soln:

  - if permissions are not appropriate, change them

    #chmod 6751 /u01/app/11.2.0/grid/bin/oracle

     chown grid:oinstall /u01/app/11.2.0/grid/bin/oracle

-------------------------------------------------------
 - Inappropriate permissions on /u01/app/oracle/product/11.2.0/dbhome_1/bin
-------------------------------------------------------

- Check  that Permission on file
  /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle   shd be

#ls -lr oracle

-r-sr-s--x 1 oracle asmadmin 173515905 Apr 10 23:29 oracle

  soln:

  - if permissions are not appropriate, change them

    #chmod 6751  /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle

     chown oracle:asmadmin  /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle


------------------------------------------------------------------------

IF CRS SERVICE IS NOT STARTING ON A NODE AND CRSD.LOG HAS AN ENTRY
LIKE

   hash-id does not match

It implies that OCR info on that node is not compatible with
other nodes. This could be the case when OCR was reconfigured
(add/delete/replace a location) and the affected node was not
online so that it's ocr.loc was not updated.

Soln: Copy /etc/oracle/ocr.loc from a runnning node to the
      affected node and try to restart crs

-------------------------------------------------------------------
RCONFIG

IF WE USE RCONFIG TO ADD AN INSTANCE, ENTRIES ARE NOT AUTOMATICALLY
ADDED TO THE TNSNAMES.ORA ON THE NEWLY ADDED NODE. THEY HAVE TO BE
ADDED MANUALLY.

RCONFIG WILL AUTOMATICALLY ADD REDO THREADS FOR THE ALL THE RUNNING NODES IN THE CLUSTER. FOR EXAMPLE, IF WE HAVE
- A SINGLE INSTANCE DATABASE (NODE1)
- 5 NODES IN THE CLUSTER AND
- 3 RUNNING NODES (NODE1, NODE2, NODE3)

IF WE USE RCONFIG TO ADD ANOTHER INSTANCE ON ONE OF THE REMAINING RUNNING NODES (NODE2), IT WILL AUTOMATICALLY ADD 2 SETS OF REDO THREADS, ONE FOR NODE2 AND ANOTHER FOR NODE3.

--------------------------------------------------------------------

IF A NODE IS DELETED USING ROOTCRS.PL etc. procedure (steps 3,4 and 7)
GRID HOME IS NOT DELETED FROM THE DELETED NODE.

TO ADD BACK SUCH NODE,

  - RUN ADDNODE.SH WITH NOCOPY OPTION

    It will update the inventories on all existing nodes and
    instantiates scripts on local node

    #cd grid_home/oui/bin

     addNode.sh -silent -noCopy ORACLE_HOME=$ORACLE_HOME "CLUSTER_NEW_NODES={host03}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={host03-vip}"


   - RUN ROOT.SH

     In case it is not available in grid home, copy root.sh
     from any of the existing nodes

     #cd /u01/app/11.2.0/grid

      ./root.sh

---------------------------------------------------------------------------------------------

WHEN AN INSTANCE IS ADDED TO A RAC DATABASE USING DBCA, TNSNAMES.ORA
ON NEWLY ADDED INSTANCE IS NOT UPDATED.

Soln: Copy entry for the database from tnsnames.ora from existing node to the newly added node.

---------------------------------------------------------------------------------------------
 Problem:

After adding a node to the cluster, sometimes the node is visible to the cluster i.e. it appears when
crsctl stat res -t is issued but it is not listed in DBCA. The reason is that its information is not updated
in the inventory of existing nodes. 

Solution:

Add the entry of the newly added node in the inventory of the existing nodes for database home.

$cd /u01/app/oraInventory/ContentsXML
  vi inventory.xml
i.e.

<HOME NAME="OraDb11g_home1" LOC="/u01/app/oracle/product/11.2.0/dbhome_1" TYPE="O" IDX="2">
   <NODE_LIST>
      <NODE NAME="host01"/>
      <NODE NAME="host02"/>
      <NODE NAME="host03"/>

--------------------------------------------------------------------------------------------------------

If a serverpool is created using srvctl e.g. test and we want to see its status, we can see it using srvctl or crsctl.
If we query using srvctl status serverpool -g test, it will display its status
If we query using crsctl status serverpool test, it will not recognize test serverpool but
if we query using crsctk status serverpool ora.test, it displays the status.

Hence, the name of the serverpool created using srvctl should be prefixed by ora. for crsctl to recognize it. 

Another issue related to the same problem is that DBCA recognizes only the pools prefixed with ora. Hence if we want to create a policy managed database ,
  - create the corresponding serverpool using srvctl
   - if we create serverpool using crsctl and prefix  the name with .ora.   i.e crsctl create serverpool ora.test , DBCA will recognize the pool but the database created using this pool will be listed as ADMIN MANAGED when its configuration is checked using srvctl although it runs in the sreverpool.

   Hence we should use srvctl to create a serverpool if we want a database to run in that pool.

-------------------------------------------------------------------------------------------------------------

In case of an admin managed database , names of the instances have the format db1, db2 etc.
In case of an policy managed database , names of the instances have the format db_1, db_2 etc.
/etc/oratab should have entries for all the instances so that we can invoke EM.
To invoke EM:
$export ORACLE_UNQNAME=db
 -- check which is instance is running on current node
#srvctl status database -d db
-- set ORACLE_SID to the instance running on current node
#export ORACLE_SID=db_n
$emctl start dbconsole


------------------------------------------------------------------------------------------------------

 Related links :

HOME

11.2 RAC GOODIES AND SERVERPOOLS
11G R2 RAC PROBLEMS AND SOLUTIONS
CLONE DATABASE HOME IN 11GR2 RAC
NIC BONDING IN 11G R2 RAC
PREREQUISITE TO CONFIGURE GNS AND GPNP IN 11G R2 RAC
SERVICES CONFUSIONS CLEARED
SETUP 10G RAC ON YOUR LAPTOP
SETUP AND PLAY WITH 11G R2 RAC AT YOUR HOME PC
http://koenigocm.blogspot.in/search/label/11g%20R2%20RAC%20-%20ADD%20INSTANCE%20MANUALLY

11g R2 RAC - LOCAL ARCHIVE WITH NFS
  http://koenigocm.blogspot.in/search/label/11g%20R2%20RAC%20-%20LOCAL%20ARCHIVE%20WITH%20NFS
 
 11g R2 RAC : CACHE FUSION DEMONSTRATED
 http://koenigocm.blogspot.in/search/label/11g%20R2%20RAC%20%3A%20CACHE%20FUSION%20DEMONSTRATED


11G R2 RAC : DYNAMIC REMASTERING DEMONSTRATED

11gR2 RAC : USE RCONFIG TO CONVERT NON RAC DATABASE TO RAC DATABASE
http://koenigocm.blogspot.in/search/label/11gR2%20RAC%20%3A%20USE%20RCONFIG%20TO%20CONVERT%20NON%20RAC%20DATABASE%20TO%20%20RAC%20DATABASE

11g R2 RAC : RECOVER VOTING DISK - A SCENARIO
http://koenigocm.blogspot.in/search/label/11g%20R2%20RAC%20%3A%20RECOVER%20VOTING%20DISK%20-%20A%20SCENARIO


11g R2 RAC : TRACING SERVICES IN A RAC DATABASE
 http://koenigocm.blogspot.in/search/label/11g%20R2%20RAC%20%3A%20TRACING%20SERVICES%20%20IN%20A%20RAC%20DATABASE

11G R2 RAC: INSTANCE RECOVERY
 http://koenigocm.blogspot.in/search/label/11G%20R2%20RAC%3A%20INSTANCE%20RECOVERY

11g R2 RAC: MANAGING THIRD PARTY APPLICATIONS USING APPLICATION VIP
 http://koenigocm.blogspot.in/search/label/11g%20R2%20RAC%3A%20MANAGING%20THIRD%20PARTY%20APPLICATIONS%20USING%20APPLICATION%20VIP

7 comments:

naveen said...


Your website is very good and nice information was provided on your site, thanks for sharing.
Oracle RAC Training in Hyderabad

Unknown said...

Your blog is so nice, and the article is very good it helps to so many people.

Oracle RAC Training in Hyderabad

Unknown said...

Thank you for your valuable information also please do visit our website
Oracle RAC Training in Bur Dubai, Dubai

Training for IT and Software Courses said...

I am Here to Get Learn Good Stuff About RAC Training, Thanks For Sharing
Database Developer Training.RAC Training in Bangalore

KITS Technologies said...

What as up, I read your blogs like every week. Your writing style is awesome, keep up the good work!
Oracle Exadata online online training
Oracle fusion order management online online training
Oracle golden gate online online training
Oracle identity manager online online training
Oracle performance tuning online online training
Oracle SCM online online training
Oracle SOA online online training
OSB online online training
OTM online online training

Bronco Austin said...


Iam so thrilled because of finding your alluring website here.Actually i was searching for Oracle RAC.Your blog is so astounding and informative too..Iam very happy to find such a creative blog. Iam also find another one by mistake while am searching the same topicOracle SQL.Thank you soo much..

Realtime Experts said...

This Post is really supportive to all of us. Eager that these kind of information you post in future also. Thanks for Sharing Otherwise if anyone want To Learn SAS Training

RAC Training in Bangalore