Pages

Tuesday, 3 January 2012

SETUP AND PLAY WITH 11G R2 RAC AT YOUR HOME PC

Before to follow below steps to setup 2-node 11g cluster ensure that DNS is up and running and you have already resolve scan name into three multiple IPs(recommended by oracle).



2- NODE 11g RAC SETUP WITH OPENFILER

PART-A  Open-filer Set-up



ü  Install openfiler on a machine (Leave 100GB free space on the hdd)

ü  Login to root user

ü  Start iSCSI target service

*      # service iscsi-target start

*      # chkconfig –level 345 iscsi-target on

PART –B Configuring Storage on openfiler



ü  From any client machine open the browser and access openfiler console (https://hostname:446) use openfiler as username and password is itself “password”

ü  Open system tab and update the local N/W configuration for both nodes with netmask (255.255.255.255).

ü  Create a single primary partition with partition type as “Physical Volume”.

ü  Create a volume group.

ü  Now create a single volume over the volume group which will be mapped to cluster nodes.

ü  Edit the property of each volume and change access to allow.



PART –C N/W Configuration on Linux Nodes

ü  Assign the ips

ü  Edit the /etc/hosts on both nodes as given below if you are not using DNS Server



Oracle RAC Node 1- (node1)
Device
IP Address
Subnet
Gateway
Purpose
Eth0
192.9.201.183
255.255.255.0
192.9.201.1
Connect node to the public network
Eth1
10.0.0.1
255.255.255.0

Connect node1 (interconnect) to node2
/etc/hosts
127.0.0.1                             localhost.localdomain                        localhost

#Public network – (eth0)
192.9.201.183                     node1.oracle.com                               node1
 192.9.201.187                    node2,oracle.com                               node2

#Private Interconnect – (eth1)
 10.0.0.1                              node1-priv.oracle.com                       node1-priv
 10.0.0.2                              node2-priv.oracle.com                       node2-priv

#Public Virtual IP (VIP Addresses) – (eth0:1)
 192.9.201.184                    node1-vip.oracle.com                         node1-vip
 192.9.201.187                    node2-vip.oracle.com                         node2-vip

#Private Storage network for openfiler
 192.9.201.182                   openfiler.oracle.com                            openfiler











Oracle RAC Node 2- (node2)

Device

IP Address

Subnet

Gateway

Purpose

Eth0

192.9.201.187

255.255.255.0

192.9.201.1

Connect node to the public network

Eth1

10.0.0.2

255.255.255.0



Connect node1 (interconnect) to node2

/etc/hosts

127.0.0.1                             localhost.localdomain                        localhost

#Public network – (eth0)
192.9.201.183                     node1.oracle.com                               node1
 192.9.201.187                    node2,oracle.com                               node2

#Private Interconnect – (eth1)
 10.0.0.1                              node1-priv.oracle.com                       node1-priv
 10.0.0.2                              node2-priv.oracle.com                       node2-priv

#Public Virtual IP (VIP Addresses) – (eth0:1)
 192.9.201.184                    node1-vip.oracle.com                         node1-vip
 192.9.201.187                    node2-vip.oracle.com                         node2-vip

#Private Storage network for openfiler
 192.9.201.182                   openfiler.oracle.com                            openfiler








Note:- Node wise perform ping operation

For node1:-

ping node2.oracle.com

ping node2

ping node2-priv.oracle.com

ping node2.priv

ping openfiler.oracle.com

ping openfiler



For node2:-

ping node1.oracle.com

ping node1

ping node1-priv.oracle.com

ping node1.priv

ping openfiler.oracle.com

ping openfiler



PART – D Node Configuration



ü  Set Kernel Parameters (/etc/sysctl.conf)


fs.aio-max-nr = 1048576
 fs.file-max = 6815744
 kernel.shmall = 2097152
 kernel.shmmax = 536870912
 kernel.shmmni = 4096
 kernel.sem = 250 32000 100 128
 net.ipv4.ip_local_port_range = 9000 65500
 net.core.rmem_default = 262144
 net.core.rmem_max = 4194304
 net.core.wmem_default = 262144
 net.core.wmem_max = 1048586


ü  Configure /etc/security/limits.conf file



*           soft    nproc   2047

            *           hard   nproc   16384

            *           soft    nofile   1024

            *           hard   nofile   65536



ü  Configure /etc/pam.d/login file



      session   required     /lib/security/pam_limits.so



ü  Create user and groups on both nodes



# groupadd oinstall

      # groupadd dba

      # groupadd oper

      #groupadd asmadmin

      #groupadd asmdba

      #groupadd asmoper

      # useradd –g oinstall –G asmadmin,asmdba,asmoper grid

      #useradd –g oinstall –G dba,oper,asmdba

      #passwd grid

      # passwd oracle



ü  Create required directories and set the ownership and permission.



# mkdir –p /u01/app/grid

#chown –R grid:oinstall /u01/app/

#chmod –R 775 /u01/app/grid

#mkdir –p /u01/app/oracle

#chown –R oracle:oinstall /u01/app/oracle/



ü  Dump all the required software



ü  Configure ssh



ü  Run cluster verifier



ü  Create partition and install & configure ASM LIBRARY

 
 
 
Configire SSH for both users Grid & Oracle
 
SSH configuration for Oracle user
 
On node1:- $ssh-keygen –t rsa
                          $ssh-keygen –t dsa
 
On node2:- $ssh-keygen –t rsa
                          $ssh-keygen –t dsa
 
On node1:- $cd .ssh
                          $cat *.pub>>node1
 
On node2:- $cd .ssh
                          $cat *.pub>>node2
 
On node1:- $scp node1 node2:/home/oracle/.ssh
On node2:- $scp node2 node2:/home/oracle/.ssh
 
On node1:- $cat node*>>authowized_keys
On node2:- $cat node*>>authowized_keys
 
SSH configuration for grid user
 
On node1:- $ssh-keygen –t rsa
                          $ssh-keygen –t dsa
 
On node2:- $ssh-keygen –t rsa
                          $ssh-keygen –t dsa
 
On node1:- $cd .ssh
                          $cat *.pub>>node1
 
On node2:- $cd .ssh
                          $cat *.pub>>node2
 
On node1:- $scp node1 node2:/home/grid/.ssh
On node2:- $scp node2 node2:/home/grid/.ssh
 
On node1:- $cat node*>>authowized_keys
On node2:- $cat node*>>authowized_keys
 
 
Now test the ssh configuration
$ vi a.sh
 
Ssh node1 hostname
Ssh node2 hostname
Ssh node1-priv hostname
Ssh node2-priv hostname
 
$chmod +x a.sh
$./a.sh
 
Execute above script from both nodes from both user grid and oracle to generate keys.
 
To run cluster verifier
 
PART –F iSCSI Volume configuration on nodes
 
Now on both nodes
1)      Open the /etc/iscsid/iscsid.conf
               ……………
               ins.address=Openfiler hostname
       
2)      start ths iscsi service
         #service iscsi restart
         #chkconfig –level 345 iscsi on
         #iscsiadm –m discovery –t sendtargets –p 192.9.201.182
         #service iscsi restart
 
3)      Now prepare a plan for creating partitions. Examples are given below
 

 
SL NO
 
Partition name

ASM disk name

For What?

Approx Size
 
1
/dev/sdb5

ASMDISK01
DATA(DISKGROUP)

2g

 
2
 /dev/sdb6
ASMDISK02
DATA(DISKGROUP)


2g

 
3
/dev/sdb7
ASMDISK03
DATA(DISKGROUP)

2g


4
/dev/sdb8
ASMDISK04
DATA1(DISKGROUP)

8g


5
/dev/sdb9
ASMDISK05
DATA1(DISKGROUP)

8g


6
/dev/sdb10
ASMDISK06
DATA1(DISKGROUP)

8g


7
/dev/sdb11
ASMDISK07
DATA1(DISKGROUP)


8g


8


9
/dev/sdb12


/dev/sdb13
ASMDISK08


ASMDISK09
FRA(DISKGROUP)


FRA(DISKGROUP)
8g


8g








5. Now you need to create 9 partitions as per above table. Create first three partitions with the size of 2gb and rest six partition with the size of 8gb. First allocate entire space as extended partition and then create logical partition within extended.



#fdisk /dev/sdb

……

:e (extended)

Part No. 1

First Cylinder: <Enter>

Last Cylinder: <Enter>

:p

:n

:l

First Cylinder: <Enter>

Last Cylinder: +2048M

…………………

……………………

…………………………..



6. Note the /dev/sdb* names.

7. #partprobe

8. Login as root user on node2 and run partprobe



On node1 login as root user and Install ASM LIBRARY



#rpm –Uvh –nodeps /software/asmlibrhel54/oracleasm-2.6.18-164.el5-2.0.5-1.el5.i686.rpm

# rpm –Uvh –nodeps /software/asmlibrhel54/oracleasm-2.6.18-164.el5PAE-2.0.5-1.el5.i686.rpm

# rpm –Uvh –nodeps /software/asmlibrhel54/oracleasmlib-2.0.4-1.el5.i386.rpm

# rpm –Uvh –nodeps /software/asmlibrhel54/oracleasm-support-2.1.3-1.el5.i386.rpm

#oracleasm configure –i

(use grid user as owner and asmadmin as group)

#oracleasm exit

#oracleasm init



Perform above steps on both the nodes.



Now Create ASM Disks from Node1



#oracleasm createdisk ASMDISK01 /dev/sdb5

#oracleasm createdisk ASMDISK02 /dev/sdb6

#oracleasm createdisk ASMDISK03 /dev/sdb7

#oracleasm createdisk ASMDISK04 /dev/sdb8

#oracleasm createdisk ASMDISK05 /dev/sdb9

#oracleasm createdisk ASMDISK06 /dev/sdb10

#oracleasm createdisk ASMDISK07 /dev/sdb11

#oracleasm createdisk ASMDISK08 /dev/sdb12

#oracleasm createdisk ASMDISK09 /dev/sdb13







Execute these commands from both the nodes to scan the ASM Disks

#oracleasm scandisks

#oracleasm listdisks







PART – E Grid Infra Installation

As grid user Install Grid Infra Software

$ cd /software/grid

$./runInstaller





PART – G RDBMS Installation

Install 11g R2 database software
------------------------------------------------------------------------------------------------------------
Related links:
HOME

11.2 RAC GOODIES AND SERVERPOOLS




 11G R2 RAC PROBLEMS AND SOLUTIONS




 CLONE DATABASE HOME IN 11GR2 RAC




NIC BONDING IN 11G R2 RAC



PREREQUISITE TO CONFIGURE GNS AND GPNP IN 11G R2 RAC

SERVICES CONFUSIONS CLEARED

SETUP 10G RAC ON YOUR LAPTOP

11g R2 RAC - ADD INSTANCE MANUALLY
http://koenigocm.blogspot.in/search/label/11g%20R2%20RAC%20-%20ADD%

11g R2 RAC - LOCAL ARCHIVE WITH NFS
  http://koenigocm.blogspot.in/search/label/11g%20R2%20RAC%20-%20LOCAL%20ARCHIVE%20WITH%20NFS
 
 11g R2 RAC : CACHE FUSION DEMONSTRATED
 http://koenigocm.blogspot.in/search/label/11g%20R2%20RAC%20%3A%20CACHE%20FUSION%20DEMONSTRATED
11G R2 RAC : DYNAMIC REMASTERING DEMONSTRATED

 11gR2 RAC : USE RCONFIG TO CONVERT NON RAC DATABASE TO RAC DATABASE
http://koenigocm.blogspot.in/search/label/11gR2%20RAC%20%3A%20USE%20RCONFIG%20TO%20CONVERT%20NON%20RAC%20DATABASE%20TO%20%20RAC%20DATABASE

  11g R2 RAC : RECOVER VOTING DISK - A SCENARIO
 http://koenigocm.blogspot.in/search/label/11g%20R2%20RAC%20%3A%20RECOVER%20VOTING%20DISK%20-%20A%20SCENARIO

11g R2 RAC: MANAGING THIRD PARTY APPLICATIONS USING APPLICATION VIP
 http://koenigocm.blogspot.in/search/label/11g%20R2%20RAC%3A%20MANAGING%20THIRD%20PARTY%20APPLICATIONS%20USING%20APPLICATION%20VIP


11g R2 RAC : TRACING SERVICES IN A RAC DATABASE
 http://koenigocm.blogspot.in/search/label/11g%20R2%20RAC%20%3A%20TRACING%20SERVICES%20%20IN%20A%20RAC%20DATABASE

11G R2 RAC: INSTANCE RECOVERY
 http://koenigocm.blogspot.in/search/label/11G%20R2%20RAC%3A%20INSTANCE%20RECOVERY

Posted by at
Labels: ,

2 comments:

Unknown said...

My favourite... Karan

3 January 2012 at 22:51
Unknown said...

great doc ,
just question..
what the benefit from using openfiler while i can use shared hard disks in vmware or virual box

Thanks again bro

21 February 2016 at 12:38

Post a Comment

Subscribe to: Post Comments (Atom)