Wednesday, June 30, 2010

Install Oracle RAC 11gR2 on Vmware with Windows 7 64-bit as host OS and Linux as guest OS (Part 4, RAC VM Setup)

He has no fear and He is not afraid of anyhthing


Index of all the posts of Gurpartap Singh's Blog


Now we are going to create the Base machine for RAC and I will name it base_rac.

Copy the base machine folder and rename the new folder as "rac_base"
Now open pen the new machine as shown below with Vmware Workstation
OR
Hit Ctrl+O (to open machine) and browse to the new base RAC directory and open the vmware config file in our case is CentOS_5_RAC.



Add 3 more host-only network cards to this machine as shown below, after doing this process for 3 times you will havein total 1 bridged card and 3 host-only cards:



click host-only





Then click options and name this machine as "base_rac" as shown below.



and then start the machine.




Once its up check its eth0 ip to connect from putty. In this case mine is 192.168.1.107



Login through putty and run following command as root:
system-config-network

and you come to the following screen. Here we will edit devices.



Update eth0



Uncheck Use dhcp
Static IP : 192.168.1.176
Netmask : 255.255.255.0
Default Gateway IP : 192.168.1.1



and then pick eth1:
Uncheck DHCP
Make static ip 10.10.1.176
Netmask 255.255.255.0








Then edit eth2 as:

Uncheck Use DHCP
Static IP 10.10.2.100
Netmask 255.255.255.0
Nothing for default gateway




Now for eth3 and eht4 as you did for eth1 but the static ips will be:
10.10.3.176 and 10.10.4.176













Now we will edit the dns configuration.


Change the name to base_rac.rac.meditate.com
and
primary DNS to 192.168.1.1 and click ok.


Now restart the network from the machine itself, as root run:
service network restart
It wil restart the full network and new ips will now be in effect.

Now connect to the machine through putty as root and run "ifconfig" and you will see the following output.
[root@ base_rac ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:9D:47:90
inet addr:192.168.1.176 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe9d:4790/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:724 errors:0 dropped:0 overruns:0 frame:0
TX packets:547 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:64021 (62.5 KiB) TX bytes:143868 (140.4 KiB)
Base address:0x2000 Memory:d8960000-d8980000

eth1 Link encap:Ethernet HWaddr 00:0C:29:9D:47:9A
inet addr:10.10.1.176 Bcast:10.10.1.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe9d:479a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:169 errors:0 dropped:0 overruns:0 frame:0
TX packets:80 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:41613 (40.6 KiB) TX bytes:15853 (15.4 KiB)
Base address:0x2080 Memory:d8980000-d89a0000

eth2 Link encap:Ethernet HWaddr 00:0C:29:9D:47:A4
inet addr:10.10.2.100 Bcast:10.10.2.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe9d:47a4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:149 errors:0 dropped:0 overruns:0 frame:0
TX packets:74 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:35176 (34.3 KiB) TX bytes:15359 (14.9 KiB)
Base address:0x20c0 Memory:d89a0000-d89c0000

eth3 Link encap:Ethernet HWaddr 00:0C:29:9D:47:AE
inet addr:10.10.3.176 Bcast:10.10.3.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe9d:47ae/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:131 errors:0 dropped:0 overruns:0 frame:0
TX packets:72 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:29702 (29.0 KiB) TX bytes:14730 (14.3 KiB)
Base address:0x2400 Memory:d89c0000-d89e0000

eth4 Link encap:Ethernet HWaddr 00:0C:29:9D:47:B8
inet addr:10.10.4.176 Bcast:10.10.4.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe9d:47b8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:100 errors:0 dropped:0 overruns:0 frame:0
TX packets:75 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:21525 (21.0 KiB) TX bytes:15022 (14.6 KiB)
Base address:0x2440 Memory:d89e0000-d8a00000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:560 (560.0 b) TX bytes:560 (560.0 b)

[root@ base_rac ~]#
Now check the hostname and it would also have been updated by now.

[root@ base_rac ~]# hostname
base_rac.rac.meditate.com

Try to ping oracle's website to make sure the internet is working on the machine as follows:

[root@base_rac ~]# ping www.oracle.com
PING a398.g.akamai.net (206.108.207.146) 56(84) bytes of data.
64 bytes from 206.108.207.146: icmp_seq=1 ttl=61 time=17.1 ms
64 bytes from 206.108.207.146: icmp_seq=2 ttl=61 time=16.6 ms
64 bytes from 206.108.207.146: icmp_seq=3 ttl=61 time=16.7 ms

--- a398.g.akamai.net ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 11017ms
rtt min/avg/max/mdev = 16.698/16.842/17.128/0.251 ms
[root@base_rac ~]#

Since ping is returning it means we are connected to the internet with this machine.

Now make your /etc/hosts look like following:

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 base.rac.meditate.com base localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6

# iSCSI SAN Server
10.10.1.175 san.rac.meditate.com san

# SCAN IP and Name
10.10.2.166 scan.rac.meditate.com

# This node primary address
192.168.1.176 rac1.rac.meditate.com rac1
# This node SAN address
10.10.1.176 rac1-san.rac.meditate.com rac1-san
# This node private interconnect address
10.10.3.176 rac1-priv.rac.meditate.com rac1-priv
# This node VIP address
10.10.2.176 rac1-vip.rac.meditate.com rac1-vip

# Repeat for each node in cluster
192.168.1.177 rac2.rac.meditate.com rac2
10.10.1.177 rac2-san.rac.meditate.com rac2-san
10.10.3.177 rac2-priv.rac.meditate.com rac2-priv
10.10.2.177 rac2-vip.rac.meditate.com rac2-vip

# Repeat for each node in cluster
192.168.1.178 rac3.rac.meditate.com rac3
10.10.1.178 rac3-san.rac.meditate.com rac3-san
10.10.3.178 rac3-priv.rac.meditate.com rac3-priv
10.10.2.178 rac3-vip.rac.meditate.com rac3-vip

Next Install the package dependencies for Oracle (answer ‘y’ when prompted), I would use yum to install the packages as:

yum install iscsi-initiator-utils \
gcc.x86_64 \
gcc-c++.x86_64 \
xorg-x11-utils.x86_64 \
xorg-x11-xinit.x86_64 \
unixODBC-devel.x86_64 \
unixODBC.i386 \
unixODBC-devel.i386 \
sysstat.x86_64 \
libstdc++.so.5 \
compat-libstdc++-33-3.2.3-61.x86_64 \
compat-db-4.2.52-5.1.x86_64 \
libaio-devel.x86_64 \
libaio-devel.i386 \
libXp.x86_64 \
libXp-devel.x86_64 \
libXpm-devel.x86_64 \
libXp.i386 \
gdb.x86_64 \
elfutils-libelf-devel.x86_64 \
compat-gcc-34.x86_64 \
compat-gcc-34-c++.x86_64 \
libgnome.x86_64 \
libc-client.x86_64 \
libc-client-devel.x86_64 \
glibc.x86_64 \
libXtst-devel.x86_64 \
libXtst.i386 \
glibc-devel.i386 \
glibc-devel.x86_64 \
pdksh.x86_64
After the above command downloads and installs the everything successfully do the following:
Copy over the downloaded Oracle-Validated RPM to the VM with Winscp.
(I hope everybody knows how to use Winscp). Though we can share the windows directories with this machine and no need to copy with Winscp but I am not using it for now.
From putty cd to the directory where you copied this file and run the following command to install the package:
rpm -Uhv oracle-validated-1.0.0-18.el5.x86_64.rpm

This package creates the oracle id and groups and sets kernel parameters. Remeber the default password of user oracle is same as username i.e. "oracle" and I am not going to change it in this article. This is much faster and reduces the chances of errors than doing it all manually. Though it should set all the kernel parameter necessary for Grid and Oracle but , it doesn't set these three parameter. So to set them manually and so that these parameters are ok in the next reboot edit as root the /etc/sysctl.conf file and update the following values for the following 3 parameters :

fs.file-max = 6815744
net.ipv4.ip_local_port_range = 9000 65500
net.core.wmem_max= 1048576
At this point make sure san machine is running and if not then start it up.
Now start the iscsi servvice on the machine by executing the following command as root:
service iscsi start

Now discover the luns with the following command:
iscsiadm -m discovery -t sendtargets -p san.rac.meditate.com
and you should see:
10.10.1.175:3260,1 iqm.mgmt.volumes-san
You should also be able to see the devices using
"fdisk -l" and
ls -l /dev/sd*
Note - sdb to sdm are the iscsi disks and if you don't see them reboot the machine and try discovery again and you should be able to see these luns.
Execute following as root so that iscsi service starts on every reboot.
chkconfig --level 345 iscsi on

With the following command change the permissions of the devices to oracle and oinstall:
Though we have more devices but we have activated just 12 of them in SAN, if you want you can add them now or you can add them later. I will not add them now.
chown oracle:oinstall /dev/sd[b-m]
Also add this line to the end of the file /etc/rc.d/rc.local
chown oracle:oinstall /dev/sd[b-m]
Make the following directories on the machine and then change their ownership to oracle.

mkdir /u01 /u02 /software
chown oracle:oinstall /u01 /u02 /software

Now tweak the machine so that installer thinks that its Redhat and not CentOS as installer fails otherwise.

echo "redhat-release-5Server-5" > /tmp/.linux_release
/usr/bin/chattr +i /tmp/.linux_release


Install X windows software Xming. Its free to download and use and start it.

http://sourceforge.net/projects/xming/
Now, open the new putty window and click on SSH under connection and clcik X11 and then check Enable X11 forwarding. The screen shot of this putty step has been covered in step 5
Now login as root to this machine from this window and do the following.

Now, install the vmware tools,click VM->Install VMware Tools


Now on putty window that we opened in the above step mount the cdrom as:
First create the cdrom directory with the following command:
mkdir /cdrom

Then mount it with teh following directory:
mount -t iso9660 -o ro /dev/cdrom /cdrom
Now shift to cdrom directory
cd /cdrom
and list the files in it:
[root@base_rac cdrom]# ls -lart
total 93674
-r--r--r-- 1 root root 1864 Oct 22 2009 manifest.txt
-r--r--r-- 1 root root 95909121 Oct 22 2009 VMwareTools-8.1.3-203739.tar.gz
dr-xr-xr-x 2 root root 2048 Oct 22 2009 .
drwxr-xr-x 28 root root 4096 Jun 29 22:51 ..
[root@base_rac cdrom]#
So we have VMwareTools-8.1.3-203739.tar.gz, we will untar it to install VMware tools as follows:
shift to /tmp directory as:
cd /tmp
Unzip it as:
gunzip -c /cdrom/VMwareTools-8.1.3-203739.tar.gz tar xf -
and now list the contents as:

[root@base_rac tmp]# ls -lart
total 28
drwxr-xr-x 7 root root 4096 Oct 22 2009 vmware-tools-distrib
-rw-r--r-- 1 root root 25 Jun 29 22:11 .linux_release
drwxrwxrwt 2 root root 4096 Jun 29 22:44 .ICE-unix
drwxr-xr-x 28 root root 4096 Jun 29 22:51 ..
drwxrwxrwt 4 root root 4096 Jun 29 22:57 .
Now shift to the following directory:
[root@base_rac tmp]#
cd /tmp/vmware-tools-distrib

At this point we are all set to run the install of the VMware tools. Just execute the following file and it will install the VMWare tools as follows, remeber to hit enter at all the questions i.e. accept the default answers and you will have the following output:

[root@base_rac vmware-tools-distrib]# ./vmware-install.pl
Creating a new VMware Tools installer database using the tar4 format.

Installing VMware Tools.

In which directory do you want to install the binary files?
[/usr/bin]

What is the directory that contains the init directories (rc0.d/ to rc6.d/)?
[/etc/rc.d]

What is the directory that contains the init scripts?
[/etc/rc.d/init.d]

In which directory do you want to install the daemon files?
[/usr/sbin]

In which directory do you want to install the library files?
[/usr/lib/vmware-tools]

The path "/usr/lib/vmware-tools" does not exist currently. This program is
going to create it, including needed parent directories. Is this what you want?
[yes]

In which directory do you want to install the documentation files?
[/usr/share/doc/vmware-tools]

The path "/usr/share/doc/vmware-tools" does not exist currently. This program
is going to create it, including needed parent directories. Is this what you
want? [yes]

The installation of VMware Tools 8.1.3 build-203739 for Linux completed
successfully. You can decide to remove this software from your system at any
time by invoking the following command: "/usr/bin/vmware-uninstall-tools.pl".

Before running VMware Tools for the first time, you need to configure it by
invoking the following command: "/usr/bin/vmware-config-tools.pl". Do you want
this program to invoke the command for you now? [yes]


Stopping VMware Tools services in the virtual machine:
Guest operating system daemon: [ OK ]
Virtual Printing daemon: [ OK ]
Unmounting HGFS shares: [ OK ]
Guest filesystem driver: [ OK ]


Found a compatible pre-built module for vmmemctl. Installing it...


Found a compatible pre-built module for vmhgfs. Installing it...


Found a compatible pre-built module for vmxnet. Installing it...


Found a compatible pre-built module for vmblock. Installing it...


[EXPERIMENTAL] The VMware FileSystem Sync Driver (vmsync) is a new feature that
creates backups of virtual machines. Please refer to the VMware Knowledge Base
for more details on this capability. Do you wish to enable this feature?
[no]

Found a compatible pre-built module for vmci. Installing it...


Found a compatible pre-built module for vsock. Installing it...


Found a compatible pre-built module for vmxnet3. Installing it...


Found a compatible pre-built module for pvscsi. Installing it...


No X install found.

Creating a new initrd boot image for the kernel.
Checking acpi hot plug [ OK ]
Starting VMware Tools services in the virtual machine:
Switching to guest configuration: [ OK ]
Paravirtual SCSI module: [ OK ]
Guest filesystem driver: [ OK ]
Mounting HGFS shares: [FAILED]
Guest memory manager: [ OK ]
VM communication interface: [ OK ]
VM communication interface socket family: [ OK ]
Blocking file system: [ OK ]
File system sync driver: [ OK ]
Guest operating system daemon: [ OK ]
Virtual Printing daemon: [ OK ]

The configuration of VMware Tools 8.1.3 build-203739 for Linux for this running
kernel completed successfully.

You must restart your X session before any mouse or graphics changes take
effect.

You can now run VMware Tools by invoking the following command:
"/usr/bin/vmware-toolbox" during an X server session.

To enable advanced X features (e.g., guest resolution fit, drag and drop, and
file and text copy/paste), you will need to do one (or more) of the following:
1. Manually start /usr/bin/vmware-user
2. Log out and log back into your desktop session; and,
3. Restart your X session.

Enjoy,

--the VMware team

Found VMware Tools CDROM mounted at /cdrom. Ejecting device /dev/cdrom ...
[root@base_rac vmware-tools-distrib]#

Now start Xming if you have not started earlier and execute the following command:

/usr/bin/vmware-toolbox
Xming will open the following window and you can check the checkbox for time synchronization and then click close. Thats it and now we are done with installing VMware tools.



We are done with BASE_RAC machine, just shut it down with the following command:
Shut down the Base RAC VM using “init 0” .

Create 2 VM nodes for the RAC cluster by making 2 copies of Base RAC vm. If you have just 4GB ram don't try for third node. If you have 6GB or 8GB then make 3rd copy of Base RAC vm and name it rac3
I will not install Grid Infrastructure on this third node i.e. rac3 but will use this to show you, how to add a new node to the existing 2 node cluster and how to add a new instance to an existing RAC database in Part 8 .

Now open all these 3 VMs and rename them. We have done renaming earlier also so I am not doing the screenshots here. Just click edit virtual machine and then click options and rename the machine to rac1 i.e. replace base_rac with rac1 and in the same way rename other two copies as rac2 and rac3.
Now start rac1 and it will not use static ip and will usse dhcp. You when you copy machines then at first reboot the machine uses dhcp. Just fix it. You have to do this just once and then at subsequent reboot it will pick static ip.
So login as root on the console and execute "ifconfig eth0" to see the ip the machine is using. In my case it was 192.168.1.106
Just logon through putty to this machine with this ip address and run :
system-config-network
and update eth0 to use static ip 192.168.1.176,netmask 255.255.255.0 and gateway 192.168.1.1
and uncheck dhcp and
update eth1 to static ip of 10.10.1.176 and netmask of 255.255.255.0 and nothing gateway and uncheck dhcp and
update eth2 to static ip of 10.10.2.100 and netmask of 255.255.255.0 and nothing gateway and uncheck dhcp and
update eth3 to static ip of 10.10.3.176 and netmask of 255.255.255.0 and nothing for gateway and uncheck dhcp and
update eth4 to static ip of 10.10.4.176 and netmask of 255.255.255.0 and nothing for gateway and uncheck dhcp.
Now edit the DNS configuration:
rename to "rac1.rac.meditate.com"
and update primary dns to 192.168.1.1

Now start rac2 and it will not use static ip and will use dhcp.

So login as root on the console and execute "ifconfig eth0" to see the ip the machine is using. In mycase it was 192.168.1.107

Just logon through putty to this machine with this ip address and run :
system-config-network

and update eth0 to use static ip 192.168.1.177,netmask 255.255.255.0 and gateway 192.168.1.1
and uncheck dhcp and

update eth1 to static ip of 10.10.1.177 and netmask of 255.255.255.0 and nothing gateway and uncheck dhcp and
update eth2 to static ip of 10.10.2.101 and netmask of 255.255.255.0 and nothing gateway and uncheck dhcp and
update eth3 to static ip of 10.10.3.177 and netmask of 255.255.255.0 and nothing for gateway and uncheck dhcp and

update eth4 to static ip of 10.10.4.177 and netmask of 255.255.255.0 and nothing for gateway and uncheck dhcp.
Now edit the DNS configuration, change name to rac2.rac.meditate.com and
update the primary DNS to 192.168.1.1


Now if you are using rac3 then start it and it will also not use static ip and will use dhcp.

So login as root on the console and execute "ifconfig eth0" to see the ip the machine is using. In mycase it was 192.168.1.108

Just logon through putty to this machine with this ip address and run :
system-config-network

and update eth0 to use static ip 192.168.1.178,netmask 255.255.255.0 and gateway 192.168.1.1
and uncheck dhcp and
update eth1 to static ip of 10.10.1.178 and netmask of 255.255.255.0 and nothing gateway and uncheck dhcp and
update eth2 to static ip of 10.10.2.102 and netmask of 255.255.255.0 and nothing gateway and uncheck dhcp and
update eth3 to static ip of 10.10.3.178 and netmask of 255.255.255.0 and nothing for gateway and uncheck dhcp and
update eth4 to static ip of 10.10.4.178 and netmask of 255.255.255.0 and nothing for gateway and uncheck dhcp.

Now update the DNS configuration and rename the machine to rac3.rac.meditate.com and
update the Primary DNS to 192.168.1.1
Now reboot all the nodes and check that they have the same ips as we have just set and also they can ping "ww.oracle.com" and you can see the luns from these machines by running
"ls -l /dev/sd*"
You should see few luns owned by oracle:oinstall to make sure that the machines can see the luns.
Now shutdown all the machines as we are ready to install grid infrastructure.
After you shoutdown all the machines just take the backup of all the machnes by copying them as in the next part we are going to install the Grid Infrastructure.
Remember:
Feel Him in and around you all the time as He is everywhere.
Posted by at 20 comments:

Tuesday, June 29, 2010

Install Oracle RAC 11gR2 on Vmware with Windows 7 64-bit as host OS and Linux as guest OS (Part 3, SAN Setup)

He is the Creator and is doing everything


Index of all the posts of Gurpartap Singh's Blog

Here are the steps to create the SAN machine.

After we copied the machine name that folder as SAN. Now open that machine with VMware workstation.



Here screen shot while opening the machine you need to browse to the location. Pick CentOS_5_BASE (Vmware file)



After opening it, click edit this machine.



click options and here change the name to "san"



Reduce the memory to 128MB and start the machine.



click I copied it on the following POP up.



In the following screen shot you can see the machine is rebooting.



After the machine reboots, login as root and execute the following command to see the ip address of the machine. in my case it picked 192.168.1.106



Login as root to this machine from putty. Putty is a small executable file that can be dowloaded easily and instead of working on machine directly, I will work with putty as its easy to copy paste stuff with that. Execute as root on putty window :
system-config-network



Hit edit devices and pick the first one as eth0 and hit enter to edit it.


Enter the following IP addresses for this machine.
1. Unselect Use DHCP
2. Static IP as 192.168.1.175
3. Netmask 255.255.255.0
4. Default Gateway 192.168.1.1
and then clcik ok.


Next select eth1 to edit i.e third row and enter the following ips.
1. Unselect Use DHCP.
2. Static IP 10.10.1.175
3. Netmask 255.255.255.0
Leave default gateway IP as empty. and click ok.


Now Edit DNS configuration

Here you will see that the hostname is already updated to "san.rac.meditate.com"
If not then update it. Just enter Primary DNS as 192.168.1.1
and click ok



And then on next screen click save and quit.

Now from m/c as root execute:
#service network restart

This will start the network and will make the ip of our machine as as 192.168.1.175

The old putty window will loose the connection so now open the new window and login as root and you should be able to ping oracle as follows:

login as: root
root@192.168.1.175's password:
Last login: Mon Jun 28 20:55:07 2010 from 192.168.1.103
[root@san ~]# ping www.oracle.com
PING a398.g.akamai.net (206.108.207.137) 56(84) bytes of data.
64 bytes from 206.108.207.137: icmp_seq=1 ttl=61 time=17.2 ms
64 bytes from 206.108.207.137: icmp_seq=2 ttl=61 time=16.2 ms

--- a398.g.akamai.net ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 16.219/16.726/17.233/0.507 ms


Edit /etc/hosts and it should look like:

[root@san ~]# cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 base.rac.meditate.com base localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
# SAN
192.168.1.175 san.rac.meditate.com san

Now as root, restart the machine:
shutdown -h now

and then login as root from putty and see all the network settings are ok as:

[root@san ~]# hostname
san.rac.meditate.com
[root@san ~]# cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 base.rac.meditate.com base localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
# SAN
192.168.1.175 san.rac.meditate.com san

[root@san ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:B2:8E:A6
inet addr:192.168.1.175 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:feb2:8ea6/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:210 errors:0 dropped:0 overruns:0 frame:0
TX packets:210 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:20520 (20.0 KiB) TX bytes:26784 (26.1 KiB)
Base address:0x2000 Memory:d8940000-d8960000

eth1 Link encap:Ethernet HWaddr 00:0C:29:B2:8E:B0
inet addr:10.10.1.175 Bcast:10.10.1.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:feb2:8eb0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:46 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:7687 (7.5 KiB)
Base address:0x2080 Memory:d8960000-d8980000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:560 (560.0 b) TX bytes:560 (560.0 b)

[root@san ~]#




Install the iSCSI package and answer 'y' to the prompts.
Run the following command as root from the putty window as root:

yum install scsi-target-utils



Create a directory for the disks
mkdir /disks

Create a set of 24 disk files each 2Gb in size

for i in 1 to 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1117 18 19 20 21 22 23 24
do
dd if=/dev/zero of=/disks/disk$i.dat bs=1M count=2048
done



[root@san ~]# mkdir /disks
[root@san ~]# for i in 1 to 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
> do
> dd if=/dev/zero of=/disks/disk$i.dat bs=1M count=2048
> done

2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 4.66889 seconds, 460 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 7.42677 seconds, 289 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 4.88252 seconds, 440 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 7.27913 seconds, 295 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 6.02063 seconds, 357 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 5.77786 seconds, 372 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 5.81557 seconds, 369 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 6.69618 seconds, 321 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 6.01836 seconds, 357 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 5.76424 seconds, 373 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 5.98724 seconds, 359 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 6.99209 seconds, 307 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 5.88205 seconds, 365 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 5.93012 seconds, 362 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 5.73615 seconds, 374 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 5.65302 seconds, 380 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 7.10604 seconds, 302 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 6.1137 seconds, 351 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 5.80223 seconds, 370 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 5.94321 seconds, 361 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 7.27233 seconds, 295 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 5.9688 seconds, 360 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 6.6014 seconds, 325 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 6.27009 seconds, 342 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 6.1728 seconds, 348 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 6.04302 seconds, 355 MB/s
[root@san ~]#



Set the iscsi target demon to start automatically
chkconfig --level 345 tgtd on



Edit the following file:
/etc/rc.d/rc.local

So it has the following entries:

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local

# Create a target
tgtadm --lld iscsi --op new --mode target --tid 1 -T iqm.mgmt.volumes-san

# Create LUNs within the target
for i in 1 2 3 4 5 6 7 8 9 10 11 12
do
tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun $i -b /disks/disk$i.dat
done

# Expose target to all
tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL



Bounce the VM again with "reboot"



Verify target is ready and the LUNs are present:
tgtadm --lld iscsi --op show --mode target

You should see something like the following:

[root@san ~]# tgtadm --lld iscsi --op show --mode target

Target 1: iqm.mgmt.volumes-san
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: None
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /disks/disk1.dat
LUN: 2
Type: disk
SCSI ID: IET 00010002
SCSI SN: beaf12
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /disks/disk2.dat
.................
.................
...............
.................
LUN: 11
Type: disk
SCSI ID: IET 0001000b
SCSI SN: beaf111
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /disks/disk11.dat
LUN: 12
Type: disk
SCSI ID: IET 0001000c
SCSI SN: beaf112
Size: 2147 MB
Online: Yes
Removable media: No
Backing store type: rdwr
Backing store path: /disks/disk12.dat
Account information:
ACL information:
[root@san ~]#



Shut down the VM with "shutdown -h now"

Now we have configured our SAN and it has 12 luns of 2GB each and can be used by RAC.
Next we will configure RAC machines. We can add other 12 luns later if required.

Remember:
Where there is no mother, father, friends or siblings, there is only the Naam, the Name of the Lord, shall be with you as your help and support. So try to do as much meditaion in the rememberance of God as much you can.
Posted by at 11 comments:

Monday, June 28, 2010

Install Oracle RAC 11gR2 on Vmware with Windows 7 64-bit as host OS and Linux as guest OS (Part 2, Setup Base Brick Machine)

His Name is Truth


Index of all the posts of Gurpartap Singh's Blog

Now we will create a Base machine, Master schema calls this as Base Brick Machine.
This machine will be used to make the SAN machine and the RAC machines. This note can also be used to install Linux on Windows 7 with Virtual machines.

Open Vmware and click file->New->Virtual Machines as follows:



Click Typical



Select installer disk image and browse the location of linux 5.4 64-bit, software in .iso format



Name hte virtual machine as CentOS_S_BASE
and type the location whereever you want to keep the snapshot. I used "C:\VM\rac11g"
and click next.





It is just a virtual size. I entered 100GB and also don't forget to click "Split Virtual Disk into 2GB files" and click next.



Though the screenshot below has "Power on this virtual Machine after creation" just Uncheck "Power on this virtual machine after creation" as we will do some modifications to this machine (after all its our base machine).





After the machine creation Edit the virtual machine and click on Network adapter and make it Bridged and check "Replicate physical network conversion state" as shown below and keep "Connect at power on" checked and then hit OK.



Increase memory to 1536 MB.



No start the machine.



We will install CentOS in graphical mode, so hit enter on the following screen to start instalation in graphical mode.


You will see something like in following screen.



On this screen click skip.



Click next.


Select English and next.



Then, US english and next.



Click yes on warning.


Check "Review and manually partioning layout"



Click yes again.



Select /ext3 as shown below and click edit.





You will see something like this.



Enter the size of the machine in MB as 88064 Or something that is the multiple of 1024 but less than 90,000 and then click ok.


Nowclick on /swap and hit edit as shown below.



Now increase the value of swap from 3040 to 5024MB and hit ok and now move to the next page.


Keep the defaults i.e. GRUB and move on.




Here we will set up the ips for our machines. Click eth0 and hit edit.



On the following screen enter its ip. My router is set on 192.168.1.x. So, I have used 192.168.1.174 and netmask as 255.255.255.0 and hit ok.



Though we don't need eth1 on this machine but I am using this. Next check eth1 and hit edit.


Enter the ip as 10.10.1.1 and netmask as 255.255.255.0 and hit OK.


Now enter the name of the machine as:
base.rac.meditate.com

(I am picking "meditate" because, "Meditation in rememberance of God is better than medication to cure all diseases in the world")
and
Enter the Gateway and Primary DNS as 192.168.1.1 (ip of your router) and hit next.


I selected Vancouver here and uncheck "Synhcronise clock with UTC" and click next.

Enter the password for root.


Uncheck first option Desktop/Genome and start the install.

Install will start and you will see something like this.

next screen during install.







and now reboot































After the reboot finishes you will have something like this, edit firewall configuration












Security level disabled and selinux disabled too.











Then edit system services and uncheck the following list of services.












and then exit.












and now login as root on the following screen.













Now, we are done with Base Machine. you can stop the machime with 'init 0' and make the copy of this machime i.e. copy the whole folder and rename it as SAN. You need to copy folder rac11g. Make two copies of it. Name one copy as base_brick_machine. Other copy as "san". Then delete all the contents under folder rac11g and move folders "base_brick_machine" and "san" under it.
Now, open these machines from vmware workstation console as file -> open -> san -> (file type ("vmware virtual machine configuration") and start the machine. On the pop up that says machine has been copied, just say yes. We are doing this just to make sure the machine comes up fine. Keep adding the machines to favorites. Just right click on the machine on right side of the window in VM workstation console and click "add to favorites". Once machine is up and running just stop it by executing "init 0" as root.


In next part we will make the SAN machine.

Remember:
This world is like a bubble in a water that wells up and disapper, there is nothing that is forever in this Universe the only thing that was, is and will remain true is God Himself, so always remember Him and try to be one with Him.



Posted by at 9 comments:
Subscribe to: Posts (Atom)