AskDBA.org Forum » Topic: ASM and Multipathing

AskDBA.org Forum » Topic: ASM and Multipathing http://www.forum.askdba.org/ Forum for Oracle DBA's en Sun, 12 Feb 2012 08:46:38 +0000 deepika.dhamija@gmail.com on "ASM and Multipathing" http://www.forum.askdba.org/topic/asm-and-multipathing#post-111 Wed, 09 Dec 2009 12:29:40 +0000 deepika.dhamija@gmail.com 111@http://www.forum.askdba.org/ Amit, I am finally able to achieve what I intended to do. Many Thanks to you for your valuable suggestions. Amit Bansal on "ASM and Multipathing" http://www.forum.askdba.org/topic/asm-and-multipathing#post-110 Wed, 09 Dec 2009 09:30:43 +0000 Amit Bansal 110@http://www.forum.askdba.org/ Hi, >>Actually I have an application that runs on every node on the cluster and creates the devices /dev/multipath/sd* Can't understand this statement. Anyways if you use udev, you can set permissions in rules file /etc/udev/rules.d/50-udev.rules I think your sysadmin can help you with that. -Amit deepika.dhamija@gmail.com on "ASM and Multipathing" http://www.forum.askdba.org/topic/asm-and-multipathing#post-109 Wed, 09 Dec 2009 09:03:19 +0000 deepika.dhamija@gmail.com 109@http://www.forum.askdba.org/ Hi Amit, Actually I have an application that runs on every node on the cluster and creates the devices /dev/multipath/sd*. I am new to this entire setup, but I can tell you that it uses udev. On Node 1 [oracle@linux1 ~]$ uname -a Linux linux1 2.6.9-78.0.13.ELsmp #1 SMP Wed Jan 7 17:52:47 EST 2009 i686 i686 i386 GNU/Linux Does this much information answer your questions? Amit Bansal on "ASM and Multipathing" http://www.forum.askdba.org/topic/asm-and-multipathing#post-108 Wed, 09 Dec 2009 07:58:21 +0000 Amit Bansal 108@http://www.forum.askdba.org/ Deepika, How have you configured multipathing? Are you using udev and device mapper? Also let me know your linux release details. -Amit deepika.dhamija@gmail.com on "ASM and Multipathing" http://www.forum.askdba.org/topic/asm-and-multipathing#post-107 Wed, 09 Dec 2009 07:10:39 +0000 deepika.dhamija@gmail.com 107@http://www.forum.askdba.org/ Thanks Amit for the prompt reply. I had ORACLE_SID SET to orcl1 and orcl2 on both the nodes respectively. On node1, I did the above steps, and I luckily was able to connect to Oracle. It did not say 'connected to idle instance'. The O/P of the query on Node1 (ORACLE_SID=+ASM1) is NAME PATH MOUNT_S HEADER_STATUS ORCL:VOL1 CLOSED UNKNOWN ORCL:VOL1 CLOSED UNKNOWN ORCL:VOL1 CLOSED UNKNOWN The O/P of the query on Node2 (ORACLE_SID=orcl2) is NAME PATH MOUNT_S HEADER_STATUS VOL1 ORCL:VOL1 OPENED UNKNOWN VOL2 ORCL:VOL2 OPENED UNKNOWN VOL3 ORCL:VOL3 OPENED UNKNOWN Does that mean that the multipath disks have not been picked up by asm at all. What about the permissions. I ran the foll commands (for sdb2, sdb3, sdb4) on Node1 # chown -R oracle:dba /dev/multipath/sdb2 # chmod -R 775 /dev/multipath/sdb2 # /etc/init.d/oracleasm scandisks # /etc/init.d/oracleasm listdisks VOL1 VOL2 VOL3 How do these permissions remain intact after a reboot to the node? Amit Bansal on "ASM and Multipathing" http://www.forum.askdba.org/topic/asm-and-multipathing#post-106 Wed, 09 Dec 2009 06:54:50 +0000 Amit Bansal 106@http://www.forum.askdba.org/ Hi, Try logging into ASM instance and query V$ASM_DISK i.e export ORACLE_SID=+ASM1 sqlplus "/ as sysdba" select name,path,mount_status,header_status from V$ASM_DISK; with regards to naming convention, I think naming convention should follow /dev/[a-z]* naming convention instead of /dev/multipath/[a-z]* It could be this or it could be that as the name of multipath device is also ending with sdb2,sdb3 and we are also excluding the disk ending with sdb2,sdb3. I am not too sure on this. I think best way would be to actually first query V$ASM_DISK output. deepika.dhamija@gmail.com on "ASM and Multipathing" http://www.forum.askdba.org/topic/asm-and-multipathing#post-105 Wed, 09 Dec 2009 06:33:44 +0000 deepika.dhamija@gmail.com 105@http://www.forum.askdba.org/ Hi Amit, Yes, I am following the above URL. On Node1: [root@linux1 ~]# /etc/init.d/oracleasm listdisks VOL1 VOL2 VOL3 [root@linux1 ~]# /etc/init.d/oracleasm querydisk /dev/multipath/sdb2 Device "/dev/multipath/sdb2" is marked an ASM disk with the label "VOL1" [root@linux1 ~]# /etc/init.d/oracleasm querydisk /dev/sdb2 Device "/dev/sdb2" is marked an ASM disk with the label "VOL1" The O/P is same for sdb3 and sdb4 (both multipath & single path devices) Is this what you mean when you say if disks are visible on node1? [oracle@linux1 ~]$ srvctl status asm -n linux1 ASM instance +ASM1 is running on node linux1. [oracle@linux1 ~]$ srvctl status instance -d orcl -i orcl1 Instance orcl1 is not running on node linux1 [oracle@linux1 ~]$ srvctl start instance -d orcl -i orcl1 PRKP-1001 : Error starting instance orcl1 on node linux1 CRS-0215: Could not start resource 'ora.orcl.orcl1.inst'. [oracle@linux1 ~]$ I am unable to start the instance orcl1 on node1. Also what is the significance of naming multipath devices in a particular way? Amit Bansal on "ASM and Multipathing" http://www.forum.askdba.org/topic/asm-and-multipathing#post-104 Tue, 08 Dec 2009 13:32:15 +0000 Amit Bansal 104@http://www.forum.askdba.org/ Hi Deepika, I believe you are following below article available on Oracle website <a href="http://www.oracle.com/technology/tech/linux/asmlib/multipath.html" rel="nofollow">http://www.oracle.com/technology/tech/linux/asmlib/multipath.html</a> First I believe that you should check ASM instance and see if you are able to see the disks on Node 1. From above output it looks that you connected to node 1 DB instance and not asm instance. This can also be seen from crs_stat output which shows that asm instance is up on node 1. Secondly name multipath device as multipatha,multibathc instead of /multipath/sdb1 etc deepika.dhamija@gmail.com on "ASM and Multipathing" http://www.forum.askdba.org/topic/asm-and-multipathing#post-103 Tue, 08 Dec 2009 09:37:34 +0000 deepika.dhamija@gmail.com 103@http://www.forum.askdba.org/ Hi I have a 2node RAC Setup 10gR2. I have three ASM Volumes configured, out of devices sdb2, sdb3, sdb4 /etc/init.d/oracleasm createdisk VOL1 /dev/sdb2 /etc/init.d/oracleasm createdisk VOL2 /dev/sdb3 /etc/init.d/oracleasm createdisk VOL3 /dev/sdb4. where VOL1 & VOL2 belong to the Disk Group ORCL_DATA1 and Vol3 to the group FLASH_RECOVERY_AREA. I want to use the multipath disks namely /dev/multipath/sdb2, /dev/multipath/sdb3 and /dev/multipath/sdb4 for asm volumes. I created the multipath disks on node1 and edited the file /etc/sysconfig/oracleasm and set ORACLEASM_SCANORDER="multipath" and ORACLEASM_SCANEXCLUDE="sdb2 sdb3 sdb4". After a Reboot, the crs_stat shows On Node1 -------------------------------------------------------------------------------- Name Type Target State Host -------------------------------------------------------------------------------- ora....SM1.asm application ONLINE ONLINE linux1 ora....X1.lsnr application ONLINE ONLINE linux1 ora.linux1.gsd application ONLINE ONLINE linux1 ora.linux1.ons application ONLINE ONLINE linux1 ora.linux1.vip application ONLINE ONLINE linux1 ora....SM2.asm application ONLINE ONLINE linux2 ora....X2.lsnr application ONLINE ONLINE linux2 ora.linux2.gsd application ONLINE ONLINE linux2 ora.linux2.ons application ONLINE ONLINE linux2 ora.linux2.vip application ONLINE ONLINE linux2 ora.orcl.db application ONLINE ONLINE linux1 ora....l1.inst application ONLINE OFFLINE ora....l2.inst application ONLINE ONLINE linux2 ora....test.cs application ONLINE ONLINE linux2 ora....cl1.srv application ONLINE OFFLINE ora....cl2.srv application ONLINE ONLINE linux2 oracle@linux1 ~$ sqlplus "/ as sysdba" SQL*Plus: Release 10.2.0.1.0 - Production on Tue Dec 8 12:04:53 2009 Copyright (c) 1982, 2005, Oracle. All rights reserved. Connected to an idle instance. SQL> select name, path from v$asm_disk; select name, path from v$asm_disk * ERROR at line 1: ORA-01034: ORACLE not available I try to start the instance manually, oracle@linux1 ~$ srvctl start instance -d orcl -i orcl1 PRKP-1001 : Error starting instance orcl1 on node linux1 CRS-0215: Could not start resource 'ora.orcl.orcl1.inst'. Also from sql SQL> startup ORA-01078: failure in processing system parameters ORA-01565: error in identifying file '+ORCL_DATA1/orcl/spfileorcl.ora' ORA-17503: ksfdopn:2 Failed to open file +ORCL_DATA1/orcl/spfileorcl.ora ORA-15077: could not locate ASM instance serving a required diskgroup On Node2 -------------------------------------------------------------------------------- oracle@linux2 ~$ sqlplus "/ as sysdba" SQL*Plus: Release 10.2.0.1.0 - Production on Tue Dec 8 11:56:13 2009 Copyright (c) 1982, 2005, Oracle. All rights reserved. Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production With the Partitioning, Real Application Clusters, OLAP and Data Mining options SQL> select name, path from v$asm_disk; NAME -------------------------------------------------------------------------------- PATH -------------------------------------------------------------------------------- VOL1 ORCL:VOL1 VOL2 ORCL:VOL2 VOL3 ORCL:VOL3 Plz note, that I have the multipath devices only on one node currently. I am new to the multipathing concept. Plz advise, on what (above) steps are wrong and what additional things are to be taken care of. And the causes/solutions to the errors.