How To Setup ASM & ASMLIB On Native Linux Multipath Mapper disks

This note apply to Oracle Database using ASM and linux S.O.

After installed and configured the Multipath, follow bellow:

1. Download and Install ASMLIB :
http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html

2. Check the disks :

ls -l /dev/mapper
brw-rw---- 1 root disk 253, 46 Jul 27 17:19 Prod_Orcl01
brw-rw---- 1 root disk 253, 67 Jul 27 17:59 Prod_Orcl02
brw-rw---- 1 root disk 253, 36 Jul 27 17:19 Prod_Orcl03
brw-rw---- 1 root disk 253, 58 Jul 27 18:00 Prod_Orcl04

3. Create the ASMLIB disks on mapper partitions as follow:

/etc/init.d/oracleasm createdisk DSKORA1  /dev/mapper/Prod_Orcl01
/etc/init.d/oracleasm createdisk DSKORA2  /dev/mapper/Prod_Orcl02

After create the disks, the ASM put a mark on the disks to know which are your own.

4. If this is a RAC configuration, then from each node execute:

/etc/init.d/oracleasm scandisks
/etc/init.d/oracleasm listdisks

5. Configure ASMLIB to use multipath  (from each node on RAC environments):

By any path the ASMLIB can found the disks, but, the best path is using the multipath :

Modify in /etc/sysconfig/oracleasm :
ORACLEASM_SCANORDER=”dm”
ORACLEASM_SCANEXCLUDE=”sd”

note: The Oracle ASMLib configuration file is located at /etc/sysconfig/oracleasm. It is a link to file /etc/sysconfig/oracleasm-_dev_oracleasm.

Restart ASMLIB (from each node on RAC environments):
/etc/init.d/oracleasm stop
/etc/init.d/oracleasm start

6. Verify if the configuration is correct:

6.1 During the disk discovery, ASMLIB uses file /proc/partitions, see :

# cat /proc/partitions

   8     0  877264896 sda
   8     1     104391 sda1
   8     2  877157032 sda2
   8    16  209715200 sdb
   8    32  382730240 sdc
   8    48  379596800 sdd
   8    64    2097152 sde
   8    80    2097152 sdf
   8    96 1169776640 sdg
 253     6  209715200 dm-6
 253     7  382730240 dm-7
 253     8  379596800 dm-8
 253     9    2097152 dm-9
 253    10    2097152 dm-10

6.2 The ASMLIB mount disks at /dev/oracleasm/disks, see:

# ls -la /dev/oracleasm/disks

brw-rw---- 1 grid asmadmin 253,  6 Ago 16 16:33 DSKORA1
brw-rw---- 1 grid asmadmin 253,  7 Ago 16 16:33 DSKORA2

6.3 Check if major and minor of disks “dm” not “sd”, is the same in /proc/partitions  and /dev/oracleasm/disks , see “253″ and “6″ bellow:

/proc/partitions 253     6  209715200 dm-6

/dev/oracleasm/disks
brw-rw---- 1 grid asmadmin 253,  6 Ago 16 16:33 DSKORA1

You can check using querydisk too :

# /etc/init.d/oracleasm querydisk -d DSKORA1
Disk "DSKORA1" is valid ASM disk on device [253, 6]

Good Luck !
======================================================================
Checking asm group with asm disk that have been created by ASMLib

bash> /etc/init.d/oracleasm listdisks
DISK1
DISK2
DISK3
DISK4
DISK5

SQL> conn / as sysasm
Connected.
SQL> col path format a15

SQL> select a.name,a.state,b.name,b.path from v$asm_diskgroup a, v$asm_disk b where a.group_number=b.group_number order by b.name;

NAME                           STATE       NAME                           PATH
------------------------------ ----------- ------------------------------ ---------------
DGROUP1                        MOUNTED     DISK1                          ORCL:DISK1
DGROUP1                        MOUNTED     DISK2                          ORCL:DISK2
DGROUP2                        MOUNTED     DISK3                          ORCL:DISK3
DGROUP2                        MOUNTED     DISK4                          ORCL:DISK4
DGROUP2                        MOUNTED     DISK5                          ORCL:DISK5