ASMLib Troubleshooting

I've noticed a few forum questions regarding ASM or indeed the OUI not being able to see devices that are managed via ASMLib. This prompted me to "upgrade" my knowledge of ASMLib and this blog is just a few extra tools for checking on your ASMLib devices.

By the way, anyone out there thinking ASMLib is not getting a whole lot of love from Oracle of late? The latest updates on the ASMLib page seems to be early 2007.

Anyway, first troubleshooting tip is a simple one, but make sure you have all three ASMLib rpms:

# rpm -qa |grep asm
oracleasm-support-2.0.3-1
oracleasmlib-2.0.2-1
oracleasm-2.6.9-22.ELsmp-2.0.3-1

You get odd behaviour without all of 'em. So what do each of these provide you:

# rpm -ql oracleasm-support
/etc/init.d/oracleasm
/etc/sysconfig/oracleasm
/usr/lib/oracleasm/oracleasm_debug_link
/usr/sbin/asmscan
/usr/sbin/asmtool

So the init.d oracleasm script is really where you configure disks and includes various options, like listing disks and querying. This is actually just a shell script that calls the executables asmscan and asmtool. There is a configuration file in /etc/sysconfig where you can change things like the pattern to scan for devices and you also have the ability to exclude devices using this configuration file. Excluding devices and explicitly setting the scanorder can be useful for multipath devices.

Once you have ran /etc/init.d/oracleasm configure you should see a new device:

# df -ha |grep asm
oracleasmfs 0 0 0 - /dev/oracleasm

# rpm -ql oracleasmlib
/opt/oracle/extapi
/opt/oracle/extapi/64
/opt/oracle/extapi/64/asm
/opt/oracle/extapi/64/asm/orcl
/opt/oracle/extapi/64/asm/orcl/1
/opt/oracle/extapi/64/asm/orcl/1/libasm.so
/usr/sbin/oracleasm-discover

So this rpm provides you with a library and an executable. Running the executable once you have configured devices is kinda nice:

# /usr/sbin/oracleasm-discover
Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so
[ASM Library - Generic Linux, version 2.0.2 (KABI_V2)]
Discovered disk: ORCL:VOL1 [121634784 blocks (62277009408 bytes), maxio 512]
Discovered disk: ORCL:VOL2 [20971488 blocks (10737401856 bytes), maxio 512]
Discovered disk: ORCL:VOL3 [20971488 blocks (10737401856 bytes), maxio 512]
Discovered disk: ORCL:VOL4 [419424957 blocks (214745577984 bytes), maxio 512]

The final rpm is the kernel module:

# rpm -ql oracleasm-2.6.9-22.ELsmp
/lib/modules/2.6.9-22.ELsmp/kernel/drivers/addon/oracleasm
/lib/modules/2.6.9-22.ELsmp/kernel/drivers/addon/oracleasm/oracleasm.ko

You want to ensure that the oracleasm has been loaded by the kernel:

# /sbin/lsmod |grep oracleasm
oracleasm 55176 1

You can find information about the module with modinfo:

# /sbin/modinfo oracleasm
filename: /lib/modules/2.6.9-22.ELsmp/kernel/drivers/addon/oracleasm/oracleasm.ko
description: Kernel driver backing the Generic Linux ASM Library.
author: Joel Becker
version: 2.0.3
license: GPL
depends:
vermagic: 2.6.9-22.ELsmp SMP gcc-3.4

Make sure the devices you are trying to use are known by the kernel you can check in /dev/ or look in /proc/partitions. ASMLib likes to work on partitions, you can create this on a device using fdisk.

A list of devices marked by ASMLib is generated with:

# /etc/init.d/oracleasm listdisks
VOL1
VOL2
VOL3
VOL4

You can cross-reference this with what is in the /dev/oracleasm/disks directory:

# ls -l /dev/oracleasm/disks/
total 0
brw-rw---- 1 oracle oinstall 8, 17 Jun 24 09:13 VOL1
brw-rw---- 1 oracle oinstall 8, 49 Jun 24 09:13 VOL2
brw-rw---- 1 oracle oinstall 8, 65 Jun 24 09:13 VOL3
brw-rw---- 1 oracle oinstall 8, 97 Jun 24 09:13 VOL4

You can use querydisk to determine which device a particular ASMLib Volume corresponds to:

# /etc/init.d/oracleasm querydisk -d VOL1
Disk "VOL1" is a valid ASM disk on device [8, 17]

You can find out which devices this represents with the following:

# grep "8 17" /proc/partitions
8 17 60817392 sdb1

Still paranoid that this might not be your device, check the contents of the disk header:

# od -c /dev/sdb1 |head -10
0000000 001 202 001 001 200 036 - W 310
0000020
0000040 O R C L D I S K V O L 1
0000060
0000100 020 \n 001 003 V O L 1
0000120
0000140 D A T A 1
0000160
0000200 V O L 1
0000220

There is also a neat trick with blkid which shows the disk headers:

#./blkid|grep asm
/dev/sdb1: LABEL="VOL1" TYPE="oracleasm"
/dev/sdd1: LABEL="VOL2" TYPE="oracleasm"
/dev/sde1: LABEL="VOL3" TYPE="oracleasm"
/dev/sdg1: LABEL="VOL4" TYPE="oracleasm"
/dev/sdo1: LABEL="VOL1" TYPE="oracleasm"
/dev/sdq1: LABEL="VOL2" TYPE="oracleasm"
/dev/sdr1: LABEL="VOL3" TYPE="oracleasm"
/dev/sdt1: LABEL="VOL4" TYPE="oracleasm"
/dev/emcpowerf1: LABEL="VOL4" TYPE="oracleasm"
/dev/emcpowerp1: LABEL="VOL3" TYPE="oracleasm"
/dev/emcpowero1: LABEL="VOL2" TYPE="oracleasm"
/dev/emcpowern1: LABEL="VOL1" TYPE="oracleasm"

You can see from the above, that I have multiple devices corresponding to the same physical device and I am using EMC Powerpath as the multipathing software.

Note not all versions of blkid (well it's actually the E2fsprogs version) pick up oracleasm as a type.

AS you can see there are various techniques to check what devices you have configured via ASMLib for using with your ASM instance!