====== Multipath mit RHEL4 (an DS4300) ====== Folgende Konstellation ist gegeben: * RedHat Enterprise Linux 4 ES U3 (Kernel 2.6.9-34.0.1ELsmp) * 2x qlogic HBA Typ 2340 * DS4300 mit 2 Controllern * HBA1 geht auf Controller A * HBA2 geht auf Controller B Um Multipath einzurichten, sind nachfolgende Schritte vorzunehmen. Siehe auch: {{wissen:ibm:blade:dm-multipath-usage.txt|dm-multipath-usage.txt}} ===== initrd erstellen ===== Creating initramfs Looking for deps of module scsi_mod Looking for deps of module sd_mod scsi_mod Looking for deps of module mptbase Looking for deps of module mptscsih mptfc mptspi mptsas mptscsi mptbase scsi_mod Looking for deps of module mptfc mptscsi mptbase scsi_mod Looking for deps of module mptscsi mptbase scsi_mod Looking for deps of module mptbase Looking for deps of module ata_piix libata scsi_mod Looking for deps of module libata scsi_mod Looking for deps of module scsi_mod Looking for deps of module scsi_mod Looking for deps of module qla2300 qla2xxx scsi_transport_fc scsi_mod Looking for deps of module qla2xxx scsi_transport_fc scsi_mod Looking for deps of module scsi_transport_fc scsi_mod Looking for deps of module scsi_mod Looking for deps of module ide-disk Looking for deps of module dm-mod Looking for deps of module ext3 jbd Looking for deps of module jbd Looking for deps of module dm-mod Looking for deps of module dm-mirror dm-mod Looking for deps of module dm-zero dm-mod Looking for deps of module dm-snapshot dm-mod Looking for deps of module dm-multipath dm-mod Using modules: ./kernel/drivers/scsi/scsi_mod.ko ./kernel/drivers/scsi/sd_mod.ko ./kernel/drivers/message/fusion/mptbase.ko ./kernel/drivers/message/fusion/mptscsi.ko ./kernel/drivers/message/fusion/mptfc.ko ./kernel/drivers/message/fusion/mptspi.ko ./kernel/drivers/message/fusion/mptsas.ko ./kernel/drivers/message/fusion/mptscsih.ko ./kernel/drivers/scsi/libata.ko ./kernel/drivers/scsi/ata_piix.ko ./kernel/drivers/scsi/scsi_transport_fc.ko ./kernel/drivers/scsi/qla2xxx/qla2xxx.ko ./kernel/drivers/scsi/qla2xxx/qla2300.ko ./kernel/drivers/md/dm-mod.ko ./kernel/fs/jbd/jbd.ko ./kernel/fs/ext3/ext3.ko ./kernel/drivers/md/dm-mirror.ko ./kernel/drivers/md/dm-zero.ko ./kernel/drivers/md/dm-snapshot.ko ./kernel/drivers/md/dm-multipath.ko /sbin/nash -> /tmp/initrd.iR6392/bin/nash /sbin/insmod.static -> /tmp/initrd.iR6392/bin/insmod /sbin/udev.static -> /tmp/initrd.iR6392/sbin/udev /etc/udev/udev.conf -> /tmp/initrd.iR6392/etc/udev/udev.conf copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/scsi/scsi_mod.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/scsi_mod.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/scsi/sd_mod.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/sd_mod.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/message/fusion/mptbase.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/mptbase.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/message/fusion/mptscsi.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/mptscsi.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/message/fusion/mptfc.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/mptfc.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/message/fusion/mptspi.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/mptspi.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/message/fusion/mptsas.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/mptsas.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/message/fusion/mptscsih.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/mptscsih.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/scsi/libata.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/libata.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/scsi/ata_piix.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/ata_piix.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/scsi/scsi_transport_fc.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/scsi_transport_fc.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/scsi/qla2xxx/qla2xxx.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/qla2xxx.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/scsi/qla2xxx/qla2300.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/qla2300.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/md/dm-mod.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/dm-mod.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/fs/jbd/jbd.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/jbd.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/fs/ext3/ext3.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/ext3.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/md/dm-mirror.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/dm-mirror.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/md/dm-zero.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/dm-zero.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/md/dm-snapshot.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/dm-snapshot.ko(elf64-x86-64) copy from /lib/modules/2.6.9-34.0.1.ELsmp/./kernel/drivers/md/dm-multipath.ko(elf64-x86-64) to /tmp/initrd.iR6392/lib/dm-multipath.ko(elf64-x86-64) /sbin/lvm.static -> /tmp/initrd.iR6392/bin/lvm /etc/lvm -> /tmp/initrd.iR6392/etc/lvm `/etc/lvm/lvm.conf' -> `/tmp/initrd.iR6392/etc/lvm/lvm.conf' Loading module scsi_mod Loading module sd_mod Loading module mptbase Loading module mptscsi Loading module mptfc Loading module mptspi Loading module mptsas Loading module mptscsih Loading module libata Loading module ata_piix Loading module scsi_transport_fc Loading module qla2xxx Loading module qla2300 Loading module dm-mod Loading module jbd Loading module ext3 Loading module dm-mirror Loading module dm-zero Loading module dm-snapshot Loading module dm-multipath ===== dmesg ===== QLogic Fibre Channel HBA Driver qla2300 0000:04:02.0: Found an ISP2312, irq 209, iobase 0xffffff0000004000 qla2300 0000:04:02.0: Configuring PCI space... qla2300 0000:04:02.0: Configure NVRAM parameters... qla2300 0000:04:02.0: Verifying loaded RISC code... qla2300 0000:04:02.0: LIP reset occured (f8f7). qla2300 0000:04:02.0: Waiting for LIP to complete... qla2300 0000:04:02.0: LIP occured (f8f7). qla2300 0000:04:02.0: LOOP UP detected (2 Gbps). qla2300 0000:04:02.0: Topology - (Loop), Host Loop address 0x7d scsi2 : qla2xxx qla2300 0000:04:02.0: QLogic Fibre Channel HBA Driver: 8.01.02-d4 QLogic QLA2340 - 133MHz PCI-X to 2Gb FC, Single Channel ISP2312: PCI-X (100 MHz) @ 0000:04:02.0 hdma+, host#=2, fw=3.03.18 IPX Vendor: IBM Model: 1722-600 Rev: 0520 Type: Direct-Access ANSI SCSI revision: 03 qla2300 0000:04:02.0: scsi(2:0:0:0): Enabled tagged queuing, queue depth 32. SCSI device sdb: 853947711 512-byte hdwr sectors (437221 MB) SCSI device sdb: drive cache: write back SCSI device sdb: 853947711 512-byte hdwr sectors (437221 MB) SCSI device sdb: drive cache: write back sdb: sdb1 Attached scsi disk sdb at scsi2, channel 0, id 0, lun 0 Vendor: IBM Model: Universal Xport Rev: 0520 Type: Direct-Access ANSI SCSI revision: 03 qla2300 0000:04:02.0: scsi(2:0:0:31): Enabled tagged queuing, queue depth 32. SCSI device sdc: 40960 512-byte hdwr sectors (21 MB) SCSI device sdc: drive cache: write through SCSI device sdc: 40960 512-byte hdwr sectors (21 MB) SCSI device sdc: drive cache: write through sdc: unknown partition table Attached scsi disk sdc at scsi2, channel 0, id 0, lun 31 ACPI: PCI interrupt 0000:05:01.0[A] -> GSI 48 (level, low) -> IRQ 217 qla2300 0000:05:01.0: Found an ISP2312, irq 217, iobase 0xffffff0000006000 qla2300 0000:05:01.0: Configuring PCI space... qla2300 0000:05:01.0: Configure NVRAM parameters... qla2300 0000:05:01.0: Verifying loaded RISC code... qla2300 0000:05:01.0: LIP reset occured (f8f7). qla2300 0000:05:01.0: Waiting for LIP to complete... qla2300 0000:05:01.0: LIP occured (f8f7). qla2300 0000:05:01.0: LOOP UP detected (2 Gbps). qla2300 0000:05:01.0: Topology - (Loop), Host Loop address 0x7d scsi3 : qla2xxx qla2300 0000:05:01.0: QLogic Fibre Channel HBA Driver: 8.01.02-d4 QLogic QLA2340 - 133MHz PCI-X to 2Gb FC, Single Channel ISP2312: PCI-X (133 MHz) @ 0000:05:01.0 hdma+, host#=3, fw=3.03.18 IPX Vendor: IBM Model: 1722-600 Rev: 0520 Type: Direct-Access ANSI SCSI revision: 03 qla2300 0000:05:01.0: scsi(3:0:0:0): Enabled tagged queuing, queue depth 32. SCSI device sdd: 853947711 512-byte hdwr sectors (437221 MB) SCSI device sdd: drive cache: write back SCSI device sdd: 853947711 512-byte hdwr sectors (437221 MB) SCSI device sdd: drive cache: write back sdd: sdd1 Attached scsi disk sdd at scsi3, channel 0, id 0, lun 0 Vendor: IBM Model: Universal Xport Rev: 0520 Type: Direct-Access ANSI SCSI revision: 03 qla2300 0000:05:01.0: scsi(3:0:0:31): Enabled tagged queuing, queue depth 32. SCSI device sde: 40960 512-byte hdwr sectors (21 MB) SCSI device sde: drive cache: write through SCSI device sde: 40960 512-byte hdwr sectors (21 MB) SCSI device sde: drive cache: write through sde: unknown partition table Attached scsi disk sde at scsi3, channel 0, id 0, lun 31 Die Platten, die für uns interessant sind, sind die 400GB LUNs (sdb,sdd). ===== lsscsi ===== RPMs sind unter http://rpms.serviert.de verfügbar. [0:0:0:0] disk LSILOGIC 1030 IM 1000 /dev/sda [0:0:8:0] process IBM 25P3495a S320 1 1 - [2:0:0:0] disk IBM 1722-600 0520 /dev/sdb [2:0:0:31] disk IBM Universal Xport 0520 /dev/sdc [3:0:0:0] disk IBM 1722-600 0520 /dev/sdd [3:0:0:31] disk IBM Universal Xport 0520 /dev/sde Hier sind ''sdb'' und ''sdd'' diesselbe pysikalische LUN, die aber über unterschiedliche Pfade erreichbar sind (unterschiedliche HBA und Storage Controller Kombination). Da die ''device mapper block devices'' keine SCSI LUNs sind, werden diese nicht angezeigt. ===== multipath config ===== # /sbin/scsi_id -g -u -s /block/sdb 3600a0b800019e99000002e1234567890 # /sbin/scsi_id -g -u -s /block/sdc 3600a0b800019e99000002e6d00000000 # /sbin/scsi_id -g -u -s /block/sdd 3600a0b800019e99000002e1234567890 D.h. sdb und sdd sind identisch. defaults { udev_dir /dev multipath_tool "/sbin/multipath -v0" polling_interval 60 selector "round-robin 0" path_grouping_policy failover getuid_callout "/sbin/scsi_id -g -u -s /block/%n" prio_callout /bin/true path_checker tur rr_min_io 100 rr_weight priorities failback immediate no_path_retry fail user_friendly_name yes } devnode_blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[a-z]" devnode "^cciss!c[0-9]d[0-9]*" devnode "^sda" } devices { device { vendor "IBM " product "1722-600 " path_grouping_policy group_by_serial path_checker tur } device { vendor "DataCore" product "SAN*" path_checker tur failback 10 } } **Erklärung** * vendor -> LUNs, die DataCore oder IBM als vendor in der SCSI INQUIRY data * product -> LUNs, die "SAN*" oder "1722-600" als product-string enthalten * path_checker tur -> Nach einem Pfad-Fehler wird der "failed Path" mit SCSI TEST UNIT READY geprüft * failback 10 -> Wenn ein "failed-path" wieder "healthy" ist, warte 10 Sekunden vor dem FailBack * polling_interval 60 -> Wenn ein Pfad defekt ist, prüfe alle 60 Seconden, ob er wieder OK ist create: 3600a0b800019e99000002e1234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 \_ 2:0:0:0 sdb 8:16 [ready] \_ round-robin 0 \_ 3:0:0:0 sdd 8:48 [ready] create: 3600a0b800019e99000002e6d00000000 [size=20 MB][features="0"][hwhandler="0"] \_ round-robin 0 \_ 2:0:0:31 sdc 8:32 [ready] create: 3600a0b800019eac20000383600000000 [size=20 MB][features="0"][hwhandler="0"] \_ round-robin 0 \_ 3:0:0:31 sde 8:64 [ready] brw-rw---- 1 root disk 253, 5 Jun 27 09:20 3600a0b800019e99000002e1234567890 brw-rw---- 1 root disk 253, 8 Jun 27 09:20 3600a0b800019e99000002e1234567890p1 brw-rw---- 1 root disk 253, 6 Jun 27 09:20 3600a0b800019e99000002e6d00000000 brw-rw---- 1 root disk 253, 7 Jun 27 09:20 3600a0b800019eac20000383600000000 Mountpoint anpassen (vorher ''/dev/sdb1'', jetzt ''/dev/mapper''), Mounten nach ''LABEL=XYZ'' ist nicht möglich, da durch die Mehrfachpfade die Fehlermeldung ''mount: LABEL=XYZ duplicate - not mounted'' beim mount erscheint. /dev/mapper/3600a0b800019e99000002e1234567890p1 /data ext3 defaults 0 0 ===== Vorbereitung reboot ===== * chkconfig multipathd on Da beim Systemstart die multipath-Devices erzeugt werden, und der ''multipath.static'' auf das Dateisystem schreiben muss, ist die Datei ''/etc/rc.sysinit'' anzupassen. # Multipath initialization if [ -x /sbin/multipath.static ]; then if ! LC_ALL=C fgrep -q "dm_multipath" /proc/modules 2>/dev/null ; then modprobe dm-multipath >/dev/null 2>&1 fi if [ -x /sbin/multipath.static ]; then if /sbin/multipath.static -v0 > /dev/null 2>&1 ; then action $"Setting up Multipathing:" /sbin/multipath.static -v0 fi fi fi ===== Reboot (dmesg) ===== Jun 28 07:19:38 ds4k kernel: QLogic Fibre Channel HBA Driver Jun 28 07:19:38 ds4k kernel: ACPI: PCI interrupt 0000:04:02.0[A] -> GSI 24 (level, low) -> IRQ 209 Jun 28 07:19:38 ds4k kernel: qla2300 0000:04:02.0: Found an ISP2312, irq 209, iobase 0xffffff0000004000 Jun 28 07:19:38 ds4k kernel: qla2300 0000:04:02.0: Configuring PCI space... Jun 28 07:19:38 ds4k kernel: qla2300 0000:04:02.0: Configure NVRAM parameters... Jun 28 07:19:38 ds4k kernel: qla2300 0000:04:02.0: Verifying loaded RISC code... Jun 28 07:19:38 ds4k kernel: qla2300 0000:04:02.0: LIP reset occured (f8f7). Jun 28 07:19:38 ds4k kernel: qla2300 0000:04:02.0: Waiting for LIP to complete... Jun 28 07:19:38 ds4k kernel: qla2300 0000:04:02.0: LIP occured (f8f7). Jun 28 07:19:38 ds4k kernel: qla2300 0000:04:02.0: LOOP UP detected (2 Gbps). Jun 28 07:19:38 ds4k kernel: qla2300 0000:04:02.0: Topology - (Loop), Host Loop address 0x7d Jun 28 07:19:38 ds4k kernel: scsi2 : qla2xxx Jun 28 07:19:38 ds4k kernel: qla2300 0000:04:02.0: Jun 28 07:19:39 ds4k kernel: QLogic Fibre Channel HBA Driver: 8.01.02-d4 Jun 28 07:19:39 ds4k kernel: QLogic QLA2340 - 133MHz PCI-X to 2Gb FC, Single Channel Jun 28 07:19:39 ds4k kernel: ISP2312: PCI-X (100 MHz) @ 0000:04:02.0 hdma+, host#=2, fw=3.03.18 IPX Jun 28 07:19:39 ds4k kernel: Vendor: IBM Model: 1722-600 Rev: 0520 Jun 28 07:19:39 ds4k kernel: Type: Direct-Access ANSI SCSI revision: 03 Jun 28 07:19:39 ds4k kernel: qla2300 0000:04:02.0: scsi(2:0:0:0): Enabled tagged queuing, queue depth 32. Jun 28 07:19:39 ds4k kernel: SCSI device sdb: 853947711 512-byte hdwr sectors (437221 MB) Jun 28 07:19:39 ds4k kernel: SCSI device sdb: drive cache: write back Jun 28 07:19:39 ds4k kernel: SCSI device sdb: 853947711 512-byte hdwr sectors (437221 MB) Jun 28 07:19:39 ds4k kernel: SCSI device sdb: drive cache: write back Jun 28 07:19:39 ds4k kernel: sdb: sdb1 Jun 28 07:19:39 ds4k kernel: Attached scsi disk sdb at scsi2, channel 0, id 0, lun 0 Jun 28 07:19:39 ds4k kernel: Vendor: IBM Model: Universal Xport Rev: 0520 Jun 28 07:19:39 ds4k kernel: Type: Direct-Access ANSI SCSI revision: 03 Jun 28 07:19:39 ds4k kernel: qla2300 0000:04:02.0: scsi(2:0:0:31): Enabled tagged queuing, queue depth 32. Jun 28 07:19:39 ds4k kernel: SCSI device sdc: 40960 512-byte hdwr sectors (21 MB) Jun 28 07:19:39 ds4k kernel: SCSI device sdc: drive cache: write through Jun 28 07:19:39 ds4k kernel: SCSI device sdc: 40960 512-byte hdwr sectors (21 MB) Jun 28 07:19:39 ds4k kernel: SCSI device sdc: drive cache: write through Jun 28 07:19:39 ds4k kernel: sdc: unknown partition table Jun 28 07:19:39 ds4k kernel: Attached scsi disk sdc at scsi2, channel 0, id 0, lun 31 Jun 28 07:19:39 ds4k kernel: ACPI: PCI interrupt 0000:05:01.0[A] -> GSI 48 (level, low) -> IRQ 217 Jun 28 07:19:39 ds4k kernel: qla2300 0000:05:01.0: Found an ISP2312, irq 217, iobase 0xffffff0000006000 Jun 28 07:19:39 ds4k kernel: qla2300 0000:05:01.0: Configuring PCI space... Jun 28 07:19:39 ds4k kernel: qla2300 0000:05:01.0: Configure NVRAM parameters... Jun 28 07:19:39 ds4k kernel: qla2300 0000:05:01.0: Verifying loaded RISC code... Jun 28 07:19:39 ds4k kernel: qla2300 0000:05:01.0: LIP reset occured (f8f7). Jun 28 07:19:39 ds4k kernel: qla2300 0000:05:01.0: Waiting for LIP to complete... Jun 28 07:19:39 ds4k kernel: qla2300 0000:05:01.0: LIP occured (f8f7). Jun 28 07:19:39 ds4k kernel: qla2300 0000:05:01.0: LOOP UP detected (2 Gbps). Jun 28 07:19:39 ds4k kernel: qla2300 0000:05:01.0: Topology - (Loop), Host Loop address 0x7d Jun 28 07:19:39 ds4k kernel: scsi3 : qla2xxx Jun 28 07:19:39 ds4k kernel: qla2300 0000:05:01.0: Jun 28 07:19:39 ds4k kernel: QLogic Fibre Channel HBA Driver: 8.01.02-d4 Jun 28 07:19:39 ds4k kernel: QLogic QLA2340 - 133MHz PCI-X to 2Gb FC, Single Channel Jun 28 07:19:39 ds4k kernel: ISP2312: PCI-X (133 MHz) @ 0000:05:01.0 hdma+, host#=3, fw=3.03.18 IPX Jun 28 07:19:39 ds4k kernel: Vendor: IBM Model: 1722-600 Rev: 0520 Jun 28 07:19:39 ds4k kernel: Type: Direct-Access ANSI SCSI revision: 03 Jun 28 07:19:39 ds4k kernel: qla2300 0000:05:01.0: scsi(3:0:0:0): Enabled tagged queuing, queue depth 32. Jun 28 07:19:39 ds4k kernel: SCSI device sdd: 853947711 512-byte hdwr sectors (437221 MB) Jun 28 07:19:39 ds4k kernel: SCSI device sdd: drive cache: write back Jun 28 07:19:39 ds4k kernel: SCSI device sdd: 853947711 512-byte hdwr sectors (437221 MB) Jun 28 07:19:39 ds4k kernel: SCSI device sdd: drive cache: write back Jun 28 07:19:39 ds4k kernel: sdd: sdd1 Jun 28 07:19:39 ds4k kernel: Attached scsi disk sdd at scsi3, channel 0, id 0, lun 0 Jun 28 07:19:39 ds4k kernel: Vendor: IBM Model: Universal Xport Rev: 0520 Jun 28 07:19:39 ds4k kernel: Type: Direct-Access ANSI SCSI revision: 03 Jun 28 07:19:39 ds4k kernel: qla2300 0000:05:01.0: scsi(3:0:0:31): Enabled tagged queuing, queue depth 32. Jun 28 07:19:39 ds4k kernel: SCSI device sde: 40960 512-byte hdwr sectors (21 MB) Jun 28 07:19:39 ds4k kernel: SCSI device sde: drive cache: write through Jun 28 07:19:39 ds4k kernel: SCSI device sde: 40960 512-byte hdwr sectors (21 MB) Jun 28 07:19:39 ds4k kernel: SCSI device sde: drive cache: write through Jun 28 07:19:39 ds4k kernel: sde: unknown partition table Jun 28 07:19:39 ds4k kernel: Attached scsi disk sde at scsi3, channel 0, id 0, lun 31 Jun 28 07:19:39 ds4k kernel: device-mapper: 4.5.0-ioctl (2005-10-04) initialised: dm-devel@redhat.com Jun 28 07:19:39 ds4k kernel: device-mapper: dm-multipath version 1.0.4 loaded ===== Failover Test ===== Hierzu wird die LUN über den eindeutigen ''/dev/mapper/''-Devicenamen gemountet und sowohl gelesen als auch geschrieben. Während dieses Vorgangs wird zunächst der eine Pfad entfernt und anschließend wieder angeschlossen und dann der andere Pfad entfernt und wieder angeschlossen. ==== Test HBA1 ==== Ziehen des FibreChannel Kabels von HBA1 und anschließendes wiederaufstecken. Nachdem der ''LOOP DOWN'' erkannt wird setzt der ''multipathd'' den einen Pfad auf ''failed faulty''. Sobald der Link wieder vorhanden ist, wechselt der Status von ''failed ready'' auf ''active ready''. Jetzt sind alle zwei Pfade wieder vorhanden. === HBA1: multipath -ll === 16:04:53 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:04:55 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [failed][faulty] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:06:06 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [enabled] \_ 2:0:0:0 sdb 8:16 [failed][faulty] \_ round-robin 0 [active] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:06:48 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [enabled] \_ 2:0:0:0 sdb 8:16 [failed][faulty] \_ round-robin 0 [active] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:06:51 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [enabled] \_ 2:0:0:0 sdb 8:16 [failed][ready] \_ round-robin 0 [active] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:06:58 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [enabled] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:01 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][faulty] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:04 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] === HBA1: syslog === Jun 28 16:04:55 ds4k kernel: qla2300 0000:04:02.0: LIP reset occured (f8ef). Jun 28 16:04:56 ds4k kernel: qla2300 0000:04:02.0: LOOP DOWN detected (2). Jun 28 16:05:55 ds4k kernel: SCSI error : <2 0 0 0> return code = 0x10000 Jun 28 16:05:55 ds4k kernel: end_request: I/O error, dev sdb, sector 135849247 Jun 28 16:05:55 ds4k kernel: end_request: I/O error, dev sdb, sector 135849255 Jun 28 16:05:55 ds4k kernel: device-mapper: dm-multipath: Failing path 8:16. Jun 28 16:05:55 ds4k kernel: SCSI error : <2 0 0 0> return code = 0x10000 ... Jun 28 16:05:55 ds4k kernel: end_request: I/O error, dev sdb, sector 269927031 Jun 28 16:05:55 ds4k kernel: SCSI error : <2 0 0 0> return code = 0x10000 Jun 28 16:05:55 ds4k multipathd: 8:16: tur checker reports path is down Jun 28 16:05:55 ds4k kernel: end_request: I/O error, dev sdb, sector 269929095 Jun 28 16:05:55 ds4k multipathd: checker failed path 8:16 in map 3600a0b80001773890000001234567890 Jun 28 16:05:55 ds4k multipathd: 3600a0b80001773890000001234567890: remaining active paths: 1 Jun 28 16:05:55 ds4k kernel: end_request: I/O error, dev sdb, sector 269929103 Jun 28 16:06:02 ds4k kernel: SCSI error : <2 0 0 0> return code = 0x10000 ... Jun 28 16:06:02 ds4k kernel: end_request: I/O error, dev sdb, sector 270066631 Jun 28 16:06:02 ds4k kernel: end_request: I/O error, dev sdb, sector 270066639 Jun 28 16:06:08 ds4k multipathd: 3600a0b80001773890000001234567890: switch to path group #1 Jun 28 16:06:50 ds4k kernel: qla2300 0000:04:02.0: LIP occured (f8ef). Jun 28 16:06:50 ds4k kernel: qla2300 0000:04:02.0: LOOP UP detected (2 Gbps). Jun 28 16:06:59 ds4k multipathd: 8:16: tur checker reports path is up Jun 28 16:06:59 ds4k multipathd: 8:16: reinstated Jun 28 16:06:59 ds4k multipathd: 3600a0b80001773890000001234567890: remaining active paths: 2 Jun 28 16:06:59 ds4k multipathd: 3600a0b80001773890000001234567890: switch to path group #1 Jun 28 16:06:59 ds4k multipathd: 3600a0b80001773890000001234567890: switch to path group #1 ==== Test HBA2 ==== Ziehen des FibreChannel Kabels von HBA2 === HBA2: multipath -ll === 16:07:06 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:08 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:10 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:12 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:14 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:17 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:19 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:21 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:23 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:26 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:28 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:31 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:33 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:36 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:38 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:41 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:43 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] 16:07:46 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [failed][faulty] 16:08:47 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [failed][faulty] 16:09:13 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [failed][faulty] 16:09:16 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [failed][ready] 16:09:26 3600a0b80001773890000001234567890 [size=407 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [enabled] \_ 3:0:0:0 sdc 8:32 [active][ready] === HBA2: syslog === Jun 28 16:07:46 ds4k kernel: qla2300 0000:05:01.0: LIP reset occured (f8e4). Jun 28 16:07:47 ds4k kernel: qla2300 0000:05:01.0: LOOP DOWN detected (2). Jun 28 16:08:45 ds4k kernel: device-mapper: dm-multipath: Failing path 8:32. Jun 28 16:08:45 ds4k multipathd: 8:32: tur checker reports path is down Jun 28 16:08:45 ds4k multipathd: checker failed path 8:32 in map 3600a0b80001773890000001234567890 Jun 28 16:08:45 ds4k multipathd: 3600a0b80001773890000001234567890: remaining active paths: 1 Jun 28 16:09:15 ds4k kernel: qla2300 0000:05:01.0: LIP occured (f8e4). Jun 28 16:09:15 ds4k kernel: qla2300 0000:05:01.0: LOOP UP detected (2 Gbps). Jun 28 16:09:25 ds4k multipathd: 8:32: tur checker reports path is up Jun 28 16:09:25 ds4k multipathd: 8:32: reinstated Jun 28 16:09:25 ds4k multipathd: 3600a0b80001773890000001234567890: remaining active paths: 2 ====== dmsetup info ====== Name: 3600a0b80001773890000001234567890 State: ACTIVE Tables present: LIVE Open count: 1 Event number: 1 Major, minor: 253, 5 Number of targets: 1 UUID: mpath-3600a0b80001773890000001234567890 Name: 3600a0b80001773890000001234567890p1 State: ACTIVE Tables present: LIVE Open count: 1 Event number: 0 Major, minor: 253, 8 Number of targets: 1 UUID: part1-mpath-3600a0b80001773890000001234567890 bzw. 3600a0b80001773890000001234567890:253:5:L--w:1:1:8:mpath-3600a0b80001773890000001234567890 3600a0b80001773890000001234567890p1:253:6:L--w:1:1:0:part1-mpath-3600a0b80001773890000001234567890 3600a0b800017738900000008437c0944 (253, 5) ~~UP~~