2

I have 2 identical Dell R720 servers running identical Oracle Enterprise Linux(RHEL)6.4.

Both servers (supposedly) configured in exactly the same way. However, one of the servers is behaving differently.

Every other reboot its local HDD name(and related partitions) flip from /dev/sda to /dev/sdj. This is problematic because this server is configured with multipathd, and if this flip happens this config does not match other server and Oracle DB(or its clusterware) complains that nodes are not configured identically.

Why does one server has a consistent device names while the other server keeps flipping back and forth?

How can I make local hdd to consistently be /dev/sda?

edit: I created a rules file in /etc/udev/rules.d with the following rule, but it doesn't seem to be working

KERNEL=="sd*" , PROGRAM="scsi_id --page=0x83 -–whitelisted --device=/dev/%k",RESULT=="36b8ca3a0e58a3f00195c25c8117a6822", NAME="sda"
Mxx
  • 2,362
  • 2
  • 28
  • 40
  • Try this: `KERNEL=="sd*", PROGRAM="scsi_id --page=0x83 -–whitelisted --device=/dev/%k", RESULT=="36b8ca3a0e58a3f00195c25c8117a6822", NAME="foo%n"` – Petter H Jul 08 '13 at 18:42
  • http://fritshoogland.wordpress.com/2012/07/23/using-udev-on-rhel-6-ol-6-to-change-disk-permissions-for-asm/ – Petter H Jul 08 '13 at 19:13
  • check your driver load order in your initrd and check modprobe.conf – c4f4t0r Jan 05 '14 at 00:06

4 Answers4

9

Better than fighting with udev to force a device name for a given device, a permanent solution is to use UUIDs. This is valid for any device known to the device-mapper.

This way, you don't have to worry if you add extra disks to your host. The UUID identifier guaranties that the right device will be used.

dawud
  • 15,096
  • 3
  • 42
  • 61
  • It is always best to use UUIDs for this very reason... To find the UUID of the drive in question use: blkid -p /dev/ – Atari911 Jun 27 '13 at 20:19
  • use `UUIDs` where? multipath is using `wwid`. Given a choice, I'd rather fight `udev` than Oracle. – Mxx Jun 27 '13 at 20:37
  • "`dmsetup` reports a UUID for partitions on multipath devices in the form: part-mpath-". Also, multipath devices are on top of the `/dev/name` devices. What I'm suggesting is to use UUIDs for the underlying devices, to avoid multipath failures, in other words, fix it at the bottom of the storage stack. – dawud Jun 27 '13 at 20:41
  • I still don't understand where exactly I should use UUID. multipath.conf uses `wwid`s. I just specify wwid and it automatically creates all the relevant mappings. – Mxx Jun 27 '13 at 21:00
  • 2
    http://www.centos.org/docs/5/html/5.2/Virtualization/sect-Virtualization-Virtualized_block_devices-Configuring_persistent_storage_in_a_Red_Hat_Enterprise_Linux_5_environment.html – GioMac Jun 27 '13 at 21:41
2

Use aliases, or user_friedly_names or do it with udev. All of these are documented in RHEL Storage Administration Guide:

Maybe you could use /dev/disk/* tree instead of mapped /dev/sd*

https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/persistent_naming.html

http://www.centos.org/docs/5/html/5.2/Virtualization/sect-Virtualization-Virtualized_block_devices-Configuring_persistent_storage_in_a_Red_Hat_Enterprise_Linux_5_environment.html

GioMac
  • 4,544
  • 4
  • 27
  • 41
  • I am using `aliases` in multipath.conf. But if you'll see sample in your 2nd link, multipath config doesn't reference to any /dev/* devices. Yet `multipath -ll` for each path shows `/dev/sd*` device... – Mxx Jun 28 '13 at 00:53
0

if you're using multipathd, then you should not have to care weather the disks name is /dev/sdaor /dev/sdj. Use the device name created with multipathd shown with multipath -ll. That name is persistent. The /dev/sd* names only refer to a single path...

Petter H
  • 3,443
  • 1
  • 16
  • 19
  • I do not worry about the multipath part, that thing works fine. I care about local hdd changing its name... – Mxx Jul 05 '13 at 20:00
  • OK but then what about using UUID or writing an udev rule like other answers suggest? you can write a simpler one then the one you have, or follow dawuds answer. You can find the disks under `/dev/disk/by-uuid/`. Just use that path instead, it is persistent. – Petter H Jul 05 '13 at 20:04
  • Again, like in the other answers, WHERE can i use UUID? How much simpler can I write that rule? It checks for wwid and tries to set device name. Yes, I can I can use those other paths, but software specifically checks for for /dev/sda because that is how other servers are already configured. – Mxx Jul 06 '13 at 00:01
  • OK I dont get it why you'd need the same name on the *local* disk, just because the other services are configured like that but if that is somehow a requirement for you then fine; use UDEV; UUID won't work. As for the where of it, how do you expect someone other than yourself to know *where* to use it? wherever you use `/dev/sda` today would be my bet. Anyway if you have to have the same device name on all your servers, then make an udev rule. If you can't get it working, debug it. – Petter H Jul 08 '13 at 15:03
  • This is a requirement for Oracle's clusterware pre-install check software. It requires for all nodes to have identical configuration, including drive volumes. – Mxx Jul 08 '13 at 15:20
  • have you tried setting another name than "sda", because chanses are that by the time ur rule is read, sda have been created already. – Petter H Jul 08 '13 at 18:06
  • I have not tried that, and it is possible that sda is already in use. :( However, ultimately it has to be sda because there are 3 other servers in this cluster that already use sda. – Mxx Jul 08 '13 at 19:31
  • make the rule run early then by setting low nr for it first in the name, byt make sure first with udevadm test, that your rule is otherwise OK... – Petter H Jul 09 '13 at 14:17
-2

Never use raw disk partitioning. Use LVM. You will end up with the very same configuration on both hosts, regardless of the UUID/WWID disk naming schema.

user204341
  • 1