0

Is there capacity within amazon/centos/linux to switch the ordering round of nitro disks?

I have an ami which consistently has devices in the incorrect order, by this I mean nvme1n1 and nvme2n1 should be switched round. If I run nvme id-ctrl -v /dev/nvme1n1 | grep sn I get a different serial number back following a reboot. I know they're "wrong" as the serial numbers are not reflective of their capacity... Hope that makes sense (I appreciate it's a bit confusing). This only ever occurs on servers with two or more disks; upon a reboot the disks are "correct"

My question is, is there a method of forcing the nvme device to disconnect and reconnect (in the hope that the mapping works as expected in the correct order).

Thanks guys

keeer
  • 783
  • 1
  • 5
  • 11

2 Answers2

0

Amazon Linux version 2017.09.01 and later contains scripts and a udev rule that automatically maps NVMe devices to /dev/xvd?. It is very briefly mentioned in the documentation, but there is not much information there.

You can obtain a copy by launching the Amazon Linux AMI, but there are also other places on the web where they have been posted. For example, I found this gist.

stefansundin
  • 2,826
  • 1
  • 20
  • 28
  • Yeah I managed to get something similar working, but it wasn't without its own limitations; namely that it's just a symbolic link rather than a block device and I have code which work on block devices. – keeer Jul 22 '20 at 13:08
0

Very simple in the end:

echo 1 > /sys/bus/pci/devices/$(readlink -f /sys/class/nvme/nvme1 | awk -F "/" '{print $5}')/remove
echo 1 > /sys/bus/pci/devices/$(readlink -f /sys/class/nvme/nvme2 | awk -F "/" '{print $5}')/remove
echo 1 > /sys/bus/pci/rescan
keeer
  • 783
  • 1
  • 5
  • 11