RHEL 6 iSCSI LUNs

Question

I have a RHEL system set up as an iSCSI target with 2 luns shared out via one target via the following configuration in /etc/tgt/targets.conf (which I built following the commented out examples in /etc/tgt/targets.conf):

<target iqn.2014-04.com.local.box:ESXi.target1>
        backing-store /dev/ESXi_VG/ESXi_LUN_01 #LUN01
        backing-store /dev/ESXi_VG/TEST_LUN_01 #LUN02
</target>

These are intended to be shared out to a pair of ESXi hosts as a Datastore.

I originally created a single 4.00 TB LVM logical volume on the vg ESXi_VG for the datastore; however I realized I'd need a second LUN for Iops testing via RDM to a vm stored on local storage on the ESXi host. To that end I created a second 5.00 GB LVM logical volume on the same volume group and added the LUN entry for the second LUN in the target configuration I showed above. I then unmounted the 4.00 Tb LUN on my ESXi host, and restarted tgtd.

I'm seeing some weird stuff now. First, Here is the output from a tgt-admin -s command to show current LUNs:

[root@box ~]# tgt-admin -s
Target 1: iqn.2014-04.com.local.box:ESXi.target1
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 5369 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            Backing store type: rdwr
            Backing store path: /dev/ESXi_VG/TEST_LUN_01
            Backing store flags:
    Account information:
    ACL information:
        ALL

As you can see, only 1 LUN is displaying, not 2 - and the LUN that is displaying is only the 5.00 GB LUN that I added at a later date. If I remove or comment out the LUN entry it only shows LUN 0, the controller, and not LUN 1.

However, ESXi can see the 4.00 TB LUN still and can mount it, format it, etc. what it can't see is the 5.00 GB partition, despite it being shared out via the same iSCSI target.

I have a couple questions as a result:

Does tgt-admin only show LUNs not currently connected to an initiator? If not why is my 4.00 TB datastore no longer showing?
How can I get the 2nd LUN to appear to ESXi so I can mount it to a VM via RDM for iops testing?

I have, multiple times - it turns out all that was needed to have the new LUN show up on the RHEL side was to run "tgt-admin --update ALL --force" as it was not reading the new configuration properly since the target was "in use" (even though I had the LUN unmounted, and even deleted it at one point). However, ESXi still does not see the 5.00 GB LUN when I rescan the HBAs. — Coxswain, Apr 24 '14 at 17:04
Many months after asking this question, I have found the issue. the tgtd service was not actually stopping when I performed a restart. The only reason I caught this was because I tried to delete my test lun and recreate it, hypothesizing that it was possible that the lun itself had an issue. When I tried to run lvremove the logical volume threw an error; I eventually discovered that the pid for an old tgtd session had grabbed the lun. I ran a kill -9 against the PID, reset the tgtd service, and can now see both of the LUNs I have created. — Coxswain, Aug 02 '14 at 04:56

score 2 · Accepted Answer · answered Aug 02 '14 at 05:07

Here is what I discovered that the problem was. After being called away from this project due to a corporate merger, it was dumped back on my plate and I decided to restart from scratch. This time I created a LUN, RH_ISCSI_LV_001, and shared it out via the iscsi daemon. I was able to mount and format it in vSphere.

Once again I unmounted the LUN and stopped tgtd to edit /etc/tgt/targets.conf to add a second test LUN which I would use for performance testing by mapping it to a guest using RDM. Once again when I started tgtd up, only 1 LUN was displayed.

I hypothesized it might be a problem with the LUN not being readable by tgtd after VMware formatted it, so I attempted to destroy it with lvremove. it threw an ioctl error (device-mapper: remove ioctl on failed: Device or resource busy) and wouldn't delete.

After some digging I ran the command lsof /dev/ESXi_VG/RH_ISCSI_LV_001 and discovered that pid 2251 was the blocking process. a quick ps -ef and I learned that tgtd had, at some point, not shut down cleanly, and as nothing prevented a second tgtd process from starting up (apparently) it retained "ownership" of the LUN and thus prevented each successive tgtd process I spawned from touching it. a quick kill -9 and service tgtd start later and both my LUNs were visible to ESXi.

RHEL 6 iSCSI LUNs

1 Answers1