ISCSI Becomes Much More Slower After An Intermediate Layer

Question

I'm new to iSCSI protocol and I have a task to catch and interpret the commands between QEMU and iSCSI server when a VM is boot from iSCSI drive.

I setup tgtd on Ubuntu and managed to install a guest OS into a disk image created by qemu-img with a format of raw, which is the backing storage of tgtd.

To achieve my target, I decided to write a transparent layer between QEMU and tgtd, which performs as a server for QEMU and a client for tgtd. After a brief reading of RFC3720, I wrote my transparent layer with the core code as

    while(1) {
        rt_val = recv(server_conn_sock, BHS, 48, 0);
        unsigned int TotalAHSLength = (unsigned int)BHS[4];
        unsigned int DataSegmentLength = ((unsigned int)BHS[5] << 16) | ((unsigned int)BHS[6] << 8) | (unsigned int)BHS[7];
        unsigned int DataSegmentLengthWithPadding = DataSegmentLength % 4 ? DataSegmentLength + 4 - (DataSegmentLength % 4) : DataSegmentLength;
        if(TotalAHSLength > 0) {
            rt_val = recv(server_conn_sock, AHS, 4 * TotalAHSLength, 0);
        }
        if(DataSegmentLengthWithPadding > 0) {
            rt_val = recv(server_conn_sock, DataSegment, DataSegmentLengthWithPadding, 0);
        }
        rt_val = send(client_sock, BHS, 48, 0);
        if(TotalAHSLength > 0) {
            rt_val = send(client_sock, AHS, 4 * TotalAHSLength, 0);
        }
        if(DataSegmentLengthWithPadding > 0) {
            rt_val = send(client_sock, DataSegment, DataSegmentLengthWithPadding, 0);
        }
    
        rt_val = recv(client_sock, BHS, 48, 0);
        TotalAHSLength = (unsigned int)BHS[4];
        DataSegmentLength = ((unsigned int)BHS[5] << 16) | ((unsigned int)BHS[6] << 8) | (unsigned int)BHS[7];
        DataSegmentLengthWithPadding = DataSegmentLength % 4 ? DataSegmentLength + 4 - (DataSegmentLength % 4) : DataSegmentLength;
        if(TotalAHSLength > 0) {
            rt_val = recv(client_sock, AHS, 4 * TotalAHSLength, 0);
        }
        if(DataSegmentLengthWithPadding > 0) {
            rt_val = recv(client_sock, DataSegment, DataSegmentLengthWithPadding, 0);
        }
        send(server_conn_sock, BHS, 48, 0);
        if(TotalAHSLength > 0) {
            send(server_conn_sock, AHS, 4 * TotalAHSLength, 0);
        }
        if(DataSegmentLengthWithPadding > 0) {
            send(server_conn_sock, DataSegment, DataSegmentLengthWithPadding, 0);
        }
    }

The code is basically forwarding everything between QEMU and tgtd. However, I found that the speed of QEMU to boot the guest OS is more than 100 times slower compared with booting with a direct connection to tgtd. My task required my intermediate layer not to affect the performance of QEMU.

So, what could be a possible reason for the difference in speed of reading disk?

ISCSI Becomes Much More Slower After An Intermediate Layer

0 Answers0