Seeking large file in c++ on windows

Question

I'm writing a small utility to parse xfs filesystem on windows. For small size image <5GB my utility is working fine: I'm able to list all files and directories. But when I try to parse large xfs image > 30GB. It is giving wrong results. I'm using _fseeki64 and _ftelli64 for seeking and fread for reading particular block. One of thing I've notice is that _fseeki64 is not wroking properly. Below is my seek function which seek to particular group number and block number.

int FileSystemReadXFS::SeekToGroupBlock(uint16_t grpNum, uint64_t blockNum)
    {
        int error = -1;
        //Seek to beginning
        if(_fseeki64(m_fileSystemInfo.fp, (__int64)0, SEEK_SET) != 0)
        {
            PRINT_SEEK_ERROR;
            goto BAILOUT;
        }
        __int64 currPtr = 0;
        //Seek to destination group
        if(grpNum > 0)
        {
            if(_fseeki64(m_fileSystemInfo.fp, (__int64)(grpNum*m_fileSystemInfo.SizeOfBlockGroup*m_fileSystemInfo.BlockSize), SEEK_SET))
            {
                PRINT_SEEK_ERROR;
                    goto BAILOUT;
            }
             currPtr = _ftelli64(m_fileSystemInfo.fp);
        }


        //Seek to destination block in group
        if(blockNum > 0)
        {
            if(_fseeki64(m_fileSystemInfo.fp, (__int64)(blockNum*m_fileSystemInfo.BlockSize), SEEK_CUR))
            {
                PRINT_SEEK_ERROR;
                goto BAILOUT;
            }
            currPtr = _ftelli64(m_fileSystemInfo.fp);
        }
        error = 0;
BAILOUT:
        return error;
    }

However the above function is taking me to wrong postion. For example when I want to seek to group number = 2 with m_fileSystemInfo.SizeOfBlockGroup = 2043982 and m_fileSystemInfo.BlockSize = 4096.

I'm expecting currPrt = 2043982*4096*2 = 16744300544 (0x3E609C000), But _ftelli64 is returning (0xE609C000). Please suggest what could be going wrong. Also please advice what is the best way to deal with large files on windows in c++.

Update::

I found that the seekOffset is getting limited to 8154365952 (0x1e609c000) instead of actual value of 16744300544 (0x3e609c000) inspite of me using __int64.

So.

_int64 seekOff = (_int64)(grpNum*m_fileSystemInfo.SizeOfBlockGroup*m_fileSystemInfo.BlockSize) = 2*2043982*4096 is giving 8154365952 instead of 16744300544.

I'm not sure what could the reason as everything is in __int64.

score 4 · Answer 1 · answered Nov 15 '13 at 13:15

4

Apparently the problem was with calculation of seek offset. It was generating integer overflow. Hence I have to convert everything to __int64 even-though I'm working on 64 bits application. I was thinking compiler might be able to do this for me.

__int64 grpNum = 2;
__int64 sizeOfBlockGroup = 2043982;
__int64 blockSize = 4096;
__int64 seekOffSet = grpNum*sizeOfBlockGroup*blockSize;

which works fine with _fseeki64 and __ftelli64.

answered Nov 15 '13 at 13:15

praks411

1,972
16
23

In the memory model used for 64bit windows, pointers are 64bit but integers are 32bit. Also, you don't have to make all the values __int64, you just have to make sure that if A * B is bigger than 32bits then either A or B is __int64. – Stuart Nov 15 '13 at 22:51

score 2 · Answer 2 · answered Nov 15 '13 at 11:20

2

Your best bet is to use Win32 API directly and not go through the C RunTime.

Use CreateFile to open the file and SetFilePointerEx to seek

The functions you're calling will end up calling these APIs anyway. In Visual studio you have the CRT code so you can step into _fseeki64 and possibly see where it goes wrong.

answered Nov 15 '13 at 11:20

shoosh

76,898
55
205
325

I found that offset calculation of 2*2043982*4096 is getting limited to 8154365952 instead of actual value of 16744300544. I've used __int64 most of the places but it doesn't help. – praks411 Nov 15 '13 at 11:57

Seeking large file in c++ on windows

2 Answers2