QT: Runtime error reading text file in QStringList

Question

with my application in Qt I'm currently trying to read a really big formatted text file (20.000.000 of lines) and collecting data in some QStringLists (plural).

Each lines generates 4 QStringList components of 10 characters each.

Always at the same point the application crashes with the "Runtume Error!".

I'm saying the same point because if I try to remove the last line read, I got the error at the next one. (I'm talking about crashing at the 16.000.000th line )

I'm not sure (nor so skilled...), but I think I sort of reached the memory capability, could it be the case? is there some work-around or some better way in doing that?

here is part of the code where I collect the data from the text file:

int lcCount=0;
QStringList loadcase, elid, plyid, eltyp, sigmax, sigmay, sigmaz, sigmaxy, sigmaxz, sigmayz;
QString currentLC, currentELtyp;
QStringList status; status<<"LCoff"<<"ELoff"<<"PLYoff";
QStringList lineList;

while(!in.atEnd())
{
    //PROGRESS BAR
    int pBarValue = countRow*100/countTotRow;
    //qDebug()<<countRow<<" - " << countTotRow;
    m_pBar->avanzamentoPCH(pBarValue);
    qApp->processEvents();

    QString line = in.readLine(); countRow++;
    if(line.contains("$TITLE")){status.replace(0,"LCoff");status.replace(1,"ELoff");}

    if (line.contains("$SUBCASE ID ="))
    {
        status.replace(0,"LCon");
        line.chop(30);
        line.remove(" ");
        line.remove("$SUBCASEID=");
        currentLC = line;
        lcCount++;
    } else if (line.contains("$ELEMENT TYPE =") && status.at(0)=="LCon")
    {
        line.chop(30);
        line.remove("$ELEMENT TYPE =");
        lineList = line.split(" ",QString::SkipEmptyParts);
        // creazione stringa del tipo di elemento
        currentELtyp = lineList.at(1);
        currentELtyp.chop(2);
        currentELtyp.prepend("C");

        if((lineList.at(1).contains("QUAD4LC")) || (lineList.at(1).contains("TRIA3LC")))
        {
            status.replace(1,"ELon");
        }
        lineList.clear();
    } else if(status.at(0)=="LCon" && status.at(1)=="ELon")
    {
        line.chop(8); // elimina gli ultimi 8 caratteri evitando possibili errori quando il numero delle righe del pch si attacca all'ultimo valore
        loadcase << currentLC;
        eltyp<<currentELtyp;

        lineList = line.split(" ",QString::SkipEmptyParts); //,QString::SkipEmptyParts
        elid << lineList.at(0);
        plyid << lineList.at(1);
        sigmax << lineList.at(2);
        sigmay << lineList.at(3);

        lineList.clear();

        line = in.readLine();countRow++;
        line.chop(8); // elimina gli ultimi 8 caratteri evitando possibili errori quando il numero delle righe del pch si attacca all'ultimo valore
        lineList = line.split(" ",QString::SkipEmptyParts);
        sigmaxy << lineList.at(1);
        sigmaxz << lineList.at(2);
        sigmayz << lineList.at(3);
        lineList.clear();
        //
        line = in.readLine();countRow++;
        line = in.readLine();countRow++;
        // solo per laminati!!
        sigmaz << "0.0";
    }
}

and here is an example of the text file: a part like this format goes on and on

650001                 1             -1.919393E+05      5.581818E+05       8
-CONT-                 -1.747859E+03     -8.388188E+01     -4.930236E+01       9
-CONT-                 -8.986650E+01      5.581859E+05     -1.919434E+05      10
-CONT-                  3.750646E+05                                          11

It's quite possible. Are you allocating the memory for your `QStringList` instances on the stack or on the heap? If on the stack, I would suggest switching to the heap. It would be really helpful if you give the code where your read the file and store it inside your lists so that we can exclude other possibilities. In addition since you think that it's related to your memory you can easily check your task manager to see what happens with your application plus provide us with the information of the size of the file (as in kilo/megabytes) and the memory your system has. — rbaleksandar, Sep 18 '17 at 16:20
I expect `QStringList` to allocate from the heap regardless. — drescherjm, Sep 18 '17 at 16:22
`QStringList` is a container. It will allocate memory dynamically as you add items to it. This allocation will be from the heap. — drescherjm, Sep 18 '17 at 16:30
I'm sorry guys but Idon't know what heap or stack are... my skills are quite limited. trying to answer you: the program crashes when it reaches 1.7 GB of memory usage. I'm adding code and text file example in the question above — piepolitb, Sep 18 '17 at 16:41
You may have reached a memory limit. I'd recommend running this in your debugger to see what the error is. — MrEricSir, Sep 18 '17 at 17:10
I would switch to 64 bit. You probably ran out of address space available to add any more data. Remember 2GB is the total user address space available to a 32 bit process on windows. There is a way to extend this limit to 3GB or 4GB under certain situations however its much easier to just build a 64 bit application. — drescherjm, Sep 19 '17 at 13:39
thank you drescherjim. How can I do that? I would give it a try and would like to learn something new... — piepolitb, Sep 20 '17 at 06:37

QT: Runtime error reading text file in QStringList

0 Answers0