Incremental Loading in QScintilla Object

Question

I am using a an object of QScintilla and I am reading the file in QScintilla Object incrementally.

Header myEditor.h

class myScintilla: public QScintilla {
     public readFile();
};

#include  "myEditor.h"
void myEditor::readFile() {

   if (FILE* fp = fopen(ofilename.toLatin1(), "r")) {
    QTextStream ts(fp, QIODevice::ReadOnly);
    int bufferSize =(1024* 1024)/2;
    do {
      QString s = ts.read(bufferSize);
      append(s);
    } while(!ts.atEnd());
}

Even after this change there will be still performance issue while reading large files. It took around

1) 25 seconds to read a file of size 1.5 GB. (Machine cores 4 , 16 GB RAM) 2 10 seconds of file of size 512MB (on same machine)

Is there any way we can load the file in QScintilla object incrementally based on movement of scrollbar?

Answer 1

I found your question interesting so did a little bit of Googling on your behalf. It seems to me that while Scintilla exposes this functionality via the Loader interface, in fact the QScintilla class does not. To make this work, it seems that what you would have to do is use the QScintillaBase class to send the SCI_CREATELOADER message to the Scintilla control.

Edit: Also, you do not want to use append in a loop. That will cause all sorts of terrible things to happen. It will likely force rendering, some sort of indexing, etc. Before using my suggestion above, I would suggest that you instead build up a gigantic QString in memory and then set that at the end. Better to pre-allocate. That might be a little faster.

FINAL ANSWER

Edit #2: OK, it was bothering me that such an industrial strength editor component like Scintilla did not support this natively but it seems that the right way to do this is by using a combination of features:

1. You start with a document allocated using SCI_ALLOCATE where the number of bytes is the size of your file
2. You listen for the SCN_UPDATEUI event
3. Then, based on where the user is scrolling to, you load that data

It should be straightforward to map the above to QScintillaBase as a test.