Efficiently reading csv file in MQL4?

Question

I placed "AAPL.csv" into the MetaTrader terminal folder's Files subfolder (MQL4/Files) to be accessible by the EA. The structure of this csv is as follows:

Date,Open,High,Low,Close,Adj Close,Volume
1980-12-12,0.1283479928970337,0.1289059966802597,0.1283479928970337,0.1283479928970337,0.10092189908027649,469033600
1980-12-15,0.12221000343561172,0.12221000343561172,0.12165199965238571,0.12165199965238571,0.09565676748752594,175884800

I want to read this, as well as many other similar csv files. All files have different lengths. My question is, what is the best practice when reading variable-length files? For now, I managed to read the content of my file by creating a 2-dimensional array:

string s[7][1000000];

although this is poor programming (what if the file only has 500 rows?) and it can still fail if I encounter a csv that is longer (what if file has 1000001 rows?). I tried using a dynamic array:

string s[7][];

but it returns '[' - invalid index value error. Yet another idea I had to use the FileSize() function and allocate just-the-necessary amount of memory to the 2-dimensional array. However,

int handle=FileOpen(FileName,FILE_CSV|FILE_READ,",");  
if(handle>0)      
{        
   int size = FileSize(handle);      
...

yielded a size that equals the product of column numbers and row numbers. I was hoping to obtain a row_count and col_count and use them to define s:

string s[col_count][row_count];

My full working code:

extern string FileName = "AAPL.csv";

int init()
  {
   int row=0,col=0;
   string s[7][1000000]; 

   ResetLastError();
   int handle=FileOpen(FileName,FILE_CSV|FILE_READ,",");
   if(handle>0)
      {
        while(True)
        {
          string temp = FileReadString(handle);
          if(FileIsEnding(handle)) break; //FileIsEnding = End of File
          s[col][row]=temp;
          if(FileIsLineEnding(handle)) //FileIsLineEnding = End of Line
          {
            col = 0; //reset col = 0 for the next row
            row++; //next row
          }
          else
          {
            col++; //next col of the same row
          }
        }
        FileClose(handle);
      }
      else
      {
        Comment("File "+FileName+" not found, the last error is ", GetLastError());
      }  
   return(0);
  }

int start()
  {
   return(0);
  }

PaulB · Answer 1 · 2023-07-07T07:52:41.720

You should use the first dimension of the array for your rows and the second dimension for your columns. You would therefore have an array statement of string s[1][7];

You can then resize your array as you loop through reading your csv file as follows:

string s[1][7];

int handle=FileOpen(FileName,FILE_CSV|FILE_READ,",");
if(handle==INVALID_HANDLE) Print("Error opening file ",GetLastError());
else
{
   int row=0;
   while(!FileIsEnding(handle))
   {
      if(row>0) ArrayResize(s,row+1);
      //carry out array reading here
      //eg s[row][0]=FileReadString(handle);
      //eg s[row][1]=FileReadString(handle);
      //etc
      row++;
   }
   FileClose(handle);
}

You can not resize the 2nd dimension of a multi dimensional array.

score 0 · Answer 2 · answered Aug 25 '23 at 00:25

For what it's worth, dingmaotu has published mql4-lib at GitHub. This API offers a number of useful MQL classes and functions, including support for file I/O in MQL4.

The README file provides an example of writing a CSV file with the API. It works - tested locally for CSV output from custom indicator data.

Efficiently reading csv file in MQL4?

2 Answers2