Fastest way to read 2mb binary file in decimal or hex?

Question

im trying to read a binary file into my own hex editor using a datagridview.

this was my initial approach

FileStream RawD = new FileStream(ECUFileName, FileMode.Open, FileAccess.Read);
BinaryReader RawB = new BinaryReader(RawD);

then i've tried just the binary reader as well...

now i'm using

MemoryMappedFile RawD = MemoryMappedFile.CreateFromFile(ECUFileName);
MemoryMappedViewStream stream = RawD.CreateViewStream();
BinaryReader RawB = new BinaryReader(stream);

all i want the code to do is read decimals and put it in a datagridview. a 2mb file takes forever.

while (RawB.BaseStream.Position <= RawB.BaseStream.Length)
{
    if (term == true) break;
    DataGridViewRow row = (DataGridViewRow)DataG.Rows[0].Clone();
    DataG.Rows.Add(row);
    for (int P = 0; P < I; P++)
    {
        if (RawB.BaseStream.Position == RawB.BaseStream.Length) break;
        Int16 Byte = RawB.ReadInt16();
        string ByteStr = string.Format("{0}",Byte);
        DataG[P, X].Value = ByteStr;
        DataG[P, X].ReadOnly = true;
        string ADR = string.Format("{0:x6}", X * 10); ;
        DataG.Rows[X].HeaderCell.Value = ADR;
        DataG.FirstDisplayedScrollingRowIndex = DataG.FirstDisplayedScrollingRowIndex + 1;
        if (term == true) break;
    }

i've just tried this, seems a bit quicker

byte[] bytes = File.ReadAllBytes(ECUFileName);
for (int i=0; i <= bytes.Length; i++)
{
    string result = Convert.ToString(bytes[i]);
    richTextBox1.AppendText(result);
}

be hex editor cant be used for what i intend to do, so i have to manually do it.

i have read tons of articles where people say read AllBytes with the file method, to have the file stored in memory, when i tried the file method, it didn't work fast as well.

Your slowness is coming from working with the grid, not reading the file. — SLaks, Feb 18 '15 at 18:26
You should look at: http://stackoverflow.com/questions/10226992/slow-performance-in-populating-datagridview-with-large-data — itsme86, Feb 18 '15 at 18:28
And is your MemoryMapped approach any faster? (Seems unlikely). — H H, Feb 18 '15 at 18:41
Have you tried loading the data into a DataTable and using that as the DataSource for the DataGridView? — Andrew Morton, Feb 18 '15 at 18:55
I can't get it faster then ~10 sec for a 2MB file. You'll have to paginate or use VirtualMode. — Hans Passant, Feb 18 '15 at 19:24
Naming a variable of type `Int16` as "Byte" may not be the best thing to do ;) — Andrew Morton, Feb 18 '15 at 21:00

score 1 · Accepted Answer · answered Feb 18 '15 at 20:58

I placed a DataGridView from the toolbox onto a Form and used this code:

static void Demo(DataGridView dgv)
{
    string src = @"C:\temp\currentData.txt";
    var data = File.ReadAllBytes(src);
    int nCols = 16; //number of columns to use in the DGV
    DataTable dt = new DataTable();

    for (int i = 0; i < nCols; i++)
    {
        dt.Columns.Add(new DataColumn { DataType = Type.GetType("System.Int16") });
    }

    var lastIndex = data.Length - nCols - 1;
    for (int i = 0; i < lastIndex; i += nCols * 2)
    {
        var dr = dt.NewRow();

        for (int j = 0; j <= nCols * 2 - 1; j += 2)
        {
            dr[j / 2] = BitConverter.ToInt16(data, i + j);
        }

        dt.Rows.Add(dr);
    }


    dgv.DataSource = dt;

}

private void Form1_Load(object sender, EventArgs e)
{
    Demo(dataGridView1);

}

Which showed the populated DGV in less than a second. (Using an old i7 920.) Any stupidity in the code (e.g. off-by-one errors) is because I manually translated it from VB.NET. As it is, it does not show the last row of data if it is not a complete row. I'm only demonstrating that it is faster this way. Also, you should make suitable adjustments if the file is not an even number of bytes in length.

The important part which makes it faster is that I populate a DataTable to use later as the DataSource for the DGV.

The display of larger files would probably be better done with VirtualMode (as suggested by Hans Passant).

Fastest way to read 2mb binary file in decimal or hex?

1 Answers1