Read binary data (image) from separated file

Question

I have a file which has records for employees data and images. Each record for one employee and his data, his image, and his wife image. I can't change the file structure

There are separators between text data and images.

Here a sample of one record:

record number D01= employee name !=IMG1= employee image ~\IMG2= wife image ^! \r\n

(D01= & !=IMG1= & ~\IMG2= & ^!) are the separators

This is the code how the file was written:

FileStream fs = new FileStream(filePath, FileMode.Create);
StreamWriter sw = new StreamWriter(fs, Encoding.UTF8);
BinaryWriter bw = new BinaryWriter(fs);

sw.Write(employeeDataString);
sw.Write("!=IMG1=");
sw.Flush();

bw.Write(employeeImg, 0, employeeImg.Length);
bw.Flush();

sw.Write(@"~\IMG2=");
sw.Flush();

bw.Write(wifeImg, 0, wifeImg.Length);
bw.Flush();

sw.Write("^!");
sw.Flush();

sw.Write(@"\r\n");
sw.Flush();

So how to read that file?

Answer 1

There many kinds of files; the three most common ways to store records are

• Fixed size records, ideally with fixed size fields. Very simple to implement random access.
• Tagged files with tags and data interwoven. A bit complicated, but highly flexible and still rather efficiently readable, since the tags hold the positions and lengths of the data.
• And then there are Separated files. Always a pain.

Two issues:

• You must be sure that the separators are never in the data. Not 100% possible when you have binary data like images..
• There is no efficient way to access individual records..

Ignoring the 1st issue, here is a piece of code that will read all records into a list of class ARecord.

FileStream fs;
BinaryReader br;
List<ARecord> theRecords;

class ARecord
{
    public string name { get; set; }
    public Image img1 { get; set; }
    public Image img2 { get; set; }
}

int readFile(string filePath)
{
    fs = new FileStream(filePath, FileMode.Open);
    br = new BinaryReader(fs, Encoding.UTF8);

    theRecords = new List<ARecord>();
    ARecord record = getNextRecord();
    while (record != null)
    {
        theRecords.Add(record);
        record = getNextRecord();
    }
    return theRecords.Count;
}

ARecord getNextRecord()
{
    ARecord record = new ARecord ();

    MemoryStream ms;
    System.Text.UTF8Encoding enc = new System.Text.UTF8Encoding();
    byte[] sepImg1 = enc.GetBytes(@"!=IMG1=");
    byte[] sepImg2 = enc.GetBytes(@"~\IMG2=");
    byte[] sepRec = enc.GetBytes(@"^!\r\n");

    record.name = enc.GetString(readToSep(sepImg1));

    ms = new MemoryStream(readToSep(sepImg2));
    if (ms.Length <= 0) return null;             // check for EOF
    record.img1 = Image.FromStream(ms);

    ms = new MemoryStream(readToSep(sepRec));
    record.img2 = Image.FromStream(ms);

    return record;
}

byte[] readToSep(byte[] sep)
{
    List<byte> data = new List<byte>();
    bool eor = false;
    int sLen = sep.Length;
    int sPos = 0;
    while (br.BaseStream.Position < br.BaseStream.Length && !eor )
    {
        byte b = br.ReadByte();
        data.Add(b);
        if (b != sep[sPos]) { sPos = 0; }
        else if (sPos < sLen - 1) sPos++; else eor = true;
    }
    if (data.Count > sLen ) data.RemoveRange(data.Count - sLen , sLen );
    return data.ToArray();
}

Notes:

• There is no error checking whatsoever.
• Watch those separators! is the @ really right??
• Expanding the code to create the record number is left to you