Read and Write to Binary File?

Question

I am saving some custom created Objects classes (Stream data) to File.

I need to be able to load the contents of the File into a TStringList so I can append a new line to the end of the File, and then Save the changes.

This is not working though because LoadFromFile cannot seem to parse the File correctly. I assume because of the funny characters the Stream Saves to File as, and TStringList expects plain textual information.

How can I do the following:

Read any Raw Binary File into a TStringList.
Add my new Line, eg StringList1.Add(MyString);
Save the Raw Binary File again.

This question actually relates to another question I asked: Save a CRC value in a file, without altering the actual CRC Checksum?

This is what I am trying to do:

Calculate the CRC Checksum of my Saved Stream File.
Add the CRC Value to the end of the File.
Re-save the File.

Then when I attempt to Open my Stream File:

Assign the CRC Value (at the end of the File) to a variable.
Delete the CRC Value from the File.
Save the Stream File as a new Temp File.
Calculate and Compare the CRC of the Temp File, with the CRC stored in the variable.
If the CRC of the File matches the internal stored CRC Value, I can process the File as normal.

But I don't know how to Read or Write the Raw Binary Data of the File.

I would be grateful if someone could give me some help and advice thanks :)

You are saving a checksum into a text file? Is that correct? I still don't understand why you need the checksum. — David Heffernan, Dec 23 '11 at 15:46
Saving to the Stream File, I am doing this so that whenever a File is opened from my Application I can determine if in fact the File was originally saved from my program. — , Dec 23 '11 at 16:04
Do you want the file to be readable in a text editor? FWIW your current approach is exceedingly wasteful with spurious read and writes. — David Heffernan, Dec 23 '11 at 16:08
What version of Delphi are you using? It matters in particular for Unicode strings (or not) — David Heffernan, Dec 23 '11 at 16:12
This doesn't make much sense. Maybe you should actually state more of what you actually intend to do with these "binary files". Personally I don't think you should be using a TStringList to hold binary data. How do you intend to decide what a "newline" (carriage return + linefeed" equivalent is? Or are you going to load one block of data (say 1024 bytes) per item in the BinaryStringList? — Warren P, Dec 23 '11 at 16:36
I am using Delphi XE. The File is saved as a Stream and as such unreadable which is fine. I don't need to edit the File outside my program, quite the opposite. I just need to add a line containing the CRC of the File to end, but this isn't working because I cannot read the File. I think I am confusing it all again and making a difficult situation out of something simple. — , Dec 23 '11 at 16:49
It is my opinion that you are making life needlessly hard for yourself. I cannot imagine why you would want to make it hard for you to read and write files. Hard for user to automate the app. And risky in case you need to change file formats. If you have to do something like this put it in another file with a base name that matches the text file. You are over-complicating and trying to solve problems that don't exist. — David Heffernan, Dec 23 '11 at 17:05
If I try to open the File that was saved from my program and the file is corrupt (eg, the stream in the file has been broken) the program still tries to open the stream file even though it cannot, it might say Out of Memory error for example, or sometimes nothing at all. Try..Except block does not seem to realise the file cannot be opened and doesn't raise an exception. So I thought I could include a CRC checksum value of the file internally and check the file integrity that way. But yes, it seems I am making it difficult for myself. — , Dec 23 '11 at 17:14
It would be much better to detect a corrupt file and report an error. If the user messes the file up then that's their problem. Store it under their user profile and there's no reason for them to mess around with it. — David Heffernan, Dec 23 '11 at 17:20
This is not a file I can just hide away in the temp directory, as it can be loaded from a TOpenDialog. I agree if anyone decides to mess up the file that is their own fault, but for some reason I cannot trap the error if the file (stream data) is corrupt - it almost always shows no error but tries to process it anyway. — , Dec 23 '11 at 17:49
I think I solved it, and for once by not creating more problems than needed! I was trying to catch the exception in my procedure that Opens the File, instead I created the try..exception block in the actual base class where I load the stream, if there is an exception I set a boolean flag to let me know if the file loaded ok. I check this flag in the Open procedure and can determine if the file was ok or not. seems to be working this way. I don't understand why I get confused and make problems more difficult :( — , Dec 23 '11 at 18:38
Craig; A free tip; Binary files are trouble. If you don't need to make custom binary file formats, then don't. Text files are much more "rugged and reliable", and easier to troubleshoot when malformed. — Warren P, Dec 23 '11 at 19:23
Warren, thanks for the tip. I may consider changing the way I save my objects to file, maybe XML would be a better choice. — , Dec 23 '11 at 20:05
@Craig XML would be one good choice. JSON is much more readable if you ever want a human to parse it. Personally my favourite is YAML but that requires a lot of effort to get into Delphi. Either JSON or XML are good choices. — David Heffernan, Dec 23 '11 at 20:13
@David I don't know anything about the File format JSON or YAML without reading up on it. I've been reading some more on XML and I can see it shouldn't be too much trouble implementing. I don't really like the way Streams save to File, and like you say at least something like XML would allow readability if desired. — , Dec 24 '11 at 21:16
I was thinking of using TXMLDocument, and saving my objects as XML nodes. — , Dec 26 '11 at 09:49

score 6 · Accepted Answer · answered Dec 23 '11 at 16:13

This class descends from TStringList and adds a check value at the end when writing to a file. This value is checked whenever the file is read.

type
  TCRCStringList = class(TStringList)
  type
    TCRC = LongWord;
  private
    function CalcCRC(Stream: TStream): TCRC;
  public
    procedure LoadFromStream(Stream: TStream; Encoding: TEncoding); override;
    procedure SaveToStream(Stream: TStream; Encoding: TEncoding); override;
  end;

function TCRCStringList.CalcCRC(Stream: TStream): TCRC;
begin
  Result := 42;  // place CRC calculation here
end;

procedure TCRCStringList.LoadFromStream(Stream: TStream; Encoding: TEncoding);
var
  crc: TCRC;
  temp: TMemoryStream;
begin
  temp := TMemoryStream.Create;
  try
    temp.CopyFrom(Stream, Stream.Size - Sizeof(crc));
    Stream.Read(crc, Sizeof(crc));
    if crc <> CalcCRC(temp) then
      raise Exception.Create('CRC error');
    temp.Position := 0;
    inherited LoadFromStream(temp, Encoding);
  finally
    temp.Free;
  end;
end;

procedure TCRCStringList.SaveToStream(Stream: TStream; Encoding: TEncoding);
var
  crc: TCRC;
  temp: TMemoryStream;
begin
  temp := TMemoryStream.Create;
  try
    inherited SaveToStream(temp, Encoding);
    temp.Position := 0;
    crc := CalcCRC(temp);
    temp.Position := temp.Size;
    temp.Write(crc, Sizeof(crc));
    Stream.CopyFrom(temp, 0); // count = 0 copies the whole stream from the beginning
  finally
    temp.Free;
  end;
end;

thanks for posting, you're code looks interesting. I have since resolved my original problem without the use of any CRC checking. I will accept your answer however as it seems very useful, I will have to look at it more closer to research it. Thanks — , Dec 23 '11 at 19:24

Read and Write to Binary File?

1 Answers1