4

I am learning C++ specially about binary file structure/manipulation, and since I am totally new to the subject of binary files, bits, bites & hexadecimal numbers, I decided to take one step backward and establish a solid understanding on the subjects.

In the picture I have included below, I wrote two words (blue thief) in a .txt file.

enter image description here

The reason for this, is when I decode the file using a hexeditor, I wanted to understand how the information is really stored in hex format. Now, don't get me wrong, I am not trying to make a living out of reading hex formats all day, but only to have a minimum level of understanding the basics of a binary file's composition. I also, know all files have different structures, but just for the sake of understanding, I wanted to know, how exactly the words "blue thief" and a single ' ' (space) were converted into those characters.

One more thing, is that, I have heard that binary files contain three types of information:

header, ftm & and the data! is that only concerned with multimedia files like audios, videos? because, I can't seem to see anything, other than what it looks like a the data chunk in this file only.

blue thief
  • 113
  • 3
  • 14
  • What's the question? You've hit on the distinction between text and binary files. A text file like your example contains just the characters you see in Notepad--no more, no less. A binary file, on Windows systems, is anything that isn't a text file. (On Unix the distinction gets more fuzzy...) It doesn't have to contain a header identifying what sort of format the data is encoded in, but most do. The header can vary tremendously in size and structure. – dodgethesteamroller Jun 17 '13 at 20:51
  • @dodgethesteamroller I just wanted to know how the letters were converted, so I can understand how text-hex conversion works and possible, more if there is anything more to know in the notepad – blue thief Jun 17 '13 at 20:56
  • OK, I see. :) Expanding my comment into an answer... – dodgethesteamroller Jun 17 '13 at 21:03
  • @dodgethesteamroller that's more like it. While, you are at it. revoke the dv perhaps :) ? – blue thief Jun 17 '13 at 21:05
  • Sorry, I'm not the person who downvoted you, but I can cancel it out to 0... – dodgethesteamroller Jun 17 '13 at 21:17

1 Answers1

6

The characters in your text file are encoded in a Windows extension of ASCII--one byte for each character that you see in Notepad. What you see is what you get.

Generally, a hard distinction is made between text and binary files on Windows systems. On Unix/Linux systems, the distinction is fuzzier... you could argue that there is no distinction, in fact.

On Windows systems, the distinction is enforced by file extensions. All files with the extension ".TXT" are assumed to be text files (i.e., to contain only hex codes that represent visible onscreen characters, where "visible" includes whitespace).

Binary files are a whole different kettle of fish. Most, as you mention, include some sort of header describing how the data that follows is encoded. These headers can vary tremendously in size depending on the type of data (again, assumed to be indicated by the extension on Windows systems as well as Unix). A simple example is the WAV format for uncompressed audio. If you open a WAV file in your hex editing program, you'll see that the first four bytes are "RIFF"--this is a marker, often called a "magic number" even though it is readable as text, indicating that the contents are an audio file. Newer versions of the WAV specification have complicated this somewhat, but originally the WAV header was just the "RIFF" tag plus a dozen or so bytes indicating the sample rate of the following data. (You can see this by comparing the raw data in a track on an audio CD to the WAV file created by ripping an uncompressed copy of that track at 44.1 KHz--the data should be the same, with just a header section added at the start of the WAV file.)

Executable files (compiled programs) are a special type of binary file, but they follow roughly the same scheme of a header followed by data in a prescribed format. In this case, though, the "data" is executable machine code, and the header indicates, among other things, what operating system the file runs on. (For example, most Linux executables begin with the characters "ELF".)

dodgethesteamroller
  • 1,272
  • 9
  • 12
  • Impressive. You just more or less, described what I wanted to know. One final question though. How can I learn more about this? Which link would you recommend that could be helpful to me to understand about this? – blue thief Jun 17 '13 at 21:29
  • @bluethief Depends what direction you want to go. For more about file extensions and different ways of encoding data, [this Wikipedia list of file formats](http://en.wikipedia.org/wiki/List_of_file_formats) looks like a good start. For more about character encodings in text files, first follow the link above for ASCII, then read about [Unicode](http://en.wikipedia.org/wiki/Unicode). If you are interested in how these representations work at a hardware level, you might read about [binary](http://en.wikipedia.org/wiki/Binary_code) and [disk drives](http://en.wikipedia.org/wiki/Disk_storage). – dodgethesteamroller Jun 17 '13 at 21:46