Writting a P6 .ppm in ASCII using ASCII/UTF8 data from Textbox. Encoding Confusion?

Question

Im having some general confusion with encoding on a little tool I'm writing.

First of all I apologise that the following code is a little butchered but of the code I have written so far, it's the closest to actually working.

If I use the following code:

/*create file*/
FileStream fileS = new FileStream(filename + ".ppm", FileMode.Create, FileAccess.ReadWrite, FileShare.None, 8, FileOptions.None);
/*create a binary writer*/
BinaryWriter bWriter = new BinaryWriter(fileS, Encoding.ASCII);

/*write ppm header*/
string buffer = "P6 ";
bWriter.Write(buffer.ToCharArray(), 0, buffer.Length);
buffer = width.ToString() + " ";
bWriter.Write(buffer.ToCharArray(), 0, buffer.Length);
buffer = height.ToString() + " ";
bWriter.Write(buffer.ToCharArray(), 0, buffer.Length);
buffer = "255 ";
bWriter.Write(buffer.ToCharArray(), 0, buffer.Length);

/*write data out*/
byte[] messageByte = Encoding.UTF8.GetBytes(ppmDataBox.Text);
bWriter.Write(messageByte, 0, messageByte.Length);

/*close writer and bWriter*/
bWriter.Close();
fileS.Close();

Then what I get is a file saved in UTF-8 format, if I open that file and re-save it as ASCII I get the PPM I expect.

However if I change the line:

 byte[] messageByte = Encoding.UTF8.GetBytes(ppmDataBox.Text);

to

 byte[] messageByte = Encoding.ASCII.GetBytes(ppmDataBox.Text);

Then I do get a file saved in ASCII format but the file is wrong, the colours are wrong and basically the data in the file does not match the data in the text box.

I am assuming that the textbox is in UTF-8 and the data I am pasting into it is actually ASCII format/characters and I first need to convert that ASCII into its corresponding UTF-8...(aka be the UTF-8 version of those characters). However if I'm totally honest this is my first venture into the world of encoding and I'm completely clueless. So please let me know if I'm talking rubbish.

Here is a sample of the kind of data i'm pasting into the text box:

ÿÿ ÿÿ ÿÿ ÿÿ aa aa aa ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿ

it is meant to be yellow with little black squares everywhere, but its coming out green and when the file is created in ASCII format the data ends up looking like this:

?? ?? ?? ?? aa aa aa ?? ?? ?? ??

Answer 1

ASCII is a 7-bit encoding (character values 0 thru 127). The ÿ character has a value greater than 127, the exact value depending on which encoding or code page is used. (In code page 1252 it has a value of 255). When the ASCII encoding tries to process a character with a value greater than 127, it just writes a question mark.

It looks like you need to map high ASCII characters (character values 128 thru 255) to single bytes. That rules out using the UTF8, UTF32 or UniCode encodings, since their GetBytes() methods will return multiple bytes for single character values greater than 127.

To map high ASCII characters to single bytes, try a code page like 1252 or 437. If those don't give the desired mapping, there are many other code pages listed here.

Here's an example using code page 1252:

using System;
using System.IO;
using System.Text;

namespace ConsoleApplication6
{
  public class Program
  {
    public static void Main(String[] args)
    {
      (new Program()).Run();
    }

    public void Run()
    {
      this.SaveData(@"c:\temp\test.ppm", "ÿÿ ÿÿ ÿÿ ÿÿ aa aa aa ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿ", 100, 200, Encoding.GetEncoding(1252));
    }

    private void SaveData(String filename, String data, Int32 width, Int32 height, Encoding encoding)
    {
      const Int32 bufferSize = 2048;

      Directory.CreateDirectory(Path.GetDirectoryName(filename));      

      if (Path.GetExtension(filename).ToLower() != ".ppm")
        filename += ".ppm";

      using (var fs = new FileStream(filename, FileMode.Create, FileAccess.ReadWrite, FileShare.None, bufferSize))
      {
        using (var bw = new BinaryWriter(fs, encoding))
        {
          var buffer = encoding.GetBytes(this.GetHeader(width, height));
          bw.Write(buffer);

          buffer = encoding.GetBytes(data);
          bw.Write(buffer);
        }
      }
    }

    private String GetHeader(Int32 width, Int32 height)
    {
      return String.Format("P6 {0} {1} 255 ", width, height);
    }
  }
}