convert ANSI data to Unicode

Question

i have a text file saved by encoding 1256.
as far as my Windows CE 5.0 on my device does not support that code page i can't open the file by that encoding in .NET CF, but the OS supports Unicode. (i showed up some hard code strings in my form) how can i read that file and convert it's data to Unicode?
how can i convert a single character to its UTF8 equivalent bytes?
THIS LINK says that in 1256 code page table the character number 200/C8 is 0x0628. so what's the relation between them? if i have 200/C8 , how can i obtain the 0x0628?

Answer 1

It would probably be easiest just to hard code the conversion yourself - create a char[] of 256 values, populate the first 128 positions with just the equivalent numbers, and then populate the rest manually. The "relation" between them isn't one you can get mathematically - it's just a somewhat-arbitrary assignment of values

For example:

private static readonly char[] CodePage1256 = GenerateCodePage1256();

private static readonly char[] GenerateCodePage1256()
{
    char[] ret = new char[256];
    for (int i = 0; i < 128; i++)
    {
        ret[i] = (char) i;
    }
    string upperCharacters =
        "\u20ac\u067e\u201a\u0192\u201e\u2026\u2020\u2021" +
        "\u02c6\u2030"; // etc - from the Wikipedia page

    for (int i = 0; i < 128; i++)
    {
        ret[i + 128] = upperCharacters[i];
    }
}

Then you have a direct byte to char mapping. Of course this is a potentially error-prone process - another possibility would be to create a file with the mapping in, on a system which does have that code page.

Anyway, once you've got the mapping, you can easily convert any array of bytes to a string or char array, at which point you can use the normal .NET classes to write out the file as UTF-8 again. For example:

using (Stream input = File.Open("input.txt"))
{
    using (StreamWriter output = File.CreateText("output.txt"))
    {
        byte[] byteBuffer = new byte[8 * 1024];
        char[] charBuffer = new char[byteBuffer.Length];
        int bytesRead;

        while ((bytesRead = input.Read(byteBuffer, 0, byteBuffer.Length)) > 0)
        {
            for (int i = 0; i < bytesRead; i++)
            {
                charBuffer[i] = CodePage1256[byteBuffer[i]];
            }
            output.Write(charBuffer, 0, bytesRead);
        }
    }
}