Reading multi-dimensional variables from text file in C#

Question

I'm trying to create a GUI in C# (Windows Form Application, Visual Studio) that reads and writes certain variables from/to a text file. I have no problem with the GUI itself, but I need a function that can read from (and eventually write to) the text file, recognize the variables and put them in internal variables or directly into the GUI controls. The text file looks something like this (it's much longer though):

     /*   Control     */
/YD/YDC/MCSTEP 20001    /YD/YDC/NCSTEP 0
/YD/YDC/DCSIZC 0.075

     /*   Properties     */
/YD/YDPE/D1PEFR     3
0.7 0.75 .5
/YD/YDPM/I2PMSET      21       3       2
       1       3       0
       1       0       1
/YD/YDPN/D3PNFAC 231 2 3 4

0               0
1e-010          0
9.2966e-008     0
1.8583e-007     0

0     0
1e-010     0
9.2966e-008     0.71221
1.8583e-007     1.4688

0               0
1e-010          0
9.2966e-008     0
1.8583e-007     0

I have a code in C that reads the text file, but it's horribly long. I have the impression that this can be done elegantly with a relatively short code in C#. I have tried using a dictionary as suggested here, but the format of my text file is too complex for it... unless some guru out there knows some tricks that can help me.

Your suggestions are welcome. Sample code would be greatly appreciated.

The following defines the format of the text file:

• All variables begin with /YD/.
• Text between /* and */ are comments (no need to read).
• Separators can be spaces or tab characters and you can have one or many of them between variables and values. You can also have empty lines.
• Sometimes there is more than one variable per line.
• Sometimes a variable and its value are in different lines.
• Some variables are single values, others are 1-D, 2-D or 3-D.
• Variables with the form xxxxxx (not necessarily fixed number of characters) are followed by its single value.
• Variables with the form x1xxxxx are 1-D. These are followed by an integer number which tells you the size of the variable, followed by the values of the variable (e.g. the variable /YD/YDPE/D1PEFR 3 0.7 0.75 .5 is a 1-D variable, with three values: 0.7, 0.75 and 0.5).
• Variables with the form x2xxxxx are 2-D. These are followed first by a number that tells you how to read the variable, then the size of the variable (2 numbers), followed by the actual values of the variable. E.g. the variable /YD/YDPM/I2PMSET 21 3 2 1 3 0 1 0 1 is a 2-D variable, where the number 21 means that first you read changing the "x" coord then the "y" coord (a 12 would mean you read first changing "y" then "x"), following numbers 3 and 2 mean the size is [3,2], and the following 6 numbers are the values of the variable in order (x1y1, x2y1, x3y1, x1y2, x2y2, x3y2).
• Finally, variables of the form x3xxxx are 3-D. These are followed by a number that tells you how to read it, then the size (3 numbers) and then the values. E.g. variable /YD/YDPN/D3PNFAC 231 2 3 4 ... is 3-D, first you read "x", then "z", then "y", and the size is [2,3,4]. The values are then read in order (x1y1z1, x2y1z1, x1y1z2, x2y1z2, x1y1z3, x2y1z3, x1y1z4, x2y1z4, x1y2z1, x2y2z1, ...).

Answer 1

Mu suggestion would be to restructure the way that file is build. That way you could use an XML structure and make all your code much much more easy to read.

Take a look at : XDocument (System.Xml.Linq)

Here is an example of what you could do:

<Root>
    <Controls>
        <Control/>
        ...
    </Controls>
    <Properties>
        <Property/>
        ...
    </Properties>
</Root>

But if you can't change the file you should use: File.ReadLines

Like so:

foreach (var line in File.ReadLines("FilePath"))
{
    //Do your work with the [line]
}

To get your variables take a look at string class. More specific look at the Split, Replace and Trim methods.

Answer 2

I don't think anything in C# is going to magically do it for you, but to avoid the spaghetti code I imagine you have in C, I would break it up into some classes that are each responsible for a given variable type. Sorry this is a bit psuedo codey but I hope it helps.

First a base class:

public abstract class DataValue {
    public string Category { get; protected set; } // value after /YD
    public string Name { get; protected set; } // value after /YD/XXX/
    public int DimensionCount { get; protected set; } // number of dimensions
    public int[] Dimensions { get; protected set; } // The size of each dimension

    protected void ParseHeader(string header) {
        // Split of the header into the Category/Name and set it
    }

    public virtual void ReadValue(string header, StreamReader reader) {
        ParseHeader(header);
    }
}

Then a class for each of your variable types:

public class ScalarValue: DataValue {
    public ScalarValue() {
        DimensionCount = 1;
        Dimensions = new [] { 1 };
    }

    public override void ReadValues(string header, StreamReader reader) {
        base.ReadValues(header, reader);

        // Do custom logic for reading a scalar here.
    }
}

You would make a class like that for each of the primary types. They are each responsible for their own reading logic. You need one primary parser loop to find variables, create the object, and dispatch them:

public static List<DataValue> ParseDataFile(string filePath) {
    var reader = new StreamReader(filePath);

    var values = new List<DataValue>();
    var variableHeader = FindNextVariable(reader);

    while(!string.IsNullOrEmpty(variableHeader)) {
        DataValue v;

        switch(FindVariableType(variableHeader)) {
            case Scalar:
                v = new ScalarValue();
            case OneDimensionalArray:
                v = new Array1DValue();
            // etc
        }

        v.ReadValue(variableHeader, reader);

        values.Add(v);

        variableHeader = FindNextVariable(reader);
    } 

    return values;
}

FindNextVariable is:

public static string FindNextVariable(StreamReader reader) {
    var i = reader.Read();

    bool commentBlock = false;
    string variable = "";

    while (i >= 0) {
        var c = (char)i;

        // if it's white space, toss it
        // if it's a / peek to see if the next is a * starting a comment block
        // if in a comment block, read until you get */ consecutively
        // if it's a / and not a * next, begin appending to variable until you hit white space again.
    }

    return variable;
}

FindVariableType is:

public enum VariableType {
    Scalar,
    OneDimensionalValue,
    TwoDimensionalValue,
    ThreeDimensionalValue,
    Other
}

public static VariableType FindVariableType(string header) {
    var fields = string.Split('/', header);

    // take the third item, parse it to determine type
}

Lots of ways to do the actual implementation, but that's the general idea.

Answer 3

OK, here is my solution. I included several comments in the code, so it's more or less self explanatory.

In summary I define a class (one class for all variables, even if they are multidimensional!), and then I store all variables in a dictionary. I think it's pretty straightforward.

The class to store my variables:

public class Variable
{
    // Properties.
    public string Database { get; set; }
    public string Name { get; set; }
    public string Type { get; set; }
    public int Dimensions { get; set; }
    public int Order { get; set; }
    public int[] Size { get; set; }
    public double[] Value { get; set; }

    // Default constructor.
    public Variable()
    {
    }

    // Constructor.
    public Y_Variable(string db, string nam, string typ, int dim, int ord, 
        int[] siz, double[] val)
    {
        this.Database = db;
        this.Name = nam;
        this.Type = typ;
        this.Dimensions = dim;
        this.Order = ord;
        this.Size = siz;
        this.Value = val;
    }
}

The code to read file and store variables in dictionary:

public class Y_File2Dictionary
    {
        public static Dictionary<string, Y_Variable> File2Dictionary(string fileName)
        {
            // Define delimiters and block comments.
            char[] delimiterChars = { ' ', '\t', '\n', '\r' };
            var blockComments = @"/\*(.*?)\*/";

            // Read the file, remove comments, and store in array with no spaces.
            string[] arrayAllVars = Regex
                .Replace(File.ReadAllText(fileName), blockComments, "",
                    RegexOptions.Singleline)
                .Split(delimiterChars, StringSplitOptions.RemoveEmptyEntries);

            // Initialize dictionary.
            Dictionary<string, Y_Variable> dictAllVars = new 
                Dictionary<string, Y_Variable>();

            // Loop to read variables and store them in dictionary.
            int i = 0;
            while (i < arrayAllVars.Length)
            {
                // Read only variables.
                if (!char.Equals(arrayAllVars[i][0], '/'))
                {
                    i++;
                    continue;
                }

                // Read variable name and separate into array
                // e.g. /YD/YDC/DCSTEC => {"", YD, YDC, DCSTEC}.
                string[] arrayVarName = Regex.Split(arrayAllVars[i++], "/");

                // Identify variable.
                string varDb = arrayVarName[2];
                string varName = arrayVarName[3];
                string varTyp = (varName[0] == 'D') ? "double" : "integer";
                int varDim = ((int)Char.GetNumericValue(varName[1]) > 0) ?
                    ((int)Char.GetNumericValue(varName[1])) : 0;

                // Initiallization of variables to store order and size.
                int varOrder = 0;
                int[] varSize = new int[3] { 1, 1, 1 };

                // Update order and size, depending on the number of dimensions.
                switch (varDim)
                {
                    case 1:
                        varOrder = 1;
                        Int32.TryParse(arrayAllVars[i++], out varSize[0]);
                        varSize[1] = 1;
                        varSize[2] = 1;
                        break;
                    case 2:
                        Int32.TryParse(arrayAllVars[i++], out varOrder);
                        Int32.TryParse(arrayAllVars[i++], out varSize[0]);
                        Int32.TryParse(arrayAllVars[i++], out varSize[1]);
                        varSize[2] = 1;
                        break;
                    case 3:
                        Int32.TryParse(arrayAllVars[i++], out varOrder);
                        Int32.TryParse(arrayAllVars[i++], out varSize[0]);
                        Int32.TryParse(arrayAllVars[i++], out varSize[1]);
                        Int32.TryParse(arrayAllVars[i++], out varSize[2]);
                        break;
                    default:
                        varOrder = 0;
                        varSize[0] = 1;
                        varSize[1] = 1;
                        varSize[2] = 1;
                        break;
                }

                // Determine total size of variable, get values as strings.
                var varTotalSize = varSize[0] * varSize[1] * varSize[2];
                string[] varValStr = new string[varTotalSize];
                Array.Copy(arrayAllVars, i, varValStr, 0, varTotalSize);

                // Convert values from string to double.
                double[] varValDbl = new double[varTotalSize];
                varValDbl = Array.ConvertAll(varValStr, Double.Parse);

                // Add variable to dictionary.
                if (dictAllVars.ContainsKey(varDb + "_" + varName))
                    dictAllVars.Remove(varDb + "_" + varName);
                dictAllVars.Add(varDb + "_" + varName, new Y_Variable(varDb, varName,
                    varTyp, varDim, varOrder, varSize, varValDbl));

                i += varTotalSize;
            }
            return dictAllVars;
        }
    }