Design lexical analyzer and parser in C#

Question

I am going to design a compiler with the help of Gold Parse Builder (GPB) to design lexical analyzer and parser. I am having input x1 = x3/(x2/x5);

and need its output like following:

[  1]   Variable         x1 at 1,1
[  2]     Equals          = at 1,4
[  3]   Variable         x3 at 1,6
[  4]     Divide          / at 1,8
[  5]  LeftParen          ( at 1,9
[  6]   Variable         x2 at 1,10
[  7]     Divide          / at 1,12
[  8]   Variable         x5 at 1,13
[  9] RightParen          ) at 1,15
[  10]       Semi          ; at 1,16

I can print this output because I have grammar for this. But my question how to get tokens one by one like at first index x1 (If I try to read it by char by char then output comes x and 1 not x1, it I try to split it from white spaces then what about x3/(x2/x5)).

Give me some idea about this

Thanks In advance.

Answer 1

You could make a Parser, either with Regular Expressions or by looping the string and reading the relevant informations, as the following piece of code.

I only include the parser itself, as the rest can be inferred from the way it gets call or how it is used. You would still need to check at which position the Token starts but I guess that should be simple enough.

public class FormulaReader : IFormulaReader
{
    protected static readonly IToken EQUALIZE = new Token(TokenType.EQUALIZER, "=");
    protected static readonly IToken ADD = new Token(TokenType.MODIFIER, "+");
    protected static readonly IToken SUBTRACT = new Token(TokenType.MODIFIER, "-");
    protected static readonly IToken DIVIDE = new Token(TokenType.MODIFIER, "/");
    protected static readonly IToken MULTIPLY = new Token(TokenType.MODIFIER, "*");
    protected static readonly IToken GROUPSTART = new Token(TokenType.GROUPSTART, "(");
    protected static readonly IToken GROUPEND = new Token(TokenType.GROUPEND, ")");

    public IFormula Parse(string textRepresentation)
    {
        IList<IToken> tokenList = new List<IToken>()
        {
            EQUALIZE, ADD, SUBTRACT, DIVIDE, MULTIPLY, GROUPSTART, GROUPEND
        };
        IFormula formula = new Formula();
        IToken currentToken = new Token(TokenType.IDENTIFIER, "");
        bool equalizerFound = false;

        for (int i = 0, len = textRepresentation.Length; i < len; i++)
        {
            string item = textRepresentation[i].ToString();
            if (item == " ")
            {
                continue;
            }
            IToken selectedToken = (from t in tokenList where string.Equals(t.Text, item) select t).FirstOrDefault();
            if (selectedToken == null)
            {
                currentToken.Text += item;
            }
            else
            {
                if (!string.IsNullOrWhiteSpace(currentToken.Text))
                {
                    // contains an identifier
                    formula.Add(currentToken, equalizerFound);
                }
                if (selectedToken.TokenType != TokenType.EQUALIZER)
                {
                    formula.Add(selectedToken, equalizerFound);
                }
                else
                {
                    equalizerFound = true;
                }
                currentToken = new Token(TokenType.IDENTIFIER, "");
            }
        }

        return formula;
    }
}