JFLEX: create custom function to count number of characters

Question

I have a project to write a lexer using JFLEX for a made up language.It is a language with a simple syntax including simple variables like strings and characters integers and floats. However i have a problem with characters. I created a state similar to STRING named CHAR which uses single quotes. Unfortunately initializations like 'hello' are acceptable. So is there a way to create a custom function to count the number of characters and print the character variable if its size is equal to 1 or throw an exception otherwise? I searched a lot but cannot find an answer.Here is my code so far:

import static java.lang.System.out;

%%

%class Lexer
%unicode
%public
%final
%integer
%line
%column

%{
    // user custom code 

    StringBuffer sb = new StringBuffer();

%}

LineTerminator = \r|\n|\r\n
WhiteSpace     = {LineTerminator} | [ \t\f] 
Comment        = "/*" [^*] ~"*/" | "/*" "*"+ "/"

Identifier     = [:jletter:] [:jletterdigit:]*
IntegerLiteral = 0 | [1-9][0-9]*

Exponent= [eE][\+\-]?[0-9]+
Float1= [0-9]+ \. [0-9]+ {Exponent}?
Float2= \. [0-9]+ {Exponent}?
Float3= [0-9]+ \. {Exponent}?
Float4= [0-9]+ {Exponent}
FloatLiteral   = {Float1} | {Float2} | {Float3} | {Float4}


%state STRING, CHAR

%%

/* reserved keywords */
    <YYINITIAL>"ﬂoat"                         { out.println("FLOAT"); } 
    <YYINITIAL>"int"                          { out.println("INTEGER"); }
    <YYINITIAL>"char"                         { out.println("CHAR"); }
    <YYINITIAL>"break"                        { out.println("BREAK"); }
    <YYINITIAL>"print"                        { out.println("PRINT"); }
    <YYINITIAL>"while"                        { out.println("WHILE"); }
    <YYINITIAL>"if"                           { out.println("IF"); }
    <YYINITIAL>"else"                         { out.println("ELSE"); }
    <YYINITIAL>"void"                         { out.println("VOID"); }
    <YYINITIAL>"return"                       { out.println("RETURN"); }
    <YYINITIAL>"continue"                     { out.println("CONTINUE");}           
    <YYINITIAL>"new"                          { out.println("NEW"); }
    <YYINITIAL>"delete"                       { out.println("DELETE"); }

    <YYINITIAL>"("                            { out.println( "LEFT ARENTHESIS" ); }
    <YYINITIAL>")"                            { out.println( "RIGHT PARENTHESIS" ); }
    <YYINITIAL>"{"                            { out.println( "LEFT CURLY BRACKET" ); }
    <YYINITIAL>"}"                            { out.println( "RIGHT CURLY BRACKET" ); }
    <YYINITIAL>"["                            { out.println( "LEFT SQ" ); }
    <YYINITIAL>"]"                            { out.println( "RIGHT SQ" ); }
    <YYINITIAL>";"                            { out.println( "SEMICOLON" ); }
    <YYINITIAL>","                            { out.println( "COMMA" ); }

    <YYINITIAL> {

    /* identifiers */ 
    {Identifier}                   { out.println("id:" + yytext()); }

    /* literals */
    {IntegerLiteral}               { out.println("integer:" + yytext()); }
    {FloatLiteral}               { out.println("float:" + yytext()); }

    \"                             { sb.setLength(0); yybegin(STRING); }
    \'                             { sb.setLength(0); yybegin(CHAR); }

    /* operators */
    "="                            { out.println("ASSIGN"); }
    "+"                            { out.println("PLUS"); }
    ";"                            { out.println("SEMICOLON"); } 
    ">"                            { out.println("GREATER THAN"); } 
    "<"                            { out.println("LESS THAN"); } 
    "!="                           { out.println("NOT EQUAL TO"); } 
    "<="                           { out.println("LESS OR EQUAL THAN"); } 
    ">="                           { out.println("GREATER OR EQUAL THAN"); } 
    "−"                            { out.println("SUBTRACTION"); } 
    "*"                            { out.println("MULTIPLICATION"); } 
    "/"                            { out.println("DIVISION"); } 
    "%"                            { out.println("PERCENT"); } 
    "=="                           { out.println("EQUALS"); } 
    "&&"                           { out.println("AND"); } 
    "||"                           { out.println("OR"); } 

    /* comments */
    {Comment}                      { /* ignore */ }

    /* whitespace */
    {WhiteSpace}                   { /* ignore */ }
    }

    <STRING> {
    \"                             { yybegin(YYINITIAL);
                                out.println("string:" + sb.toString()); }

    [^\n\r\"\\]+                   { sb.append(yytext()); }
    \\t                            { sb.append('\t'); }
    \\n                            { sb.append('\n'); }


    \\r                            { sb.append('\r'); }
    \\\"                           { sb.append('\"'); }
    \\                             { sb.append('\\'); }
    }

    <CHAR> {
    \'                             { yybegin(YYINITIAL);
                                out.println("char:" + sb.toString()); }

    [^\n\r\0\'\\]+                 { sb.append(yytext()); }
    \\t                            { sb.append('\t'); }
    \\n                            { sb.append('\n'); }
    \\0                            { sb.append('\0'); }                        

    \\r                            { sb.append('\r'); }
    \\\'                           { sb.append('\''); }
    \\                             { sb.append('\\'); }
    }


    /* error fallback */
    [^]                                { throw new RuntimeException((yyline+1) + ":" + (yycolumn+1) + ": illegal character <"+ yytext()+">"); }

score 0 · Answer 1 · answered Mar 25 '18 at 11:29

After may tries i found a solution which is acceptable for me i just replaced this block of code inside the CHAR state :

\'                             { yybegin(YYINITIAL);
                            out.println("char:" + sb.toString()); }

with the following:

\'                             {
                                if(sb.length() > 1 ){
                                   yybegin(YYINITIAL);
                                   throw new RuntimeException("you can not instatiate a char variable with more than one characters\n");

                                }else{
                                    yybegin(YYINITIAL);
                                    out.println("char:" + sb.toString());
                                }

Hope this will help someone with a similar problem

JFLEX: create custom function to count number of characters

1 Answers1