Lex - not detecting token after adding a new line

Question

I'm trying to write a program that reads action and its components from a PDDL file. The program should first look for token :action( and then action name etc.

Following is my lex code:

Note: The answer to this question is mostly related to :(action token, action name token and newline token, so you might only look at those. By SC_ACNAME I mean scan action name etc.

%{
#include "myscanner.h"
%}

%START SC_ACNAME SC_PARAMETERS SC_PREDICATE SC_BODY

NAME                                [0-9a-zA-Z_\-]+
TEXT                                [0-9a-zA-Z?_\-( \t]+
TEXT_PARAM                          [0-9a-zA-Z?_\-( \t\n]+
TEXTBREAK                           [ \t\n]+

    int lb_flag=0;
    int pred_lb_flag = 0;

%%

\(:action                                                               {lb_flag++; BEGIN SC_ACNAME; return ACTION;}
<SC_ACNAME>{TEXT}                                                       {BEGIN SC_BODY; return ACTION_NAME;}
<SC_BODY>:parameters                                                    {BEGIN SC_PARAMETERS; return PARAMETER;}
<SC_PARAMETERS>{TEXT_PARAM}                                             {lb_flag++; BEGIN SC_BODY; return PARAMETERS;}  
<SC_BODY>:precondition                                                  {BEGIN SC_PREDICATE; return PRECONDITION;}
<SC_BODY>:effect                                                        {BEGIN SC_PREDICATE; return EFFECT;}
<SC_PREDICATE>\(and                                                     {pred_lb_flag++; return CONJUNCTION;}
<SC_PREDICATE>\(not                                                     {pred_lb_flag++; return NEGATION;}
<SC_PREDICATE>\({NAME}({TEXTBREAK}\?{NAME})+\)                          return PREDICATE;
<SC_PREDICATE>\)                                                        {if(pred_lb_flag) pred_lb_flag--; if(!pred_lb_flag) BEGIN SC_BODY; return RIGHT_BRACKET;}
<SC_BODY>\)                                                             {if(lb_flag) lb_flag--; if(!lb_flag) BEGIN 0; return RIGHT_BRACKET;}
<SC_PREDICATE,SC_BODY,SC_ACNAME>{TEXTBREAK}                             ;   

%%

int yywrap(void)
{
    return 1;
}

Header File (myscanner.h):

#define ACTION 1
#define ACTION_NAME 2
#define PARAMETER 3
#define PARAMETERS 4
#define PRECONDITION 5
#define EFFECT 6
#define CONJUNCTION 7
#define NEGATION 8
#define PREDICATE 9
#define RIGHT_BRACKET 10

Following in my C program:

#include <stdio.h>
#include "myscanner.h"

extern int yylex();
extern int yylineno;
extern char* yytext;

int main(void)
{
    int ntoken, vtoken;

    ntoken = yylex();
    while(ntoken)
    {
        printf("%d", ntoken);
        print("%s", yytext);
        ntoken = yylex();
    }
    return 0;
}

When I try to tokenise file which contains (:action move it detects :(action and action name as desired and returns ACTION and ACTION_NAME. But when I enter a newline after move (action name), it doesn't detect anything. Why is it not recognising the token (:action?

EDIT:

Pass Test Case: domain.pddl

(:action move

Failure Test Case: domain.pddl

(:action move<nl>

indicates newline

I do not understand _when I enter a newline after move (action name)_, I need to enter a newline after _(:action move_ to finish the input, and all seems ok. Please edit your question to give the input(s) — bruno, Jan 08 '19 at 19:05
@bruno Hmm...I did this via a file. Can you try entering new line in a file and then test it? ./a.out < file.pddl — Laschet Jain, Jan 08 '19 at 19:08
again, to avoid misunderstanding, give an example of input file, not a description of that file, if you want use to indicate where the nl are — bruno, Jan 08 '19 at 19:09

bruno · Accepted Answer · 2019-01-08T19:23:41.493

1

I encourage you to add for instance putchar('\n'); before the return 0; in main to flush what is produced by printf, may be you have nothing printed because of that miss ?

If 'f' contains the line (:action move<nl> and I execute a.out < f that prints 1(:action2 move and it is the same when the line is not terminated in 'f' (no at the end)

edited Jan 08 '19 at 19:23

answered Jan 08 '19 at 19:20

bruno

32,421
7
25
37

Can you please explain _printf_s part? – Laschet Jain Jan 08 '19 at 19:22
Can you please explain that flush part in a bit more detail. Thanks. My program worked but still i didn't get the concept. – Laschet Jain Jan 08 '19 at 19:26
that depend on the used OS, but when you print on stdout (rather than stderr) the output is buffered, so a flush is needed to see the output, one way to force the flush is to print a newline. – bruno Jan 08 '19 at 19:28
Thanks. I'm using MacOS. – Laschet Jain Jan 08 '19 at 19:29
Hmm...it's kinda funny that some files in my laptop are running fine while this one is giving error. – Laschet Jain Jan 08 '19 at 19:31
Just test with and without the `putchar('\n');` before the `return 0;` . At the minimum when you start the execution is a shell the newline allows to see the shell prompt on the a new line – bruno Jan 08 '19 at 19:31
I didn't said in my answer, but the definition of TEXT and TEXT_PARAM containing space tab and newline seems strange for me while you also have them in TEXTBREAK – bruno Jan 08 '19 at 19:32

Lex - not detecting token after adding a new line

1 Answers1