4

I am trying to create a reentrant parser using flex and bison. I want to add a parameter to save some state, but I failed to add it to yylex().

Here is the example, it is not expected to compile, just shows the generated code.

foo.l

%option reentrant
%option bison-bridge
%option header-file="foo.tab.h"
%{
#include "foo.tab.h"
%}
%%
"{" { return "{"; }
")" { return '}'; }
%%

foo.y

%define api.pure full
%define parse.error verbose
%parse-param {void *scanner}
%parse-param {int *pint}
%lex-param {void *scanner}
%lex-param {int *pint}
%token '(' ')'
%%
foo : '(' | ')' ;
%%

run with:

bison -d -b foo foo.y
flex foo.l
gcc -E lex.yy.c | less

We can see int yylex (YYSTYPE * yylval_param , yyscan_t yyscanner) {...} So pint is gone. But I think I have specified it at foo.y. So what I need to do more to make yylex accept pint?

Environment: Gentoo Linux stable with Bison-3.0.4 and Flex 2.5.39

Brian Tompsett - 汤莱恩
  • 5,753
  • 72
  • 57
  • 129
OstCollector
  • 51
  • 1
  • 5
  • Bison directives (eg. `%lex-param`) have no influence on flex. How could they? Flex doesn't even know the name of the bison file. – rici Sep 25 '16 at 21:31
  • Anyway, see if http://stackoverflow.com/questions/34418381/how-to-reference-lex-or-parse-parameters-in-flex-rules/34420950#34420950 helps. If not, I'll try to write something up later. – rici Sep 25 '16 at 21:43
  • @rici , I am not a native English speaker, and I misunderstood some documents. I though `%lex-param` automatically add additional args in flex. – OstCollector Sep 26 '16 at 02:31
  • 2
    bison and flex are not as well-documented as one might like; even native English speakers have trouble. But they don't violate that laws of physics. Flex produces the `yyflex` function, with a defined parameter list. `bison` produces code which calls `yyflex`, providing an actual argument lust. They are separate tools; they don't see each other's files. But obviously the declaration and the call must agree. So the source files for the two tools need to make sure that both tools generate compatible outputs. – rici Sep 26 '16 at 03:16
  • 1
    Turns out that it is easier to pass custom data with `yyextra` (rather than modifying `yylex` signature) as explained in the answer to [this question](https://stackoverflow.com/questions/49221106/remove-default-yy-decl-from-flex-output) and in [flex docs](https://westes.github.io/flex/manual/Extra-Data.html). – gudok Oct 23 '18 at 14:54

1 Answers1

11

%lex-param says for bison to invoke yylex with extra parameters, but does not say anything to flex.

The default definition of the yylex() function can be changed by defining the YY_DECL macro in your foo.l file's definition part. In order to have only the int *pint as argument, it looks like this:

#define YY_DECL int yylex(int *pint)

If yylval_param and yyscanner are also needed, then:

#define YY_DECL int yylex(YYSTYPE * yylval_param, yyscan_t yyscanner, int *pint)

xsnpdngv
  • 119
  • 1
  • 4
  • 1
    This answer is technically wrong: defining `YY_DECL` in your `*.l` file has no effect actually. What's important is to include a file that defines `YY_DECL` in your program **before** you include `*.lex.h` header. – wvxvw Feb 27 '18 at 16:31