Undefined symbols for architecture x86_64 flex/bison

Question

I am trying to make a simple toy (LET) language as practice using flex/bison. However, I keep getting the following error when I run the command g++ -Wall -std=c++11 repl.cpp -v -o LET:

Undefined symbols for architecture x86_64:
  "yy_scan_string(char const*)", referenced from:
      _main in repl-030403.o
  "yy_delete_buffer(yy_buffer_state*)", referenced from:
      _main in repl-030403.o
  "yyparse()", referenced from:
      _main in repl-030403.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I have the following files defined,

LET.y:

%{
#include <iostream>
#include <cstdio>
#include <string>

#include "AST.hpp"
#include "GENERATED_CODE/LET_parser.hpp"
#include "GENERATED_CODE/LET_lexer.hpp"

using namespace std;

extern int yylex();
extern char* yytext;
extern int yyparse();
extern Pgm* prog;
void yyerror (char const *s) {
   fprintf (stderr, "%s\n", s);
}
%}

%output "./PARSER/GENERATED_CODE/LET_parser.cpp"
%defines "./PARSER/GENERATED_CODE/LET_parser.hpp"

%union {
    Pgm* prog;
    int value;
    char* var;
    char cval;
    Exp* exp;
    string* str;
}

%token TTRUE
%token TFALSE
%token TLET
%token TIN
%token TIF
%token TTHEN
%token TELSE
%token TISZERO
%token TASSIGN
%token TLPAREN
%token TRPAREN
%token TPLUS
%token TMINU
%token TMULT
%token TDIVI
%token TREMI
%token TCOMMA
%token <value> TNUM
%token <var> TVAR

%type <prog> Prog
%type <exp> Exp
%type <str> Vari
/* Causes Bison to give more detailed error messages when parsing */
%error-verbose

%%

/* Bison Grammar Declarations */
Prog: Exp                 { prog = new Pgm($1); }

Vari: TVAR        { $$ = new string(yytext);}

Exp: TNUM                                 { $$ = new ConstExp(atoi(yytext)); }
   | TVAR                                 { $$ = new VarExp(yytext); } 
   | TTRUE                                { $$ = new TrueExp; }
   | TFALSE                               { $$ = new FalseExp; }
   | TISZERO TLPAREN Exp TRPAREN          { $$ = new IsZeroExp($3); }
   | TMINU TLPAREN Exp TCOMMA Exp TRPAREN { $$ = new DiffExp($3, $5); }
   | TPLUS TLPAREN Exp TCOMMA Exp TRPAREN { $$ = new SumExp($3, $5); }
   | TMULT TLPAREN Exp TCOMMA Exp TRPAREN { $$ = new MultExp($3, $5); }
   | TDIVI TLPAREN Exp TCOMMA Exp TRPAREN { $$ = new QuotExp($3, $5); }
   | TREMI TLPAREN Exp TCOMMA Exp TRPAREN { $$ = new RemiExp($3, $5); }
   | TIF Exp TTHEN Exp TELSE Exp          { $$ = new IfExp($2, $4, $6); }
   | TLET Vari TASSIGN Exp TIN Exp        { $$ = new LetExp(*$2, $4, $6); }

LET.l:

%{
#include <iostream>
#include <cstdio>

#include "AST.hpp"
#include "GENERATED_CODE/LET_parser.hpp"

using namespace std;

int yylex(); 
void yyerror (char const *s) {
   fprintf (stderr, "%s\n", s);
}
%}

%option c++
%option header-file = "./PARSER/GENERATED_CODE/LET_lexer.hpp"
%option outfile     = "./PARSER/GENERATED_CODE/LET_lexer.cpp"
%option noyywrap

digit [0-9]
alpha [a-zA-Z]
ws [ \t\n]+

%%
"#".*[\n]                      { /* No Action Taken */ }
{ws}                           { /* No Action Taken */ }
"true"                         { return TTRUE; }
"false"                        { return TFALSE; }
"let"                          { return TLET; }
"in"                           { return TIN; }
"if"                           { return TIF; }
"then"                         { return TTHEN; }
"else"                         { return TELSE; }
"zero?"                        { return TISZERO; }
"="                            { return TASSIGN; }
"("                            { return TLPAREN; }
")"                            { return TRPAREN; }
"+"                            { return TPLUS; }
"-"                            { return TMINU; }
"*"                            { return TMULT; }
"/"                            { return TDIVI; }
"%"                            { return TREMI; }
","                            { return TCOMMA; }
"-"?{digit}+                   { return TNUM; }
[a-zA-Z][a-z|0-9|\_|\'|\-|\?]* { return TVAR; }

Makefile:

generate-parser: clean flex bison

clean:
    rm -rf PARSER/GENERATED_CODE
    mkdir -p PARSER/GENERATED_CODE

flex: 
    flex PARSER/LET.l

bison:
    bison -d PARSER/LET.y

repl.cpp:

#include <cstdio>

#include "./PARSER/AST.hpp"
#include "./PARSER/GENERATED_CODE/LET_parser.hpp"
#include "./PARSER/GENERATED_CODE/LET_lexer.hpp"

using namespace std;

extern YY_BUFFER_STATE yy_scan_string(const char * str);
extern void yy_delete_buffer(YY_BUFFER_STATE buffer);

int main() {
    char input[] = "RAINBOW UNICORN 1234 UNICORN";

    YY_BUFFER_STATE state = yy_scan_string(input);
    yyparse();
    yy_delete_buffer(state);

    return 0;
}

Answer 1

The basic problem is that you do not seem to be compiling the generated scanner and parser, nor are you linking the compiled scanner and parser into your executable. So when you attempt to create the executable, none of the symbols defined in the scanner (yy_scan_string and yy_delete_buffer) nor in the parser (yyparse) exist.

When you add the compilation lines to your Makefile, remember that the scanner depends on the parser because it must include the header file generated by bison in order to have the token types defined.

That being the case, the parser cannot depend on the scanner, which is why it is not correct to #include the scanner's header file in the bison prologue. It is also incorrect to #include the parser header file in the bison prologue, because the bison-generated parser already includes the necessary prototypes and definitions.

Finally, you are asking flex to generate a C++ lexer, but your code uses the C interface (which includes the functions yy_scan_string and yy_delete_buffer). Personally, I would just delete the %option c++ from your scanner prologue. If you really want to use the C++ API, you will need to carefully read the flex manual chapter on C++ scanners, and then make appropriate changes to your bison definition so that it knows how to invoke the lexer.

Although this is not related to your question, the use of yytext in a parser is almost never correct, since the contents of yytext will normally correspond to the next token in the input. (LR(1) parsers always lookahead one token.) You should do the necessary semantic conversions in your scanner actions.