7

I'm trying to use flex and bison in my project to generate a parser code for a file structure. Main programming language is C++ and project is on an OO design mainly running in parallel.

I heard that flex and bison generated parsers are C codes and they're not reenterant. Googling, I found flex++ and bisonc++. Unfortunately there is no simple tutorial to get started. Most examples are based on bison/flex. Some people somehow integrated bison/flex parsers in their C++ code. They supposed to be "tricky"...

Documentation of flex++ and bisonc++ doesn't help me and. Tutorials and examples, they all get input from stdin and print some messages on stdout.

I need these features in my parser:

  1. Parser should be a C++ class, defined in normal manner (a header and a cpp file)
  2. Parser receives data from either an std::string or std::stringstream or a null-terminated char*.

I feel so confused. Should I use flex++/bisonc++ or flex/bison? And how to do that, satisfying above conditions?

Lesmana
  • 25,663
  • 9
  • 82
  • 87
sorush-r
  • 10,490
  • 17
  • 89
  • 173

3 Answers3

3

There are flex/bison, flex++/bison++ and flexc++/bisonc++. I think it's best to pick one of these three pairs, instead of mixing/matching flex++ and bisonc++. Here are the user guides for Flexc++ and Bisonc++.

From the Flexc++ website:

Flexc++, contrary to flex and flex++, generates code that is explicitly intended for use by C++ programs. The well-known flex(1) program generates C source-code and flex++(1) merely offers a C++-like shell around the yylex function generated by flex(1) and hardly supports present-day ideas about C++ software development. Contrary to this, flexc++ creates a C++ class offering a predefined member function lex matching input against regular expressions and possibly executing C++ code once regular expressions were matched. The code generated by flexc++ is pure C++, allowing its users to apply all of the features offered by that language.

From the Bisonc++ website:

Bisonc++ is a general-purpose parser generator that converts a grammar description for an LALR(1) context-free grammar into a C++ class to parse that grammar. Once you are proficient with bisonc++, you may use it to develop a wide range of language parsers, from those used in simple desk calculators to complex programming languages. Bisonc++ is highly comparable to the program bison++, written by Alain Coetmeur: all properly-written bison++ grammars ought to be convertible to bisonc++ grammars after very little or no change. Anyone familiar with bison++ or its precursor, bison, should be able to use bisonc++ with little trouble. You need to be fluent in using the C++ programming in order to use bisonc++ or to understand this manual.

So flexc++/bisonc++ are more than just wrappers around the old flex/bison utilities. They generate complete C++ classes to be used for re-entrant scanning / parsing.

TemplateRex
  • 69,038
  • 19
  • 164
  • 304
  • I even can't compile flexc++! Just downloaded source code. No makefile, no cmake nothing! – sorush-r Jul 26 '12 at 18:27
  • @sorush-r there's a Linux package: http://packages.debian.org/sid/flexc++ Why would you need a makefile? Just include the headers and build from your own environment. – TemplateRex Jul 26 '12 at 18:31
  • 1
    I didn't heard about `icmake` before. seems to be the build system used to compile `flexc++`. Thanks – sorush-r Jul 26 '12 at 18:33
  • 1
    @sorush-r The author Frank Brokken makes a lot of his own utilities. Apart from flexc++, bisonc++, icmake he also did a markup language (yodl) and much more. Oh and he is the author of the C++ Annations document http://cppannotations.sourceforge.net/ – TemplateRex Jul 26 '12 at 18:36
  • I'm compiling `flexc++`. Hope can use it successfully! I really like C++ Annations a lot! It's my main resource for cpp. In fact I always do `sudo apt-get install c++-annotations-pdf` ;) It has a section for bisonc++/flex (last section in book) Unfortunately it's so complicated for me. – sorush-r Jul 26 '12 at 18:51
  • 1
    @sorush-r If you mean section 23.9 http://www.icce.rug.nl/documents/cplusplus/cplusplus23.html#l576, that's the one. If you follow this line by line, you should get a compilable example by the time you reach 23.9.2. – TemplateRex Jul 26 '12 at 18:54
  • I'll take a look at (4th time!). I think all stuff about `lex`, `flex`, `yacc`, `flex`, `bison`, etc, are very complicated. started to study about two weeks ago, since now, no success. yet! – sorush-r Jul 26 '12 at 18:59
  • 2
    @sorush-r Perseverance will pay off for you, just try to understand one step at the time. I had the good fortune to be a student at the same university where Frank Brokken was a teacher, at the time when the Annotations were being written! That was even before the old C++98 standard was ready, with broken compilers etc. But somehow, I got through that class. Now it's a lot easier, so good luck to you! – TemplateRex Jul 26 '12 at 19:06
2

Flex can generate a reentrant C scanner. See Section 19 Reentrant C scanners in the Flex manual.

Similarly, Bison can generate a reentrant C parser. See Section 3.8.11 A Pure (Reentrant) Parser in the Bison manual for details.

Do you absolutely need to have a C++ parser and std::string/stringstream based parser data?

Have you looked at Boost.Spirit as an alternative?

rici
  • 234,347
  • 28
  • 237
  • 341
Void - Othman
  • 3,441
  • 18
  • 18
  • No. I can change my code to generate `char*`s to be parsed. Bu not able to read input from `stdin` – sorush-r Jul 26 '12 at 17:47
  • 1
    @sorush-r: You should be able to change the source of your Flex C scanner input by redefining the `YY_INPUT()` macro. See [Section 9 The Generated Scanner](http://flex.sourceforge.net/manual/Generated-Scanner.html#Generated-Scanner) in the Flex manual. I'd take a serious look at Boost.Spirit, too. – Void - Othman Jul 26 '12 at 18:09
0

The LRSTAR product (LR(k) parser and DFA lexer generator) is C++ based. Runs on Widowns and has six Visual Studio projects. The code also compiles with "gcc" and other compilers. There are classes for lexer and parser, symbol-table, AST. Complete source code is available. It gets good reviews. I should know. I am the author.