4

I have written a verilog parser with bison, and use boost::variant to store all the difference cases for each variant of each rules. I use a small example, the BNF rule of expression, to show my data structure:

expression :
  primary
  | expression + expression
primary :
  (expression)
  | number

The data structure to store it is:

typedef boost::variant<
  std::shared_ptr<exp1>,
  std::shared_ptr<exp2>,
> expression
typedef boost::variant<
  std::shared_ptr<prim1>,
  std::shared_ptr<prim2>,
> primary

Class exp1/2 and prim1/2 are used to store the two different cases in expression and primary:

class exp1 : public BaseClass {
  public :
    std::shared_ptr<primary> mem1;
    exp1(std::shared_ptr<primary> i1):
    mem1(i1)
    {}
}

For simplification, I only show exp1, while exp2 and prim1/2 are similar. In bison file, the rules and their actions are written like this :

expression :
  primary  {
   $$= std::make_shared<expression>(std::make_shared<exp1>($1));
  }

such a solution leads to two problem:

1 compiling is veeeeeeeeeeeery slow , cost almost 1 minute with g++ 4.8.4
2 run time is not very fast

I have a similar parser written in ocaml and ocamlyacc, it support very elegent specification for variant, and compile with 1 seconds, and the run speed is very similar to g++ version mentioned above.

Is there any problem in my style of using boost::variant?

==============

I change all the variant to class with constructor accepting shared_ptrs:

class ComponentBase {
};
Class VariantBase{
};
class prim1;
class prim2;
class exp1;
class exp2;
class expression : public VariantBase {
  expression (shared_ptr<ComponentBase> i1):
    VariantBase(i1) {}
}
class primary : public VariantBase {
  primary (shared_ptr<ComponentBase> i1):
    VariantBase(i1) {}
}

Then there is NO any improvement on compilation. It seems that code generated by the yacc is the source of slow.

Any suggestion?

shengyushen
  • 289
  • 3
  • 13
  • I assume the syntax tree is more complicated than this. So really, 1 minute is too long? – StoryTeller - Unslander Monica Jan 11 '16 at 14:48
  • As for the run-time, you'd have to examine the cpp files bison gives out. If the source code is hard to optimize, or you have optimizations turned off, your run-time will suffer. – StoryTeller - Unslander Monica Jan 11 '16 at 14:53
  • Actually the bison source file contain 3700~ lines, 200~ rules and 500~ components. It is a large one for complete verilog 2005 language. But I have a similar one written in ocaml language, whose compile time is only 3 seconds. – shengyushen Jan 12 '16 at 01:01

1 Answers1

6

Update Added demos in Boost Spirit Qi instead (because I'm not versed in flex/bison), see block-quote below

If your AST is using shared pointers, I suggest that runtime performance is not the concern.

If your AST is using variants, I suggest that compiletime performance is not the concern. (Therefore there is no cause for concern :))


The shared_ptrs go against the variants, conceptually. Shared_ptrs facilitate runtime polymorphism with lifetime-managed dynamic allocation of nodes.

Variants facilitate static polymorphism with automatic storage durations.

Go Runtime Polymorphism

If you're fine with runtime polymorphic AST nodes (which are often quite convenient in transformations on the AST) then I suggest you do not want to use the variants. Instead, make them part of the same node-hierarchy.

Rough sketch:

enter image description here


Go Static Polymorphism

Dropping the runtime polymorphism (and the variant header, preferrably) is going to reduce compilation time. Compilation times grow when there are many combinations of template instantiations to be made, inlined and optimized.

UPDATE

This demonstrates dropping the shared-pointer and runtime polymorphism for a simpler AST.

The afore-mentioned explosion of template instantiations + inlining thereof explains that the "old-fashioned" Qi implementation would compile slow, possibly slower than your original code. The X3 version doesn't have this problem.

sehe
  • 374,641
  • 47
  • 450
  • 633
  • Your drawing seems to suggest me to break the recursion in expression and primary definition? but that recursion is necessary, because the language to be parse require such recursion. – shengyushen Jan 12 '16 at 01:04
  • one even serious problem is, not only the parser.cpp generated from bison compile slow, all other files that include the variant definition head files also compile very slow. – shengyushen Jan 12 '16 at 01:14
  • To be honest, I think how you represent the AST doesn't have to match the productions. Here's what I know better: [boost spirit x3](http://coliru.stacked-crooked.com/a/1418d7b5d9df4a92). Compiles reasonably fast, just value semantics; prepared for different operators, includes parsing and printing the AST. – sehe Jan 12 '16 at 01:35
  • Here's the Qi version which compiles on [c++11](http://coliru.stacked-crooked.com/a/2e38fe6565e9cc80) as well as [c++03](http://coliru.stacked-crooked.com/a/ade3676e51b9d604). **WARNING**: The Qi version is heavy on the compiler, so it's likely to take _longer_ to compile. Of course, a bit of pImpl-idiom gets rid of that in the header file. – sehe Jan 12 '16 at 01:49
  • you example seems very elegant in specifying scanner and parser all in the same c++ file. But I find that spirit is for LL language, which is not compatible with LR or LALR use in yacc, right? – shengyushen Jan 12 '16 at 02:30