Questions tagged [abstract-syntax-tree]

Abstract syntax trees (ASTs) represent the recursive structure of a formal document (program source code).

Abstract Syntax Trees (abbreviated "AST") represent the structure of a formal document (often a computer source program). Nodes in the tree represent syntactically significant chunks of document (function definitions, declarations, statements, expressions and subexpressions). Children of a node represent the parts of that chunk (for a function definition node, the children are likely to be "name", "signature" and "body").

ASTs are widely used in program analysis and transformation systems, as well as classical tools such as compilers.

They are typically constructed by a parsing activity, which is driven by the BNF rules that describe the formal document structure. If one captures precisely how the parser matches the BNF rules to the formal document, the result is a so-called concrete syntax tree (CST). Usually CSTs contain a lot of unnecessary detail (such as parentheses and keywords) that are not needed by the tool using the tree, since the tree nodes themselves essentially represent grammar rules and this information is thus redundant. So parsers often construct ASTs during the parsing process to produce relatively compact trees compared to what a CST would be. See What is the difference between an Abstract Syntax Tree and a Concrete Syntax Tree? for more discussion on this topic.

There is an "ast" tag for StackOverflow; you should use the "abstract-syntax-tree" tag instead.

2945 questions
24
votes
2 answers

How does a simple calculator with parentheses work?

I want to learn how calculators work. For example, say we have inputs in infix notation like this: 1 + 2 x 10 - 2 The parser would have to respect common rules in math. In the above example this means: 1 + (2 x 10) - 2 = 19 (rather than 3 x 10 - 2 =…
Proud Member
  • 40,078
  • 47
  • 146
  • 231
24
votes
3 answers

Can I get an XML AST dump of C/C++ code with clang without using the compiler?

I managed to compile successfully clang for windows with cmake and visual studio 10. I would like to get an XML file as AST representation of the source code. There is one option that provides the result with clang with gcc under linux (ubuntu) but…
jdehaan
  • 19,700
  • 6
  • 57
  • 97
24
votes
4 answers

How to make use of Clang's AST?

I am looking at making use of the Clang's AST for my C code and do some analysis over the AST. Some pointers on where to start, how to obtain the Clang's AST, tutorials or anything in this regard will be of great help!!! I have been trying to find…
bsoundra
  • 930
  • 2
  • 13
  • 27
24
votes
2 answers

Comments out of order after adding item to Go AST

The following test attempts to use AST to add fields to a struct. The fields are added correctly, but the comments are added out of order. I gather the position may need to be specified manually, but I've so far drawn a blank finding an…
David Brophy
  • 849
  • 9
  • 19
24
votes
5 answers

How to exclude headers from AST in clang?

I'm generating AST using clang. I've got following file (lambda.cpp) to parse: #include void my_lambda() { auto lambda = [](auto x, auto y) {return x + y;}; std::cout << "fabricati diem"; } I'm parsing this using following…
Kao
  • 7,225
  • 9
  • 41
  • 65
24
votes
2 answers

How can we get the Syntax Tree of TypeScript?

Is there a process on getting a syntax tree of a compiler. We had been assigned on a project that needs to access typescript's syntax tree (which is opensource so we could see the whole compiler's code). But we don't know how to get it. I've been…
Daj
  • 241
  • 2
  • 3
23
votes
2 answers

How can I use the java Eclipse Abstract Syntax Tree in a project outside Eclipse? (ie not an eclipse plugin)

How can I use the java Eclipse Abstract Syntax Tree in a project outside Eclipse? (ie not an eclipse plugin) All the Eclipse AST examples that I've seen are for eclipse plugins. Is there a way (ie an example) of a project that uses the eclipse AST…
hawkeye
  • 34,745
  • 30
  • 150
  • 304
22
votes
6 answers

A parser for regular expressions in PHP?

I need to parse regular expressions into their components in PHP. I have no problem creating the regular expressions or executing them, but I want to display information about the regular expression (e.g. list the capture groups, attach repetition…
Jan Fabry
  • 7,221
  • 2
  • 36
  • 41
22
votes
4 answers

Parser Combinators, separating grammar and AST construction

I'm writing a simple functional programming language in Scala using the parser-combinators library. The syntax is specified here: https://github.com/hejfelix/Frase/blob/master/src/main/scala/it/vigtig/lambda/ParserLike.scala There is one thing which…
Felix
  • 8,385
  • 10
  • 40
  • 59
22
votes
2 answers

How can I compare two source code files/ ast trees?

I'm generating some source code using the templates package( is there a better method? )and part of the testing I need to check if the output matches the expected source code. I tried a string comparison but it fails due the extra spaces / new…
themihai
  • 7,903
  • 11
  • 39
  • 61
22
votes
4 answers

Go parser not detecting Doc comments on struct type

I am trying to read the assocated Doc comments on a struct type using Go’s parser and ast packages. In this example, the code simply uses itself as the source. package main import ( "fmt" "go/ast" "go/parser" "go/token" ) //…
Matt Sherman
  • 8,298
  • 4
  • 37
  • 57
22
votes
4 answers

How to get source corresponding to a Python AST node?

Python AST nodes have lineno and col_offset attributes, which indicate the beginning of respective code range. Is there an easy way to get also the end of the code range? A 3rd party library?
Aivar
  • 6,814
  • 5
  • 46
  • 78
22
votes
2 answers

How can I dump an abstract syntax tree generated by gcc into a .dot file?

I think the question's title is self-explanatory, I want to dump an abstract syntax tree generated by gcc into a .dot file (Those files generated by Graphviz) because then I want to view it in a .png file or similar. Is there any way I can do that?
asdrubalivan
  • 1,149
  • 2
  • 9
  • 17
21
votes
4 answers

elegant way to test python ASTs for equality (not reference or object identity)

Not sure of the terminology here, but this would be difference between eq? and equal? in scheme, or the difference between == and strncmp with C strings; where in each case the first would return false for two different strings that actually have…
Wang
  • 3,247
  • 1
  • 21
  • 33
21
votes
2 answers

What are motivations behind compiling to byte-code?

I'm working on my own toy programming language. For now I'm interpreting the source language from AST and I'm wondering what advantages compiling to a byte-code and then interpreting it could provide me. For now I have three things in…
sinan
  • 6,809
  • 6
  • 38
  • 67