building a clang AST programmatically

Question

I am writing a transpiler which can translate the source language to C++. I am using flex, bison, and clang-AST for this purpose. I will start with an empty AST and with each semantic action in parsing I will add nodes to the clang-AST.

The problem is that I can't find a way to build the AST programmatically. Let say I want to generate an AST for the following code without RecursiveASTVisitor(because my source language is not C++)

#include <iostream>

int main()
{
  std::cout << "Hello World " << std::endl;

  return 0;
}

So the corresponding code which can generate the AST should look like this

#include "clang/AST/ASTContext.h"

int main(int argc, const char **argv)
{

    //Create an empty context
    clang::ASTContext *context = new clang::ASTContext(NULL, NULL, NULL, NULL, NULL);

    //Declare a main function
    clang::FunctionDecl *FD = clang::FunctionDecl::Create(Context, "main");

    //Declare params
    std::vector<clang::ParmVarDecl*> NewParamInfo;
    NewParamInfo.push_back(clang::ParmVarDecl::Create(Context, FD, "argc"));
    NewParamInfo.push_back(clang::ParmVarDecl::Create(Context, FD, "argv"));

    //set params to function
    FD->setParams(ArrayRef<clang::ParmVarDecl*>(NewParamInfo));

    //Declare a compund stament
    clang::CompoundStmt *CS = new (Context) clang::CompoundStmt(clang::SourceLocation());

    //Declare a Statement to print
    clang::Stmt *S = new (Context) clang::ReturnStmt("std::cout << \"Hello World \" << std::endl;");

    //Add print statement to compund statement
    CS->setStmts(Context, S, 1);

    //Add compund statement to body of function
    FD->setBody(CS);

    return 0;
}

The code mentioned above is not a working code and looking at the docs of llvm or clang is PITA can anybody help me out who has done any similar work?

AFAIK, their is no public API to modify or build AST. What's the purpose to build the AST manually if your goal it to transpile other language to C++. You just emit the C++ source code, no need to use clang. — llllllllll, Dec 27 '18 at 14:11
Why would you transpile to C++ ? This is unusual (and you need to explain why you want to do so). Notice that practically speaking, C++ is slow to compile. You'll better transpile to C, and if you really want to generate C++ you should explain why and emit C++ code in a file. — Basile Starynkevitch, Dec 27 '18 at 14:21
If you code a compiler, better have it emit [LLVM](http://llvvm.org/) or some similar thing (e.g. [libgccjit](https://gcc.gnu.org/onlinedocs/jit/)...). I see no point in emitting C++ AST. If you want to emit C++, emit source code. — Basile Starynkevitch, Dec 27 '18 at 14:25
Notice that `#include ` is pulling many dozens of thousands lines (and there is no equivalent in your AST generating program) — Basile Starynkevitch, Dec 27 '18 at 14:27
I guess that `clang::ReturnStmt("std::cout << \"Hello World \" << std::endl;")` is *not* doing what you want. — Basile Starynkevitch, Dec 27 '18 at 14:34

score 1 · Answer 1 · answered Apr 05 '21 at 16:13

There is writing clang's AST from scratch as clang itself does, not with an API but with its internal functions; conceivably you could go that route but suffer churn in the nonpublic not-API functions that you'd be using. It is not clear that your intended purpose needs the bleeding edge of freshness of the AST-authoring functions that clang itself needs for developing C++20 and C++2b and onward.
But then there is the refactoring API, as mentioned in Clang: write a function's AST from original file to a new file but your intended purpose isn't refactoring per se, but you can contort your thinking a tad to utilize the relatively-stable refactoring API within clang to build up the AST that you seek as “refactoring” from nothingness/scratch.

Also many commenters are deriding the usage of clang's AST as a data-structure. They would eschew clang's AST to “emit source code”, but to emit nontrivial source code one must re-invent your own AST. Why bother reinventing the C++-AST wheel? Just build up clang's AST via the refactoring API instead of designing an AST of your own volition from scratch. Then as the final step after all of your “refactorings” of your intended transpilation have completed, read the generated C++ source code back out from the clang AST by traversing it with a visitor.

building a clang AST programmatically

1 Answers1