1

I've been working on a python program with pycparser which is supposed to generate a JSON-file with dependencies of a given function and its outputs. For an example function:

int Test(int testInput)
{
    int b = testInput;
    return b;
}

Here I would expect b to be dependent on testInput. But ofcourse it can get a lot more complicated with structs and if-statements etc. The files I'm testing also have functions in a specific form that are considered inputs and outputs as in:

int Test(int testInput)
{
    int anotherInput = DatabaseRead(VariableInDatabase);
    int b = testInput;
    int c;

    c = anotherInput + 1;
    DatabaseWrite(c);
    return b;
}

Here c would be dependent on VariableInDatabase, and b same as before. I've run into a wall with this analysis in pycparser as mostly structs and pointers are really hard for me to handle, and it seems like there'd be a better way. I've read into ASTs and CFGs, and other analysis tools like Frama-C but I can't seem to find a clear answer if this is even a thing.

Is there a known way to do this kind of analysis, and if so, what should I be looking into? It's supposed to thousands of files and be able to output these dependencies into a JSON, so plugins for editors doesn't seem like what I'm looking for.

Zpooks
  • 90
  • 8

1 Answers1

3

You need data flow analysis of your code, and then you want to follow the data flow backwards from a result to its sources, up to some stopping point (in your case, you stopped at a function parameter but you probably also want to stop at any global variable).

This is called program slicing in the literature.

Computing data flows is pretty hard, especially if you have a complex language (C is fun: you can have data flows through indirectly called functions that read values; now you need indirect points-to analysis to support your data flow, and vice versa).

Here's fun example:

 // ocean of functions:
 ...
 int a(){ return b; }
 ...
 int p(){ return q; }
 ...
  void foo( int()* x )
 {  return (*x)(); }

Does foo depend on b? on q? You can't know unless you know that foo calls a or b. But foo is handed a function pointer... and what might that point to?

Using just ASTs and CFGs is necessary but not sufficient; data flow analysis algorithms are hard, especially if you have scale (as you suggest you do); you need a lot of machinery to do this that is not easy to build [We've done this on C programs of 16 million lines]. See my essay on Life After Parsing.

Ira Baxter
  • 93,541
  • 22
  • 172
  • 341
  • Currently my program is able to retrieve dependencies with using just an AST, with the exclusion of structs (Which currently looks like I have a solution for) and pointers. All written in Python, with pycparser. Could I expect to run into problems later on than this which would make this impossible? Edit: I'm going to go ahead and read that essay aswell. – Zpooks Aug 08 '16 at 13:54
  • 1
    @Zpooks: Yes, you will have trouble. You didn't seem to appreciate the pointer-to-function text discussion so I added an example of indirect function calls. You can't do that with "just ASTs". You can however do that by reinventing (yourself) all the classic analyses required to compute data flows, but that isn't doing it with "just ASTs". – Ira Baxter Aug 08 '16 at 14:45