1

C newbie here. My use case is to give a header file posted here to the python library cffi so that I can bind to a C library. The header file in the previous link has macros. cffi only accepts a header file without macros, I think.

  • Is there a way to give a header file with macros to cffi by specifying any options?
  • If not, how do I preprocess that file so that I can give it to cffi? I tried gcc -E ddlog.h > ddlog-processed.h but when I ran the following code with cffi, it errors out.

Steps to reproduce:

  • Download ddlog.h
  • pip install cffi
  • gcc -E ddlog.h > ddlog-processed.h
  • In a file build.py in the same folder as ddlog.h, place
import cffi
import pathlib

ffi = cffi.FFI()
this_dir = pathlib.Path().absolute()
h_file_name = this_dir / "ddlog-processed.h"
with open(h_file_name) as h_file:
    ffi.cdef(h_file.read())
  • python build.py

It gives the error:

Traceback (most recent call last):
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/cffi/cparser.py", line 305, in _parse
    ast = _get_parser().parse(fullcsource)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/pycparser/c_parser.py", line 152, in parse
    debug=debuglevel)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/pycparser/ply/yacc.py", line 331, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/pycparser/ply/yacc.py", line 1199, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/pycparser/ply/yacc.py", line 193, in call_errorfunc
    r = errorfunc(token)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/pycparser/c_parser.py", line 1861, in p_error
    column=self.clex.find_tok_column(p)))
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/pycparser/plyparser.py", line 67, in _parse_error
    raise ParseError("%s: %s" % (coord, msg))
pycparser.plyparser.ParseError: /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/sys/_types/_int8_t.h:30:18: before: char

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/bin/invoke", line 10, in <module>
    sys.exit(program.run())
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/invoke/program.py", line 384, in run
    self.execute()
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/invoke/program.py", line 566, in execute
    executor.execute(*self.tasks)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/invoke/executor.py", line 129, in execute
    result = call.task(*args, **call.kwargs)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/invoke/tasks.py", line 127, in __call__
    result = self.body(*args, **kwargs)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/tasks.py", line 48, in build_cffi
    ffi.cdef(h_file.read())
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/cffi/api.py", line 112, in cdef
    self._cdef(csource, override=override, packed=packed, pack=pack)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/cffi/api.py", line 126, in _cdef
    self._parser.parse(csource, override=override, **options)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/cffi/cparser.py", line 358, in parse
    self._internal_parse(csource)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/cffi/cparser.py", line 363, in _internal_parse
    ast, macros, csource = self._parse(csource)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/cffi/cparser.py", line 307, in _parse
    self.convert_pycparser_error(e, csource)
  File "/Users/rabraham/Documents/dev/ddlog/playpen_ddlog/venv/lib/python3.7/site-packages/cffi/cparser.py", line 336, in convert_pycparser_error
    raise CDefError(msg)
cffi.CDefError: parse error
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/sys/_types/_int8_t.h:30:18: before: char
RAbraham
  • 5,956
  • 8
  • 45
  • 80
  • Is it choking on linemarkers? https://stackoverflow.com/questions/2946898/how-to-remove-lines-added-by-default-by-the-c-preprocessor-to-the-top-of-the-out – Carl Norum Apr 07 '20 at 20:30
  • A complete self-contained example would really help here. – Carl Norum Apr 07 '20 at 20:31
  • It's choking on `typedef __signed char int8_t;` – RAbraham Apr 07 '20 at 20:50
  • If it can't handle typedefs, you're going to have a lot harder time. `typedef` is a language feature, not a preprocessor feature. You can't just strip those out. That header is not so long though - why not just rewrite it in a way that works? – Carl Norum Apr 07 '20 at 20:58
  • re: self-contained example, I have just updated the question with `Steps to reproduce`. Thanks @CarlNorum! – RAbraham Apr 07 '20 at 20:58
  • I think its the `__signed` token and not necessarily `typedef` which is a problem? I ask because I took out `__signed` and it went ahead to the next error. I hesitate to clean it by hand as `ddlog.h` is generated by the library authors(not me) and it may change in different versions? – RAbraham Apr 07 '20 at 21:03

1 Answers1

2

There are two steps to using CFFI. The first one is to decide if you're using the ABI or the API mode---ffi.dlopen() versus ffi.set_source(); see https://cffi.readthedocs.io/en/latest/overview.html#abi-versus-api for details. Then you need to write the call to ffi.cdef() accordingly. I generally recommend to use the API mode, which is much more flexible at the cost of requiring a C compiler at install-time (just like if you wrote a standard CPython C extension module). In both modes, you should copy manually parts of the C header inside the call to ffi.cdef(); in the API mode you can leave many more details out and replace them with ... (dot-dot-dot).

In neither of the two modes can you just paste a random C header making use of all the standard C features. Calling gcc -E just makes the problem even harder (but has been done in some cases with very large libraries, with lots of custom post-processing). The point of ffi.cdef() is that normally, you paste a simplified version of just the features you need.

To answer your precise question:

  • macros that are just constants, like an integer, are directly supported. In API mode you can also write #define MY_CONSTANT ... with dot-dot-dot.

  • macros that work like functions are supported in the API mode (only) by using the following trick: write them in ffi.cdef() as if they were regular functions. That's enough to be able to call them.

  • for more complex cases, you might have to write a real C function that wraps the usage of the macro or macros as needed. Just invent a new function name, declare the function in ffi.cdef(), and finally implement the function directly in ffi.set_source().

In the ABI mode, the last two cases cannot be supported at all---macros don't exist in the ABI any more.

Armin Rigo
  • 12,048
  • 37
  • 48